Message-based multiprocessor system and method for operating the same

Abstract

To provide a message-based multiprocessor system that includes a message exchange network (R, L) and a plurality of processor clusters (Ci, j) that can exchange messages with each other.SOLUTION: In a multiprocessor system, a processor cluster (Ci,j) includes one or more processor cluster elements (PCEs) and a message generator (MG). The system is configured as a neural network processor system having multiple neural network processing layers (NL1,...,NL5). Each of the neural network processing layers is assigned one or more processor clusters. A logic module of the message generator that responds to an activation signal (Sact([X, Y])) of the processor cluster elements selectively generates and transmits a message for each cluster of a set of destination processor clusters.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This application relates to a message-based multiprocessor system that can be configured as a deep neural network.

[0002] The present application further relates to a method for operating a message-based multiprocessor system. [Background technology]

[0003] A deep neural network comprises multiple neural network layers. Each neural network layer comprises multiple neural network computation elements. A neural network computation element in a layer may receive weighted input from a neural network computation element in a preceding layer or an input device and may have an output to a neural network computation element in a subsequent layer. The specific manner in which a neural network layer is connected to a subsequent layer depends on the type of neural network layer. For example, in a fully connected layer (FC), each neural network computation element may receive input from a neural network computation element in a subsequent layer. In a convolutional layer, each neural network core element may receive input from a neural network core element in a subsequent layer that is within a convolution kernel centered at a local address corresponding to the local address in the convolutional layer. A pooling layer is used to reduce spatial dimensions. Each neural network core element in the pooling layer corresponds to a respective set of neural network core elements in the subsequent layer. The pooling operation for each neural network core element of a pooling layer may involve, for example, selecting a value from each set of neural network core elements in the preceding layer, e.g., sampling the maximum, minimum, median, or one particular value of each set of neural network core elements. Alternatively, the pooling operation may involve calculating an average value from each set of neural network core elements in the preceding layer.

[0004] A message-based multiprocessor system includes a message-passing network and multiple processor clusters that can exchange messages with each other via the message-passing network. Each processor cluster typically includes one or more processor cluster elements. The message-based multiprocessor system can be configured as a deep neural network. In this case, the processor cluster elements are configured as neural network core elements. This can be achieved by providing the processor cluster elements as dedicated hardware. Alternatively, the processor cluster elements can be programmed to function as neural network core elements. The processor cluster elements may have their own processors, or alternatively, they may share a common processor. In practice, the number of neural network core elements in a neural network layer may be greater than the number of processor elements in a processor cluster. Therefore, when configuring a message-based multiprocessor system, multiple processor clusters are assigned to the neural network layer. Summary of the Invention [Problem to be solved by the invention]

[0005] In operation, a processor cluster element configured as a neural network core element may receive input messages and send output messages. Because a neural network layer typically comprises multiple processor clusters, sending an output message requires a preparatory step in which it is determined which processor cluster of the subsequent destination layer should be addressed. A known approach uses a lookup table with an entry for each processor cluster element of the core that specifies the appropriate destination core whose processor cluster element is to receive the message. However, this approach requires a significant amount of memory. [Means for solving the problem]

[0006] It is an object of the present disclosure to provide an improved message-based multiprocessor system that can be configured as a deep neural network while requiring less memory.

[0007] This object is achieved by a message-based multiprocessor system as claimed in claim 1. The improved message-based multiprocessor system claimed therein comprises a message-switching network and a plurality of processor clusters. Each processor cluster comprises a plurality of processor cluster elements and a message generator, and the processor cluster elements can use the message generator to send messages to destination cores via the message-switching network. The claimed message-based multiprocessor system can be configured as a neural network processor system having a plurality of neural network processing layers, each assigned to one or more of the processor clusters, and the associated processor cluster elements of a processor cluster are neural network processing elements in the neural network processing layer. Multiple neural network processing layers can also be assigned to a common processor cluster. In some embodiments, the processor cluster elements are full-fledged processors with their own processing and storage capabilities. In other embodiments, the processor cluster elements share a common processing capability that performs the processing tasks of the processor cluster elements in a time-shared manner. The message generator of the processor cluster associated with the neural network processing layer comprises a logic module and an associated message generator control storage space comprising respective message generation control data for each destination processor cluster in the set of destination processor clusters. In operation, the logic module performs a respective computation using the message generation control data for each destination processor cluster in the set of destination processor clusters to determine whether the respective destination processor cluster is a target of the processor cluster element. In this manner, large-scale neuromorphic networks can be implemented in a massively parallel multiprocessor system.

[0008] The message generator control storage space may be provided in rewritable memory so that the message-based multiprocessor system can be reconfigured. Alternatively, the storage space may be permanent so that the message-based multiprocessor system can be configured only once for a specific purpose. Hybrid approaches are also possible, where some processor clusters have permanent message generator control storage space while other processor clusters have rewritable storage space, or where some control data is stored permanently and other control data is rewritable.

[0009] A neural network layer may represent one or more feature maps. A feature map specifies the feature value of a feature as a function in coordinate space. As a basic example, a feature may be the color blue in an image, and the feature map specifies the intensity value of blue as a function of spatial coordinates in the image, e.g., a two-dimensional or three-dimensional image. A feature may also be a vector, e.g., specifying the intensity values ​​of red, green, and blue, respectively. In that case, the feature map specifies a vector with the intensity values ​​of red, green, and blue as a function of spatial coordinates in the image, e.g., a two-dimensional or three-dimensional image. In practice, any feature or vector thereof may be mapped, e.g., a vector indicating the probability that an edge exists in the image for each spatial direction. A feature may also represent spatial relationships learned by a neural network processor from available training data. A feature map may be represented in a neural network in the sense that each element in the feature map is assigned a processor cluster element of a processor cluster. This suggests that the state of a processor cluster element indicates the value of the feature for a particular coordinate in the feature map. If the features are components of a feature vector, each component of the feature vector is represented by a respective processor cluster element.

[0010] A further object is to provide an improved method of operating a message-based multiprocessor system as claimed in claim 10. According to which the message-based multiprocessor system operates as follows.

[0011] In a preparation step, the message-based multiprocessor system is configured as a neural network processor having multiple neural network processing layers, whereby one or more respective subsets of processor clusters with associated processor cluster elements are assigned to each neural network processing layer, so that the processor cluster elements form neural network processing elements in the neural network processing layer. The preparation step includes writing respective sets of control data for respective destination processor clusters in subsequent neural network processing layers to respective storage entries in a message generator control storage space of a source processor cluster, whereby each source processor cluster is provided with control data related to its associated destination processor cluster.

[0012] Upon completion of the preparation phase, the multiprocessor system based on the configured messages can undertake the operation phase. The operation phase may start immediately after completion of the preparation phase, or alternatively, at a later time. During the operation phase, the processor cluster element may transmit information about the state of a neural network processing element (with a given ID) to all affected cores. In some embodiments, the processor cluster element is configured as a spiking neural processor cluster element (i.e., a state value is emitted only if certain requirements are met). In that case, the activation signal indicates a spiking condition. Alternatively, the activation signal may be associated with an internal clock signal of the processor cluster element. The processor cluster element that emits the activation signal is referred to as the source processor cluster element. The processor cluster comprising the source processor cluster element is referred to as the source processor cluster.

[0013] In response to an activation signal associated with a neural network processing element identified by a neural network processor element ID, a message generator is activated to generate a message for each set (i=1, .., n) of control data for the source processor cluster. d ), where n d is the number of the destination processor cluster for the source processor cluster. The processor element ID may be a single number or a set of coordinates.

[0014] a) The message generator retrieves the i-th set of control data from the i-th respective storage entry.

[0015] b) Based on the control data, the logical unit determines whether the message should be sent by the source processor cluster element to the destination processor cluster specified by the control data.

[0016] c) If the result of the determination is positive, the output message is sent to the i-th designated processor cluster.

[0017] On the other hand, a separate storage entry is required for each destination core, and the number of destination cores is typically much smaller than the number of processor cluster elements. By way of example, a processor cluster may include hundreds of thousands of processor cluster elements, but the number of destination cores may not be large, typically less than 10, for example in the range of 1 to 5. This allows processor clusters in improved message-based multiprocessor systems to require significantly less memory. A modest number of control data elements is sufficient for the operation of the logic module. These control data elements include at least an offset indication, a destination size indication, and a destination core indication.

[0018] The destination core indication specifies the address of the destination core. The address may, for example, be a relative indication, i.e., indicating the relative coordinates of the destination core in the message switching network with respect to the coordinates of the sending processor cluster. Alternatively, the address may be provided as an absolute indication. The arrangement of processor clusters in the message switching network need not reflect the organization of layers in the configured neural network processing system. Furthermore, the arrangement of processor clusters in a neural network processor layer does not necessarily impose constraints on the arrangement of processor clusters in the message switching network. Nevertheless, it is advantageous if processor clusters in a neural network processor layer are relatively close to processor clusters in a preceding neural network processor layer and / or processor clusters in a succeeding neural network processor layer, thereby enabling messages to be sent while minimizing the amount of hops between network nodes.

[0019] Typically, a processor cluster element configured as a neural network processing element has unique coordinates (x1, x2,...x n ), where n is an integer greater than or equal to 1. For example, ID=x1+N1.x2+N1.N2.x3+...Πi=1,..n-1N i .x n As such, a processor cluster element may have a scalar processor ID that is related to a spatial coordinate by a linear relationship. The coordinates may comprise, for example, one or more spatial coordinates and one or more feature coordinates. For example, a processor cluster element at coordinates (x1, x2, x3) has position (x1, x2) in two-dimensional space and represents the feature indicated by x3.

[0020] If the processor cluster elements are configured as neural network processing elements in a two-dimensional space, the processor clusters may similarly be arranged in a two-dimensional space. In that case, the processor clusters may be identified by their coordinates in that space, but this is not required. Alternatively, the processor clusters may have a scalar indicator for their identification, for example, specified in the control data. It is sufficient that the destination core indication uniquely identifies the destination core.

[0021] The offset indication can be calculated in a preliminary stage from the coordinates of the message-sending processor cluster in the source layer and the message-receiving processor cluster in the destination layer. The coordinates of a processor cluster in a layer are understood to mean the coordinates of a reference processor cluster element of the processor cluster in the coordinate system of that layer. The reference processor cluster element is usually the processor cluster element having the smallest coordinate value in the coordinate system of the layer. The coordinate value of a processor cluster element representing a neural network layer element of a neural network layer is the vector sum of the coordinate value of the processor cluster in the neural network processor layer and the local coordinate value of the processor cluster element in the processor cluster (i.e., relative to the reference processor cluster element).

[0022] The destination size indication represents the size of the space represented by the core at the destination layer. If the space is multidimensional, the size of each dimension may be specified in the destination size indication. Alternatively, less control data may be required if the size is assumed equal in two or more of the dimensions.

[0023] In a message-based multiprocessor system, whether a message should be sent to a particular destination core can be determined in a small number of computationally efficient steps.

[0024] In the first step, coordinate limit values ​​are calculated based on the local coordinates of the source processor cluster elements and the control data elements of the control data for a particular destination core. The lower limit value is calculated by adding an offset value to the local coordinate. If the processor cluster elements are arranged according to a one-dimensional coordinate system, the lower limit value is equal to the sum of their local coordinates and the offset value. If the processor cluster elements are arranged according to a higher-dimensional coordinate system, the lower limit value is calculated for each dimension as the sum of the local coordinate for that dimension and the offset value for that dimension. For example, for a two-dimensional coordinate [Y,X], the lower limit value [Ymin,Xmin] is [Ymin,Xmin]=[Y,X]+[Yoffs,Xoffs] where [Yoffs, Xoffs] is the corresponding pair of offset values ​​in the control data.

[0025] The upper limit value, in this example [Ymax,Xmax], is calculated by adding the kernel size indicator KC to it. The kernel size indicator may be related to the size W of the convolution kernel using the following relationship: W=1+KC<<1 or equivalently KC=(W-1)>>1 where "<<k"および"> The notation ">k" denotes a binary left shift operation and a binary right shift operation of k positions, respectively. That is, these operations are 2 k Multiplication with and 2 k This results in division by . Therefore, [Ymax,Xmax] = [Ymin,Xmin] + KC.

[0026] This allows the value of the kernel size indicator to be pre-computed by a simple subtraction and right-shift operation. The convolution kernel is assumed to have the same size in all dimensions. Alternatively, a separate kernel size indicator may be provided for each dimension. In some configurations, the kernel size indicator is 0. In that case, the size W of the convolution kernel is 1, i.e., the source processor cluster elements and the destination processor cluster elements have a common value for their [X,Y] coordinates.

[0027] As a final step, for each spatial coordinate, it is determined whether at least one of the associated lower and upper bounds is within the size of the range taken by the destination core.

[0028] Upon determining that this condition is met, the message generator proceeds by preparing and sending an output message to a destination processor cluster specified in the control data, the message comprising the coordinates of a processor cluster element in the destination processor cluster, and the destination processor cluster applies the message to only that processor cluster element or to a set of processor cluster elements comprising that processor cluster element, depending on whether the kernel size indicator is different from 0.

[0029] In some embodiments, a designated processor cluster that is a recipient of a message is provided with an improved message receiving function that is capable of applying the event message to a set of processor cluster elements according to a pattern specified by an indicator in the message. The indicator in the message may directly or indirectly reference the pattern to be applied. In a direct reference embodiment, a parameter in the received message is an index into the pattern memory of the recipient processor cluster. In an indirect reference embodiment, a parameter in the received message is an index into the pattern header memory of the recipient processor cluster, which references the pattern memory.

[0030] In one embodiment, the logic module includes a respective logic module section for calculating, for coordinate values ​​of the processor cluster element associated with the activation signal, a possible destination range having minimum and maximum coordinate values ​​for each coordinate in the coordinate system of the destination processor cluster. In that embodiment, the logic module also includes a further logic module section for determining whether a condition is met that, for each of the coordinates, at least one of the calculated minimum value and the calculated maximum value is within the corresponding range for that coordinate. The further logic module section is configured to enable message transmission if the condition is met and to disable message transmission if, for any of the coordinates, neither the calculated minimum value nor the calculated maximum value is within the corresponding range.

[0031] In a working example of this embodiment, the further logic module section comprises a respective first comparator module for providing, for each coordinate, a signal indicating that the calculated minimum value for that coordinate is within a corresponding range, and a respective second comparator module for providing a first match signal indicating that the calculated maximum value for that coordinate is within a corresponding range. It further comprises a logical OR gate for providing, for each coordinate, a second match signal indicating that at least one first match signal is valid. The further logic module section further comprises a logical AND gate for providing a message transmission enable signal if the further match signal is valid for each coordinate.

[0032] In one particularly preferred example of these practical implementations, the comparator module includes at least one mask register with a respective mask bit each representing a respective power of two, a respective logic gate for bit-by-bit comparison with the corresponding bit of the calculated minimum or maximum value, and a synthesis module for issuing an invalid match signal when at least one of the logic gates indicates that the bit of the calculated minimum / maximum value is set but the corresponding mask bit is not set. In this latter exemplary embodiment, partitions may be defined according to powers of two. If the minimum or maximum value being tested, for which the mask bit is not set, exceeds the specified power of two, the corresponding logic gate issues an output signal that causes the synthesis module to issue an invalid match signal. Bit-by-bit comparison allows for very low delay and modest silicon area compared to embodiments using full comparators.

[0033] These and other aspects of the present disclosure are shown in greater detail in the accompanying drawings. [Brief explanation of the drawings]

[0034] [Figure 1]1 is a schematic diagram illustrating an exemplary embodiment of an improved message-based multiprocessor system configured as a neural network processor, with the upper portion of FIG. 1 illustrating elements of the improved message-based multiprocessor system and the lower portion illustrating a neural network processor configured therewith. [Figure 2] FIG. 1 illustrates an exemplary processor cluster in an embodiment of an improved message-based multiprocessor system. [Figure 3] FIG. 10 is another example diagram illustrating the assignment of processor clusters with associated processor cluster elements to neural network processing elements in a neural network layer. [Figure 4] FIG. 10 illustrates another example of the allocation of processor clusters with associated processor cluster elements to neural network processing elements in a neural network layer. [Figure 5] FIG. 10 shows an example where convolution C1 is followed by subsampling by ½ of S2 in X and Y to reduce the XY dimensions of the feature map. [Figure 5A] FIG. 6 is a diagram showing conditions related to FIG. 5. [Figure 5B] FIG. 6 is a diagram showing conditions related to FIG. 5. [Figure 5C] FIG. 6 is a diagram showing conditions related to FIG. 5. [Figure 5D] FIG. 6 is a diagram showing conditions related to FIG. 5. [Figure 6] FIG. 10 shows an example where the operation is upsampling (i.e., adding zeros) to be performed before applying the convolution. [Figure 7A] FIG. 2 illustrates aspects of the components of a processor cluster in more detail. [Figure 7B] FIG. 2 illustrates aspects of the components of a processor cluster in more detail. [Figure 7C] FIG. 2 illustrates aspects of the components of a processor cluster in more detail. [Figure 7D]FIG. 2 illustrates aspects of the components of a processor cluster in more detail. [Figure 7E] FIG. 2 illustrates aspects of the components of a processor cluster in more detail. [Figure 8] FIG. 1 illustrates a schematic diagram of a method for operating a message-based multiprocessor system. [Figure 9] FIG. 1 illustrates an example in which an improved message-based multiprocessor system is configured as a neural network processor with a neural network processing layer for performing padded convolutions. [Figure 10] FIG. 1 illustrates aspects of a configuration for convolution without padding. [Figure 11] FIG. 1 illustrates aspects of a configuration for transposed convolution. [Figure 12A] FIG. 10 illustrates aspects of a configuration for concatenating independent feature maps into one larger feature map in the next layer. [Figure 12B] FIG. 10 illustrates aspects of a configuration for concatenating independent feature maps into one larger feature map in the next layer. [Figure 13A] FIG. 1 illustrates aspects of a configuration for splitting a feature map into multiple feature maps. [Figure 13B] FIG. 1 illustrates aspects of a configuration for splitting a feature map into multiple feature maps. [Figure 14] FIG. 1 illustrates aspects of the configuration for depth-wise convolution. [Figure 15] FIG. 10 illustrates an aspect of a configuration for adding a feature map. [Figure 16] FIG. 10 illustrates aspects of an arrangement for integrating convolution followed by flattening. [Figure 17] FIG. 10 illustrates aspects of a configuration for combining an average pooling operation with a subsequent convolution. [Figure 18] FIG. 10 is a diagram showing a more specific example of this configuration. [Figure 19] FIG. 10 is a diagram illustrating another example. DETAILED DESCRIPTION OF THE INVENTION

[0035] The upper part of Fig. 1 shows schematically a message-based multiprocessor system 1 comprising a message-switching network with routers R and links L and a number of processor clusters Ci,j that are able to exchange messages with each other via the message-switching network. As shown in more detail in Fig. 2, a processor cluster Ci,j comprises one or more processor cluster elements PCE and a message generator MG.

[0036] The message-based multiprocessor system 1 can be configured as a neural network processor system having multiple neural network processing layers, each assigned to one or more of the processor clusters, and the associated processor cluster elements of a processor cluster are neural network processing elements in the neural network processing layers. As an illustrative example, the bottom of FIG. 1 shows how the exemplary message-based multiprocessor system 1 can be configured as a neural network processor system having five neural network processing layers: NL1, NL2, NL3, NL4, and NL5. A neural network processor system configured in this manner has, for example, an 82x82x3 data input to provide RGB image data with a resolution of 82x82 pixels. Each of the neural network processing layers NL1, NL2, NL3, and NL4 is a 3x3 convolutional layer.

[0037] The first convolutional neural network processing layer, NL1, is a convolutional layer with 10 feature maps at a resolution of 80x80 pixels and is assigned to processor clusters C1,0 and C0,1. The second convolutional neural network processing layer, NL2, has 20 feature maps at a resolution of 40x440 pixels and is assigned to processor clusters C2,0, C1,1, and C2,0. The third convolutional neural network processing layer, NL3, has 42 feature maps at a resolution of 38x38 pixels and is assigned to processor clusters C2,1 and C1,2. The fourth convolutional neural network processing layer, NL4, has 50 feature maps at a resolution of 19x19 pixels, and the fifth neural network processing layer, NL5, is a fully connected layer. Both neural network processing layers, NL4 and NL5, are assigned to processor cluster C2,2. It should be noted that the processor cluster C0,0 is not used to configure the neural network processor system, but may be used for other purposes.

[0038] It should be understood that this is only a simplified example. In practice, a core may include thousands of processor cluster elements, and a message-based multiprocessor system may include hundreds or more such cores arranged in a message-passing network. Also, there is not necessarily a clear geometric relationship between the location of cores in a message-based processor system and the allocation of cores to neural network processing layers.

[0039] In Figure 2, the message generator MG of a processor cluster Ci,j is shown to comprise a logic module MGL and an associated message generator control storage space MGM. The message generator logic module MGL receives activation signals S of the processor cluster elements. act ([X,Y]). actIn response to ([X,Y]), it selectively generates and transmits messages for each of a set of destination processor clusters according to the respective message generation control data CD1, CD2, CD3 for the destination processor clusters stored in the message generator control storage space MGM. (Note that in this case, there are three control data words that are all reachable by the source feature map cut, since the destination feature map is cut across three cores.) In this connection, the destination processor clusters of a processor cluster are the cores configured to potentially receive messages from that processor cluster. In the example of FIG. 1, processor cluster C0,0 has two destination cores, i.e., cores C0,1 and C1,0, and processor cluster C0,2 has no destination core. Also, processor cluster C2,2, assigned to the last neural network processor layer NL5, has no destination core. The other processor clusters have exactly one destination core.

[0040] FIG. 3 illustrates another example of the allocation of processor clusters with associated processor cluster elements to neural network processing elements in a neural network layer. Here, the upper part of FIG. 3 illustrates an exemplary neural network layer NLS of a neural network, and the lower part of FIG. 3 illustrates another neural network layer NLD of the neural network, where NLD is the destination of the first mentioned neural network layer. For this purpose, the neural network layer NLS of the upper part of FIG. 3 is labeled the "source neural network layer," and the neural network layer NLD of the lower part of FIG. 4 is labeled the "destination neural network layer." In the illustrated embodiment, the neural network layers NLS, NLD each have a feature map. Each feature map specifies the distribution of feature values ​​in a feature space. In this case, the space is two-dimensional, having coordinates (X, Y), and each feature map has a unique coordinate Z. In other embodiments, the feature maps may be provided in a one-dimensional space, or alternatively, in a space with more than two dimensions. In the illustrated embodiment, the source neural network layer NLS has eight feature maps, each with Z = 0, 1, ..., 7. The destination neural network layer NLD has sixteen feature maps, each with Z = 0, 1, ..., 15. Each of the feature maps has a coordinate range extending from (X, Y) = (0, 0) to (7, 7). In the illustrated example, the coordinate range (4, 4) to (7, 7) of feature maps 4 through 7 of the source neural network layer NLS is assigned to processor cluster PCS1. The other coordinate ranges are assigned to other processor clusters.

[0041] The processor cluster PCS1 assigned to this range is - First destination processor cluster PCD1 for the range (Xdst0,Ydst0,Zdst0) = (0,0,0) to (3,7,15) - Second destination processor cluster PCD2 for the range (Xdst0,Ydst0,Zdst0)=(4,0,0) to (7,7,7) - Third destination processor cluster PCD3 for range (Xdst0,Ydst0,Zdst0)=(4,0,8) to (7,7,15) , and has a destination in the destination feature map assigned to

[0042] The following describes in more detail how the source processor cluster PCS selectively generates and sends messages to each of these destination processor clusters PCD1, PCD2, PCD3 based on an evaluation using the respective message generation control data (CD1, CD2, CD3) stored in the message generator control memory space MGM.

[0043] Starting with the first destination processor cluster PCD1, the evaluation is as follows:

[0044] The (X,Y) position relative to the origin of the feature map is calculated from the ID of the processor cluster element carrying the activation signal Sact[X,Y], also referred to herein as the "firing processor cluster element" or simply the "firing core element." In this example, the activation signal Sact[X,Y] specifies that the local coordinates of the firing core element within the cluster PCS are X=0, Y=1.

[0045] Adding these coordinates to the coordinates of the origin of the source processor cluster PCS in the global coordinate system (Xsrc0=4, Ysrc0=4) gives the global coordinates (Xsrc, Ysrc) of the firing core element in the complete logical source feature map. X src =X+X src0 Y src =Y+Y src0

[0046] The coordinates of the origin of the destination core PCD1 in the global coordinate system are subtracted from this intermediate result. For the destination core PSD1, the global coordinates of its origin are (Xdst=0, Ydst0=0).

[0047] Then, assuming the kernel is square, an offset (ΔXmin, ΔYmin) is subtracted to get the first destination (X,Y) affected by the firing core element (in the example of a zero-padded 3x3 convolution, the kernel size is 3 and the value to be subtracted is (KernelSize-1) / 2=1). Alternatively, the kernel shape can be rectangular and specified by a kernel size pair.

[0048] Instead of repeating all the calculations every time an activation signal is received, the calculations are simplified by pre-calculating the offset pairs. Xoffs=Xsrc0-Xdst0-ΔXmin Yoffs=Ysrc0-Ydst0-ΔYmin

[0049] The pre-calculated offset pair values ​​are stored as part of the message generation control data (CD1, CD2, CD3).

[0050] Upon receiving an activation signal Sact[X,Y], the minimum coordinate pair is calculated by adding the offset value pair (Xoffs, Yoffs) to the coordinate pair (X,Y) provided by the activation signal. (Xmin,Ymin)=(X,Y)+(Xoffs,Yoffs), where Xmin and Ymin are the minimum X and Y values, respectively.

[0051] The maximum coordinate pair is also calculated by adding the kernel size indicator to the minimum coordinate pair, i.e. (Xmax,Ymax)=(Xmin,Ymin)+(Kx,Ky) (=(Xmin,Ymin)+(K,K) for a rectangular kernel), where Xmax and Ymax are the maximum X and Y values, respectively. Furthermore, K=KernelSize-1, or in the more general case, Kx=KernelSizeX Ky=KernelSizeY.

[0052] Subsequently, in response to the activation signal Sact[X,Y], it is determined by the processor logic module MGL whether the outgoing message is to be sent to the destination processor cluster PSD1.

[0053] To do this, the values ​​of the minimum coordinate pair (Xmin, Ymin) and the maximum coordinate pair are compared with the dimensions of the range spanned by the destination processor cluster PSD1 in the coordinate space, denoted W for the X direction and H for the Y direction. The sending of an output message is enabled if at least one of the values ​​Xmin, Xmax is within the range [0,W) and at least one of the values ​​Ymin, Ymax is within the range [0,H).

[0054] In these steps, at most one message needs to be sent to each destination cluster or feature map in the destination cluster. The message only needs to specify a single destination coordinate, and the destination cluster applies the message to the set of destination processor cluster elements specified by the pattern. The identity of the pattern to be applied is typically included in the message. For destination processor cluster PCD1, the pair of offset values ​​is (Xoffs=3, Yoffs=3). In this example, where the activation signal Sact[X,Y] comes from the firing core element with local coordinates X=0, Y=1, the minimum and maximum coordinate pairs are (Xmin,Ymin)=(0,1)+(3,3)=(3,4) (Xmax,Ymax)=(3,4)+(2,2)=(5,6) It is calculated as follows.

[0055] The message generation control data (CD1) of the processor cluster (PCS1) specifies that the destination processor cluster (PCD1) has a width W=4 and a height H=8. Therefore, at least one of the values ​​Xmin, Xmax, in this case the value Xmin=3, is within the range [0,W). Also, at least one of the values ​​Ymin, Ymax, in this case both the values ​​Ymin=4, Ymax=6, is within the range [0,H). This causes the logic module (MGL) to determine that a message should be generated and sent to the destination processor cluster PCD1.

[0056] The message to be generated and sent may comprise the following data: a) A cluster address specified in the message generation control data (CD1) and a pair of values ​​(Xmin, Ymin) calculated by the logic module (MGL). b) In an exemplary embodiment, the message includes a value. This is not required. In some cases, the presence or absence of a message may be considered to indicate a Boolean value. Also, the time distance between subsequent messages may indicate a value to the recipient processor cluster (PCS1). c) In an exemplary embodiment, the message alternatively or additionally comprises pattern identification information. The pattern identification information enables the destination processor cluster (PCS1) to select one of multiple patterns of weight values ​​to be applied to the processor cluster elements of PCS1 to which the message is to be applied. Alternatively, the destination processor cluster (PCS1) may apply a standard pattern. The pattern identification information may be determined as the sum of a base ID (PatternID0) and a coordinate value Z indicating a feature map index associated with the processor cluster element (PCE) in the processor cluster (PCS1) carrying the activation signal. The receiving destination processor cluster (PCD1) applies the message value according to the pattern weight to the processor cluster element (PCE) of PCD1. Here, the message value is either a value explicitly specified in the message, or an implicit value based on the presence or absence of a message, or the length of time elapsed since the previous message, and the pattern is either a default pattern or a pattern specified by the pattern identification information in the message.

[0057] For the destination core PSD2, the global coordinates of its origin are (Xdst0=4, Ydst0=0). Therefore, the offset value pair for this destination core (PSD2) is: Xoffs=Xsrc0-Xdst0-ΔXmin=4-4-1=-1 Yoffs=Ysrc0-Ydst0-ΔYmin=4-0-1=3 is calculated in advance.

[0058] In this example, where the activation signal Sact[X,Y] comes from the firing core element with local coordinates X=0, Y=1, the minimum and maximum coordinate pairs are (Xmin,Ymin)=(0,1)+(-1,3)=(-1,4) (Xmax,Ymax)=(-1,4)+(2,2)=(1,6) It is calculated as follows.

[0059] The message generation control data (CD2) of processor cluster (PCS2) further specifies that destination processor cluster (PCD2) has width W=4 and height H=8. Therefore, at least one of the values ​​Xmin, Xmax, here value Xmax=1, is within the range [0,W). Also, at least one of the values ​​Ymin, Ymax, in this case both values ​​Ymin=4, Ymax=6, is within the range [0,H). This causes logic module (MGL) to determine that a message should be generated and sent to destination processor cluster PCD2.

[0060] The message generation control data (CD3) relates to a third processor cluster (PCD3) that has the same global coordinates of the origin in the XY plane as the processor cluster PCD2 (Xdst0=4, Ydst0=0), but a different Z value, i.e., Zdst=8. This implies that the message to the third destination processor cluster (PCD3), when sent, will have a different pattern ID than the second destination processor cluster (PCD3).

[0061] 4 shows another example in which a neural network processor system formed by a message-based multiprocessor system 1 includes a fully-connected neural network processing layer FC2. That is, layer FC2 is fully connected to neural network processing layer NL1. In the example shown in FIG. 4, neural network processing layer NL1 is formed by processor cluster PCS1 and a further source processor cluster not shown here. Fully-connected neural network processing layer FC2 is formed by destination processor cluster PCD1 and destination processor cluster PCD2.

[0062] The fully connected layer FC2 defines the mapping 1x1xZ1 → 1x1xZ2, where Z1 and Z2 are the layer depths of the source NL1 and destination layer FC2, respectively. This mapping is logically equivalent to a 1x1 convolution on the feature maps with X and Y equal to 1.

[0063] The source FM has a size of (1,1,1024). In the example shown, the coordinate range of the neural network processing layer NL1 is (Xsrc0, Ysrc0, Zsrc0) = (0,0,0) to (0,0,1023). In this example, the processor cluster PCS1 is assigned to the section of this coordinate range (Xsrc0, Ysrc0, Zsrc0) = (0,0,512) to (0,0,1023). The destination processor clusters PCD1 and PCD2 are each assigned to the section (Xdst0,Ydst0,Zdst0)=(0,0,0) to (0,0,1023) and (Xdst0, Ydst0, Zdst0) = (0, 0, 1024) to (0, 0, 1067) in the coordinate range of the fully connected neural network processing layer FC2 is assigned to.

[0064] As an example, it is assumed that the neural network processor element identified by coordinates (X,Y,Z)=(0,0,N) (i.e., NeuronID=FMstart+N) produces an activation signal. In this example, it is further assumed that the range of PatternIDs in destination processor cluster PCD1 ranges from 0 (source neuron 0) to 1067 (source neuron 1067), and that the range of PatternIDs in cluster PCD2 ranges from 99 (source neuron 0) to 1166 (source neuron 1067).

[0065] [Table 1]

[0066] When the activation signal Sact(0,0,N) is detected, the logic module MGL of the processor cluster PCS1 calculates the minimum and maximum value pairs [Xmin,Ymin] and [Xmax,Ymax] using the coordinates of the processor cluster element PCE in the XY plane. Since the XY coordinates are (0,0) and the offset value and KC value are 0, the calculated values ​​of the minimum and maximum value pairs are [Xmin,Ymin]=[0,0] [Xmax,Ymax]=[0,0] is.

[0067] Therefore, at least one of the values ​​Xmin, Xmax, in this case both, is within the range [0,W). Also, at least one of the values ​​Ymin, Ymax, in this case both, is within the range [0,H), because in a fully connected layer, Xmin, Xmax, Ymin, and Ymax are always 0 (because as long as KernelSize is always 1, Xsrc, Ysrc, Xdst, and Ydst are always 1, which means that as long as Xoffset and Yoffset are always 0, the x and y of the firing neuron are always 0). The message sent to the destination processor cluster PCD1 comprises the following data: address of PCD1, Xmin=0, Ymin=0, PatternID=PatternID0+N=512+N, Value.

[0068] The above applies equally to validating messages to destination processor cluster PCD2, even though the destination address and the selected pattern are different: the message sent for destination processor cluster PCD2 comprises the following data: Address of PCD2, Xmin=0, Ymin=0, PatternID=PatternID0+N=611+N, Value.

[0069] Figure 5 shows an example in which a convolution C1 is followed by a 1 / 2-fold subsampling S2 in X and Y to reduce the XY dimensions of the feature map. This is logically equivalent to jumping by 2 in the input kernel movement (commonly called striding). The convolution and subsampling operations can be combined to reduce the number of processor cluster elements PCEs required to be configured as neural processing layer elements. This can reduce the number of processor cluster elements X required by 75%. Essentially, only processor cluster elements corresponding to neural elements with even X and Y coordinates need remain.

[0070] As shown in FIGS. 5A-5D, there may be four cases for the minimum pair [Xmin, Ymin] for the affected region: Both Xmin and Ymin are odd numbers (Figure 5A) Xmin is odd and Ymin is even (Figure 5B) Xmin is even and Ymin is odd (Figure 5C) Both Xmin and Ymin are even numbers (Figure 5D)

[0071] In the first case shown in FIG. 5A, the operation amounts to a convolution with a kernel of size KH×KH, where KH is related to the size of the kernel (KernelSize) of the convolution C1 as follows: KH=(KernelSize-1)>>1 This is for example a 1x1 convolution with w(0,0), where w(.,.) denotes the weight of the convolution kernel C1, i.e. w(0,0) is the weight of the convolution kernel C1 for the coordinate (0,0).

[0072] In the second case, as shown in Figure 5B, the combined operation is equivalent to a convolution with KH x (KH + 1), which in this example is a 1 x 2 convolution with [w(0, -1); w(0, 1)].

[0073] In the third case shown in Figure 5C, the combined operation reduces to a convolution with (KH+1) x KH, which in this example is a 2 x 1 convolution with [w(-1,0);w(1,0)].

[0074] In the fourth case shown in Figure 5D, the combined operation is equivalent to a convolution with (KH+1) x (KH+1), which in this example is a 2 x 2 convolution with [w(-1,-1);w(1,-1);w(-1,1);w(1,1)].

[0075] This example typically considers odd kernel widths / heights, but remains the same for even kernel shapes, in which case only the shapes of the four sub-convolutions change.

[0076] FIG. 6 shows yet a further example in which the operation is upsampling (i.e., adding zeros) to be performed before applying a convolution. The upsampling can be 2x, 4x, 8x, 16x, etc. (whereas downsampling is always 1 / 2x). The improved message-based multiprocessor system 1 facilitates the integration of the upsampling operation and the subsequent convolution operation in a single convolution layer, as described in more detail below. In the example shown, the first operation is a 4x upsampling and the second operation is a 3x3 convolution (zero-padded).

[0077] Essentially, neurons X and Y are left-shifted (i.e., multiplied by 2) before applying the deconvolution.

[0078] Figures 7A to 7E show aspects of the message generator MG in more detail, where Figure 7A shows a first module MGLY that calculates the minimum and maximum Y values ​​(Ymin, Ymax) based on the Y values ​​of the processor cluster elements PCE from which the activation signal (Sact([X,Y])) comes.

[0079] FIG. 7B shows a second module MGLX that calculates minimum and maximum X values ​​(Xmin, Xmax) based on the X values ​​of the processor cluster elements PCE.

[0080] FIG. 7C shows a third module MGLC that determines the pattern ID to be sent with the message.

[0081] FIG. 7D shows an exemplary implementation of the part XMN in the third module MGLC.

[0082] FIG. 7E shows a fourth module that determines whether a message should be sent.

[0083] As is evident from these figures, the logic module MGL determines in a computationally efficient manner whether a message is to be sent to a particular destination processor cluster.

[0084] For illustrative purposes, Figures 7A-7E show each module as a pseudo-hardware implementation. It will be understood by those skilled in the art that the same operations may be performed by one or more programmable processors.

[0085] More specifically, the first module shown in FIG. 7A performs the following operations:

[0086] In Figure 7A, the Y value indicated in the activation signal is optionally left-shifted in element GLY1 by a factor indicated by the parameter Upsamp contained in the message generation control data CD1, CD2, .... The number of bits to which the left-shift operation is applied is thereby 2It is equal to Log(UpsamplingFactor). It is assumed that only upsampling by a power of 2 is required. If no upsampling is required, the input value of Y is passed to the output of element GLY1. It may be contemplated to define an upsampling factor other than a power of 2. In that case, the left shift operation should be replaced by a multiplier, which is computationally more expensive.

[0087] In adder GLY2, Yoffset is added to the output value Y<<Upsamp from element GLY2. The output of adder GLY2 is supplied to right shift element GLY4 and adder GLY5. Further, the first bit (Yodd) of this output is supplied to multiplexer GLY3.

[0088] In the operation, the logic module performs the following steps. The initial lower limit values [Ymin, Xmin] for the mapping window are calculated as follows. Ymin = Y<<CDi.UpSamp + CDi.Yoffset (refer to elements GLY1 and GLY2 in Figure 7A respectively) Xmin = X<<CDi.UpSamp + CDi.Xoffset (refer to elements GLX1 and GLX2 in Figure 7B respectively)

[0089] Here, CDi.x represents the parameter x in the message generation control data CDi stored in the message generator control memory space (MGM), and "i" is the index associated with the current destination processor cluster. For example, CD1.UpSamp is the upsampling factor for the destination processor cluster with index 1.

[0090] In the elements GLY1, GLX1, the Y and X values ​​indicated by the activation signal are optionally left-shifted by a factor indicated by the parameter CDi.Upsamp contained in the message generation control data CDi. The number of bits to which the left-shift operation is applied is thereby equal to 2Log(UpsamplingFactor). It is assumed that only upsampling by powers of 2 is required. If no upsampling is required, the Y input value is passed to the output of the elements GLY1, GLX1. It may be attempted to define upsampling factors other than powers of 2. In that case, the left-shift operation would have to be replaced by a multiplier, which is computationally more expensive.

[0091] The initial upper limit is Ymax = Ymin + CDi.KC (see adder GLY5 in Figure 7A) Xmax = Xmin + CDi.KC (see adder GLX5 in Figure 7B) It is calculated as follows.

[0092] The final lower bound for the mapping window [Ymin,Xmin] is Ymin=CDi.S2*Yodd+Ymin>>CDi.S2 (see multiplexer GLY3, right shift element GLY4, and adder GLY6 in Figure 7A, respectively) Xmin=CDi.S2*Xodd+Xmin>>CDi.S2 (see multiplexer GLX3, right shift element GLX4, and adder GLX6 in Figure 7B, respectively) It is calculated as follows.

[0093] Here, the value pair [Xodd, Yodd] is assigned as [Ymin[0], Xmin[0]], i.e., the least significant bits of Ymin and Xmin in the output of GLY2, GLX2.

[0094] From Figures 7A and 7B it becomes clear that the multiplications CDi.S2*Yodd and CDi.S2*Xodd are simplified to simple selection operations performed by GLY3 and GLX3, respectively.

[0095] The final upper bound for the mapping window [Ymax,Xmax] is Ymax = Ymax >> CDi.S2 (see right shift element GLY7 in Figure 7A) Xmax = Xmax >> CDi.S2 (see right shift element GLX7 in Figure 7B) It is calculated as follows.

[0096] These values ​​[Xmin, Ymin], [Xmax, Ymax] determine whether the message has a destination in the destination processor cluster, according to the associated message generation control data CDi, in the section of the logic module MGL shown in Figure 7C. To that end, it is determined whether at least one of the values ​​Xmin, Xmax is in the range [0, W) and, additionally, whether at least one of the values ​​Ymin, Ymax is in the range [0, H).

[0097] In the pseudo-hardware implementation shown in FIG. 7C, comparators XMN and XMX determine whether the values ​​Xmin and Xmax, respectively, are within the range [0,W). Then, logic element ORX determines whether this is true for at least one of the values ​​Xmin and Xmax. Similarly, comparators YMN and YMX determine whether the values ​​Ymin and Ymax, respectively, are within the range [0,H), and logic element ORY determines whether this is true for at least one of the values ​​Ymin and Ymax. Note that if the comparisons XMN and XMX in the sections are performed sequentially, e.g., in the case of a software implementation, the second comparison can be skipped if the first comparison indicates that the tested value Xmin or Xmax was within range. This is also true for the sequential comparisons performed for Ymin and Ymax. Logic gate ANDXY emits the signal "Hit" only if the outputs of both ORX and ORY are positive.

[0098] In some embodiments, the comparator modules XMN, XMX, YMN, YMX may be provided as full-fledged comparator modules that perform comparisons for arbitrary values. Alternatively, the comparator modules may be provided as bitwise comparators, allowing the acceptable target coordinate range to be selected from powers of two.

[0099] By way of example, such an embodiment of comparator module XMN is shown in FIG. 7D. Comparator modules XMX, YMN, and YMX may be provided similarly. Comparator module XMN shown in FIG. 7D comprises a mask register MR with a respective mask bit each representing a respective power of 2, respective logic gates BC2, ..., BC6 for bit-wise comparison with corresponding bits Xmin[2], ..., Xmin[6] of the calculated minimum value Xmin, and a combination module BCM. The combination module issues an invalid match signal if at least one of logic gates BC2, ..., BC6 indicates that a bit of the calculated minimum / maximum value is set but the corresponding mask bit is not set. For example, if bits MR2 and MR3 are set true but the remaining bits are set false, any input value greater than or equal to 16 results in an invalid match signal. Similarly, the range may be constrained to <4, <8, <32, or <64. In this embodiment, bit lines Xmin[7] and Xmin[8] are provided directly as inputs to the compositing module BCM. In this example, it is assumed that the X and Y coordinate range is constrained between 0 and 127. Therefore, in either case, an invalid match signal is asserted if the input value is greater than or equal to 128, or if the input value is negative as specified by bit 8 of input signal Xmin.

[0100] When the logic module section of Figure 7C, optionally comprising the comparator module of Figure 7D, emits a signal Hit, an output message is generated and sent to the associated destination processor cluster. The message generation comprises calculating a pattern ID offset value as follows: PatternIDoffset=(Z<<2*S2)+S2(2*Yodd+Xodd)

[0101] As shown in FIG. 7E, this result can be determined very computationally efficiently using, for example, a left shift of 2 to the left, shifting on [Yodd,Xodd] instead of "00" as in a normal shift operation.

[0102] 8 shows a schematic diagram of a method of operating a message-based multiprocessor system 1. The message-based multiprocessor system is provided, for example, according to an embodiment as shown in and described with reference to FIGS. 1 and 2. In summary, the message-based multiprocessor system comprises a message-exchanging network R, L and a plurality of processor clusters Ci,j (see, for example, FIG. 2) capable of exchanging messages with each other via the message-exchanging network. Each processor cluster Ci,j comprises one or more processor cluster elements PCE and a message generator MG with a logic module MGL and a message generator control memory space MGM.

[0103] The exemplary embodiment of the method shown in FIG. 8 proceeds according to the following steps.

[0104] In the initialization step S1, the following input parameters are obtained: ControlDataStart, ControlDataPreStop, N, Value, FMstart, FMsizeZv, FMsizeY, FMsizeX. Note that specifying the value FMstart allows the option of mapping multiple FMs to a common layer. Note further that alternatively, it is possible to provide X, Y, Z, and Nrel directly as input and do this independently of the message generator. This simplifies subsequent calculations.

[0105] The input parameters ControlDataStart and ControlDataPreStop are obtained from the pattern memory, where the input parameter ControlDataStart indicates the location of the first set of control data, and the input parameter ControlDataPreStop indicates the end of the last set of control data.

[0106] The input parameters N and Value specify the ID (e.g., coordinate value of the processor cluster element) and value of the firing neural processor layer element within the processor cluster. The frame is specified by FMstart, FMsizeZv, FMsizeY, and FMsizeX.

[0107] where FMstart denotes the index of the first processor cluster element of the current processor cluster. The parameters FMsizeZv, FMsizeY, and FMsizeX denote the size of the feature map or the portion of the feature map represented by the processor cluster. That is, the parameters FMsizeX and FMsizeY denote the size of the feature map in the spatial directions X and Y, and FMsizeZv denotes the number of feature maps represented by the processor cluster.

[0108] With these input parameters, the following initial steps are performed: The neuron id Nrel for the starting position is Nrel=N-FMstart It is calculated as follows.

[0109] The position [X,Y,Z] of the processor cluster element that emits the activation signal Sact(X,Y,Z) in the source feature map is determined from its relative address. This operation is [X,Y,Z]=getXYZ(Nrel,FMsizeZv;FMsizeY,FMsizeX) can be expressed by the symbol:

[0110] A relative processor cluster element Nrel may be related to a position [X,Y,Z] as Nrel=X+FMsizeX*Y+FMsizeX*FMsizeY*Z. In this example, it is assumed that neuron IDs are assigned in the following layout / scheme: X first, then Y, and finally Z. Note that any other way of assigning neuron IDs is possible, as long as it is well enough defined to allow the reconstruction of coordinates X, Y, Z from NeuronID.

[0111] In one embodiment, the values ​​of FMsizeX and FMsizeY are powers of 2, so the value of Nrel is Nrel=X+Y<< 2 logFMsizeX+Z<<( 2 logFMsizeX+ 2 logFMsizeY).

[0112] Therefore, the coordinates X, Y, and Z are X=Nrel[0: 2 logFMsizeX-1] Y=Nrel[ 2 logFMsizeX: 2 logFMsizeX+ 2 logFMsizeY-1] Z=Nrel[ 2 logFMsizeX+ 2 logFMsizeY:] It can be derived from Nrel as

[0113] Instead of calculating coordinate values ​​for each instance, the coordinate values ​​may alternatively be calculated incrementally. For example, the processor cluster element states may be updated cyclically, starting from the first processor cluster element in the cluster having coordinates (0,0,0) to the last processor cluster element, while incrementally updating the coordinate values.

[0114] The value of the control parameter DestNum, which indicates the number of the destination processor cluster, is initialized.

[0115] The value of the further control parameter DestInd is initialized to 0. This further control parameter is an index that specifies the respective set of message control parameters for the respective destination processor cluster.

[0116] In step S2 the value of the control parameter DestNum is verified. If the value of DestNum is 0 the procedure ends. If the value of DestNum is different from 0 one or more of steps S3 to S9 of the procedure are executed as specified below.

[0117] In step S3, the message generation control data CDi for the destination processor cluster referenced by the destination index DestInd is read from the message generator control storage space MGM.

[0118] In step S4 the Boolean value of the further message type indication "Flatten" is determined. If the Boolean value is True, step S5 is executed followed by step S6. If the Boolean value is False, the procedure continues directly to step S6.

[0119] In step S5, the coordinates [X, Y, Z] are assigned as follows: [X,Y,Z]=[0,0,Nrel]

[0120] In step S6, the following calculations are performed: The initial lower bounds for the mapping window [Ymin,Xmin] are Ymin=Yoffset+Y< <UpSamp Xmin=Xoffset+X< <UpSamp The initial upper limit is calculated as follows: Ymax=Ymin+KC Xmax=Xmin+KC It is calculated as follows.

[0121] The value pair [Xodd,Yodd] is assigned as [Ymin[0],Xmin[0]].

[0122] The final lower bound for the mapping window [Ymin,Xmin] is Ymin=S2*Yodd+Ymin>>S2 Xmin=S2*Xodd+Xmin>>S2 It is calculated as follows.

[0123] The final upper bound for the mapping window [Ymax,Xmax] is Ymax=Ymax>>S2 Xmax=Xmax>>S2 It is calculated as follows.

[0124] Using these values, the following function is used to determine if the message has a destination in the destination processor cluster corresponding to the message generation control data: Hit=hitDetect(Ymin,Ymax,Xmin,CutHeight,CutWidth)

[0125] If it is determined in step S7 that the Boolean Hit was set to True in the previous step, then in step S8 a message is prepared to be sent to the destination address as follows: The message comprises the following information: a) DstClsIDY and DstClslDX, which together specify the network address of the destination processor cluster. b) the address of a destination processor cluster element within that destination processor cluster, DestN c) SynType, which indicates the type of operation to be performed by the destination processor cluster. d) Spatial pattern, e.g., convolution pattern designation, PatternID0+PatternID e) Value: In the case of a non-packed message, or for a set of processor cluster elements within a range defined by a spatial pattern, typically centered around and including the destination processor cluster element specified in the message, a value to be used in the operation to be performed by the processor cluster for the specifically referenced processor cluster element.

[0126] Regardless of the value of the Boolean Hit, in step S9 the control value indicating the control word to be used is updated according to, and if it is determined that the processor cluster has a further destination processor cluster, the same procedure is applied from step S3 onwards for this further destination processor cluster.

[0127] example Exemplary configurations of embodiments of improved message-based multiprocessor systems are discussed below.

[0128] Convolution with padding In one example shown in FIG. 9, the improved message-based multiprocessor system is configured as a neural network processor with a neural network processing layer for performing padded convolutions.

[0129] In this configuration, the XY size (W, H) of the destination feature map is the same as the XY size of the source feature map. Furthermore, the Ymin and Xmin values ​​in the destination feature map are Xmin=X-(kernelSizeX-1) / 2 Ymin=Y-(kernelSizeY-1) / 2 Here, kernelSizeX and kernelSizeY are the dimensions of the convolution kernel. They may have the same value, i.e., KernelSizeX=KernelSizeY=KernelSize. That is,

number

[0130] Convolution without padding Figure 10 shows a convolution without padding. In this configuration of the improved message-based multiprocessor system, the partitioning of the destination feature map (hit detection) may be the same as in the zero-padding case.

[0131] The XY size of the destination feature map is equal to the XY size of the source feature map minus (KernelSize-1). Ymin and Xmin in the destination feature map are equal to Y and X in the source minus (KernelSize-1), respectively. Ymax and Xmax in the destination feature map are equal to X and Y in the source.

[0132] Transposed convolution Also, in the case of transposed convolution (see Figure 11), often called deconvolution, the partitioning of the destination feature map (hit detection) can be the same as in regular convolution. The XY size of the destination FM is equal to the XY size of the source FM plus (KernelSize-1). The values ​​of Ymin and Xmin in the destination feature map are equal to Y and X in the source FM. The values ​​of Ymax and Xmax in the destination FM are equal to the values ​​of X and Y in the source FM plus (KernelSize-1).

[0133] Feature map concatenation As shown in Figures 12A and 12B, independent feature maps may be concatenated into one larger FM in the next layer. The concatenation may be combined with subsequent convolutions to reduce the number of neural network layer elements, i.e., processor cluster elements allocated as neural network layer elements. The same principle applies to partitioning feature maps and allocating the partitions to respective processor clusters.

[0134] In one example (see FIG. 12A), three 2D FMs are concatenated into one 3D FM, where each of the three incoming PatternIDs (events from different sources) points to the same processor element address (located in the 3D coordinate system), but to different pattern memory starting points (so that each source has a different weight), and a transformation to one 3D FM is obtained on the fly while applying the convolution.

[0135] As further shown in FIG. 12B, the division in the XY plane can be controlled simply by setting the OffsetY and OffsetX values.

[0136] Feature map division As shown in Figures 13A and 13B, a feature map may be split into multiple FMs, which may be combined with subsequent convolutions to reduce the number of neural network layer elements, i.e., processor cluster elements allocated as neural network layer elements.

[0137] In the example shown in Figure 13A, a single 3D FM is split into two 3D FMs. A neural network layer with spatial coordinates (X, Y) and Z coordinates that indicate a specific feature is denoted as 3D-FM. Generalizations are possible, and multiple feature maps are defined in more than two spatial dimensions (X1, X2, ..., Xn).

[0138] When the processor cluster element PCE generates an activation signal (representing a neuron in source FM layer 0 or 1 firing), PatternID0 or PatternID0+1 is on an outgoing event. This implies that in the destination cluster, PatternID0 and PatternID0+1 go through the population memory to FM-3D.0 (the green part), and the convolution is applied only to the green part.

[0139] When a neuron in source FM layer 2 or 3 fires, PatternID0+2 or PatternID0+3 is on the outgoing event. As a result, in the destination cluster, PatternID0+2 and PatternID0+3 go through NPM to FM-3D.1 (yellow part).

[0140] In some embodiments, the logic module MGL of the message generator may be extended with an additional detection section that also detects for the Z coordinate whether at least one of the calculated minimum value Zmin and the calculated maximum value Zmax is within the corresponding range SizeZ for that coordinate. Alternatively, if there is no such additional detection section and it is unavoidable to send messages to destination processor clusters that would otherwise be excluded, these otherwise excluded destination processor clusters may apply a zero operation pattern, i.e. a pattern with a single zero weight value, so that the processor cluster elements within that pattern are effectively unaffected by the message as if the message had not been directed to the otherwise excluded destination processor cluster at all.

[0141] In the example shown in Figure 13B, the feature map is divided in the XY plane. In general, the same principles apply to partitioning a single feature map across several clusters. FM-2D has two sets of message generation control data to specify partitioning in XY. Partitioning in the Z direction as well as the XY plane can be achieved by a combination of the principles specified in this example and the example shown in Figure 13A.

[0142] Depth-wise convolution Figure 14 shows an example of depth-wise convolution, i.e., convolution with a convolution pattern extending in the Z direction. Depth-wise convolution is relatively simple to configure, since each Z layer of the source / current FM has a different Pattern-ID in the destination layer (e.g., PatternID = PatternID0 + Z). The Pattern-ID of the current layer points to the correct FMstart in the destination, and then the 2D pattern is read.

[0143] Therefore, each destination is simply one 2D FM, and the Pattern-ID of the 2D FM is classified. This suggests that the 3D FM is simply split into parallel 2D FMs that are combined with the convolution (see the preceding example of splitting FMs). To reduce memory and processing capacity requirements, the 2D FMs may be combined into one 3D FM (see the examples of concatenating FMs in Figures 12A and 12B).

[0144] Adding feature maps ResNet requires a depth-first 1x1 convolution for each feature map with a kernel of [w(0,0)=1] pointing to the same destination FM (see Figure 15). Therefore, apart from the offset values ​​specified herein, the source feature map (X,Y) uses the same message generation control data. Both convolutions are summed. This summation can further be merged with the subsequent convolution (H) due to the distributed nature of the convolution (the kernel becomes KernelSize×KernelSize). That is, (X+Y)*H=(X*H)+(Y*H). This results in a 50% reduction in the required number of processor cluster elements.

[0145] Combining flattening with a preceding convolution Figure 16 shows that convolution followed by flattening may be combined to reduce the required number of processor cluster elements. The pattern memory for the destination neural network layer indicates that the destination neural network layer is configured as a convolutional layer (i.e., X, Y, Z with KC>0) so that convolution is performed. The message generation control data for the source processor cluster in this case includes a flag ("flatten") to interpret the destination neural network layer DL as a 1x1xZ layer instead. See the top of Figure 16. In hardware, this simply adds three multiplexers for the (X, Y, Z) computation. See the bottom of Figure 16. N ID -FMStart=FMsizeY·FMsizeZ·X+FMsizeZ·Y+Z

[0146] Average Pooling As shown in Figure 17, the average pooling operation can also be combined with subsequent convolutions to be performed by simple convolutional layers using striding, thereby achieving a 50% reduction in processor cluster elements.

[0147] An example 3x3 convolution after average pooling As an example, Figure 18 shows how average pooling followed by a KS x KS convolution can be achieved in one step by an even (i.e., asymmetric) 2KernelSize x 2KernelSize convolution with a stride of 2. The resulting strided convolution has weights w'(-KernelSize+1,-KernelSize+1) to w'(KernelSize,KernelSize).

[0148] Thus, the average pooling followed by a 3x3 convolution simplifies to a single strided 6x6 convolution, as shown in Figure 19. The message control data in this example is ΔX min =KernelSize-1=2 ΔY min =KernelSize-1=2 KC=2KernelSize-1=5 Specify.

[0149] A more formal proof is given in Appendix 1.

[0150] Appendix 1 1 Proof that average pooling can be integrated with subsequent convolutions using strides The initial / input layer is I∈BF 2mx2m where BF denotes a 16-bit Brain floating-point number. Furthermore, the intermediate averaging layers and the output layer (after subsequent convolutions) are denoted by A∈BF. mxn and O∈BF mxn (Without loss of generality, zero-padded convolution is assumed here.) The output O is a KS × KS kernel (weights: ω i,j ), so the following expression represents the output layer:

number

number

number

number

number

number

number

[0151]

number

[0152] 1. Multiprocessor systems

Claims

1. A message-based multiprocessor system (1), comprising: - a message exchange network (R, L), a plurality of processor clusters (Ci,j) capable of exchanging messages with each other via said message exchange network, each processor cluster (Ci,j) comprising one or more processor cluster elements (PCE), and a message generator (MG); Equipped with the message-based multiprocessor system (1) is configurable as a neural network processor system having a plurality of neural network processing layers (NL1,...,NL5), each of the plurality of neural network processing layers being assigned one or more of the processor clusters with associated processor cluster elements being neural network processing elements in the neural network processing layer; A message-based multiprocessor system (1), wherein the message generator (MG) of a processor cluster (Ci,j) (associated with a neural network processing layer) comprises a logic module (MGL), an associated message generator control memory space (MGM) comprises respective message generation control data (CD1, CD2, CD3) for each destination processor cluster in a set of destination processor clusters, and the logic module of the message generator, responsive to an activation signal (Sact([X,Y])) of a processor cluster element, is configured to perform the respective computations using the message generation control data (CD1, CD2, CD3) for each destination processor cluster in the set of destination processor clusters to determine whether the respective destination processor cluster is a target of the processor cluster element and to selectively generate and send a message to each destination processor cluster determined as a target by the respective computation.

2. The logic module (MGL) comprises respective logic module sections (MGLX, MGLY) for calculating a possible destination range having a minimum coordinate value ([Xmin, Ymin]) and a maximum coordinate value ([Xmax, Ymax]) of each coordinate [X, Y] in the coordinate system of the destination processor cluster (PCD1) for a coordinate value of the processor cluster element (PCE) associated with the activation signal (Sact([X, Y])), and the logic module (MGL) calculates the calculated minimum value ([Xmin 2. The message-based multiprocessor system of claim 1, further comprising a further logic module section (MGLC) for determining whether a condition is met that at least one of the calculated minimum value ([Xmax,Ymin]) and the calculated maximum value ([Xmax,Ymax]) is within a corresponding range ([0,W);[0,H)) for that coordinate, and for enabling message transmission if the condition is met and for disabling message transmission if neither the calculated minimum value nor the calculated maximum value for any of the coordinates is within the corresponding range.

3. 3. The message-based multiprocessor system of claim 2, wherein the further logic module section (MGLC) comprises: a respective first comparator module (XMN, YMN) for providing, for each coordinate, a signal indicating that the calculated minimum value (Xmin, Ymin) for that coordinate is within the corresponding range ([0,W), [0,H)); and a respective second comparator module (XMX, YMX) for providing a first match signal indicating that the calculated maximum value (Xmax, Ymax) for that coordinate is within the corresponding range ([0,W), [0,H)); and further comprises a logical OR gate (ORX, ORY) for providing, for each coordinate, a second match signal indicating that at least one first match signal is valid; and the further logic module section (MGLC) further comprises a logical AND gate (ANDXY) for providing a message transmission enable signal if the further match signal for each coordinate is valid.

4. 4. The message-based multiprocessor system of claim 3, wherein the comparator module (XMN) comprises at least one mask register (MR) having a respective mask bit each representing a respective power of two, respective logic gates (BC2,...,BC6) for bit-wise comparison with a corresponding bit (Xmin[2],...,Xmin[6]) of the calculated minimum value (Xmin) or maximum value, and a combination module (BCM) for issuing an invalid match signal when at least one of the logic gates (BC2,...,BC6) indicates that a bit of the calculated minimum / maximum value is set but the corresponding mask bit is not set.

5. The control data further includes predetermined data indicating offsets (Xoffs, Yoffs), Xoffs = Xsrc0 - Xdst0 - ΔXmin, Yoffs=Ysrc0-Ydst0-ΔYmin 5. The message-based multiprocessor system of claim 1, wherein (Xsrc0, Ysrc0) is a coordinate pair representing a first location of the processor cluster that is a source of the message in an associated neural network processing layer, and (Xdst0, Ydst0) is a coordinate pair representing a first location of the processor cluster specified by the control data as a destination of the message in an associated neural network processing layer, and wherein the values ​​-ΔXmin, -ΔYmin are related to convolution kernel sizes Wx, Wy.

6. 6. A message-based multiprocessor system as described in any one of claims 1 to 5, wherein the control data further comprises an indicator (Flat) which, when set, indicates that the processor cluster elements of the destination processor cluster are arranged in one dimension and have coordinate values ​​in that dimension proportional to the index of the processor cluster element in the destination processor cluster.

7. 7. The message-based multiprocessor system of claim 1, wherein the control data further comprises an indicator (S2) specifying a stride change.

8. 8. The message-based multiprocessor system of claim 1, wherein the control data further comprises an indicator (UpSamp) specifying a scale factor.

9. 9. A message-based multiprocessor system as claimed in any one of claims 1 to 8, wherein the destination processor cluster further comprises a pattern memory function (PMEM), each entry in the PMEM specifying a spatial pattern of processor cluster elements in the space of the neural network processing layer associated with the destination processor cluster, and the control data further comprises a reference (PatternID) to an entry in the pattern memory function.

10. A method of operating a message-based multiprocessor system (1), said system comprising: - comprising a message switching network (R,L) and a plurality of processor clusters (Ci,j) capable of exchanging messages with each other via said message switching network, said processor clusters (Ci,j) comprising one or more processor cluster elements (PCE) and a message generator (MG) with a logic module (MGL) and a message generator control memory space (MGM), The method comprises: - in a preliminary stage, configuring said message-based multiprocessor system (1) as a neural network processor having a plurality of neural network processing layers (NL1,...,NL5), whereby to each neural network processing layer one or more respective subsets of said processor clusters with associated processor cluster elements, said associated processor cluster elements forming the neural network processing elements in said neural network processing layer; - during said preparation stage, writing respective sets of control data (CD1, CD2, CD3) for respective destination processor clusters in the subsequent neural network processing layer into respective storage entries of said message generator control storage space (MGM) of the source processor cluster; - during an operational phase, a source processor cluster element (PCE) of said source processor cluster activates said message generator (MG); In response to the activating step, the message generator, for each set (CD1, CD2, CD3) of control data for the source processor cluster: a) retrieving each said set of control data from said respective storage entry; b) performing a respective calculation using said message generation control data (CD1, CD2, CD3) for each destination processor cluster in said set of destination processor clusters to determine whether said respective destination processor cluster is a target of said processor cluster element; and c) if the result of said determination is positive, performing a step of sending an output message to said designated processor cluster; A method comprising:

11. 11. The method of claim 10, wherein the control data further comprises an indication of a message distribution pattern, the transmitted output message conveys the indication, and the destination processor cluster receiving the message applies the message to a set of core elements of the destination processor cluster according to a pattern specified by the indication.

Images