Sparsity-based mapping of neural networks to computational units in system-on-chips

Abstract

A method for an artificial neural network includes receiving a set of input values ​​to be convolved with a plurality of kernels for execution via a plurality of computational units. One or more thermally stressed computational units among the plurality of computational units are determined. The plurality of kernels are mapped to a plurality of computational units of a system on a chip (SOC) based on the one or more thermally stressed computational units. Convolution of the set of input values ​​with a sparsest kernel among the plurality of kernels is performed on the most thermally stressed computational unit.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. patent application No. 17 / 194,202, filed on March 5, 2021, entitled “SPARSITY-BASED NEURAL NETWORKMAPPING TO COMPUTING UNITS IN A System-on-chip,” the disclosure of which is expressly incorporated herein by reference in its entirety.

[0003] public domain

[0004] field

[0005] Aspects of the present disclosure relate generally to artificial neural networks, and more particularly to improved processing and mapping.

[0006] background

[0007] An artificial neural network may include groups of interconnected artificial neurons (e.g., neuron models). An artificial neural network may be a computing device or represented as a method to be executed by a computing device. A convolutional neural network (such as a deep convolutional neural network) is a feedforward artificial neural network. A convolutional neural network may include layers of neurons that may be arranged in a tiled receptive field.

[0008] Deep convolutional neural networks (DCNs) are used in various technologies, such as image recognition, speech recognition, autonomous driving, and Internet of Things (IoT) devices. Embedded IoT devices may have limited resources, such as limited on-chip memory. Therefore, the use of DCNs on these devices may be limited. It would be desirable to improve the use of DCNs on devices with limited resources.

[0009] Overview

[0010] In one aspect of the present disclosure, a method for an artificial neural network is provided. The method includes receiving a set of input values ​​to be convolved with a plurality of cores via a plurality of computational units. The method also includes determining one or more thermally stressed computational units from the plurality of computational units. The method additionally includes mapping the plurality of cores to a plurality of computational units of a system on a chip (SOC) based on the one or more thermally stressed computational units. Furthermore, the method includes performing a convolution of the set of input values ​​with the sparsest core among the plurality of cores on the computational unit with the greatest thermal stress.

[0011] In one aspect of the present disclosure, a device for an artificial neural network is provided. The device includes a memory and one or more processors coupled to the memory. The processor(s) are configured to receive a set of input values ​​to be convolved with a plurality of cores via a plurality of computing units. The processor(s) are further configured to determine one or more thermally stressed computing units among the plurality of computing units. In addition, the processor(s) are configured to map the plurality of cores to a plurality of computing units of a system on a chip (SOC) based on the one or more thermally stressed computing units. In addition, the processor(s) are configured to perform the convolution of the set of input values ​​with the sparsest core among the plurality of cores on the computing unit with the greatest thermal stress.

[0012] In one aspect of the present disclosure, a device for an artificial neural network is provided. The device includes a device for receiving a set of input values ​​to be convolved with a plurality of cores via a plurality of computing units. The device also includes a device for determining one or more thermally stressed computing units among the plurality of computing units. Additionally, the device includes a device for mapping the plurality of cores to a plurality of computing units of a system on a chip (SOC) based on the one or more thermally stressed computing units. In addition, the device includes a device for performing a convolution of the input value set with the sparsest core among the plurality of cores on the computing unit with the greatest thermal stress.

[0013] In one aspect of the present disclosure, a non-transitory computer-readable medium is provided. The computer-readable medium is encoded with program code for an artificial neural network. The program code is executed by a processor and includes code for receiving a set of input values ​​to be convolved with a plurality of cores via a plurality of computing units. The program code also includes code for determining one or more thermally stressed computing units among the plurality of computing units. Additionally, the program code includes code for mapping the plurality of cores to a plurality of computing units of a system on a chip (SOC) based on the one or more thermally stressed computing units. In addition, the program code includes code for performing a convolution of the input value set with the sparsest core among the plurality of cores on the computing unit with the greatest thermal stress.

[0014] Additional features and advantages of the present disclosure are described below. It will be appreciated by those skilled in the art that the present disclosure can be readily used as a basis for modifying or designing other structures for carrying out the same purposes as the present disclosure. It will also be appreciated by those skilled in the art that such equivalent constructions do not depart from the teachings of the present disclosure as set forth in the appended claims. The novel features which are believed to be characteristic of the present disclosure, both in terms of its organization and method of operation, together with further objects and advantages, will be better understood when the following description is considered in conjunction with the accompanying drawings. However, it is to be clearly understood that each of the figures is provided for illustration and description purposes only and is not intended as a definition of limitations of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The features, nature and advantages of the present disclosure will become more apparent when the detailed description set forth below is read in conjunction with the accompanying drawings, in which like reference numerals designate correspondingly throughout.

[0017] Figure 1 An example implementation of designing a neural network using a system on a chip (SOC) including a general-purpose processor in accordance with certain aspects of the present disclosure is illustrated.

[0018] Figure 2A 、 2B 2C are diagrams illustrating neural networks according to aspects of the present disclosure.

[0019] Figure 2D is a diagram illustrating an exemplary deep convolutional network (DCN) according to aspects of the present disclosure.

[0020] Figure 3 is a block diagram illustrating an exemplary deep convolutional network (DCN) according to aspects of the present disclosure.

[0021] Figure 4 is a block diagram illustrating an exemplary system on a chip (SOC) according to aspects of the present disclosure.

[0022] Figure 5A -B illustrates an example of remapping neural network partitions to compute units in a system on a chip (SOC) according to aspects of the present disclosure.

[0023] Figure 6 Illustrated are a pair of example heat maps illustrating temperature balancing and hot spot reduction in accordance with aspects of the present disclosure.

[0024] Figure 7 is a block diagram illustrating an architecture for a mapping process for a neural network in accordance with aspects of the present disclosure.

[0025] Figure 8 is a block diagram illustrating a neural network container file according to aspects of the present disclosure.

[0026] Figure 9A -C is a set of diagrams illustrating sparsity-based remapping according to aspects of the present disclosure.

[0027] Figure 10 is a block diagram illustrating exemplary sparsity-based remapping applied to mitigate temperature issues in accordance with aspects of the present disclosure.

[0028] Figure 11 A flow chart illustrating a method according to an aspect of the present disclosure is shown.

[0029] Detailed description

[0030] The detailed description set forth below in conjunction with the accompanying drawings is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. This detailed description includes specific details in order to provide a thorough understanding of the various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid obscuring such concepts.

[0031] Based on this teaching, it will be appreciated by those skilled in the art that the scope of the present disclosure is intended to cover any aspect of the present disclosure, regardless of whether it is implemented independently or in combination with any other aspect of the present disclosure. For example, any number of aspects set forth can be used to implement a device or practice method. In addition, the scope of the present disclosure is intended to cover such devices or methods that are practiced using other structures, functionality, or structure and functionality that are supplementary to or different from the various aspects of the present disclosure set forth. It should be understood that any aspect of the present disclosure disclosed can be implemented by one or more elements of the claims.

[0032] The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0033] Although specific aspects are described herein, numerous variations and permutations of these aspects fall within the scope of this disclosure. While some benefits and advantages of preferred aspects are mentioned, the scope of this disclosure is not intended to be limited to a particular benefit, use, or objective. On the contrary, various aspects of this disclosure are intended to be broadly applicable to different technologies, system configurations, networks, and protocols, some of which are illustrated as examples in the accompanying drawings and the following description of preferred aspects. The detailed description and drawings merely illustrate and do not limit the disclosure, the scope of which is defined by the appended claims and their equivalents.

[0034] A neural network can be divided into several sub-parts from a round of processing. These sub-parts or parts can be called blocks. These parts can be mapped to computing units, such as, for example, artificial intelligence (AI) accelerators, tensor processing units, neural signal processors (NSPs), neural processing units (NPUs), etc. A system on a chip (SOC) is an integrated circuit (IC) that integrates the components of a computing system (e.g., processing units, memory, input / output ports, and auxiliary storage). An SOC can include one or more such computing units on a single IC or chip to improve processing performance. As neural networks are processed, power density or temperature may not be uniform across the SOC. That is, even if tensors (matrices, blocks) of the same size are mapped to each computing unit, some parts of the SOC may have a higher temperature than other parts of the SOC. This is because traditionally, kernel convolutions are naively assigned to computing units in a preset order.

[0035] Under real-world conditions, compute units on the same SOC may have different temperatures and power consumption at the same operating frequency, voltage, and utilization. For example, the temperature differences may be due to differences in heat conduction paths, variations in on-die leakage, thermal coupling from other adjacent functional blocks, or differences in the workloads encountered by the corresponding compute units. Uneven temperatures (e.g., hot spots) can lead to performance inefficiencies—throttling (degradation by reducing operating frequency and executing pipelining). In some cases, hot spots can lead to thermal runaway (a rapid, uncontrolled temperature rise), which can trigger a SOC shutdown.

[0036] Various aspects of the present disclosure relate to improving the processing of artificial neural networks by mapping network partitions or blocks to computational units of a SOC based on statistical information about the cores of each layer of the artificial neural network (e.g., sparsity, sparsity percentage, or average effective bits of weight values), and in some aspects dynamically mapping to computational units of the SOC. For example, remapping can occur periodically at runtime, such as once every 30 seconds. In some aspects, the temperature or current consumption of each computational unit can be detected and monitored. Therefore, in some aspects, the detected temperature or current consumption can be used together with the statistical information so that mapping can be performed to reduce the detected temperature or current consumption of one or more computational units of the SOC.

[0037] Figure 1 An example implementation of a system on a chip (SOC) 100 according to certain aspects of the present disclosure is illustrated, which may include a central processing unit (CPU) 102 or a multi-core CPU configured to map neural network partitions to processing elements. Variables (e.g., neural signals and synaptic weights), system parameters associated with a computing device (e.g., a neural network with weights), latency, frequency bin information, and task information may be stored in a memory block associated with a neural processing unit (NPU) 108, a memory block associated with CPU 102, a memory block associated with a graphics processing unit (GPU) 104, a memory block associated with a digital signal processor (DSP) 106, a memory block 118, or may be distributed across multiple blocks. Instructions executed at CPU 102 may be loaded from a program memory associated with CPU 102 or may be loaded from memory block 118.

[0038] The SOC 100 may also include additional processing blocks tailored for specific functions, such as a GPU 104, a DSP 106, a connectivity block 110 (which may include fifth generation (5G) connectivity, fourth generation long term evolution (4G LTE) connectivity, Wi-Fi connectivity, USB connectivity, Bluetooth connectivity, etc.), and a multimedia processor 112 that may, for example, detect and recognize gestures. In one implementation, the NPU is implemented in the CPU, DSP, and / or GPU. The SOC 100 may also include a sensor processor 114, an image signal processor (ISP) 116, and / or a navigation module 120 (which may include a global positioning system).

[0039] The SOC 100 may be based on the ARM instruction set. In one aspect of the present disclosure, the instructions loaded into the general-purpose processor 102 may include code for receiving a set of input values ​​to be convolved with a plurality of cores for execution via a plurality of computing units. The general-purpose processor 102 may also include code for determining one or more thermally stressed computing units among the plurality of computing units. Additionally, the general-purpose processor 102 may include code for mapping the plurality of cores to a plurality of computing units of a system on a chip (SOC) based on the one or more thermally stressed computing units. The general-purpose processor 102 may further include code for performing a convolution of the input value set with the sparsest core among the plurality of cores on the computing unit with the greatest thermal stress.

[0040] Deep learning architectures perform object recognition tasks by learning to represent inputs at successively higher levels of abstraction at each layer, thereby constructing useful feature representations of the input data. In this way, deep learning addresses a major bottleneck of traditional machine learning. Before the advent of deep learning, machine learning approaches to object recognition problems might rely heavily on human-engineered features, perhaps combined with shallow classifiers. A shallow classifier might be, for example, a two-class linear classifier, where the weighted sum of the feature vector components is compared to a threshold to predict which class the input belongs to. Human-engineered features might be templates or kernels customized for a specific problem domain by engineers with domain expertise. In contrast, deep learning architectures learn to represent features similar to those that a human engineer might design, but they do so through training. Furthermore, deep networks can learn to represent and recognize new types of features that humans might not have considered.

[0041] Deep learning architectures can learn hierarchies of features. For example, if a first layer is presented with visual data, it can learn to recognize relatively simple features (such as edges) in the input stream. In another example, if a first layer is presented with auditory data, it can learn to recognize spectral power in specific frequencies. A second layer, taking the output of the first layer as input, can learn to recognize combinations of features, such as simple shapes for visual data or combinations of sounds for auditory data. For example, higher layers can learn to represent complex shapes in visual data or words in auditory data. Even higher layers can learn to recognize common visual objects or spoken phrases.

[0042] Deep learning architectures can perform particularly well when applied to problems with a natural hierarchical structure. For example, the classification of motor vehicles can benefit from first learning to recognize wheels, windshields, and other features. These features can then be combined in different ways at higher levels to identify cars, trucks, and airplanes.

[0043] Neural networks can be designed with various connectivity patterns. In a feedforward network, information is passed from lower layers to higher layers, with each neuron in a given layer communicating to neurons in a higher layer. As described above, hierarchical representations can be constructed in successive layers of a feedforward network. Neural networks can also have reflow or feedback (also known as top-down) connections. In a reflow connection, the output from a neuron in a given layer can be communicated to another neuron in the same layer. The reflow architecture can help identify patterns that span more than one chunk of input data delivered sequentially to the neural network. The connections from neurons in a given layer to neurons in lower layers are called feedback (or top-down) connections. A network with many feedback connections can be helpful when the recognition of high-level concepts can assist in discerning specific low-level features of the input.

[0044] The connections between layers of a neural network can be fully connected or partially connected. Figure 2A Illustrated is an example of a fully connected neural network 202. In the fully connected neural network 202, a neuron in a first layer may communicate its output to every neuron in a second layer, such that every neuron in the second layer will receive input from every neuron in the first layer. Figure 2BAn example of a locally connected neural network 204 is illustrated. In the locally connected neural network 204, neurons in a first layer may be connected to a limited number of neurons in a second layer. More generally, the locally connected layers of the locally connected neural network 204 may be configured such that each neuron in a layer will have the same or similar connectivity pattern, but their connection strengths may have different values ​​(e.g., 210, 212, 214, and 216). The locally connected connectivity pattern may produce spatially distinct receptive fields in higher layers because higher layer neurons in a given area may receive inputs that are tuned through training to properties of a limited portion of the total input to the network.

[0045] An example of a locally connected neural network is a convolutional neural network. Figure 2C An example of a convolutional neural network 206 is illustrated. The convolutional neural network 206 can be configured such that the connection strengths associated with the inputs to each neuron in the second layer are shared (e.g., 208). Convolutional neural networks may be well suited for problems where the spatial location of the inputs is meaningful.

[0046] One type of convolutional neural network is a deep convolutional network (DCN). Figure 2D A detailed example of a DCN 200 designed to recognize visual features from an image 226 input from an image capture device 230 (such as an onboard camera) is illustrated. The DCN 200 of the current example can be trained to identify traffic signs and the numbers provided on traffic signs. Of course, the DCN 200 can be trained for other tasks, such as identifying lane markings or identifying traffic lights.

[0047] DCN 200 can be trained using supervised learning. During training, an image (such as image 226 of a speed limit sign) can be presented to DCN 200, and a forward pass can then be computed to produce output 222. DCN 200 can include a feature extraction section and a classification section. Upon receiving image 226, convolution layer 232 can apply a convolution kernel (not shown) to image 226 to generate a first set of feature maps 218. As an example, the convolution kernel of convolution layer 232 can be a 5x5 kernel that generates a 28x28 feature map. In this example, since four different feature maps are generated in first set of feature maps 218, four different convolution kernels are applied to image 226 at convolution layer 232. Convolution kernels can also be referred to as filters or convolution filters.

[0048] The first set of feature maps 218 may be subsampled by a max pooling layer (not shown) to generate a second set of feature maps 220. The max pooling layer reduces the size of the first set of feature maps 218. That is, the size of the second set of feature maps 220 (such as 14x14) is smaller than the size of the first set of feature maps 218 (such as 28x28). The reduced size provides similar information to subsequent layers while reducing memory consumption. The second set of feature maps 220 may be further convolved via one or more subsequent convolutional layers (not shown) to generate one or more subsequent sets of feature maps (not shown).

[0049] exist Figure 2D In the example of , the second set of feature maps 220 is convolved to generate a first feature vector 224. In addition, the first feature vector 224 is further convolved to generate a second feature vector 228. Each feature of the second feature vector 228 may include a number corresponding to a possible feature of the image 226 (such as "sign," "60," and "100"). A softmax function (not shown) may convert the numbers in the second feature vector 228 into probabilities. Thus, the output 222 of the DCN 200 is the probability that the image 226 includes one or more features.

[0050] In this example, the probabilities for "logo" and "60" in output 222 are higher than the probabilities for other features of output 222 (such as "30," "40," "50," "70," "80," "90," and "100"). Before training, output 222 generated by DCN 200 is likely incorrect. Thus, the error between output 222 and the target output can be calculated. The target output is the true value of image 226 (e.g., "logo" and "60"). The weights of DCN 200 can then be adjusted to more closely align output 222 of DCN 200 with the target output.

[0051] To adjust the weights, the learning algorithm can calculate a gradient vector for the weights. The gradient can indicate the amount by which the error will increase or decrease if the weights are adjusted. At the top layer, the gradient can directly correspond to the value of the weight connecting the activated neurons in the penultimate layer to the neurons in the output layer. In lower layers, the gradient can depend on the value of the weights and the calculated error gradient of the higher layers. The weights can then be adjusted to reduce the error. This way of adjusting weights can be called "backpropagation" because it involves a backward pass through the neural network.

[0052] In practice, the error gradient of the weights may be calculated over a small number of examples so that the calculated gradient approximates the true error gradient. This approximation method may be called stochastic gradient descent. Stochastic gradient descent may be repeated until the error rate achievable by the entire system has stopped decreasing or until the error rate has reached a target level. After learning, a new image (e.g., image 226 of a speed limit sign) may be presented to the DCN and a forward pass through the network may produce output 222, which may be considered an inference or prediction of the DCN.

[0053] Deep Belief Network (DBN) is a probabilistic model comprising multiple layers of hidden nodes. DBN can be used to extract hierarchical representations of training data sets. DBN can be obtained by stacking multiple layers of restricted Boltzmann machines (RBM). RBM is a type of artificial neural network that can learn probability distributions on input sets. Since RBM can learn probability distributions without information about which class each input should be classified into, RBM is often used in unsupervised learning. Using a hybrid unsupervised and supervised paradigm, the bottom RBM of the DBN can be trained in an unsupervised manner and can be used as a feature extractor, while the top RBM can be trained in a supervised manner (on the joint distribution of the input and target class from the previous layer) and can be used as a classifier.

[0054] A deep convolutional network (DCN) is a network of convolutional networks with additional pooling and normalization layers. DCNs have achieved state-of-the-art performance on many tasks. DCNs can be trained using supervised learning, where both the input and output targets are known for many examples and are used to modify the network weights using gradient descent.

[0055] The DCN can be a feedforward network. In addition, as described above, the connections from neurons in the first layer of the DCN to the neuron groups in the next higher layer are shared across the neurons in the first layer. The feedforward and shared connections of the DCN can be used to perform fast processing. The computational burden of the DCN can be much smaller than, for example, the computational burden of a similarly sized neural network that includes recurrent or feedback connections.

[0056] The processing of each layer of the convolutional network can be thought of as a spatially invariant template or basis projection. If the input is first decomposed into multiple channels, such as the red, green, and blue channels of a color image, the convolutional network trained on this input can be thought of as three-dimensional, with two spatial dimensions along the axes of the image and a third dimension that captures color information. The output of the convolutional connection can be thought of as forming a feature map in the subsequent layer, each element in which receives input from a range of neurons in the previous layer (e.g., feature map 218) and from each of the multiple channels. The values ​​in the feature map can be further processed with nonlinearities (such as rectification, max(0,x)). The values ​​from adjacent neurons can be further pooled (which corresponds to downsampling) and can provide additional local invariance and dimensionality reduction. Normalization can also be applied by lateral inhibition between neurons in the feature map, which corresponds to whitening.

[0057] The performance of deep learning architectures improves as more labeled data points become available or as computing power increases. Modern deep neural networks are routinely trained using thousands of times more computing resources than were available to typical researchers just fifteen years ago. New architectures and training paradigms can further boost deep learning performance. Rectified linear units can alleviate the training problem known as vanishing gradients. New training techniques can reduce overfitting and thus enable larger models to achieve better generalization. Encapsulation techniques can abstract the data within a given receptive field and further improve overall performance.

[0058] Figure 3 3 is a block diagram illustrating a deep convolutional network 350. The deep convolutional network 350 may include multiple layers of different types based on connectivity and weight sharing. Figure 3 As shown in FIG, the deep convolutional network 350 includes convolution blocks 354A and 354B. Each of the convolution blocks 354A and 354B may be configured with a convolution layer (CONV) 356, a normalization layer (LNorm) 358, and a maximum pooling layer (MAX POOL) 360.

[0059] The convolution layer 356 may include one or more convolution filters that can be applied to the input data to generate a feature map. Although only two convolution blocks 354A, 354B are shown, the present disclosure is not limited thereto, and instead any number of convolution blocks 354A, 354B may be included in the deep convolutional network 350 according to design preferences. The normalization layer 358 may normalize the output of the convolution filter. For example, the normalization layer 358 may provide whitening or lateral suppression. The maximum pooling layer 360 may provide spatial downsampling aggregation to achieve local invariance and dimensionality reduction.

[0060] For example, the parallel filter banks of the deep convolutional network can be offloaded onto the CPU 102 or GPU 104 of the SOC 100 to achieve high performance and low power consumption. In alternative embodiments, the parallel filter banks can be offloaded onto the DSP 106 or ISP 116 of the SOC 100. In addition, the deep convolutional network 350 can access other processing blocks that may be present on the SOC 100, such as the sensor processor 114 and the navigation module 120 dedicated to sensors and navigation, respectively.

[0061] The deep convolutional network 350 may also include one or more fully connected layers 362 (FC1 and FC2). The deep convolutional network 350 may further include a logistic regression (LR) layer 364. Between each layer 356, 358, 360, 362, 364 of the deep convolutional network 350 are weights (not shown) to be updated. The output of each layer (e.g., 356, 358, 360, 362, 364) can be used as input to a subsequent layer (e.g., 356, 358, 360, 362, 364) in the deep convolutional network 350 to learn a hierarchical feature representation from the input data 352 (e.g., image, audio, video, sensor data, and / or other input data) supplied from the first convolutional block 354A. The output of the deep convolutional network 350 is a classification score 366 for the input data 352. The classification score 366 can be a set of probabilities, where each probability is a probability that the input data includes a feature from a feature set.

[0062] Figure 4 4 is a block diagram illustrating a system on a chip (SOC) 400 according to aspects of the present disclosure. The SOC 400 may include one or more computing units 402a-z. Each of the computing units 402a-z may be, for example, a neural processing unit (NPU), a neural signal processor (NSP), a tensor processing unit (TPU), an artificial intelligence accelerator, or other processing unit. The computing units (e.g., 402a-z) of the SOC 400 may be interconnected via a bus 404 so that the computing units (e.g., 402a-z) may execute artificial neural networks, such as Figure 3 Convolutional neural network 350. However, the present disclosure is not so limited, and a fabric, a network on a chip (NOC), or any suitable interconnect may interconnect the computing units.

[0063] The temperature and power consumption of the compute units (e.g., 402a-z) across the SOC 400 can vary. Under real-world conditions, the compute units 402a-z on the same SOC 400 may have different temperatures and power consumption at the same operating frequency, voltage, and utilization. For example, these variations can be due to different thermal resistances and conduction paths from the compute units 402a-z to the SOC 400 package (e.g., the SOC floorplan), differences in heat from neighboring functional blocks (e.g., neighboring compute units), on-die variations in silicon leakage (e.g., current (Idd) current consumption in quiescent state (IDDQ)), and differences in the sparsity of weights of the blocks mapped to the compute units 402a-z.

[0064] Temperature variations (e.g., hot spots) in the compute units 402a-z may cause performance inefficiencies—throttling (degradation by reducing operating frequency and pipelining). In some cases, hot spots may lead to thermal runaway (rapid, uncontrolled temperature rise), which may trigger a SOC shutdown.

[0065] Figure 5A -B illustrates an example of remapping neural network partitions to compute units in a SOC according to aspects of the present disclosure. Figure 5A is a block diagram illustrating the mapping of neural networks to computational units according to aspects of the present disclosure. Figure 5A As shown, a set of input values ​​502a-n can be received for processing via input 510. The set of input values ​​502a-n can represent an image, for example. The set of input values ​​502a-n can be processed via a neural network to derive an output (e.g., image classification). In some aspects, input 510 can be different from or the same as other input blocks. The set of input values ​​502a-n can be convolved with a set of weight matrices or kernels (e.g., kernels #0-7) to produce a set of output values ​​504a-m via output 512. In some aspects, output 512 can be different from or the same as other output blocks. The output values ​​504a-m can be provided as input values ​​to subsequent layers of the neural network to operate the neural network to perform a desired task (e.g., classification of the input image).

[0066] like Figure 5AAs illustrated, the operation of the artificial neural network can be performed via parallel processing. The input value sets 502a-n can be considered as blocks that can be mapped to individual computational units (e.g., computational units 402a-z) for performing convolution operations. For example, the input values ​​502a-n can be convolved in parallel with each of eight kernels (e.g., kernel #0-kernel #7). Each such block kernel convolution can be performed on a separate computational unit (e.g., 520a) to generate a layer of output values ​​(504a-m). For example, the operation of convolving the input value sets 502a-n with a kernel (e.g., kernel #0) can be assigned to computational unit 520a to produce output layer #0 of output values ​​(e.g., 504a-m). In another example, the operation of convolving the input value sets 502a-n with kernel #1 can be assigned to computational unit 520b to produce output layer #1 of output values ​​(e.g., 504a-m). It should be noted that although cores #0-7 are shown, the number of cores is not limited thereto, and any number of cores may be used.

[0067] Figure 5B The operation of an artificial network via parallel processing utilizing dynamic mapping (or remapping) to computational units according to aspects of the present disclosure is illustrated. Figure 5B As shown, a sparsity parameter (e.g., the number of zero values) can be determined for each core (e.g., cores #0-7). In some aspects, the sparsity parameter can be a binary label for each core (e.g., sparse / non-sparse), or a ranking of the cores in order of sparsity (whether or not a specific sparsity level is calculated), or other sparsity metrics. After the sparsity of each core (e.g., cores #0-7) is determined, the cores can be sorted and arranged or arranged according to the determined sparsity (e.g., in order of decreasing sparsity). Figure 5B In the example shown in FIG. 1 , core #3 has the greatest sparsity among the cores and is therefore ranked first in the order. On the other hand, core #5 has the least sparsity among the cores and is therefore ranked last in the order. Thus, a block or operation that convolves an input portion or quantity (e.g., 552 a-n) with a core (cores #0-7) can be mapped to a computational unit (e.g., computational units 402 a-z) based on the determined sparsity order.

[0068] In some aspects, the temperature can also be measured via a temperature sensor (e.g., Figure 11 . The temperature at or near each computing unit (e.g., computing units 402a-z) is detected by a temperature sensor (e.g., sensor 114) in the SOC. Additionally, the temperature at or near the computing units can be continuously monitored to protect the device throughout the operation of the neural network. Element 560 is a chart graphically illustrating the temperatures of exemplary computing units NSP#0-7 of the SOC. As shown in element 560, computing unit NSP#1 has the highest temperature and is a hot spot on the SOC. On the other hand, the temperature sensor (e.g., sensor 114) has detected that computing unit NSP#7 has the lowest temperature. According to aspects of the present disclosure, the mapping of neural networks to computing units can be determined based on the temperatures detected at or near the computing units.

[0069] By way of example only and not limitation, Figure 5B As shown, a block (e.g., 570a) for operations that convolve a kernel with the greatest sparsity (e.g., kernel #3) with a corresponding input portion or quantity can be assigned or mapped to the computational unit (e.g., NSP #1) with the highest detected temperature. In doing so, the workload of NSP #1 can be reduced because the number of convolution operations to be performed is reduced, or because kernel #3 can be the smallest kernel among the kernels. This is because when a convolution operation includes a weight value of zero, the corresponding multiplication and accumulation operations can be skipped. Consequently, the temperature subsequently detected at or near that computational unit (e.g., NSP #1) can be reduced. Similarly, if computational unit NSP #2 is determined to have the next highest detected temperature, a block (e.g., 570b) for operations that convolve a kernel with the next greatest sparsity (e.g., kernel #7) with a corresponding input portion or quantity (e.g., 552a-n) can be assigned or mapped to the computational unit (e.g., NSP #2) with the next highest detected temperature. Remapping can be performed in this manner until all neural network partitions have been processed. For example, computation unit NSP#7 is determined to have the lowest detected temperature, and therefore, a block (e.g., 570n) for convolving a kernel (e.g., kernel #5) with the smallest sparsity with the corresponding input (e.g., 552a-n) can be assigned or mapped to the computation unit (e.g., NSP#7) with the lowest detected temperature. Computation unit NSP#7 can therefore perform the convolution of kernel #5 with the corresponding input (e.g., 552a-n) to produce output #5. Each output portion can then be provided to a subsequent layer of the neural network to perform a desired task.

[0070] In this way, the temperatures of the compute units (e.g., NSP #0-7) across the SOC can be balanced, thereby reducing hot spots to some extent. This is because temperature is correlated with power consumption and is inversely correlated with (one hundred percent (100%) - sparsity percentage). That is, as the sparsity of cores mapped to a compute unit increases, the number of calculations to be performed by such a compute unit decreases, thereby reducing power consumption and temperature. Furthermore, temperature balancing and hot spot reduction can be beneficially performed without reducing operating frequency or stopping the processing of commands or threads.

[0071] Figure 6 A pair of example heat maps showing temperature balancing and hot spot reduction according to aspects of the present disclosure are illustrated. Figure 6 , heat map 600 illustrates a non-uniform thermal distribution of SOC 602. For example, temperatures detected via one or more temperature sensors (e.g., sensor 114) may be reflected in the heat map with varying degrees of shading. A higher degree of shading (e.g., a darker shading) illustrates a higher temperature in the indicated area compared to areas with a lower degree of shading (e.g., a lighter shading). For example, areas 604a-f are shown as having a higher degree of shading than other areas of SOC 602 and may be considered hot spots. Heat map 650 illustrates an exemplary thermal distribution of SOC 602 while processing a neural network after a remapping process based on sparsity and temperature neural network partitioning. The processing of the neural network has been remapped so that convolution operations involving kernels with higher sparsity (e.g., a larger number of zero values) are mapped to computational units (e.g., NSPs) at or near higher temperatures in regions 604a-f of the heat map 600, and convolution operations involving kernels with lower sparsity (e.g., a lower number of zero values) are mapped to computational units (NSPs) at or near lighter shaded regions, with corresponding regions 654a-f of the heat map 650 being shown with a lower degree of shading. The lower degree of shading indicates that a lower temperature is detected for such corresponding regions 654a-f than for regions 604a-f.

[0072] Figure 7 is a block diagram illustrating an architecture 700 for a remapping process for a neural network according to aspects of the present disclosure. Figure 7, architecture 700 includes temperature sensors 702, current sensors 704, kernel statistics unit 706, metadata unit 708, dynamic mapping unit 710, and compute units 712. As conditions relative to the SOC (e.g., SOC 400) change during neural network operation, one or more temperature sensors 702 can detect the temperature of each compute unit (e.g., 402a-z) and surrounding areas and / or components. Similarly, one or more current sensors 704 can detect the current consumption of each compute unit (e.g., 402a-z). Temperature sensors 702 and current sensors 704 can be used to continuously monitor the temperature and current consumption of the compute units, respectively, during runtime operation of the neural network.

[0073] The kernel statistics unit 706 can calculate statistics for each kernel in each layer of the neural network. The calculated kernel statistics can include, for example, the sparsity of weight values ​​in the kernel, the sparsity percentage of weight values ​​in each kernel, and / or the average of the significant bits of weights in each kernel. The calculated kernel statistics can be provided to the metadata unit 708.

[0074] The data processing unit 714 can receive the kernel (weight values) of the neural network and can perform error correction, channel decoding, channel or feature map pruning, intra-kernel level pruning, training to induce weight sparsity, quantization, regularization (e.g., L1 regularization), or otherwise adapt the data or kernel (e.g., weight values). The data processing can be performed before processing the neural network. In operation, the data processing unit 714 can provide the adapted kernel information to the kernel statistics unit 706 for calculating kernel statistics (e.g., sparsity, sparsity percentage, mean effective bit, or other kernel statistics). The metadata unit 708 can, in turn, receive the kernel statistics information and generate metadata that can be provided to the dynamic mapping unit 710.

[0075] The dynamic mapping unit 710 receives core statistics (e.g., sparsity, sparsity percentage, average effective bits, etc.) included in the metadata and can allocate processing of the neural network based on the core statistics. The temperature sensor 702 and the current sensor 704 can provide detected temperature information and detected current information, respectively, to the dynamic mapping unit 710. In turn, the dynamic mapping unit 710 can allocate computing units 712 for processing the neural network based on one or more of the temperature information, the current information, and the metadata including the core statistics. In some aspects, computing unit information (e.g., configuration information, processing capabilities, and processing workload) can be provided to the dynamic mapping unit 710 to allocate processing of the neural network.

[0076] Figure 8800 is a block diagram illustrating a neural network container file 800 according to aspects of the present disclosure. The neural network container file 800 may include network architecture information 802 (e.g., a description of the network architecture), network parameters 804, and tensor information about layers 806 of the neural network. Additionally, the neural network container file 800 may include metadata 808, such as kernel statistics (e.g., sparsity, sparsity percentage per kernel per layer, or mean effective bits). In some aspects, the neural network container file 800 may be generated at compile time. Thus, when a SOC (e.g., SOC 400) loads data or the neural network container file 800 into memory, the information included in the neural network container file (e.g., metadata 808) may be used to dynamically determine the mapping to compute units (e.g., compute units 402a-z).

[0077] Figure 9A -C is a set of diagrams illustrating sparsity-based remapping according to various aspects of the present disclosure. Figure 9A , a set of kernels 902 can be applied to an input image, for example, to generate a set of output values ​​(e.g., an output feature map). The output values ​​can be provided to subsequent layers of the neural network to perform the desired task (e.g., image classification). Each of kernels 0-7 can be pruned to induce sparsity. That is, the kernel can be encouraged to have more zero values. Pruning can be applied to all kernels or on an intra-kernel basis. Additionally, pruning can be performed using a training algorithm to induce weight sparsity (e.g., iteratively setting low-value weights to zero and continuing to train the network) (e.g., via quantization (e.g., rounding low-value weights to zero) or via regularization during training (e.g., L1 regularization)).

[0078] Low weight values ​​are represented by darker shading in the set of kernels 902a. For example, in kernel 0, elements a, b, f, g, and h have darker shading indicating lower values. Similarly, in kernel 2, elements a, b, d, e, f, g, and h have lower values.

[0079] like Figure 9B As shown, core 902 ( Figure 9A ) have been set to zero, as indicated by the black shaded elements. For example, in core 0, elements a, b, f, g, and h have been set to zero, as indicated by the black shading. Similarly, in core 2, elements a, b, d, e, f, g, and h have been set to zero, as indicated by the black shading.

[0080] like Figure 9C As shown, the set of cores 902 can be further sorted according to sparsity. For example, core 2 ( Figure 9B ) has maximum sparsity (zero values ​​indicated by black shading) in core 902 and is mapped to core 0 (in Figure 9CCore 0 (in Figure 9B ) has the second largest sparsity in core 902 (zero values ​​indicated by black shading) and is mapped to core 0 (in Figure 9C On the other hand, core 3 (in Figure 9B ) has the least sparsity and can be mapped to core 7 (in Figure 9C middle).

[0081] Figure 10 1004 . This is a block diagram illustrating an exemplary sparsity-based remapping process for alleviating temperature issues according to various aspects of the present disclosure. A set of blocks are initially mapped to a set of virtual IDs of compute units (see 1002). A sparsity percentage for each mapped block is also provided. The virtual IDs of the compute units are sorted based on the sparsity percentage of the mapped blocks 1004. Although in this example, blocks are mapped to virtual IDs or virtual compute units, the present disclosure is not limited thereto, and various aspects of the present disclosure can be implemented without using virtual compute units.

[0082] Additionally, the neural network may also determine and monitor metrics regarding the condition and operation of the physical computing units. In this exemplary aspect, as shown in table 1008, the temperature of each computing unit is monitored. Of course, as previously disclosed, other metrics or statistics may be determined and / or monitored. Figure 10 As shown, the temperature distribution between the physical compute units is non-uniform. Notably, physical compute unit 2 operates at a temperature of 103 degrees Celsius, while physical compute unit 7 operates at a temperature of 91 degrees Celsius. To improve the temperature distribution between the compute units, the temperatures can be sorted from highest to lowest, and the blocks can be mapped to the physical compute units based on the sparsity percentage (see Table 1006). In this way, the compute unit with the highest temperature (e.g., compute unit 2 at 103 degrees Celsius) is mapped to the block with the highest sparsity percentage (e.g., virtual ID 2 with 88% sparsity), while the compute unit with the lowest temperature (e.g., compute unit 7 at 91 degrees Celsius) is mapped to the block with the lowest sparsity percentage (e.g., virtual ID 3 with 0%). Therefore, the temperature of the compute units of the SOC can be reduced.

[0083] Figure 11 10 illustrates a flow chart of a method 1100 according to an aspect of the present disclosure. At block 1102, the method 1100 receives a set of input values ​​to be convolved with a plurality of kernels for execution via a plurality of computational units. Figure 5A As shown, a set of input values ​​502a-n may be received for processing via input 510. The set of input values ​​502a-n may, for example, represent an image. The set of input values ​​502a-n may be processed via a neural network to derive an output (e.g., an image classification).

[0084] At block 1104, the method 1100 determines at least one thermal stress calculation unit from the plurality of calculation units. Figure 5B As discussed, it is also possible to use a temperature sensor (e.g. Figure 1 1 . The temperature at or near each computing unit (e.g., computing units 402a-z) is detected by a temperature sensor (e.g., sensor 114) in the SOC. Additionally, the temperature at or near the computing units can be continuously monitored to protect the device throughout the operation of the neural network. Element 560 is a chart graphically illustrating the temperatures of exemplary computing units NSP#0-7 of the SOC. As shown in element 560, computing unit NSP#1 has the highest temperature and is a hot spot on the SOC. On the other hand, the temperature sensor (e.g., sensor 114) has detected that computing unit NSP#7 has the lowest temperature. According to aspects of the present disclosure, the mapping of neural networks to computing units can be determined based on the temperatures detected at or near the computing units.

[0085] At block 1106, the method 1100 maps the plurality of cores to a plurality of computing units of a system on chip (SOC) based on the at least one thermally stressed computing unit. Figure 5B As shown, the operation of convolving the kernel with the greatest sparsity (e.g., kernel #3) with the corresponding input portion or amount can be assigned or mapped to the computational unit with the highest detected temperature (e.g., NSP #1). In doing so, the workload of NSP #1 can be reduced because the number of convolution operations to be performed is reduced, or because kernel #3 can be the smallest kernel among the kernels. This is because when a convolution operation includes a weight value of zero, the corresponding multiplication and accumulation operation can be skipped.

[0086] At block 1108, the method 1100 performs a convolution of the input value set with the sparsest kernel among the plurality of kernels on the computational unit with the greatest thermal stress. Figure 5B As discussed, computation unit NSP#7 is determined to have the lowest detected temperature, and therefore, a block (e.g., 570n) for convolving a kernel (e.g., kernel #5) with the smallest sparsity with the corresponding input (e.g., 552a-n) can be assigned or mapped to the computation unit (e.g., NSP#7) with the lowest detected temperature. Thus, computation unit NSP#7 can perform the convolution of kernel #5 with the corresponding input (e.g., 552a-n) to produce output #5. Each output portion can then be provided to a subsequent layer of the neural network to perform the desired task.

[0087] In one aspect, the receiving means, determining means, mapping means, and / or means for performing convolution may be the CPU 102, a program memory associated with the CPU 102, a dedicated memory block 118, a fully connected layer 362, and / or a routing connection processing unit 216 configured to perform the functions recited. In another configuration, the aforementioned means may be any module or any equipment configured to perform the functions recited by the aforementioned means.

[0088] The various operations of the methods described above may be performed by any suitable device capable of performing the corresponding functions. These devices may include various hardware and / or software components and / or modules, including but not limited to circuits, application-specific integrated circuits (ASICs), or processors. Generally speaking, where there are operations illustrated in the figures, those operations may have corresponding counterpart means-plus-function components with similar numbering.

[0089] Implementation examples are provided in the following numbered clauses:

[0090] 1. A method for an artificial neural network, comprising:

[0091] receiving a set of input values ​​to be convolved with a plurality of kernels for execution via a plurality of computational units;

[0092] determining at least one thermal pressure calculation unit among the plurality of calculation units;

[0093] mapping the plurality of cores to a plurality of computing units of a system on chip (SOC) based on the at least one thermally stressed computing unit; and

[0094] Convolution of the set of input values ​​with the sparsest kernel among the plurality of kernels is performed on the most thermally stressed compute unit.

[0095] 2. The method of clause 1, further comprising:

[0096] determining a first order of the computational units based on an amount of thermal stress and determining a second order of the cores based on a sparsity metric; and

[0097] Each of the plurality of cores is assigned to one of the plurality of computing units based on a first order and a second order.

[0098] 3. The method of clause 2, wherein the first order comprises the plurality of cores arranged in a layer according to decreasing sparsity, and the second order comprises the plurality of computational units arranged according to increasing thermal tolerance; and

[0099] Each core is assigned to one of the plurality of compute units in order such that the least sparse cores are assigned to the least thermally stressed compute unit and the most sparse cores are assigned to the most thermally stressed compute unit.

[0100] 4. The method of clause 1, further comprising: detecting at least one of a temperature or a current draw of each of a plurality of computing units of the SOC, and wherein the mapping is based on the temperature or the current draw.

[0101] 5. The method of clause 4, wherein mapping is performed for a computing unit of the plurality of computing units having a temperature or current exceeding a threshold.

[0102] 6. The method of clause 1, further comprising: calculating statistical information for each core of the plurality of cores, and wherein the mapping is further based on the statistical information.

[0103] 7. The method of any of clauses 1-6, wherein the mapping comprises dynamically assigning the plurality of cores to a plurality of computing units of the SOC at runtime.

[0104] 8. An apparatus for an artificial neural network, comprising:

[0105] Memory; and

[0106] at least one processor coupled to the memory, the at least one processor configured to:

[0107] receiving a set of input values ​​to be convolved with a plurality of kernels for execution via a plurality of computational units;

[0108] determining at least one thermal pressure calculation unit among the plurality of calculation units;

[0109] mapping the plurality of cores to a plurality of computing units of a system on chip (SOC) based on the at least one thermally stressed computing unit; and

[0110] Convolution of the set of input values ​​with the sparsest kernel among the plurality of kernels is performed on the most thermally stressed compute unit.

[0111] 9. The apparatus of clause 8, wherein the at least one processor is further configured to:

[0112] determining a first order of the computational units based on an amount of thermal stress and determining a second order of the cores based on a sparsity metric; and

[0113] Each of the plurality of cores is assigned to one of the plurality of computing units based on a first order and a second order.

[0114] 10. The apparatus of clause 9, wherein the first order comprises the plurality of cores arranged in a layer according to decreasing sparsity, and the second order comprises the plurality of computational units arranged according to increasing thermal tolerance; and

[0115] Wherein the at least one processor is further configured to assign each core to one of the plurality of compute units in order such that the least sparse cores are assigned to the least thermally stressed compute unit and the sparsest cores are assigned to the most thermally stressed compute unit.

[0116] 11. The apparatus of clause 8, wherein the at least one processor is further configured to:

[0117] detecting at least one of a temperature or a current consumption of each of a plurality of computing units of the SOC, and

[0118] The plurality of cores are mapped to the plurality of computing units based on temperature or current consumption.

[0119] 12. The apparatus of clause 11, wherein the at least one processor is further configured to assign a computing unit of the plurality of computing units having a temperature or current exceeding a threshold.

[0120] 13. The apparatus of clause 8, wherein the at least one processor is further configured to:

[0121] calculating statistics for each of the plurality of cores; and

[0122] The plurality of cores are mapped to the plurality of computing units based on the statistical information.

[0123] 14. The apparatus of any of clauses 8-13, wherein the at least one processor is further configured to dynamically assign the plurality of cores to a plurality of computing units of the SOC at runtime.

[0124] 15. An apparatus for an artificial neural network, comprising:

[0125] means for receiving a set of input values ​​to be convolved with a plurality of kernels for execution via a plurality of computational units;

[0126] means for determining at least one thermal stress calculation unit among the plurality of calculation units;

[0127] means for mapping the plurality of cores to a plurality of computing units of a system on a chip (SOC) based on the at least one thermally stressed computing unit; and

[0128] Means for performing, on the most thermally stressed compute unit, a convolution of the set of input values ​​with the sparsest kernel among the plurality of kernels.

[0129] 16. The apparatus of clause 15, further comprising:

[0130] means for determining a first order of the computational elements based on the amount of thermal stress and determining a second order of the cores based on the sparsity measure; and

[0131] Means for assigning each of the plurality of cores to one of the plurality of computing units based on a first order and a second order.

[0132] 17. The apparatus of clause 16, wherein the first order comprises a plurality of cores arranged in a layer according to decreasing sparsity and the second order comprises a plurality of computational units arranged according to increasing thermal stress; and further comprising means for assigning each core to one of the plurality of computational units in order such that the least sparse core is assigned to the computational unit with the least thermal stress and the sparsest core is assigned to the computational unit with the greatest thermal stress.

[0133] 18. The apparatus of clause 15, further comprising:

[0134] means for detecting at least one of a temperature or a current draw of each of a plurality of computing units of the SOC; and

[0135] Means for assigning the plurality of cores to the plurality of computing units based on temperature or current consumption.

[0136] 19. The apparatus of clause 15, further comprising: means for assigning a core of the plurality of cores to a computing unit of the plurality of computing units having a temperature or current exceeding a threshold.

[0137] 20. The apparatus of clause 15, further comprising: means for computing statistical information for each core of the plurality of cores; and means for mapping cores of the plurality of cores to compute units of the plurality of compute units based on the statistical information.

[0138] 21. The apparatus of any of clauses 15-20, further comprising: means for dynamically assigning the plurality of cores to a plurality of computing units of the SOC at runtime.

[0139] 22. A non-transitory computer-readable medium having recorded thereon a program code for an artificial neural network, the program code being executed by a processor and comprising:

[0140] program code for receiving a set of input values ​​to be convolved with a plurality of kernels for execution via a plurality of computational units;

[0141] program code for determining at least one thermal stress calculation unit among the plurality of calculation units;

[0142] program code for mapping the plurality of cores to a plurality of compute units of a system on a chip (SOC) based on the at least one thermally stressed compute unit; and

[0143] Program code is provided for performing, on a maximum thermally stressed compute unit, a convolution of the set of input values ​​with a sparsest kernel among the plurality of kernels.

[0144] 23. The non-transitory computer-readable medium of clause 22, further comprising:

[0145] Program code for determining a first order of the compute units based on the amount of thermal stress and determining a second order of the cores based on the sparsity measure; and

[0146] Program code is provided for assigning each of the plurality of cores to one of the plurality of compute units based on a first order and a second order.

[0147] 24. The non-transitory computer-readable medium of clause 23, wherein the first order comprises the plurality of cores arranged in a layer according to decreasing sparsity, and the second order comprises the plurality of computing units arranged according to increasing thermal tolerance; and

[0148] Further included is program code for assigning each core to one of the plurality of compute units in order such that the least sparse cores are assigned to the least thermally stressed compute unit and the most sparse cores are assigned to the most thermally stressed compute unit.

[0149] 25. The non-transitory computer-readable medium of clause 22, further comprising:

[0150] program code for detecting at least one of a temperature or a current consumption of each of a plurality of computing units of the SOC; and

[0151] Program code for mapping the plurality of cores to the plurality of compute units based on temperature or current consumption.

[0152] 26. The non-transitory computer-readable medium of clause 25, further comprising: program code for assigning a computing unit of the plurality of computing units having a temperature or current exceeding a threshold.

[0153] 27. The non-transitory computer-readable medium of clause 22, further comprising:

[0154] program code for calculating statistics for each of the plurality of cores; and

[0155] Program code is provided for mapping the plurality of cores to the plurality of compute units based on the statistical information.

[0156] 28. The non-transitory computer-readable medium of any of clauses 22-27, further comprising: program code for dynamically assigning the plurality of cores to a plurality of computing units of the SOC at runtime.

[0157] As used herein, the term "determine" encompasses a wide variety of actions. For example, "determine" may include calculating, computing, processing, deriving, investigating, searching (e.g., searching in a table, database, or another data structure), ascertaining, and the like. Additionally, "determine" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Furthermore, "determine" may include resolving, selecting, choosing, establishing, and the like.

[0158] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items, including single members. As an example, "at least one of a, b, or c" is intended to encompass: a, b, c, ab, ac, bc, and abc.

[0159] The various illustrative logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array signal (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any commercially available processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0160] The steps of the method or algorithm described in conjunction with the present disclosure can be implemented directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module can reside in any form of storage medium known in the art. Some examples of usable storage media include random access memory (RAM), read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, CD-ROMs, and the like. The software module can include a single instruction or many instructions and can be distributed over several different code segments, distributed between different programs, and distributed across multiple storage media. A storage medium can be coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, a storage medium can be integrated into a processor.

[0161] The methods disclosed herein include one or more steps or actions for achieving the described method. These method steps and / or actions may be interchangeable with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of the specific steps and / or actions may be modified without departing from the scope of the claims.

[0162] The functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in hardware, an example hardware configuration may include a processing system in a device. The processing system may be implemented using a bus architecture. Depending on the specific application and overall design constraints of the processing system, the bus may include any number of interconnecting buses and bridges. The bus may link together various circuits including a processor, a machine-readable medium, and a bus interface. The bus interface may be used to connect a network adapter, etc., to the processing system via the bus. The network adapter may be used to implement signal processing functions. For certain aspects, a user interface (e.g., a keypad, a display, a mouse, a joystick, etc.) may also be connected to the bus. The bus may also link various other circuits, such as timing sources, peripherals, voltage regulators, power management circuits, and the like, which are well known in the art and will not be described further.

[0163] The processor may be responsible for managing the bus and general processing, including executing software stored on a machine-readable medium. The processor may be implemented with one or more general and / or special processors. Examples include microprocessors, microcontrollers, DSP processors, and other circuit systems capable of executing software. Software should be broadly interpreted to mean instructions, data, or any combination thereof, whether referred to as software, firmware, middleware, microcode, hardware description language, or other. As an example, a machine-readable medium may include random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, magnetic disks, optical disks, hard drives, or any other suitable storage medium, or any combination thereof. The machine-readable medium may be implemented in a computer program product. The computer program product may include packaging material.

[0164] In a hardware implementation, the machine-readable medium can be a part of the processing system that is separate from the processor. However, as will be readily appreciated by those skilled in the art, the machine-readable medium or any part thereof can be external to the processing system. As an example, the machine-readable medium can include a transmission line, a carrier wave modulated by data, and / or a computer product separate from the device, all of which can be accessed by the processor via a bus interface. Alternatively or additionally, the machine-readable medium or any part thereof can be integrated into the processor, as may be the case with a cache and / or general register file. Although the various components discussed can be described as having specific locations, such as local components, they can also be configured in various ways, such as with certain components being configured as part of a distributed computing system.

[0165] The processing system can be configured as a general-purpose processing system having one or more microprocessors providing processor functionality and external memory providing at least a portion of the machine-readable medium, all linked together with other supporting circuitry via an external bus architecture. Alternatively, the processing system can include one or more neuromorphic processors for implementing the neuron model and neural system model described herein. As another alternative, the processing system can be implemented using an application-specific integrated circuit (ASIC) with a processor, bus interface, user interface, supporting circuitry, and at least a portion of the machine-readable medium integrated into a single chip, or implemented using one or more field programmable gate arrays (FPGAs), programmable logic devices (PLDs), controllers, state machines, gating logic, discrete hardware components, or any other suitable circuitry, or any combination of circuits capable of performing the various functionalities described throughout this disclosure. Depending on the specific application and the overall design constraints imposed on the overall system, one skilled in the art will recognize how to best implement the functionality described with respect to the processing system.

[0166] The machine-readable medium may include several software modules. These software modules include instructions that, when executed by a processor, cause a processing system to perform various functions. These software modules may include a transmitting module and a receiving module. Each software module may reside in a single storage device or be distributed across multiple storage devices. As an example, when a triggering event occurs, a software module may be loaded from a hard drive into RAM. During execution of the software module, the processor may load some instructions into a cache to increase access speed. One or more cache lines may then be loaded into a general register file for execution by the processor. When describing the functionality of a software module below, it will be understood that such functionality is implemented by the processor when the processor executes instructions from the software module. In addition, it should be appreciated that various aspects of the present disclosure result in improvements to the functionality of a processor, computer, machine, or other system that implements such aspects.

[0167] If implemented in software, each function may be stored as one or more instructions or codes on or transmitted by a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any media that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a computer. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (such as infrared (IR), radio, and microwave), then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology (such as infrared, radio, and microwave) is included in the definition of medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Thus, in some aspects, computer-readable media may include non-transitory computer-readable media (e.g., tangible media). Additionally, for other aspects, computer-readable media may include transient computer-readable media (e.g., signals). Combinations of the above should also be included within the scope of computer-readable media.

[0168] Thus, some aspects may include a computer program product for performing the operations presented herein. For example, such a computer program product may include a computer-readable medium having stored (and / or encoded) thereon instructions, which are executable by one or more processors to perform the operations described herein. For some aspects, the computer program product may include packaging materials.

[0169] In addition, it should be appreciated that the modules and / or other appropriate means for performing the methods and techniques described herein can be downloaded and / or otherwise obtained by the user terminal and / or base station where applicable. For example, such a device can be coupled to a server to facilitate the transfer of the means for performing the methods described herein. Alternatively, the various methods described herein can be provided via a storage device (e.g., RAM, ROM, a physical storage medium such as a compact disc (CD) or floppy disk, etc.) so that once the storage device is coupled to or provided to the user terminal and / or base station, the device can obtain the various methods. In addition, any other suitable technology suitable for providing the methods and techniques described herein to a device can be utilized.

[0170] It will be understood that the claims are not limited to the precise configuration and components illustrated above. Various changes, substitutions and variations may be made in the arrangement, operation and details of the methods and apparatus described above without departing from the scope of the claims.

Claims

1. A method for an artificial neural network, comprising: receiving a set of input values ​​to be convolved with a plurality of kernels for execution via a plurality of computing units of a system on a chip (SOC); detecting a temperature associated with each of the plurality of calculation units of the SOC; mapping the plurality of cores to the plurality of compute units of the SOC based on the detected temperature associated with each of the plurality of compute units and the sparsity of each of the plurality of cores; performing a convolution operation of the set of input values ​​with the plurality of kernels using the plurality of computational cells, wherein a sparsest kernel among the plurality of kernels is convolved with the set of input values ​​on a computational cell associated with a highest temperature among the plurality of computational cells; as well as Inferences are generated based on the convolution operation.

2. The method of claim 1, further comprising: determining a first order of the computational units based on the detected temperature associated with each of the plurality of computational units and determining a second order of the cores based on the sparsity; as well as Each of the plurality of cores is assigned to one of the plurality of computing units based on the first order and the second order.

3. The method of claim 2, wherein a first order comprises the plurality of cores arranged in a layer according to decreasing sparsity, and a second order comprises the plurality of computational units arranged according to increasing temperature; and Each core is assigned to one of the plurality of computing units in order such that the least sparse core is assigned to the computing unit associated with the lowest detected temperature of the plurality of computing units and the most sparse core is assigned to the computing unit associated with the highest temperature of the plurality of computing units.

4. The method of claim 1, further comprising: A current consumption of each of the plurality of calculation units of the SOC is detected, and wherein the mapping is based on the current consumption. 5 . The method of claim 4 , wherein the mapping is performed for a computing unit among the plurality of computing units having a temperature or a current exceeding a threshold value.

6. The method of claim 1, further comprising: Statistics are calculated for each of the plurality of cores, and wherein the mapping is further based on the statistics. 7 . The method of claim 1 , wherein the mapping comprises dynamically assigning the plurality of cores to the plurality of computing units of the SOC at runtime.

8. An apparatus for an artificial neural network, comprising: at least one memory; as well as at least one processor coupled to the at least one memory, the at least one processor configured to: receiving a set of input values ​​to be convolved with a plurality of kernels for execution via a plurality of computing units of a system on a chip (SOC); detecting a temperature associated with each of the plurality of calculation units of the SOC; mapping the plurality of cores to the plurality of compute units of the SOC based on the detected temperature associated with each of the plurality of compute units and the sparsity of each of the plurality of cores; performing a convolution operation of the set of input values ​​with the plurality of kernels using the plurality of computational cells, wherein a sparsest kernel among the plurality of kernels is convolved with the set of input values ​​on a computational cell associated with a highest temperature among the plurality of computational cells; as well as Inferences are generated based on the convolution operation.

9. The apparatus of claim 8, wherein the at least one processor is further configured to: determining a first order of the computational units based on the detected temperature associated with each of the plurality of computational units and determining a second order of the cores based on the sparsity; and Each of the plurality of cores is assigned to one of the plurality of computing units based on the first order and the second order.

10. The apparatus of claim 9, wherein the first order comprises the plurality of cores arranged in a layer according to decreasing sparsity, and the second order comprises the plurality of computational units arranged according to increasing thermal tolerance; and wherein the at least one processor is further configured to assign each core to one of the plurality of computing units in order such that the least sparse core is assigned to the computing unit associated with the lowest detected temperature of the plurality of computing units and the most sparse core is assigned to the computing unit associated with the highest temperature of the plurality of computing units.

11. The apparatus of claim 8, wherein the at least one processor is further configured to: detecting a current consumption of each of the plurality of computational units of the SOC, and wherein the mapping is based on the current consumption, and The plurality of cores are mapped to the plurality of computing units based on the temperature or the current consumption. 12 . The apparatus of claim 11 , wherein the at least one processor is further configured to assign a computing unit of the plurality of computing units having a temperature or current exceeding a threshold.

13. The apparatus of claim 8, wherein the at least one processor is further configured to: calculating statistics for each of the plurality of cores; and The plurality of cores are mapped to the plurality of compute units based on the statistical information.

14. The apparatus of claim 8, wherein the at least one processor is further configured to dynamically assign the plurality of cores to the plurality of computing units of the SOC at runtime.

15. An apparatus for an artificial neural network, comprising: means for receiving a set of input values ​​to be convolved with a plurality of kernels for execution via a plurality of computing units of a system on a chip (SOC); means for detecting a temperature associated with each of said plurality of calculation units of said SOC; means for mapping the plurality of cores to the plurality of compute units of the SOC based on the detected temperature associated with each of the plurality of compute units and the sparsity of each of the plurality of cores; means for performing a convolution operation of the set of input values ​​with the plurality of kernels using the plurality of computational cells, wherein a sparsest kernel among the plurality of kernels is convolved with the set of input values ​​on a computational cell associated with a highest temperature among the plurality of computational cells; as well as Means for generating an inference based on the convolution operation.

16. The apparatus of claim 15, further comprising: means for determining a first order of computational cells based on the detected temperature associated with each of the plurality of computational cells and determining a second order of kernels based on the sparsity; as well as Means for assigning each of the plurality of cores to one of the plurality of computing units based on the first order and the second order.

17. The apparatus of claim 16 , wherein the first order comprises the plurality of cores arranged in a layer according to decreasing sparsity, and the second order comprises the plurality of computational units arranged according to increasing temperature; and the apparatus further comprises means for assigning each core to one of the plurality of computational units in order such that the least sparse core is assigned to the computational unit associated with the lowest detected temperature of the plurality of computational units and the most sparse core is assigned to the computational unit associated with the highest temperature of the plurality of computational units.

18. The apparatus of claim 15, further comprising: means for detecting a current draw of each of the plurality of calculation units of the SOC, and wherein the mapping is based on the current draw; as well as Means for assigning the plurality of cores to the plurality of compute units based on the current consumption.

19. The apparatus of claim 18, further comprising: Means for assigning a core of the plurality of cores to a computing unit of the plurality of computing units having a temperature or current exceeding a threshold.

20. The apparatus of claim 15, further comprising: means for calculating statistics for each of the plurality of cores; and means for mapping cores of the plurality of cores to compute units of the plurality of compute units based on the statistical information.

21. The apparatus of claim 15, further comprising: Means for dynamically assigning the plurality of cores to the plurality of computing units of the SOC at runtime.

22. A non-transitory computer-readable medium having recorded thereon a program code for an artificial neural network, the program code being executed by a processor and comprising: program code for receiving a set of input values ​​to be convolved with a plurality of kernels for execution via a plurality of computing units of a system on a chip (SOC); program code for detecting a temperature associated with each of the plurality of calculation units of the SOC; program code for mapping the plurality of cores to the plurality of compute units of the SOC based on a detected temperature associated with each of the plurality of compute units and a sparsity of each of the plurality of cores; program code for performing a convolution operation of the set of input values ​​with the plurality of kernels using the plurality of computational cells, wherein a sparsest kernel among the plurality of kernels is convolved with the set of input values ​​on a computational cell associated with a highest temperature among the plurality of computational cells; as well as Program code is provided for generating inferences based on the convolution operation.

23. The non-transitory computer-readable medium of claim 22, further comprising: program code for determining a first order of each of the plurality of computational cells based on the detected temperature associated with the computational cells and determining a second order of cores based on the sparsity; as well as Program code for assigning each of the plurality of cores to one of the plurality of compute units based on the first order and the second order.

24. The non-transitory computer-readable medium of claim 23, wherein the first order comprises the plurality of cores arranged in a layer according to decreasing sparsity, and the second order comprises the plurality of computational units arranged according to increasing temperature; and Further including: Program code for assigning each core to one of the plurality of computing units in order such that the least sparse core is assigned to the computing unit associated with the lowest detected temperature of the plurality of computing units and the most sparse core is assigned to the computing unit associated with the highest temperature of the plurality of computing units.

25. The non-transitory computer-readable medium of claim 22, further comprising: program code for detecting a current draw of each of the plurality of computational units of the SOC, and wherein the mapping is based on the current draw, and Program code is provided for mapping the plurality of cores to the plurality of compute units based on the current consumption.

26. The non-transitory computer-readable medium of claim 25, further comprising: Program code is provided for designating a computing unit of the plurality of computing units having a temperature or current exceeding a threshold.

27. The non-transitory computer-readable medium of claim 22, further comprising: program code for calculating statistics for each of the plurality of cores; as well as Program code for mapping the plurality of cores to the plurality of compute units based on the statistical information.

28. The non-transitory computer-readable medium of claim 22, further comprising: Program code for dynamically assigning the plurality of cores to the plurality of computing units of the SOC at runtime.

Images