Modifying processing data streams to reduce power impact during parallel processing

Abstract

It is an object of the present invention to provide a modification of a processing data stream to reduce the power impact during parallel processing.SOLUTION: Certain aspects of the present disclosure provide a method for performing parallel data processing, the method including receiving data for parallel processing from a data processing requester, generating a plurality of data sub-blocks, determining a plurality of data portions in each data sub-block of the plurality of data sub-blocks, permuting the plurality of data portions in at least one data sub-block of the plurality of data sub-blocks, providing the plurality of data sub-blocks to a plurality of processing units for parallel processing, the plurality of data sub-blocks including at least one data sub-block including a permuted order of the plurality of data portions, and receiving processed data associated with the plurality of data sub-blocks from the plurality of processing units.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] Claiming priority under 35 U.S.C. § 119 This application claims priority to U.S. Patent Application No. 16 / 834,986, filed March 30, 2020, and entitled "PROCESSING DATA STREAM MODIFICATION TO REDUCE POWER EFFECTS DURING PARALLEL PROCESSING," which is incorporated herein in its entirety.

[0002] Aspects of the present disclosure relate to techniques for modifying processing data streams to reduce unintended, disruptive power effects during parallel processing. [Background technology]

[0003] Parallel processing is increasingly being implemented to improve the performance of large-scale data processing tasks. For example, the performance of machine learning and artificial intelligence systems can often be improved by implementing parallel processing in or across processing systems that include multiple processing units.

[0004] While parallel processing can yield significant performance benefits in many large-scale data processing contexts, it can also create significant distortions in the processing system. Such distortions can lead to disruptive power effects, such as current spikes and voltage droop, that adversely affect processing system performance. For example, if multiple processing units in a processing system, such as a system-on-chip (SoC), simultaneously initiate intensive data processing tasks, the entire processing system may enter a degraded or unstable state during the ensuing current spike and / or voltage droop. In such cases, the benefits of the parallel processing architecture are significantly reduced, and performance to the end user is compromised. These issues can be particularly severe in mobile devices, where power issues are amplified. Summary of the Invention [Problem to be solved by the invention]

[0005] Therefore, what is needed is a system and method for modifying processing data streams to reduce disruptive power effects during parallel processing. [Means for solving the problem]

[0006] Some aspects provide a method for parallel data processing, the method including steps of receiving data for parallel processing from a requester, generating a plurality of data sub-blocks, determining a plurality of data portions in each data sub-block of the plurality of data sub-blocks, permuting the plurality of data portions in at least one data sub-block of the plurality of data sub-blocks, providing the plurality of data sub-blocks, including at least one data sub-block including the permuted order of the plurality of data portions, to a plurality of processing units for parallel processing, and receiving processed data associated with the plurality of data sub-blocks from the plurality of processing units.

[0007] A further aspect provides a method for performing parallel data processing, the method including steps of receiving data for parallel processing from a requester; determining a plurality of density ranges, each density range of the plurality of density ranges being different from each other density range of the plurality of density ranges; determining a plurality of data portions of the data based on the density ranges; generating a plurality of data sub-blocks having reordered data portions; providing the plurality of data sub-blocks to a plurality of processing units for parallel processing; and receiving processed data associated with the plurality of data sub-blocks from the plurality of processing units.

[0008] A further aspect provides a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of a processing system, perform the aforementioned methods as well as methods further described herein.

[0009] A further aspect provides a computer program product embodied on a computer-readable storage medium that includes code for performing the foregoing methods as well as methods further described herein.

[0010] A further aspect provides a processing system having one or more processors and / or other means configured to perform the aforementioned methods, as well as methods further described herein.

[0011] The following description and the associated drawings set forth in detail certain illustrative features of the one or more examples.

[0012] The accompanying drawings illustrate some aspects of the one or more examples and therefore should not be considered limiting of the scope of the present disclosure. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 illustrates an exemplary parallel data processing architecture. [Figure 2] FIG. 10 illustrates an example of generating multiple data sub-blocks based on a received data block by a processing coordinator. [Figure 3] FIG. 1 illustrates another exemplary parallel data processing architecture. [Figure 4] FIG. 1 illustrates an example of dynamic rearrangement of data for parallel processing to reduce power impact. [Figure 5] FIG. 1 illustrates an exemplary method for dynamically rearranging data for parallel processing. [Figure 6] FIG. 10 illustrates another example of dynamic reordering of data for parallel processing to reduce power impact. [Figure 7]FIG. 1 illustrates an exemplary method for dynamically rearranging data for parallel processing using density binning. [Figure 8] FIG. 1 illustrates an exemplary processing system configured for parallel processing using density-based rearrangement. [Figure 9] FIG. 10 is a chart illustrating an exemplary relationship between density in convolutional neural network data and power consumption of a processing unit. [Figure 10] FIG. 10 is a diagram illustrating an example of a parallel processing multidimensional data block. [Figure 11A] FIG. 2 illustrates exemplary power impacts associated with parallel processing. [Figure 11B] FIG. 2 illustrates exemplary power impacts associated with parallel processing. [Figure 11C] FIG. 2 illustrates exemplary power impacts associated with parallel processing. [Figure 12] FIG. 1 illustrates an example of using density binning on machine learning data. [Figure 13] FIG. 1 illustrates a comparison of conventional processing of machine learning data blocks using kernels and density rearrangement processing of machine learning data blocks. [Figure 14A] FIG. 1 illustrates exemplary power impacts associated with parallel processing using density binning. [Figure 14B] FIG. 1 illustrates exemplary power impacts associated with parallel processing using density binning. [Figure 14C] FIG. 1 illustrates exemplary power impacts associated with parallel processing using density binning. [Figure 15] FIG. 1 illustrates useful properties of some machine learning operations when using density-reordered input data. [Figure 16] FIG. 1 illustrates an exemplary electronic device that may be configured to perform the methods described herein. [Figure 17] FIG. 1 illustrates an exemplary multi-processor processing system that may be implemented using examples described herein. DETAILED DESCRIPTION OF THE INVENTION

[0014] For ease of understanding, where possible, identical reference numerals have been used to designate identical elements common to the figures. It is contemplated that elements and features of one example may be advantageously incorporated into other examples without further description.

[0015] Aspects described herein relate to systems and methods for modifying processing data streams to reduce unintended, disruptive power effects during parallel processing. Such systems and methods improve the performance of parallel processing systems, such as parallel processing systems that include multiple processing units and / or that coordinate multiple processing systems.

[0016] In general, parallel processing capabilities may be provided in a single processor having multiple processing units (e.g., multiple processing cores of a processor), by multiple processors each having one or more processing units, by multiple processing systems each having its own processor, and combinations thereof. In general, multiple processors used for parallel processing may be homogeneous or heterogeneous in design, such as multiple central processing units (CPUs), or a single central processing unit, graphical processing unit (GPU), neural signal processing unit (NPU), digital signal processor (DSP), etc. acting in concert. NPUs as described herein are generally specialized circuits configured to implement all the necessary control and arithmetic logic for executing machine learning algorithms, such as algorithms for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc.

[0017] As described in more detail below, modifying the processing data stream to reduce disruptive power effects during parallel processing may be achieved by analyzing the data for parallel processing and reordering (or rearranging, reordering, or modifying) the data to avoid simultaneous power spikes from multiple processing units. The reordered processing data stream may thus reduce or eliminate undesirable power effects in the processing system, such as processing power spikes, voltage droop, abnormally high temperatures, additional leakage currents, etc., which advantageously improves the performance and reliability of the processing system.

[0018] Furthermore, the methods described herein may advantageously reduce peak current during parallel processing without using traditional, undesirable performance-degrading methods such as frequency throttling and command stalling, to name a few.

[0019] Furthermore, avoiding large processing power spikes may advantageously reduce the heat load on the system (from high, transient current flows). Reduced heat load may advantageously reduce cooling requirements and therefore positively impact packaging possibilities. Furthermore, reduced heat load may reduce thermally induced leakage currents and therefore positively impact power efficiency.

[0020] The aforementioned advantages can significantly improve the performance of mobile devices, particularly due to their inherent power design constraints.

[0021] Exemplary Parallel Data Processing Architecture 1 shows an exemplary parallel data processing architecture 100. In the illustrated example, an application 102 provides data for processing to a data processing system 110 that includes a data processing coordinator 104 for coordinating parallel processing. Application 102 may be a local application (e.g., on the processing system) or a remote application (e.g., in a client-server architecture) and may be generally referred to as a data processing requestor.

[0022] The processing coordinator 104 may generally be a function implemented within a processing system configured to coordinate parallel processing of data, such as within a multi-processor computing system (e.g., a computer, a server, a mobile device, a system-on-chip (SoC), an Internet of Things (IoT) or an edge device), or generally within any multi-processing unit system.

[0023] In this example, data processing coordinator 104 receives data from application 102 and coordinates parallel processing of the data by providing data sub-blocks A-D to processing cores 106A-D. Data sub-blocks A-D may be any type of data for parallel processing. In some examples, data sub-blocks A-D may include machine learning or artificial intelligence model input data, such as image data, video data, audio data, or other quantitative data.

[0024] A data sub-block may generally relate to a block, tile, tensor, etc. of data sent to multiple processing units (e.g., processing cores 106A-D) for parallel processing. In some examples, each data sub-block may represent a unique subset of a set of data for processing, or each data sub-block may represent a repeated set of data that is processed differently on each of the processing units, such as processing the same data with different models. Thus, generating the data sub-blocks may include generating mutually exclusive data subsets, or repeated data sets, for distribution to the multiple processing units.

[0025] FIG. 2 illustrates an example of generating multiple data sub-blocks by a data processing coordinator, such as processing coordinator 104 of FIG.

[0026] In this example, a multidimensional data block 202 is provided to a processing coordinator 104 for processing by multiple processing devices, such as multiple processing cores 106A-D (e.g., for parallel processing). In some examples, the data block 202 may be a tensor. In general, a tensor of rank n in an m-dimensional space has n indices and m n A tensor is a mathematical object that has components and obeys certain transformation rules. Each index of a tensor spans the number of dimensions of the space. Thus, a tensor may be a generalization of scalars (no index), vectors (having exactly one index), and matrices (having exactly two indexes) to any number of indices.

[0027] Data block 202 includes different portions 202A-D having different densities (or conversely, sparseness), generally indicated by their different shadings. In this example, portion 202B is the darkest shaded portion and is therefore the densest data portion of data 202, while portions 202A and 202C are one shade lighter than portion 202B and are therefore less dense, but more dense than portion 202D. Data portion 202D is the lightest shade of data portion in data block 202 and is the least dense (or sparsest).

[0028] In one example, the density of a multidimensional data instance may be measured by the proportion of non-null entries in various dimensions of the data, such as an array, vector, matrix, or other multidimensional data format. Similarly, the sparseness of a multidimensional data instance may be measured by the proportion of null entries in various dimensions of the data. As a simple example, a data vector with entries [1, 0, 1] would have a density of 2 / 3 and a sparseness of 1 / 3. In general, denser data requires more processing power and sparser data requires less processing power.

[0029] In particular, the different densities and layouts of the data portions (202A-D) in Figure 2 are intended to be merely a simple example for ease of explanation, as data may have much more complex density characteristics.

[0030] The processing order 203 of the data 202 indicates that during processing of the data 202, the processing cores 106A-D encounter data of different densities at different times.

[0031] In this example, processing coordinator 104 divides data 202 into four sub-blocks 204A-D, each containing a subset of portions 202A-D, as indicated by corresponding shaded bands. As shown in chart 206, processing of data sub-blocks 204A-D by processing cores 106A-D according to processing direction 203 results in a significant spike in total processing system power at 208, coinciding with time segment 2 along the time axis. This is caused by each processing core (106A-D) arriving at the denser data portion 202B of data sub-blocks 204A-D at the same (or nearly the same) time.

[0032] The power spike at 208 is an example of an undesirable power effect that can cause various side effects to the processing system, such as voltage droop on the processing system power bus, resulting in associated degradation such as processor frequency throttling, command stalls, system instability, etc. Furthermore, to mitigate these potential problems, a higher operating voltage may be required, which increases power dissipation.

[0033] As described in more detail below, the problem illustrated in chart 206 can be overcome by determining the data density of the data sub-blocks (e.g., data sub-blocks 204A-D) and rearranging the data to avoid simultaneous occurrence of peak processing demands at each of the processing cores (e.g., 106A-D in this example).

[0034] Parallel Data Processing Architecture Using Data Rearrangement 3 shows another exemplary parallel data processing architecture 300. In the illustrated example, an application 302 provides data for processing to a data processing system 310 that includes a data processing coordinator 304, a data density analyzer 305, and a data sequencer 307. Again, application 302 may be referred to as a data processing requestor and may be local or remote to processing system 310.

[0035] As noted above, data processing coordinator 304 may generally be configured to coordinate parallel processing of data in a multi-processing unit data processing system, such as processing system 310. In this example, data processing coordinator 304 receives data from application 302 and generates data sub-blocks. As noted above, the received data may be tensors.

[0036] In some examples, processing coordinator 304 may be implemented as part of a compiler configured to determine how to divide and allocate data sub-blocks to processing cores during compile time. In other examples, processing coordinator 304 may be implemented as part of a task scheduler or kernel of an operating system configured to determine how to divide and allocate data sub-blocks to processing cores at run time. Notably, these are just two example implementations, and others are contemplated.

[0037] 1 and 2, here the data sub-blocks are then analyzed by a density analyzer 305 before processing. The density analyzer 305 is configured to determine the density (or alternatively, the sparseness) of the incoming data sub-blocks. In some examples, the density determination results in identifying multiple different data portions associated with different densities (or density ranges). In some examples, the data portions are arranged along the processing direction.

[0038] The density analyzer 305 provides the data sub-blocks and the determined density to the sequencer 307, which then rearranges the individual data sub-blocks based on the determined density to reduce or eliminate the simultaneous occurrence of peak density data portions being processed by multiple processing cores at once.

[0039] In one example, the sequencer 307 may generate rearranged data sub-blocks by changing the order (e.g., by shuffling or shifting) of the data portions in each sub-block (e.g., in processing order or time) to reduce the coincidence of peak densities in the processing order of the data portions across multiple processing units, such as processing cores 306A-D.

[0040] In another example, the sequencer 307 may generate rearranged data sub-blocks by retrieving data from different density bins in a particular ordering. For example, the sequencer 307 may rearrange the data sub-blocks by retrieving data from density bins in a circular order of data density, as described in more detail with respect to FIG. 6. This also reduces the coincidence of peak densities in the processing order across multiple processing units, such as processing cores (306A-D).

[0041] The sequencer 307 may be configured to not only reduce the co-occurrence of peak density data portions, but also to reduce the average difference in data density (or sparseness) between successive data portions (e.g., along the processing order). In one example, this reduces the density x i x i - 1 <x i < x i + 1 or x i - 1 > x i >x i + 1 This may be done by rearranging the various data portions in order of increasing or decreasing density, such that: This density-based (or sparseness) ordering advantageously reduces the size of the processing power change (e.g., in current draw) between data portions of varying density.

[0042] In the illustrated embodiment, sequencer 307 provides the reordered data sub-blocks to processing cores 306A-D, respectively, as further shown and described with respect to Figure 4. In other examples, as further defined with respect to Figure 6, each processing core (e.g., 306A-D) may be assigned a default density-based order and configured to read data portions based on the default density-based order.

[0043] In some examples, after processing, sequencer 307 may receive the processed data and reorder it to match the original sequence order. In this manner, density determination and ordering may be performed without affecting existing parallel data processing coordination functions. However, in other examples, if the sequence of the data processing results is not important to the overall data processing results, processing cores 306A-D may provide the processed data directly to processing coordinator 304.

[0044] Examples of the density analyzer (305) and / or the sequencer (307) may be implemented in hardware and / or software logic. For example, in some examples, the density analyzer (305) and the sequencer (307) are implemented as part of a DMA that includes hardware logic and low-level firmware, as described in more detail below with respect to FIG.

[0045] Although shown as separate embodiments in this example, in alternative examples, the functionality of density analyzer 305 and sequencer 307 may be integrated into a single element, such as processing coordinator 304, or into another embodiment of processing system 310. These various functions are shown separately in Figure 3 for clarity.

[0046] Example of dynamic data rearrangement for parallel processing FIG. 4 shows an example of dynamic reordering of data for parallel processing to reduce power impact.

[0047] In the illustrated example, a multidimensional data block 302 is provided by an application such as that described above in FIG. 3 for parallel data processing. Here again, data block 302 has portions (e.g., 302A-D) of varying data density (or sparseness). In this example, portion 302B is the darkest shaded portion and therefore the densest data portion, while portions 302A and 302C are one shade lighter than portion 302B and therefore less dense, but more dense than portion 302D. Data portion 302D is the lightest shade of data portion in data block 302 and is the least dense (or sparsest).

[0048] Notably, in this example, the density of each portion 302A-D is uniform, but in other examples, the density across one or more dimensions of the multidimensional data block 302 may not be uniform. For example, the data block 302 may represent image data, with each of the data portions 302A-D representing a different channel (or layer) of the image, with its own measurable density. As noted above, density may be measured by the number of null or zero entries in a portion of the data, such as a slice of image data having dimensions of width (w) x height (h) x depth (d).

[0049] The data processing coordinator 304 receives the data block 302 and generates multiple data sub-blocks for parallel processing, which are then provided to the density analyzer 305. The density analyzer 305 analyzes the data sub-block, for example, to identify portions of the data sub-block that are of varying density. In one example, the portions may correspond to multiple predetermined density ranges.

[0050] The density analyzer 305 then provides the data sub-blocks and density data to the sequencer 307, which dynamically rearranges the data sub-blocks based on the determined density data.

[0051] In the example shown in FIG. 4, the original relative order of the data portions (302A-D) is maintained in the rearranged data sub-blocks 404A-D when considered in a circular manner, but the processing time for each data portion is shifted for each data sub-block due to the rearrangement. Specifically, in this example, the processing order of the data portions in data sub-block 404A is the same as in the original data block 302, but the processing order of the data portions in each subsequent data sub-block is incremented by one position, or in other words, one position for data sub-block 404B, two positions for data sub-block 404C, and three positions for data sub-block 404D. As shown, when a data portion reaches the end of a sub-block, the next increment returns it to the starting position in a circular manner. This rearrangement of the data portions along the processing order of the data sub-blocks may generally be referred to as process time rearrangement. The process time rearrangement may have a defined stride that defines the number of positions to shift a particular portion. In the illustrated example, the stride is 1.

[0052] An advantageous result of the process time reordering of FIG. 4 is that the rearranged data sub-blocks 404A-D have varying densities of data portions in different time segments of the processing order 406. The distribution of peak density data portions (and therefore peak processing power) across different time segments for different data sub-blocks advantageously reduces the peak total processing system power compared to chart 206 of FIG. 2. Specifically, the process time reordering of this example results in the densest data portions being processed in the second time segment for data sub-block 404A, the third time segment for data sub-block 404B, the fourth time segment for data sub-block 404C, and the first time segment for data sub-block 404D (returning it to the first segment in the processing order). Furthermore, the total system processing power across time segments 1-4 remains unchanged in chart 408 compared to having a pronounced peak in chart 206 of FIG. 2.

[0053] More generally, process time reordering generates multiple processing orders for different data portions of data sub-blocks 404A-D. In this example, each data sub-block 404A-D has a different processing order for its data portion, but in other examples, there may be more sub-blocks than data portions per sub-block, in which case some processing orders may be repeated (as shown in FIG. 6).

[0054] In particular, while the number of data portions is the same as the number of data sub-blocks in this example, this need not be the case and is used in this example for ease of explanation. Furthermore, while the number of data sub-blocks (404A-D) in this example matches the number of processing cores (306A-D), this also need not be the case. In some processing systems, the number of available cores may be dynamic, and this process may occur dynamically according to available processing resources.

[0055] Although not shown in FIG. 4 , in other examples, the data portions of the sub-blocks (404A-D) may be further rearranged according to decreasing or increasing density order. In other words, the first data sub-block may be rearranged so that the first data portion (at the beginning of the processing order) has the highest density and each subsequent data portion has a lower density than the previous data portion, i.e., in decreasing density order. The subsequent data sub-blocks may then be time-rearranged as described above. Alternatively, the first data sub-block may be rearranged so that the first data portion has the lowest density and each subsequent data portion has a higher density than the previous data portion, i.e., in increasing density order. This may be generally referred to as density-order rearrangement and may advantageously reduce the variation in density, and therefore the variation in processing power, between adjacent data portions. Once the first data sub-block has its data portions ordered by density, process time ordering may then be applied as described above.

[0056] Exemplary Method for Dynamically Rearranging Data for Parallel Processing 5 illustrates an exemplary method 500 for dynamically rearranging data for parallel processing. Method 500 may be performed by a processing system, such as, for example, processing system 310 of FIG. 3 and processing system 804 of FIG. 8. As described further herein, in some embodiments, aspects of method 500 may be performed by a DMA controller, as described with respect to FIG. 8.

[0057] Method 500 begins at step 502 with receiving data for parallel processing from a data processing requestor, such as those described with respect to Figures 3, 4, and 6. In various embodiments, the data processing requestor may be a local application, a remote application, a local processing subsystem (e.g., another processor in the same processing system), a local or remote process, etc.

[0058] Method 500 then proceeds to step 504 and generates a plurality of data sub-blocks, as described with respect to Figures 3, 4, and 6. As noted above, in some aspects, each data sub-block represents a subset of data that is processed in a similar manner, while in other aspects, each sub-block is an overlapping set of data that is processed differently.

[0059] The method 500 then proceeds to step 506, where a plurality of data portions are determined in each data sub-block of the plurality of data sub-blocks. As noted above, the data portions may be determined with respect to a plurality of densities or density ranges, for example, by binning according to density.

[0060] In some aspects, determining the plurality of data portions in each data sub-block of the plurality of data sub-blocks includes determining a plurality of density ranges, each density range of the plurality of density ranges being different from each other density range of the plurality of density ranges, and determining the plurality of data portions based on the plurality of density ranges.

[0061] The method 500 then proceeds to step 508, where the reordering of the data portions in at least one of the plurality of data sub-blocks is performed. In this example, the reordering of the data portions in the data sub-block forms a reordered data sub-block.

[0062] In some aspects, changing the order of the plurality of data portions for at least one data sub-block of the plurality of data sub-blocks includes time-shifting each data portion of the plurality of data portions along a processing time direction in a circular order, as described with respect to Figures 4 and 6.

[0063] In some aspects, changing the order of the plurality of data portions for at least one data sub-block of the plurality of data sub-blocks includes rearranging the plurality of data portions according to one of an increasing density order or a decreasing density order, as described with respect to FIGS. 4 and 6.

[0064] The method 500 then proceeds to step 510, where a plurality of data sub-blocks, including at least one data sub-block that includes a permuted order of the plurality of data portions, are provided to a plurality of processing units for parallel processing.

[0065] Method 500 then proceeds to step 512, where processed data associated with the plurality of data sub-blocks is received from the plurality of processing units.

[0066] The method 500 then proceeds to step 514, where the processed data is provided to the data processing requestor.

[0067] In some aspects, the method 500 further includes, prior to providing the processed data to the data processing requester, reordering the processed data associated with the plurality of data sub-blocks based on an original order of the plurality of data portions in each data sub-block of the plurality of data sub-blocks.

[0068] In some aspects, the method 500 further includes permuting the plurality of data portions in each data sub-block of the plurality of data sub-blocks.

[0069] In some aspects, determining the multiple data portions in each data sub-block of the multiple data sub-blocks is performed by a direct memory access (DMA) controller in data communication with the multiple processing units, as described with respect to FIG.

[0070] In some aspects, permuting the plurality of data portions in at least one of the plurality of data sub-blocks is performed by a DMA controller, as described with respect to FIG.

[0071] In some aspects, the data for parallel processing includes machine learning data associated with a machine learning model.

[0072] In some embodiments, the plurality of processing units includes at least one neural signal processing unit or AI acceleration unit, as described with respect to FIG. 16.

[0073] In some aspects, method 500 may be used to perform neural network convolution processing across many neural signal processing units (NPUs) (or AI acceleration cores). Similarly, method 500 may be used to perform neural network convolution processing across many arithmetic logic units (ALUs) within a GPU, or across many GPUs, or across many CPUs. For example, method 500 may be used to perform general image processing across many ALUs within a GPU, or across many GPUs.

[0074] An example of dynamic data rearrangement for parallel processing based on density binning FIG. 6 shows another example of dynamic reordering of data for parallel processing to reduce power impact.

[0075] 6 illustrates a processing coordinator 604, which includes a density analyzer 606 and a sequencer 610. In this example, the processing coordinator 604 receives a multidimensional data block 602, which may be a tensor in some aspects. The processing coordinator 604 then uses the density analyzer 606 to examine the data block 602.

[0076] The density analyzer 606 identifies portions of the data block 602 that have densities within predefined ranges associated with density bins 608A-C. The density analyzer 606 then places the identified portions of the data block 602, including 602A-D, into bins 608A-C based on the determined densities. Generally, multiple density bins will cover the full range of possible densities, and every data portion can be assigned to a unique bin.

[0077] As shown in this example, data portion 602B is the highest density data portion and is stored in bin 608A based on its determined density falling within the density range associated with bin 608A (e.g., density > 66%), data portion 602A is a medium density data portion and is stored in bin 608B based on its determined density falling within the density range associated with bin 608B (e.g., 33% < density < 66%), and data portions 602C and 602D are low density data portions that are stored in bin 608C based on their determined density falling within the density range associated with bin 608C (e.g., density < 33%).

[0078] In this example, sequencer 610 then generates rearranged data sub-blocks 612A-D by drawing data portions from density bins (608A-C) in a defined order that is modified (e.g., incremented) for each data sub-block.

[0079] In the illustrated example, a density bin sequence map 611 is used by the sequencer 610 to map each processing core to a bin sequence (or order) in a circular manner. For example, core 1 (616A) is mapped to the sequence 1-2-3 (i.e., bins 608A-608B-608C), core 2 (616B) is mapped to the sequence 3-1-2 (i.e., bins 608C-608A-608B), etc. In this example, the bin sequence is each incremented by one bin for each successive processing core in the circular order. Further, in this example, the sequence repeats with the fourth processing core because there are only three bins. In other aspects, there may be more or fewer bins and more or fewer processing cores.

[0080] In other examples, processing cores 616A-D may be configured to read data portions from density bins 608A-C in a predetermined bin sequence order (as shown at 611) without the need for action by sequencer 610. In such examples, the predetermined density bins that dictate the allocation may be configured at install time, for example, in a processing core firmware image.

[0081] After processing by processing cores 616A-D, the processed data may be reordered and recombined to form the processed data output. In some examples, a bin sequence map may be used to reorder the processed data to match the original sequence of the data input. In this way, data processing requesters do not need to be modified to receive the processed data.

[0082] Exemplary Method for Dynamically Rearranging Data for Parallel Processing Using Density Binning FIG. 7 illustrates an example method 700 for dynamically rearranging data for parallel processing using density binning.

[0083] Method 700 begins at step 702 by receiving data for parallel processing from a data processing requester.

[0084] The method 700 then proceeds to step 704 to determine a plurality of density ranges, each density range of the plurality of density ranges being different from each other density range of the plurality of density ranges.

[0085] The method 700 then proceeds to step 706 to determine a plurality of data portions of the data based on density ranges.

[0086] In some embodiments, determining the plurality of data portions of the data based on the density ranges includes determining a density for each respective data portion of the plurality of data portions and assigning each respective data portion of the plurality of data portions to one density bin of the plurality of density bins based on the determined density for the respective data portion, wherein each density bin is associated with one density range of the plurality of density ranges, as described above with respect to FIG.

[0087] In some aspects, determining the multiple data portions of the data based on density ranges is performed by a direct memory access (DMA) controller in data communication with multiple processing units, as described with respect to Figure 8. An advantage of having the density (or sparseness) calculation in the DMA is that it hides the associated computational latency, which can be done in parallel with moving and buffering data for each data portion.

[0088] The method 700 then proceeds to step 708, where it generates a plurality of data sub-blocks having the rearranged data portions.

[0089] In some aspects, generating a plurality of data sub-blocks having rearranged data portions includes adding data portions to the plurality of data sub-blocks according to a plurality of bin sequences, each bin sequence of the plurality of bin sequences including a unique sequence of a plurality of density bins, as described with respect to FIG.

[0090] In some aspects, generating the plurality of data sub-blocks having rearranged data portions is performed by a DMA controller, as described with respect to FIG.

[0091] The method 700 then proceeds to step 710, where the multiple sub-blocks of data are provided to multiple processing units for parallel processing.

[0092] In some aspects, providing multiple sub-blocks of data to multiple processing units for parallel processing is performed by a DMA controller, as described with respect to FIG.

[0093] The method 700 then proceeds to step 712, where processed data associated with the plurality of data sub-blocks is received from the plurality of processing units.

[0094] In some aspects, receiving the processed data associated with the plurality of data sub-blocks from the plurality of processing units is performed by a DMA controller, as described with respect to FIG.

[0095] Method 700 then proceeds to step 714, where the processed data is provided to a data processing requestor. In various embodiments, the data processing requestor may be a local application (as described with respect to FIGS. 3 and 6), a remote application, a local processing subsystem (e.g., another processor in the same processing system), a local or remote process, etc.

[0096] In some aspects, the method 700 further includes assigning each bin sequence of the plurality of bin sequences to one or more processing units of the plurality of processing units, and each respective data sub-block of the plurality of data sub-blocks is provided to one processing unit of the plurality of processing units based on the assigned bin sequence of the respective data sub-block.

[0097] In some aspects, the method 700 further includes determining a plurality of bin sequences according to one of increasing or decreasing density order and a processing order time shift.

[0098] In some aspects, the data for parallel processing is machine learning model input data for a convolutional neural network model.

[0099] In some aspects, each data portion of the plurality of data portions includes a feature map. In some embodiments, the feature map may represent the output of a filter applied to a previous layer of a neural network, such as a convolutional neural network. In some cases, the feature map may be referred to as an intermediate feature map, and the intermediate feature map may represent the output from an intermediate layer in a deep neural network, such as a convolutional neural network.

[0100] In some aspects, method 700 further includes rearranging a convolution kernel (or filter) for each respective data sub-block of the plurality of data sub-blocks based on a sequence of portions of data in the respective data sub-blocks, and applying the rearranged convolution kernel to the respective data sub-blocks, as described with respect to FIG. 15. In various embodiments, a convolution kernel or filter is a set of trainable values ​​or parameters that are applied to data being processed by a layer of a convolutional neural network model to generate layer output data, such as a feature map. A convolution kernel such as that shown in FIG. 15 may include multiple layers, and the multiple layers may be rearranged to match rearranged input data, such as a rearranged input feature map.

[0101] In some embodiments, the plurality of processing units includes at least one neural signal processing unit or AI acceleration unit, as described with respect to FIG. 16.

[0102] In some aspects, method 700 may be used to perform neural network convolution processing across many neural signal processing units (NPUs) (or AI acceleration cores). Similarly, method 700 may be used to perform neural network convolution processing across many arithmetic logic units (ALUs) within a GPU, or across many GPUs, or across many CPUs. For example, method 700 may be used to perform general image processing across many ALUs within a GPU, or across many GPUs.

[0103] Exemplary System for Parallel Processing Using Density-Based Rearrangement FIG. 8 illustrates an exemplary processing system 804 configured for parallel processing using density-based reordering as described herein.

[0104] In this example, processing system 804 receives data from application 802 to be processed in parallel using multiple processing units, including neural signal processing units (NPUs) 812A-F, which may alternatively be referred to as AI acceleration cores or units, as described in further detail below. Notably, in other examples, other numbers and / or types of processing units may be used in addition to or instead of NPUs 812A-F, such as the various types of processing units described below with respect to FIG.

[0105] Data from application 802 is received by a direct memory access (DMA) controller 806, which in this example includes a density analyzer 808, such as those described above, and a sequencer 810. Notably, in other examples, one or more of density analyzer 808 and sequencer 810 may be separate from DMA controller 806. Further, as in FIG. 6 above, density analyzer 808 and sequencer 810 may be part of a processing coordinator (not shown). Alternatively, a separate processing coordinator may be provided and function as described herein.

[0106] In another example, each NPU may be assigned a default density bin reading order that it can follow for each sub-block, thus eliminating the need for sequencer 810 to perform that function. The density bin ordering assignments may be defined, for example, at install time, in, for example, an NPU firmware image. However, in some embodiments, sequencer 810 may be used to override the default ordering.

[0107] In this example, DMA controller 806 performs density analysis and reordering functions (as described above) and provides reordered data sub-blocks (not shown) to one or more of NPUs 812A-F for parallel processing. The reordered data sub-blocks may be stored in memory associated with each of the NPUs, such as, for example, tightly coupled memory (TCM) 814A-F. Generally, TCM provides low-latency memory that a processor can use without the unpredictability inherent in caches. TCM may be used, for example, to hold critical routines, such as interrupt handling routines or real-time tasks where cache uncertainty is undesirable.

[0108] In one example, the processing system 804 may be configured to perform neural network convolution processing across NPUs 812A-F.

[0109] In particular, FIG. 8 is only one example of some embodiments of a processing system; other embodiments may exist and other configurations are possible.

[0110] Exemplary Parallel Processing of Neural Network Data Using Density-Based Rearrangement Parallel processing of machine learning data, such as for neural networks, places a heavy burden on processing systems. Many factors can lead to increased peak current draws and increased current ramps over short periods of time, which are detrimental to the processing system performance described above. For example, processing systems for machine learning data may generally operate at high frequencies to quickly process the various instructions and threads associated with machine learning operations. Furthermore, machine learning data may generally be dense.

[0111] To illustrate the impact of dense machine learning data on processing power, FIG. 9 shows a chart 900 illustrating an example relationship between the density of convolutional neural network data and the power consumption of a processing unit. As shown, the amount of processing power used generally increases as the density of the convolutional neural network data increases (i.e., as sparseness decreases). Furthermore, the increase in power consumption with increasing density further increases as data size and computational complexity increase (e.g., floating-point multiply-accumulate consumes more power than integer multiply-accumulate). In this particular example, 16-bit floating-point convolutional data processing consumes 40% more power compared to 8-bit integer convolutional data processing at full density, such as in saturated tensors.

[0112] 10 illustrates an example of parallel-processed multidimensional data sub-blocks 1002A-D. In this example, each data sub-block 1002A-D includes multiple feature maps, which are processed with convolution kernels 1004A-D (e.g., in a convolutional layer of a neural network). In this example, the density of the data sub-blocks 1002A-D varies along each data sub-block's various feature maps, indicated by different shadings of the feature maps along each sub-block's convolutional processing sequence 1010; thus, each feature map in this example may be considered a portion of its respective data sub-block.

[0113] Chart 1006 shows the power consumption of four processing units 1008A-D. Because data sub-blocks 1002A-D each have denser portions (e.g., feature maps) at the same location along the data sub-block's convolutional processing sequence 1010, the power consumption of each processing unit 1008A-D is highest at the same time, thus causing a significant increase in overall processing power, as shown at 1012. As explained above, this significant increase is detrimental to system performance because it can cause disruptive power effects such as voltage droop (discussed in more detail with respect to FIGS. 11A-C ), which can further affect other parts of the system, such as by causing throttling of processing operations.

[0114] FIG. 11A shows an example chart of power consumption for multiple processing cores, similar to chart 1006 of FIG.

[0115] FIG. 11B shows an example of total processing power consumption 1102 for cores 0-3 from FIG. 11A. As shown, the total processing power generally varies over time, but reaches a power spike maximum at 1105, which coincides with time increment 4 in FIG. 11A. Furthermore, a change in current 1104 (e.g., Δi) over a change in time 1106 (e.g., Δt) results in a steep ramp 1107 (e.g., Δi / Δt) in power consumption. This abrupt change in power consumption can cause an abrupt change in current, potentially resulting in disruptive power effects such as the voltage droop shown in FIG. 11C.

[0116] Figure 11C shows an example of system voltage 1008 over time, for times consistent with Figures 11A and 11B. As shown, a voltage droop 1112 occurs at time 4, which coincides with the power consumption peak 1105 in Figure 11B. In particular, the voltage droop 1112 can cause the system voltage to move out of the stable system voltage band 1110, causing system instability and performance degradation actions on parts of the system to maintain stability, such as frequency throttling.

[0117] FIG. 12 illustrates an example of using density binning on machine learning data such as the feature maps shown in FIG. 10. Specifically, bin 1 (1202A) contains address pointers to feature map data 1204A having a density between 0 and 25%. Similarly, bin 2 (1202B) contains address pointers to feature map data 1204B having a density between 26 and 50%, bin 3 (1202C) contains address pointers to feature map data 1204C having a density between 51 and 75%, and bin 4 (1202D) contains address pointers to feature map data 1204D having a density between 76 and 100%. Binning of address pointers to various feature map data of varying densities may be performed, for example, by a density analyzer such as described above (e.g., with respect to FIG. 6).

[0118] The multiple processing units may then process feature maps from the bins in a power-efficient manner, such as by reducing the peak power ramp (Δi / Δt). For example, a first processing unit may process one feature map from each bin (e.g., 1202A) before processing the feature map from the next bin (e.g., 1202B), with the resulting processing order in this example being bin 4 (highest power), bin 3, bin 2, bin 1 (lowest power).

[0119] Additionally, as explained above, multiple processing units may stagger their bin processing start order to minimize the initial power ramp and gather current, thereby minimizing peak and leakage power of the total processing power. Using the example of Figure 12, the processing start order for four processing units may be: Processing Unit 1: Bin 4, Bin 3, Bin 2, Bin 1; Processing Unit 2: Bin 3, Bin 2, Bin 1, Bin 4; Processing Unit 3: Bin 2, Bin 1, Bin 4, Bin 3; and Processing Unit 4: Bin 1, Bin 4, Bin 3, Bin 2. Notably, these orders are merely examples and others are contemplated.

[0120] Once bin processing has begun by a parallel processing unit, the next binning operation may be pipelined with the current processing unit, for example, using a circular buffer for processing.

[0121] 13 shows a comparison between conventional processing of machine learning data sub-blocks 1002A-D with kernels 1004A-D using processing units 1308A-D and density-reordered processing of machine learning data sub-blocks 1302A-D with kernels 1304A-D. In particular, this is an example where each data sub-block 1002A-D and 1302A-D operates on the same data with a different kernel.

[0122] As shown, the distribution of data density in sub-blocks 1002A-D is uniform, resulting in power spikes at the densest (darkest shading) data portions as explained above. In contrast, the distribution of density in data sub-blocks 1302A-D varies due to their density-based reordering, resulting in the benefits described above and further with respect to Figures 14A-14C. Specifically, in this example, data sub-blocks 1302A-D have undergone both time reordering and density-order reordering.

[0123] Figure 14A shows an exemplary chart of power consumption for multiple processing cores processing data sub-blocks 1302A-D of Figure 13. In particular, the peak power of each core is distributed as a result of distributing the densest portions of the processed data as shown in data sub-blocks 1302A-D of Figure 13. This contrasts with the concentration of peak processing power in Figure 11A, which is based on the distribution of density in data sub-blocks 1002A-D of Figure 10.

[0124] FIG. 14B shows an example of the total processing power consumption 1402 for cores 0-3 from FIG. 14A. As shown, the total processing power still varies over time, but the power spike shown at 1105 in FIG. 11B has been eliminated. Furthermore, the change in current 1404 (e.g., Δi) with a change in time 1406 (e.g., Δt) results in a much shallower power ramp 1407 (e.g., Δi / Δt) in power consumption. These improved power consumption characteristics avoid disruptive power effects such as voltage droop, as discussed above with respect to FIG. 11C.

[0125] Figure 14C shows an example of system voltage over time 1408 for times consistent with Figures 14A and 14B. As shown, there is no significant voltage droop and the system voltage remains within a safe operating band 1410 compared to the voltage droop 1112 shown in Figure 11C. Thus, system instability and performance-degrading actions on parts of the system are avoided while processing the same data simply in a density-based reordered order.

[0126] Figure 15 illustrates useful properties of some machine learning operations when using density-reordered input data.

[0127] As shown, applying the original sequence of feature maps 1502A to kernel 1504A results in output feature map 1506. Similarly, to obtain the same output feature map 1506, a rearranged kernel 1504B may be applied to a rearranged (e.g., time- and / or density-reordered) input feature map 1502B. Thus, the mathematical result after rearrangement is the same as the previous mathematical result, and the output feature map does not need to be separately rearranged in this scenario. Furthermore, the same mathematical result is obtained with improved system performance, as described above.

[0128] Exemplary Electronic Device for Parallel Data Processing Using Data Rearrangement FIG. 16 illustrates an exemplary electronic device 1600 that may be configured to perform parallel data processing using data reordering, for example, as described herein with respect to FIGS.

[0129] The electronic device 1600 includes a central processing unit (CPU) 1602, which in some examples may be a multi-core CPU. Instructions executed in the CPU 1602 may be loaded from, for example, a program memory associated with the CPU.

[0130] The electronic device 1600 also includes additional processing components tailored to specific functions, such as a graphics processing unit (GPU) 1604, a digital signal processor (DSP) 1606, a neural signal processing unit (NPU) 1608, a multimedia processing unit 1610, and wireless connectivity components 1612.

[0131] An NPU, such as the 1608, is generally a specialized circuit configured to implement all the necessary control and arithmetic logic to execute machine learning algorithms, such as algorithms for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. An NPU may alternatively be referred to as a neural signal processor (NSP), tensor processing unit (TPU), neural network processor (NNP), intelligence processing unit (IPU), vision processing unit (VPU), or graph processing unit.

[0132] NPUs such as 1608 are configured to accelerate the execution of common machine learning tasks such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip, such as a system-on-chip (SoC), while in other examples, they may be part of a dedicated neural network accelerator.

[0133] NPUs may be optimized for training or inference, or in some cases may be configured to balance training and inference performance. In NPUs that can perform both training and inference, the two tasks may nevertheless generally be performed independently.

[0134] NPUs designed to accelerate training are typically configured to accelerate the curation of new models, a computationally intensive operation that involves inputting an existing dataset (often labeled or tagged), iterating over the dataset, and then adjusting model parameters such as weights and biases to improve model performance. Optimization based on mispredictions typically involves backpropagating through each layer of the model and determining gradients to reduce prediction error.

[0135] NPUs designed to accelerate inference are generally configured to operate on complete models. Thus, such NPUs may be configured to input new data and rapidly process the data through an already trained model to generate model output (e.g., inference information).

[0136] In one implementation, the NPU 1608 is part of one or more of the CPU 1602, the GPU 1604, and / or the DSP 1606.

[0137] In some examples, the wireless connectivity component 1612 may include sub-components for, for example, third-generation (3G) connectivity, fourth-generation (4G) connectivity (e.g., 4G LTE), fifth-generation connectivity (e.g., 5G or NR), Wi-Fi connectivity, Bluetooth connectivity, and other wireless data transmission standards. The wireless connectivity processing component 1612 is further connected to one or more antennas 1614.

[0138] The electronic device 1600 may include one or more sensor processing units 1616 associated with any type of sensor, one or more image signal processors (ISPs) 1618 associated with any type of image sensor, and / or a navigation processor 1620 that may include satellite-based positioning system components (e.g., GPS or GLONASS) and inertial positioning system components.

[0139] The electronic device 1600 may include one or more input and / or output devices 1622, such as a screen, a touch surface (including a touch display), physical buttons, a speaker, a microphone, etc.

[0140] In some examples, one or more of the processors of electronic device 1600 may be based on the ARM or RISC-V instruction set.

[0141] Electronic device 1600 also includes memory 1624, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, memory 1624 includes computer-executable components, which may be executed by one or more of the aforementioned processors of electronic device 1600. Although not shown in FIG. 16 , the electronic device may also include a direct memory access (DMA) controller.

[0142] In particular, in this example, memory 1624 includes a transmitting component 1624A, a receiving component 1624B, a processing component 1624C, a determining component 1624D, an output component 1624E, a sub-block component 1624F, a density estimating component 1624G, a sequence (or order) component 1624H, and an adjusting component 16241. The illustrated components and other components not illustrated may be configured to perform various aspects of the methods described herein.

[0143] In general, the electronic device 1600 and / or its components may be configured to perform the methods described herein.

[0144] 17 illustrates an exemplary multi-processor processing system 1700 that may be implemented using the examples described herein. For example, the multi-processing system 1700 may represent various processors of the electronic device 1600 of FIG.

[0145] In this example, system 1700 includes processors 1701, 1703, and 1705, although in other examples, any number of individual processors may be used. Additionally, processors 1701, 1703, and 1705, while depicted similarly, may represent various different types of processors in electronic devices, such as CPUs, GPUs, DSPs, NPUs, etc., as described herein.

[0146] Each of processors 1701, 1703, and 1705 includes an instruction scheduler, various hardware subcomponents (e.g., Hardware X, Hardware Y, and Hardware Z), and local memory. In some examples, the local memory may be tightly coupled memory (TCM). Note that while the components of each of processors 1701, 1703, and 1705 are shown as the same in this example, in other examples, some or each of processors 1701, 1703, and 1705 may have different hardware configurations, different hardware elements, etc.

[0147] Each of the processors 1701, 1703, and 1705 is also in data communication with a global memory, such as a DDR memory, or other type of volatile working memory. For example, the global memory 1707 may represent the memory 1624 of FIG.

[0148] In some implementations, in a multiprocessor processing system such as 1700, one of the processors may act as a master processor. For example, in this example, processor 1701 may be the master processor. The master processor may include a compiler that, when executed, can determine how a model, such as a neural network, is processed by various components of processing system 1700. For example, a hardware parallel configuration may be implemented by mapping some of the model's processing to various hardware (e.g., hardware X, hardware Y, and hardware Z) within a given processor (e.g., processor 1701) and some of the model's processing to other processors (e.g., processors 1703 and 1705) and their associated hardware. For example, parallel blocks in the parallel block processing architectures described herein may be mapped to different portions of each of the various hardware in processors 1701, 1703, and 1705.

[0149] (Example) Example 1 1. A method for parallel data processing, comprising: receiving data for parallel processing from a data processing requester; generating a plurality of data sub-blocks; determining a plurality of data portions in each data sub-block of the plurality of data sub-blocks; permuting the plurality of data portions in at least one data sub-block of the plurality of data sub-blocks; providing the plurality of data sub-blocks, including at least one data sub-block containing the permuted order of the plurality of data portions, to a plurality of processing units for parallel processing; and receiving processed data associated with the plurality of data sub-blocks from the plurality of processing units.

[0150] Example 2 2. The method of claim 1, wherein the step of determining a plurality of data portions in each data sub-block of the plurality of data sub-blocks includes: determining a plurality of density ranges, each density range of the plurality of density ranges being different from each other density range of the plurality of density ranges; and determining the plurality of data portions based on the plurality of density ranges.

[0151] Example 3 3. The method of any one of Examples 1 or 2, wherein the step of changing the order of the plurality of data portions for at least one data sub-block of the plurality of data sub-blocks includes the step of time-shifting each data portion of the plurality of data portions along a processing time direction in a circular order.

[0152] Example 4 4. The method of any one of Examples 1 to 3, wherein the step of rearranging the plurality of data portions for at least one data sub-block of the plurality of data sub-blocks includes the step of arranging the plurality of data portions according to one of an increasing density order or a decreasing density order.

[0153] Example 5 The method of any one of Examples 1 to 4, further comprising: reordering the processed data associated with the plurality of data sub-blocks based on an original order of the plurality of data portions in each data sub-block of the plurality of data sub-blocks; and providing the processed data to the data processing requester.

[0154] Example 6 6. The method of any one of examples 1 to 5, further comprising permuting the plurality of data portions in each data sub-block of the plurality of data sub-blocks.

[0155] Example 7 7. A method as in any one of claims 1 to 6, wherein the step of determining the plurality of data portions in each data sub-block of the plurality of data sub-blocks is performed by a direct memory access (DMA) controller in data communication with the plurality of processing units.

[0156] Example 8 8. The method of embodiment 7, wherein the step of permuting the plurality of data portions in at least one of the plurality of data sub-blocks is performed by a DMA controller.

[0157] Example 9 9. The method of any one of examples 1 to 8, wherein the step of permuting the plurality of data portions in at least one data sub-block of the plurality of data sub-blocks comprises reading, by a plurality of processing units, the plurality of data portions in a predefined order.

[0158] Example 10 10. The method of any one of Examples 1 to 9, wherein the plurality of processing units comprises at least one neural signal processing unit.

[0159] Example 11 11. A processing system for parallel data processing, comprising: a memory containing computer-executable instructions; and one or more processors configured to execute the computer-executable instructions to cause the processing system to perform a method according to any one of Examples 1 to 10.

[0160] Example 12 A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of a processing system, cause the processing system to perform the method of any one of Examples 1 to 10.

[0161] Example 13 A computer program product embodied on a computer-readable storage medium, the computer program product comprising code for performing the method according to any one of Examples 1 to 10.

[0162] Example 14 1. A method for parallel data processing, comprising: receiving data for parallel processing from a data processing requester; determining a plurality of density ranges, each density range of the plurality of density ranges being different from each other density range of the plurality of density ranges; determining a plurality of data portions of the data based on the density ranges; generating a plurality of data sub-blocks having rearranged data portions; providing the plurality of data sub-blocks to a plurality of processing units for parallel processing; and receiving processed data associated with the plurality of data sub-blocks from the plurality of processing units.

[0163] Example 15 15. The method of claim 14, wherein the step of determining a plurality of data portions of the data based on a density range includes determining a density for each respective data portion of the plurality of data portions; and assigning each respective data portion of the plurality of data portions to one density bin of a plurality of density bins based on the determined density for the respective data portion, wherein each density bin is associated with one density range of the plurality of density ranges.

[0164] Example 16 16. The method of example 15, wherein generating a plurality of data sub-blocks having rearranged data portions includes adding data portions to the plurality of data sub-blocks according to a plurality of bin sequences, each bin sequence of the plurality of bin sequences including a unique sequence of a plurality of density bins.

[0165] Example 17 17. The method of embodiment 16, further comprising assigning each bin sequence of the plurality of bin sequences to one or more processing units of the plurality of processing units, wherein each respective data subblock of the plurality of data subblocks is provided to one processing unit of the plurality of processing units based on the assigned bin sequence of the respective data subblock.

[0166] Example 18 17. The method of example 16, further comprising determining a plurality of bin sequences according to one of increasing density order or decreasing density order and a processing order time shift.

[0167] Example 19 19. The method of any one of Examples 14 to 18, further comprising providing the processed data to a data processing requester.

[0168] Example 20 20. The method of any one of Examples 14 to 19, wherein the data for parallel processing is machine learning model input data for a convolutional neural network model, each data portion of the plurality of data portions including a feature map, and the method further comprises: rearranging a convolution kernel for each respective data sub-block of the plurality of data sub-blocks based on a sequence of the data portions in the respective data sub-block; and applying the rearranged convolution kernel to the respective data sub-block.

[0169] Example 21 21. A method according to any one of claims 14 to 20, wherein the step of determining the plurality of data portions of the data based on the density range is performed by a direct memory access (DMA) controller in data communication with the plurality of processing units.

[0170] Example 22 22. The method of claim 21, wherein the step of generating a plurality of data sub-blocks having rearranged data portions is performed by a DMA controller.

[0171] Example 23 21. The method of any of Examples 14 to 20, wherein the plurality of processing units comprises at least one neural signal processing unit.

[0172] Example 24 24. A processing system for parallel data processing, comprising: a memory containing computer-executable instructions; and one or more processors configured to execute the computer-executable instructions to cause the processing system to perform a method according to any one of Examples 14 to 23.

[0173] Example 25 A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of a processing system, cause the processing system to perform the method of any one of Examples 14 to 23.

[0174] Example 26 24. A computer program product embodied on a computer-readable storage medium, the computer program product comprising code for performing the method according to any one of Examples 14 to 23.

[0175] Additional Considerations The foregoing description is provided to enable any person skilled in the art to practice the various examples described herein. The examples described herein are not intended to limit the scope, applicability, or examples set forth in the claims. Various modifications to these examples will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other examples. For example, changes may be made in the function and arrangement of described elements without departing from the scope of the disclosure. Various examples may omit, substitute, or add various procedures or components as appropriate. For example, described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Also, features described with respect to some examples may be combined in some other examples. For example, an apparatus may be implemented or a method may be practiced using any number of the aspects described herein. Additionally, the scope of the disclosure is intended to extend to such apparatuses or methods practiced using other structure, functions, or structure and functions in addition to or other than the various aspects of the disclosure described herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of a claim.

[0176] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects.

[0177] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items, including single members. As an example, "at least one of a, b, or c" is intended to encompass a, b, c, ab, ac, bc, and abc, as well as any combination having multiples of the same element (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other permutation of a, b, and c).

[0178] As used herein, the term "determining" encompasses a wide variety of actions. For example, "determining" may include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, database, or another data structure), ascertaining, etc. Also, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), etc. Also, "determining" may include resolving, selecting, electing, establishing, etc.

[0179] The methods disclosed herein include one or more steps or actions for achieving the method. Method steps and / or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims. Furthermore, various operations of the methods described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and / or software components and / or modules, including, but not limited to, circuits, application-specific integrated circuits (ASICs), or processors. Generally, where operations are illustrated in figures, those operations may have corresponding equivalent means-plus-function components that are similarly numbered.

[0180] The following claims are not intended to be limited to the examples set forth herein but are to be accorded the full scope consistent with the language of the claims. Within the claims, reference to an element in the singular does not mean "one and only one," but rather "one or more." Unless otherwise expressly stated, the term "some" refers to one or more. Claim elements are not to be construed under the provisions of 35 U.S.C. § 112(f) unless the element is expressly recited using the phrase "means for," or, in the case of a method claim, unless the element is recited using the phrase "step for." All structural and functional equivalents of the elements of the various aspects described throughout this disclosure that are known or later become known to those skilled in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is made public, regardless of whether such disclosure is expressly recited in the claims. [Explanation of symbols]

[0181] 100 Parallel Data Processing Architecture 102 Applications 104 Data Processing Coordinator 106 processing cores 110 Data Processing System 202 Multidimensional data block, data block, data 204 data sub-blocks 300 Parallel Data Processing Architecture 302 Application 304 Data Processing Coordinator 305 Data Density Analyzer 306 processing cores 307 Data Sequencer 310 Data Processing System 404 Data Sub-Block 602 data blocks 604 Processing Coordinator 606 Density Analyzer 608 Density Bins 610 Sequencer 611 Density Bin Sequence Map 616 processing cores 802 Application 804 Processing System 806 Direct Memory Access (DMA) Controller 808 Density Analyzer 810 Sequencer 812 Neural Signal Processing Unit (NPU) 814 Tightly Coupled Memory (TCM) 1600 Electronic Devices 1602 Central Processing Unit (CPU) 1604 Graphics Processing Unit (GPU) 1606 Digital Signal Processor (DSP) 1608 Neural Signal Processing Unit (NPU) 1610 Multimedia Processing Unit 1612 Wireless Connectivity Components 1614 Antenna 1616 Sensor Processing Unit 1618 Image Signal Processor (ISP) 1620 Navigation Processor 1622 Input / Output Devices 1624 memory 1700 Multi-Processing System 1701 processor 1703 processor 1705 processor 1707 Global Memory

Claims

1. 1. A method for parallel data processing, comprising: receiving data for parallel processing from a data processing requester; generating a plurality of data sub-blocks based on the data; determining a plurality of data portions in each data sub-block of the plurality of data sub-blocks; permuting the plurality of data portions in at least one data sub-block of the plurality of data sub-blocks; providing the plurality of data sub-blocks, including the at least one data sub-block containing the permuted order of the plurality of data portions, to a plurality of processing units for parallel processing; receiving processed data associated with the plurality of data sub-blocks from the plurality of processing units; A method comprising:

2. determining a plurality of data portions in each data sub-block of the plurality of data sub-blocks, determining a plurality of density ranges, each density range of the plurality of density ranges being different from each other density range of the plurality of density ranges; determining the plurality of data portions based on the plurality of density ranges; 2. The method of claim 1, comprising:

3. 2. The method of claim 1, wherein the step of changing the order of the plurality of data portions for the at least one data sub-block of the plurality of data sub-blocks comprises the step of time-shifting each data portion of the plurality of data portions along a processing time direction in a circular order.

4. 2. The method of claim 1, wherein rearranging the order of the plurality of data portions for the at least one data sub-block of the plurality of data sub-blocks comprises arranging the plurality of data portions according to one of an increasing density order or a decreasing density order.

5. reordering the processed data associated with the plurality of data sub-blocks based on an original order of the plurality of data portions in each data sub-block of the plurality of data sub-blocks; providing the processed data to the data processing requester; The method of claim 1 further comprising:

6. The method of claim 1 , further comprising permuting the plurality of data portions in each data sub-block of the plurality of data sub-blocks.

7. 2. The method of claim 1, wherein determining the plurality of data portions in each data sub-block of the plurality of data sub-blocks is performed by a direct memory access (DMA) controller in data communication with the plurality of processing units.

8. 8. The method of claim 7, wherein the permuting of the plurality of data portions in the at least one data sub-block of the plurality of data sub-blocks is performed by the DMA controller.

9. 2. The method of claim 1, wherein the step of permuting the plurality of data portions in the at least one data sub-block of the plurality of data sub-blocks comprises reading the plurality of data portions in a predefined order by the plurality of processing units.

10. The method of claim 1 , wherein the plurality of processing units includes at least one neural signal processing unit.

11. 1. A processing system for parallel data processing, comprising: a memory containing computer-executable instructions; Executing the computer-executable instructions, the processing system receiving data for parallel processing from a data processing requester; generating a plurality of data sub-blocks based on the data; determining a plurality of data portions in each data sub-block of the plurality of data sub-blocks; permuting the plurality of data portions in at least one data sub-block of the plurality of data sub-blocks; providing the plurality of data sub-blocks, including the at least one data sub-block containing the permuted order of the plurality of data portions, to a plurality of processing units for parallel processing; receiving processed data associated with the plurality of data sub-blocks from the plurality of processing units; one or more processors configured to cause A processing system comprising:

12. To determine a plurality of data portions in each data sub-block of the plurality of data sub-blocks, the one or more processors: determining a plurality of density ranges, each density range of the plurality of density ranges being different from each other density range of the plurality of density ranges; determining the plurality of data portions based on the plurality of density ranges; The processing system of claim 11 , further configured to:

13. 12. The processing system of claim 11, wherein the one or more processors are further configured to time-shift each data portion of the plurality of data portions along a processing time direction in a circular order to change the order of the plurality of data portions for the at least one data sub-block of the plurality of data sub-blocks.

14. 12. The processing system of claim 11, wherein to vary the ordering of the plurality of data portions for the at least one data sub-block of the plurality of data sub-blocks, the one or more processors are further configured to arrange the plurality of data portions according to one of an increasing density order or a decreasing density order.

15. the one or more processors: permuting the processed data associated with the plurality of data sub-blocks based on an original order of the plurality of data portions in each data sub-block of the plurality of data sub-blocks; providing the processed data to the data processing requester; The processing system of claim 11 , further configured to:

16. The processing system of claim 11 , wherein the one or more processors are further configured to permute the plurality of data portions within each data sub-block of the plurality of data sub-blocks.

17. 12. The processing system of claim 11, further comprising a direct memory access (DMA) controller in data communication with the plurality of processing units, configured to determine the plurality of data portions in each data sub-block of the plurality of data sub-blocks.

18. 20. The processing system of claim 17, wherein the DMA controller is further configured to permute the order of the plurality of data portions in the at least one data sub-block of the plurality of data sub-blocks.

19. 12. The processing system of claim 11, wherein the one or more processors are configured to read the plurality of data portions in a predefined order to vary the order of the plurality of data portions in the at least one data sub-block of the plurality of data sub-blocks.

20. The processing system of claim 11 , wherein the plurality of processing units includes at least one neural signal processing unit.

21. 1. A method for parallel data processing, comprising: receiving data for parallel processing from a data processing requester; determining a plurality of density ranges, each density range of the plurality of density ranges being different from each other density range of the plurality of density ranges; determining a plurality of data portions of the data based on the density range; generating a plurality of data sub-blocks having the rearranged data portions; providing the plurality of data sub-blocks to a plurality of processing units for parallel processing; receiving processed data associated with the plurality of data sub-blocks from the plurality of processing units; A method comprising:

22. determining the plurality of data portions of the data based on the density ranges, determining a density for each respective data portion of the plurality of data portions; assigning each respective data portion of the plurality of data portions to one density bin of a plurality of density bins based on the determined density for the respective data portion; Including, The method of claim 21 , wherein each density bin is associated with one density range of the plurality of density ranges.

23. generating the plurality of data sub-blocks having rearranged data portions, adding data portions to the plurality of data sub-blocks according to a plurality of bin sequences; Including, The method of claim 22 , wherein each bin sequence of the plurality of bin sequences comprises a unique sequence of the plurality of density bins.

24. assigning each bin sequence of the plurality of bin sequences to one or more processing units of the plurality of processing units; further comprising 24. The method of claim 23, wherein each respective data sub-block of the plurality of data sub-blocks is provided to one processing unit of the plurality of processing units based on the assigned bin sequence of the respective data sub-block.

25. one of an increasing density order or a decreasing density order; Processing order time shift and 24. The method of claim 23, further comprising determining the plurality of bin sequences according to:

26. 22. The method of claim 21, further comprising providing the processed data to the data processing requestor.

27. the data for parallel processing is machine learning model input data for a convolutional neural network model; each data portion of the plurality of data portions includes a feature map; The method comprises: rearranging a convolution kernel for each respective data sub-block of the plurality of data sub-blocks based on a sequence of data portions in the respective data sub-block; applying the reordered convolution kernel to each of the data sub-blocks; 22. The method of claim 21, further comprising:

28. 22. The method of claim 21, wherein determining the plurality of data portions of the data based on the density ranges is performed by a direct memory access (DMA) controller in data communication with the plurality of processing units.

29. 30. The method of claim 28, wherein generating the plurality of data sub-blocks having rearranged data portions is performed by the DMA controller.

30. The method of claim 21 , wherein the plurality of processing units includes at least one neural signal processing unit.

31. 1. A processing system for parallel data processing, comprising: a memory containing computer-executable instructions; Executing the computer-executable instructions, the processing system receiving data for parallel processing from a data processing requester; determining a plurality of density ranges, each density range of the plurality of density ranges being different from each other density range of the plurality of density ranges; determining a plurality of data portions of the data based on the density range; generating a plurality of data sub-blocks having the rearranged data portions; providing the plurality of data sub-blocks to a plurality of processing units for parallel processing; receiving processed data associated with the plurality of data sub-blocks from the plurality of processing units; one or more processors configured to cause A processing system comprising:

32. To determine the plurality of data portions of the data based on the density range, the one or more processors: determining a density for each respective data portion of the plurality of data portions; assigning each respective data portion of the plurality of data portions to one density bin of a plurality of density bins based on the determined density for the respective data portion; further configured to:

32. The processing system of claim 31, wherein each density bin is associated with one density range of the plurality of density ranges.

33. To generate the plurality of data sub-blocks having rearranged data portions, the one or more processors: adding data portions to the plurality of data sub-blocks according to a plurality of bin sequences; further configured as follows:

33. The processing system of claim 32, wherein each bin sequence of the plurality of bin sequences comprises a unique sequence of the plurality of density bins.

34. the one or more processors: assigning each bin sequence of the plurality of bin sequences to one or more processing units of the plurality of processing units; further configured as follows:

34. The processing system of claim 33, wherein each respective data sub-block of the plurality of data sub-blocks is provided to one processing unit of the plurality of processing units based on the assigned bin sequence of the respective data sub-block.

35. the one or more processors: one of an increasing density order or a decreasing density order; Processing order time shift and 34. The processing system of claim 33, further configured to determine the plurality of bin sequences according to:

36. 32. The processing system of claim 31, wherein the one or more processors are further configured to provide the processed data to the data processing requester.

37. the data for parallel processing is machine learning model input data for a convolutional neural network model; each data portion of the plurality of data portions includes a feature map; the one or more processors: rearranging a convolution kernel for each respective data sub-block of the plurality of data sub-blocks based on a sequence of data portions in the respective data sub-block; applying the reordered convolution kernel to each of the data sub-blocks; 32. The processing system of claim 31, further configured to:

38. 32. The processing system of claim 31, further comprising a direct memory access (DMA) controller in data communication with the plurality of processing units and configured to determine the plurality of data portions of the data based on the density range.

39. 39. The processing system of claim 38, wherein the DMA controller is further configured to generate the plurality of data sub-blocks having reordered data portions.

40. 32. The processing system of claim 31, wherein the plurality of processing units includes at least one neural signal processing unit.

Images