Apparatus for use in processing equipment and apparatus and methods for artificial neural networks

Abstract

Apparatuses for use in processing devices and apparatuses and methods for artificial neural networks are disclosed. An apparatus for use in a processing device includes a processor configured to process data having a predefined data structure. A memory abstraction apparatus is coupled to the processor and configured to determine an address of packed data for the processor. The packed data is stored on a memory apparatus coupled to the processor. The memory abstraction apparatus is further configured to provide output data to the processor based on the address of the packed data, wherein the output data is configured according to the predefined data structure.

Description

Technical Field

[0001] This disclosure pertains to the field of neural networks, and more specifically, relates to increasing processor-based neural network computation. Background Technology

[0002] In machine learning, a convolutional neural network (CNN or ConvNet) is a type of feedforward artificial neural network (ANN). In feedforward ANNs, the connection patterns between neurons are inspired by the organization of the visual cortex in animals. Individual cortical neurons respond to stimuli in confined spatial regions called receptive fields. The overlapping of receptive fields from different neurons partially flattens the visual field. The response of a single neuron to stimuli within its receptive field can be mathematically approximated by a convolution operation. Inspired by biological processes, convolutional networks are a variant of multilayer perception designed to use minimal preprocessing. They have wide applications in image and video recognition, recommender systems, and natural language processing. Convolutional neural networks can also be called shift-invariant or spatially invariant artificial neural networks (SIANNs), a name based on their shared weight architecture and shift-invariant features.

[0003] Sensing and control applications can benefit from the application of algorithms employing ANNs. However, compact digital signal processor (DSP) and microcontroller (uC) designs, suitable for high-volume, cost-sensitive applications, exhibit poor performance when performing ANN calculations. The limited memory of such devices can mean that ANNs are implemented using packed / compressed subword data. Unfortunately, unpacking such data can cause stream address generation, gathering, or (re)formatting operations that are unsuitable for the full-word instruction set / aligned-word instruction set and sequential pipelined microarchitecture of compact uC and DSP systems. Therefore, only a fraction of the processing power is available for useful payload calculations (e.g., multiplication / accumulation, minimum / maximum values, etc.) when performing ANN calculations. The processing resources (and power / energy) of compact uC and DSP systems are inefficiently used to unpack narrow subword data using full-word arithmetic / logic operations. This performance bottleneck limits the scope for applying other highly attractive ANN algorithms in sensing / control devices employing such compact uCs.

[0004] ANN algorithms can be implemented on the central processing unit (CPU) using extended single-instruction, multiple-data (SIMD) instruction sets that include packing / unpacking and scatter / aggregate memory fetch operations. However, these approaches are not always unsuitable for compact uCs. For example, instruction-level parallelism is required to mask the latency of scatter / aggregate memory fetch operations. The area and power constraints of compact uC designs preclude instruction-level parallelism, significantly reducing the potential value of such instruction set extensions. Furthermore, the cost-effective payload and memory microarchitecture typically employed in compact uC designs cannot directly support the multiple source and target instruction formats required for scatter / aggregate unpacking and packing instructions. Extensions supporting such operations (e.g., additional register file ports, complex pipeline pause / drain logic, etc.) directly impact critical path logic elements, and therefore the associated edge area and power costs can be disproportionately high. Additionally, the general-purpose SIMD instructions required for efficient use of SIMD scatter / aggregate unpacking and packing instructions may not be necessary. Therefore, the effective edge area and power costs for ANN acceleration will be more tightly coupled with the hardware required for efficient packing and unpacking processes.

[0005] Furthermore, loosely coupled accelerator peripherals using streaming direct memory access (DMA) are typically used to address performance issues for specific signal processing applications. However, for ANN acceleration in compact uC systems, loosely coupled accelerator peripherals cannot significantly reduce the bottleneck of data aggregation and unpacking, and low-latency DMA to and from the coprocessor is often unavailable in compact uC systems. This can significantly reduce the ability to accelerate ANN algorithms in compact uC systems because coprocessor resources wait for data aggregation and unpacking before completing ANN computation, thus significantly increasing the execution time of the ANN algorithm. Summary of the Invention

[0006] According to one aspect of the present invention, an apparatus for use in a processing device is provided. The apparatus includes: a processor configured to process data having a predefined data structure; a memory extraction device coupled to the processor and configured to: determine an address for packaged data for the processor, wherein the packaged data is stored on a memory device coupled to the processor; and provide output data to the processor based on the address of the packaged data, wherein the output data is configured according to a predefined data structure.

[0007] According to another aspect of the present invention, an apparatus for an artificial neural network is provided. The apparatus includes: a microcontroller configured to be coupled to a memory device and configured to process a data stream, wherein the data stream is configured to match at least one of a predefined precision, a predefined layout, or a predefined data structure; and a data stream extraction device coupled to the memory device and the microcontroller and configured to process data stored on the memory device and provide a continuous output stream to the microcontroller, wherein the data stream extraction device includes: an address generator component configured to generate a plurality of data addresses of the data stored on the memory device; a data extraction component configured to import selected data from the memory device based on the plurality of data addresses; and a data formatting component configured to provide a continuous output stream to the microcontroller.

[0008] According to another aspect of the present invention, a method for an artificial neural network is provided. The method includes: determining, via a memory extraction device, a plurality of addresses of packaged data stored on a memory device and a suppression indicator, wherein the suppression indicator identifies a subset of the plurality of addresses associated with generated packaged data in the memory device; extracting the stored packaged data from the memory device based on the plurality of addresses and the suppression indicator; generating the generated packaged data based on the suppression indicator; and constructing formatted data based on the stored packaged data or the generated packaged data, via the memory extraction device, according to a predefined data structure of a microcontroller. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating a convolutional neural network (CNN) system according to the described aspects (implementation methods).

[0010] Figure 2 This is a block diagram illustrating a CNN based on the described aspects.

[0011] Figure 3 A diagram showing the convolution operation based on the described aspects and some related components is presented.

[0012] Figure 4 A diagram showing aspects of pooling operations based on the described aspects is presented.

[0013] Figure 5 Another diagram is shown, illustrating aspects of convolution and pooling operations according to the described aspects.

[0014] Figure 6 This is a block diagram illustrating one or more data extraction units tightly coupled to the CPU according to the described aspects.

[0015] Figure 7 This is a block diagram showing the various components of the data extraction unit based on the described aspects.

[0016] Figure 8 The memory layout for an input array for a two-dimensional convolution process, in which adjacent kernel footprints overlap the memory layout, is shown, according to the described aspects.

[0017] Figure 9 A table is shown representing the various input / output values ​​generated by the components of the data extraction unit according to the described aspects.

[0018] Figure 10 It is a flowchart illustrating the processing flow based on the described aspects. Detailed Implementation

[0019] This disclosure will now be described with reference to the accompanying drawings, wherein the same reference numerals are always used to refer to the same elements, and wherein the structures and apparatus shown are not necessarily drawn to scale. As used herein, the terms “component,” “system,” “interface,” etc., are intended to refer to computer-related entities, hardware, software (e.g., in execution), and / or firmware. For example, a component can be a processor, a process running on a processor, a controller, an object, an executable file, a program, a storage device, and / or a computer having processing power. By way of illustration, an application running on a server and the server itself can also be a component. One or more components may reside within a process, and components may be located on one computer and / or distributed among two or more computers. This document may describe a group of elements or a group of other components, wherein the term “group” can be interpreted as “one or more.”

[0020] Furthermore, these components can be executed from various computer-readable storage media on which various data structures, such as modules, are stored. Components can communicate via local and / or remote processes, for example, based on signals having one or more data packets (e.g., data from a component interacting with another component in a local system, a distributed system, and / or interacting with other systems across networks such as the Internet, a local area network, a wide area network, or similar networks).

[0021] As another example, a component can be a device having specific functions provided by mechanical parts operated by an electric or electronic circuitry system, wherein the electric or electronic circuitry system can be operated by a software application or firmware application executed by one or more processors. The one or more processors can be internal or external to the device and can execute at least a portion of the software or firmware application. As yet another example, a component can be a device that provides specific functions through electronic parts rather than mechanical parts; the electronic parts may include software and / or firmware that at least partially endow the electronic parts with the functions described above.

[0022] The exemplary use of words is intended to present the concept in a concrete manner. As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clearly indicated by the context, "X adopts A or B" is intended to mean any of the natural inclusive arrangements. That is, if X adopts A; X adopts B; or X adopts A and B, then "X adopts A or B" is satisfied in any of the foregoing cases. Additionally, unless otherwise specified or clearly indicated by the context, the articles "a" and "an" as used in this application and the appended claims should generally be interpreted as meaning "one or more". Furthermore, with regard to the terms "including", "includes", "having", "has", "with", or variations thereof used in the detailed description and any of the claims, these terms are intended to be included in a manner similar to the term "comprising".

[0023] Considering the aforementioned shortcomings of compact microcontrollers (uCs) executing ANN algorithms, various aspects / implementations enable the acceleration of ANN algorithm computation in compact uC systems. The aspects / implementations used as solutions in this disclosure increase the throughput and power efficiency of ANN algorithms implemented on compact uC systems. The embodiments or aspects described herein accelerate the execution of data aggregation and unpacking for ANN computations while minimizing the use of CPU or coprocessor resources. Therefore, the disclosed methods and systems can improve the processing speed and power efficiency of ANN algorithms implemented on compact uC systems. For example, the data fetching and unpacking bottlenecks in compact uC systems executing ANN algorithms (or similar numerical computations) can be accelerated by sending the output data stream to a tightly coupled memory fetching unit of the main CPU. The memory fetching unit is tightly coupled to the main CPU and configured to fetch and unpack data from the main CPU in parallel with the execution of ANN computations on the main CPU. This accelerates the parameter and activation extraction of the inner loop for sparse product and / 2D / 1D sliding window convolution computations, which include a large amount of computation used to perform the ANN algorithm.

[0024] One or more memory fetching units are each configured to accelerate the fetching of the input stream required for ANN computation from packed or compressed subword data. The storage location, layout, and packing format of the input stream can be programmed by the main CPU via configuration registers / memory. Aggregated and unpacked data from one or more memory fetching units are written to output registers / memory in a format suitable for direct use in CPU arithmetic / logic instructions, readable by the main CPU. The aggregated and unpacked data can then be fetched from the output registers / memory by the main CPU, configured to perform ANN computations, thereby reducing the main CPU's need to unpack, fetch, or format data and increasing the speed of the ANN algorithm. Even during CPU wait periods, the memory fetching units are configured to fetch new data items concurrently with CPU activity. This further contributes to the speedup gain of the ANN algorithm. Additional aspects and details of this disclosure are further described below with reference to the accompanying drawings.

[0025] Figure 1An example of a CNN system 100 is shown, which can reliably classify images into specific categories / classes, thereby allowing driver assistance systems to make critical decisions, such as reducing engine speed, braking, stopping, or other functions involving motor vehicles, other vehicles, or equipment. For example, the CNN system 100 may include one or more sensors 102 that provide datasets as input to a CNN 104, which includes multiple different computation / processing layers with various components. The CNN 104 then provides outputs to one or more ADAS detectors 106 to 112 (e.g., pedestrian detector 106, car detector 108, bicycle or motorcycle detector 110, traffic sign detector 112, or other detectors), which detect the presence of conditions associated with categories (pedestrian features, car functions, bicycle functions, traffic sign features, etc.) corresponding to the data from any of the detectors 106 to 112. Based on whether a given detector determines the presence of a condition according to the output data of the CNN, detectors 106 to 112 may signal to the application, such as a collision warning 116, a stop / slow down braking function, the start / stop of something, or other application functions associated with a condition that is part of the function of an autonomous vehicle navigation system or other system / device.

[0026] In one example, sensor 102 may include an image capture device, such as a camera, radar, light detection and ranging (LIDAR), other image scanning devices, or other sensors for detecting and processing data from images—such as those received on signals from the image capture device. The dataset from sensor 102 can then be fed to CNN 104 to take the signal and then attempt to determine what the signal represents. For example, a car traveling on a highway with a front-facing camera sensor attached may return images of traffic signs, and if any, images of lampposts, and images of the car itself, if any, or if there is a car driving ahead, as part of the sensor data.

[0027] Typically, in an autopilot system where the driver sits in the back seat, the person makes many decisions, such as decelerating, applying the brakes, honking the horn, or other actions, which are essentially done by the autopilot / navigation system. A dedicated computer processing unit is on the board, and it's important for them to decide what to do next, and what's important is understanding what's in front of them. Therefore, the main purpose of the system and CNN104 is to determine what's contained in these images / pictures, and then make further decisions based on what's found in these images.

[0028] For example, if the analysis of one or more images indicates a situation / pattern in the image—such as the presence of a person on the road approximately 100 meters away—the onboard computer will noticeably slow down and possibly bring the vehicle to a stop. Alternatively, if the determination is based on acceleration of the situation within the image, it will simply guide the vehicle to change lanes and then accelerate. Therefore, these or other decisions can be derived from the properties of the data analyzed within the image as characteristic patterns returned by the system, and then analyzed for any number of given situations present in the image. The system can be robust through initial training with filters or kernel filters while learning to detect possible variations to identify situations in various environments.

[0029] CNN 104 can include a series of different computations that can be interconnected and dependent on each other as CNN computation layers, including various components for providing the computation results of a given layer to the next layer. These layers in Figure 1 The image can be viewed as layers with different shades within a CNN block of 104. Outputs (e.g., Pp, Pcar, Pcycle, Pts) appear at the ends of the computation channels, which can include different probabilities from various possible states within a subset of the image or a sector of the image. For example, by summing these probabilities, the resulting value can be equal to one (1). The object type or category / class type with the highest probability can then be the object type or category / class type corresponding to the image or image sector, detected by one or more detectors 106 to 112 and acted upon by one or more applications 114 and 116.

[0030] For example, if the image contains pedestrians, the pedestrian ADAS detector 106 can be configured to send a signal to the braking system to apply the brakes. Any number of items can be detected within the image by one or more detectors 106 to 112, not just the pedestrian as an example. Furthermore, in response to patterns of object / category types detected by the convolutional neural network system, signals can be provided to its triggering components to make further decisions based on one or more predefined criteria. For example, the pedestrian detector can send an image message to the collision warning component / system indicating that a pedestrian has been detected. The collision warning system can then immediately take action, perhaps applying brakes to slow down, decelerating the vehicle if it moves to the right or left, or making another decision based on pattern recognition of the pedestrian in the image, thereby avoiding a fatal collision. Similar examples can be made from other patterns or conditions detected in the image from sensor data, such as car detection, bicycles, traffic signs, or any other external pattern that triggers action when detected.

[0031] Reference Figure 2 In particular, Figure 2This is another example of the various computational layers that make up CNN 104. CNN 104 may include a number of computational layers, including convolutional layer 202, rectified linear unit (RELU) layer 204, pooling layer 206, fully connected (FC) layer 208 (artificial neural network layer), and output layer 210. Although five computational layers have been shown, as will be understood by those skilled in the art, more or fewer computational layers can be envisioned. As used herein, a layer or computational (a1) layer may refer to one or more components that operate in a similar functional manner through mathematical or other functional means to process received inputs to generate / derive outputs for the next layer, and one or more other components for further processing within the convolutional neural network system.

[0032] For example, convolutional layer 202 may include one or more convolutional units 212, which extract data slices of image 232 as a dataset. Convolutional layer 202 may be combined with rectified linear unit (RELU) layer 204 so that it is also considered or referred to as a computational layer 230, or generally as convolutional layer 230. In particular, convolution operations may be performed by convolutional unit 212 to extract features from received images as inputs from the input end (e.g., camera data, radar data, other sensor data, etc.). For example, faces or other features may be detected from the convolutional data generated by the convolutional operation as a result of the output of convolutional unit 212. Convolution processing may be performed on a set / segment / subset / part of image data, for example, along a portion of image 232 for specific features. Initially, the convolutional operations generated by convolutional unit 212 can extract low-level features. This can be achieved using a sliding window of a specific size (e.g., an n×n window for the convolution operation, where n is a positive integer, or n×p, where n and p are also different positive integers), where the sliding convolution is performed incrementally across the entire image 232 using overlapping data from image 232. Then, in subsequent passes or iterations, additional higher-level features can be extracted by combining the results of previous convolution operations performed by the same or different components for lower-level feature extraction. Iteration can then continue to generate data at different hierarchical levels using one or more different iterations or passes of the image.

[0033] For example, a face can have contours, diagonal lines, curves, etc. Low-level features (lines, slopes, curves, etc.) can be extracted from a set of convolutional data, for example, to determine the location of an ellipse in the image, features that look like the edge of an ear, or other features that constitute the basic structural data for a specific feature / pattern. This specific feature / pattern can be determined based on a combination of hierarchical datasets to determine whether there is part of a certain pattern or condition in the image 232, such as a person on the collision path of a car or other system including components of a convolutional neural network.

[0034] These convolutional operations can produce feature maps. A feature map can be considered as the output of a convolutional component 212 applied to the image 232 from the sensor or the output data of a previous computational layer. For example, a filter instance is drawn across its receptive field in the output of a previous layer, moving one or more pixels at a time. The receptive field can be referred to as a small region / subset of the image connected to the filter (filter kernel). For example, a filter can analyze the presence of triangles as a feature. For example, instead of a filter sliding across the entire image looking for the presence of triangles, various filters can analyze / manage various regions of the image for triangles or other specific features.

[0035] Convolution at each location of a filter within its receptive field may result in a measurement of the presence of a feature represented by the kernel at that location. The measurement of the presence of the feature at all convolution locations can be a feature map of the filter kernel. For example, if a convolutional filter slides one pixel at a time, the new convolutional field can overlap with the previous convolutional field by (field width - 1) of the input value. For instance, in a 32×32 image, dragging a 5×5 convolutional kernel over the input image data, with a stride width of one pixel, produces a feature map of size 28×28 (32-5+1×32-5+1) for each convolutional filter output. The terms “convolutional kernel,” “convolutional filter kernel,” “filter,” “kernel,” and “filter kernel” are used interchangeably throughout this document, with additional details provided below (e.g., see reference). Figure 3 Similarly, as used in this paper, the stride width or length can limit how much the sliding window shifts or how the filter performs convolution around the amount of input data.

[0036] After extracting the lowest-level information as low-level features, the output of convolutional information or a convolutional dataset can be obtained by combining the convolutional components 212 of convolutional layer 202, for example, by combining two edges / lines and analyzing, for example, at detectors 106 to 112, whether the combination corresponds to a nose or an eye. The combination of these low-level features (e.g., edges, lines, etc.) can also form a dataset of slightly standard features (e.g., medium-level) corresponding to conditions in image 232 (e.g., face, nose, ears, eyes, forehead, facial hair, etc.), with each set in the hierarchy of convolutional datasets including extracted data features of greater complexity. Furthermore, for example, the dataset can be continuously combined along the feature maps of the image and further processed along computational layers until the system is able to discern high-level features and identify a complete face as an object / high-level feature within the image.

[0037] In this way, low-level features (e.g., curves, edges, lines, slopes, etc.) can be one level of dataset hierarchy, and another level of convolutional datasets (e.g., combinations of shapes, patterns, etc.) can form another level, and any number of convolutional datasets can be generated based on convolutional operations in the hierarchy of datasets to determine higher-level patterns, including faces, people, animals, bicycles, motorcycles, signs, details of signs, or any number of situations (e.g., people in a collision path, animals entering a collision path, or other situations occurring in one or more images).

[0038] Since convolution operations can be linear, convolutional layer 202 does not fully and always reflect what might be considered real-world behavior, which typically does not follow a fixed path and is often random. Therefore, to generate convolutional outputs with more randomness, ReLU layer 204 can be configured as part of a CNN system / component 104 and further combined as part of combined convolutional layer 230. Convolutional component 212 can be connected to ReLU component 214 of the ReLU layer. ReLU component 214 can be configured to generate a set of non-linear convolutional output data by removing linearity from the convolutional data output from convolutional component 212 to the ReLU input. For example, ReLU component 214 can monitor inputs for negative values ​​that might occasionally be included in the convolutional output. Then, based on the convolutional dataset received as these inputs, a hierarchy of datasets similar to the non-linear convolutional output data is generated. In response to detecting any negative values, ReLU component 214 replaces them with zero or positive values. In this way, the linear output of convolutional layer 202 can be transformed into at least slightly or partially non-linear data as non-linear convolutional output data.

[0039] By extracting and linearly removing features, the pooling layer 206 at pooling component 216 receives non-linear convolutional output data to keep its scale invariant. Therefore, regardless of the location, lighting, or clutter of the image being analyzed, the CNN system can be intelligent and robust enough to detect variations in the same picture / image.

[0040] Therefore, convolutional layer 202 generates feature extraction. ReLU layer 204 removes linearity from the data to make them consistent with the real data. Pooling layer 206 is responsible for volume reduction and keeping the network image scale invariant, which can be compared to or similar to other image processing techniques such as Scale Invariant Feature Transform (SIFT) and Histogram of Oriented Gradients (HOG).

[0041] Pooling component 216 is configured to generate pooled outputs through pipelined processing in parallel with convolutional combination layer 230. In embodiments, pooling component 216 can initiate scale-invariant processing and perform statistical operations on a first set / subset of convolutional data or nonlinear convolutional output data. Thus, for example, two different computational layers can be pipelined and operate in parallel with each other's functions, or processed simultaneously as a sliding convolutional window outputs a portion or some of the overall convolutional data on image 232. Convolution / ReLU operations of convolutional layer 230, convolutional component 212, or ReLU component 214 operate as pipelined processing with pooling component 216. These pooling components 216 perform statistical operations on nonlinear convolutional output data based on pooling windows of subsets of the nonlinear convolutional output data.

[0042] For example, the pooling operation reduces the size of the input data (non-linear convolution output data) provided by the ReLU layer 204. When each subset of data is received, a single non-linear convolution output dataset can be processed, its size reduced at the pooling layer 206, and then classified at the fully connected (FC) layer 208, and finally transformed into a human-understandable form by the classifier 210, which will be further detailed below.

[0043] The FC (Full-Connected) network layer 208 of the Artificial Neural Network (ANN) represents the function of the human brain with neurons. For example, different values ​​(e.g., 1, 5, 60,000, and 0.25, or other sets of values) can appear at the input of the ANN to enable the determination of patterns or conditions present in image data (e.g., man, woman, person, etc.). If the FC component sees a different set of numbers, the system output will detect that it may be something else.

[0044] The output of pooling component 216 of pooling layer 206 can be fed into an artificial neural network, also known as FC layer 208, which further determines the most likely candidates contained in the image / picture. Therefore, these numbers, for example, as the output of the FC component, can then be converted into probabilities by content called output layer 210 and fed into various detectors (e.g., Figure 1 (e.g., pedestrian detector 106 or traffic sign detector 112) and take necessary actions at application 114 or 116 in response.

[0045] FC unit 218 forms an ANN with a neuronal structure formed by network functions 222, 226 and output functions 224, 228, based on the corresponding first FC layer (FC1) and second (subsequent) FC layer (FC2). The output of each neuron N1 to Nx and N4 to Ny may depend on the availability of input data (e.g., I1, I2) and input weights (e.g., w1, w2), rather than the fetched program counter and external instructions. This means that each FC unit 218 may include one or more data stream processors that operate without fetching instructions from memory, but rather process input data in response to the availability of input data at one or more inputs (e.g., I1, I2) to process the input data immediately / transiently by executing a specified function. The inputs (e.g., I1, I2) may be provided by the pooled output of pooling unit 216 or by previous FC units / layers (e.g., FC1, N1 to Nx) of data available at the FC2 unit inputs (e.g., I1, I2).

[0046] For example, dataflow processing can be contrasted with or different from von Neumann control flow processing / von Neumann machine / processor, which fetches instructions from a memory with a program counter to process data, rather than processing data solely based on the availability of certain data at the input. FC unit 218 can also individually receive the same multiple inputs (e.g., I1, I2) at each FC unit within a layer (e.g., FC1). For example, N1 through NX all receive inputs from each pooling unit 216, and these inputs (e.g., I1, I2) can be identical. Additionally, each second (or subsequent) layer of an FC unit (e.g., FC2, N4 through Ny) can also receive the same inputs (e.g., I1, I2) from previous layers of the unit (e.g., FC1, N1 through Nx, previous computation layers, or other computation layers), etc., across various FC layers 208 (e.g., FC1, FC2, etc.), and each FC layer 208 can be one or more different layers of the formed computation unit.

[0047] Furthermore, each network function of a neuron in the FC layer 208 or FC component 218 may include mathematical operations, such as the dot product of the inputs and weighted summation (as weight factors for each input). The inputs (e.g., I1, I2) may be the same, and the weights (e.g., w1, w2) may be different. Additionally, the sum of the dot product of the inputs (I1, I2) and the weights (e.g., w1, w2) of each input (I1, I2) may also be performed at each FC component. Different FC components 218 may have different weights (e.g., w1, w2), or alternatively, for each FC component receiving the same group / set / multiple inputs (e.g., I1, I2), different inputs (I1, I2) may have different weights (e.g., w1, w2). Alternatively, these weights may be similar to or different from each other, such that at least some FC components 218 may have the same weights at the input (e.g., w1, w2), while other FC components 218 may have different weights (e.g., w1, w2), where different values ​​are assigned to the corresponding input weights and arbitrary quantities used in the decomposition process.

[0048] Furthermore, the output functions 224, 228 of each FC component 218 may also include mathematical functions that operate on the results of network functions 222, 226. For example, the output functions may include trigonometric functions (e.g., tanh, tan, etc.) or other mathematical functions that operate on the weighted input and the results of summation or other operations performed on it.

[0049] Finally, output layer 210 may include a classifier function 220 that receives the output of the FC layer or the ANN output. For example, these classifier functions may include a SoftMax function, which can compress / squeeze the output of the FC layer 208 to values ​​ranging from 0 to 1. The output of classifier function 220 may include probabilities, where the most likely output class corresponding to the input data has the highest probability. For example, output layer 210 may also be a standard classifier, such as a Support Vector Machine (SVM) or other classifiers.

[0050] In embodiments of this disclosure, the execution of convolutional layer 230 and pooling layer 206 can be pipelined (by performing pipelined or piped processing), resulting in significant speedup and efficiency. As described above, pooling layer 208 can operate in parallel or simultaneously with convolutional layer 230 based on the availability of data from the first convolutional sliding window operation, and can begin processing simultaneously or during subsequent sliding window operations on overlapping or subsequent subsets of image data.

[0051] The execution of pipelined or pipelining processing may be referred to herein as a series of data processing elements, components, or functions connected in series, where the output of one component or element is the input of the next component or element. The pipelined elements or components may be executed in parallel or in a time-sliced ​​manner; in this case, a certain amount of buffer memory may be inserted between the elements. For example, instead of waiting for a previous component to fully complete its operation on a set of input data, a subsequent component or element can immediately begin processing a partial or incomplete set of results from the previous component or element. In this way, for example, a component of a pooling layer may be configured to operate on at least a portion of the results from the convolutional combination layer 230 as an accelerated process.

[0052] In other embodiments, the execution of FC layer 208 can also be pipelined, thus contributing to additional speedup and efficiency. Furthermore, FC components 218, as neurons, can also be pipelined to begin processing in response to partial output data from a previous or preceding FC component 218, so that they can operate in parallel with each other. For example, the network function 222 of a neuron in FC1 (e.g., N2) can begin processing before completing the output function 224 of the preceding FC component (e.g., N1), and in parallel with the processing operation of the output function 224 of the preceding FC component (e.g., N1) of FC1. Subsequently, FC components can also operate in parallel with each other through such pipelined processing operations.

[0053] Reference Figure 3 This illustrates an example of representative convolutional data result 300 according to various aspects / implementations of this document. Convolutional operations can be performed via the convolution component 212 of the convolutional layer 202 or as described above. Figure 2 The combined convolutional layer 230 with ReLU component 214 is utilized. For example, the convolution component 212 of the convolutional layer 230 can be configured to generate weighted multiplications and sums. In signal processing, these operations can be one-dimensional, and in the case of image processing, the convolution operation can be two-dimensional, allowing for various weights along the X and Y axes, with each weight weighting the input differently or similarly, representing an amount that is decomposed or considered in further processing.

[0054] In one example, the number of two-dimensional arrays 302 can be stored in memory represented by memory 304 (memory-1). These values ​​can represent regions of data or annotation values ​​provided by sensors (e.g., image / camera / video capture sensors, etc.) as input to one or more convolutional components 212. These values ​​302 can, for example, correspond to images or sub-parts / subsets of image data, and are representative here for illustrative purposes of potential values ​​for convolution operations performed by the components of the convolutional component or combination layer 230. For example, these values ​​may be real numbers, imaginary numbers, functions of one or more variables, or other representations of the data received or stored in memory 304. For example, data 306 can be similar to data 302, but slightly different in representation, showing linear storage of data to be stored in memory 304 as provided by sensor 102.

[0055] Convolution is a mathematical operation in which filter kernels can slide across an image, with the aim of detecting the presence of features represented by the filter kernels. In a CNN system, filter kernels can have fixed regions in image 302 to search for their features. Therefore, multiple instances of filters can be connected to non-overlapping regions of the image called local receptive fields. Each of the filter instances searches for the same exact feature, but only within its local receptive field in memory 304. Thus, it can be said that for each feature that must be searched, a corresponding set of filter kernel instances connected to the local receptive field is provided. The convolutional output of all filter kernel instances of a feature represents the feature map of that feature. Multiple filter kernels extend to other features intended to be detected.

[0056] For example, a representation of the nose, or an image of the nose within a specific filter kernel, can be placed on top of an image (captured by sensor data) at a certain location or subset for multiplying numbers below the filter kernel, and the sum of the multiplication results can be performed by the convolution unit 212 to obtain a specific value that is subsequently stored. The kernel can then slide a column along one direction (e.g., to the right). The computation is then repeated, where given mathematical operations such as dot product multiplication and summation can be used to repeatedly calculate the results to be stored.

[0057] These sliding operations can continue along the x-axis and then along a row to the end of the x-axis. The kernel can return to the first column, but slide down one row from where the operation is repeated again from the position of convolution unit 212. The overlapping image region 308 of image 302 represents the convolution window. It is on and along these convolution windows 308 where convolution is performed by convolution unit 212. On a data-parallel processor (e.g., a SIMD processor), the data belonging to the convolution window and the data belonging to the filter kernel can both be loaded into a vector register and fed to the vector multiplication-accumulation (MAC) unit. For example, a vector MAC operation can produce a convolution output. This convolution output can then be stored later in memory 2 (memory 310). After the filter kernel has completed convolution with all overlapping convolution windows 308, the complete convolution output can be obtained in convolution output memory 310.

[0058] According to the convolution process, hierarchical datasets can be generated in parallel or simultaneously with each other, and then nonlinearly processed by the ReLU component 214 to determine whether the image contains a picture of a nose and its location within the image. If the determination is affirmative, those parts of the image containing the nose will tend to have higher convolution values ​​in the output relative to other parts of the image.

[0059] In one example, under a 3×3 (or other n×n or n×p size, where n and p are positive integers that can be the same or different) convolutional sliding window 308, a set of smaller numbers (e.g., 1, 2, 3, 7, 8, 9, 2, 4, 5) can be derived, and then another sequence (e.g., 2, 3, 4, 8, 9, 1, 4, 5, 9) can be immediately generated. These are representative values ​​that can represent values ​​obtained from or derived from an image. The sequence of numbers (e.g., 1, 2, 3, 7, 8, 9, 2, 4, 5) can be derived from an initial region / subset of numbers as convolutional sliding window data. Then, by ignoring the first column and focusing on the numbers starting from the second, third, and fourth columns, another sequence (e.g., 2, 3, 4, 8, 9, 1, 4, 5, 9) can be obtained on the first, second, and third rows.

[0060] Further utilizing the 3×3 window 308, a small portion of the image data, 3×3 in size, can be analyzed based on the image data stored in memory 304. Other window sizes can also be utilized, and the example size is not necessarily exhaustive or restrictive of all aspects of this paper. Another sliding window can be formed by using another column of sliding windows, resulting in a window sequence of 3, 4, 5, 9, 1, 3, 5, 9, 3. Then, a dot product multiplication operation can be performed to obtain each number as the output of the sliding window convolution, which can be represented as AB CDEFGH in the lower right corner annotated by memory-2 (e.g., 310). For example, a value or representative result (e.g., A) can essentially represent the convolutional output of a filter kernel with a first window (containing 1, 2, 3, 7, 8, 9, 2, 4, 5), while B represents the convolutional output of a kernel representing the nose (or other features) with a second window containing 2, 3, 4, 8, 9, 1, 4, 5, 9, etc., where similarly, for example, H is the convolution of a filter kernel representing the nose (or other features) with a final window of 1, 3, 2, 9, 3, 5, 1, 1, 3.

[0061] A, B, C, D, E, F, G, and H can indicate the availability level of the nose in any region / subset / area of ​​the data corresponding to a given letter symbol, or as a level in hierarchical data. For example, if A is a very high value, there is a high chance that a nose (or other features corresponding to the kernel or sliding convolution window of the dataset) can be located at that position. Therefore, convolution can provide a true estimate of whether a feature is likely to be available in image data and where a feature might be located in the image data. Similarly, other features, or any number of features, can be analyzed separately by convolutional component 212 and combined computation layer 230 to detect one or more patterns or conditions within one or more images, which can be referred to as feature detection; for example, its output can be shown by convolutional output 310.

[0062] See Figure 4 This illustrates another example of the pooling operation in the CNN system 100. For example, the output of convolution 310 is here further expanded with an alphabet memory-2, where letters represent any value or function from a larger image. The pooling component 216 of pooling layer 206 processes the convolution output 310 by processing the data through a pruning, trimming, or compression process that reduces the output of convolution 310.

[0063] For example, a set of non-overlapping blocks can be represented in different shades or blocks drawn in memory-2 (memory 310). For example, blocks A, B, I, J are different from blocks / windows C, D, K, L, and blocks / windows C, D, K, L are different from blocks E, F, M, N and blocks G, H, O, P, etc. Therefore, Figure 2The pooling unit 216 can process the outputs of the convolution and sector non-overlapping regions 2×2 by performing statistical operations along both the x and y axes in each of these 2×2 units, so as to store the pooling outputs 402 in memory-3. These pooling outputs 402 represent scale-invariant features.

[0064] Although this paper shows and describes a 2×2 non-overlapping region as an example, other sizes can also be envisioned and utilized, allowing the use of p×n non-overlapping blocks / windows, where p or n can be any positive integer that is different from or the same as each other except for zero.

[0065] The statistical operations performed by the pooling component 216 may include minimum, maximum, mean, median, or other statistical operations on the sliding convolution window (e.g., A, B, I, j). The pooling component 216 can perform pooling operations on any statistical operation, but in most implementations, the statistical operation used by the pooling component may be a maximum value.

[0066] Therefore, based on each of these 2×2 block windows (or other block / window sizes) on the output of the sliding convolution 310, a value (e.g., representative B) can be selected. In this example, B has been selected, K has been selected from the next block, and subsequently E, P and Z, A, U, X, as stored in memory-3402, and pooling outputs as scale-invariant features or as volume-reduced outputs derived from the convolution output 310 from convolution unit 212 or the nonlinear convolution output dataset from ReLU unit 214 can be selected. Thus, pooling unit 216 can generate a reduction in data size from its received inputs based on the pooling operation. The output of the pooling operation can then be fed into an artificial neural network or FC layer 208, which further attempts to determine patterns / conditions within the image in order to make further decisions.

[0067] In various aspects, specific component hardware can be assigned to perform these computations, requiring all main processors (e.g., engine control units or other main processing units) to inform the convolutional neural network component hardware of: i) the existence of an image to be analyzed; and ii) an indication of the results after determining the content of the image. Thus, the computations of the individual layers of CNN 104 can be offloaded to the CNN accelerator, for example, to perform computations in the background to return the results to the main processor. While this may be a good solution in terms of power and performance efficiency, it can be costly because such hardware units can occupy a significant area on the microcontroller and consume a large amount of power for functionality. Therefore, the aspects presented in this paper also eliminate the need for dedicated hardware accelerators, instead reusing certain on-chip infrastructure by using one or more smaller, lighter components, rather than as densely as in dedicated CNN accelerators. Thus, costs can be reduced while maintaining performance without overloading the main processor.

[0068] In implementation, pooling processor 216 can be pipelined by operating in parallel with convolution unit 212 or in parallel with combined convolution + ReLU layer 230 and related processes. Instead of each layer of CNN 104 initiating its action only after all computational operations of the previous layer have been completed (e.g., all outputs of sliding convolution 310 are generated iteratively over the entire image for a certain feature level or subset of image parts), pooling unit 216 can initiate pooling operations on pooling windows once such windows have been generated by its predecessor (e.g., convolution unit 212), unlike typically waiting for all pooling windows to be generated by their predecessors.

[0069] For example, as a triggering event (e.g., also known as accessing the matching or observation (al) unit output), a data block can be available at the input used for processing to the next layer, and thus, the next layer immediately begins computation. Therefore, once the convolutional layers 202 / 212 have computed a block / window of the computed output (e.g., one of the shaded blocks A, B, I, J), the processing gain can be increased by initiating a pooling operation.

[0070] Instead of waiting for the convolutional layers to generate convolutional outputs A, B, C, D, E, F, G, etc., all the way up to F in the lower right corner of the output 310 of memory-2 to initiate pooling, the pooling unit 216 can process data based on availability in response to a triggering event or indication (or access match). For example, a triggering event could be writing to the location in memory-2 corresponding to the last value for a given window (e.g., J of the first shaded block of a 2×2 window). Thus, once an element, value, or any factor of the 2×2 block is written to memory or a specific memory location, the first pooling unit 216 can be triggered to immediately begin processing pooling operations on available data.

[0071] In another implementation / aspect, the trigger indication in this example may additionally or alternatively be the availability of the output of a sliding convolution corresponding to the last value, position, or result of a p×p window (e.g., a 2×2 non-overlapping data block), where p can be any positive integer. Furthermore, the trigger can be extended to include the availability of data corresponding to all inputs, such that once all inputs have received data, as in a data stream processing flow, regardless of whether the reception of the data itself is complete / finished, or whether a previous layer (e.g., combination layer 230, convolution unit 212, ReLU unit 214, etc.) has completed, processing in the next layer (e.g., via pooling unit 216) can be triggered.

[0072] In response to the availability of a value or indication of a write operation at a specific location, a pooling unit 216 can initiate a completed block or non-overlapping window (e.g., a 2×2 window of non-linear convolution output data or the output of a sliding convolution 310). The pooling unit 216 can perform operations simultaneously or in parallel with one or more convolution units 212. (See reference...) Figure 5 When convolution unit 212 provides sliding convolution output 310, pooling unit 216 can provide a final pooled output with output address 502 for each block (e.g., eight blocks J, L, N, P, Z, B, D, and F). Thus, for example, once J is obtained from convolution unit 216, the maximum values ​​of A, B, I, and J can be found as example statistical operations as part of the pooling operation of pooling unit 216, without waiting for other results from other windows or subsets of data within the same or different tomographic levels in the image, such as C, D, E, and F. Furthermore, once N is available, pooling operations for E, F, M, and N begin without waiting for O to be written to memory. Similarly, once P is available, for example, for G, H, O, and P, pooling calculations can be initiated for each block of output sliding convolution data, or also referred to as nonlinear output data 310.

[0073] Reference Figure 6 An example of an architectural ANN system or device 600 for operating CNN layers is shown, according to various aspects or implementations. System 600 includes a central processing unit (CPU) 604 coupled to each other via a processor interconnect 602, one or more memory extraction units 606, and a memory storage device 614. In one aspect, CPU 604 can be configured as a microcontroller unit (UC), compact uC, or digital signal processing (DSP) device with a sequential pipelined microarchitecture. For example, CPU 604 can be configured as a compact uC with an ordered, simple pipelined / single-issue microarchitecture.

[0074] CPU 604 is configured to facilitate the control and operation of components of system 600 and is directly coupled to memory storage device 614 and one or more memory retrieval units 606. Furthermore, CPU 604 is configured to process data with a predefined data structure. The predefined data structure is defined by CPU 604 and stored in a configuration register / memory (not shown) coupled to CPU 604. Additionally, memory storage device 614 is configured to store packaged / compressed data with another predefined data structure. This other predefined data structure of the packaged / compressed data stored on memory storage device 614 may, for example, differ from the predefined data structure of the data that CPU 604 is configured to process. In embodiments, the packaged / compressed data may correspond to a dataset (e.g., image, video data, waveform data, etc.) collected by sensor 102, which is provided as input to CNN 104. The packaged / compressed data may include packaged subword data whose storage location, layout, and packaging format can be programmed by CPU 604 via configuration registers / memory. Therefore, CPU 604 can facilitate the storage of packaged / compressed data on memory storage device 614 based on the dataset collected by sensor 102.

[0075] In some implementations, the packaged / compressed data includes stored elements (i.e., stored data or necessary data) stored in the memory storage device 614, and generated elements (i.e., generated data or non-necessary data) implicit in a specified packaging format of the packaged / compressed data. For example, generated elements may include padding data (i.e., padding at the edges) or elements skipped in sparse data.

[0076] In one embodiment, the memory decimation unit 606 and the CPU 604 may be operable to support or include the convolution unit 212 of the convolutional layer 202, the pooling unit 216 of the pooling layer 206, or the FC unit of the FC layer 208 for pipelined processing. For example, the CPU 604 includes arithmetic / logic instructions configured to perform processing associated with the convolution unit 212, the pooling unit 216, or the FC unit (e.g., multiplication / accumulation calculations, min / max calculations, etc.), and the memory decimation unit 606 is configured to provide the CPU 604 with output data suitable for processing for the aforementioned processing. In another embodiment, the memory decimation unit 606 may each be referred to as a memory decimation device.

[0077] Each memory extraction unit 606 is configured to determine the address of packaged / compressed data in the memory storage device 614 and provide corresponding output data to the CPU 604 based on the address of the packaged / compressed data and a predefined data structure. In an embodiment, the memory extraction unit 606 generates output data such that the output data is configured according to the predefined data structure. Therefore, the memory extraction unit 606 can unpack the packaged / compressed data and can decompress or (re)format the subsequently unpacked data in a manner suitable for processing by the sequential pipelined microarchitecture of the CPU 604. This reduces or eliminates the need for the CPU 604 to unpack or reformat the packaged / compressed data, thereby freeing up computational power that can be used to perform arithmetic / logical operations on the output data instead. This alleviates the performance bottleneck of unpacking and reformatizing packaged data and increases the speedup gain for performing ANN algorithms on the CPU 604.

[0078] In another embodiment, the stored data of the packaged / compressed data includes one or more redundant memory words, wherein the memory extraction unit 606 may each include a buffer (e.g., a circular buffer) configured to store one or more redundant memory words. The memory extraction unit 606 is configured to selectively extract stored data based on the redundant memory words stored in the buffer. In one aspect, the memory extraction unit 606 is configured to selectively extract redundant memory words of stored data from the memory device at a time.

[0079] Memory fetch units 606 may, for example, each include one or more coprocessors configured to perform the functions / processes of memory fetch units 606. Components of memory fetch units 606 are tightly coupled to CPU 604 or memory storage device 614, wherein memory storage device 614 includes instruction memory and data memory. For example, memory fetch units 606 may be attached as coprocessors directly coupled to the core of CPU 604, wherein CPU 604 includes special function instructions, special function registers (e.g., IFRs in RISC-V), local memory, bus agents, etc., whose resources can be directly accessed or shared by memory fetch units 606. Furthermore, CPU 604 may include an instruction fetching device configured to fetch fetched data (e.g., instruction data stored in instruction memory) from memory storage device 614. In embodiments, memory fetch units 606 are each configured to share resources with the instruction fetching device of CPU 604 and may be configured to fetch packed / compressed data at least partially from the instruction memory of memory storage device 614. By leveraging the shared resources between the memory extraction unit 606 and the CPU 604, each memory extraction unit 606 can access memory via the CPU / coprocessor memory interface, enabling the use of high-performance CPU memory ports (e.g., dedicated program memory, dedicated instruction memory, or temporary data memory). This further alleviates the performance bottleneck of data aggregation and unpacking, thereby further increasing the speedup gain for performing the ANN algorithm on the CPU 604. Therefore, in one embodiment, the memory extraction unit 606 accesses or performs memory extraction operations on the memory storage device 614 via the CPU 604 (e.g., the memory extraction unit 606 accesses memory via the CPU 606 through the coprocessor interface). In another embodiment, each memory extraction unit 606 can be configured to have a memory management unit (MMU) configured by the MMU special function registers (SFRs), wherein the output data unpacked / aggregated by the MMU is mapped to a virtual address by the MMU. Furthermore, each memory extraction unit 606 can be configured to share or include MMU resources (e.g., cache, bus proxy, high-bandwidth CPU interface, etc.). In another embodiment, the memory extraction unit 606 can be coupled to the bus of the CPU 604 as a peripheral device using streaming direct memory access (DMA), wherein the memory extraction unit 606 can read data from the memory storage device 614 via DMA and includes programmable input / output (I / O). In yet another embodiment, the memory extraction unit 606 is integrated within the CPU 604, i.e., the memory extraction unit 606 is part of the CPU 604.

[0080] In one implementation, CPU 604 includes an arithmetic logic unit (ALU) configured to perform ALU processing. In such an implementation, the ALU is configured to perform parallel execution of processes associated with, for example, convolution unit 212, pooling unit 216, or FC unit (e.g., multiplication / accumulation calculations, min / max calculations, etc.). Memory decimation units 606 are each configured to perform decimation, unpacking, decompression, or decoding processes on packed / compressed data simultaneously with the ALU operations performed on CPU 604, thereby accelerating ANN computation without requiring complex SMID expansion. For example, memory decimation units 606 are configured to generate new output data for CPU 604 while CPU 604 performs ALU processing on previously output data provided by memory decimation units 606.

[0081] In each implementation, the memory extraction unit 606 includes an extraction address generator unit 608, a sparse aggregation unit 610, and a data unpacking unit 612. The extraction address generator unit 608 is configured to determine or generate address and subword data item selection information for packaged / compressed data stored in the memory storage device 614. The addresses generated by the extraction address generator unit 608 include a first set of addresses associated with storage elements in the packaged / compressed data and a second set of addresses associated with generated elements in the packaged / compressed data. Furthermore, the sparse aggregation unit 610 is configured to selectively extract stored packaged / compressed data from the memory storage device 614 while ignoring (i.e., not extracting) the packaged / compressed data generated based on the address and subword data item selection information from the extraction address generator unit 608. By ignoring the generated packaged / compressed data, the number of read / extraction operations performed on the memory storage device 614 can be reduced, thereby reducing the time required to extract the packaged / compressed data. In some implementations, the stored packaged / compressed data corresponds to a first set of addresses associated with the stored elements, and the generated packaged / compressed data corresponds to a second set of addresses associated with the generated elements. Furthermore, the data unpacking unit 612 is configured to generate output data based on the stored packaged / compressed data extracted from the memory storage device 614 and a predefined data structure. The output data is configured according to the predefined data structure and subsequently streamed to the CPU 604 for data processing, or stored in an output register that can be extracted by the CPU 604 at a later time. In other implementations, the memory extraction unit 606 can extract, unpack, and format new packaged / compressed data from the memory storage device 614 simultaneously with the CPU 604 processing the output data from the output register. This can further increase the speedup gain for performing ANN algorithms on the system 600.

[0082] Reference Figure 7This illustrates various aspects or implementation methods. Figure 6 Detailed examples of the components of the memory extraction unit 606 are provided. The memory extraction unit 606 includes a memory extraction bus unit 704, an extraction address generator unit 608, a sparse aggregation unit 610, a data unpacking unit 612, a configuration unit 706, and a register interface unit 708, all interconnected via an extraction unit interconnect 702. In one aspect, the extraction unit interconnect 702 may be coupled to a processor interconnect 602. In an embodiment, the memory extraction unit 606 accesses the memory storage device 614 or performs memory read operations via a CPU 604 (e.g., the memory extraction unit 606 accesses the memory via a coprocessor interface through the CPU 606).

[0083] The memory access bus unit 704 is coupled to the CPU 604 and the memory storage device 614, and is configured to perform memory read operations on the memory storage device 614. The configuration unit 706 is configured to provide configuration parameters to other components in the memory access unit 606. For example, the configuration parameters may include memory location, layout, and packing format information of the packed subword data stored on the memory storage device 614.

[0084] The decimation address generator unit 608 is configured to generate a stream of word addresses and subword data item selections using layout and packing format information of the packed / compressed data. In one implementation, the generated word addresses and subword data item selections correspond to the sequence of input data required for implementing the inner loop of an ANN operation on the convolution unit 212 of the convolutional layer 202 or on the FC unit of the FC layer 208 (e.g., scanning kernels on a short-axis array of inputs for convolution operations or on a single neuron in a dense layer). For example, the decimation address generator unit 608 may be configured to generate a stream of word addresses and subword data item selections for short-axis iterations of two-dimensional convolution operations on a dense two-dimensional array of compressed subword data of a fixed size. In such an implementation, word addresses and subword data item selections can be generated for a data stream corresponding to the kernel “trace” of a two-dimensional kernel filter sliding along the short axis of the data array. Therefore, the address generator unit 608 includes kernel convolution parameters that can be used to perform convolution operations (e.g., on CPU 604), such as row stride, column stride, kernel filter width, kernel filter height, etc. Furthermore, the stream for generating word addresses (and corresponding sub-word selections) for one-dimensional input / kernel filters can be implemented as discussed above regarding two-dimensional kernel filters, where the kernel filter height is set to 1 and the kernel filter width is equal to the size of the one-dimensional input data.

[0085] The subword data item selection information includes the number and location of packed data items associated with the address of a memory word (i.e., the number and location of subword items); and a suppression indicator for packed data items whose extraction should be suppressed and replaced with zero / null values ​​in subsequent processing. Therefore, the subword data item selection information indicates a first set of addresses associated with storage elements (e.g., packed data items associated with the addressed memory word) in the packed data stored on the memory storage device 614, and a second set of addresses associated with elements generated in the packed data (e.g., packed data items whose extraction should be suppressed and replaced with zero / null values ​​in subsequent processing). Furthermore, the subword data item selection information includes a padding mask corresponding to whether a packed data item will be extracted or ignored. Therefore, the padding mask can be used as a suppression indicator for packed data items whose extraction should be suppressed and replaced with zero / null values ​​in subsequent processing. For example, a padding mask containing all "0"s (e.g., a 4-bit data structure containing 0, 0, 0, 0) indicates that the extraction of the corresponding packed data item should be suppressed in subsequent processing steps.

[0086] The generated elements associated with the second address set can correspond to zero "padding" of the input to the convolutional layer, thus exhibiting suppressed decimation, as indicated by the subword data item selection information. Therefore, the input padding of the convolutional layer's input is defined according to a "virtual" (i.e., physically non-existent) memory location that allows uniform processing of all kernel footprint positions. By suppressing the decimation of generated elements from memory storage device 614, the number of read / decimation operations performed by memory decimation unit 606 is reduced, thereby reducing the time required to decimate packaged data items in subsequent processing steps. In some embodiments, the generated elements are left-aligned to the input array if the short axis of the convolutional layer's input array is word-aligned with the left padding corresponding to a row position with coordinates less than 0 (high msb). By positioning the generated elements of the convolutional layer's input array to the left of the input array, the operation of memory decimation unit 606 can be efficiently implemented while simultaneously sliding the filter kernel across the input array during convolution.

[0087] By specifying the packaged data items to be extracted based on the word address and sub-word item position, packing formats with packaged data items narrower than a byte or not aligned with the byte address can be supported. The optimal bit width for quantizing ANN weights and activations depends on the application, thus this flexibility in choosing the width of the packaged data items increases the flexibility of the memory extraction unit 606. However, in some implementations, a packing format that segments packaged data items across word boundaries can disproportionately complicate the implementation, and the trade-off gains diminish with increasing bit width. Therefore, packaged data items with a word-aligned layout without excessive unused bits alleviate implementation complexity and reduce the trade-off gains. For example, if the memory storage device 614 is configured to store a 32-bit word, the 32-bit word includes 1, 2, 3, 4, 5, 6, 8, 10, 16, and 32-bit data items, where 4, 6, 8, 10, and 16-bit items can optimize the extraction of packaged data items from the memory storage device 614. Furthermore, it should be understood that the memory storage device 614 storing words with other bit values ​​is within the scope of this disclosure.

[0088] In one implementation, CPU 604 can reprogram the format information of the packed / compressed data, allowing the decimation address generator unit 608 to change between different packing formats within a single kernel convolution (e.g., different quantization parameters for different channels in a convolutional network layer). Alternatively, a format information indicator can be coupled to the decimation address generator 608 and can facilitate the programming of multiple parameter sets corresponding to different packing formats of the packed / compressed data, such that changing the format information indicator can facilitate rapid switching between different packing formats of the current packed / compressed dataset being processed by the decimation address generator 608.

[0089] In one implementation, the memory extraction unit 606 is referred to as a data stream device, which is configured to process data stored on the memory storage device 614 and provide a continuous output stream to the CPU 604. Furthermore, the memory extraction unit 606 is configured to provide a continuous output stream simultaneously with the CPU 604 processing the output stream. On one hand, the extraction address generator unit 608 may be referred to as an address generator component, which is configured to generate multiple data addresses for the data stored on the memory storage device 614. Furthermore, the extraction address generator unit 608 is configured to generate a suppression indicator (e.g., a padding mask) for each of the multiple data addresses, and wherein selected data is introduced based on the suppression indicator. The suppression indicator is associated with data to be replaced by the generated data (e.g., bit padding) according to a predefined data structure.

[0090] The sparse aggregation unit 610 is configured to selectively extract stored packaged data items from the memory storage device 614 while ignoring (i.e., not extracting) generated packaged data items based on word addresses and subword data item selection information from the extraction address generator unit 608. The extracted stored packaged data items are then transmitted to the data unpacking unit 612 for decompression, unpacking, or reformatting. In one aspect, the generated packaged data items may be implicitly included in a predefined format, such as skipped elements in sparse data or zero padding at edges. In an embodiment, the packaged data items may be or include sparse data such that the subword data item selection information includes a corresponding sparse mapping bit vector. The sparse mapping bit vector may be generated by the extraction address generator unit 608 or may be mapped in memory within the memory storage device 614. The sparse mapping bit vector can be used to identify stored packaged data items that must be extracted from the memory storage device 614 and to generate packaged data items whose extraction is suppressed.

[0091] In one embodiment, the sparse clustering unit 610 may be referred to as a data extraction component, which is configured to import selected data from memory storage device 614 based on word addresses. In one aspect, the sparse clustering unit 610 is also configured to import the selected data at least partially from the instruction memory of CPU 604, and is further configured to share resources with the instruction extraction means of CPU 604. Furthermore, the sparse clustering unit 610 is configured to skip data based on predefined control information or data extracted from memory, wherein the control information may include word data item selection information, information related to a predefined format of the imported selected data (e.g., skipped elements in the sparse data or zero padding at edges), a sparse mapping bit vector corresponding to the imported selected data, etc. In other embodiments, the sparse clustering unit 610 is configured to decompress a continuous sequence of variable-length encoded data that may be associated with the imported selected data. For example, the variable-length encoded data may include Huffman-coded data. In this implementation, the selected data includes multiple data items, and the sparse aggregation unit 610 is configured to transmit multiple data items in parallel with the data unpacking unit 612, which is supplying a continuous output stream to the CPU 604.

[0092] In some implementations, the packaged data items on memory storage device 614 may be or include dense data, which includes a corresponding "all-one" bit vector indicating the presence or absence of data. To improve efficiency, the all-one bit vector may be generated internally in sparse clustering unit 610 or may be extracted from memory. To further improve efficiency and reduce the total time required to complete the decompression, unpacking, or reformatting of packaged data, multiple stored packaged data items may be transmitted to data unpacking unit 612 in parallel. Furthermore, the bus interface between sparse clustering unit 610 and data unpacking unit 612 may be configured to support multi-word burst transmission. In such implementations, sparse clustering unit 610 may transmit stored packaged data items to a read-ahead cache (e.g., a small 4- to 8-entry cache unit) directly coupled to data unpacking unit 612. This further improves the efficiency of memory extraction unit 606. In some implementations, sparse clustering unit 610 is configured to selectively extract new stored packaged data items from memory storage device 614 simultaneously with data unpacking unit 612 decompressing, unpacking, or reformatting stored packaged data items. In such an implementation, newly stored packaged data items can be stored in a read-ahead cache, while the unpacking unit 612 receives / processes the stored packaged data items from the read-ahead cache.

[0093] In some implementations, the sparse aggregation unit 610 is configured to selectively extract stored sparsely packed data items from memory using a generated word address, a suppression indicator (e.g., a padding mask), and a sparse mapping bit vector. The sparse mapping bit vector may be generated by the data extraction unit 606 or mapped to memory. Sparsely packed data items marked as non-existent by the sparse mapping bit vector can be programmed as part of the stream parameters output from the sparse aggregation unit 610 to the data unpacking unit 612. For example, sparsely packed data items marked as non-existent can be skipped (e.g., omitted from the output parameter stream) or replaced with programmable zero / null values, thereby further improving computation and unpacking efficiency. In one implementation, by packing non-zero weights (e.g., kernel weights) and using a corresponding sparse mapping bit vector for skipping input data corresponding to zero weights, zero-weighted products used to calculate weighted sums can be skipped, thereby improving the processing efficiency of the ANN algorithm on CPU 604.

[0094] The circular buffer buffer 710 may be part of the sparse aggregation unit 610 or may be coupled to the sparse aggregation unit 610 via the decimation unit interconnect 702. For example, the circular buffer buffer 710 may be directly attached / directly electrically coupled to the sparse aggregation unit 610. The circular buffer buffer 710 is configured to store memory words (e.g., stored packaged data items) decimated by the sparse aggregation unit 610 based on word addresses provided by the decimation generator unit 608. As the decimation generator unit 608 provides a new word address to the sparse aggregation unit 610, the memory word associated with the new word address may correspond to a memory word already stored on the circular buffer buffer 710. Therefore, the memory word associated with the new word address is a duplicate word. In such an implementation, the sparse aggregation unit 610 may suppress the decimation of duplicate words from the memory storage device 614 associated with the new word address. Furthermore, the sparse aggregation unit 610 may read stored packaged data items associated with duplicate words from the memory words stored on the circular buffer buffer 710. This reduces or eliminates redundant decimation performed on memory storage device 614, thereby improving the decimation efficiency of sparse clustering unit 610. In an implementation, the repeated word may be referred to as a redundant memory word.

[0095] For example, a convolution operation could include iterating over the minor axis of a dense two-dimensional array with a minor axis stride of 1, thereby causing a sequential kernel footprint over the data array (see, for example, [link to previous section]). Figure 8The word address stream for sequential kernel footprints generated by the extraction generator unit 608 is deterministic and highly repetitive. This is because adjacent kernel footprints overlap, such that the memory packaged data items corresponding to the word address stream may include one or more duplicate words. The circular buffer buffer 710 is configured to eliminate redundant extraction of one or more duplicate words from the memory storage device 614. In an implementation, a last-in-first-out (LIFO) order is achieved in the circular buffer buffer 710 if the buffer order of the circular buffer buffer 710 is transposed and the minor axis is reversed relative to the order of the generated word addresses. For example, the kernel memory word address generated for the kernel footprint position corresponding to the kernel footprint of the largest W×H word in row-minor order (where W and H are positive integers that may be the same or different) can be represented as: [(0,0), (1,0), ..., (W-1,0), (0,1), ..., (W-1,1), ..., (0,H-1), ..., (W-1,H1-1)]. Furthermore, after extracting the memory word based on the word address corresponding to the first position of the kernel footprint, the extracted memory word can be stored on a circular buffer register 710, such that the contents of the circular buffer register 710 are represented as: […, (W-1, 0), W-1, 1), …, (W-1, H-1), (W-2, 0), …, (W-2, H-1), …, (0, 0), …, (0, H-1)]. Additionally, the circular buffer register 710 includes a B-word circular buffer memory segment (where B is a positive integer, such as 8, 16, 32, or other suitable value) and a current end position e, such that the value of the kernel memory word (x, y) (where x and y are zero or can be the same or different positive integers) will be cached at the address of the circular buffer register 710, as in the equation |e+(y, H… x )| B This indicates that each time the top-left kernel footprint is shifted to an adjacent kernel footprint position, a column of memory words is added and d columns of memory words are deleted (where a and d are zero or can be the same or different positive integers), and e is incremented by dH (the number of words no longer in the current kernel footprint). In such an implementation, memory words from column [0, H-1-a] can be retrieved from the circular buffer buffer 710, while new memory words corresponding to the new column [Ha, H-1] can be retrieved from the memory storage device 614 and copied to the circular buffer buffer 710.

[0096] In one implementation, the circular buffer cache 710 can be used for kernels with kernel footprints exceeding the cache size by caching only a suitable subset of values. For example, this can be achieved by setting W to less than the actual maximum number of kernel footprint columns and extracting kernel locations (x, y) from memory where W-1-x is less than zero. In other implementations, the multiplication performed to compute the location of the kernel word cached in the circular buffer cache 710 may not require a large multiplier block. For example, for compact uC, an ANN algorithm with a sufficiently small parameter set may be sufficient with an 8-bit kernel footprint dimension (H, W). Furthermore, for a one-dimensional kernel filter (where W can be large), H is 1, allowing the multipliers used to determine the location of the kernel word cached in the circular buffer cache 710 to be omitted, thereby improving processing efficiency. In an alternative implementation, the location of the kernel word cached in the circular buffer can be determined iteratively by adding / subtracting the H*(W-1) value pre-computed by the CPU 604.

[0097] Furthermore, the data unpacking unit 612 is configured to generate output data based on stored packaged data items selectively extracted by the sparse aggregation unit 610. In embodiments, the data unpacking unit 612 may decompress, unpack, or format the stored packaged data items into a form suitable for the CPU 604 to perform arithmetic or logical operations on the corresponding output data. In one aspect, the output data generated by the data unpacking unit 612 is configured to match at least one of a predefined precision, a predefined layout, or a predefined data structure. For example, the predefined precision, predefined layout, or predefined data structure may each include information including the data size, data format, data offset value, or data shift value of the generated output data. In some embodiments, the predefined data structure may be or include a padded dense 2D array, a padded 1D data vector, an expanded padded 2D array and an expanded padded 1D vector, a 1D vector with skipped elements according to the corresponding sparse mapping (i.e., a sparsely mapped bit vector), etc. The register interface unit 708 may include one or more output registers configured to store the output data generated by the data unpacking unit 612. Therefore, the output data can be extracted from the output register by the CPU 604 for subsequent processing steps. Furthermore, according to a predefined data structure, the output data can be configured with signed / zero extension to support a specified word width and repackaging structure defined by the CPU 604 to match the CPU 604's SIMD data format. Additionally, the data unpacking unit 612 is configured to decode the stored packed data items based on at least one of offset values, shift values, scaling, rounding, lookup tables, or variable-length encoding (e.g., Huffman coding), each of which can be stored in memory. In some embodiments, the stored packed data items can be encoded using a scaling / offset quantization scheme, such that decoding the corresponding stored packed data item based on the offset value or shift value reduces the time required to perform decoding. Furthermore, for example, the stored packed data items can be encoded using a non-linear quantization scheme (e.g., logarithmic encoding for statistical clustering), such that decoding the corresponding stored packed data item based on a lookup table reduces the time required to perform decoding. This reduces the time required to generate output data and further improves the power efficiency of the memory extraction unit 606. In some implementations, the packaged data items stored may not be formatted before being stored on the memory storage device 614, so that the data unpacking unit 612 does not perform formatting (e.g., does not perform decompression or compression). This can improve the efficiency of the memory extraction unit 606.

[0098] On one hand, the data unpacking unit 612 can be referred to as a data formatting component, which is configured to generate output data as a continuous stream to the CPU 604. Furthermore, the data unpacking unit 612 is configured to provide a data format and a data instruction set of the CPU 604 that match the stored packaged data items, wherein the data format and data instruction set are provided by the CPU 604 or the memory storage device 614.

[0099] To efficiently transfer output data to the CPU 604, a hardware / software (HW / SW) interface can be implemented between components of the memory fetch unit 606 (e.g., the output register, the fetch address generator unit 608, the data unpacking unit 612, etc.) and the CPU 604. In such an implementation, an observation unit can be coupled between the CPU 604 and components of the memory fetch unit 606. Once the observation unit determines that the CPU 604 fetches an unpacked data item from the output data stored in the output register, the observation unit can detect this event and provide a trigger via the HW / SW interface to the data unpacking unit 612 or the fetch address generator unit 608 to initiate the unpacking of the next packed data item, switch to the next line of the core, or perform other specified actions in parallel with the CPU 604's processing of the fetched unpacked data item. For example, the output data can be presented on the output register at multiple register addresses, so that once the CPU 604 fetches output data from a specific address in the output register, the observation unit can detect which address is accessed and trigger a specific action based on the corresponding specific address. Furthermore, by pausing CPU 604 in a wait state instead of requiring programmed polling of state information, CPU 604 decimation can be synchronized with the availability of newly provided output data from sparse aggregation unit 610. This reduces processor wait time that would otherwise be available for polling state information, thereby reducing processor wait time and further improving power efficiency during ANN computation.

[0100] Interconnect 702 may include interconnects (e.g., on-chip bus interconnects) that connect various components of memory extraction unit 606 to each other. Memory extraction bus unit 704 may be configured as a memory extraction bus master or as a uC coprocessor memory interface such that components of memory extraction unit 606 are tightly coupled to CPU 604 or memory storage device 614 via memory extraction bus unit 704. Alternatively, for example, memory extraction bus unit 704 may be used by extraction address generator unit 608 to extract data from memory and store data back to memory (e.g., such as ring buffer cache 710 or another suitable memory). Register interface unit 708 may be configured as a configuration and output bus slave or as a uC coprocessor register interface such that components of memory extraction unit are tightly coupled to registers / memory of memory storage device 614 or CPU 604. Furthermore, for example, register interface unit 708 may be used by data unpacking unit 612 to program registers / memory of memory storage device 614 or CPU 604.

[0101] The components of memory extraction unit 606 can be pipelined relative to each other or relative to CPU 604, resulting in significant speedup gains by making the data stream processor an output sector of each component immediately available. In some embodiments, the components of memory extraction unit 606 can be configured as a simple sequential state machine or as a deeply pipelined parallel implementation, where parallel processing of multiple address / extraction / data items is possible. In embodiments, the components of memory extraction unit 606 are configured to provide throughput commensurate with the capacity of CPU 604 for processing extracted and unpacked data items. This improves the power / area efficiency of compact uC devices. For example, for compact uC devices without SIMD capabilities, the components of memory extraction unit 606 configured as a fully sequential state machine will facilitate throughput commensurate with the capacity of the compact uC device for processing corresponding output data, while minimizing power consumption and increasing cost savings. In another example, for high-performance uC devices (e.g., higher clock speeds relative to memory throughput and SIMD processing), pipelined components of memory extraction unit 606, capable of delivering one or more output data items per CPU clock cycle, will facilitate throughput commensurate with the capacity of the high-performance uC device. This can allow for the generation of unpacked data items from an extracted memory word and subsequent output for data processing while simultaneously performing memory reads to extract subsequent packed data items, further contributing to additional speedup gains. Furthermore, pipelined components of memory extraction unit 606 can allow for increased stream aggregation / unpacking efficiency when performing channeled convolution operations (e.g., pooling, depthwise separable convolution) using a row-secondary (CHW) memory layout that avoids the need for memory storage of partial results, regardless of the layout, while storing pruned parameter sets (e.g., sparse data). In one aspect, memory extraction unit 606 may include a small buffer memory (e.g., a circular buffer cache) that can be directly managed by the pipeline, thereby achieving efficient caching performance.

[0102] Reference Figure 8 This illustrates an example of a memory layout for an input array used to illustrate a two-dimensional convolution process for packaged data items to be extracted from adjacent kernel footprints 802 to 808, according to various aspects / implementations of this document. The convolution operation can be utilized by the convolution component 212 of the convolutional layer 202, or as described above. Figure 2The combined convolutional layer 230 with ReLU component 214 is described for use. For example, the convolution component 212 of the convolutional layer 230 can be configured to generate weighted multiplication and summation. In signal processing, these operations can be one-dimensional, and in the case of image processing, the convolution operation can be two-dimensional, allowing for various weights along the X and Y axes, which weight the input differently or similarly, and for each of these weights, the amount that is decomposed or considered in further processing.

[0103] In one implementation, the packaged data item may be or include a two-dimensional array of values ​​stored in a memory storage device 614 with a corresponding memory address 810. These values ​​may represent regions of packaged data or packaged annotation values ​​that can be provided as input to one or more convolutional units 212 after being extracted and unpacked from memory. The values ​​in the two-dimensional array may have been captured by sensor 102 (e.g., an image / camera / video capture sensor, etc.) and subsequently stored in memory storage device 614 as packaged data items. Values ​​accessible at memory address 810 may, for example, correspond to a sub-part / subset of an image or image data and may be used by components of convolutional unit 212 or combination layer 230 for convolution operations. Furthermore, the packaged data item may be divided into four sub-word data items with corresponding addresses A', B', C', D', E', F', G', H', and I', respectively. For example, address A' can represent a first memory word comprising four sub-word data items with starting coordinates (-4, 0) in a two-dimensional array value, and address B' can represent a second memory word comprising four sub-word data items with starting coordinates (1, -4) in a two-dimensional array value, and so on. Furthermore, the first sub-word data item of the first memory word can be accessed at address A'[0], the second sub-word data item of the first memory word can be accessed at address A'[1], and so on.

[0104] As referenced above Figure 3 As described above, convolution is a mathematical operation in which a filter kernel can slide across an image, with the aim of detecting the presence of features represented by the filter kernel. For example... Figure 8As shown, multiple core footprints 802 to 808 are each located on a fixed region of memory address 810. For example, each core footprint 802 to 808 may have a 3×3 core and the filter core may have a step size of 1. Core footprints 802 to 808 illustrate locations / regions where filter cores can be placed to detect the presence of features in data items stored at memory address 810. For example, the filter cores may be configured to slide from the first core footprint 802 (e.g., from left to right) to the second core footprint 804, from the second core footprint 804 (e.g., from left to right) to the third core footprint 806, and subsequently from the third core footprint 806 (e.g., from left to right) to the fourth core footprint 808. Furthermore, word memory layout 812 illustrates an example layout of sub-word data items stored at corresponding memory addresses 810. The “virtual” word address corresponds to the implicit zero-padding of the two-dimensional input array, and the word address may correspond to a stored packed sub-word data item. Implicit zero-filling (no memory overhead) can be achieved by using virtual word addresses (whose extraction is suppressed later), while maintaining, for example, a simple control state machine.

[0105] Reference Figure 9 Table 900 illustrates examples of representative address generation and sparse aggregation input / output based on a sparse kernel 902 according to various aspects / implementations of this document. The sparse kernel 902 may be stored in memory as a packing vector 904 (e.g., containing W0, W2, W5, W8) with a decimation / skip vector 905 (e.g., containing 1, 0, 1, 0, 0, 1, 0, 0, 1). The sparse kernel 902 includes word addresses W0, W1, W2, W3, W4, W5, W6, W7, and W8 corresponding to sparse data. The word addresses in the sparse kernel 902 include a first set of word addresses corresponding to stored packing data items (e.g., word addresses in packing vector 904) and a second set of word addresses (e.g., containing W1, W3, W4, W6, W7) corresponding to elements in the sparse data that may be skipped (i.e., not decimated). The decimation / skip vector 905 serves as an indicator of which values ​​in the word addresses should be skipped and which values ​​should be decimated for processing (e.g., for convolution). The decimation / skip vector 905 can be configured as, or referred to as, a sparsely mapped bit vector associated with the sparse kernel 902. In other embodiments, the sparse kernel 902 can be configured as a one-dimensional extended input kernel, a two-dimensional extended input kernel, etc. The word addresses W0, W2, W5, and W8 of the packing vector 904 can correspond to... Figure 8 The kernel footprint positions are 802 to 808, where the packed data items at the corresponding word addresses W0, W2, W5 and W8 can be used for subsequent convolution processing performed on CPU 604.

[0106] Table 900 includes multiple columns 906 to 928, which correspond to the input / output values ​​of the decimation address generator unit 608 and the sparse aggregation unit 610, and simultaneously process the input of the sparse kernel 902. The decimation address generator unit 608 includes five columns from Table 900: kernel item coordinate column 906, decimation word address column 908, first item number column 910, item quantity column 912, and padding mask column 914. The kernel item coordinate column 906 includes the top-left coordinates of the starting position corresponding to the kernel footprint location. For example, kernel item coordinates (-1, 0) can correspond to the top-left coordinate of the first kernel footprint 802, kernel item coordinates (0, 0) can correspond to the top-left coordinate of the second kernel footprint 804, and so on. Extraction address generator unit 608 is configured to generate a word address stream (e.g., extraction word address column 908) and sub-word data item selection (e.g., first item number column 910, item number column 912, and fill mask column 914) for each core item coordinate in column 906.

[0107] Extracted word address column 908 includes extracted word addresses corresponding to at least one segment of word address within the window corresponding to the core footprint location. For example, the first core footprint 802 is at least partially located on the segment of word addresses A' to F' that corresponds to the extracted word address of the first core footprint 802, the second core footprint 804 is at least partially located on the segment of word addresses D' to F' that corresponds to the extracted word address of the second core footprint 804, and so on.

[0108] The first item number column 910 indicates the position of the first sub-word data item within the window of the kernel footprint position for each extracted word address in column 908. The item quantity column 912 includes the number of sub-word data items within the window of the kernel footprint position for each extracted word address in column 908. Furthermore, the padding mask column 914 indicates the padding mask for each extracted word address in column 908. In this embodiment, the padding mask serves as a suppression indicator regarding whether to extract or ignore the packed sub-word data item at the corresponding extracted word address. The padding mask can be determined based on the coordinates of the packed sub-word data item stored at the corresponding extracted word address. This is because padding or skipping data elements are implicitly included in the generation element of the packed data stored at the extracted word address. Furthermore, a padding mask including all "0"s indicates that the extraction of the corresponding packed sub-word data item should be suppressed by the sparse aggregation unit 610.

[0109] The cache of memory retrieval unit 606, such as a circular buffer cache 710, includes three columns as shown in Table 900: an add / delete column 916, a cache coordinate column 918, and a cache address column 920. The add / delete column 916 corresponds to the column of packed memory words added to or removed from the cache during the corresponding core footprint position. The number of columns added to or removed from the cache can be determined based on how the left and right "edges" of the core footprint cross word boundaries as the core moves from a core footprint position to an adjacent footprint position. For example, at the first core footprint position 802, two columns of packed memory words are added to the cache based on the corresponding retrieval word addresses (e.g., packed memory words corresponding to memory addresses A' to C' and packed memory words corresponding to memory addresses D' to F'). Furthermore, when the core slides from the first core footprint position 802 to the second core footprint position 804, the left-hand edge of the core crosses a word boundary (e.g., between column -1 and column 0), which causes the cache to delete one column of packed memory words (corresponding to memory addresses A' to C'). This process can be repeated until all core footprint locations are used up. Furthermore, if the right edge of a core crosses a word boundary, a column of packed memory words can be added to the cache. Cache coordinate column 918 includes the addresses of the core memory words generated for the core footprint locations, where the maximum trace word dimension is represented by (W, H) (where W and H are positive integers that can be the same or different). Additionally, cache address column 920 includes the addresses in the cache corresponding to the locations of the corresponding core memory words.

[0110] The sparse aggregation unit 610 of the memory decimation unit 606 includes four columns as shown in Table 900: a remaining decimation / skip column 922, a word-aligned decimation / skip column 924, a memory read column 926, and an output item column 928. The remaining decimation / skip column 922 shows the underlined bits of the decimation / skip vector 905, which correspond to the number of packet subword data items that can be decimated for the corresponding decimated word address. The word-aligned decimation / skip column 924 corresponds to bits indicating the packet subword data items to be decimated for the corresponding decimated word address. The memory read column 926 indicates when to perform a memory read operation to decimate packet subword data for the corresponding decimated word address, or when to suppress a memory read operation for the corresponding decimated word address. For example, the unitalicized extract word address in memory read column 926 indicates when the sparse clustering unit 610 directly extracts the corresponding pack word from memory (e.g., from memory storage device 614), and the italicized and underlined extract word address indicates when the sparse clustering unit 610 directly extracts the corresponding pack word from cache (e.g., from circular buffer cache 710).

[0111] In this implementation, each extracted word address includes the number of data items (for column 912 corresponding to the extracted word address) that can be extracted by the sparse aggregation unit 610 and subsequently output for subsequent processing (e.g., to the data unpacking unit 612). The number of data items can be used to shift one or more of the remaining extract / skip vector R (column 922) to the left-hand side, such that the remaining extract / skip vector R is consumed when the corresponding kernel footprint is transferred from the top to the bottom of column 922. Furthermore, the first data item number n (column 910) can be represented as 0 padding bits and configured to word-align the bits of the shifted-out vector R. For example, for the extracted word address A' at kernel coordinates (-1, 0), the first data item number n can be configured as 0, 0, 0 padding bits, and the padding bits are subsequently appended to the bits of the shifted-out vector R, such that the resulting word-aligned extract / skip mask can be represented as 0, 0, 0, 1. Subsequently, a bitwise AND operation is performed on the obtained word-aligned decimation / skip mask (containing 0, 0, 0, 1) and the corresponding padding mask (containing 0, 0, 0, 0) to generate a corresponding word-aligned decimation / skip vector F (column 926) containing 0, 0, 0, 0. The sparse aggregation unit 610 is configured to repeat this process for each decimated word address in column 908, thereby generating vector F for each decimated word address in column 908.

[0112] Furthermore, the word-aligned decimation / skip vector F is configured to indicate the decimation position corresponding to the packaged sub-word data item decimated from the corresponding decimated word address, and the decimation suppression position which may correspond to the undecimated packaged sub-word data item, however the corresponding undecimated packaged sub-word data item may be output as a zero value. The bit values ​​of the vector F(i) containing a value of "1" indicate the decimation position for the corresponding decimated word address, and a value of "0" indicates the decimation suppression position for the corresponding decimated word address, where i is an index that may include 0, 1, 2, 3. For example, the vector F(i) corresponding to the decimated word address D' at kernel coordinates (0, 0) may be represented as 1, 0, 1, 0, where the first packaged sub-word data item of the decimated word address D' is decimated from the memory (e.g., storage device or cache unit) corresponding to address D'[0], and the third packaged sub-word data item of the decimated word address D' is decimated from the memory corresponding to address D'[2]. In such an implementation, the second and fourth packet sub-word data items corresponding to the extracted word addresses D'[1] and D'[3] respectively can have their extraction suppressed and can be output as zero values.

[0113] The sparse aggregation unit 612 is configured to determine the output item column 928 for each decimated word address. Furthermore, null or zero values ​​in the output item column 928 can be output as padding data for subsequent processing steps. In one embodiment, the sparse aggregation unit 612 is configured to sequentially output packed sub-word data items from each decimated word address. For example, for kernel coordinates (0, 0), the two packed sub-word data items that need to be output for the decimated memory word D' are output as D'[0], followed by D'[2].

[0114] Reference Figure 10 This disclosure illustrates a process flow 1000 for accelerating the assembly and unpacking of data for ANN computation on a compact microcontroller (uC) device, according to various aspects / implementations / details described herein. Method 1000 begins at 1002, where a plurality of addresses and suppression indicators (e.g., padding masks) of packaged data stored on a memory device (e.g., memory retrieval unit 606) are determined, wherein the suppression indicators identify a subset of the plurality of addresses associated with the packaged data generated on the memory device (e.g., memory retrieval unit 614). In this embodiment, the plurality of addresses correspond to retrieval word addresses associated with kernel footprints corresponding to kernel filters.

[0115] At 1004, the stored packaged data is extracted from the memory using a memory extraction device based on multiple addresses and suppression indicators.

[0116] At position 1006, the generated packaged data is generated based on the suppression indicator by means of a memory extraction device.

[0117] At 1008, formatted data is constructed from stored or generated packaged data using a memory extraction device based on a predefined data structure of the microcontroller (e.g., CPU 604). In an embodiment, constructing the formatted data includes decompressing or compressing the stored packaged data based on at least one of offset, bit shift, scaling, rounding, table lookup, and variable-length encoding (e.g., Huffman encoding).

[0118] Based on the embodiments and examples described herein, the examples (implementations) may include, for example, the following subjects: methods or apparatus for performing actions or blocks of a method; at least one machine-readable medium including instructions that, when executed by a machine (e.g., a processor with memory, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc.), cause the machine to perform actions of a method, apparatus, or system for concurrent communication using various communication technologies.

[0119] The first example is an apparatus used in a processing device, the apparatus comprising: a processor configured to process data having a predefined data structure; a memory extraction device coupled to the processor, the memory extraction device being configured to determine an address for packaged data for the processor, wherein the packaged data is stored in a memory device coupled to the processor; and providing output data to the processor based on the address of the packaged data, wherein the output data is configured according to a predefined data structure.

[0120] The second example may include the first example, wherein the memory extraction device is configured to extract packaged data at least partially from the instruction memory.

[0121] The third example may include the first or second example, wherein the memory extraction device is configured to share resources with the processor's instruction extraction device.

[0122] The fourth example may include any one or more of the first to third examples, wherein the packaged data includes stored data and generated data, and wherein the memory extraction device is configured to determine a first set of addresses associated with the stored data and a second set of addresses associated with the generated data.

[0123] The fifth example may include any or more of the first through fourth examples, wherein the generated data includes padding data.

[0124] The sixth example may include any one or more of the first to fifth examples, wherein the memory extraction device is configured to selectively extract stored data according to a first address set, and wherein the memory extraction device is further configured to ignore a second address set associated with the generated data.

[0125] The seventh example may include any one or more of the first to sixth examples, wherein the stored data includes one or more redundant memory words, wherein the memory extraction device includes a circular buffer configured to store the redundant memory words, and wherein the stored data is selectively extracted based on the redundant memory words stored in the circular buffer.

[0126] The eighth example may include any one or more of the first to seventh examples, wherein the memory extraction device is configured to selectively extract redundant memory words of stored data from the memory device one at a time based on a first set of addresses.

[0127] The ninth example may include any or more of the first through eighth examples, wherein the memory extraction device is further configured to provide output data by formatting the stored data of the packaged data according to a predefined data structure of the processor.

[0128] The tenth example may include any or more of the first to ninth examples, wherein the memory extraction device is further configured to provide output data by further decompressing the formatted stored data.

[0129] The eleventh example could be an apparatus for an artificial neural network (ANN), comprising: a microcontroller configured to be coupled to a memory device and configured to process a data stream, wherein the data stream is configured to match at least one of a predefined precision, a predefined layout, or a predefined data structure; a data stream extraction device coupled to the memory device and the microcontroller, the data stream extraction device being configured to process data stored on the memory device and provide a continuous output stream to the microcontroller, wherein the data stream extraction device includes: an address generator component configured to generate a plurality of data addresses of the data stored on the memory device; a data extraction component configured to import selected data from the memory device based on the plurality of data addresses; and a data formatting component configured to provide a continuous output stream to the microcontroller.

[0130] The twelfth example may include the eleventh example, wherein the data formatting component is further configured to provide a data format and a data instruction set of the microcontroller that match the selected data.

[0131] The thirteenth example may include any or more of the eleventh to twelfth examples, wherein the data extraction component is further configured to import selected data at least partially from the instruction memory, and wherein the data extraction component is further configured to share resources with the microcontroller's instruction extraction device.

[0132] The fourteenth example may include any or more of the eleventh to thirteenth examples, wherein the address generator component is configured to generate a suppression indicator for each of a plurality of data addresses, wherein the selected data is introduced based on the suppression indicator, and wherein the suppression indicator is associated with the data to be replaced by the generated data according to a predefined data structure.

[0133] The fifteenth example may include any or more of the eleventh to fourteenth examples, wherein the data extraction component is also configured to skip data based on predefined control information or extracted from memory.

[0134] The sixteenth example may include any or more of the eleventh to fifteenth examples, wherein the data extraction component is further configured to decompress a continuous series of variable-length encoded data.

[0135] The seventeenth example may include any or more of the eleventh to sixteenth examples, wherein a subset of multiple data addresses is associated with one or more repeating words, wherein the data extraction component includes a circular buffer cache configured to store one or more repeating words such that the data extraction component introduces one or more repeating words from the memory device at a time, and wherein selected data is extracted from the circular buffer cache while it has already been stored in the circular buffer cache.

[0136] The eighteenth example could be a method for an artificial neural network (ANN), the method comprising: determining, via a memory extraction device, multiple addresses and suppression indicators of packaged data stored on a memory device, wherein the suppression indicators identify a subset of multiple addresses associated with packaged data generated on the memory device; extracting the stored packaged data from memory based on the multiple addresses and suppression indicators via the memory extraction device; generating generated packaged data based on the suppression indicators via the memory extraction device; and constructing formatted data based on the stored packaged data or the generated packaged data, via the memory extraction device, according to a predefined data structure of a microcontroller.

[0137] The nineteenth example may include the eighteenth example, and also includes: skipping data via a memory extraction device based on predefined or memory-extracted control information.

[0138] The twentieth example includes any or more of the eighteenth to nineteenth examples, wherein constructing formatted data includes decompressing or compressing the stored packaged data according to at least one of offset, bit shift, scaling, rounding, lookup table, and variable length encoding.

[0139] Applications (e.g., program modules) may include routines, programs, components, data structures, etc., that perform specific tasks or implement specific abstract data types. Furthermore, those skilled in the art will understand that the disclosed operations can be practiced with other system configurations, including single-processor or multi-processor systems, minicomputers, mainframe computers, personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, each of which can be operatively coupled to one or more associated devices.

[0140] Computing devices can typically include a variety of computer-readable media. Computer-readable media can be any available medium that can be accessed by a computer, and includes both volatile and non-volatile media, and removable and non-removable media. By way of example and not limitation, computer-readable media can include computer storage media and communication media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media (e.g., one or more data stores) can include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CDROM, Digital Universal Disc (DVD) or other optical disc storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store the required information and is accessible by a computer.

[0141] Communication media typically contain computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals, such as carrier waves or other transmission mechanisms, and include any information transmission medium. The term "modulated data signal" means a signal having one or more characteristics set or altered in a manner that encodes information in the signal. By way of example and not limitation, communication media include wired media such as wired networks or direct wired connections, and wireless media such as acoustic, RF, infrared, and other wireless media. Any combination of the above should also be included within the scope of computer-readable media.

[0142] It should be understood that the aspects described herein can be implemented by hardware, software, firmware, or any combination thereof. When implemented in software, functionality can be stored on or transmitted via a computer-readable medium as one or more instructions or code. A computer-readable medium includes both computer storage media and communication media that include any medium facilitating the transfer of a computer program from one location to another. A storage medium can be any available medium accessible by a general-purpose or special-purpose computer. By way of example, and not limitation, such a computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store desired program code elements in the form of instructions or data structures and is accessible by a general-purpose or special-purpose computer or a general-purpose processor or a special-purpose processor. Additionally, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0143] The various exemplary logics, logic blocks, modules, and circuits described in conjunction with the aspects disclosed herein can be implemented or executed by a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or combinations thereof, designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, a combination of one or more microprocessors with a DSP core, or any other such configuration. Furthermore, at least one processor may include one or more modules operable to perform one or more of the actions and / or functions described herein.

[0144] For software implementation, the techniques described herein can be implemented using modules (e.g., processes, functions, etc.) that perform the functions described herein. Software code can be stored in memory units and executed by a processor. Where the memory units are communicatively coupled to the processor by various means known in the art, the memory units can be implemented within or outside the processor. Furthermore, at least one processor may include one or more modules operable to perform the functions described herein.

[0145] The technologies described herein can be used in various wireless communication systems, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and others. The terms "system" and "network" are often used interchangeably. CDMA systems can implement radio technologies such as Universal Terrestrial Radio Access (UTRA), CDMA2000, etc. UTRA includes Wideband CDMA (W-CDMA) and other variants of CDMA. Furthermore, CDMA2000 encompasses the IS-2000, IS-95, and IS-856 standards. TDMA systems can implement radio technologies such as the Global System for Mobile Communications (GSM). OFDMA systems can implement radio technologies such as Evolved UTRA (E-UTRA), Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, Flash-OFDM, etc. UTRA and E-UTRA are part of the Universal Mobile Telecommunications System (UMTS). 3GPP Long Term Evolution (LTE) is the UMTS version using E-UTRA. 3GPP Long Term Evolution (LTE) employs OFDMA on the downlink and SC-FDMA on the uplink. UTRA, E-UTRA, UMTS, LTE, and GSM are described in documents from an organization called the "3rd Generation Partnership Project" (3GPP). Furthermore, CDMA2000 and UMB are described in documents from an organization called "3rd Generation Partnership Project 2" (3GPP2). In addition, such wireless communication systems can additionally include peer-to-peer (e.g., mobile-to-mobile) self-organizing network systems that typically use unpaired unlicensed spectrum, 802.xx wireless LANs, BLUETOOTH, and any other short-range or long-range wireless communication technologies.

[0146] Single-carrier frequency division multiple access (SC-FDMA) utilizing single-carrier modulation and frequency-domain equalization is a technique that can be used in conjunction with the disclosed aspects. SC-FDMA systems have similar performance and substantially similar overall complexity to OFDMA systems. SC-FDMA signals exhibit a lower peak-to-average power ratio (PAPR) due to their inherent single-carrier structure. SC-FDMA can be used for uplink communication, where the lower PAPR allows mobile terminals to benefit from improved transmit power efficiency.

[0147] Furthermore, the various aspects or features described herein can be implemented as methods, apparatus, or articles of art using standard programming and / or engineering techniques. As used herein, the term "article of art" is intended to include computer programs accessible from any computer-readable device, carrier, or medium. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic stripes, etc.), optical discs (e.g., compact discs (CDs), digital versatile discs (DVDs), etc.), smart cards, and flash memory devices (e.g., EPROMs, cards, levers, key drives, etc.). Additionally, the various storage media described herein may represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data. Furthermore, computer program products may include computer-readable media having one or more instructions or codes operable to cause a computer to perform the functions described herein.

[0148] Furthermore, the actions and / or functions of the methods or algorithms described in conjunction with the aspects disclosed herein can be directly implemented as hardware, software modules executed by a processor, or combinations thereof. Software modules can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. Exemplary storage media can be coupled to a processor such that the processor can read information from and write information to the storage medium. In alternative embodiments, the storage medium can be a component of the processor. Furthermore, in some aspects, the processor and storage medium can reside in an ASIC. Additionally, the ASIC can reside in a user terminal. In alternative embodiments, the processor and storage medium can reside as discrete components of a user terminal. Additionally, in some aspects, the actions and / or functions of the methods or algorithms can reside as one or any combination or set of code and / or instructions that can be incorporated into a computer program product on a machine-readable and / or computer-readable medium.

[0149] The above description of the embodiments shown in this disclosure (including the description in the abstract) is not intended to be exhaustive or to limit the disclosed embodiments to the exact forms disclosed. Although specific embodiments and examples have been described herein for illustrative purposes, various modifications are possible within the scope of such embodiments and examples, as will be appreciated by those skilled in the art.

[0150] In this regard, although the disclosed subject matter has been described in conjunction with various embodiments and corresponding figures, it should be understood where applicable that other similar embodiments may be used, or modifications and additions may be made to the described embodiments without departing from the described implementation, to perform the same, similar, alternative, or substitute functions of the disclosed subject matter. Therefore, the disclosed subject matter should not be limited to any single embodiment described herein, but should be interpreted in breadth and scope according to the appended claims.

[0151] In particular, regarding the various functions performed by the components or structures (components, devices, circuits, systems, etc.) described above, unless otherwise indicated, the terminology used to describe such components (including references to "device") is intended to correspond to any component or structure that performs the specified function of the described component (e.g., functionally equivalent), even where the function performing the exemplary implementations of the invention shown herein is not structurally equivalent to the disclosed structure. Furthermore, although specific features may have been disclosed with respect to only one of several implementations, such features can be combined with one or more other features of other contemplated implementations and may be advantageous for any given or particular application.

Claims

1. A method for an artificial neural network, the method comprising: Using a memory extraction device, the layout and packing format information of the packed data stored on the memory device are used to determine the address stream of the packed data and a suppression indicator, wherein the suppression indicator identifies a subset of addresses in the address stream associated with the generated packed data in the memory device. The memory extraction device selectively extracts stored packaged data from the memory device based on the address stream and the suppression indicator, while ignoring the generated packaged data. The memory extraction device sequentially extracts the stored packaged data from the memory device and processes the stored packaged data sequentially. The generated packaged data is generated based on the suppression indicator using the memory extraction device; and The memory extraction device constructs formatted data based on a predefined data structure of a microcontroller with a single-issue microarchitecture, according to the stored or generated packaged data.

2. The method according to claim 1, further comprising: Data can be skipped using the memory extraction device based on predefined control information or control information extracted from the memory device.

3. The method according to claim 1, wherein, Constructing the formatted data includes decompressing or compressing the stored packaged data based on at least one of offset, shift, scaling, rounding, lookup table, and variable length encoding.

4. The method according to claim 1, further comprising: The microcontroller reprograms the format information of the packaged data, wherein the memory extraction device changes between different packaging formats when generating the generated data.

5. An apparatus for use in a processing device, the apparatus comprising: A processor configured to process data having a predefined data structure, wherein the processor has a single-issue microarchitecture and includes special function registers, bus agents, and instruction fetching means; A memory extraction device coupled to the processor, wherein the memory extraction device is configured as a sequential state machine and configured to: Determine the address of the packaged data for the processor, wherein the packaged data is stored on a memory device coupled to the processor; The packaged data is at least partially extracted from the instruction memory; Sharing resources with the processor's instruction fetching device, special function register, and bus agent, wherein the instruction fetching device is configured to fetch the instruction memory; and Output data is provided to the processor based on the address of the packaged data, wherein the output data is configured according to the predefined data structure, and wherein the memory extraction device is configured to process the packaged data sequentially.

6. The apparatus according to claim 5, wherein, The packaged data includes stored data and generated data, wherein the memory extraction device is configured to determine a first set of addresses associated with the stored data and a second set of addresses associated with the generated data.

7. The apparatus according to claim 6, wherein, The memory extraction device is configured to at least match the capacity of the processor to process the output data, wherein the processor has an ordered microarchitecture.

8. The apparatus according to claim 6, wherein, The memory extraction device is configured to selectively extract the stored data according to the first address set, and wherein the memory extraction device is further configured to ignore the second address set associated with the generated data.

9. The apparatus according to claim 8, wherein, The stored data includes one or more redundant memory words, wherein the memory extraction device includes a circular buffer configured to store the redundant memory words, and wherein the stored data is selectively extracted based on the redundant memory words stored in the circular buffer.

10. The apparatus according to claim 9, wherein, The memory extraction device is configured to selectively extract redundant memory words of the stored data from the memory device one at a time based on the first address set.

11. The apparatus according to claim 6, wherein, The memory extraction device is further configured to provide the output data by formatting the stored data of the packaged data according to the predefined data structure of the processor.

12. The apparatus according to claim 11, wherein, The memory extraction device is further configured to provide the output data by further decompressing the formatted stored data.

13. The apparatus according to claim 5, wherein, The processor does not have Single Instruction Multiple Data (SIMD) capability.

14. An apparatus for an artificial neural network, the apparatus comprising: A microcontroller configured to be coupled to a memory device and configured to process a data stream, wherein the data stream is configured to match at least one of a predefined precision, a predefined layout, or a predefined data structure; A data stream extraction device, coupled to the memory device and the microcontroller, and configured to process packaged data on the memory device and provide a continuous output stream to the microcontroller, wherein the packaged data includes stored packaged data and generated packaged data, wherein the generated packaged data exists in the packaged data based on a predefined packaging format, wherein the data stream extraction device is configured to process the packaged data sequentially, and wherein the data stream extraction device includes: An address generator component is configured to generate an address stream and sub-word data item information for the packaged data, wherein the sub-word data item information includes the number and position of sub-words of the stored packaged data on the memory device and associated with the address stream; A data extraction component is configured to selectively extract the stored packaged data from the memory device based on the address stream and the sub-word data item information, and to selectively suppress the extraction of the generated packaged data; and A data formatting component is configured to provide the continuous output stream to the microcontroller.

15. The apparatus according to claim 14, wherein, The data formatting component is also configured to provide a data format that matches the stored packaged data and the data instruction set of the microcontroller.

16. The apparatus according to claim 14, wherein, The address generator component is configured to generate a suppression indicator for each address in the address stream, wherein the stored packaged data is extracted based on the suppression indicator, and wherein the suppression indicator identifies the generated packaged data that is replaced by generated data according to the predefined data structure.

17. The apparatus according to claim 14, wherein, The stored packaged data includes data that is not aligned with the byte address, wherein the data stream extraction device is configured to process the data that is not aligned with the byte address based at least in part on the number of sub-words and the position of the sub-words.

18. The apparatus according to claim 14, wherein, The microcontroller is configured to perform convolution on the data stream and reprogram the format information of the packaged data on the memory device, wherein the data stream extraction device is configured to switch from processing the packaged data on the memory device using the predefined packaging format to processing the packaged data on the memory device using a second packaging format within a single convolution performed by the microcontroller, wherein the predefined packaging format is different from the second packaging format.

19. The apparatus according to claim 14, wherein, A subset of addresses in the address stream is associated with one or more repeating words, wherein the data extraction component includes a circular buffer cache configured to store the one or more repeating words such that the data extraction component introduces the one or more repeating words from the memory device at a time, and wherein the stored packaged data is extracted from the circular buffer cache after it has already been stored in the circular buffer cache.

20. The apparatus according to claim 14, wherein, The stored packaged data includes one or more redundant memory words, wherein the data stream extraction device is configured to selectively extract the one or more redundant memory words from the memory device at a time.

Images