Efficient selection of single instruction multiple data operations for neural processing units

Abstract

Systems and methods for efficient selection of single instruction multiple data operations for neural processing units. An example processor system includes a matrix processor configured to perform a convolution associated with a neural network and a single instruction multiple data (SIMD) processor in communication with the matrix processor, the SIMD processor configured to perform a set of operations based on a current location associated with processing the neural network, and the set of operations is selected from a plurality of SIMD programs, and the set of operations is selected from the SIMD program according to the current location.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 364,451, entitled “EFFICIENT SELECTION OF SINGLEINSTRUCTION MULTIPLE DATA OPERATIONS FOR NEURAL PROCESSING UNITS,” filed on May 10, 2022, the disclosure of which is hereby incorporated herein by reference. Technical Field

[0003] The present disclosure relates to processing units, and more particularly, to single instruction multiple data processing units. Background Art

[0004] Neural networks are relied upon for different purposes and increasingly form the basis of technology. For example, a neural network may be utilized to perform object classification on an image obtained via a user device (e.g., a smartphone). In this example, the neural network may represent a convolutional neural network that applies convolutional layers, pooling layers, and one or more fully connected layers to classify objects depicted in an image. As another example, a neural network may be utilized for text translation between languages. For this example, the neural network may represent a recursive neural network.

[0005] With respect to convolutional neural networks, the network can be divided into convolutional layers, pooling layers, and the like. Example convolutional layers can cause a volume of a filter or kernel to be applied to input data. For example, a first convolutional layer can cause a convolution between image data and a filter or kernel. As another example, subsequent convolutional layers can cause a convolution between a feature map (e.g., output from a previous layer) and a filter or kernel. These layers can require a large number of operations. For example, there can be millions or billions of multiplications and additions to process a convolutional layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1A is a block diagram illustrating an example processor system including a matrix processor and one or more single instruction multiple data (SIMD) processors.

[0007] Figure 1B is a block diagram illustrating details of an example SIMD program that may be selected for implementation by a SIMD processor.

[0008] Figure 2 A representation of an example SIMD program selected during processing of a layer of a neural network is shown.

[0009] Figure 3is a flow chart of an example process for processing a neural network using SIMD asterisks.

[0010] Figure 4 is a block diagram illustrating an example vehicle including a vehicle processor system.

[0011] Embodiments of the present disclosure and their advantages may be best understood by reference to the following detailed description. It should be understood that the same reference numerals are used to identify the same elements shown in one or more of the accompanying drawings, wherein the display therein is for the purpose of illustrating embodiments of the present disclosure, rather than for the purpose of limiting embodiments of the present disclosure. DETAILED DESCRIPTION

[0012] The present application describes techniques for efficiently causing the execution of operations or programs (e.g., groups of operations) based on a current position, processing step, or processing operation (collectively referred to herein as a position) associated with a neural network. As will be described, these operations can be implemented by one or more single instruction multiple data (SIMD) processors that are connected to a convolution engine or processor (e.g., matrix processor 102) or are otherwise capable of receiving information from and providing information to a convolution engine or processor. For example, there can be 1, 2, 5, 17, etc. SIMD processors. In some embodiments, there can be 2, 5, 17, etc. Operations can be associated with quantizing received data, normalizing (e.g., evenly redistributing data values ​​within a range of bit sizes used for calculations), clearing state, loading constants, storing statistics associated with processing a neural network or a portion thereof (e.g., a layer), etc.

[0013] As known to those skilled in the art, prediction is an example technique for selectively performing operations based on the state of a mask or prediction (e.g., a prediction register). For example, predictions can use conditional instructions associated with corresponding predictions that are evaluated to determine whether they are true. In this example, conditional instructions are evaluated during each pass through a portion of the code. Therefore, a "no" operation can generally be issued for any prediction that is negatively evaluated. Therefore, predictions can shut down instructions based on the prediction of the instruction, but these instructions can also generally not be skipped to avoid the substantial complexity associated with dynamically skipping instructions. On the contrary, even for predictions that are negatively evaluated, computation cycles are consumed. Therefore, predictions can reduce fetch performance due to skipped instructions.

[0014] Certain computing tasks, such as processing of neural networks, may have predictable processing paths that are repeated during the operation of the computing task. For example, a convolutional neural network may process data using an ordered layer that causes the execution of operations on the input data and subsequently routes the data to subsequent layers. In this example, a convolution engine or processor may be used to perform computationally intensive convolutions. SIMD processors may be used for other tasks, such as quantizing data as described above. Using predictions for operations performed by a SIMD processor may cause a large amount of wasted computing cycles and increase energy usage. For example, different operations may be performed depending on a specific location in a neural network. In this example, predictions may cause a considerable number of operations to be skipped until operations specific to a specific location are reached.

[0015] In contrast, the present application describes efficient techniques for selecting operations (e.g., a processing layer, an output channel, a portion of an output channel, etc.) based on a location in a neural network. The location may also be referred to herein as a section and indicates an operation or a set of operations to be performed by a SIMD processor. Advantageously, a program counter may be set based on a section (e.g., using a state machine). Thus, the program counter may be limited to operations between the start and end of operations associated with a section. These operations may then be implemented by the SIMD processor.

[0016] In this way, the techniques described herein can avoid the added computation cycles used by the prediction to evaluate the prediction. Instead, due to the hierarchical structure of the processing neural network, the location (e.g., a section) can indicate a specific operation to be performed. For example, the program can be partitioned or otherwise divided into a set of operations that correspond to the natural nested order of the neural network processing (e.g., the beginning of a layer, the beginning of an output channel, the end of an output channel, the end of a layer, etc.). Thus, instruction fetches can start and end at different points in the program, but for example, will be a continuous set of instructions.

[0017] Although neural networks and SIMD processors (e.g., neural processing units) associated with neural network processing are described herein, it is understood that the techniques described herein can be applied to other processors. For example, any application-specific processor that utilizes software routines at predictable points in a data flow can utilize the techniques of the present disclosure.

[0018] Example Block Diagram

[0019] Figure 1A1 is a block diagram showing an example processor system 100 including a matrix processor 102 and one or more single instruction multiple data (SIMD) processors 110A to 110N. In some embodiments, there are more than one SIMD processors 110A to 110N. In some embodiments, the processor system 100 can be used to implement automatic or semi-automatic driving of a vehicle. For example, the processor system 100 can be included in a vehicle (e.g., an electric vehicle) and use sensor data to implement automatic or semi-automatic driving. Example sensors can include image sensors, radars, ultrasonic sensors, lidars, etc.

[0020] In some embodiments, the matrix processor 102 can be used to perform convolutions associated with a convolutional neural network or a transfer network. For example, the input data 104 and the weight data 106 can be convolved. The matrix processor 102 may include multiple multiplication and accumulation units that perform convolutions. As an example, the matrix processor can use input and weight data that have been organized or formatted to facilitate larger convolution operations. For example, the input data 104 can be in the form of a three-dimensional matrix (e.g., two-dimensional data across multiple input channels). In this example, the output data can span multiple output channels. Therefore, the matrix processor 102 can process larger input data by merging or flattening each two-dimensional output channel into a vector, so that the matrix processor 102 can process the entire channel or a major part thereof. As another example, data can be efficiently reused so that weight data can be shared across convolutions. With respect to output channels, the weight data 106 can represent the weight data (e.g., kernel) used to calculate the output channel.

[0021] In some embodiments, the techniques described herein may be applied to the example matrix processors described in U.S. Patent No. 11,157,287, U.S. Patent Publication No. 2019 / 0026250, and U.S. Patent No. 11,157,441, which are incorporated herein by reference in their entirety and form a part of the present disclosure, as if set forth herein.

[0022] Therefore, in some embodiments, the matrix processor 102 can continue to calculate the convolution for a particular output channel. For example, the matrix processor 102 can calculate the convolution based on the weight data 106 and the input data 104, which can represent at least a portion of the convolution for a particular output channel. It is understood that there may be millions or billions of operations required to calculate a particular output channel. In some embodiments, the calculation portion associated with a particular output channel can be referred to as a corresponding path.

[0023] As will be described, during the above-mentioned convolutional neural network calculations, SIMD processors 110A to 110N can perform different operations. For example, SIMD processors 110A to 110N can receive outputs from matrix processor 102. In this example, the output can represent the processing results associated with convolution input data 104 and weight data 106. Example operations may include quantizing the processing results. Additionally, the SIMD processor can determine statistics associated with processing a specific output channel or a layer being determined for a specific output channel. Therefore, SIMD processors 110A to 110N can monitor statistics such as average values, minimum values, maximum values, etc. Similarly, SIMD processors 110A to 110N can determine statistical information for each output channel being determined for a layer.

[0024] As another example, the SIMD processors 110A to 110N may provide information to the matrix processor. For example, the SIMD processors 110A to 110N may load constants required for processing of a particular layer. In this example, the constants may be loaded into the matrix processor or into the SIMD processor for use. As another example, the SIMD processors 110A to 110N may clear states associated with processing a layer, output channel, or path.

[0025] SIMD processors 110A to 110N can perform operations according to the current position (also referred to herein as a section) associated with processing the above-mentioned convolutional neural network. For example, the operations performed by SIMD processors 110A to 110N can be grouped. Therefore, the compiler can divide the program including the operation into blocks (e.g., individual programs), which can be executed separately by SIMD processors 110A to 110N during operation. As an example, during the operation of the processor system 100, a group of operations or more than one group of operations can be selected for execution by SIMD processors 110A to 110N. In some embodiments, a program counter can be used to limit the execution of the operation to a group of operations according to the current section. As an example, the program counter can be limited to the start and end pointers so that a group of operations (e.g., a specific program) is performed by SIMD processors 110A to 110N. Therefore, the hardware (e.g., program counter, hardware-based state machine) included in the processor system 100 can control the following selection and implementation:

[0026] In this manner, SIMD processors 110A to 110N may perform specific operations depending on whether a specific layer has begun to be processed, whether a specific output channel has begun to be processed, whether a specific pass has begun, etc. This flexibility is possible without requiring computationally expensive predictions as described above. Thus, the techniques described herein limit the extent to which operations are skipped and not issued.

[0027] SIMD processors 110A to 110N may also be used with Figure 1A For example, the memory may represent an SRAM, and the SIMD processors 110A to 110N may provide processing results for storage in the SRAM. In this example, the processing results may represent a quantized version of the convolution implemented by the matrix processor 102.

[0028] Figure 1B 1 is a block diagram illustrating details of an example SIMD program 152 that may be selected for implementation by SIMD processors 110A to 110N. Figure 1A As described, SIMD processors 110A to 110N may perform operations based on the current section. For example, the current section may indicate a location associated with processing neural network 150. An example location may indicate a layer to be processed. Another example location may indicate an output channel to be processed. Another example location may indicate a channel associated with processing an output channel to be processed. In some embodiments, the example location may indicate a specific layer, a specific output channel, a specific path, etc.

[0029] In the illustrated embodiment, the SIMD program 152 is separated (e.g., grouped) into operations associated with layers, output channels, paths, etc. During operation of the processor system 100, these different groups may be selected (e.g., via a program counter) and implemented by the SIMD processors 110A to 110N based on the current section. As shown, the selected operations 154 from the SIMD program 152 are provided to the SIMD processors 110A to 110N.

[0030] As shown, SIMD programs 152 can be organized according to normal (e.g., conventional) processing through neural network 150. For example, different SIMD programs 152 can be selected 154 for implementation based on the progress through neural network 150. In this example, a state machine can be used to select a SIMD program, for example, based on a program counter. Therefore, a specific operation can be selected based on the layer that is activated (e.g., "layer A"). Similarly, when the layer is activated, a specific output channel (e.g., "output channel A") can be activated. Similarly, a first path can be activated. When the first path is completed, one or more remaining paths can be executed to determine the output channel. Optionally, a specific operation can be performed by SIMD processors 110A to 110N for the first path and the remaining paths. Then the next output channel can be activated, which can have the same or different operations as output channel A. As described herein, example operations can include quantizing data, determining statistical information, etc.

[0031] The following are additional example operations including example pseudocode. Each example includes one or more operations to be performed at different sections and forming a larger operation.

[0032] Example operations may include a parametric maximum operation. For example, the operation may output the maximum value (or index of the maximum value) within an element path. The element (e.g., a new element) may represent an input element (e.g., a single value) sent from a matrix processor to a SIMD channel (e.g., a SIMD processor) for post-processing. The following includes example pseudocode that references a specific section:

[0033] Festival Pseudo-SIMD code Open Layer Open output channel Open access curr_max=0 all curr_max=max(new_element,curr_max) Close the passage store out curr_max Close output channel Turn off layer

[0034] Another example operation may include a per-channel reduction. This operation may cause the SIMD processors 110A to 110N to find an average value included in a channel, such as a value between the SIMD processors 110A to 110N. For this operation, data may be moved between SIMD processors to achieve the reduction. The following includes example pseudocode:

[0035]

[0036]

[0037] Another example operation may include optionally collecting and storing per-layer averages while also storing output input elements. Example pseudocode may include:

[0038]

[0039] Therefore, SIMD program 152 can provide programmers with arbitrary flexibility so that operations are performed by SIMD processors 110A to 110N. It is understood that in some embodiments, these operations can represent corresponding single instructions that cause lockstep operations of SIMD processors 110A to 110N. In some embodiments, SIMD processors 110A to 110N can be grouped separately. For example, a subgroup of SIMD processors 110A to 110N can execute a first SIMD program and a different subgroup can execute a second SIMD program. In this way, there can be a group of SIMD processors that perform different operations.

[0040] Figure 2 A representation of an example SIMD program selected during processing of a layer of a neural network is shown. Figure 2, an example process flow 200 associated with processing a neural network is shown. For example, the process flow 200 identifies different locations or sections associated with processing. These locations can form a nested hierarchy of operations (e.g., a start group of operation A, a start group of operation B, a stop group of operation B, a start group of operation C, a stop group of operation C, a stop group of operation A).

[0041] In block 202, the position indicates the beginning of a layer. For example, a layer may be a convolutional layer in a neural network. As another example, a layer may represent a different layer (e.g., a pooling layer). In block 204, the position indicates the beginning of an output channel. In some embodiments, and as described above in Figure 1A As described in , the layer 202 can be processed based on processing the output channels (eg, processing the output channels individually). However, this processing is by way of example, and the techniques described herein are not so limited.

[0042] At block 206, the position indicates the start path of output channel 204. In the example shown, the SIMD program has been grouped between the start of output channel 204 and the end of output channel 208. For example, when the path is started, the SIMD program can make the current state cleared, load constants, etc. "All" blocks can represent operations that are always performed when the position is reached. Example operations can include quantized data. For example, the start path can represent the first path, and subsequent paths may not perform a set of operations for the start path. However, they can perform operations for "all" blocks. Therefore, once any block in blocks 202 to 208 is reached, "all" blocks can be optionally processed.

[0043] Ending a path may represent an operation that is performed once the path is completed. Ending a path may also represent the final path associated with an output channel. Example operations may include determining statistics for a single path for or for a path associated with an output channel. Similarly, the end of an output channel may represent an operation associated with completing an output channel or an operation associated with completing all output channels for a layer. Example operations may include writing per-channel statistics. For example, average, maximum, minimum, for each output channel or for all channels (e.g., for a layer). Additional operations are described in more detail above, for example in conjunction with pseudocode.

[0044] Example Flowchart

[0045] Figure 33 is a flow chart of an example process 300 for processing a neural network using SIMD sections. For convenience, process 300 is described as being performed by a system (e.g., processor system 100). In some embodiments, a program counter may identify a current section and cause selection of an operation to be performed by a SIMD processor (e.g., SIMD processors 110A to 110N).

[0046] At block 302, the system causes execution of a neural network. As described herein, a neural network may be referred to as a convolutional neural network. For example, a neural network may be used in part for autonomous or semi-autonomous driving. In this example, the neural network may be used to identify objects around the vehicle and characteristics of the objects (e.g., classification, speed, acceleration, location, etc.).

[0047] At block 304, the system identifies a current location associated with processing the neural network. As described above, the current location can represent a node associated with the neural network. Example locations can include a layer, an output channel, a pathway, an end of a pathway or multiple pathways, an end of an output channel, an end of a layer, etc.

[0048] In some embodiments, the program counter can be used to identify the current position. It can be appreciated that the program counter can indicate the current instruction (e.g., a pointer to the current instruction). Based on the current position, the program counter can identify the operation for the current position. For example, the program counter can be limited (e.g., by a processor) to the instruction range that includes the operation (e.g., a group of operations) identified. As mentioned above, the operation of the program can be grouped. These groups can be associated with corresponding start and end pointers (e.g., by a compiler). Therefore, the program counter can be used to limit the execution of a group of operations for the current position based on the start and end pointers.

[0049] At block 306, the system obtains output from the matrix processor. For some positions, the SIMD processor may use the output from the matrix processor. For example, the SIMD processor may perform operations to quantize the output. For other positions, the SIMD processor may not use or may not yet have access to the output from the matrix processor. As an example, the SIMD processor may load constants (e.g., into the matrix processor or SIMD processor for use in quantization, determining statistics, etc.).

[0050] At block 308, the system causes execution of a SIMD program associated with the processing location. As described in block 304, a particular program (e.g., a set of operations) may be selected for execution by a SIMD processor. For some operations, the SIMD processor may use output from a matrix processor (e.g., for quantization).

[0051] Vehicle Block Diagram

[0052] Figure 4 A block diagram of a vehicle 400 (e.g., vehicle 102) is shown. The vehicle 400 may include one or more motors 402 that move the vehicle 400. The motors 402 may include, for example, induction motors, permanent magnet motors, etc. As known to those skilled in the art, batteries 404 (e.g., one or more battery packs each including a plurality of batteries) may be used to power the motors 402.

[0053] Vehicle 400 also includes a propulsion system 406 that can be used to set a gear (eg, direction of propulsion) for the vehicle. With respect to electric vehicles, propulsion system 406 can adjust operation of electric motor 402 to change the direction of propulsion.

[0054] Additionally, the vehicle includes a processor system 100 that includes one or more single instruction multiple data processors (e.g., SIMD processors 110A to 110N) as described herein. The processor system 100 can process data, such as images received from image sensors located around the vehicle 400. The processor system 100 can additionally output information to a display 408 and receive information (e.g., user input) from the display 408, which is included in the vehicle 400.

[0055] Other embodiments

[0056] All processes described herein can be implemented and fully automated by software code modules executed by a computing system including one or more computers or processors. The code modules can be stored in any type of non-transient computer readable medium or other computer storage device. Some or all of the methods can be implemented in dedicated computer hardware.

[0057] According to the present disclosure, many other changes except those described herein will be apparent.For example, depending on the embodiment, some actions, events or functions of any algorithm described herein can be performed in different orders, or can be added, merged or completely ignored (for example, not all described actions or events are necessary for the practice of the algorithm). In addition, in certain embodiments, actions or events can be performed concurrently rather than sequentially, for example, by multithreading, interrupt processing or multiple processors or processor cores or on other parallel architectures. In addition, different tasks or processes can be performed by different machines and / or computing systems that can work together.

[0058] The various illustrative logic blocks, modules, and engines described in conjunction with the embodiments disclosed herein may be implemented or executed by a machine designed to perform the functions described herein (such as a processing unit or processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof). The processor may be a microprocessor, but in an alternative, the processor may be a controller, a microcontroller, or a state machine, a combination thereof, or the like. The processor may include a circuit system configured to process computer executable instructions. In another embodiment, the processor includes an FPGA or other programmable device that performs logic operations without processing computer executable instructions. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in combination with a DSP core, or any other such configuration. Although this article is primarily described with respect to digital technology, the processor may also primarily include analog components. For example, some or all of the signal processing algorithms described herein may be implemented in an analog circuit system or a mixed analog and digital circuit system. The computing environment may include any type of computer system, including but not limited to a microprocessor-based computer system, a mainframe computer, a digital signal processor, a portable computing device, a device controller, or a computing engine within an appliance, and the like.

[0059] Unless otherwise specifically stated, conditional language such as "can," "can," "might," or "may" is understood in the context to be generally used to express that certain features, elements, or steps are included in some embodiments and not included in other examples. Therefore, such conditional language is generally not intended to imply that features, elements, and / or steps are in any way required for one or more embodiments, or that one or more embodiments must include logic for deciding whether such features, elements, and / or steps are included in any particular embodiment or are to be performed in any particular embodiment with or without user input or prompting.

[0060] Unless specifically stated otherwise, disjunctive language, such as the phrase "at least one of X, Y, or Z," is understood along with the context to be generally used to indicate that an item, term, etc. may be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is generally not intended to, and should not, imply that certain embodiments require that at least one of X, at least one of Y, or at least one of Z are all present.

[0061] Any process description, element or block in the flowcharts described herein and / or depicted in the accompanying drawings should be understood to potentially represent a module, segment or portion of code including one or more executable instructions for implementing a specific logical function or element in the process. Alternative implementations are included within the scope of the embodiments described herein, where elements or functions may be deleted, executed from the order shown or discussed, including substantially simultaneously or in reverse order, depending on the functions involved as will be understood by those skilled in the art.

[0062] Unless expressly stated otherwise, articles such as "a" or "an" should generally be interpreted as including one or more of the described items. Thus, phrases such as "the device is configured to" are intended to include one or more of the described devices. Such one or more of the described devices may also be configured together to implement the described statements. For example, "a processor configured to implement statements A, B, and C" may include a first processor configured to execute statement A, which works together with a second processor configured to execute statements B and C.

[0063] It should be emphasized that many changes and modifications may be made to the above-described embodiments, and the elements of these embodiments should be understood to be among other acceptable examples. All such modifications and changes are intended to be included within the scope of the present disclosure.

Claims

1. A processor system, comprising: a matrix processor configured to perform convolutions associated with the neural network; as well as one or more single instruction multiple data (SIMD) processors in communication with the matrix processor, wherein the SIMD processor is configured to perform a set of operations based on a current position associated with processing the neural network, Wherein the set of operations is selected from a plurality of SIMD programs, and wherein the set of operations is selected from the SIMD programs according to the current position.

2. The processor system of claim 1 , wherein the matrix processor is configured to perform a convolution associated with an output layer of a convolutional layer included in the neural network.

3. The processor system of claim 1, wherein the SIMD processor comprises a plurality of SIMD processors.

4. The processor system of claim 1, wherein the current position indicates one or more of a processing layer, an output channel, or a path.

5. The processor system of claim 4, wherein the path represents a processing block or a portion of a process associated with an output channel.

6. The processor system of claim 4, wherein the current position indicates a beginning or ending layer, output channel, or path.

7. The processor system of claim 1, wherein the set of operations is selected using a program counter.

8. The processor system of claim 7, wherein the program counter is used to limit execution between a start pointer and an end pointer, and wherein the start pointer and the end pointer identify the set of operations from the SIMD program.

9. The processor system of claim 1, wherein the set of operations is associated with quantized output from the matrix processor.

10. The processor system of claim 1, wherein the set of operations is associated with determining statistical information.

11. The processor system of claim 1 , wherein the SIMD program is accessed according to the processing flow of the neural network, and wherein prediction is not used such that each operation included in the SIMD program prior to the current position is not evaluated.

12. A method implemented by a processor system, the method comprising: causing execution of the neural network; identifying a current location associated with the neural network; as well as Execution of a SIMD program associated with the current location is caused via one or more SIMD processors, wherein the SIMD program is selected from a plurality of SIMD programs, and wherein the SIMD program is selected from the SIMD programs based on the current location.

13. The method of claim 12, wherein the SIMD program is associated with a quantized output from a matrix processor.

14. The method of claim 12, wherein the SIMD program is associated with determining statistical information.

15. The method of claim 12, wherein the processor system is configured to perform a convolution associated with an output layer of a convolutional layer included in the neural network.

16. The method of claim 12, wherein the current position indicates one or more of a processing layer, an output channel, or a pathway.

17. The method of claim 16, wherein the path represents a processing block or a portion of processing associated with an output channel.

18. The method of claim 12, wherein the current position indicates a starting or ending layer, output channel, or path.

19. The method of claim 12, wherein the SIMD program is selected using a program counter, wherein the program counter is used to limit execution between a start pointer and an end pointer, and wherein the start pointer and the end pointer identify the SIMD program from the plurality of SIMD programs.

20. The method of claim 12, wherein the SIMD program is accessed according to the processing flow of the neural network, and wherein prediction is not used such that each operation included in the SIMD program prior to the current position is not evaluated.

Images