Efficient direct folding using SIMD instructions

Abstract

To provide systems and methods for filtering of signals using convolution operations.SOLUTION: A computer comprising one or more processors offering vector instructions may implement a direct convolution on a source data set. The source data set may be one-dimensional or multi-dimensional. For a given vector width w of the vector instructions, w consecutive data elements of an output data set are computed in parallel using vector instructions. For multi-dimensional data sets, multiple vectors of the output data set are computed for a single load of a set of vectors from the source data set. New vector instructions are disclosed to improve the performance of the convolution and to enable full utilization of arithmetic logic units within the one or more processors.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present disclosure relates generally to digital signal processing, machine learning and neural networks, and more particularly to systems and methods for filtering signals using convolution operations. [Background technology]

[0002] In digital signal processing, a convolution is a weighted sum of an input value and its immediate neighbors in a data set that produces an output value in an output data set. These data sets may be one-dimensional or multi-dimensional with corresponding data structures that determine the values ​​that are immediately neighbors of the input value. The weighting values ​​of the convolution are specified in a convolution kernel, where the kernel has the same dimensionality as the input and output data sets. The convolution operation is useful in a wide variety of signal processing applications and is essential to the operation of many aspects of deep neural networks.

[0003] Convolution is often used widely in machine learning applications. In this context, it is common to apply convolution to a single data set with many small convolution kernels. As a result, convolution is often a computationally intensive part of machine learning applications, and efficient means of performing convolution operations are of increasing value.

[0004] One common approach to improving convolution efficiency, convolution lowering, involves transforming the convolution operation into an equivalent computational process that can be solved by optimized dense linear algebra. This allows the use of optimized General Matrix Multiply (GEMM) routines that are adept at subdividing the computations, exploiting the inherent reuse of data, minimizing the need for off-chip bandwidth, and distributing it across many cores. This approach also suffers from the significant drawback that convolution lowering adds additional processor overhead and significantly increases data size. The increased data size makes caching difficult, increases off-chip bandwidth usage, and leads to solutions that favor expensive, high-performance memory instead of caching and traditional memory techniques.

[0005] The second approach is to perform the convolution in the frequency domain. This requires transforming the input data set and the convolution kernel into the frequency domain, multiplying the two transformed data sets, and applying an inverse frequency transform to the product. This approach offers similar advantages and disadvantages as convolution drop, and the multiplication operation can again be implemented by GEMM routines, but adds additional computational overhead and the memory footprint can increase dramatically for small kernel sizes. In machine learning applications where many small kernels are used, the need to transform many small kernels into the frequency domain makes convolution in the frequency domain unattractive.

[0006] A third approach, commonly referred to as "direct convolution", applies convolution filters directly to the data in the time or spatial domain. This approach avoids the additional computational overhead and memory increase of previous approaches, but suffers from much lower processor utilization, less parallelism and less use of reuse or dedicated hardware just to unwind the convolutions. Typically, such hardware does not leverage existing resources of traditional hardware and is non-trivial to program and optimize for a given deep neural network. Summary of the Invention

[0007] A method and system for providing efficient direct convolution operations using a central processing unit (CPU) that provides vector instructions is described. Efficient direct convolution operations are useful in machine learning and deep neural network applications where convolution operations can require significant memory bandwidth and computational resources. These convolution applications are generally characterized by using smaller convolution kernel sizes, where direct convolution approaches are otherwise memory bandwidth constrained. By implementing direct convolution on the main CPU using the approach disclosed herein, the system can avoid the need for dedicated hardware accelerators or general-purpose GPU (GPGPU) solutions, thus enabling better performance scaling in these applications while fully utilizing computational resources as well as cache and memory bandwidth at the smaller kernel sizes typically used in machine learning applications.

[0008] To increase data-level parallelism in hardware, processors may include vector instructions that provide parallel execution lanes for each instruction executed, such as in Single Instruction Multiple Data (SIMD) microarchitectures. This type of microarchitecture can provide higher throughput for certain applications than single-lane or general-purpose microarchitectures. General-purpose microarchitectures operate on a single lane of data contained in a register, while SIMD microarchitectures operate on data contained in vectors that consist of multiple independent lanes of data.

[0009] In some embodiments, multiple SIMD source vectors are loaded, one source vector aligned with the destination vector to be calculated, one or more source vectors include left portion data with respect to the destination vector, and one or more source vectors include right portion data with respect to the destination vector. Each of these source vectors is multiplied with a respective convolution coefficient vector and added to an accumulator vector. When all source vectors have been processed, the accumulator vector is written to the destination vector.

[0010] In some embodiments, the convolution performed may be a multidimensional convolution. In these embodiments, multiple accumulator vectors may be calculated simultaneously using the same loaded source vector. Different source vectors may be multiplied with other convolution coefficient vectors corresponding to other dimensions of the convolution kernel, and multiple accumulator vectors may not be written until all source vectors for a given destination vector have been processed.

[0011] In some embodiments, a SIMD instruction may be provided to extract an unaligned SIMD vector from a pair of aligned SIMD vectors. In these embodiments, the aligned source vector and multiple shifted source vectors may be loaded from a common data source.

[0012] In some embodiments, a SIMD instruction may be provided to perform a SIMD scalar multi-add operation in which each lane of a data vector is multiplied by a scalar amount and added to each lane of a base vector.

[0013] In some embodiments, convolution operations may be performed simultaneously on multiple input data sets by computing partial resolution output data sets using pre-interleaved source data, in these embodiments, SIMD instructions may be provided to perform de-interlacing operations on the computed data vectors to extract the partial resolution output data vectors.

[0014] The various disclosed embodiments transform direct convolution computations from an operation traditionally limited by memory bandwidth to one that fully utilizes the computational resources and bandwidth of cache and memory subsystems, while only requiring the main CPU; while these embodiments are particularly suited to direct convolution computations, it should be noted that these embodiments may also be applied more generally to a variety of computational tasks, such as, for example, general parallel traversal techniques. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram of one embodiment of a computer system. [Figure 2A] FIG. 2 is a block diagram illustrating a one-dimensional convolution of data elements with a three-element convolution kernel. [Figure 2B] FIG. 2 is a block diagram illustrating a two-dimensional convolution of data elements with a 3×3 element convolution kernel. [Diagram 3] FIG. 1 is a block diagram illustrating left-shifted and right-shifted data streams supporting aligned loading of source vectors for use with a three element convolution kernel, in accordance with at least some embodiments. [Figure 4] 14 illustrates the computation of an output vector of a three-element, one-dimensional convolution using aligned, left-shifted, and right-shifted source vectors with respective kernel vectors, according to at least some embodiments. [Diagram 5] FIG. 1 is a flow diagram illustrating one embodiment of a method for computing a set of output vectors for a dataset that includes a three-element, one-dimensional convolution of an input dataset. [Figure 6A] 1 illustrates an extract vector instruction that generates a data vector that includes a subset of the concatenation of two source vectors, according to at least some embodiments. [Figure 6B]6B illustrates use of the extract vector instruction of FIG. 6A to load a source vector set for a convolution operation with an aligned sequential source input vector set, according to at least some embodiments. [Figure 7] FIG. 6B is a flow diagram illustrating one embodiment of a method for computing a set of output vectors for an input data set comprising a three-element, one-dimensional convolution of the data set using the extract vector instruction of FIG. 6A. [Figure 8] A scalar multi-add vector instruction is shown that derives a scalar value, scales each of the data lanes of a vector by the scalar value to generate a scaled vector, and adds the scaled vector to a base vector to generate an output vector. [Figure 9] FIG. 9 is a flow diagram illustrating one embodiment of a method for computing a series of output vectors for an input data set using the scalar multiply-add vector instruction of FIG. 8 , involving a three element, one dimensional convolution of the data set. [Figure 10] 13 illustrates partial computation of multiple output vectors for a 3×3 element, 2-dimensional convolution with left-shifted, right-shifted source vectors aligned with the respective kernel vectors, according to at least some embodiments. [Figure 11] FIG. 11 is a flow diagram illustrating one embodiment of a method for computing a column of output vectors for an input data set that involves a 3×3 element, two-dimensional convolution of the data set using the scalar multiply-add vector instruction of FIG. 8 and the extract vector instruction of FIG. 6A. [Figure 12A] 1 illustrates one embodiment of a shuffle vector instruction that generates a data vector including shuffled data lanes of a concatenation of multiple interlaced source vectors including a stride factor and a starting source lane. [Figure 12B] 1 illustrates one embodiment of a shuffle vector instruction that generates a data vector including del-interlaced data lanes from an interlaced source vector including a stride factor, a starting source lane, and a starting output lane. [Figure 13]6A and 12A illustrate use of the extract vector instruction of FIG. 6A and the shuffle vector instruction of FIG. 12A to load a set of source vectors for a convolution operation with aligned sequential source input vectors, including a stride value of 2, in accordance with at least some embodiments.

[0016] Although the present disclosure is described herein by way of example with respect to several embodiments and exemplary drawings, those skilled in the art will recognize that the present disclosure is not limited to the described embodiments or drawings. It should be understood that the drawings and detailed description herein are not intended to limit the present disclosure to the particular form disclosed, but rather that the present disclosure is intended to cover all modifications, equivalents, and alternatives within the spirit and scope defined by the appended claims. Any headings used herein are for organizational purposes only and are not intended to limit the scope of the specification or the claims. As used herein, the term "may" is used in an permissive sense (i.e., meaning to have the possibility), rather than a mandatory sense (i.e., must mean), and similarly, the terms "include", "including" and "includes" mean to include, but are not limited to.

[0017] Various units, circuits, or other components may be described as being "configured" to perform a task or tasks. In this context, "configured" is a broad definition of a structure that generally means "having circuitry" to perform a task or tasks during operation. Thus, a unit / circuit / component may be configured to perform a task even if the unit / circuit / component is not currently turned on. In general, the circuitry that forms the structure corresponding to "configured" may include hardware circuitry. Similarly, various units / circuits / components may be described as performing a task or tasks for convenience of description. Such descriptions should be construed to include the term "configured." It is clearly intended that specifying a unit / circuit / component as being configured to perform one or more tasks does not invoke an interpretation of 35 U.S.C. § 112(f) for that unit / circuit / component.

[0018] This specification includes references to "one embodiment" or "an embodiment." The appearance of the phrase "in one embodiment" or "in an embodiment" does not necessarily refer to the same embodiment, but generally contemplates an embodiment including any combination of features unless expressly disclaimed herein. Particular features, structures, or characteristics may be combined in any suitable manner consistent with the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Deep neural networks have become increasingly important in solving a wide range of general problems with large datasets. However, these networks spend a significant amount of processing time on convolutions and matrix multiplications during the training phase to update the neuron weights, with more than twice the processing time per pass. For networks that converge slowly, an even larger number of passes may be required. As a result, efficient convolution computation is becoming increasingly important.

[0020] A method and system are described for providing efficient direct convolution operations using a central processing unit (CPU) that provides vector instructions. Efficient direct convolution operations are useful in deep neural network applications and other applications where convolution operations require significant computational resources. By implementing the convolution in the main CPU, the system can avoid the need for dedicated hardware accelerators or general-purpose GPU (GPGPU) solutions, thus enabling better performance scaling in convolution neural network applications.

[0021] FIG. 1 provides a block diagram of one embodiment of a computer system. Computer system 100 includes processor 110, memory 120, and mass storage device 140 attached to IO bus 130. Processor 110 includes execution core 112, including register file 114, coupled to cache 116. It should be noted that while certain components are shown and described in computer system 100, different components and different numbers of components may be present in alternative embodiments. For example, computer system 100 may not include mass storage device 140 and may include multiple processors similar to processor 110. Additionally, while a single cache 116 is shown internal to processor 110, there may be a hierarchical cache including multiple levels both internal and external to processor 110. Computer system 100 may also include a graphics processor, user interface devices, network cards, and / or other peripheral devices coupled to processor 110 using IO bus 130, a network, or other communication channels.

[0022] In various embodiments, processor 110 may represent a general-purpose processor that performs computational operations. For example, processor 110 may be a central processing unit (CPU), such as a microprocessor, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like. Processor 110 may be a stand-alone component or may be integrated with other components (e.g., other processors, other components of a system on chip (SOC) or FPGA), or may be a component in a multi-chip module (MCM) with other components. Processor 110 may include one or more mechanisms for vector processing (e.g., a vector execution unit).

[0023] As shown in FIG. 1, the processor 110 may include an execution core 112. The execution core 112 may be configured to execute instructions defined in an instruction set architecture implemented by the processor 110. The execution core 112 may have various micro-architectural and implementation features, as desired. For example, the execution core 112 may include a superscalar or scalar implementation. The execution core 112 may include an in-order or out-of-order implementation, and a speculative or non-speculative implementation. The execution core 112 may include any combination of the above features. The implementation may include microcode in some embodiments. The execution core 112 may also include various execution units, each configured to perform various types of operations (e.g., integer, floating point, vector, multimedia, load / store, etc.). The execution core 112 may include a different number of pipeline stages and various other performance enhancing features, such as branch prediction. Execution core 112 may include one or more of an instruction decode unit, a scheduler or reservation station, a reorder buffer, a memory management unit, an I / O interface, and the like.

[0024] Register file 114 may include a set of registers that can be used to store operands for various instructions. Register file 114 may include registers of various data types based on the type of operands execution core 112 is configured to store in the registers (e.g., integer, floating point, multimedia, vector, etc.). Register file 114 may include architected registers (i.e., registers specified in the instruction set architecture implemented by processor 110). Alternatively or additionally, register file 114 may include physical registers (e.g., if register renaming is implemented in execution core 112).

[0025] Cache 116 may exemplify any cache structure. For example, cache 116 may be implemented as a Harvard architecture (separate instruction cache for instruction fetching and data cache for data read / write by execution units for memory reference operations), as a shared instruction, data cache, etc. In some embodiments, a load / store execution unit may be provided for executing memory reference operations.

[0026] An instruction may be an executable entity defined in an instruction set architecture implemented by processor 110. There are a variety of instruction set architectures (e.g., the x86 architecture originally developed by Intel, ARM from ARM Holdings, Power and PowerPC from IBM / Motorola, etc.). Each instruction is defined in an instruction set architecture including its encoding in memory, its operation, and its effect on registers, memory locations, and / or other processor state. A given implementation of an instruction set architecture may execute each instruction directly, although its form may be altered by decoding and other manipulations in the processor hardware. Another implementation may decode at least some instructions into multiple instruction operations for execution by an execution unit in processor 110. Some instructions may be microcoded in some embodiments.

[0027] Mass storage device 140, memory 120, and cache 116 are storage devices that collectively form a memory hierarchy that stores data 124 and instructions 122 for processor 110. More specifically, mass storage device 140 may be large capacity non-volatile memory such as disk drives, large flash memory units, etc., with long access times, while cache 116 and memory 120 are smaller but have shorter access times. These fast semiconductor memories store copies of frequently used data. Memory 120 may represent a memory device in the dynamic random access memory (DRAM) family of memory devices. The size of memory 120 is typically larger than cache 116, while cache 116 is typically implemented using smaller devices in the static random access memory (SRAM) family of devices. In some embodiments, cache 116, memory 120, and mass storage device 140 are shared among one or more processors in computer system 100.

[0028] It should be noted that data and instructions may be stored in a non-transitory computer readable storage device, which may be any device or storage medium capable of storing code and / or data used by a computer system (e.g., computer system 100). In general, non-transitory computer readable storage devices include, but are not limited to, volatile memory, non-volatile memory, disk drives, magnetic tape, compact disks (CDs), digital versatile disks or digital video disks (DVDs), and other media capable of storing computer readable media now known or later developed. Thus, mass storage device 140, memory 120, and cache 116 are all examples of non-transitory computer readable storage media.

[0029] As described above, the execution core 112 may be configured to execute vector instructions. Vector instructions may be defined in the classical sense as single instruction multiple data (SIMD) instructions in that they may define the same operation to be performed on multiple data elements in parallel. The data elements operated on by an instance of an instruction may be referred to as vectors and are organized into data lanes. These data elements may include data of various multi-bit data types (e.g., integer, floating point, multimedia, vector, etc.) that the execution core 112 is configured to store in registers based on the type of operands. For simplicity, the vectors of this disclosure are represented as including eight data lanes, although it should be understood that this is not intended to limit the disclosure to this particular vector configuration. Rather, one skilled in the art will recognize that vectors may be configured to provide any number of data lanes. In some embodiments, there may be 32 vector registers in a vector register file, with each vector register including 128 bits. However, in other embodiments, the number of vector registers may be different and / or the number of bits per register may be different. In general, vector lengths may be implemented as powers of two (e.g., 2, 4, 8, etc.). However, in some embodiments, vector lengths need not be powers of two. Additionally, embodiments that implement register renaming may include any number of physical registers that may be assigned to architected vector registers. An architected register may be a register that can be specified as an operand in a vector instruction.

[0030] In digital signal processing, a convolution is a weighted sum of an input value and its immediate neighbors in a data set that produces an output value in an output data set. These data sets may be one-dimensional or multidimensional with corresponding data structures that determine the neighbors of the input value. The weights of the convolution are specified in the convolution kernel, where the kernel has the same dimensionality as the input and output data sets.

[0031] 2A illustrates a one-dimensional convolution operation 225. In FIG. 2A, data element s2 of source data 200 is convolved with a three-element convolution kernel 210 to generate data element o2 in output data 220. Additionally, data elements s1 and s3 of source data 200 are required for convolution as the immediate left and right neighbors of data element s2. The output data is a weighted sum of the source data elements with respective weighting coefficients k1-k3 of the convolution kernel 210, i.e., o2=k1*s1+k2*s2+k3*s3.

[0032] FIG 2B illustrates a two-dimensional convolution operation 275. In FIG 2B, data element s5 of source data 250 is convolved with a 3x3 element convolution kernel 260 to generate data element o5 in output data 270. Additionally, data elements s1-s4 and s6-s9 of source data 250 are required for convolution as the left, right, top and bottom immediate neighbors of data element s2. The output data is a weighted sum of the source data elements using the respective weighting coefficients k1-k9 of the convolution kernel 260, i.e. o5=k1*s1+k2*s2+k3*s3+k4*s4+k5*s5+k6*s6+k7*s7+k8*s8+k9*s9.

[0033] It should be understood that the examples of one-dimensional and two-dimensional convolutions are not intended to limit the present disclosure to these particular convolution configurations. On the contrary, one skilled in the art will recognize that convolutions may be performed in any number of dimensions. Furthermore, this example is not intended to limit the convolutions to three elements per dimension, and one skilled in the art will recognize that convolutions may be performed in other numbers of elements per dimension.

[0034] FIG. 3 illustrates a method of pre-shifting a source data set according to some embodiments. In addition to aligned data 300, FIG. 3 illustrates left-shifted data 310, which corresponds to a data set 350 with vector alignment shifted one data lane left 360, and right-shifted data 320, which corresponds to a data set 350 with vector alignment shifted one data lane right 370. The three data sets, 300, 310, and 320, correspond to a convolution operation using a convolution kernel consisting of three columns. Such a convolution may be a one-dimensional convolution or a multidimensional convolution. Furthermore, it should be understood that the pre-shifting of a data set is not limited to a three-column convolution. Those skilled in the art will understand that any number of columns may be supported, for example, a five-column convolution may have a total of five source data sets, the three shown in FIG. 3, 300, 310, and 320, as well as a data set shifted twice left and a data set shifted twice right.

[0035] FIG. 4 illustrates the computation of a convolution output vector using a one-dimensional three-element convolution kernel given a pre-shifted source vector. Pre-shifted source vectors 400A, 400B, and 400C may be loaded from a pre-shifted data set as shown in FIG. 3 in some embodiments, but may be loaded using various other methods in other embodiments. Kernel vectors 410A, 410B, and 410C are vector representations of the convolution kernel 210 of FIG. 2A. Each kernel vector is composed of a respective convolution kernel weighting value replicated in all data lanes of the kernel vector. For example, kernel vector 410B corresponds to a medium weighting value of the convolution kernel 210, and thus its weighting value k2 is replicated in all data lanes thereof. Thus, output vector 420 is the sum of source vector 400 multiplied by each kernel vector 410. It should be noted that the convolution computation illustrated in FIG. 4 is not limited to performing a convolution with only three kernel columns, but may be extended to implement any number of kernel columns.

[0036] Figure 5 is a flow diagram illustrating a one-dimensional 3-element convolution of multiple data vectors according to some embodiments. The method begins at step 502 with the initialization of three kernel vectors vKLeft, vKCenter and vKRight, which correspond to kernel vector 410 of Figure 4. These vectors remain constant for the duration of the convolution operation and therefore may be loaded only once.

[0037] The method proceeds to step 504 with loading the aligned and shifted source vectors vLeft, vCenter, and vRight corresponding to vector 400 of Figure 4. In some embodiments, vLeft, vCenter, and vRight may be loaded from a pre-shifted data set as shown in Figure 3, while in other embodiments they may be loaded via other means.

[0038] The method then proceeds to step 506, which performs three weighted sums that are added to the vector accumulator vAcc. Each weighted value is calculated using a multiply-add vector instruction common to many SIMD architectures, but in some embodiments may be implemented using a combination of arithmetic vector instructions. In step 506, the multiply-add vector instruction is represented by a MADD operation that takes three operands: a first operand is a base vector, a second operand is a source vector, and a third operand is a kernel vector. The MADD operation multiplies the second and third operands, then adds the product to the first operand and returns a result vector. In the first MADD operation, a specified base vector vZero corresponds to a vector that contains a value of zero in all data lanes and is common to many SIMD architectures. Using vZero in this manner provides an efficient means of initializing the vector accumulator vAcc, although in some alternative embodiments, in SIMD architectures that do not provide an equivalent vZero vector, the vector accumulator vAcc may first be initialized by an alternative means. For the remaining two MADD operations, the specified base vector is the accumulator vAcc. The three source vectors vLeft, vCenter, and vRight are multiplied by the respective kernel vectors vKLeft, vKCenter, and vKRight and added to the accumulator vAcc.

[0039] Once the MADD operation is complete, the method proceeds to step 508 where the accumulator vector is written to the destination address. The number of remaining vectors is checked in step 510 and the process continues at step 504 until all vectors have been generated.

[0040] While the embodiment disclosed in FIG. 5 performs a one-dimensional convolution with a three element kernel, it should be understood that this embodiment is not intended to be limited to three elements and may include various kernel widths. Although the embodiment of FIG. 5 implements an efficient computation of the weighted sum, it relies on pre-shifted source data as shown in FIG. 3, and for wider kernel widths, the use of these pre-shifted data sets results in greater demands on memory and off-chip bandwidth. For this reason, a vector instruction for deriving the shifted source data is disclosed in FIG. 6A. The use of this vector instruction, when combined with the embodiment of FIG. 5, minimizes off-chip bandwidth requirements, especially at wider kernel widths.

[0041] In some embodiments, an extract vector instruction may be used to generate a shifted data vector required for a convolution calculation using a pair of aligned source vectors. The use of an extract vector instruction allows for the use of the originally provided source data for the convolution operation, thus allowing for efficient use of memory and cache bandwidth. FIG. 6A shows an example of such an extract vector instruction 620. In FIG. 6A, two consecutive source vectors 600A and 600B are concatenated and a subset of the resulting double-wide vector is extracted starting from an offset data lane specified by shift 605 to generate an output vector 610. Optionally, an output width 615 may be specified that allows the output vector to include less than the full vector width of the output data lanes. In this case, the rightmost lane of the output vector 610 includes zeros. In FIG. 6A, a vector width of eight data lanes is shown with a shift 605 value of four and an output width 615 value of six.

[0042] Using the extract vector instruction, various embodiments of the convolution operation may load three aligned source vectors, a center aligned vector, and an aligned vector shifted with the immediate left and immediate right aligned vectors of the center aligned vector for a wide variety of kernel widths.

[0043] FIG. 6B illustrates the loading of source vectors for a one-dimensional three-element convolution as shown in FIG. 4. Three aligned source vectors are loaded: a center source vector 625B, a source vector 625A immediately left of the center vector, and a source vector 625C immediately right of the center vector. An extract left operation 630 is performed using the left source vector 625A and the center source vector 625B and the extract vector instruction 620 of FIG. 6A with a shift value equal to the number of data lanes in the vector, 8 minus 1 in the example embodiment. This operation results in a left vector 650. Similarly, an extract right operation 640 is performed using the center source vector 625B and the right source vector 625C and the extract vector instruction 620 of FIG. 6A with a shift value of 1. This operation results in a right vector 670. The set of vectors 650, 660, and 670 can then be used in a convolution as shown in FIG. 5. Those skilled in the art will appreciate that the method illustrated in Figure 6B is not intended to be limited to 3-element convolution operations, and may be used to generate vectors necessary to perform convolutions of various kernel widths, often without the need to load additional source vectors 625 beyond those illustrated in Figure 6B. Additionally, the use of the extract vector instruction 620 eliminates the need for pre-shifted data as shown in Figure 3, which eliminates an increase in data size for the convolution operation.

[0044] 7 is a flow diagram illustrating a one-dimensional 3-element convolution of multiple data vectors using the extraction method of FIG. 6B according to some embodiments. The convolution of FIG. 7 minimizes the number of loads and stores of vector data to main memory and uses the original source data without pre-computation of the vector data. Furthermore, the sequence of loads and stores to main memory allows for high performance of system caches, resulting in a high performance convolution implementation that balances computational demands with cache and memory bandwidth.

[0045] The method begins at step 702 with the initialization of three kernel vectors vKLeft, vKCenter, and vKRight, which correspond to the kernel vectors 410 in Figure 4. These vectors may be loaded only once, since they remain constant for the duration of the convolution operation. Additionally, a left-aligned source vector vSrcLeft is initialized, and the first source vector is loaded into vCenter.

[0046] The method proceeds to step 704 where the right-aligned source vector vSrcRight is loaded. Since vSrcLeft and vCenter are already loaded, once vSrcRight is loaded, the method proceeds to step 706 where the extract vector instruction 620 is used to generate left-shifted and right-shifted vectors vLeft and vRight for the convolution calculation. In step 706, the extract vector instruction 620 is represented by an EXTRACT operation that takes four operands (the first and second operands are the source vectors, the third operand is the shift value, and the fourth operand is the output width). Step 706 generates left-shifted and right-shifted source vectors vLeft and vRight which, when combined with the center vector vCenter, form the vector used to perform the convolution.

[0047] The method then proceeds to step 708 with three weighted sums added to the vector accumulator vAcc. Each weighted value is calculated using a multiply-add vector instruction common to many SIMD architectures, but in some embodiments may be implemented using a combination of arithmetic vector instructions. In step 708, the multiply-add vector instruction is represented by a MADD operation that takes three operands, the first operand being a base vector, the second operand being a source vector, and the third operand being a kernel vector. The MADD operation multiplies the second and third operands, then adds the product to the first operand and returns a result vector. In the first MADD operation, the specified base vector vZero corresponds to a vector containing a value of 0 in all data lanes and is common to many SIMD architectures. Using vZero in this manner provides an efficient means of initializing the vector accumulator vAcc, although in some alternative embodiments, in SIMD architectures that do not provide an equivalent vZero vector, the vector accumulator vAcc may first be initialized by an alternative means. For the remaining two MADD operations, the specified base vector is the accumulator vAcc. The three source vectors vLeft, vCenter, and vRight are multiplied by the respective kernel vectors vKLeft, vKCenter, and vKRight and added to the accumulator vAcc.

[0048] Once the MADD operation is complete, the method proceeds to step 710 where the accumulator vector is written to the destination address. In step 712, the vCenter and vSrcRight vectors are shifted to become the vSrcLeft and vCenter vectors, respectively, and then in step 714 the number of remaining vectors is checked and the process continues at step 704 until all vectors have been generated.

[0049] In some embodiments, a scalar multiply-add vector instruction may be used. In a conventional multiply-add vector instruction, two vectors are multiplied and the product is added to a third vector to generate an output vector. In the disclosed scalar multiply-add vector instruction 860 shown in FIG. 8, each lane of a source vector 800 is multiplied by a scalar amount specified by a lane 820 of a kernel vector 810 to form a scale vector 830, which is added to a base vector 840 to generate an output vector 850. The use of this vector instruction allows the weighting coefficients of a convolution kernel to be stored compactly in the individual lanes of the vector, rather than using a vector for each weighting coefficient. This reduces the use of the register file 114 and off-chip bandwidth requirements.

[0050] Figure 9 is a flow diagram illustrating a one-dimensional 3-element convolution of multiple data vectors using the scalar multiply-add vector instruction 860 of Figure 8, according to some embodiments. The method begins at step 902 with initialization of a single kernel vector vKernel with weighting coefficients in the various lanes of the vector. This vector may be loaded only once, as it remains constant for the duration of the convolution operation.

[0051] The method proceeds to step 904 with loading the aligned and shifted source vectors vLeft, vCenter, and vRight corresponding to vector 400 of Figure 4. In some embodiments, vLeft, vCenter, and vRight may be loaded from a pre-shifted data set as shown in Figure 3, while in other embodiments they may be loaded via other means, such as using the extract vector instruction 620 and method as shown in Figures 6A and 6B.

[0052] The method then proceeds to step 906, where three weighted sums are performed on the vector accumulator vAcc. Each weighted value is calculated using the scalar multiply-add vector instruction 860 of FIG. 8. In step 906, the scalar multiply-add vector instruction 860 is represented by a SCALAR_MADD operation that takes four operands: a first operand is a base vector, a second operand is a source vector, a third operand is a kernel vector, and a fourth operand is a lane of the kernel vector that includes a scaling factor. The SCALAR_MADD operation multiplies each lane of the second operand by the lanes specified by the third and fourth operands, and then adds the product to the first operand to return a result vector. In the first SCALAR_MADD operation, the specified base vector vZero corresponds to a vector that contains a value of zero in all data lanes and is common to many SIMD architectures. Using vZero in this manner provides an efficient means of initializing the vector accumulator vAcc, however, in some alternative embodiments, in SIMD architectures that do not provide an equivalent vZero vector, the vector accumulator vAcc may first be initialized by alternative means. For the remaining two SCALAR_MADD operations, the specified base vector is the accumulator vAcc. The three source vectors vLeft, vCenter, and vRight are multiplied by their respective kernel scalar values ​​loaded from their respective data lanes of the kernel vector vKernel and added to the accumulator vAcc.

[0053] Once the SCALAR_MADD operation is complete, the method proceeds to step 908 where the accumulator vector is written to the destination address. The number of remaining vectors is checked in step 910 and the process continues at step 904 until all vectors have been generated.

[0054] While the embodiment disclosed in Figure 9 performs a one-dimensional convolution with a three-element kernel, it should be understood that the embodiment is not intended to be limited to three elements and may include various kernel widths. Additionally, while the various embodiments described above implement a one-dimensional convolution operation, it should be understood that the methods discussed may be applied to multidimensional convolution operations as well. For multidimensional convolution, a set of source vectors such as those shown in Figure 4 may be applied to multiple accumulator vectors simultaneously to implement the multidimensional convolution.

[0055] FIG. 10 illustrates partial computation of a convolution output vector using a two-dimensional 3×3 element convolution kernel given a pre-shifted source vector. The pre-shifted source vectors, including left vector 1002, center vector 1004, and right vector 1006, may be loaded from a pre-shifted data set as shown in FIG. 3 in some embodiments, or may be loaded using a variety of other methods, including using the extract vector instructions 620 and methods as shown in FIG. 6A and FIG. 6B. The various weighting coefficients of the 3×3 kernel 1010 may be stored as a kernel vector 410 as shown in FIG. 4 in some embodiments, or may be stored as individual data lanes of the kernel vector for use with the scalar multiply add vector instruction 860 of FIG. 8 in other embodiments. The left vector 1002, center vector 1004, and right vector 1006 have left, center, and right columns, respectively, in the 3×3 kernel 1010, where each column contains three rows of weighting coefficients. Each row of the 3×3 kernel 1010 has a respective accumulator vector, a leading accumulator 1018 for the first row, a central accumulator 1016 for the second row, and a lagging accumulator 1014 for the third row.

[0056] The multiply-add vector instruction is executed three times for each of the source vectors 1002, 1004, and 1006, and once for each of the three accumulator vectors. This results in a total of nine multiply-add vector instructions, one designated for each kernel's weighting coefficient. For each multiply-add vector instruction, the kernel's row and column are selected based on the operation's respective source and accumulator vectors. Thus, the leading accumulator 1018 is a weighted sum of the left vector 1002 using coefficient k1, the middle vector 1004 using coefficient k2, and the right vector 1006 using coefficient k3. Similarly, the middle accumulator 1016 is a weighted sum of the left vector 1002 using coefficient k4, the middle vector 1004 using coefficient k5, and the right vector 1006 using coefficient k6. Finally, the lagging accumulator 1014 is a weighted sum of the left vector 1002 with coefficient k7, the center vector 1004 with coefficient k8, and the right vector 1006 with coefficient k9.

[0057] Figure 10 shows only a partial calculation of the output vectors for a single set of source vectors. For the complete calculation of each output vector, a weighted sum of the vectors immediately above and below each destination vector must also be performed. By iterating the method shown in Figure 10 over the columns of source data, a complete convolution calculation may be performed, loading each source vector, along with its associated left-shift and right-shift vectors, only once.

[0058] 10 illustrates a convolution calculation with a 3×3 convolution kernel for simplicity, but one skilled in the art will recognize that the calculation shown is not limited to either three rows or three columns, but may be used to support other numbers of rows, columns, or rows and columns. Additionally, the calculation shown is not intended to be limited to two-dimensional convolution operations, but may be employed to perform convolutions in greater than two dimensions.

[0059] FIG. 11 is a flow diagram illustrating a portion of a two-dimensional three-element convolution of multiple data vectors using the scalar multiply-add vector instruction 860 of FIG. 8 and the extract vector instruction 620 of FIG. 6A, according to some embodiments. The method assumes that kernel vectors vKernel and vKernel2 have already been initialized with weighting factors. In this example, the vector is assumed to contain eight lanes of data, but nine weighting factors are required, so two kernel vectors are used, although one skilled in the art will recognize that more or fewer kernel vectors may be used depending on the vector and convolution kernel configuration. The method begins with step 1100, which loads a central source vector, vCenter. Once this vector is loaded, it is immediately available for convolution calculations.

[0060] The method then proceeds to step 1102, which performs two functions. First, a left-aligned source vector is loaded. Once this vector is loaded, it may be combined with vCenter to generate a left-shifted vector vLeft. Second, the vCenter vector loaded in the previous step is used to perform a weighted sum calculation using the scalar multiply-add vector instruction 860 of FIG. 8.

[0061] The scalar multiply add vector instruction 860 is represented by a SCALAR_MADD operation that takes four operands (the first operand is a base vector, the second operand is a source vector, the third operand is a kernel vector, and the fourth operand is a lane of the kernel vector that contains scaling factors. The SCALAR_MADD operation multiplies each lane of the second operand by the lanes specified by the third and fourth operands, then adds the products to the first operand and returns a result vector.

[0062] The vTop and vMid accumulator vectors are not initialized since they may have partial results from a previous iteration of the method. However, the vBtm accumulator must be initialized. The base vector vZero specified in the vBtm calculation of step 1102 corresponds to a vector containing zero values ​​in all data lanes and is common to many SIMD architectures. Using vZero in this manner provides an efficient means of initializing the vector accumulator vBtm, although in some alternative embodiments, in SIMD architectures that do not provide an equivalent vZero vector, the vector accumulator may first be initialized by alternative means. Step 1102 calculates three partial sums using the vCenter source vector and three weighting values ​​corresponding to the center column of the 3×3 convolution kernel using a SCALAR_MADD operation.

[0063] The method then proceeds to step 1104, which loads the vSrcRight source vector. Step 1104 also loads the vLeft vector using the already loaded vSrcLeft and vCenter vectors and the extract vector instruction 620 of Figure 6A. The extract vector instruction 620 is represented by an EXTRACT operation that takes four operands: the first and second operands are the source vectors, the third operand is a shift value, and the fourth operand is the output width.

[0064] The method then proceeds to step 1106 where the vRight vector is loaded using the already loaded vCenter and vSrcRight vectors and the extract vector instruction 620 of Figure 6A. Additionally, the vLeft vector loaded in step 1104 is used to perform a weighted sum calculation using the scalar multiply add vector instruction 860 of Figure 8 using three weighting values ​​corresponding to the left column of the 3x3 convolution kernel.

[0065] The method then uses the vRight vector loaded in step 1106 to perform a weighted sum calculation using scalar multiply-add vector instruction 860 of FIG. 8 with the three weighting values ​​corresponding to the right column of the 3×3 convolution kernel.

[0066] The method then proceeds to step 1110 where the partial sums of the source vector have been completed for the source rows. At this point, the accumulator vTop has completed all rows and may be written to the destination vector. The method proceeds to step 1112 where the two remaining accumulators, vMid and vBtm, are shifted to the vTop and vMid positions, respectively, in preparation for the next row of calculation.

[0067] Although the flow diagram of Figure 11 illustrates a convolution calculation with a 3x3 convolution kernel for simplicity, one skilled in the art will recognize that the calculation shown is not limited to either three rows or three columns, but may be used to support other numbers of rows, columns, or rows and columns. Additionally, the calculation shown is not intended to be limited to two-dimensional convolution operations, but may be used to perform convolutions in more than two dimensions.

[0068] Although the various embodiments of the convolution operation discussed herein may be applied to a wide variety of applications, the various embodiments may be particularly useful when applied to machine learning and deep neural networks. In these applications, the convolution step is recognized as a fairly time-intensive and resource-inefficient operation. As a result, processors may be tuned to accelerate this operation. Convolution stages are increasingly being added to a wide range of deep neural networks (DNNs), including convolutional networks (D-CNNs), recurrent neural networks such as LSTMs, general adversarial networks (GANs), and the like. By performing the convolution operation directly using SIMD instructions available on general-purpose CPU cores, efficient or more efficient convolutions may be performed that fit into conventional processor pipelines, thus allowing the use of specialized spatial convolutional neural network hardware while minimizing or eliminating extra hardware. Furthermore, because convolutions in these applications are often employed iteratively on a given data set using many small convolution kernels, this approach allows for better processor utilization since individual convolution operations can be dedicated to individual processor cores, allowing the system to scale well by simply adding additional processor resources. Finally, various embodiments of the SIMD instructions described herein may provide benefits for a wide range of operations beyond convolution.

[0069] In some embodiments, strides may be used to reduce the computational requirements and output size of a convolution operation. In this case, the stride value specifies the number of data elements that can be skipped for every data element computed. For example, if a stride of 2 is employed, all other data elements are skipped and only half of the data elements are computed. For multi-dimensional convolution dimensions, strides can be employed efficiently by skipping rows of data elements, but for one-dimensional convolutions in various embodiments described herein, the stride factor serves to lower the utilization of the arithmetic logic units of the execution core. For this reason, shuffled data sets can be employed to enable the utilization of data lanes that are otherwise unused for vector instructions. In this case, various embodiments of shuffle vector instructions are disclosed that allow shuffled source data vectors to be loaded for implementation of convolutions with a stride value greater than one.

[0070] FIG. 12A shows an example of such a shuffle vector instruction 1240, according to some embodiments. In FIG. 12A, source vectors 1200, such as 1200A, 1200B, are concatenated and a subset of the resulting vector is extracted, starting from source lane 1210 and implementing a stride value specified by stride 1220. The result is written to output vector 1230. The number of source vectors 1200 required is equal to stride 1220, resulting in an output vector that includes data elements in all data lanes. In some embodiments, each of the source vectors 1200 may be specified using an instruction operand. In other embodiments, the source vectors 1200 may be specified using an operand that specifies the range of the vector.

[0071] FIG. 12B illustrates another example of a shuffle vector instruction 1290, according to some embodiments. In FIG. 12B, a fixed number of source vectors 1250, which may include one or more vectors, are concatenated, and a subset of the resulting vectors are extracted, starting from the source lane 1260 and implementing a stride value specified by stride 1265. The result is written to an output vector 1280. Because the number of source vectors 1250 is fixed and may be smaller than the stride 1265, the output vector may only include a contiguous subset of all data lanes. For this reason, an additional operand may be provided to specify the first output lane 1270 to extract the output data lane.

[0072] FIG. 13 illustrates the loading of source vectors for a one-dimensional 3-element convolution with a stride value of 2. Three aligned source vectors are loaded, a center source vector 1300B, a source vector 1300A immediately to the left of the center vector, and a source vector 1300C immediately to the right of the center vector. An extract operation 1310A is performed using the left and center source vectors 1300A and 1300B of FIG. 6A and the extract vector instruction 620 with a shift value equal to the number of data lanes in the vector, 8 minus 1 in the example embodiment. Similarly, an extract operation 1310B is performed using the center and right source vectors 1300B and 1300C and the extract vector instruction 620 of FIG. 6A with a shift value of 1. The extract operations 1310A and 1310B result in intermediate source vectors 1320A and 1320B, which are input to a shuffle operation 1330A. The shuffle operation 1330A employs the shuffle vector instruction 1240 of FIG. 12A with a source lane 1210 value of zero and a stride 1220 value of two to generate a left vector 1340A that includes alternating data lanes of the original source vector 1300 starting with data lane l8 of the source vector 1300A.

[0073] The source vectors 1300B and 1300C are then input to shuffle operations 1330B and 1330C. The shuffle operation 1330B uses the shuffle vector instructions 1240 of FIG. 12A with source lane 1210 values ​​of zero and stride 1220 values ​​of two to generate a center vector 1340B, and the shuffle operation 1330C uses the shuffle vector instructions 1240 of FIG. 12A with source lane 1210 values ​​of one and stride 1220 values ​​of two to generate a right vector 1340B. Thus, the center vector 1340B includes alternating data lanes of the original source vector 1300 starting from data lane c1 of the source vector 1300B, and the right vector 1340C includes alternating data lanes of the original source vector 1300 starting from data lane c2 of the source vector 1300B.

[0074] The set of vectors 1340 may then be used in a convolution as shown in FIG. 5 or in combination with the method shown in FIG. 11 to implement a multidimensional convolution with strides. Those skilled in the art will recognize that the method shown in FIG. 13 is not intended to be limited to a 3-element convolution operation or a stride value of 2, and may be used to generate the vectors necessary to perform convolutions for various kernel widths and stride values ​​without the need to load additional source vectors 1300 beyond those shown in FIG. 13. Those skilled in the art will also recognize that embodiments implementing kernel widths greater than 3 and / or strides greater than 2 may require additional source vectors 1300 beyond those shown in FIG. 13, and that additional extract operations 1310 are required by these particular embodiments. The use of the extract vector instruction 620 and shuffle vector instructions 1240 or 1290 eliminates the need for pre-shifted data as shown in FIG. 3, eliminating the increase in data size for convolution operations implementing the stride feature.

Claims

1. 1. A system comprising: a processor configured to execute a vector instruction to generate an output vector from a first source vector and a second source vector, the output vector, the first source vector and the second source vector each including a plurality of data lanes, and to execute the vector instruction, the processor concatenating the first source vector and the second source vector to generate a combined source vector; extracting the output vector from the combined source vectors starting at a data lane offset greater than 0, the output vector including at least one data lane of the first source vector and at least one data lane of the second source vector; and writing the extracted output vector to a vector register of the processor; To extract the output vector from the combined source vector starting at the data lane offset, the processor: extracting a number of data lanes according to an extraction width identified as an operand of the vector instruction, the extraction width being less than a number of data lanes of the vector register; and adding a rightmost data lane number, each containing a zero value, to the output vector to generate an extracted output vector having a number of data lanes that matches the number of data lanes in the vector register.

2. 2. The system of claim 1, wherein the first source vector and the second source vector are accessed from respective source vector registers of the processor identified by respective operands of the vector instruction.

3. The system of claim 1 or 2, wherein the data lane offset is identified as an operand of the vector instruction.

4. The vector instruction is executed as part of performing a convolution on a vector, the output vector being a left source vector of the convolution and the second source vector being a center source vector of the convolution, and to perform the convolution the processor loading a plurality of source vector registers with each source vector of the vectors, including the first source vector, the second source vector, and a third source vector; concatenating the second source vector and the third source vector to generate another combined source vector; extracting a right source vector from the other combined source vector starting at another data lane offset, the right source vector including at least one data lane of the second source vector and at least one data lane of the third source vector; writing the right source vector to another vector register of the processor; loading a kernel vector including a plurality of weighting values ​​including a center weighting value, a left weighting value and a right weighting value; generating one or more output vectors, each of the output vectors comprising a weighted sum of a plurality of source vectors including the left source vector, the center source vector, and the right source vector, wherein to generate a particular one of the one or more output vectors, the processor: loading a scalar weighting value from a lane of the kernel vector, the lane being assigned to a particular vector instruction according to the particular output vector, and a particular source vector of the plurality of source vectors being assigned as an operand of the particular vector instruction; scaling each of a plurality of data lanes of the particular source vector by the loaded scalar weighting value to generate a scaled vector; adding the scaled vector to an accumulator vector stored in a vector accumulator to generate a new output vector; and storing the new output vector in the vector accumulator.

5. The system of claim 4 , wherein the different data lane offset is determined according to the data lane offset and a number of data lanes of the vector register.

6. 5. The system of claim 4, wherein the processor is configured to perform the convolution as part of a convolutional neural network.

7. 1. A method comprising: Executing, by a processor, a vector instruction to generate an output vector from a first source vector and a second source vector, the output vector, the first source vector, and the second source vector each including a plurality of data lanes, the executing comprising: concatenating the first source vector and the second source vector to generate a combined source vector; extracting the output vector from the combined source vector starting at a data lane offset, the output vector including at least one data lane of a first source vector and at least one data lane of a second source vector; writing the extracted output vector to a vector register of the processor; Extracting the output vector from the combined source vector starting at the data lane offset comprises: extracting a number of data lanes according to an extraction width identified as an operand of the vector instruction, the extraction width being less than a number of data lanes of the vector register; and adding a rightmost number of data lanes that contain zero values ​​to the output vector to generate an extracted output vector having a number of data lanes that matches the number of data lanes in the vector register.

8. 8. The method of claim 7, wherein the first source vector and the second source vector are accessed from respective source vector registers of the processor identified by respective operands of the vector instruction.

9. The method of claim 7 or 8, wherein the data lane offset is identified as an operand of the vector instruction.

10. The vector instruction is executed as part of performing a convolution on a vector, the output vector being a left source vector of the convolution and the second source vector being a center source vector of the convolution, and performing the convolution comprises: loading a plurality of source vector registers with each source vector of the vectors, including the first source vector, the second source vector, and a third source vector; executing another vector instruction to generate a right source vector from the second source vector and the third source vector; concatenating the second source vector and the third source vector to generate another combined source vector; extracting the right source vector from another combined source vector starting at another data lane offset, the right source vector including at least one data lane of the second source vector and at least one data lane of the third source vector; writing the right source vector to another vector register of the processor; loading a kernel vector including a plurality of weighting values ​​including a center weighting value, a left weighting value and a right weighting value; generating one or more output vectors each including a weighted sum of a plurality of source vectors including the left source vector, the center source vector, and the right source vector, wherein generating a particular one of the one or more output vectors includes: loading a scalar weighting value from a lane of the kernel vector, the lane being assigned to a particular vector instruction according to the particular output vector, and a particular source vector of the plurality of source vectors being assigned as an operand of the particular vector instruction; scaling each of a plurality of data lanes of the particular source vector by the loaded scalar weighting value to generate a scaled vector; adding the scaled vector to an accumulator vector stored in a vector accumulator to generate a new output vector; storing the new output vector in the vector accumulator; and executing the particular vector instruction.

11. The method of claim 10 , wherein the different data lane offset is determined according to the data lane offset and a number of data lanes of the vector register.

12. The method of claim 10 , wherein the processor is configured to perform the convolution as part of a convolutional neural network.

13. One or more non-transitory computer-accessible storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to: causing a processor to execute a vector instruction to generate an output vector from a first source vector and a second source vector, the output vector, the first source vector, and the second source vector each including a plurality of data lanes, the executing comprising: concatenating the first source vector and the second source vector to generate a combined source vector; extracting the output vector from the combined source vector starting at a data lane offset, the output vector including at least one data lane of a first source vector and at least one data lane of a second source vector; writing the extracted output vector to a vector register of the processor; Extracting the output vector from the combined source vector starting at the data lane offset comprises: extracting a number of data lanes according to an extraction width identified as an operand of the vector instruction, the extraction width being less than a number of data lanes of the vector register; and adding a rightmost data lane number, each including a zero value, to the output vector to generate an extracted output vector having a number of data lanes that matches the number of data lanes in the vector register.

14. 14. The one or more non-transitory computer-accessible storage media of claim 13, wherein the first source vector and the second source vector are accessed from respective source vector registers of the processor identified by respective operands of the vector instruction.

15. 15. The one or more non-transitory computer-accessible storage media of claim 13 or 14, wherein the data lane offset is identified as an operand of the vector instruction.

16. The vector instruction is executed as part of performing a convolution on a vector, the output vector being a left source vector of the convolution and the second source vector being a center source vector of the convolution, and performing the convolution comprises: loading a plurality of source vector registers with each source vector of the vectors, including the first source vector, the second source vector, and a third source vector; executing another vector instruction to generate a right source vector from the second source vector and the third source vector; concatenating the second source vector and the third source vector to generate another combined source vector; extracting the right source vector from another combined source vector starting at another data lane offset, the right source vector including at least one data lane of the second source vector and at least one data lane of the third source vector; writing the right source vector to another vector register of the processor; loading a kernel vector including a plurality of weighting values ​​including a center weighting value, a left weighting value and a right weighting value; generating one or more output vectors each including a weighted sum of a plurality of source vectors including the left source vector, the center source vector, and the right source vector, wherein generating a particular one of the one or more output vectors includes: loading a scalar weighting value from a lane of the kernel vector, the lane being assigned to a particular vector instruction according to the particular output vector, and a particular source vector of the plurality of source vectors being assigned as an operand of the particular vector instruction; scaling each of a plurality of data lanes of the particular source vector by the loaded scalar weighting value to generate a scaled vector; adding the scaled vector to an accumulator vector stored in a vector accumulator to generate a new output vector; Storing the new output vector in the vector accumulator.

17. 20. The one or more non-transitory computer-accessible storage media of claim 16, wherein the processor is configured to perform the convolution as part of a convolutional neural network.

Images