Efficient direct convolution using SIMD instructions

Abstract

The invention relates to efficient direct convolution using SIMD instructions. A computer including one or more processors providing vector instructions may implement direct convolution on a source data set. The source data set may be one-dimensional or multi-dimensional. For a given vector width w of the vector instruction, w consecutive data elements of the output data set are parallel-computed using the vector instruction. For a multi-dimensional data set, multiple vectors of an output data set are calculated for a single load of a set of vectors from a source data set. New vector instructions are disclosed to improve the performance of convolution and make full use of arithmetic logic units in one or more processors.

Description

[0001] This application is a divisional application of the invention patent application with application number 201880066852.7, filed on September 6, 2018, and invention name “Efficient direct convolution using SIMD instructions”. Technical Field

[0002] The present disclosure relates generally to digital signal processing, machine learning, and neural networks, and more particularly to systems and methods for filtering signals using convolution operations. Background Art

[0003] In digital signal processing, a convolution is a weighted sum of an input value and its immediate neighbors in a data set, producing an output value for an output data set. These data sets can be one-dimensional or multi-dimensional, where the corresponding data organization determines the values ​​that are the immediate neighbors of the input value. The weighted values ​​for the convolution are specified in a convolution kernel, where the kernel processes the same number of dimensions as the input and output data sets. The convolution operation is useful in a wide variety of signal processing applications and is an integral part of the operation of many aspects of deep neural networks.

[0004] Convolutions can be used extensively in machine learning applications. In this context, convolutions are usually applied to a single dataset using many small convolution kernels. As a result, convolutions are often a computationally intensive part of machine learning applications, so there is increasing value in performing convolution operations in an efficient manner.

[0005] A popular approach to improving the efficiency of convolutions is to reduce the convolution, converting the convolution operation into an equivalent computational process that can be solved by optimized dense linear algebra. This allows the use of optimized general matrix multiplication (GEMM) routines that are good at taking advantage of the inherent reuse of data, minimizing off-chip bandwidth requirements, and subdividing calculations that can be distributed across many cores. This approach also has significant disadvantages. Reduced convolution adds additional processor overhead and significantly increases data size. The increase in data size makes it difficult to cache and increases the use of off-chip bandwidth, resulting in solutions that tend to use expensive high-performance memories instead of caches and conventional memory technologies.

[0006] The second approach is to perform the convolution in the frequency domain. This requires transforming the input dataset and the convolution kernel separately to the frequency domain, multiplying the two transformed datasets together, and applying the inverse frequency transform to the product. This approach has similar advantages and disadvantages to reduced convolution, the multiplication operation can again be implemented by the GEMM routine, but with the added computational overhead, and the memory footprint can increase dramatically for small kernel sizes. In machine learning applications that use many small kernels, convolution in the frequency domain is unpopular due to the need to transform many small kernels to the frequency domain.

[0007] The third approach, often called "direct convolution", applies the convolution filter directly to the data in the time or spatial domain. This approach avoids the additional computational overhead and memory growth of the previous approaches, but with much lower processor utilization, reduced parallelism and reuse, or hardware dedicated only to solving the convolution problem. Typically, such hardware does not take advantage of existing resources of conventional hardware and is non-trivial to program and optimize for a given deep neural network. Summary of the invention

[0008] Methods and systems are described for providing efficient direct convolution operations using a central processing unit (CPU) that provides vector instructions. Efficient direct convolution operations are useful in machine learning and deep neural network applications, where convolution operations may require a large amount of memory bandwidth and computational resources. These convolution applications are often characterized by the use of smaller convolution kernel sizes, otherwise direct convolution methods will be limited by memory bandwidth. By implementing direct convolution on a host CPU using the methods disclosed herein, the system can avoid the need for dedicated hardware accelerators or general purpose GPU (GPGPU) solutions, thereby achieving better performance scaling in these applications while fully utilizing computational resources and cache and memory bandwidth in smaller kernel sizes typically used for machine learning applications.

[0009] To increase data-level parallelism in hardware, processors can include vector instructions that provide parallel execution channels for each instruction executed in a microarchitecture such as a single instruction multiple data (SIMD). This type of microarchitecture can provide higher throughput than a single channel or general microarchitecture for certain applications. While general microarchitectures operate on a single channel of data contained in a register, SIMD microarchitectures operate on data contained in a vector consisting of multiple independent data channels.

[0010] In some embodiments, multiple SIMD source vectors are loaded, one source vector is aligned with the destination vector to be calculated, one or more source vectors include the left half data relative to the destination vector, and one or more source vectors include the right half data relative to the destination vector. Each of these source vectors is multiplied with a respective convolution coefficient vector and then added to the accumulator vector. Once all source vectors have been processed, the accumulator vector is written to the destination vector.

[0011] In some embodiments, the convolution to be performed may be a multi-dimensional convolution. In these embodiments, multiple accumulator vectors may be calculated simultaneously using the same loaded source vector. Various source vectors may be multiplied by other convolution coefficient vectors corresponding to other dimensions of the convolution kernel, and the multiple accumulator vectors may not be written until all source vectors for a given destination vector have been processed.

[0012] In some embodiments, SIMD instructions may be provided to extract unaligned SIMD vectors from aligned SIMD vector pairs. In these embodiments, an aligned source vector and a plurality of shifted source vectors may be loaded from a common data source.

[0013] In some embodiments, SIMD instructions may be provided to perform a SIMD scalar multiply-add operation, where each lane of a data vector is multiplied by a scalar and added to a corresponding lane of a basis vector.

[0014] In some embodiments, convolution operations may be performed on multiple input data sets simultaneously by computing partially resolved output data sets using pre-interleaved source data. In these embodiments, SIMD instructions may be provided to perform de-interleaving operations on the computed data vectors to extract partially resolved output data vectors.

[0015] The various disclosed embodiments convert the computation of direct convolution from an operation that is traditionally limited by memory bandwidth to an operation that fully utilizes the computational resources and bandwidth of the cache and memory subsystem of the main CPU, and these embodiments are particularly suitable for direct convolution computation. It should be noted that they can be more generally applied to various computing tasks, for example, to general parallel traversal techniques. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a block diagram of one embodiment of a computer system.

[0017] Figure 2A is a block diagram illustrating a one-dimensional convolution of data elements with a three-element convolution kernel.

[0018] Figure 2B is a block diagram illustrating a two-dimensional convolution of data elements with a 3x3 element convolution kernel.

[0019] Figure 3 is a block diagram illustrating a data flow supporting left and right shifting of aligned loading of a source vector for use with a three-element convolution kernel in accordance with at least some embodiments.

[0020] Figure 4 Illustrated is the computation of an output vector of a three-element one-dimensional convolution using aligned, left-shifted, and right-shifted source vectors and respective kernel vectors, in accordance with at least some embodiments.

[0021] Figure 5 is a flow chart illustrating one embodiment of a method for computing a series of output vectors for a data set comprising a three-element one-dimensional convolution of an input data set.

[0022] Fig. 6AAn Extract vector instruction is illustrated that generates a data vector comprising a subset of a concatenation of two source vectors, in accordance with at least some embodiments.

[0023] Figure 6B Illustrated is a method for using a Fig. 6A The Extract Vector instruction of to load a set of source vectors for a convolution operation using a set of aligned, sequential source input vectors.

[0024] Figure 7 is a diagram for computing a series of output vectors for a data set comprising a three-element one-dimensional convolution of an input data set and using Fig. 6A A flow chart of one embodiment of a method of extracting vector instructions.

[0025] Figure 8 Illustrated is a scalar multiply-add vector instruction that derives a scalar value, scales each data lane of a vector by the scalar value to produce a scaled vector, and adds the scaled vector to a basis vector to generate an output vector.

[0026] Fig. 9 is a diagram for computing a series of output vectors for a data set comprising a three-element one-dimensional convolution of an input data set and using Figure 8 A flow chart of one embodiment of a method of performing a scalar multiply-add vector instruction.

[0027] Fig.10 Illustrated is a partial computation of multiple output vectors of a two-dimensional convolution of 3x3 elements using aligned, left-shifted, and right-shifted source vectors and respective kernel vectors, in accordance with at least some embodiments.

[0028] Fig.11 is a column diagram of the output vector of a data set for computing a two-dimensional convolution of 3x3 elements of the input data set and using Figure 8 Scalar multiply-add vector instructions and Fig. 6A A flow chart of one embodiment of a method of extracting vector instructions.

[0029] Fig. 12A One embodiment of a Shuffle Vector instruction is illustrated that generates a data vector comprising a concatenation of a plurality of interleaved source vectors that include a stride factor and a starting source lane.

[0030] Fig. 12B One embodiment of a shuffle vector instruction is illustrated that generates a data vector including del-interlaced data lanes from an interleaved source vector including a stride factor, a starting source lane, and a starting output lane.

[0031] Fig.13 Illustrated is a method of using a Fig. 6A Extract vector instructions and Fig. 12A The shuffle vector instruction loads a set of source vectors with a stride value of 2 and a convolution operation with a set of aligned sequential source input vectors.

[0032] Although several embodiments and illustrative drawings are described herein by way of example, those skilled in the art will recognize that the present disclosure is not limited to the described embodiments or drawings. It should be understood that the drawings and detailed description herein are not intended to limit the present disclosure to the specific forms disclosed, but on the contrary, the present disclosure will cover all modifications, equivalents and alternative forms falling within the spirit and scope defined by the appended claims. Any title used herein is only for organizational purposes and is not meant to limit the scope of the specification or claims. As used herein, the word "may" is used in a permissive sense (i.e., meaning that it is possible) rather than a mandatory sense (i.e., meaning that it must). Similarly, the word "include" means including but not limited to.

[0033] Various units, circuits, or other components may be described as being "configured to" perform one or more tasks. In this context, "configured to" is a broad description of a structure, generally referring to a "circuit system" that "has" the ability to perform one or more tasks during operation. Thus, a unit / circuit / component may be configured to perform a task even if the unit / circuit / component is not currently turned on. In general, the circuit system that forms the structure corresponding to "configured to" may include hardware circuits. Similarly, for ease of description, various units / circuits / components may be described as performing one or more tasks. Such a description should be interpreted as including the phrase "configured to". The description of a unit / circuit / component that is configured to perform one or more tasks is explicitly not intended to invoke 35 U.S.C. §112(f) for the interpretation of that unit / circuit / component.

[0034] This description includes references to "one embodiment" or "an embodiment." The appearances of the phrases "in one embodiment" or "in an embodiment" do not necessarily refer to the same embodiment, but generally contemplate embodiments that include any combination of features unless expressly stated otherwise herein. The particular features, structures, or characteristics may be combined in any suitable manner consistent with the present disclosure. DETAILED DESCRIPTION

[0035] Deep neural networks have become increasingly important in solving a variety of general-purpose problems associated with large datasets. However, these networks spend a lot of processing time in convolutions and matrix multiplications, and during the training phase, the processing time more than doubles for each pass to update the neuron weights. Many passes may be required to slowly converge the network. Therefore, efficient convolution computation has become increasingly important.

[0036] Methods and systems are described for providing efficient direct convolution operations using a central processing unit (CPU) that provides vector instructions. Efficient direct convolution operations are useful in deep neural network applications and other applications where convolution operations may require significant computational resources. By implementing convolution on the main CPU, the system can avoid the need for dedicated hardware accelerators or general purpose GPU (GPGPU) solutions, thereby achieving better performance scaling in convolutional neural network applications.

[0037] Figure 1 A block diagram of one embodiment of a computer system is provided. Computer system 100 includes processor 110, memory 120, and mass storage device 140 attached to IO bus 130. Processor 110 includes execution core 112 coupled to cache 116, which includes register file 114. Note that although specific components are shown and described in computer system 100, different components and numbers of components may be present in alternative embodiments. For example, computer system 100 may not include mass storage device 140, but may include multiple processors similar to processor 110. In addition, although a single cache 116 is shown inside processor 110, there may be multiple levels of hierarchical caches including internal and external to processor 110. Computer system 100 may also include a graphics processor, a user interface device, a network card, and / or other peripherals coupled to processor 110 using IO bus 130, a network, or other communication channels.

[0038] In various embodiments, processor 110 may represent a general purpose processor that performs computing operations. For example, processor 110 may be a central processing unit (CPU), such as a microprocessor, a microcontroller, an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA). Processor 110 may be a standalone component, or may be integrated with other components (e.g., other processors or other components in a system on a chip (SOC) or FPGA), or may be a component in a multi-chip module (MCM) with other components. Processor 110 may include one or more mechanisms for vector processing (e.g., a vector execution unit).

[0039] like Figure 1As shown, the processor 110 may include an execution core 112. The execution core 112 may be configured to execute instructions defined in an instruction set architecture implemented by the processor 110. As needed, the execution core 112 may have various micro-architecture features and implementation features. For example, the execution core 112 may include a superscalar or scalar implementation. The execution core 112 may include an in-order or out-of-order implementation, as well as a speculative or non-speculative implementation. The execution core 112 may include any combination of the above features. In some embodiments, the implementation may include microcode. The execution core 112 may also include various execution units, each of which is configured to perform various types of operations (e.g., integer, floating point, vector, multimedia, load / store, etc.). The execution core 112 may include different numbers of pipeline stages and various other performance enhancement features, such as branch prediction. The execution core 112 may include one or more of an instruction decoding unit, a scheduler or reservation station, a reorder buffer, a memory management unit, an I / O interface, etc.

[0040] Register file 114 may include a set of registers that may be used to store operands for various instructions. Register file 114 may include registers of various data types based on the type of operands that execution core 112 is configured to store in registers (e.g., integer, floating point, multimedia, vector, etc.). Register file 114 may include architectural registers (i.e., those registers specified in the instruction set architecture implemented by processor 110). Alternatively or additionally, register file 114 may include physical registers (e.g., if register renaming is implemented in execution core 112).

[0041] Cache 116 may exemplify any cache structure. For example, cache 116 may be implemented as a Harvard architecture (a separate instruction cache for instruction fetch and a data cache for data read / write by an execution unit for memory reference operations), a shared instruction and data cache, etc. In an embodiment, a load / store execution unit may be provided to perform memory reference operations.

[0042] Instructions may be executable entities defined in an instruction set architecture implemented by the processor 110. There are a variety of instruction set architectures (e.g., x86 architecture originally developed by Intel, ARM Holdings' ARM, Power and PowerPC from IBM / Motorola, etc.). Each instruction is defined in an instruction set architecture, including its encoding in memory, its operation, and its effect on registers, memory locations, and / or other processor states. A given implementation of an instruction set architecture may directly execute each instruction, but its form may be changed by decoding and other manipulations in the processor hardware. Another implementation may decode at least some instructions into multiple instruction operations to be executed by an execution unit in the processor 110. In some embodiments, some instructions may be microcoded.

[0043] The mass storage device 110, memory 120, and cache 116 are storage devices that together form a memory hierarchy that stores data 124 and instructions 122 for the processor 110. More specifically, the mass storage device 110 can be a high-capacity non-volatile memory, such as a disk drive or a large flash memory unit with a long access time, while the cache 116 and memory 120 can be smaller, but have a shorter access time. These faster semiconductor memories store copies of frequently used data. The memory 120 can represent a memory device in the dynamic random access memory (DRAM) family of memory devices. The size of the memory 120 is generally larger than the cache 116, which is generally implemented using a smaller device in the static random access memory (SRAM) family of devices. In some embodiments, the cache 116, memory 120, and mass storage device 110 are shared between one or more processors in the computer system 100.

[0044] Note that data and instructions may be stored on a non-transitory computer-readable storage device, which may be any device or storage medium that can store code and / or data for use by a computer system (e.g., computer system 100). In general, non-transitory computer-readable storage devices include, but are not limited to, volatile memory, non-volatile memory, magnetic and optical storage devices such as disk drives, magnetic tapes, compact disks (CDs), digital versatile disks or digital video disks (DVDs), or other media now known or later developed that are capable of storing computer-readable media. Thus, mass storage device 110, memory 120, and cache 116 are all examples of non-transitory computer-readable storage media.

[0045] As described above, the execution core 112 can be configured to execute vector instructions. Vector instructions can be defined as single instruction multiple data (SIMD) instructions in the traditional sense, because they can define the same operation to be performed in parallel on multiple data elements. The data elements operated by the instance of the instruction can be referred to as vectors and organized in data channels. These data elements can contain data of various multi-bit data types based on the type of operands (e.g., integers, floating point numbers, multimedia, vectors, etc.) that the execution core 112 is configured to store in registers. For simplicity, the vectors in the present disclosure are represented as including eight data channels, but it should be understood that this is not intended to limit the present disclosure to this particular carrier organization. On the contrary, those skilled in the art will recognize that vectors can be configured to provide any number of data channels. In some embodiments, there can be 32 vector registers in the vector register file, and each vector register can include 128 bits. However, in alternative embodiments, there can be different numbers of vector registers and / or different bit numbers for each register. In general, the vector length can be implemented as a power of two (e.g., two, four, eight, etc.). However, in some embodiments, the vector length does not have to be a power of two. Furthermore, embodiments implementing register renaming may include any number of physical registers that may be allocated to architectural vector registers.An architectural register may be a register that may be specified as an operand in a vector instruction.

[0046] In digital signal processing, convolution is the weighted sum of an input value and its immediate neighbors in a data set, producing an output value in an output data set. These data sets can be one-dimensional or multi-dimensional, where the corresponding data organization determines the values ​​that are the immediate neighbors of the input value. The weights of the convolution are specified in the convolution kernel, where the convolution has the same number of dimensions as the input and output data sets.

[0047] Figure 2A A one-dimensional convolution operation 225 is depicted. Figure 2A , data element s2 of source data 200 will be convolved with three-element convolution kernel 210 to produce data element o2 in output data 220. In addition, the convolution requires data elements s1 and s3 of source data 200 as the immediate left and right neighbors of data element s2. The output data is the weighted sum of the source data elements with the respective weighting factors k1-k3 of convolution kernel 210, in other words, o2=k1*s1+k2*s2+k3*s3.

[0048] Figure 2B A two-dimensional convolution operation 275 is depicted. Figure 2B, data element s5 of source data 250 will be convolved with the 3x3 element convolution kernel 260 to produce data element o5 in output data 270. In addition, the convolution requires data elements s1-s4 and s6-s9 of source data 250 as the immediate neighbors to the left, right, above, and below of data element s2. The output data is the weighted sum of the source data elements with the respective weighting factors k1-k9 of the convolution kernel 260. In other words,

[0049] o5=k1*s1+k2*s2+k3*s3+k4*s4+k5*s5+k6*s6+k7*s7+k8*s8+k9*s9.

[0050] It should be understood that the examples of one-dimensional and two-dimensional convolutions are not intended to limit the present disclosure to these specific convolutional organizations. Instead, those skilled in the art will recognize that convolutions can be performed in any number of dimensions. Furthermore, the examples are not intended to limit convolutions to three elements per dimension, and those skilled in the art will recognize that convolutions can be performed with other numbers of elements per dimension.

[0051] Figure 3 A method of pre-shifting a source data set according to some embodiments is illustrated. In addition to aligning the data 300, Figure 3 Also shown are left-shifted data 310 corresponding to a data set 350 in which the vector alignment has been shifted to the left by one data channel 360 and right-shifted data 320 corresponding to a data set 350 in which the vector alignment has been shifted to the right by one data channel 370. The three data sets 300, 310, and 320 correspond to convolution operations using a convolution kernel consisting of three columns. Such a convolution can be a one-dimensional convolution or a multi-dimensional convolution. In addition, it should be understood that the pre-shifting of the data sets is not limited to convolutions of three columns. Those skilled in the art will recognize that any number of columns can be supported, for example, a five-column convolution can have a total of five source data sets, i.e., Figure 3 300, 310 and 320 shown in FIG. 1 , as well as a double left-shifted data set and a double right-shifted data set.

[0052] Figure 4 The calculation of the convolution output vector using a one-dimensional three-element convolution kernel given a pre-shifted source vector is illustrated. In some embodiments, the convolution output vector may be obtained from, for example, Figure 3 The pre-shifted data set shown in FIG. 1 loads the pre-shifted source vectors 400A, 400B, and 400C, but in other embodiments, various other methods may be used to load the pre-shifted source vectors. Kernel vectors 410A, 410B, and 410C are Figure 2A410B corresponds to the middle weight value of the convolution kernel 210 and therefore has the weight value k2 copied to all its data channels. Therefore, the output vector 420 is the sum of the source vectors 400 multiplied by their respective kernel vectors 410. Figure 4 The convolution computation shown is not limited to performing convolutions involving only three kernel columns, but can be extended to implement any number of kernel columns.

[0053] Figure 5 is a flowchart illustrating a one-dimensional three-element convolution of multiple data vectors according to some embodiments. The method begins at step 502, which initializes the corresponding Figure 4 The three kernel vectors vKLeft, vKCenter and vKRight of the kernel vector 410. Since these vectors remain unchanged during the convolution operation, they can only be loaded once.

[0054] The method proceeds to step 504, which loads Figure 4 The aligned and shifted source vectors vLeft, vCenter, and vRight corresponding to the vector 400 in FIG. In some embodiments, vLeft, vCenter, and vRight may be obtained from, for example, Figure 3 The pre-shifted data sets shown in are loaded, while in other embodiments they may be loaded by other means.

[0055] The method then proceeds to step 506, which performs three weighted sums added to the vector accumulator vAcc. Each weighted value is calculated using a multiply-add vector instruction common to many SIMD architectures, but in some embodiments, it can be implemented using a combination of arithmetic vector instructions. In step 506, the multiply-add vector instruction is represented by a MADD operation that takes three operands: the first operand is a basis vector, the second operand is a source vector, and the third operand is a kernel vector. The MADD operation multiplies the second and third operands, then adds the product to the first operand, and returns a result vector. In the first MADD operation, the specified basis vector vZero corresponds to a vector containing 0 values ​​in all data channels and is common to many SIMD architectures. Using vZero in this way provides an efficient means of initializing the vector accumulator vAcc, but in some alternative embodiments, for SIMD architectures that do not provide an equivalent vZero vector, the vector accumulator vAcc can first be initialized by alternative means. In the remaining two MADD operations, the specified basis vector is the accumulator vAcc. The three source vectors vLeft, vCenter, and vRight are multiplied by their respective kernel vectors vKLeft, vKCenter, and vKRight, respectively, and then added to the accumulator vAcc.

[0056] Once the MADD operation is complete, the method proceeds to step 508 where the accumulator vector is written to the destination address. The number of remaining vectors is checked in step 510 and the process continues at step 504 until all vectors have been generated.

[0057] Although Figure 5 The embodiment disclosed in performs a one-dimensional convolution using a three-element kernel, but it should be understood that the embodiment is not intended to be limited to three elements, but may include various kernel widths. Figure 5 The embodiment of achieves efficient computation of the weighted sum, but it relies on pre-shifted source data, such as Figure 3 , and for wider core widths, using these pre-shifted data sets results in greater demands on memory and off-chip bandwidth. For this reason, Fig. 6A The vector instructions for deriving the source data for the shift are disclosed in Figure 5 When used in conjunction with an embodiment of the present invention, the use of the vector instructions minimizes off-chip bandwidth requirements, especially when the core width is wide.

[0058] In some embodiments, the extract vector instruction can be used to generate the shifted data vector required for the convolution calculation using a pair of aligned source vectors. The use of the extract vector instruction allows the use of the source data originally provided for the convolution operation, thereby enabling efficient use of memory and cache bandwidth. Fig. 6A An example of such an extract vector instruction 620 is shown. Fig. 6A , two consecutive source vectors 600A and 600B are concatenated and a subset of the resulting double-width vector is extracted starting from the offset data channel specified by shift 605 to produce output vector 610. Optionally, an output width 615 may be specified that allows the output vector to contain less than the full vector width of the output data channel. In this case, the rightmost channel of output vector 610 will contain zeros. Fig. 6A , a vector width for eight data channels is shown, with a shift 605 value of four and an output width 615 value of six.

[0059] Using the extract vector instruction, various embodiments of the convolution operation may load shifted source vectors for various kernel widths using only three aligned source vectors, a center-aligned vector, and aligned vectors to the immediate left and immediate right of the center-aligned vector.

[0060] Figure 6B The diagram shows the Figure 4 Loading of source vectors for a one-dimensional three-element convolution as shown. Load three aligned source vectors, namely the center source vector 625B, the source vector immediately to the left of the center vector 625A, and the source vector immediately to the right of the center vector 625C. Using the left and center source vectors 625A and 625B and Fig. 6A Extract vector instruction 620 performs an Extract Left operation 630 using an offset value equal to the number of data lanes in the vector (eight in this example) minus one. This operation produces a left vector 650. Similarly, using center and right source vectors 625B and 625C and Fig. 6A Extract vector instruction 620 performs an Extract Right operation 640 using a shift value of 1. This operation produces a right vector 670. This can then be used in a program such as Figure 5 The set of vectors 650, 660 and 670 are used in the convolution shown in FIG. Those skilled in the art will recognize that Figure 6B The method shown in is not intended to be limited to three-element convolution operations, but can be used to generate the vectors needed to perform convolutions of various kernel widths, generally without loading anything other than Figure 6B 625. In addition, the use of the extract vector instruction 620 eliminates the need for instructions such as Figure 3 The need for pre-shifting the data as shown eliminates the growth in data size for the convolution operation.

[0061] Figure 7 is to demonstrate the use of some embodiments Figure 6B Flowchart of the extraction method for one-dimensional three-element convolution of multiple data vectors. Figure 7 The convolution minimizes the number of times vector data is loaded and stored to main memory and uses the original source data without pre-computing the vector data. In addition, the order of loading and storing to main memory can achieve high performance of the system cache, thereby achieving a high-performance convolution implementation that can balance computational requirements with cache and memory bandwidth.

[0062] The method begins at step 702, which initializes the Figure 4 The three kernel vectors vKLeft, vKCenter and vKRight of the kernel vector 410 of FIG. Since these vectors remain unchanged during the convolution operation, they can be loaded only once. In addition, the left-aligned source vector vSrcLeft is initialized and the first source vector is loaded into vCenter.

[0063] The method proceeds to step 704, which loads the right-aligned source vector vSrcRight. Since vSrcLeft and vCenter have already been loaded, once vSrcRight is loaded, the method proceeds to step 706, where the extract vector instruction 620 is used to generate left-shifted and right-shifted vectors vLeft and vRight for convolution calculation. In step 706, the extract vector instruction 620 is represented by an EXTRACT operation, which uses four operands: the first and second operands are the source vectors, the third operand is the shift value, and the fourth operand is the output width. Step 706 generates left-shifted and right-shifted source vectors vLeft and vRight, which, when combined with the center vector vCenter, form a vector for performing convolution.

[0064] The method then proceeds to step 708, which performs three weighted sums added to the vector accumulator vAcc. Each weighted value is calculated using a multiply-add vector instruction common to many SIMD architectures, but in some embodiments, it can be implemented using a combination of arithmetic vector instructions. In step 708, the multiply-add vector instruction is represented by a MADD operation that takes three operands: the first operand is a basis vector, the second operand is a source vector, and the third operand is a kernel vector. The MADD operation multiplies the second and third operands, then adds the product to the first operand, and returns a result vector. In the first MADD operation, the specified basis vector vZero corresponds to a vector containing 0 values ​​in all data channels and is common to many SIMD architectures. Using vZero in this way provides an efficient means of initializing the vector accumulator vAcc, but in some alternative embodiments, for SIMD architectures that do not provide an equivalent vZero vector, the vector accumulator vAcc can first be initialized by alternative means. In the remaining two MADD operations, the specified basis vector is the accumulator vAcc. The three source vectors vLeft, vCenter, and vRight are multiplied by their respective kernel vectors vKLeft, vKCenter, and vKRight, respectively, and then added to the accumulator vAcc.

[0065] Once the MADD operation is complete, the method proceeds to step 710 where the accumulator vector is written to the destination address. In step 712, the vCenter and vSrcRight vectors are shifted to become vSrcLeft and vCenter vectors, respectively, and then the number of remaining vectors is checked in step 714, and the process continues at step 704 until all vectors have been generated.

[0066] In some embodiments, a scalar multiply-add vector instruction may be used. In a conventional multiply-add vector instruction, two vectors are multiplied and the product is added to a third vector to produce an output vector. Figure 8 In the scalar multiply-add vector instruction 860 of the present disclosure shown, each channel of the source vector 800 is multiplied by the scalar specified by the channel 820 of the kernel vector 810 to form a scaled vector 830, which is added to the basis vector 840 to produce the output vector 850. Using this vector instruction allows the weight coefficients of the convolution kernel to be compactly stored in each channel of the vector, rather than using a vector for each weight coefficient. This reduces the use of the register file 114 and the off-chip bandwidth requirements.

[0067] Fig. 9 According to some embodiments, the use Figure 8860 is used to demonstrate a flow chart of a one-dimensional three-element convolution of multiple data vectors. The method starts at step 902, which initializes a single kernel vector vKernel using weight coefficients in each channel of the vector. Since this vector remains unchanged during the convolution operation, it can be loaded only once.

[0068] The method proceeds to step 904, which loads Figure 4 The aligned and shifted source vectors vLeft, vCenter, and vRight corresponding to the vector 400 in FIG. In some embodiments, the vector 400 may be obtained from, for example, Figure 3 The pre-shifted data set shown in FIG. 1 loads vLeft, vCenter, and vRight, while in other embodiments, the pre-shifted data set shown in FIG. 1 may be loaded with vLeft, vCenter, and vRight by using, for example, Fig. 6A and Figure 6B and the extract vector instruction 620 loads them.

[0069] The method then proceeds to step 906, which performs three weighted sums of the vector accumulator vAcc. Figure 8 The scalar multiply-add vector instruction 860 is used to calculate each weighted value. In step 906, the scalar multiply-add vector instruction 860 is represented by a SCALAR_MADD operation, which takes four operands: the first operand is a basis vector, the second operand is a source vector, the third operand is a kernel vector, and the fourth operand is a channel of the kernel vector containing the scaling factor. The SCALAR_MADD operation multiplies each channel of the second operand by the channel specified by the third and fourth operands, then adds the product to the first operand, and returns a result vector. In the first SCALAR_MADD operation, the specified basis vector vZero corresponds to a vector containing 0 values ​​in all data channels and is common to many SIMD architectures. Using vZero in this way provides an efficient means of initializing the vector accumulator vAcc, but in some alternative embodiments, for SIMD architectures that do not provide an equivalent vZero vector, the vector accumulator vAcc can first be initialized by alternative means. In the remaining two SCALAR_MADD operations, the specified basis vector is the accumulator vAcc. The three source vectors vLeft, vCenter and vRight are multiplied by their respective kernel scalar values ​​loaded from the corresponding data channels of the kernel vector vKernel and added to the accumulator vAcc.

[0070] Once the SCALAR_MADD operation is completed, the method proceeds to step 908 where the accumulator vector is written to the destination address. The number of remaining vectors is checked in step 910 and the process continues at step 904 until all vectors have been generated.

[0071] Although Fig. 9 The embodiments disclosed in the present invention utilize a three-element kernel to perform a one-dimensional convolution, but it should be understood that the embodiments are not intended to be limited to three elements, but may include various kernel widths. In addition, although the various embodiments discussed previously have implemented a one-dimensional convolution operation, it should be understood that the methods discussed may also be applied to multi-dimensional convolution operations. In the case of multi-dimensional convolution, such as Figure 4 The set of source vectors shown can be applied to multiple accumulator vectors simultaneously to implement multi-dimensional convolution.

[0072] Fig.10 The diagram illustrates a portion of the computation of a convolution output vector using a two-dimensional 3x3 element convolution kernel given a pre-shifted source vector. In some embodiments, the pre-shifted source vector including the left vector 1002, the center vector 1004, and the right vector 1006 can be obtained from, for example, Figure 3 The pre-shifted data set is loaded as shown, or can be loaded using various other methods, including using Fig. 6A and Figure 6B The various weighting coefficients of the 3x3 kernel 1010 may be stored in some embodiments as Figure 4 The kernel vector 410 shown, or in other embodiments stored as Figure 8 860 together with the scalar multiply-add vector instruction 860. The left, center, and right vectors 1002, 1004, and 1006 have corresponding left, center, and right columns in the 3x3 kernel 1010, wherein each column includes three rows of weight coefficients. Each row of the 3x3 kernel 1010 has a corresponding accumulator vector, a leading accumulator 1018 for the first row, a center accumulator 1016 for the second row, and a lagging accumulator 1014 for the third row.

[0073] Each vector in source vector 1002,1004 and 1006 is executed three times multiply-add vector instruction, and each vector in three accumulator vectors is executed once.The result is nine multiply-add vector instructions in total, one for each kernel weight coefficient.For each multiply-add vector instruction, the row and column of kernel is selected based on the corresponding source vector and accumulator vector of operation.Therefore, leading accumulator 1018 is the weighted sum of the left vector 1002 using coefficient k1, the center vector 1004 using coefficient k2 and the right vector 1006 using coefficient k3.Similarly, center accumulator 1016 is the weighted sum of the left vector 1002 using coefficient k4, the center vector 1004 using coefficient k5 and the right vector 1006 using coefficient k6.Finally, lagging accumulator 1014 is the weighted sum of the left vector 1002 using coefficient k7, the center vector 1004 using coefficient k8 and the right vector 1006 using coefficient k9.

[0074] Fig.10 Only a partial calculation of the output vectors for a single set of source vectors is illustrated. To fully calculate each output vector, a weighted sum of the vectors immediately above and immediately below each destination vector must also be performed. This is done by iterating over the columns of the source data. Fig.10 The method shown in can perform a complete convolution calculation while loading each source vector and its associated left and right shift vectors only once.

[0075] Although for simplicity, Fig.10 A convolution calculation with a 3x3 convolution kernel is illustrated, but those skilled in the art will recognize that the calculation shown is not limited to only three rows or columns, but can also be used to support other numbers of rows, columns, or rows and columns. In addition, the calculation shown is not intended to be limited to two-dimensional convolution operations, but can be used to perform convolutions greater than two dimensions.

[0076] Fig.11 According to some embodiments, the use Figure 8 Scalar multiply-add vector instructions 860 and Fig. 6A 100 , which is a flowchart illustrating a portion of a two-dimensional three-element convolution of multiple data vectors using the extract vector instruction 620. The method assumes that the kernel vectors vKernel and vKernel2 have been initialized with weighting coefficients. In this example, it is assumed that the vectors contain eight data channels and nine weighting coefficients are required, so two kernel vectors are used, but those skilled in the art will recognize that more or fewer kernel vectors may be used depending on the vectors and the convolution kernel configuration. The method begins at step 1100, which loads the center source vector vCenter. Once the vector is loaded, it is immediately available for convolution calculations.

[0077] The method then proceeds to step 1102 which performs two functions. First, the left-aligned source vector is loaded. Once the vector is loaded, it can be combined with vCenter to produce a left-shifted vector vLeft. Second, using Figure 8 The scalar multiply-add vector instruction 860 uses the vCenter vector loaded in the previous step to perform the weighted sum calculation.

[0078] The scalar multiply-add vector instruction 860 is represented by a SCALAR_MADD operation that takes four operands: a first operand is a basis vector, a second operand is a source vector, a third operand is a kernel vector, and a fourth operand is a channel of the kernel vector containing the scaling factor. The SCALAR_MADD operation multiplies each channel of the second operand by the channel specified by the third and fourth operands, then adds the product to the first operand, and returns a result vector.

[0079] The vTop and vMid accumulator vectors may have partial results from a previous iteration of the method and are therefore uninitialized. However, the vBtm accumulator must be initialized. The basis vector vZero specified in the vBtm calculation of step 1102 corresponds to a vector containing zero values ​​in all data channels and is common to many SIMD architectures. Using vZero in this manner provides an efficient means of initializing the vector accumulator vBtm, but in some alternative embodiments, for SIMD architectures that do not provide an equivalent vZero vector, the vector accumulator may first be initialized by alternative means. Step 1102 uses the SCALAR_MADD operation to calculate three partial sums using the vCenter source vector and three weighted values ​​corresponding to the center column of the 3x3 convolution kernel.

[0080] The method then proceeds to step 1104, which loads the vSrcRight source vector. Step 1104 also uses the already loaded vSrcLeft and vCenter vectors and Fig. 6A The extract vector instruction 620 is used to load the vLeft vector. The extract vector instruction 620 is represented by an EXTRACT operation that takes four operands: the first and second operands are source vectors, the third operand is a shift value, and the fourth operand is an output width.

[0081] The method then proceeds to step 1106, which uses the already loaded vCenter and vSrcRight vectors and Fig. 6A The extract vector instruction 620 is used to load the vRight vector. In addition, the vLeft vector loaded in step 1104 is used to use Figure 8 The scalar multiply-add vector instruction 860 performs a weighted sum calculation using the three weight values ​​corresponding to the left column of the 3x3 convolution kernel.

[0082] The method then proceeds to step 1108, where the vRight vector loaded in step 1106 is used to Figure 8 The scalar multiply-add vector instruction 860 performs a weighted sum calculation using the three weight values ​​corresponding to the right column of the 3x3 convolution kernel.

[0083] The method then proceeds to step 1110, where the partial sum of the source vector for the source row has been completed. At this point, the accumulator vTop will complete all rows and can be written to the destination vector. The method proceeds to step 1112, where the remaining two accumulators vMid and vBtm are shifted to the vTop and vMid positions, respectively, to prepare for the next calculation row.

[0084] Although for simplicity, Fig.11The flowchart of illustrative embodiment illustrates a convolution calculation with a 3x3 convolution kernel, but those skilled in the art will recognize that the calculation shown is not limited to three rows or columns, but can be used to support other numbers of rows, columns, or rows and columns. In addition, the calculation shown is not intended to be limited to two-dimensional convolution operations, but can be used to perform convolutions greater than two dimensions.

[0085] Although the various embodiments of the convolution operation discussed herein can be applied in a wide variety of applications, they may be particularly useful when the various embodiments are applied to machine learning and deep neural networks. In these applications, the convolution step is considered to be a very time-consuming and resource-inefficient operation. Therefore, the processor can be customized to accelerate the operation. The convolution stage is increasingly added to a variety of deep neural networks (DNNs), including convolutional networks (D-CNNs), recurrent neural networks (such as LSTMs), generative adversarial networks (GANs), and the like. By performing direct convolution operations using SIMD instructions available on general-purpose CPU cores, convolutions can be performed as efficiently or more efficiently as using dedicated spatial convolutional neural network hardware, while being loaded into conventional processor pipelines, thereby minimizing or eliminating additional hardware. In addition, since convolutions in these applications typically use many small convolution kernels that are reused for a given data set, this method can achieve excellent processor utilization because each convolution operation can be dedicated to each processor core, allowing the system to scale well by simply adding additional processor resources. Finally, the various embodiments of the SIMD instructions described herein can benefit a wide range of operations other than convolution.

[0086] In some embodiments, strides can be used to reduce the computational requirements and output size of convolution operations. In this case, the stride value specifies how many data elements can be skipped for each calculated data element. For example, when a stride of 2 is used, every other data element will be skipped and only half of the data elements will be calculated. For multi-dimensional convolution dimensions, strides can be efficiently adopted by skipping rows of data elements, but for one-dimensional convolutions in various embodiments described herein, the stride factor will be used to reduce the utilization of the arithmetic logic unit of the execution core. For this reason, shuffled data sets can be used to enable the use of otherwise unused data channels of vector instructions. In this case, various embodiments of shuffled vector instructions are disclosed, which enable loading of shuffled source data vectors to achieve convolutions with stride values ​​greater than one.

[0087] Fig. 12A An example of such a shuffle vector instruction 1240 is illustrated in accordance with some embodiments. Fig. 12A, starting from source channel 1210 and achieving the stride value specified by stride 1220, source vectors 1200, such as 1200A and 1200B, are cascaded and a subset of the resulting vector is extracted. The result is written to output vector 1230. The number of source vectors 1200 required will be equal to stride 1220, thereby generating an output vector including data elements in all data channels. In some embodiments, each source vector 1200 can be specified using an instruction operand. In other embodiments, a source vector 1200 can be specified using an operand specifying a vector range.

[0088] Fig. 12B Another example of a shuffle vector instruction 1290 is illustrated in accordance with some embodiments. Fig. 12B , starting from source channel 1260 and achieving the stride value specified by stride 1265, a fixed number of source vectors 1250 including one or more vectors are concatenated, and a subset of the resulting vectors is extracted. The result is written to output vector 1280. Since the number of source vectors 1250 is fixed and can be less than stride 1265, the output vector can only include a continuous subset of all data channels. For this reason, an additional operand can be provided to specify the first output channel 1270 for extracting the output data channel.

[0089] Fig.13 1300A and 1300B and a stride value of 2. Fig. 6A The extract vector instruction 620 performs the extract operation 1310A using an offset value equal to the number of data lanes in the vector (eight in this example) minus one. Similarly, using the center and right source vectors 1300B and 1300C and Fig. 6A Extract vector instruction 620 performs extract operation 1310B using a shift value of 1. Extract operations 1310A and 1310B produce intermediate source vectors 1320A and 1320B, which are then input to shuffle operation 1330A. Shuffle operation 1330A uses Fig. 12A A shuffle vector instruction 1240 is executed with source lane 1210 value being zero and stride 1220 value being 2 to produce a left vector 1340A comprising alternating data lanes of the original source vector 1300 starting with data lane 18 of source vector 1300A.

[0090] Source vectors 1300B and 1300C are then input into shuffle operations 1330B and 1330C. Shuffle operation 1330B uses Fig. 12AA shuffle vector instruction 1240 is executed with a source lane 1210 value of zero and a stride 1220 value of 2 to produce a center vector 1340B, while a shuffle operation 1330C uses Fig. 12A A shuffle vector instruction 1240 is executed with a source lane 1210 value of 1 and a stride 1220 of 2 to produce a right vector 1340B. Thus, center vector 1340B includes alternating data lanes of the original source vector 1300 starting with data lane c1 of source vector 1300B, while right vector 1340C includes alternating data lanes of the original source vector 1300 starting with data lane c2 of source vector 1300B.

[0091] The vector set 1340 can then be used in a Figure 5 Used in the convolution shown, or with Fig.11 The methods shown are combined to implement multi-dimensional convolution with strides. Those skilled in the art will recognize that Fig.13 The method shown in is neither intended to be limited to three-element convolution operations nor to a stride value of 2, but can be used to generate the vectors needed to perform convolutions of various kernel widths and stride values, generally without the need to load Fig.13 Additional source vectors 1300 in addition to the source vectors shown. Those skilled in the art will also recognize that implementing embodiments with kernel widths greater than three and / or strides greater than 2 may require additional source vectors 1300 in addition to the source vectors shown. Fig.13 Additional source vectors 1300 other than those shown, and additional extract operations 1310 required for those particular embodiments. The use of extract vector instructions 620 and shuffle vector instructions 1240 or 1290 eliminates the need for, for example, Figure 3 The need for pre-shifting the data as shown eliminates the growth in data size for convolution operations that implement stride features.

Claims

1. A system comprising: A processor configured to execute a vector instruction to generate an output vector from a first source vector and a second source vector, wherein the output vector, the first source vector and the second source vector individually include a plurality of data lanes, and wherein, to execute the vector instruction, the processor is configured to: cascading the first source vector and the second source vector to generate a combined source vector; and extracting an output vector starting from the combined source vector with a data channel offset greater than zero, wherein the output vector includes at least one data channel of the first source vector and at least one data channel of the second source vector; and Write the extracted output vector to the processor's vector register.

2. The system of claim 1, wherein: The first source vector and the second source vector are accessed from respective source vector registers of the processor identified by respective operands of the vector instruction.

3. The system of claim 1 or 2, wherein: The data lane offset is identified as an operand of a vector instruction.

4. The system of claim 1, wherein to extract the output vector starting with the data lane offset from the combined source vector, the processor is configured to: fetching a number of data lanes according to a fetch width of an operand identified as a vector instruction, wherein the fetch width is less than the number of data lanes of a vector register; and A number of rightmost data lanes containing corresponding zero values ​​are added to the output vector to generate an extracted output vector having a number of data lanes matching the number of data lanes of the vector register.

5. The system of claim 1 or 4, wherein: The vector instruction is executed as part of performing a convolution on a vector, wherein the output vector is a left source vector for the convolution, wherein the second source vector is a center source vector for the convolution, and wherein to perform the convolution, the processor is further configured to: loading a plurality of source vector registers with corresponding source vectors of the vector, the corresponding source vectors of the vector comprising a first source vector, a second source vector, and a third source vector; cascading the second source vector and the third source vector to generate another combined source vector; extracting a right source vector starting from the another combined source vector with another data channel offset, wherein the right source vector includes at least one data channel of the second source vector and at least one data channel of the third source vector; as well as Write the right source vector to another vector register of the processor; loading a kernel vector, the kernel vector comprising a plurality of weighted values, the plurality of weighted values ​​comprising a center weighted value, a left weighted value, and a right weighted value; and Generate one or more output vectors, the one or more output vectors respectively comprising a weighted sum of a plurality of source vectors including a left source vector, a center source vector, and a right source vector, wherein in order to generate a specific output vector in the one or more output vectors, the processor is configured to: loading a scalar weight value from a lane of a kernel vector, the lane being assigned to the particular vector instruction based on the particular output vector and a particular source vector of the plurality of source vectors being assigned as an operand to the particular vector instruction; scaling corresponding data channels of the plurality of data channels of the particular source vector by the loaded scalar weight value to generate a scaled vector; adding the scaled vector to an accumulator vector stored in a vector accumulator to generate a new output vector; and The new output vector is stored in the vector accumulator.

6. The system of claim 5, wherein: The another data lane offset is determined according to the data lane offset and the number of data lanes of the vector register.

7. The system of claim 5, wherein: The processor is configured to perform the convolution as part of a convolutional neural network.

8. A method comprising: executing, by a processor, a vector instruction to generate an output vector from a first source vector and a second source vector, wherein the output vector, the first source vector, and the second source vector individually comprise a plurality of data lanes, and wherein the executing comprises: cascading the first source vector and the second source vector to generate a combined source vector; and extracting an output vector from the combined source vector starting at a data channel offset, wherein the output vector includes at least one data channel of the first source vector and at least one data channel of the second source vector; and Write the extracted output vector to the processor's vector register.

9. The method of claim 8, wherein: The first source vector and the second source vector are accessed from respective source vector registers of the processor identified by respective operands of the vector instruction.

10. The method of claim 8 or 9, wherein the data lane offset is identified as an operand of a vector instruction.

11. The method of claim 8, wherein extracting an output vector starting with a data channel offset from the combined source vector comprises: extracting a number of data lanes according to an extract width of an operand identified as a vector instruction, wherein the extract width is less than the number of data lanes of a vector register; and A number of rightmost data lanes containing corresponding zero values ​​are added to the output vector to generate an extracted output vector having a number of data lanes matching the number of data lanes of the vector register.

12. The method according to claim 8 or 11, wherein: The vector instruction is executed as part of performing a convolution on a vector, wherein the output vector is a left source vector for the convolution, wherein the second source vector is a center source vector for the convolution, and wherein performing the convolution further comprises: loading a plurality of source vector registers with corresponding source vectors of the vector, the corresponding source vectors of the vector comprising a first source vector, a second source vector, and a third source vector; Executing another vector instruction to generate a right source vector from the second source vector and the third source vector includes: cascading the second source vector and the third source vector to generate another combined source vector; extracting a right source vector starting from the another combined source vector with another data channel offset, wherein the right source vector comprises at least one data channel of the second source vector and at least one data channel of the third source vector; and Write the right source vector to another vector register of the processor; loading a kernel vector, the kernel vector comprising a plurality of weighted values, the plurality of weighted values ​​comprising a center weighted value, a left weighted value, and a right weighted value; and Generate one or more output vectors, the one or more output vectors respectively comprising a weighted sum of a plurality of source vectors including a left source vector, a center source vector, and a right source vector, wherein generating a specific output vector of the one or more output vectors comprises executing a specific vector instruction, the specific vector instruction executing: loading a scalar weight value from a lane of a kernel vector, the lane being assigned to the particular vector instruction based on the particular output vector and a particular source vector of the plurality of source vectors being assigned as an operand of the particular vector instruction; scaling corresponding data channels of the plurality of data channels of the particular source vector by the loaded scalar weight value to generate a scaled vector; adding the scaled vector to an accumulator vector stored in a vector accumulator to generate a new output vector; and The new output vector is stored in the vector accumulator.

13. The method of claim 12, wherein: The another data lane offset is determined according to the data lane offset and the number of data lanes of the vector register.

14. The method of claim 12, wherein: The processor is configured to perform the convolution as part of a convolutional neural network.

15. One or more non-transitory computer-accessible storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to perform: The vector instruction is executed by the processor to generate an output vector from the first source vector and the second source vector, wherein: The output vector, the first source vector, and the second source vector individually include a plurality of data channels, and wherein the performing includes: cascading the first source vector and the second source vector to generate a combined source vector; and extracting an output vector from the combined source vector starting at a data channel offset, wherein the output vector includes at least one data channel of the first source vector and at least one data channel of the second source vector; and Write the extracted output vector to the processor's vector register.

16. The one or more non-transitory computer-accessible storage media of claim 15, wherein: The first source vector and the second source vector are accessed from respective source vector registers of the processor identified by respective operands of the vector instruction.

17. One or more non-transitory computer-accessible storage media as claimed in claim 15 or 16, wherein: The data lane offset is identified as an operand of a vector instruction.

18. The one or more non-transitory computer-accessible storage media of claim 15, wherein extracting an output vector starting at a data lane offset from the combined source vector comprises: extracting a number of data lanes according to an extract width of an operand identified as a vector instruction, wherein the extract width is less than the number of data lanes of a vector register; and A number of rightmost data lanes containing corresponding zero values ​​are added to the output vector to generate an extracted output vector having a number of data lanes matching the number of data lanes of the vector register.

19. One or more non-transitory computer-accessible storage media as claimed in claim 15 or 18, wherein: The vector instruction is executed as part of performing a convolution on a vector, wherein the output vector is a left source vector for the convolution, wherein the second source vector is a center source vector for the convolution, and wherein performing the convolution further comprises: loading a plurality of source vector registers with corresponding source vectors of the vector, the corresponding source vectors of the vector comprising a first source vector, a second source vector, and a third source vector; Executing another vector instruction to generate a right source vector from the second source vector and the third source vector includes: cascading the second source vector and the third source vector to generate another combined source vector; extracting a right source vector starting from the another combined source vector with another data channel offset, wherein the right source vector comprises at least one data channel of the second source vector and at least one data channel of the third source vector; and Write the right source vector to another vector register of the processor; loading a kernel vector, the kernel vector comprising a plurality of weighted values, the plurality of weighted values ​​comprising a center weighted value, a left weighted value, and a right weighted value; and Generate one or more output vectors, the one or more output vectors respectively comprising a weighted sum of a plurality of source vectors including a left source vector, a center source vector, and a right source vector, wherein generating a specific output vector of the one or more output vectors comprises executing a specific vector instruction, the specific vector instruction executing: loading a scalar weight value from a lane of a kernel vector, the lane being assigned to the particular vector instruction based on the particular output vector and a particular source vector of the plurality of source vectors being assigned as an operand to the particular vector instruction; scaling corresponding data channels of the plurality of data channels of the particular source vector by the loaded scalar weight value to generate a scaled vector; adding the scaled vector to an accumulator vector stored in a vector accumulator to generate a new output vector; and The new output vector is stored in the vector accumulator.

20. The one or more non-transitory computer-accessible storage media of claim 19, wherein: The processor is configured to perform the convolution as part of a convolutional neural network.

Images