Convolution acceleration method and equipment based on AVX instruction, and medium

Abstract

The invention provides a convolution acceleration method and device based on an AVX instruction and a medium, belongs to the field of high-performance computing neural network operator libraries, and aims to enable computation to be equivalent to the sum of a plurality of convolutions with the step length of 1 by utilizing segmentation input and weight, so that a WinoGrad algorithm can use accelerated convolution, the algorithm is improved, the application range of the algorithm is widened, and the method and the device have the advantages that the method and the device are easy to implement. And meanwhile, the algorithm is optimized by utilizing an AVX2 instruction of the x86-CPU. On one hand, the method can cover performance acceleration of more convolution type operators related to an image processing model, calculation delay is reduced, and meanwhile calculation resources in a CPU-GPU heterogeneous calculation system can be fully utilized; and on the other hand, the method also provides a new thought and reference for convolution acceleration of the GPU.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of convolution computing, and specifically to a convolution acceleration method, device, and medium based on AVX instructions. Background Art

[0002] The learning neural network in computer vision is mainly composed of convolution operators. In terms of accelerating the convolution algorithm, WinoGrad (Winograd fast Fourier convolution algorithm) is usually used in the existing technology. However, the WinoGrad algorithm is only suitable for the case where the convolution kernel step size is 1.

[0003] Therefore, how to improve the convolution calculation speed based on AVX instructions is an urgent problem to be solved. Summary of the Invention

[0004] The object of the present invention is to provide a convolution acceleration method, device and medium based on AVX instructions, so as to at least partially solve the above problems.

[0005] According to one aspect of the present invention, a convolution acceleration method based on AVX instructions is proposed, comprising: Step S1: Input the data to be convolved and determine whether the convolution kernel is 5x5 or 3x3. If so, proceed to step S2. Step S2: Determine whether the convolution kernel step size is 1. If not, proceed to step S3. If yes, proceed directly to step S4. Step S3, using interval sampling to split the weights and input into multiple convolution calculations, Step S4, converting the weights and inputs in the data format into the first data format according to WinoGrad form, Step S5: Calculate each convolution multiplication based on the first data format and the data to be calculated, and perform point-add convolution multiplication.

[0006] Preferably, weights and inputs in NHWC format are converted to nChw8c format.

[0007] Preferably, the convolution input data block is divided into submatrix of .

[0008] Preferably, during the CPU calculation process, cache is used as a buffer. Specifically, two rows of data are read into the in_reg2x4 register each time, the intermediate results are saved into the tmp_2x4 register, and written back to the cache. The reading is repeated twice, and the results are read out of the cache and the final result is obtained through matrix transformation.

[0009] The embodiment of the present application further provides a convolution acceleration device based on AVX instructions, comprising: The first judgment module is used to input the data to be convolved and determine whether the convolution kernel is 5x5 or 3x3. The second judgment module is used to judge whether the convolution kernel step size is 1. Splitting module, which is used to split weights and inputs into multiple convolution calculations using interval sampling, A format conversion module, used to convert the weights and inputs in the data format into the first data format according to the WinoGrad form, The calculation module is used to calculate each convolution multiplication based on the first data format and the data to be calculated, and to perform point-add convolution multiplication.

[0010] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps in the method of any of the above embodiments by calling the computer program stored in the memory.

[0011] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program is characterized in that when executed by a processor, the steps of the method in any of the above embodiments are executed.

[0012] The present invention provides a convolution acceleration method based on AVX instructions, which belongs to the field of high-performance computing neural network operator library. By splitting the input and weights, the calculation is equivalent to the sum of several convolutions with a step size of 1, so that the WinoGrad algorithm can use accelerated convolution, and the algorithm is improved to enhance the scope of application of the algorithm. At the same time, the algorithm is optimized by using the AVX2 instructions of the x86-CPU. On the one hand, this method can cover more convolution operator performance acceleration, especially convolution operators related to image processing models, reduce computing latency, and fully utilize the computing resources in the CPU-GPU heterogeneous computing system, reducing the consumption of hardware resources. On the other hand, this method also provides new ideas and references for GPU convolution acceleration. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of a convolution acceleration method based on AVX instructions provided in an embodiment of the present application.

[0014] Figure 2 A schematic diagram of the WinoGrad matrix conversion provided in an embodiment of the present application.

[0015] Figure 3 This is an example diagram of a 5x5 convolution kernel and a 7x7 input sub-block provided in an embodiment of the present application.

[0016] Figure 4 This is an example diagram of a 3x3 convolution kernel and a 5x5 input sub-block provided in an embodiment of the present application.

[0017] Figure 5 Schematic diagram of AVX2 WinoGrad matrix transpose calculation provided in an embodiment of the present application.

[0018] Figure 6 Schematic diagram of dividing the convolution input provided in an embodiment of the present application into 7x7 sub-matrix blocks.

[0019] Figure 7 A schematic diagram of a convolution acceleration device based on AVX instructions provided in an embodiment of the present application.

[0020] Figure 8 A schematic diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following is a further description of specific embodiments of the present invention in conjunction with the accompanying drawings. It should be noted that the description of these embodiments is intended to facilitate understanding of the present invention and does not constitute a limitation of the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0022] It should be noted that, in the description of the present invention, the terms "up", "down", "left", "right", "front", "back", etc. indicating directions or positional relationships are based on the description of the structure of the present invention shown in the accompanying drawings, and are only for the convenience of describing the present invention, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, it cannot be understood as a limitation on the present invention.

[0023] The "first" and "second" in this technical solution are only used to distinguish the same or similar structures, or corresponding structures with similar functions, and are not an arrangement of the importance of these structures, nor do they have any ranking, size comparison, or other meanings.

[0024] In addition, unless otherwise expressly specified or limited, the terms "installed" and "connected" should be understood broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be internal communication between two structures. Those skilled in the art can understand the specific meanings of the above terms in the present invention based on the overall principles of the present invention and the specific context of this solution.

[0025] Figure 1 The present invention provides a convolution acceleration method based on AVX instructions, which specifically includes the following steps: Step S1: Input the data to be convolved and determine whether the convolution kernel is 5x5. If so, proceed to step S2.

[0026] Among them, convolution calculation is usually used in deep learning artificial neural network calculation, generally using odd convolution kernels, which makes it easier to identify the position of the convolution kernel. After each convolution kernel operation, the original input dimension will be reduced and edge information will be lost. Generally, a filling method is used to complete it, and using an odd convolution kernel is more conducive to filling information symmetry.

[0027] Preferably, this embodiment uses a large core of 5x5 to facilitate subsequent splitting, so that the calculation process can use the WinoGrad algorithm.

[0028] Step S2, determine whether the convolution kernel step size is 1, if not, go to step S3, if so, directly go to step S4.

[0029] As shown in the above formula, the vectors [w0, w1, w2] are dot-multiplied with [k0, k1, k2] and [k1, k2, k3], respectively. When the stride is 2, the vectors [w0, w1, w2] are dot-multiplied with [k0, k1, k2] and [k2, k3, x], respectively. Therefore, when the stride is 2, the above 4x4 matrix conversion to a 4x9 matrix is not applicable. WinoGrad convolution generally outperforms direct convolution due to fewer multiplications. However, WinoGrad convolution requires a stride of 1 and cannot be directly used with a stride of 2.

[0030] In step S3, the weights and inputs are split into multiple convolution calculations using interval sampling.

[0031] In order to use the WinoGrad convolution algorithm even when the step size is not 1, this embodiment uses interval sampling splitting for input and weight calculation, such as Figure 3 As shown, Figure 3 The number in each element represents a hue, and elements of the same hue represent dot product. Figure 3 This is an example of a 5x5 convolution kernel and a 7x7 input sub-block. After splitting, the 7x7 matrix keeps the elements of the sub-matrix block continuous, and fills gray and white with 0.

[0032] At the same time, if Figure 4 The figure shows an example of a 3x3 convolution kernel and a 5x5 input sub-block. In this embodiment, the F(2x2, 2x2) WinoGrad convolution multiplication can be used. Here, the 5x5 sub-block is finally split into four 3x3 input matrices. The first convolution kernel is 3x3, the second convolution kernel is 3x2, the third convolution kernel is 2x3, and the fourth convolution kernel is 2x2. Figure 4The upper data is the input, the gray part of the input is padded with zeros to form a 4x4 matrix, and the middle data is the convolution kernel blank padded with zeros to form a 3x3 matrix.

[0033] Step S4, converting the weights and inputs in the data format into the first data format in accordance with WinoGrad format. Among them, the WinoGrad algorithm, also known as the Winograd fast Fourier convolution algorithm, can reduce the number of multiplication calculations in convolution calculations. The specific calculation process is as follows: The CPU-based WinoGrad algorithm's F(2x2,3x3) requires a stride of 1. For example, the MNN, TNN, and Openvino algorithm implementations require a 4x4 input sub-block and a 3x3 weight, resulting in a 2x2 output. F(2x2, 3x3) here indicates a 2x2 output matrix and a 3x3 weight matrix.

[0034] like Figure 2 As shown in the figure, the matrix conversion diagram is converted to matrix multiplication for the F(2x2, 3x3) WinoGrad convolution algorithm. The following formula represents the schematic diagram of 2D WinoGrad: ; ; ; ; ;

[0035] The final merge result is , where g is the convolution kernel weight, G is the convolution kernel conversion matrix, d is the convolution input, and B is the input conversion matrix, where G and B are 0, 1 matrices, For offline calculation, For online calculation, the B matrix is:

[0036] The convolution calculation in the formula is the multiplication of two 4x4 matrices, so the first, second, and third convolutions require 16 multiplications. The fourth convolution pads the last row and column with zeros, so it requires 12 multiplications. The total number of multiplications required is 60, while the direct convolution requires 100. Finally, add the calculation results of each matrix.

[0037] Specifically, if Figure 5As shown, to accommodate AVX2 instructions, the input weight format is required to be NHWC and then converted to nChw8c. For convolution input, the data needs to be loaded with a stride of 2 on both the horizontal and vertical axes, and then the wingorad conversion is performed. In the actual CPU implementation of the F[2x2, 3x3] algorithm, the convolution input data block is clamped into the data vector register with a stride of 2 and read into a 4x4 register matrix. Each register stores a vector of 16 floating-point numbers, and The matrix is transformed (actual calculations are only addition and subtraction), and the result is stored in the buffer register; it is transformed again with the B matrix and finally stored in the result register, so at least 32 vector registers are required. AVX512 has 32 vector registers, while AVX2 only has 16 vector registers, and each vector register holds 8 floating-point numbers. Therefore, we read two rows of data at a time into the in_reg2x4 register, save the intermediate result into the tmp_2x4 register, and write it back to the cache. This read is repeated twice, and then the result is read from the cache and transformed with the B(0,1) matrix. This way, using the cache as a buffer only uses 16 registers.

[0038] Preferably, if Figure 6 As shown, the convolution input is divided into 7x7 sub-matrix blocks.

[0039] The present invention utilizes split inputs and weights, transforming the computation into the sum of several convolutions with a stride of 1. This allows the WinoGrad algorithm to use accelerated convolutions, improves its algorithmic scope, and optimizes it using AVX2 instructions. This approach accelerates the performance of more convolution operators, reduces computational latency, and reduces hardware resource consumption.

[0040] Another exemplary embodiment of the present invention provides a convolution acceleration device based on AVX instructions, such as Figure 7 As shown, the device includes: The first judgment module 701 is used to input the data to be convolved and judge whether the convolution kernel is , The second judgment module 702 is used to judge whether the convolution kernel step size is 1. Splitting module 703 is used to split weights and inputs into multiple convolution calculations using interval sampling, The format conversion module 704 is used to convert the weights and inputs in the data format into the first data format according to the WinoGrad format. The calculation module 705 is configured to calculate each convolution multiplication based on the first data format and the data to be calculated, and perform point-add convolution multiplication.

[0041] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0042] The present invention utilizes split inputs and weights, transforming the computation into the sum of several convolutions with a stride of 1. This allows the WinoGrad algorithm to use accelerated convolutions, improves its algorithmic scope, and optimizes it using AVX2 instructions. This approach accelerates the performance of more convolution operators, reduces computational latency, and reduces hardware resource consumption.

[0043] It should be noted that the AVX instruction-based convolution acceleration device provided in the above embodiment is only illustrated by the division of the above functional units. In actual applications, the above functions can be assigned to different functional units as needed. In addition, the AVX instruction-based convolution acceleration device provided in the above embodiment and the AVX instruction-based convolution acceleration method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0044] Accordingly, an embodiment of the present application further provides an electronic device, which may be a terminal or a server. Figure 8 As shown, Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0045] The electronic device 800 includes a processor 801 having one or more processing cores, a memory 802 having one or more computer-readable storage media, and a computer program stored in the memory 802 and executable on the processor. The processor 801 is electrically connected to the memory 802. Those skilled in the art will appreciate that the electronic device structure shown in the figures does not limit the electronic device and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0046] The processor 801 is the control center of the electronic device 800. It uses various interfaces and lines to connect various parts of the entire electronic device 800. By running or loading software programs (computer programs) and / or units stored in the memory 802 and calling data stored in the memory 802, it executes various functions of the electronic device 800 and processes data, thereby monitoring the electronic device 800 as a whole.

[0047] In an embodiment of the present application, the processor 801 in the electronic device 800 will load the instructions corresponding to the processes of one or more applications into the memory 802 according to the following steps, and the processor 801 will run the applications stored in the memory 802 to implement various functions.

[0048] Optional, such as Figure 8 As shown, the electronic device 800 further includes: an acceleration module 803, a communication module 804, an input unit 805 and a power supply 806. Among them, the processor 801 is electrically connected to the acceleration module 803, the communication module 804, the input unit 805 and the power supply 806 respectively. It can be understood by those skilled in the art that Figure 8 The electronic device structure shown in the figure does not constitute a limitation to the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0049] The acceleration module 803 may be used to implement convolution acceleration based on AVX instructions.

[0050] The communication module 804 can be used to communicate with other devices.

[0051] The input unit 805 may be configured to receive input digital, character information, or user feature information (such as fingerprint, iris, or facial information), and generate keyboard, mouse, joystick, optical, or trackball signal input related to user settings and function control.

[0052] Power supply 806 is used to supply power to various components of electronic device 800. Optionally, power supply 806 can be logically connected to processor 801 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. Power supply 806 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0053] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0054] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0055] To this end, an embodiment of the present application provides a computer-readable storage medium storing a plurality of computer programs, which can be loaded by a processor to execute the steps of a convolution acceleration method based on AVX instructions provided in an embodiment of the present application. For example, the computer program can execute the following steps: Step S1: Input the data to be convolved and determine whether the convolution kernel is 5x5 or 3x3. If so, proceed to step S2. Step S2: Determine whether the convolution kernel step size is 1. If not, proceed to step S3. If yes, proceed directly to step S4. Step S3, using interval sampling to split the weights and input into multiple convolution calculations, Step S4, converting the weights and inputs in the data format into the first data format according to WinoGrad form, Step S5: Calculate each convolution multiplication based on the first data format and the data to be calculated, and perform point-add convolution multiplication.

[0056] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0057] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0058] Since the computer program stored in the storage medium can execute the steps of any convolution acceleration method based on AVX instructions provided in the embodiments of the present application, the beneficial effects that can be achieved by any convolution acceleration method based on AVX instructions provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0059] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0060] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0061] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0062] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. It is apparent to those skilled in the art that various changes, modifications, substitutions, and variations to these embodiments may be made without departing from the principles and spirit of the present invention, and these changes and modifications still fall within the scope of protection of the present invention.

Claims

1. A convolution acceleration method based on AVX instructions, characterized in that: The method comprises: Step S1: Input the data to be convolved and determine whether the convolution kernel is 5x5 or 3x3. If so, proceed to step S2. Step S2: Determine whether the convolution kernel step size is 1. If not, proceed to step S3. If yes, proceed directly to step S4. Step S3, using interval sampling to split the weights and input into multiple convolution calculations, Step S4, converting the weights and inputs in the data format into the first data format according to WinoGrad form, Step S5: Calculate each convolution multiplication based on the first data format and the data to be calculated, and perform point-add convolution multiplication.

2. The method according to claim 1, wherein: Convert weights and inputs in NHWC format to nChw8c format.

3. The method according to claim 1, wherein: Divide the convolution input data blocks into submatrix of .

4. The method according to claim 1, wherein: During the CPU calculation process, the cache is used as a buffer. Specifically, two rows of data are read into the in_reg2x4 register each time, the intermediate results are saved into the tmp_2x4 register, and written back to the cache. The reading is repeated twice, and the results are read from the cache and the final result is obtained through matrix transformation.

5. A convolution acceleration device based on AVX instructions, characterized in that: The method comprises: The first judgment module is used to input the data to be convolved and determine whether the convolution kernel is 5x5 or 3x3. The second judgment module is used to judge whether the convolution kernel step size is 1. Splitting module, which is used to split weights and inputs into multiple convolution calculations using interval sampling, A format conversion module, used to convert the weights and inputs in the data format into the first data format according to the WinoGrad form, The calculation module is used to calculate each convolution multiplication based on the first data format and the data to be calculated, and to perform point-add convolution multiplication.

6. The device according to claim 5, characterized in that: Convert weights and inputs in NHWC format to nChw8c format.

7. The device according to claim 5, characterized in that: Divide the convolution input data blocks into submatrix of .

8. The device according to claim 1, characterized in that: During the CPU calculation process, the cache is used as a buffer. Specifically, two rows of data are read into the in_reg2x4 register each time, the intermediate results are saved into the tmp_2x4 register, and written back to the cache. The reading is repeated twice, and the results are read from the cache and the final result is obtained through matrix transformation.

9. An electronic device, characterized in that: The method comprises a memory storing executable program code and a processor coupled to the memory; wherein the processor calls the executable program code stored in the memory to execute the method according to any one of claims 1 to 4.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is executed.

Images