Data processing device, data processing method, and data processing program

Abstract

This data processing device, which includes a neural network including convolution processing using a Winograd algorithm, comprises: an acquisition unit which acquires target data to be processed; and a processing unit which processes the target data by using the neural network including the convolution processing. The processing unit is set to: calculate a Hadamard product of the result of kernel transform processing based on the Winograd algorithm and a kernel transform matrix when the convolution processing is performed; obtain the result of the convolution processing by using the calculation result of the Hadamard product via multiplication, wherein the value of each element of the kernel transform matrix is a power of 2 and has a different integer value corresponding to a divisor of division that is required for the kernel transform processing when the kernel transform matrix is not applied; and exclude the division from operation processing before executing the multiplication.

Description

Technical Field

[0001] The technology of the present disclosure relates to a data processing device, a data processing method, and a data processing program.

Background Art

[0002] The need for deep learning has been increasing, and its application to various fields such as autonomous driving, monitoring, and surveillance is expected. In particular, in recent years, in order to enable large-scale arithmetic processing of deep learning in edge terminals such as cameras, the development of accelerators, which are dedicated hardware, has been actively carried out. In the accelerator described in Non-Patent Document 1, the data handled in the convolutional arithmetic processing of deep learning is limited to 8-bit fixed-point data, and the Winograd algorithm is used to reduce the amount of data and the amount of calculation.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Applying the Winograd algorithm to convolution operations requires various data transformations before and after multiplication. Implementing these data transformations in hardware requires additional resources such as adders and dividers (or shifters). Accelerators, which are hardware specifically designed for deep learning, typically have a high degree of parallelism in the convolution unit to improve throughput. Therefore, even if each of the additional resources required for this transformation is small, depending on the degree of parallelism in the convolution unit, it may have an impact on the overall system resources.

[0005] The disclosed technology was developed in view of the above points, and aims to provide a data processing device, a data processing method, and a data processing program that can reduce the circuit size while maintaining computational accuracy in convolution operations using the Winograd algorithm. [Means for solving the problem]

[0006] A first aspect of this disclosure is a data processing device including a neural network that includes a convolution operation using the Winograd algorithm, comprising: an acquisition unit that acquires target data to be processed; and a processing unit that processes the target data using the neural network that includes the convolution operation, wherein when performing the convolution operation, the processing unit calculates the Hadamard product of the result of the kernel transformation operation based on the Winograd algorithm and the kernel transformation matrix, and obtains the result of the convolution operation by using the calculation result of the Hadamard product for multiplication, wherein the value of each element of the kernel transformation matrix is ​​a power of 2 and has different constant values ​​corresponding to the divisor of the division required for the kernel transformation operation when the kernel transformation matrix is ​​not applied, and is set so that division is not included in the arithmetic processing before the multiplication is performed.

[0007] A second aspect of this disclosure is a data processing method in a data processing device including a neural network that includes a convolution operation using the Winograd algorithm, wherein an acquisition unit acquires target data to be processed, and a processing unit processes the target data using the neural network that includes the convolution operation, wherein when the processing unit performs the convolution operation, it calculates the Hadamard product of the result of the kernel transformation operation based on the Winograd algorithm and the kernel transformation matrix, and obtains the result of the convolution operation by using the calculation result of the Hadamard product for multiplication, wherein the value of each element of the kernel transformation matrix is ​​a power of 2 and has different constant values ​​corresponding to the divisor of the division required for the kernel transformation operation when the kernel transformation matrix is ​​not applied, and is set so that division is not included in the arithmetic processing before the multiplication is performed.

[0008] A third aspect of this disclosure is a data processing program for causing a computer including a neural network including a convolution operation using the Winograd algorithm to acquire target data to be processed and to process the target data using the neural network including the convolution operation, wherein when performing the convolution operation, the Hadamard product of the result of the kernel transformation operation based on the Winograd algorithm and the kernel transformation matrix is ​​calculated, and the result of the convolution operation is obtained by using the calculation result of the Hadamard product for multiplication, wherein the value of each element of the kernel transformation matrix is ​​a power of 2 and has different constant values ​​corresponding to the divisor of the division required for the kernel transformation operation when the kernel transformation matrix is ​​not applied, and is set so that division is not included in the arithmetic operation before the multiplication is performed. [Effects of the Invention]

[0009] According to the disclosed technology, it is possible to reduce the circuit size while maintaining computational accuracy in convolution operations using the Winograd algorithm. [Brief explanation of the drawing]

[0010] [Figure 1]This is a schematic block diagram of an example of a computer that functions as a data processing device in this embodiment. [Figure 2] This figure shows an example of the layer structure of a convolutional neural network. [Figure 3] This block diagram shows an example of the hardware configuration of the accelerator in this embodiment. [Figure 4] This block diagram shows an example of the hardware configuration of the accelerator PE in this embodiment. [Figure 5] This figure shows an example of the hardware configuration and data flow of the MAC arithmetic unit of the accelerator in this embodiment. [Figure 6] (a) is a diagram showing how the feature map and kernel are multiplied after the Winograd pre-transformation process in the comparative example, and (a) is a diagram showing how the feature map and kernel are multiplied after the Winograd pre-transformation process in this embodiment. [Figure 7] This is a block diagram showing the functional configuration of the data processing device of this embodiment. [Figure 8] This is a block diagram showing the functional configuration of the learning unit of the data processing device of this embodiment. [Figure 9] This is a block diagram showing the functional configuration of the inference unit of the data processing device of this embodiment. [Figure 10] This is a flowchart illustrating the learning process flow of this embodiment. [Figure 11] This is a flowchart illustrating the flow of the convolution process in the learning process and data processing of this embodiment. [Figure 12] This is a flowchart illustrating the data processing flow of this embodiment. [Modes for carrying out the invention]

[0011] Hereinafter, an example of an embodiment of the disclosed technology will be described while referring to the drawings. In each drawing, the same or equivalent components and parts are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for the convenience of explanation and may be different from the actual ratios.

[0012] <Overview of Embodiment of Disclosed Technology> In the disclosed technology, in the convolution operation processing of data fixed-point quantized to low bits, the circuit scale required when applying the Winograd algorithm is reduced, and the generated error is reduced.

[0013] As a method for reducing the number of multiplications required for convolution operation processing, the Winograd algorithm is known. In order to apply the Winograd algorithm, it is necessary to perform a predetermined conversion (hereinafter referred to as Winograd conversion) on the input data and kernel required for the convolution operation as shown in the following formula before executing the multiplication. Here, when realizing the Winograd conversion of the fixed-point quantized kernel by hardware, rounding (rounding off) and saturation processing are required before inputting the data to the multiplier (the first conventional method).

[0014] JPEG0007910628000001.jpg30163

[0015] As shown in the above formula, the Winograd conversion is performed, and the convolution kernel input to the multiplier is obtained.

[0016] In order to reduce the rounding process, a method of multiplying by a constant value so that the rounding process is unnecessary for the entire kernel and then inputting it to the multiplier is also conceivable (the second conventional method). For example, in the case of the Winograd conversion process of F(2×2, 3×3), as shown in the following formula, the rounding process can be reduced by multiplying the entire kernel by 4.

[0017] JPEG0007910628000002.jpg20163

[0018] The disclosed technology modifies the Winograd transform formula as follows: It calculates the Hadamard product of a matrix with different constant values ​​for each element to eliminate the need for rounding, and the kernel after the Winograd transform, and inputs this product to a multiplier. Since the values ​​of each element of the matrix newly added to the calculation process are always fixed according to the coefficient position of the kernel, as long as the element values ​​are powers of 2, it can be implemented with a fixed shifter and can be implemented with almost no increase in hardware size.

[0019] JPEG0007910628000003.jpg29164

[0020] Compared to the first conventional method described above, the disclosed technology can reduce the rounding circuit before the multiplier. As shown above, when viewed in units of a 4x4 kernel matrix after the Winograd transform, a total of 12 rounding circuits can be reduced.

[0021] Furthermore, compared to the second conventional method described above, the disclosed technology allows for the saturation processing circuit to be reduced for some coefficients (for example, four coefficients). Also, for coefficients K'0~K'4, K'7~K'8, and K'11~K'15, the constant values ​​multiplied by the kernel are smaller than in the second conventional method, thus reducing the errors caused by saturation processing. Focusing on K'0 as an example, the second conventional method could potentially generate an error of up to 75% due to saturation processing (the error becoming 1 / 4 of the original value), but this can be reduced to 0%.

[0022] <Configuration of the data processing device according to this embodiment> Figure 1 is a block diagram showing the hardware configuration of the data processing device 10 of this embodiment.

[0023] As shown in Figure 1, the data processing device 10 includes a CPU (Central Processing Unit) 11, ROM (Read Only Memory) 12, RAM 13, storage 14, input unit 15, display unit 16, communication interface (I / F) 17, and accelerator 18. Each component is connected to the others via a bus 19 so as to be able to communicate with each other.

[0024] The CPU 11 is a central processing unit that executes various programs and controls various components. Specifically, the CPU 11 reads a program from the ROM 12 or storage 14 and executes the program using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various calculations according to the program stored in the ROM 12 or storage 14. The CPU 11 also controls the execution timing of the camera module (not shown) and accelerator 18 connected via the communication interface 17. In this embodiment, the ROM 12 or storage 14 stores a learning program for performing learning processing on a neural network and a data processing program for performing data processing using a neural network. The learning program and the data processing program may be a single program or a group of programs composed of multiple programs or modules.

[0025] ROM12 stores various programs and data. RAM13 temporarily stores programs or data as a working area. Storage14 consists of an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs, including the operating system, and various data.

[0026] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used for various types of input.

[0027] The input unit 15 accepts training data for training the neural network as input. For example, the input unit 15 accepts training data as input, which includes the target image to be processed and the processing results for the target image that have been determined in advance.

[0028] Furthermore, the input unit 15 receives the target image to be processed, captured by the camera module, as input. The camera module can capture still images or videos at a predetermined frame rate, and the input unit 15 sequentially stores the captured images into the storage unit 14.

[0029] The display unit 16 is, for example, a liquid crystal display and displays various information, including processing results. The display unit 16 may also function as an input unit 15 by employing a touch panel system.

[0030] The communication interface 17 is an interface for communicating with other devices, and standards such as Ethernet®, FDDI, and Wi-Fi® can be used.

[0031] The accelerator 18 performs operations including convolution in the convolutional layer of the neural network. Specifically, the accelerator 18 reads the target image and kernel stored in the storage 14, and performs operations including convolution by the neural network on the read target image (for example, object detection).

[0032] Referring to Figure 2, an example of object detection processing performed by accelerator 18 will be described. Figure 2 shows an example of the layer structure of a convolutional neural network for realizing object detection processing. In the example shown in Figure 2, the input image is 448 pixels wide and 448 pixels high, and is composed of three color components: RGB. The feature extraction unit performs convolutional operations using multiple kernels different in each layer, or pooling operations, on the input image to generate a feature map. Subsequently, the detection unit performs a fully connected network on the feature map to generate the data for the final layer. In the case of object detection processing, the data for the final layer includes coordinate information indicating the relative position of the object to the input image, a confidence level indicating whether or not an object exists at that coordinate, or a classification probability indicating what class the object belongs to (e.g., person, car, dog, cat). By referring to this information, the CPU 11 can detect what kind of object is in the input image and at what position, and obtain the processing result. In this embodiment, the individual feature quantities constituting the feature map, and the parameter values ​​such as kernels and biases used during convolutional operations, are assumed to be 8-bit fixed-point data. This significantly reduces the circuit size of the accelerator 18 and the required capacity of the storage 14 compared to handling floating-point data such as 32 bits.

[0033] Figure 3 is a block diagram showing an example of the hardware configuration of the accelerator 18 in this embodiment. The accelerator 18 consists of an arithmetic processing unit 50 and a cache memory 52, the cache memory 52 is connected to the storage 14 via a bus 19. The cache memory 52 acts as a buffer located between the arithmetic processing unit 50 and the storage 14, playing a role in reducing the data transfer bandwidth between the arithmetic processing unit 50 and the storage 14. The arithmetic processing unit 50 consists of a control unit 54, a DMAC (Direct Memory Access Controller) 56, and a plurality of PEs (Processing Engines) 58. The control unit 54 sets operating parameters for the DMAC 56 and each PE 58, and manages the data supplied to each PE 58. The DMAC 56 reads from the cache memory 52 the feature map, the kernel and bias parameters necessary for convolution operations, and quantization step information for quantizing the feature map into 8-bit fixed-point data, according to the operating parameters set by the control unit 54. The read data is supplied to each PE 58, and each PE 58 executes arithmetic processing in parallel. The feature map generated by the PE58's calculation is stored in the cache memory 52 via the DMAC56 and read out from the cache memory 52 again during the calculation of the next layer. Here, each PE58 has two operating modes: a Winograd mode in which the Winograd algorithm is applied to the convolution operation, and a non-Winograd mode in which the Winograd algorithm is not applied. The control unit 54 sets each PE58 to operate in Winograd mode when the size of the kernel used for the convolution operation is 3x3 and the convolution application interval (stride) is 1. If the above conditions are not met, the control unit 54 sets each PE58 to operate in non-Winograd mode. Also, when each PE58 operates in Winograd mode, the DMAC56 supplies each PE58 with a feature map having a size of "width 4 x height 4 x number of input channels 1 (hereinafter referred to as 4x4)" and a kernel having a size of "width 3 x height 3 x number of input channels 1 (hereinafter referred to as 3x3)".On the other hand, when each PE58 operates in non-Winograd mode, the DMAC56 supplies each PE58 with a feature map having a size of "width 1 × height 1 × number of input channels 4" and a kernel.

[0034] Figure 4 is a block diagram showing an example of the hardware configuration of PE58. The MAC calculation unit 60 performs a convolution operation using a feature map and a kernel. The convolution operation result is then processed by the bias addition unit 62 and the activation function processing unit 64, and quantized to have a set quantization step by the quantization unit 66 before being output.

[0035] Figure 5 shows an example of the hardware configuration and data flow of the MAC arithmetic unit 60. The MAC arithmetic unit 60 has two data paths, one for Winograd mode and one for non-Winograd mode, and Figure 5 shows the data flow when operating in Winograd mode. When operating in Winograd mode, the MAC arithmetic unit 60 receives a 4x4 feature map and a 3x3 kernel as input. These data are subjected to conversion processing by the pre-Winograd conversion unit 70, multiplication by the multiplier 74, conversion processing by the post-Winograd conversion unit 76, cumulative addition by the cumulative addition unit 82, and quantization by the quantization unit 80, ultimately outputting a 2x2 feature map. The multiplier 74 of the MAC arithmetic unit 60 has 16 circuits that multiply two 8-bit fixed-point data. In the Winograd algorithm, the calculation process to obtain an mxm size output using an rxr size filter is generally written as F(mxm,rxr), and the MAC arithmetic unit 60 can realize the F(2x2,3x3) process. Here, the process to obtain the matrix Y, which is the result of processing F(2×2,3×3), can be written as follows.

[0036] JPEG0007910628000004.jpg25164 JPEG0007910628000005.jpg4386 JPEG0007910628000006.jpg3481 JPEG0007910628000007.jpg3573 JPEG0007910628000008.jpg5560 JPEG0007910628000009.jpg2276

[0037] Matrix d represents the 4x4 input feature map, and matrix g represents the 3x3 input kernel. Furthermore, matrix B represents the transformation matrix of the input feature map, and matrix G represents the transformation matrix of the input kernel. JPEG0007910628000010.jpg810 This represents element-wise multiplication (Hadamard product) of matrices. Matrix A is used to transform the multiplication result again to obtain the output. Here, the result of the kernel transformation process is GgG T Focusing on this, we perform the following algebraic manipulation.

[0038] JPEG0007910628000011.jpg69164

[0039] According to the above formula, the result of the kernel's conversion process is GgG T This requires addition, subtraction, and division of multiple kernel coefficients. When implementing this kernel transformation in hardware, the following circuit resources are typically required.

[0040] 1. A 1-bit or 2-bit right shifter to perform division.

[0041] 2. Rounding circuit for rounding the lower bits of the division result.

[0042] 3. Saturation processing circuit to ensure that the result of the kernel conversion process falls within a range that can be represented by the number of input bits of the multiplier (8 bits in this embodiment).

[0043] In this embodiment, the processing F(2×2,3×3) commonly used in the Winograd algorithm is modified as follows by introducing the kernel transformation matrix C, matrix D, and coefficient α.

[0044] JPEG0007910628000012.jpg14164 JPEG0007910628000013.jpg2437 JPEG0007910628000014.jpg2337

[0045] α = 1 / 4

[0046] The kernel transformation matrix C, matrix D, and coefficient α are equivalent to the calculation results of the algorithm before modification, and each element value of the matrix and the coefficient value are set to a power of 2. In addition, the values ​​of each element of kernel transformation matrix C are the kernel transformation process when kernel transformation matrix C is not applied (the result of the kernel transformation process GgG T It is a constant value that indicates the divisor required for division.

[0047] JPEG0007910628000015.jpg1341 The calculation process for each element is set so that division is not required. In the example above, the values ​​of the kernel transformation matrix C, matrix D, and coefficient α shown in this embodiment are merely examples, and the values ​​do not necessarily have to be those shown in this embodiment as long as the above conditions are met, and it is applicable to Winograd algorithms other than F(2×2,3×3).

[0048] The calculation process performed by the Winograd pre-transformation unit 70 of this embodiment will now be described. The Winograd pre-transformation unit 70 performs the feature map transformation matrix B T dB and kernel transformation matrix JPEG0007910628000016.jpg1451 We calculate each of these. Since the coefficients of each element of the kernel transformation matrix C are all powers of 2, the matrix JPEG0007910628000017.jpg1451 The Hadamard product involved in the calculation process can be implemented with a 1-bit or 2-bit left shifter, and can be implemented without increasing circuit resources. Here, the matrix JPEG0007910628000018.jpg1448 To find this, we can write it as follows:

[0049] JPEG0007910628000019.jpg34164

[0050] From the above equation, the Winograd pre-transformation unit 70 is the kernel transformation matrix C and the result of the kernel transformation process GgG T By calculating the Hadamard product with , division becomes unnecessary, reducing the circuit resources required for kernel transformation, such as rounding circuits for the lower bits. As shown in the above equation, if we consider a 4x4 matrix as one unit, there are usually 12 elements that require division per unit, so a total of 12 rounding circuits can be eliminated.

[0051] Here, we consider the calculation precision for the F(2×2,3×3) algorithm using Figure 6. Figure 6 shows the feature map after the Winograd pre-transformation process and how the kernels are multiplied. Figure 6(a) shows a case where division is required in the Winograd pre-transformation process as a comparative example, and a rounding error occurs in the least significant bit of the kernel input to the multiplier 74. Therefore, the lower bits of the 16-bit multiplication result are affected by the rounding error. On the other hand, Figure 6(b) shows a case where division is not required in the Winograd pre-transformation process, as in this embodiment, and no rounding error occurs in the least significant bit of the kernel input to the multiplier 74. Therefore, the lower bits of the 16-bit multiplication result are not affected by the rounding error, and the calculation precision is improved compared to using a normal algorithm.

[0052] Next, returning to Figure 5, the Winograd post-conversion unit 76 of this embodiment will be described. The 4x4 multiplication result output from the multiplier 74 is a matrix. JPEG0007910628000020.jpg1180 With this in mind, the post-Winograd transformation unit 76 applies matrix D, matrix A, and coefficient α to matrix R as follows. Specifically, it calculates the Hadamard product of matrix D and matrix R, and applies matrix A to the result from both sides to obtain a 2x2 matrix. Furthermore, by multiplying all elements of the 2x2 matrix by coefficient α, the final 2x2 convolution result is obtained.

[0053] JPEG0007910628000021.jpg1875

[0054] Since the coefficients of each element of matrix D are all powers of 2, JPEG0007910628000022.jpg1329 The multiplication of the Hadamard product can be achieved with a 1-bit or 2-bit left shifter without increasing circuit resources. On the other hand, in this embodiment, it is necessary to divide each element of the 2x2 matrix by multiplying by the coefficient α, which necessitates rounding of the lower bits. However, rounding the lower bits of the convolution result to quantize to 8 bits, etc., is common in accelerators, which are hardware that perform convolution operations using fixed-point data, and is not unique to this embodiment. Furthermore, the division by the coefficient α does not necessarily have to be performed in the post-Winograd conversion unit 76; it is also possible to perform the division together with the quantization process in the subsequent quantization unit 80.

[0055] The convolution result output by the post-Winograd conversion unit 76 is quantized to 8 bits by the quantization unit 80 so that each PE 58 has the quantization step set, and is held in the cumulative addition unit 82. Then, the convolution results for the number of pre-set input channels are cumulatively added and output from the MAC calculation unit 60.

[0056] Next, the functional configuration of the data processing device 10 will be described. Figure 7 is a block diagram showing an example of the functional configuration of the data processing device 10.

[0057] Functionally, the data processing device 10 comprises a learning unit 20 and an inference unit 22, as shown in Figure 7.

[0058] As shown in Figure 8, the learning unit 20 includes an acquisition unit 30, a processing unit 32, and an update unit 34.

[0059] The acquisition unit 30 acquires the target image and processing result of the input training data.

[0060] The processing unit 32 processes the target images of the training data using a neural network that includes a convolution operation using the Winograd algorithm. When performing the convolution operation, the processing unit 32 calculates the Hadamard product of the kernel transformation result based on the Winograd algorithm and the kernel transformation matrix, and obtains the result of the convolution operation by using the result of the Hadamard product as a multiplication. The processing using the neural network is executed using the accelerator 18. At this time, the target images of the training data and the kernel are input to the accelerator 18, and the processing result is output from the accelerator 18.

[0061] Here, the values ​​of each element of the kernel transformation matrix are powers of 2 and have different constant values ​​corresponding to the divisors of the division required for the kernel transformation process when the kernel transformation matrix is ​​not applied, so that division is not included in the arithmetic operations before multiplication is performed.

[0062] The calculation of the Hadamard product between the kernel transformation result based on the Winograd algorithm and the kernel transformation matrix consists solely of fixed shifters.

[0063] When performing convolutional processing, accelerator 18 operates in Winograd mode if the kernel of the layer is of a specific size (e.g., 3x3), and in non-Winograd mode if the kernel of the layer is not of a specific size.

[0064] The update unit 34 updates the neural network parameters so that the result of processing the target image using the neural network matches the pre-determined processing result.

[0065] The processing steps of the processing unit 32 and the update unit 34 are repeated until a predetermined iteration termination condition is met. This allows the neural network to learn.

[0066] As shown in Figure 9, the inference unit 22 includes an acquisition unit 40 and a processing unit 42.

[0067] The acquisition unit 40 acquires the target image that has been input for processing.

[0068] The processing unit 42 processes the target image using a neural network that includes a convolution operation using the Winograd algorithm. When performing the convolution operation, the Hadamard product of the kernel transformation result based on the Winograd algorithm and the kernel transformation matrix is ​​calculated, and the result of the convolution operation is obtained by using the result of the Hadamard product calculation in multiplication.

[0069] Processing using a neural network is performed using accelerator 18. At this time, the target image and kernel are input to accelerator 18, and the processing result is output from accelerator 18.

[0070] The results of processing the target image using a neural network are displayed on the display unit 16.

[0071] <Operation of the data processing device according to this embodiment> Next, the operation of the data processing device 10 according to this embodiment will be described.

[0072] Figure 10 is a flowchart showing the flow of the learning process by the data processing device 10. The CPU 11 reads the learning process program from the ROM 12 or storage 14, loads it into the RAM 13, and executes it, thereby performing the learning process. At the same time, learning data is input to the data processing device 10. The learning process is an example of a data processing method.

[0073] In step S100, the CPU 11, acting as the acquisition unit 30, acquires the target image and processing result of the input training data.

[0074] In step S102, the CPU 11, as the processing unit 32, uses the accelerator 18 to process the target images of the training data using a neural network that includes convolutional processing.

[0075] In step S104, the CPU 11, acting as an update unit 34, updates the parameters of the neural network so that the results of processing the target images of the training data using the neural network match the pre-determined processing results.

[0076] In step S106, the CPU 11 determines whether a predetermined iteration termination condition has been met. If the iteration termination condition has not been met, the process returns to step S102, and the processing in the processing unit 32 and the update unit 34 are repeated. This allows the neural network to be trained.

[0077] Step S102 described above performs the computational processing for each layer of the neural network. Here, the computational processing for the convolutional layer is implemented by the processing routine shown in Figure 11.

[0078] In step S110, the accelerator 18, as the processing unit 32, determines whether or not to operate in Winograd mode based on the size of the kernel of the convolutional layer. If it is determined to operate in Winograd mode, the process proceeds to step S112. On the other hand, if it is determined not to operate in Winograd mode, the process proceeds to step S114.

[0079] In step S112, the accelerator 18, as the processing unit 32, performs convolution processing using the data path for Winograd mode shown in Figure 5. At this time, the selection units 72 and 78 select Winograd mode.

[0080] In step S114, the accelerator 18, as the processing unit 32, performs convolution processing using the data path for non-Winograd mode shown in Figure 5. At this time, the selection units 72 and 78 select non-Winograd mode.

[0081] Then, the processing routine ends, the feature map is output, and this feature map is used as the input for the next layer.

[0082] Figure 12 is a flowchart showing the data processing flow by the data processing device 10. The CPU 11 reads a data processing program from the ROM 12 or storage 14, loads it into the RAM 13, and executes it, thereby performing data processing. The target image is also input to the data processing device 10. This is an example of a data processing method.

[0083] In step S120, the CPU 11, acting as the acquisition unit 40, acquires the input target image.

[0084] In step S122, the CPU 11, acting as the processing unit 42, uses the accelerator 18 to process the target image using the neural network trained through the learning process described above. The result of processing the target image using the neural network is then displayed on the display unit 16.

[0085] Step S122 described above performs the computational processing for each layer of the neural network. Here, the computational processing for the convolutional layer is implemented by the processing routine shown in Figure 11 above.

[0086] As described above, the data processing device according to this embodiment calculates the Hadamard product of the kernel transformation result based on the Winograd algorithm and the kernel transformation matrix when performing convolution processing, and obtains the result of the convolution processing by using the result of the Hadamard product for multiplication. The value of each element of the kernel transformation matrix is ​​a power of 2 and has different constant values ​​corresponding to the divisor of the division required for kernel transformation processing when the kernel transformation matrix is ​​not applied, and is set so that division is not included in the calculation processing before multiplication is performed. As a result, the circuit size can be reduced while maintaining calculation accuracy in convolution operations using the Winograd algorithm.

[0087] It should be noted that the present invention is not limited to the apparatus configuration and operation of the embodiments described above, and various modifications and applications are possible without departing from the spirit of this invention.

[0088] For example, although the explanation used the case where the data to be processed is an image, it is not limited to this; it can also be data other than images, such as sound data.

[0089] Furthermore, although the explanation described an example where the data processing device includes a learning unit and an inference unit, it is not limited to this. The device with the learning unit and the device with the inference unit may be configured as separate devices.

[0090] Alternatively, the learning unit may train a neural network that includes standard convolutional operations, rather than using the Winograd algorithm.

[0091] Furthermore, while the example given was a kernel size of 3x3 when operating in Winograd mode, it is not limited to this. The kernel size when operating in Winograd mode may also be 5x5 or 7x7. In this case, the implementation should be configured to operate in Winograd mode for a kernel size of 5x5 or 7x7.

[0092] Furthermore, the various processes that the CPU reads and executes in the above embodiment may be executed by various processors other than the CPU. Examples of such processors include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits such as ASICs (Application Specific Integrated Circuits) which have a circuit configuration specifically designed to execute a particular process. In addition, the learning process and data processing may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (for example, multiple FPGAs, and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements.

[0093] Furthermore, although the above embodiments describe a configuration in which the learning processing program and the data processing program are pre-stored (installed) in the storage 14, the invention is not limited thereto. The programs may be provided in a form stored on a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), and USB (Universal Serial Bus) memory. Alternatively, the programs may be provided in a form downloaded from an external device via a network.

[0094] The following additional information is disclosed regarding the embodiments described above.

[0095] (Additional note 1) A data processing device including a neural network that includes a convolution operation using the Winograd algorithm, Memory and At least one processor connected to the memory, Includes, The aforementioned processor, Obtain the target data to be processed, The system is configured to process the target data using a neural network that includes the aforementioned convolution process, When performing the aforementioned convolution process, the Hadamard product of the kernel transformation result based on the Winograd algorithm and the kernel transformation matrix is ​​calculated, and the result of the convolution process is obtained by using the result of the Hadamard product in multiplication. Each element of the kernel transformation matrix is ​​a power of 2 and has different constant values ​​corresponding to the divisor of the division required for the kernel transformation process when the kernel transformation matrix is ​​not applied, and is set so that division is not included in the arithmetic process before multiplication is performed. Data processing device.

[0096] (Additional note 2) A non-temporary storage medium that stores a computer-executable program including a neural network that includes a convolution operation using the Winograd algorithm, The aforementioned data processing is, Obtain the target data to be processed, This includes processing the target data using a neural network that includes the aforementioned convolution process, When performing the aforementioned convolution process, the Hadamard product of the kernel transformation result based on the Winograd algorithm and the kernel transformation matrix is ​​calculated, and the result of the convolution process is obtained by using the result of the Hadamard product in multiplication. Each element of the kernel transformation matrix is ​​a power of 2 and has different constant values ​​corresponding to the divisor of the division required for the kernel transformation process when the kernel transformation matrix is ​​not applied, and is set so that division is not included in the arithmetic process before multiplication is performed. Non-transitory storage medium. [Explanation of Symbols]

[0097] 10 Data Processing Devices 11 CPU 13 RAM 18 Accelerator 20 Learning Department 22 Reasoning part 30 Acquisition Department 32 Processing Units 34 Update section 40 Acquisition Department 42 Processing Unit 58 PE 70 Winograd Pre-Conversion Section 74 Multiplier 76 Post-Winograd Conversion Section

Claims

1. A data processing device including a neural network that includes a convolutional operation using the Winograd algorithm, An acquisition unit that acquires the target data to be processed, The system includes a processing unit that processes the target data using a neural network that includes the aforementioned convolution process, The processing unit, when performing the convolution process, calculates the Hadamard product of the kernel transformation result based on the Winograd algorithm and the kernel transformation matrix, and uses the result of the Hadamard product calculation in multiplication to obtain the result of the convolution process. Each element of the kernel transformation matrix is ​​a power of 2 and has different constant values ​​corresponding to the divisor of the division required for the kernel transformation process when the kernel transformation matrix is ​​not applied, and is set so that division is not included in the arithmetic process before multiplication is performed. Data processing device.

2. The data processing apparatus according to claim 1, wherein the value of each element of the kernel transformation matrix is ​​the divisor of the division required for the kernel transformation process when the kernel transformation matrix is ​​not applied.

3. The data processing apparatus according to claim 1, wherein the calculation of the Hadamard product of the result of the kernel transformation process based on the Winograd algorithm and the kernel transformation matrix is ​​performed using only a fixed shifter.

4. The data processing apparatus according to claim 1, wherein the target data is an image.

5. A data processing method in a data processing device including a neural network that includes a convolution operation using the Winograd algorithm, The acquisition unit acquires the target data to be processed. The processing unit includes processing the target data using a neural network that includes the convolution process, The processing unit, when performing the convolution process, calculates the Hadamard product of the kernel transformation result based on the Winograd algorithm and the kernel transformation matrix, and uses the result of the Hadamard product calculation in multiplication to obtain the result of the convolution process. Each element of the kernel transformation matrix is ​​a power of 2 and has different constant values ​​corresponding to the divisor of the division required for the kernel transformation process when the kernel transformation matrix is ​​not applied, and is set so that division is not included in the arithmetic process before multiplication is performed. Data processing method.

6. A computer including a neural network that includes convolutional processing using the Winograd algorithm, Obtain the target data to be processed, A data processing program for performing the processing of the target data using a neural network including the aforementioned convolution operation, When performing the aforementioned convolution process, the Hadamard product of the kernel transformation result based on the Winograd algorithm and the kernel transformation matrix is ​​calculated, and the result of the convolution process is obtained by using the result of the Hadamard product in multiplication. Each element of the kernel transformation matrix is ​​a power of 2 and has different constant values ​​corresponding to the divisor of the division required for the kernel transformation process when the kernel transformation matrix is ​​not applied, and is set so that division is not included in the arithmetic process before multiplication is performed. Data processing program.

Images