Data processing device, data processing program, and data processing method

Abstract

A data processing device 1 with a minimum precision of N bits for convolution operations performs a convolution operation on two pieces of input data with a width of 2M×N bits (N is a positive integer and M is an integer equal to or larger than 0), and performs minimum-precision product-sum operations when performing a plurality of consecutive processes corresponding to the integer M. If the value of the integer M is not 0, the data processing device 1 performs shift operations on the results of the minimum-precision product-sum operations while performing sign operations in the convolution operation on the input data, reflects a sign, which has been held until a reset signal is received, in the outputs of the shift operations according to the value of the integer M, and calculates a cumulative sum of the outputs of the shift operations with the reflected sign.

Description

Technical Field

[0001] The disclosed technology , predetermined calculation relates to a data processing device, a data processing program, and a data processing method for performing

Background Art

[0002] A convolutional neural network (CNN) is mainly used in image recognition and is characterized by having a "convolutional layer" that performs a convolution operation for extracting feature amounts of an input image. In recent years, object detection algorithms such as YOLO (You Only Look Once) and pose estimation algorithms such as OpenPose based on CNN have been disclosed (Non-Patent Documents 1, 2), and their application to edge AI systems that require real-time performance, such as surveillance cameras mounted on autonomous driving vehicles and drones, has been under consideration. These systems are assumed to require different convolution operation accuracies for each application, and it has been an issue to realize miniaturization while having a mechanism capable of switching the accuracy in one system.

[0003] Therefore, for example, Non-Patent Document 3 shows a processing method for realizing three convolution operation accuracies of 4 bits, 8 bits, and 16 bits with a shared circuit.

[0004] (Non-Patent Document 1) Joseph Redmon, Ali Farhadi, "YOLOv3: An Incremental Improvement", <URL: https: / / arxiv.org / abs / 1804.02767> (Non-Patent Document 2)

[0005] Zhe Cao et al., "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields", <URL: https: / / arxiv.org / pdf / 1611.08050.pdf> (Non-patent document 3)

[0006] Hao Zhang et al., "New Flexible Multiple-Precision Multi1py-Accumulate Unit for Deep Neural Network Training and Inference" [Overview of the project] [Problems that the invention aims to solve]

[0007] Figure 12 illustrates a conventional 3D convolution operation method. In one layer of the network model, if the number of input channels is n (an integer n>0), the n-channel input feature map (iFmap) is multiplied by the n-channel kernels, which are weights for extracting features from the input feature map. If the number of output channels is m (an integer m>0), the multiply-accumulate operation is repeated for m channels to generate an m-channel output feature map (oFmap). The resulting m-channel oFmap becomes the iFmap of the next layer. Note that in the first layer, the input is not an iFmap but video data, and the input channels are typically RGB (3 channels). When implementing the above process with general hardware, if the design reads the iFmap from the memory where it is stored in one cycle, the memory and wiring must be designed to match the largest data size (x, y where x×y is maximized in Figure 12), which increases the circuit size. To avoid increasing the circuit size, the maximum value of iFmap is divided into several blocks, and an iFmap is input to each block to perform a convolution operation and output the result.

[0008] Figure 13 shows a pixel-by-pixel processing method using the technology disclosed in Non-Patent Document 3. A multiply-accumulate circuit for performing convolution operations is provided that corresponds to the maximum value of the operation mode (e.g., 16 bits). By using the same multiply-accumulate circuit when performing convolution operations in 8-bit mode and 4-bit mode, it becomes unnecessary to have separate circuits for each mode. In Figure 13, black circles indicate that the 8-bit multiply-accumulate unit is being used, and white circles indicate that the 8-bit multiply-accumulate unit is not being used.

[0009] In 16-bit mode, all arithmetic units are used to perform multiply-accumulate operations between the input pixel blocks (blk_l, where l is the block number, l>0) obtained by dividing the iFmap into multiple parts and the kernel, and the intermediate results of the oFmap are stored in the cumulative storage memory. This process is repeated for a number of blocks and input channels (iCH_n, where n is the maximum input channel) corresponding to the size of the iFmap, and the cumulative sum is added to generate the oFmap corresponding to the output channel (oCH_m, where m is the maximum output channel).

[0010] In 8-bit mode, the number of blocks is doubled (equivalent to two pixels if we focus on one pixel), and the process is executed in parallel, achieving twice the processing speed. Similarly, in 4-bit mode, the process is executed in parallel with four blocks.

[0011] However, Non-Patent Document 3 describes a processing method that requires the preparation of a multiply-accumulate circuit to match the highest precision calculation mode (16 bits in the above example). Therefore, when using a calculation mode with lower precision than the highest precision mode, the utilization efficiency of both logic and memory becomes worse than when using the highest precision calculation mode. Furthermore, since convolution calculations account for the majority of AI inference processing, there is a problem in that the circuit area is overwhelmingly larger when hardware is prepared to support the highest precision calculation mode compared to when hardware is prepared to support other calculation modes.

[0012] The disclosed technology was developed in view of the above points, and aims to provide a data processing device, a data processing program, and a data processing method that can efficiently perform combined processing of the highest-precision calculation mode and other calculation modes, even when using only the minimum necessary hardware, rather than hardware matched to the highest-precision calculation mode that is possible. [Means for solving the problem]

[0013] The first aspect of this disclosure is: Predetermined calculation The minimum precision is N bits, and two 2 M ×N bit (where N is a positive integer and M is a non-negative integer) width input data A predetermined operation is performed on the A data processing device that performs processing corresponding to a plurality of consecutive M, wherein the minimum precision Multiplication Perform Multiplication part And, if the value of M is not 0, Multiplication part A shifter that performs a shift operation on the calculation result, and if the value of M is not 0, the input data The predetermined calculation for A sign calculation unit that performs sign calculations in the input data The predetermined calculation for A sign holding unit that holds the sign calculated by the sign calculation unit until it receives a reset signal notified each time the process is completed, and reflects the held sign in the output of the shifter according to the value of M; and a cumulative adder unit that cumulatively adds the outputs of the shifter in which the sign has been reflected by the sign holding unit. The predetermined calculation The system includes a cumulative storage memory that stores the cumulative addition calculation result output from the cumulative addition unit during the process.

[0014] A second aspect of this disclosure is: Predetermined calculation The minimum precision is N bits, and two 2 M ×N bit (where N is a positive integer and M is a non-negative integer) width input data The predetermined calculation against A data processing program for performing a process corresponding to a plurality of consecutive M, wherein the minimum precision Multiplication If the value of M is not 0, the minimum precision is MultiplicationPerform a shift process on the calculation result, and when the value of M is not 0, the input data The predetermined calculation for Perform a sign operation on the input data The predetermined calculation for Retain the calculated sign until a reset signal notified each time the input data The predetermined calculation ends, reflect the retained sign in the output of the shift process according to the value of M, accumulate and add the output of the shift process with the sign reflected,

[0015] The third aspect of the present disclosure is Predetermined calculation The minimum precision is N bits, and two 2 M × N-bit (N is a positive integer, M is an integer of 0 or more) width of input data The predetermined calculation against is performed, and a data processing method for performing processes corresponding to a plurality of consecutive M, and the minimum precision Multiplication is performed, and when the value of M is not 0, the minimum precision Multiplication Perform a shift process on the calculation result, and when the value of M is not 0, the input data The predetermined calculation for Perform a sign operation on the input data The predetermined calculation for Retain the calculated sign until a reset signal notified each time the input data The predetermined calculation ends, reflect the retained sign in the output of the shift process according to the value of M, accumulate and add the output of the shift process with the sign reflected,

Advantages of the Invention

[0016] According to the data processing device, data processing program, and data processing method of the present disclosure, even when using the minimum necessary hardware instead of hardware corresponding to the highest possible precision operation mode, the highest possible precision operation mode and combined processing of other operation modes can be efficiently performed.

Brief Description of the Drawings

[0017] [Figure 1]This is a schematic diagram showing the data processing method of the data processing device according to the first embodiment in 16-bit mode. [Figure 2] This figure shows an example of code reflection in 16-bit mode of the data processing device according to the first embodiment. [Figure 3] This figure shows an example of the functional configuration of a data processing device according to the first embodiment. [Figure 4] This is a block diagram showing an example of the hardware configuration of a data processing device according to the first embodiment. [Figure 5] This flowchart shows an example of the flow of a predetermined arithmetic processing in 16-bit mode according to the first embodiment. [Figure 6] This flowchart shows an example of the flow of a predetermined arithmetic processing in 8-bit mode according to the first embodiment. [Figure 7] This is a schematic diagram showing the data processing method of the data processing device according to the second embodiment in 4-bit mode. [Figure 8] This is a schematic diagram showing the data processing method of the data processing device according to the second embodiment in 8-bit mode. [Figure 9] This figure shows an example of the functional configuration of the data processing device according to the second embodiment. [Figure 10] This flowchart shows an example of the flow of a predetermined arithmetic processing in 4-bit mode according to the second embodiment. [Figure 11] This flowchart shows an example of the flow of a predetermined arithmetic processing in 8-bit mode according to the second embodiment. [Figure 12] This is a schematic diagram illustrating a conventional, general 3D convolution operation method. [Figure 13] This is a schematic diagram illustrating a convolution operation method using a multiply-accumulate circuit that supports the maximum possible processing precision. [Modes for carrying out the invention]

[0018] Hereinafter, an example of an embodiment relating to the disclosed technology will be described with reference to the drawings. The same reference numerals will be used throughout the drawings for identical or equivalent components, parts, and processes, and redundant descriptions will be omitted.

[0019] <First Embodiment> In the first embodiment, multiple compatible options Predetermined calculation It has an arithmetic unit that corresponds to the lowest precision (hereinafter referred to as the "minimum precision arithmetic unit"), and by combining the minimum precision arithmetic units, each Predetermined calculation Accuracy-based Predetermined calculation The data processing device 1 (see Figure 3) that realizes this will be described. For the sake of explanation, the data processing device 1 can handle multiple accuracies. Predetermined calculation Of these, the one with the lowest calculation precision Predetermined calculation to "minimum precision" Predetermined calculation And, with a calculation precision higher than the minimum precision, Predetermined calculation This refers to "high precision" Predetermined calculation The data processing device 1 divides the input parameter to be calculated into two data points, the upper bit and the lower bit, both with the same bit width, and performs calculations on the upper bit and lower bit in a time-division multiplexer to achieve high accuracy. Predetermined calculation To achieve this.

[0020] The data processing method according to the first embodiment involves iFmap and kernel Predetermined calculation When the minimum precision of is N bits (N>0, N is an integer), two 2 M Among the input data with a width of ×N bits (where index M is a non-negative integer), a plurality of data defined by any consecutive index M Predetermined calculation This technology is capable of handling high precision. However, as an example, this document describes the data processing method and the configuration of the data processing device 1 for input data where the minimum precision is N=8 and the index is M=0, 1, i.e., when the input data is represented by 8 bits and 16 bits.

[0021] [Data processing method in 16-bit mode] First, we will explain the data processing method in 16-bit mode using an 8-bit arithmetic unit. If the upper 8 bits and lower 8 bits of the 16-bit iFmap are "x" and "y" respectively, and the upper 8 bits and lower 8 bits of the 16-bit kernel are "a" and "b" respectively, and the operator for multiplication is "*", then iFmap*kernel can be expressed as shown in equation (1). Note that "^" is the operator for exponentiation.

[0022] (Math 1) iFmap*kernel ={256*x+y}*{256*a+b} =256^2*ax+256*(ay+bx)+by ···(1)

[0023] Equation (1) shows that by performing a left shift operation that shifts ax to the left by about 16 bits, and also performing left shift operations that shift ay and bx to the left by about 8 bits each, and then adding by to the results of each shift operation, it is possible to perform multiplication of 16-bit data using an 8-bit arithmetic unit. The process of performing a bit shift operation on some value in this way is called a shift operation.

[0024] Figure 1 is a schematic diagram of a 16-bit mode data processing method using the 8-bit arithmetic unit shown in equation (1). In Figure 1, the 8-bit operations for each term are performed in the order from left to right, i.e., operation [1] → operation [2] → operation [3] → operation [4]. Operation [1] represents the operation for the 256^2 * ax term, operation [2] represents the operation for the 256 * bx term, operation [3] represents the operation for the 256 * ay term, and operation [4] represents the operation for the by term. Note that in Figure 1, multiplication is represented by "mul". Thus, in each figure, multiplication may be represented by "mul" and "×" as needed to clearly indicate that it is a multiplication operation.

[0025] First, the data processing unit 1 multiplies the iFmap and the upper 8 bits of kernel, and stores the result of the multiplication shifted left by about 16 bits as the cumulative result in memory (Figure 1: Operation [1]).

[0026] Predetermined calculation Generally, since the calculations are performed on signed data, the data processing device 1 retains the sign determined in calculation [1] until the processing of calculation [4] is completed, and performs calculations on numerical values ​​only in the remaining calculations [2] to calculations [4] without being aware of the sign.

[0027] After the operation [1], the data processing unit 1 multiplies the upper 8 bits of iFmap by the lower 8 bits of kernel, and then multiplies the lower 8 bits of iFmap by the upper 8 bits of kernel. It then shifts the result of each multiplication to the left by about 8 bits and adds the result of the previous operation to the result of the operation, and stores it in memory (Figure 1: Operation [2], Operation [3]).

[0028] Finally, the data processing unit 1 adds the result of multiplying the lower 8 bits of iFmap by the lower 8 bits of kernel to the results of operations [1] to [3] (Figure 1: operation [4]), and reflects the sign determined in operation [1] in the cumulative result of operations [1] to [4], thereby obtaining the final cumulative result shown in Figure 2.

[0029] The data processing unit 1 obtains an oFmap by repeating operations [1] to [4] for all pixels of the iFmap and for the total number of input channels iCH_n. Note that operation [1] must be performed first to determine the sign, but the order of operations [2] to [4] can be changed.

[0030] According to the data processing method of this disclosure, the sign of the cumulative result is determined by processing the upper 8 bits of iFmap and kernel in operation [1], so it is not necessary to input a new sign bit in operations [2] to [4]. Therefore, it is not necessary to have 1-bit wide data representing the sign, and the bit width of the arithmetic unit can be reduced by 1 bit.

[0031] In the disclosed data processing method, an example is shown in which each operation in the operations [1] to [4] is performed on one pixel at a time and on one input channel iCH at a time, but the data processing method is not limited to this. For example, the data processing device 1 may process multiple pixels in parallel within the same input channel iCH, or it may process pixels included in different input channels iCH in parallel.

[0032] [Data processing method in 8-bit mode] Next, we will explain the data processing method in 8-bit mode using an 8-bit arithmetic unit. In 8-bit mode, input data can be directly input to the 8-bit arithmetic unit, so the data processing unit 1 performs calculations with the 8-bit arithmetic unit without splitting the input data into upper and lower bits as in 16-bit mode. That is, the data processing unit 1 multiplies the 8-bit iFmap and the 8-bit kernel, and adds the results of each multiplication without bit shifting to obtain the cumulative result. In this case, there is no need to perform calculations on 16-bit input data in four separate steps as in 16-bit mode, so the processing performance of the data processing unit 1 is four times that of 16-bit mode.

[0033] Figure 3 is a diagram showing an example of the functional configuration of the data processing device 1. As shown in Figure 3, the data processing device 1 is: Multiplication part It includes the following functional units: 2. shifter 3, sign calculation unit 4, sign holding unit 5, cumulative addition unit 6, and cumulative storage memory 7.

[0034] Multiplication part 2 accepts iFmap and kernel and performs minimum precision Multiplication To do so.

[0035] Shifter 3 is used when the value of index M is not 0, i.e., when the calculation mode is high precision. Multiplication part Perform a shift operation on the result of the calculation in step 2.

[0036] Cumulative storage memory 7 is Multiplication part Implemented by 2 and Shifta 3 Predetermined calculationThe cumulative sum of intermediate oFmaps obtained during this process is stored. "Intermediate oFmap" refers to: Predetermined calculation This refers to the intermediate results of oFmap obtained during the process.

[0037] The sign calculation unit 4, when the calculation mode is high precision, Multiplication part Implemented by 2 and Shifta 3 Predetermined calculation Perform a sign operation.

[0038] The code holding unit 5 uses iFmap and kernel Predetermined calculation The code calculated by the sign calculation unit 4 is held until a reset signal, which is notified each time the process is completed, is received, and the held code is reflected in the output of the shifter 3 according to the value of index M.

[0039] The cumulative addition unit 6 is, Multiplication part Implemented by 2 and Shifta 3 Predetermined calculation The intermediate oFmap obtained during this process, in which the sign has been reflected by the sign holding unit 5, is added to the cumulative sum result stored in the cumulative storage memory 7 to update the cumulative sum of the intermediate oFmap.

[0040] The operation of the shifter 3 and the sign calculation unit 4 changes according to an ON / OFF control signal set, for example, according to the calculation mode.

[0041] Specifically, in the 8-bit mode, which is the minimum precision for the data processing device 1, the value of the ON / OFF control signal is set to OFF. When the value of the ON / OFF control signal is set to OFF, the shifter 3 does not perform any shift processing and continues as is. Multiplication part The result of the calculation in step 2 is output to the cumulative addition unit 6. Also, the sign calculation unit 4 does not perform the sign calculation when the value of the ON / OFF control signal is set to OFF.

[0042] On the other hand, in the 16-bit mode, which is a high-precision calculation mode for the data processing device 1, the value of the ON / OFF control signal is set to ON. When the value of the ON / OFF control signal is set to ON, the shifter 3 Multiplication partA shift operation is performed on the result of operation 2. The amount of shift in the shift operation is set according to which of the operations [1] to [4] shown in Figure 1 is being performed. An ON / OFF control signal, set to ON, is input to the sign calculation unit 4 each time operation [1] is performed. When the value of the ON / OFF control signal is set to ON, the sign calculation unit 4 calculates the sign using the most significant bits of the iFmap and kernel input while the value of the ON / OFF control signal is ON, and outputs it to the sign holding unit 5.

[0043] Subsequently, when the calculation shown in Figure 1 [4] is completed in the data processing unit 1, a reset signal is input to the sign holding unit 5. Until the reset signal is input, the sign holding unit 5 reflects the held sign in the calculation result output from the shifter 3 and outputs it to the cumulative adder 6. That is, when the data processing unit 1 is operating in 16-bit mode, Multiplication part 2 Multiplication Every four executions, a reset signal is input to the code holding unit 5, and the code held by the code holding unit 5 is reset.

[0044] Next, an example of the hardware configuration of the data processing device 1 according to the first embodiment of this disclosure will be described. Figure 4 is a block diagram showing an example of the hardware configuration of the data processing device 1. As shown in Figure 4, the data processing device 1 is configured using a computer 10 and has a CPU (Central Processing Unit) 11, ROM (Read Only Memory) 12, RAM (Random Access Memory) 13, storage 14, input unit 15, display unit 16, and communication interface (I / F) 17. Each component is connected to each other so as to be able to communicate with each other via a bus 19.

[0045] The CPU 11 is a central processing unit, which is an example of a processor, and it executes programs and controls various parts. That is, the CPU 11 reads a program from the ROM 12 or storage 14 and executes the program using the RAM 13 as a working area. The CPU 11 controls each functional part shown in Figure 3 and performs various calculations according to the program stored in the ROM 12 or storage 14. As an example, in the first embodiment, the ROM 12 or storage 14 contains, Predetermined calculation This contains data processing programs for executing the processing.

[0046] ROM12 stores various programs and data. RAM13 temporarily stores programs or data as a working area. Storage14 consists of a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) and stores various programs, including the operating system, and various data.

[0047] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used for various types of input.

[0048] The display unit 16 is, for example, a liquid crystal display and displays various information. The display unit 16 may also function as an input unit 15 by employing a touch panel system.

[0049] Communication I / F17 is an interface for communicating with other devices. This communication may utilize wired communication standards such as Ethernet® or FDDI, or wireless communication standards such as 4G, 5G, or Wi-Fi®.

[0050] Depending on the circumstances, the input unit 15, display unit 16, and communication interface 17 do not necessarily have to be included in the computer 10.

[0051] Next, the operation of the data processing device 1 according to the first embodiment will be described.

[0052] Figure 5 shows the execution performed by the CPU 11 of the data processing unit 1 in 16-bit mode. Predetermined calculation This is a flowchart showing an example of the processing flow.

[0053] Predetermined calculation The data processing program that defines the processing is pre-stored in, for example, the ROM 12 of the data processing device 1. The CPU 11 of the data processing device 1 reads the data processing program stored in the ROM 12. Predetermined calculation The process will be executed. Note that CPU11 Predetermined calculation Before executing the process, for example, the cumulative sum value stored in RAM13 is initialized to "0".

[0054] When an iFmap and kernel for any one of the input channels iCH_n are input, in step S10, the CPU 11 selects one pixel from the iFmap and obtains the pixel value of the selected pixel from the iFmap and the kernel value of the kernel corresponding to the selected pixel from the iFmap. Both the pixel value obtained from the iFmap and the kernel value obtained from the kernel are represented in 16 bits. For the sake of explanation, the value of the selected iFmap pixel is called the "selected pixel value".

[0055] In step S20, the CPU 11 divides the selected pixel value into upper 8 bits and lower 8 bits, and also divides the kernel value into upper 8 bits and lower 8 bits. The divided upper 8 bits and lower 8 bits of the selected pixel value correspond to "x" and "y" shown in equation (1), respectively. The divided upper 8 bits and lower 8 bits of the kernel value correspond to "a" and "b" shown in equation (1), respectively.

[0056] In step S30, the CPU 11 selects one of the following combinations according to equation (1): selected pixel value "x" and kernel value "a", selected pixel value "x" and kernel value "b", selected pixel value "y" and kernel value "a", and selected pixel value "y" and kernel value "b". However, in order to determine the sign of the calculation result, the CPU 11 selects the combination of selected pixel value "x" and kernel value "a" in the first selection.

[0057] In step S40, the CPU 11 performs a multiplication operation by multiplying the combinations selected in step S30. If a combination of selected pixel value "x" and kernel value "a" was selected in step S30, the CPU 11 stores the sign of the multiplication result in RAM 13.

[0058] In step S50, the CPU 11 performs a shift operation on the multiplication result of step S40 by a shift amount uniquely determined from equation (1) for each combination of divided selected pixel value and kernel value.

[0059] In step S60, the CPU 11 applies the sign stored in the RAM 13 in step S40 to the calculation result of step S50, and performs an accumulation addition process in which it adds the calculation result with the applied sign to the accumulated sum value.

[0060] In step S70, the CPU 11 determines whether all combinations of selected pixel values ​​and kernel values ​​based on equation (1) have been selected. If there are any unselected combinations, the process proceeds to step S30, where one of the unselected combinations is selected, and steps S30 to S70 are repeated until all combinations are selected. As already explained, in the case of 16-bit wide input data, steps S30 to S70 are repeated four times for each pixel included in the iFmap. On the other hand, if all combinations have been selected, the process proceeds to step S80. In this case, the CPU 11 deletes the codes stored in RAM 13 in step S40 and resets the codes.

[0061] In step S80, the CPU 11 determines whether all pixels included in the input iFmap have been selected. If the iFmap contains unselected pixels, the process proceeds to step S10, where one of the unselected pixels is selected, and steps S10 to S80 are repeated until all pixels are selected. On the other hand, if all pixels included in the iFmap have been selected, the process proceeds to the 16-bit mode shown in Figure 5. Predetermined calculation Processing will now be terminated.

[0062] Therefore, the relationship between iFmap and kernel for one channel Predetermined calculation It has finished, Predetermined calculation The cumulative sum obtained is stored in RAM13 as the pixel value of oFmap. If there are n channels of iFmap, the CPU11 is as shown in Figure 5. Predetermined calculation The process should be repeated as many times as there are input channels.

[0063] In 16-bit mode Predetermined calculation The process involves time-division multiplexing, as shown in Non-Patent Document 3. Predetermined calculation While the processing performance is halved compared to the previous method, the area of ​​hardware resources required for the arithmetic unit is reduced to one-quarter because only one 8-bit arithmetic unit is needed.

[0064] Note that the following is shown in Figure 5. Predetermined calculation During processing, the CPU 11 receives input data such as iFmap and kernel, and then divides the input data according to the minimum precision of the arithmetic unit (see step S20 in Figure 5). However, there are no constraints on the timing of the input data division. For example, the CPU 11 may divide the pixel values ​​of oFmap into the minimum precision bit width before storing them in RAM 13.

[0065] Figure 5 shows the data processing unit 1 in 16-bit mode. Predetermined calculation Now that I've explained the process, next I will explain the data processing unit 1 in 8-bit mode. Predetermined calculation Let me explain the process.

[0066] Figure 6 shows the execution performed by the CPU 11 of the data processing unit 1 in 8-bit mode. Predetermined calculation This is a flowchart showing an example of the processing flow. The difference between the flowchart in Figure 6 and the flowchart in Figure 5 is that steps S20, S30, S50, and S70 have been removed, and steps S40 and S60 have been replaced with steps S40A and S60A, respectively. Note that CPU 11 is in 16-bit mode. Predetermined calculation Similar to the process, Predetermined calculation The cumulative sum is initialized to "0" before executing the process.

[0067] When an iFmap and kernel for any one of the input channels iCH_n are input, in step S10, the CPU 11 selects one pixel from the iFmap and obtains the pixel value of the selected pixel from the iFmap and the kernel value of the kernel corresponding to the selected pixel from the iFmap. Both the selected pixel value and the kernel value are represented by 8 bits.

[0068] In step S40A, the CPU 11 performs a multiplication operation, which involves multiplying the selected pixel value by the kernel value.

[0069] In step S60A, the CPU 11 performs an accumulation addition process in which it adds the multiplication result obtained in step S40A to the accumulated sum.

[0070] In step S80, the CPU 11 determines whether all pixels included in the input iFmap have been selected. If the iFmap contains unselected pixels, the process proceeds to step S10, where one of the unselected pixels is selected, and steps S10 to S80 are repeated until all pixels are selected. On the other hand, if all pixels included in the iFmap are selected, the process proceeds to the 8-bit mode shown in Figure 6. Predetermined calculation The process is terminated. Thus, in the case of 8-bit wide input data, steps S40A and S60A are performed only once for each pixel included in the iFmap.

[0071] In the first embodiment, the data processing device 1 in 8-bit mode and 16-bit mode Predetermined calculation Although processing examples have been described, the bit widths of input data that can be handled by the data processing device 1 are not limited to just 8 bits and 16 bits. The data processing device 1 can also handle input data with multiple other bit widths, such as 4 bits, 8 bits, and 16 bits. Predetermined calculation Processing can be performed. In this case, since the minimum precision is 4 bits, the data processing device 1 uses a 4-bit arithmetic unit. Predetermined calculation The process will be carried out.

[0072] When using a 4-bit arithmetic unit, the data processing unit 1 divides the 8-bit and 16-bit wide input data into 4-bit wide input data, and performs the time-division multiplexing process described above on the divided data, thereby achieving the corresponding precision. Predetermined calculation The processing should be performed. Specifically, the data processing device 1 should perform the processing once in 4-bit mode, four times in 8-bit mode, and sixteen times in 16-bit mode. Multiplication By repeating this process, 4-bit, 8-bit, and 16-bit operations can be achieved, respectively.

[0073] Thus, according to the data processing device 1 of the first embodiment, even when using hardware resources matched to the minimum calculation precision among the available calculation precisions, it is possible to efficiently perform combined processing of high-precision calculation modes.

[0074] <Second Embodiment> In the first embodiment, an example was shown in which a high-precision arithmetic mode is realized by dividing the input data to match the minimum precision and performing time-division multiplexing on the divided input data, based on the minimum precision. However, when realizing a higher precision arithmetic mode based on the minimum precision arithmetic mode, there is a tendency for the processing performance to worsen as the precision of the arithmetic mode increases. For example, in the case of three arithmetic modes, 4-bit, 8-bit, and 16-bit, the processing performance of the 8-bit mode is 1 / 4 and the processing performance of the 16-bit mode is 1 / 16 compared to the processing performance of the minimum precision 4-bit mode.

[0075] In the second embodiment, instead of using the lowest precision calculation mode as the reference, other precision calculation modes are used as the reference, and the time-division processing shown in the first embodiment is used only when the calculation mode is higher than the reference precision. Predetermined calculation The data processing device 1A that performs the processing will be described below. Hereafter, the pre-set reference calculation precision will be referred to as "reference precision".

[0076] The data processing method according to the second embodiment is the same as the data processing method according to the first embodiment, using iFmap and kernel Predetermined calculation When the minimum precision of is N bits, two 2 M Among the input data with a width of ×N bits, multiple data defined by an arbitrary consecutive index M Predetermined calculationThis technology is capable of handling high precision. However, as an example, this document describes the data processing method and the configuration of the data processing device 1A for input data where the minimum precision is N=4 and the index is M=0, 1, or 2, i.e., when the input data is represented by 4 bits, 8 bits, and 16 bits. In the data processing device 1A, the minimum precision is 4 bits, but the reference precision is 8 bits. That is, the minimum granularity of the arithmetic unit in the data processing device 1A is a 4-bit arithmetic unit, but the data processing device 1A has a configuration that allows for 8-bit arithmetic as a hardware resource.

[0077] [Data processing method in 4-bit mode] First, we will explain the data processing method in 4-bit mode when the reference precision is 8 bits.

[0078] As explained above, the arithmetic unit of the data processing unit 1A is a 4-bit arithmetic unit, but the data processing unit 1A has hardware resources capable of 8-bit operations. Therefore, in 4-bit mode, the data processing unit 1A can process input data for two channels in parallel, such as input channel iCH x 2 and output channel oCH x 2, and output the calculation results for two channels in parallel.

[0079] To perform parallel operations on the output channel oCH, the amount of kernel supplied must be doubled compared to performing operations on a single output channel oCH. However, since the input channel iCH is parallel and its bit width is halved, the processing is no different from when the iFmap input bus width is 8 bits.

[0080] Based on the above, the 4-bit mode data processing method in data processing device 1A will be explained in detail with reference to Figure 7.

[0081] To input the iFmaps of two input channels iCH (for example, iCH_0 and iCH_1) in parallel, Figure 7 shows that for an 8-bit wide input with reference precision, the iFmap of the odd input channel iCH_1 is set to the upper 4 bits, and the iFmap of the even input channel iCH_0 is set to the lower 4 bits.

[0082] The data processing device 1A sets kernel_o_i corresponding to input channel iCH_0 and output channel oCH_0, and input channel iCH_1 and output channel oCH_1, respectively, and multiplies kernel_o_i by the iFmap of input channel iCH_0, and by kernel_o_i by the iFmap of output channel oCH_0. Here, in kernel_o_i, "o" is the number of output channel oCH, and "i" is the number of input channel iCH, and o and i are positive integers. Specifically, the kernels corresponding to input channel iCH_0 and output channel oCH_0 are kernel_0_0, kernel_1_0, kernel_0_1, and kernel_1_1, respectively.

[0083] After multiplying kernel_o_i by the iFmap of input channel iCH_0, and kernel_o_i by the iFmap of output channel oCH_0, the data processing unit 1A adds the multiplication results for each output channel oCH. Specifically, the data processing unit 1A adds the terms of the multiplication results with kernel_o_i that have the same output channel oCH number, such as “iCH_0*kernel_0_0+iCH_1*kernel_0_1” and “iCH_0*kernel_1_0+iCH_1*kernel_1_1”.

[0084] Then, the data processing device 1A accumulates and sums the sum of the multiplication results for each output channel oCH, and stores them in the cumulative storage memory as intermediate results for the oFmap of output channel oCH_0 and the oFmap of output channel oCH_1, respectively.

[0085] The above MultiplicationBy repeatedly performing this for each pixel included in iFmap, the final oFmap for output channel oCH_0 and the final oFmap for output channel oCH_1 are obtained. Multiplication By repeating this process for the number of output channels oCH_m, the oFmap for all output channels oCH can be obtained.

[0086] These Multiplication Therefore, four 4-bit arithmetic units are needed, each corresponding to two input channels iCH and two output channels oCH. However, since the reference precision is 8 bits, the four 4-bit arithmetic units can be used in parallel.

[0087] [Data processing method in 8-bit mode] Next, we will explain a data processing method in 8-bit mode using a 4-bit arithmetic unit when the reference precision is 8 bits.

[0088] In 8-bit mode, similar to the [data processing method in 16-bit mode] shown in the first embodiment, the data processing device 1A splits the 8-bit input data of iFmap[7:0] and kernel[7:0] into the upper 4 bits (iFmap[7:4] and kernel[7:4]) and the lower 4 bits (iFmap[3:0] and kernel[3:0]), respectively, and performs a multiplication of iFmap[7:0]*kernel[7:0]. "[p:q]" is a symbol representing the range from the q-th bit (q≧0, q is an integer) to the p-th bit (p>q, p is an integer). Therefore, for example, iFmap[7:0] represents the 8 bits from the 0th bit to the 7th bit of iFmap.

[0089] The principle by which iFmap[7:0]*kernel[7:0] can be calculated by splitting iFmap[7:0] and kernel[7:0] into iFmap[7:4], kernel[7:4], iFmap[3:0], and kernel[3:0] is as explained in the first embodiment. Therefore, if iFmap[7:4] is represented as iCH(h), iFmap[3:0] as iCH(l), kernel[7:4] as kernel(h), and kernel[3:0] as kernel(l), then iFmap[7:0]*kernel[7:0] can be expressed as shown in equation (2).

[0090] (Math 2) iFmap[7:0]*kernel[7:0] =2^8^2*iCH(h)*kernel(h)+ 2^8*(iCH(h)*kernel(l)+ iCH(l)*kernel(h))+ iCH(l)*kernel(l) ···(2)

[0091] Equation (2) shows that multiplication of 8-bit data using a 4-bit arithmetic unit can be achieved by 4-bit multiplication, left shift operation, and addition. Since the data processing unit 1A with a reference precision of 8 bits has four 4-bit arithmetic units, by using the four 4-bit arithmetic units in parallel, the multiplication in equation (2) can be performed all at once without time-division multiplexing.

[0092] Figure 8 is a schematic diagram of an 8-bit mode data processing method using a 4-bit arithmetic unit when the reference precision shown in equation (2) is 8 bits. Figure 8 shows an example of multiplication between the input channel iCH_0 and kernel_0_0 corresponding to the input channel iCH_0 and the output channel oCH_0, respectively.

[0093] Because the reference precision is 8 bits, it is not possible to process input data for two channels in parallel in 8-bit mode as in 4-bit mode. The data processing device 1A uses input channels iCH_0 and kernel_0_0 to perform multiplication, left shift operation, and addition of iFmap and kernel, which are divided into 4-bit widths, and stores the cumulative sum of the operation results in the cumulative storage memory as an intermediate result in output channel oCH_0.

[0094] The above Multiplication By repeatedly performing this operation for each pixel included in the iFmap of the input channel iCH_0, the final oFmap of the output channel oCH_0 is obtained. Multiplication By repeating this process for the number of output channels oCH_m, the oFmap for all output channels oCH can be obtained.

[0095] As already explained, Predetermined calculation Generally, when performing calculations on signed data, the most significant bit of the input data is assigned to the sign. However, for 8-bit operations combining the upper and lower data after partitioning, the data processing device 1A performs the operation shown in equation (2) using the pixel value of the iFmap of the input channel iCH and the upper and lower data of the kernel, excluding the most significant bit. Then, the data processing device 1A performs an xnor operation between the most significant bit, which is the sign bit of the iFmap of the input channel iCH, and the most significant bit, which is the sign bit of the kernel, and outputs it as the final sign of the oFmap.

[0096] [Data processing method in 16-bit mode] Next, we will explain a 16-bit mode data processing method using a 4-bit arithmetic unit when the reference precision is 8 bits.

[0097] Since the reference precision is 8 bits, the bit width of data that can be processed in a batch by the data processing device 1A is limited to 8 bits. Therefore, as described in the first embodiment, the data processing device 1A divides the pixel value of the 16-bit iFmap into the upper 8 bits and the lower 8 bits, and also divides the kernel value of the 16-bit kernel into the upper 8 bits and the lower 8 bits, and performs time-division multiplexing on each of the divided 8-bit data in four steps from operation [1] to operation [4].

[0098] However, since the arithmetic unit according to the second embodiment is a 4-bit arithmetic unit, when the data processing device 1A performs calculations on 8-bit data, the method described in [Data processing method in 8-bit mode] in the second embodiment will be used.

[0099] Thus, the data processing device 1A has a reference accuracy. Predetermined calculation By repeating this process, input data with a bit width larger than the reference precision can be processed. Predetermined calculation It is possible to do so.

[0100] Figure 9 shows an example of the functional configuration of the data processing device 1A. The difference between the example of the functional configuration of the data processing device 1A shown in Figure 9 and the example of the functional configuration of the data processing device 1 according to the first embodiment shown in Figure 3 is that a precision increasing adder 8 is added. Multiplication part 2. The sign calculation unit 4 and the sign holding unit 5 are respectively Multiplication part The key difference is that it has been replaced by 2A, the sign calculation unit 4A, and the sign holding unit 5A.

[0101] Multiplication part 2A accepts iFmap and kernel and uses the minimum precision arithmetic unit to perform a reference precision calculation. Multiplication To do so.

[0102] The sign calculation unit 4A performs an xnor operation between the most significant bit, which is the sign bit of the pixel value of the iFmap, and the most significant bit, which is the sign bit of the kernel value, to determine the sign and outputs it to the sign holding unit 5A.

[0103] The code holding unit 5A, upon receiving the output control signal, reflects the held code in the intermediate oFmap output by the precision-increasing adder 8, which will be described later. The output control signal is input to the code holding unit 5A at the same time that the precision-increasing adder 8 outputs the intermediate oFmap.

[0104] The precision-increasing addition unit 8 performs addition to generate a reference precision calculation result from the minimum precision calculation result. Specifically, the precision-increasing addition unit 8 performs addition to the minimum precision calculation result from which the left shift calculation was performed by the shifter 3 according to the specified shift amount. Multiplication The results of each operation are added together, and the input data has a reference precision of a bit width that is a multiple of at least twice the minimum precision (in this case, 8 bits). Predetermined calculation Generates the result of the calculation.

[0105] Furthermore, in 4-bit mode, as already explained, in order to perform parallel calculations on the output channel oCH, the amount of kernel supplied must be doubled compared to when calculating on a single output channel oCH. Therefore, as shown in Figure 9... Multiplication part The input bit width of the kernel input to 2A and the sign operation unit 4A is twice the input bit width of the kernel of the data processing device 1 according to the first embodiment shown in Figure 3. However, if it is not necessary to perform parallel operations on the output channel oCH, Multiplication part The input bit width of the kernel input to 2A and the sign arithmetic unit 4A may be the same as the input bit width of the kernel of the data processing device 1 according to the first embodiment shown in Figure 3.

[0106] Here, as an example, the input bit width of the kernel is set to twice the minimum precision, but it may also be set to K times the minimum precision (where K is an integer greater than or equal to 2).

[0107] Furthermore, the data processing device 1A can also be configured using the computer 10 shown in Figure 4, just like the data processing device 1 according to the first embodiment.

[0108] Next, the operation of the data processing device 1A according to the second embodiment will be described.

[0109] Figure 10 shows the execution performed by the CPU 11 of the data processing unit 1A in 4-bit mode. Predetermined calculation This is a flowchart showing an example of the processing flow.

[0110] Predetermined calculation The data processing program that defines the processing is pre-stored in, for example, the ROM 12 of the data processing device 1A. The CPU 11 of the data processing device 1A reads the data processing program stored in the ROM 12. Predetermined calculation The process will be executed. Note that CPU11 Predetermined calculation Before executing the process, for example, the cumulative sum value stored in RAM13 is initialized to "0".

[0111] When four kernel_o_i are input, namely iFmaps for any two channels of the input channel iCH_n and kernels for two channels corresponding to each iFmap, in step S100, the CPU 11 selects one pixel from each iFmap and obtains the pixel value of the selected pixel from each iFmap and the kernel value of the kernel_o_i corresponding to the selected pixel from each iFmap. Both the selected pixel value and the kernel value obtained from kernel_o_i are represented by 4 bits.

[0112] For the sake of explanation, the following examples will use the iFmaps of iCH_0 and iCH_1, and the input kernel_0_0, kernel_1_0, kernel_0_1, and kernel_1_1, as shown in Figure 10. Predetermined calculation I will explain the process.

[0113] Since the reference precision of the arithmetic unit in computer 10 is 8 bits, in step S110, the CPU 11 generates an 8-bit wide parallel pixel value where the selected pixel value of iCH_1 is the upper 4 bits and the selected pixel value of iCH_0 is the lower 4 bits, and two 8-bit wide parallel kernel values ​​obtained from kernel_o_i that share the same output channel oCH, thereby aligning the pixel value and kernel value to the reference precision.

[0114] In step S120, the CPU 11 performs a multiplication operation between the parallel pixel value generated in step S110 and the two parallel kernel values, multiplying the upper 4 bits and lower 4 bits of each. This yields the multiplication results of "iCH_0*kernel_0_0", "iCH_0*kernel_1_0", "iCH_1*kernel_0_1", and "iCH_1*kernel_1_1".

[0115] In step S130, the CPU 11 adds together the terms resulting from multiplication with the same output channel oCH and kernel_o_i. This yields “iCH_0*kernel_0_0+iCH_1*kernel_0_1” and “iCH_0*kernel_1_0+iCH_1*kernel_1_1” as the sum of the multiplication results for each output channel oCH.

[0116] Then, the CPU 11 performs a cumulative addition process, which adds the sum of the multiplication results for each output channel oCH to the cumulative sum value prepared for each output channel oCH.

[0117] In step S140, the CPU 11 determines whether all pixels included in each input iFmap have been selected. If each iFmap contains unselected pixels, the process proceeds to step S100, where one of the unselected pixels is selected from each iFmap, and steps S100 to S140 are repeated until all pixels are selected. On the other hand, if all pixels included in each iFmap have been selected, the process proceeds to the 4-bit mode shown in Figure 10. Predetermined calculation Processing will now be terminated.

[0118] Therefore, the relationship between the iFmap and kernel for 2 channels Predetermined calculation The event took place. Predetermined calculation The cumulative sum obtained is stored in RAM13 as the pixel value of oFmap. If there are n channels of iFmap, the CPU11 is as shown in Figure 10. Predetermined calculationThe process should be repeated until n channels of iFmap have been processed.

[0119] Next, the data processing unit 1A in 8-bit mode Predetermined calculation Let me explain the process.

[0120] Figure 11 shows the execution performed by the CPU 11 of the data processing unit 1A in 8-bit mode. Predetermined calculation This is a flowchart showing an example of the processing flow.

[0121] When an iFmap and kernel for any one of the input channels iCH_n are input, in step S200, the CPU 11 selects one pixel from the iFmap and obtains the pixel value of the selected pixel from the iFmap and the kernel value of the kernel corresponding to the selected pixel from the iFmap. Both the selected pixel value obtained from the iFmap and the kernel value obtained from the kernel are represented in 8 bits.

[0122] In step S210, the CPU 11 performs an xnor operation on the most significant bit of the selected pixel value and the most significant bit of the kernel value to determine the sign of the oFmap. The CPU 11 stores the result of the xnor operation representing the sign in the RAM 13.

[0123] In step S220, the CPU 11 divides the selected pixel value into upper 4 bits and lower 4 bits, and also divides the kernel value into upper 4 bits and lower 4 bits. The divided upper 4 bits and lower 4 bits of the selected pixel value correspond to “iCH(h)” and “iCH(l)” shown in equation (2), respectively. Similarly, the divided upper 4 bits and lower 4 bits of the kernel value correspond to “kernel(h)” and “kernel(l)” shown in equation (2), respectively.

[0124] In step S230, the CPU 11 performs a multiplication operation using four 4-bit arithmetic units to calculate equation (2), which involves simultaneously performing the operations on iCH(h)*kernel(h), iCH(h)*kernel(l), iCH(l)*kernel(h), and iCH(l)*kernel(l).

[0125] In step S240, the CPU 11 performs a shift operation on each of the multiplication results of the divided selected pixel value and kernel value by a shift amount uniquely determined from equation (2). Specifically, the CPU 11 left-shifts iCH(h)*kernel(h) by about 16 bits, left-shifts iCH(h)*kernel(l) and iCH(l)*kernel(h) by about 8 bits, and does not perform a left-shift operation on iCH(l)*kernel(l).

[0126] In step S250, the CPU 11 applies the sign stored in RAM 13 in step S210 to the sum of the calculation results for which the shift processing was performed in step S240, and performs a cumulative addition process in which the sum of the added values ​​with the applied signs is added to the cumulative sum.

[0127] In step S260, the CPU 11 determines whether all pixels included in the input iFmap have been selected. If the iFmap contains unselected pixels, the process proceeds to step S200, where one of the unselected pixels is selected, and steps S200 to S260 are repeated until all pixels are selected. On the other hand, if all pixels included in the iFmap are selected, the 8-bit mode shown in Figure 11 is used. Predetermined calculation Processing will now be terminated.

[0128] Therefore, the relationship between iFmap and kernel for one channel Predetermined calculation The event took place. Predetermined calculation The cumulative sum obtained is stored in RAM13 as the pixel value of oFmap. If there are n channels of iFmap, the CPU11 is as shown in Figure 11. Predetermined calculationThe process should be repeated as many times as there are input channels.

[0129] Furthermore, the data processing unit 1A in 16-bit mode Predetermined calculation The processing is performed in 16-bit mode by the data processing device 1 according to the first embodiment shown in Figure 5. Predetermined calculation The same processing can be performed. However, the minimum precision of the arithmetic unit in the data processing device 1 according to the first embodiment is 8 bits, and the minimum precision of the arithmetic unit in the data processing device 1A according to the second embodiment is 4 bits. Therefore, when performing 8-bit operations on the pixel values ​​of the iFmap and the kernel values ​​of the kernel, which are divided into 8-bit widths in step S20 of Figure 5, the 8-bit operations will be performed by a 4-bit arithmetic unit through the processing shown in steps S220 to S250 of Figure 11.

[0130] Thus, according to the data processing device 1A of the second embodiment, by adding a precision-increasing adder 8 to the data processing device 1 of the first embodiment, it is possible to realize a reference precision calculation using a minimum precision arithmetic unit. Furthermore, the data processing device 1A has a reference precision Predetermined calculation By repeating this process multiple times, calculations with a precision greater than the reference precision can also be achieved.

[0131] In the first and second embodiments, the case where the bit width of the iFmap pixel value and the bit width of the kernel value are the same was described, but this is just an example, and the bit width of the iFmap pixel value and the bit width of the kernel value may be different.

[0132] Although one embodiment of the data processing devices 1 and 1A has been described above, the disclosed embodiment of the data processing devices 1 and 1A is merely an example, and the embodiment of the data processing devices 1 and 1A is not limited to the scope described in each embodiment. Various modifications or improvements can be made to each embodiment without departing from the gist of this disclosure, and such modified or improved embodiments are also included in the technical scope of the disclosure. For example, without departing from the gist of this disclosure, as shown in Figures 5, 6, 10, and 11. Predetermined calculation The internal processing order in the process may be changed.

[0133] Furthermore, this disclosure provides an example. Predetermined calculation The method of implementing the processing in software has been described. However, the same processing as the flowcharts shown in Figures 5, 6, 10, and 11 may also be implemented in hardware, for example, using an ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or PLD (Programmable Logic Device). In this case, Predetermined calculation Compared to implementing the process in software, this method allows for faster processing.

[0134] Thus, the CPU 11 of the data processing units 1 and 1A may be replaced with a dedicated processor specialized for a specific type of processing, such as an ASIC, FPGA, PLD, GPU (Graphics Processing Unit), or FPU (Floating Point Unit).

[0135] Predetermined calculation The processing can be implemented by a single CPU 11, or by a combination of two or more processors of the same or different types, such as multiple CPUs 11, or a combination of a CPU 11 and an FPGA.

[0136] Furthermore, Predetermined calculation The processing may be achieved, for example, through the collaboration of processors located in physically separate locations connected via the Internet.

[0137] Furthermore, while each embodiment describes an example in which the data processing program is stored in the ROM 12 of the data processing devices 1 and 1A, the storage location of the data processing program is not limited to the ROM 12. The data processing program of this disclosure can also be provided in a form recorded on a storage medium readable by the computer 10. For example, the data processing program may be provided in a form recorded on an optical disc such as a CD-ROM (Compact Disk Read Only Memory) or DVD-ROM (Digital Versatile Disk Read Only Memory). Alternatively, the data processing program may be provided in a form recorded on a portable semiconductor memory such as a USB (Universal Serial Bus) memory or a memory card.

[0138] ROM12, storage14, CD-ROM, DVD-ROM, USB, and memory cards are examples of non-transitory storage media.

[0139] Furthermore, data processing devices 1 and 1A may download data processing programs from external devices via the communication interface 17 and store the downloaded data processing programs in, for example, the storage 14. In this case, data processing devices 1 and 1A read the data processing programs downloaded from the external devices. Predetermined calculation Execute the process.

[0140] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0141] With regard to the embodiments described above, the following additional information is disclosed.

[0142] (Additional note 1) Predetermined calculation The minimum precision is N bits, and two 2 M ×N bit (where N is a positive integer and M is a non-negative integer) width input data A predetermined operation is performed on the A data processing device that performs a process corresponding to a plurality of consecutive M, Memory and At least one processor connected to the memory, Includes, The aforementioned processor, The minimum precision of Multiplication Perform If the value of M is not 0, Multiplication Perform a shift operation on the result of the calculation. If the value of M is not 0, the input data The predetermined calculation for Perform a sign operation in The aforementioned input control The predetermined calculation for The calculated code is held until a reset signal, which is notified each time the process is completed, is received, and the held code is reflected in the output of the shift process according to the value of M. The output of the shift process, with the sign reflected, is accumulated and added together. The predetermined calculation The result of the cumulative addition calculation obtained during this process is stored in the memory. Data processing device.

[0143] (Additional note 2) Predetermined calculation The minimum precision is N bits, and two 2 M ×N bit (where N is a positive integer and M is a non-negative integer) width input data The predetermined calculation against A non-temporary storage medium storing a data processing program that can be executed by a computer to perform data processing corresponding to a plurality of consecutive M, The aforementioned data processing is, The minimum precision of Multiplication Perform If the value of M is not 0, Multiplication Perform a shift operation on the result of the calculation. If the value of M is not 0, the input data The predetermined calculation for Perform a sign operation in The aforementioned input control The predetermined calculation forThe calculated code is held until a reset signal, which is notified each time the process is completed, is received, and the held code is reflected in the output of the shift process according to the value of M. The output of the shift process, with the sign reflected, is accumulated and added together. The predetermined calculation The result of the cumulative addition calculation obtained during this process is stored in the memory. Non-transitory storage medium.

Claims

1. The minimum precision of a predetermined operation is N bits, and two 2 Ｍ A data processing device that performs the predetermined operation on input data with a width of ×N bits (where N is a positive integer and M is a non-negative integer) and performs time-division processing corresponding to a plurality of consecutive M values, A multiplication unit that performs the minimum precision multiplication, If the value of M is not 0, a shifter performs a shift operation on the result of the calculation in the multiplication unit, If the value of M is not 0, a sign calculation unit performs a sign calculation in the predetermined operation on the input data, A code holding unit holds the code calculated by the code calculation unit until it receives a reset signal that is notified each time the processing of all combinations of calculation targets in the predetermined calculation on the input data is completed, and reflects the held code in the output of the shifter according to the value of M. A cumulative summing unit that cumulatively adds the output of the shifter, in which the code has been reflected by the code holding unit, A cumulative storage memory that stores the cumulative addition calculation result output from the cumulative addition unit during the predetermined calculation process, A data processing device equipped with the following features.

2. The multiplication unit performs the predetermined operation on the input data by repeating the minimum precision multiplication a predetermined number of times according to the value of M. The shifter performs a left shift operation on the result of the minimum precision multiplication operation according to a shift amount that is set in advance according to the combination of the targets of the minimum precision multiplication operation that is repeatedly performed in the multiplication unit for the predetermined operation on the input data. The data processing device according to claim 1.

3. When the multiplication unit performs the predetermined operation on each of the input data, it first performs multiplication of the most significant N-bit units of each of the input data, which are divided into N-bit units. The data processing apparatus according to claim 2.

4. The system further includes a precision-increasing adder that adds the results of each of the minimum-precision multiplication operations performed by left-shifting according to the shift amount, and generates the result of the predetermined operation on the input data, with a reference precision of twice or more the minimum precision bit width. The data processing apparatus according to claim 2.

5. By repeatedly performing the predetermined calculation for the reference precision using the multiplication unit, the shifter, and the precision-increasing addition unit, the predetermined calculation is performed on the input data with a bit width larger than the reference precision. The data processing device according to claim 4.

6. One of the input data is image data, and the other of the input data is a kernel for extracting features of the image. The input bit width of the kernel input to the multiplication unit and the sign operation unit is K times the minimum precision bit width (where K is an integer of 2 or more). The data processing device according to claim 5.

7. The minimum precision of a predetermined operation is N bits, and two 2 Ｍ A data processing program for performing a predetermined operation on input data with a width of ×N bits (where N is a positive integer and M is a non-negative integer), and for executing time-division processing corresponding to a plurality of consecutive M values, Perform the minimum precision multiplication as described above. If the value of M is not zero, a shift operation is performed on the result of the minimum precision multiplication operation. If the value of M is not 0, perform the sign operation in the predetermined operation on the input data. The calculated code is held until a reset signal is received, which is issued each time the processing of all combinations of calculation targets in the predetermined calculation on the input data is completed, and the held code is reflected in the output of the shift process according to the value of M. The output of the shift process, with the sign reflected, is accumulated and added together. The cumulative sum calculation result obtained during the predetermined calculation process is stored. A data processing program that causes a computer to perform a task.

8. The minimum precision of a predetermined operation is N bits, and two 2 Ｍ A data processing method that performs the predetermined operation on input data with a width of ×N bits (where N is a positive integer and M is a non-negative integer), and performs time-division processing corresponding to a plurality of consecutive M values, Perform the minimum precision multiplication as described above. If the value of M is not zero, a shift operation is performed on the result of the minimum precision multiplication operation. If the value of M is not 0, perform the sign operation in the predetermined operation on the input data. The calculated code is held until a reset signal is received, which is issued each time the processing of all combinations of calculation targets in the predetermined calculation on the input data is completed, and the held code is reflected in the output of the shift process according to the value of M. The output of the shift process, with the sign reflected, is accumulated and added together. The cumulative sum calculation result obtained during the predetermined calculation process is stored. A data processing method in which a computer performs the processing.

Images