Data processing device

Abstract

Provided is a data processing device 100A comprising a multiplication unit 102 that multiplies an input unit by a multiplier, an addition unit 104 that adds together the output from the multiplication unit 102 and a polynomial coefficient and outputs the resultant sum, a retention unit 106 that retains the output from the addition unit 104, and a selection unit 101 that selects and outputs a multiplier for use by the multiplication unit 102 from among data retained by the retention unit 106 and polynomial coefficients outputted to the multiplication unit 102.

Description

[Technical Field]

[0001] The disclosed technology relates to data processing equipment. [Background technology]

[0002] In artificial intelligence (AI) neural networks, the activation function is a function that converts any input value into a different numerical value when outputting from one neuron to the next. There are several types of activation functions, such as the sigmoid function, tanh function, and ReLU, and the function used varies depending on the AI ​​model being handled. In recent years, AI-based object detection models such as YOLO (You Only Look Once) (Non-Patent Literature 1) and pose estimation models such as OpenPose (Non-Patent Literature 2) have been disclosed. Attention is now being drawn to edge AI, which involves equipping these models with small devices such as drones and surveillance cameras.

[0003] When implementing inference processing for multiple AI models on resource-limited devices such as those used in edge AI, it is necessary to prepare a separate circuit for each type of activation function corresponding to each model, which increases hardware resources. Furthermore, since only circuits for functions determined at the time of design can be prepared, there is a lack of scalability for the future.

[0004] To address such challenges, one method for implementing multiple types of activation function processing with low resources is to represent the activation function using a piecewise polynomial approximation formula. Piecewise polynomial approximation is a method that divides the domain of the input into equal intervals and approximates the output y of each interval with an n-th degree polynomial. For example, Non-Patent Document 3 discloses a configuration for realizing piecewise polynomials.

[0005] Figure 7 shows the configuration for realizing a piecewise polynomial, as disclosed in Non-Patent Document 3. A coefficient is selected by selector 1001A, and the input x and the coefficient selected by selector 1001A are multiplied by multiplication unit 1002A. Another coefficient is selected by selector 1003A, and the output of multiplication unit 1002A and the coefficient selected by selector 1003A are added by addition unit 1004A.

[0006] Furthermore, input x is held by the holding unit 1005A, and input x and the output of the adding unit 1004A are multiplied by the multiplier unit 1002B. Also, a coefficient is selected by the selector 1003B, and the output of the multiplier unit 1002B and the coefficient selected by the selector 1003B are added by the adder unit 1004B.

[0007] With this configuration, the output y for an input x can be expressed by the polynomial (1) below, allowing for implementation with a simple multiplier and adder configuration, thus reducing hardware resources. Furthermore, the coefficients are stored in memory, and by rewriting the memory, it is possible to support multiple types of activation functions. y=C n x n +C n-1 x n-1 +···+C1x+C0(1) [Prior art documents] [Non-patent literature]

[0008] [Non-Patent Document 1] Joseph Redmon et al., "YOLOv3: An Incremental Improvement", Internet<URL:https: / / arxiv.org / abs / 1804.02767> [Non-Patent Document 2] Zhe Cao et.al, "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields", Internet<URL:https: / / arxiv.org / pdf / 1611.08050.pdf> [Non-Patent Document 3] Shinobu Nagayama, et al., "Numerical Calculation Circuit Based on Polynomial Approximation Suitable for FPGA Implementation," IEICE Technical Report [Overview of the Initiative] [Problems that the invention aims to solve]

[0009] However, while conventional configurations eliminate the need to prepare separate circuits for each type of activation function, they have the drawback of increasing circuit size as the number of sets of holders, multipliers, adders, and selectors increases proportionally to the degree n of the polynomial that can be handled. Furthermore, as n increases, the number of bits in the multiplication result also increases, thus increasing the bit width of the subsequent adder. In other words, the circuit size of a configuration that corresponds to an n-th degree polynomial approximation increases due to both the increase in the number of sets of holders, multipliers, adders, and selectors and the increase in the bit width of each arithmetic unit. This increase in circuit size is a critical problem for devices intended for edge AI.

[0010] The disclosed technology was developed in view of the above points, and aims to provide a data processing device that reduces the circuit size of the circuit that performs numerical calculations based on polynomial approximation compared to conventional configurations. [Means for solving the problem]

[0011] A first aspect of this disclosure is a data processing device comprising: a multiplication unit that multiplies an input value by a multiplier; an addition unit that adds the output from the multiplication unit and a polynomial coefficient and outputs the result; a holding unit that holds the output from the addition unit; and a selection unit that selects the multiplier from the data held in the holding unit and the polynomial coefficient output to the multiplication unit and outputs the result. A second bit reduction unit reduces the data length of the data output by the addition unit and outputs it, and an input conversion unit converts the input value according to a predetermined compression method and outputs it to the multiplication unit. Includes The input conversion unit converts the input value into a value determined by the data length of the input value and the number of divisions in the piecewise polynomial approximation. . [Effects of the Invention]

[0012] According to the disclosed technology, increasing the types of activation functions that can be processed or the order of the polynomial approximation does not increase the circuit size. Therefore, it is possible to provide a data processing device that reduces the circuit size of the circuit that performs numerical calculations based on polynomial approximation compared to conventional configurations.

Brief Description of the Drawings

[0013] [Figure 1] It is a diagram showing the configuration of a data processing apparatus according to the first embodiment. [Figure 2] It is a flowchart showing the flow of polynomial approximation processing by the data processing apparatus. [Figure 3] It is a diagram showing the configuration of a data processing apparatus according to the second embodiment. [Figure 4] It is a flowchart showing the flow of polynomial approximation processing by the data processing apparatus. [Figure 5] It is a diagram showing the configuration of a data processing apparatus according to the third embodiment. [Figure 6] It is a flowchart showing the flow of polynomial approximation processing by the data processing apparatus. [Figure 7] It is a diagram showing the configuration of a conventional data processing apparatus.

Mode for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of the disclosed technology will be described while referring to the drawings. In each drawing, the same or equivalent components and parts are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for convenience of explanation and may be different from the actual ratios.

[0015] (First Embodiment) FIG. 1 is a diagram showing the configuration of a data processing apparatus according to the first embodiment. The data processing apparatus 100A shown in FIG. 1 includes a first selector 101, a multiplication unit 102, a second selector 103, an addition unit 104, a switch 105, and a holding unit 106.

[0016] The first selector 101 is an example of a selection unit in this disclosure, and selects either a coefficient to be multiplied by the input x in the multiplication unit 102 or a value held in the holding unit 106, and outputs it to the multiplication unit 102. When the data processing device 100A performs an operation on a linear polynomial, the first selector 101 selects a coefficient to be multiplied by the input x in the multiplication unit 102. When performing an operation on a polynomial of degree n or higher, if the operation on the nth degree has not been completed, the first selector 101 selects a value held in the holding unit 106.

[0017] The multiplication unit 102 consists of a multiplier capable of processing a predetermined number of bits, and multiplies the input x input to the data processing device 100A as the multiplier and the output from the first selector 101 as the multiplicand, and outputs the result.

[0018] The second selector 103 selects the polynomial coefficients to be added to the output from the multiplication unit 102 in the addition unit 104 and outputs them to the addition unit 104.

[0019] The adder 104 consists of an adder capable of processing a predetermined number of bits, and adds the output of the multiplication process from the multiplication 102 and the coefficient output from the second selector 103, and outputs the result.

[0020] Switch 105 switches whether the output of the addition process in the adder 104 is output as output y or to the holding unit 106. When the data processing device 100A performs an operation on a linear polynomial, switch 105 switches to outputting the output of the addition process in the adder 104 as output y. When the data processing device 100A performs an operation on a polynomial of degree n or higher, switch 105 switches to outputting the output of the addition process in the adder 104 as output y if the nth-degree operation is completed, and switches to outputting the output of the addition process in the adder 104 to the holding unit 106 if the nth-degree operation is not completed. In this embodiment, a switch is used to switch whether the output of the addition process in the adder 104 is output as output y or to the holding unit 106, but this disclosure is not limited to this example. A demultiplexer may be used to switch whether the output of the addition process in the adder 104 is output as output y or to the holding unit 106. When a demultiplexer is used, the destination of the demultiplexer's output controls whether or not to accept that output value as an output.

[0021] The holding unit 106 is a buffer for matching the timing of the input and holds the output of the addition process in the addition unit 104. The value held in the holding unit 106 is output to the multiplication unit 102 when performing an operation on a polynomial of degree n or higher, and the nth-order operation has not yet been completed. In other words, when the data processing device 100A performs a second-order polynomial operation, and only the first-order operation has been completed, the contents held in the holding unit 106 are output to the multiplication unit 102 by the first selector 101.

[0022] By providing the holding unit 106, the data processing device 100A can perform an n-th degree polynomial approximation calculation by performing operations using a set of multiplication units 102, addition units 104, first selector 101, and second selector 103 multiple times. The coefficients sent to the multiplication unit 102 and addition unit 104 are stored, for example, in memory or registers, and the storage location is not defined in this embodiment. The values ​​of these coefficients differ depending on the type of activation function and the domain of the input. By rewriting these values, the data processing device 100A can realize polynomial approximation processing of multiple types of activation functions.

[0023] Next, the operation of the data processing device 100A will be explained.

[0024] Figure 2 is a flowchart showing the flow of polynomial approximation processing by the data processing device 100A. In Figure 2, n=2 is used to calculate a quadratic polynomial (output y=C2x 2 An example of polynomial approximation of (+C1x+C0) is given.

[0025] The data processing device 100A first selects the coefficient C2 with the first selector 101 and then calculates C2 × x with the multiplication unit 102 (step S101).

[0026] Following step S101, the data processing device 100A selects the coefficient C1 with the second selector 103 and calculates C2x + C1 with the adder 104 (step S102).

[0027] Following step S102, the data processing device 100A determines whether the calculation is complete using the linear approximation formula (step S103).

[0028] If the result of the judgment in step S103 is that the calculation is not completed with a linear approximation formula (step S103; No), the data processing device 100A uses the first selector 101 to select C2x+C1, which is the addition result of the adder 104 in step S102, and uses the multiplication unit 102 to perform the calculation (C2x+C1)×x (step S104).

[0029] Following step S104, the data processing device 100A selects coefficient C0 with the second selector 103 and adds C2x with the adder 104. 2 Calculate +C1x+C0 (step S105).

[0030] Following step S105, the data processing device 100A outputs C2x as output y. 2 Output +C1x+C0 (step S106).

[0031] On the other hand, if the calculation is completed using a first-order approximation formula as a result of the judgment in step S103 (step S103; Yes), the data processing device 100A outputs C2x + C1 as output y (step S107).

[0032] Although the flowchart in Figure 2 shows an example of a quadratic approximation polynomial, the data processing device 100A can perform calculations on quadratic or higher approximation polynomials by repeating the process in steps S103 to S106 multiple times. In this case, the data processing device 100A selects each time, by switching the switch 105, whether to treat the output from the adder 104 as output y or to output it to the holding unit 106.

[0033] As explained above, the data processing device 100A can perform an n-th degree polynomial approximation calculation by performing operations using a set of multiplication units 102, addition units 104, first selector 101, and second selector 103 multiple times.

[0034] (Second Embodiment) In the first embodiment, an example was shown in which the circuit size is reduced even in the case of an n-dimensional approximate polynomial by sharing the multiplication unit, addition unit, and selector for processing. In the second embodiment, a configuration and processing method are shown in which the circuit size of the multiplication unit and addition unit itself is reduced by reducing the number of bits handled by the multiplication unit and addition unit.

[0035] Figure 3 shows the configuration of a data processing device according to the second embodiment. The data processing device 100B shown in Figure 3 includes a first selector 101, a multiplication unit 102, a second selector 103, an addition unit 104, a switch 105, a holding unit 106, a first bit reduction unit 107, and a second bit reduction unit 108. In the following description, the first bit reduction unit 107 and the second bit reduction unit 108, which are components added from the first embodiment, will be described in detail.

[0036] The first bit reduction unit 107 is located after the multiplication unit 102 and reduces the number of bits in the output data from the multiplication unit 102 to a number of bits that can be processed by the adder unit 104. For example, if the multiplication unit 102 is composed of a k-bit multiplier and the adder unit 104 is composed of an l-bit adder, the first bit reduction unit 107 reduces the bit width of the output data from the multiplication unit 102 to l bits.

[0037] The second bit reduction unit 108 is located downstream of the switch 105 and reduces the number of bits in the output data from the adder 104 to a number of bits that can be processed by the multiplier 102. For example, if the multiplier 102 is composed of a k-bit multiplier and the adder 104 is composed of an l-bit adder, the second bit reduction unit 108 reduces the bit width of the output data from the adder 104 to k bits.

[0038] The first bit reduction unit 107 and the second bit reduction unit 108 remove bits from the least significant bit of the output data to match the bit width of the subsequent multiplication unit 102 or addition unit 104, and then perform rounding or truncation.

[0039] The data processing device 100B, by including a first bit reduction unit 107 and a second bit reduction unit 108, can reduce the circuit size of the multiplier inside the multiplication unit 102 and the adder inside the addition unit 104. By including the first bit reduction unit 107 and the second bit reduction unit 108, the data processing device 100B can also reduce the overall size of the device. Although the circuit size of the newly added first bit reduction unit 107 and second bit reduction unit 108 increases the circuit size of that part, the circuit size of the first bit reduction unit 107 and the second bit reduction unit 108 is small compared to the multiplier and adder corresponding to an n-th degree approximation polynomial. In particular, as the corresponding n becomes larger, the size of the multiplier and adder increases, so the effect of this embodiment becomes greater.

[0040] In this embodiment, the output y is shown as being output without any bit reduction in the data length, but this disclosure is not limited to this example. For example, a second bit reduction unit 108 may be provided before the switch 105 to output data with a shortened data length as output y.

[0041] In AI inference models, the output value after activation function processing becomes the input value for the next layer; therefore, the increase in bit width due to n-th degree polynomial operations is always reduced before input to the next layer. In this embodiment, considering the processing characteristics of the AI ​​inference model, bits are reduced during the activation function processing, making it possible to suppress accuracy degradation due to bit reduction.

[0042] Next, the operation of the data processing device 100B will be explained.

[0043] Figure 4 is a flowchart showing the flow of polynomial approximation processing by the data processing device 100B. In Figure 4, n=2 is used to calculate a quadratic polynomial (output y=C2x 2 An example of polynomial approximation of (+C1x+C0) is given.

[0044] The data processing device 100B first selects the coefficient C2 with the first selector 101 and then calculates C2 × x with the multiplication unit 102 (step S111).

[0045] Following step S111, the data processing device 100B reduces the data length of C2×x in the first bit reduction unit 107 (step S112).

[0046] Following step S112, the data processing device 100B selects the coefficient C1 by the second selector 103 and calculates C2x + C1 in the adder 104 (step S113).

[0047] Following step S113, the data processing device 100B determines whether the operation has ended with the first-order approximation formula (step S114).

[0048] As a result of the determination in step S114, if the operation has not ended with the first-order approximation formula (step S114; No), the data processing device 100B reduces the data length of C2x + C1 in the second bit reduction unit 108 (step S115).

[0049] Following step S115, the data processing device 100B selects C2x + C1, which is the addition result of the adder 104 in step S113, by the first selector 101 and calculates (C2x + C1)×x in the multiplier 102 (step S116).

[0050] Following step S116, the data processing device 100B reduces the data length of (C2x + C1)×x = C2x 2 + C1x in the first bit reduction unit 107 (step S117).

[0051] Following step S117, the data processing device 100B selects the coefficient C0 by the second selector 103 and calculates C2x 2 + C1x + C0 in the adder 104 (step S118).

[0052] Following step S118, the data processing device 100B outputs C2x 2 + C1x + C0 as the output y (step S119).

[0053] On the other hand, if the calculation is completed using a linear approximation formula as a result of the judgment in step S114 (step S114; Yes), the data processing device 100B outputs C2x + C1 as output y (step S120).

[0054] The processing flow in Figure 4 shows a flow in which the input x and coefficient C2 are multiplied in one step in the multiplication unit, but this disclosure is not limited to this example. For example, the data processing device 100B may perform the calculation of C2x in multiple steps. By performing the calculation of C2x in multiple steps, the data processing device 100B can further reduce the size of the multiplier in the multiplication unit 102, and thus further reduce the size of the entire device. Note that the execution of calculations in multiple steps may be applied to at least one or both of the multiplication unit 102 and the addition unit 104.

[0055] (Third embodiment) In the second embodiment, a configuration and processing method are shown in which the circuit size of the multiplication and addition units themselves is reduced by reducing the number of bits handled by the multiplication and addition units. In the third embodiment, in addition to the second embodiment, a configuration and processing method are shown in which the input data length to the multiplication unit 102 is shortened by applying a transformation to the input x to get Δx, thereby reducing the data bit width handled by the multiplication unit 102.

[0056] Specifically, it converts the input domain into a number of inputs with equal interval widths. For example, if the data width of the original input x is 8 bits and the number of divisions for the domain of input x is 64, then the number of inputs with equal interval widths is 2 8 / 64 = 4. In this case, we only need to represent the data for 4 inputs, so the transformed input data only needs to be 2 bits. Using these 2 bits, we represent the transformed Δx of input x as -2, -1, 0, and 1. Each coefficient in the polynomial is calculated in advance using Δx and saved so that it can be selected from a selector.

[0057] Figure 5 shows the configuration of the data processing device according to the third embodiment. The data processing device 100C shown in Figure 5 includes a first selector 101, a multiplication unit 102, a second selector 103, an addition unit 104, a switch 105, a holding unit 106, a first bit reduction unit 107, a second bit reduction unit 108, and an input conversion unit 109. In the following description, the input conversion unit 109, which is a configuration added from the second embodiment, will be described in detail.

[0058] The input conversion unit 109 performs a predetermined conversion process on the input x and outputs Δx. Specifically, the input conversion unit 109 performs a conversion process on the input x that compresses the number of bits using a predetermined compression method and outputs the result.

[0059] The conversion process in the input conversion unit 109 will be explained in general terms. If the data length of the input x is d bits and the number of divisions is N divisions, then in two's complement representation... Δx = -(2 d / N) / 2~+((2 d / N) / 2-1) It will be converted to this.

[0060] The bit width of Δx is log2(2 d The bit width that can be reduced is d-log2(2 d This is a value where the number of divisions N can be expressed as a power of 2 (N=2). m ) If so, the bit width that can be reduced is, d-log2(2 d / N)=d-log2(2 d / 2 m )=d-log22 (d-m) =d-(dm)=m In other words, by providing the input conversion unit 109, the number of bits in the input x is compressed, and the multiplication unit 102 can be reduced in width by m bits to achieve this.

[0061] By providing an input conversion unit 109 that converts input x to Δx, the data processing device 100C reduces the circuit size of the multiplier in the multiplication unit 102, and also reduces the circuit size of the adder in the subsequent addition unit 104. As a result, the overall size of the data processing device 100C can be further reduced compared to data processing devices 100A and 100B.

[0062] Next, the operation of the data processing device 100C will be explained.

[0063] Figure 6 is a flowchart showing the flow of polynomial approximation processing by the data processing device 100C. In Figure 6, n=2 is used to calculate a quadratic polynomial (output y=C2x 2 An example of polynomial approximation of (+C1x+C0) is given.

[0064] The data processing device 100C first converts the input x to Δx in the input conversion unit 109 and outputs it (step S121).

[0065] Following step S121, the data processing device 100C selects the coefficient C2 with the first selector 101 and calculates C2 × x with the multiplication unit 102 (step S122).

[0066] Following step S122, the data processing device 100C reduces the data length C2×x in the first bit reduction unit 107 (step S123).

[0067] Following step S123, the data processing device 100C selects the coefficient C1 with the second selector 103 and calculates C2x + C1 with the adder 104 (step S124).

[0068] Following step S124, the data processing device 100C determines whether the calculation is complete using the first-order approximation formula (step S125).

[0069] If, as a result of the determination in step S125, the calculation is not completed using a first-order approximation formula (step S125; No), the data processing device 100C reduces the data length of C2x + C1 using the second bit reduction unit 108 (step S126).

[0070] Following step S126, the data processing device 100C uses the first selector 101 to select C2x+C1, which is the addition result of the adder 104 in step S124, and the multiplication unit 102 performs the calculation (C2x+C1)×x (step S127).

[0071] Following step S127, the data processing device 100C performs (C2x+C1)×x=C2x in the first bit reduction unit 107. 2 Reduce the data length of +C1x (step S128).

[0072] Following step S128, the data processing device 100C selects coefficient C0 with the second selector 103 and adds C2x with the adder 104. 2 Calculate +C1x+C0 (step S129).

[0073] Following step S129, the data processing device 100C outputs C2x as output y. 2 Output +C1x+C0 (step S130).

[0074] On the other hand, if the calculation is completed using a linear approximation formula as a result of the judgment in step S125 (step S125; Yes), the data processing device 100C outputs C2x + C1 as output y (step S131).

[0075] The processing flow in Figure 6 shows a flow in which the input x and coefficient C2 are multiplied in one step in the multiplication unit, but this disclosure is not limited to this example. For example, the data processing device 100C may calculate C2x in multiple steps. By calculating C2x in multiple steps, the data processing device 100C can further reduce the size of the multiplier in the multiplication unit 102, and thus further reduce the size of the entire device. Note that calculations performed in multiple steps may also be applied to the addition unit 104.

[0076] In the third embodiment, the first bit reduction unit 107, the second bit reduction unit 108, and the input conversion unit 109 were all included, but this disclosure is not limited to this example. At least one of the first bit reduction unit 107, the second bit reduction unit 108, and the input conversion unit 109 may be provided. [Explanation of Symbols]

[0077] 100A, 100B, 100C Data Processing Units 101 First Selector 102 Multiplication part 103 Second Selector 104 Addition section 105 switches 106 Holding part 107 First bit reduction section 108 Second bit reduction section 109 Input Conversion Unit

Claims

1. A multiplication unit that multiplies the input value by the multiplier, An adder that adds the output from the multiplier and the polynomial coefficients and outputs the result, A holding unit that holds the output from the summing unit, A selection unit selects and outputs the multiplier from the data held in the holding unit and the polynomial coefficients output to the multiplication unit, A second bit reduction unit reduces the data length of the data output by the summing unit and outputs it, An input conversion unit that converts the input value according to a predetermined compression method and outputs it to the multiplication unit, Equipped with, The input conversion unit is a data processing device that converts the input value into a value determined by the data length of the input value and the number of divisions in the piecewise polynomial approximation.

2. The data processing device according to claim 1, wherein when performing a polynomial operation of a quadratic or higher degree, the selection unit selects the contents held in the holding unit as the multiplier.

3. The data processing apparatus according to claim 1, further comprising a first bit reduction unit that reduces the data length of the data output by the multiplication unit and outputs it to the addition unit.

4. The data processing apparatus according to claim 1, which repeats the operation of either or both of the multiplication unit or the addition unit multiple times.

Images