Calculation device

Abstract

To provide a technique capable of suppressing power consumption while suppressing an increase in a time required for computing in the case that the power consumption or the like increases.SOLUTION: Arithmetic units 100, 100B, 100C, 100D include: an arithmetic part 10 for executing a neural network model to perform arithmetic processing; an acquisition part 20 for acquiring any one or more of power consumption of the arithmetic units, a temperature of the arithmetic units and an operation rate of the arithmetic part as a determination value; and switching parts 30, 30B capable of switching a weight of the neural network model used in the arithmetic processing by a high bit weight and a low bit weight being the number of bits lower than the high bit weight. The switching parts switch the weight from the high bit weight to the low bit weight in the case that a determination value is equal to or more than a predetermined first threshold.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a computing device. [Background technology]

[0002] It is known that the power consumption of a computing device that performs arithmetic processing to execute a neural network model increases when a computation load is imposed. Dynamic Voltage and Frequency Scaling (DVFS) control is known, which suppresses power consumption by controlling voltage and clock frequency. Patent Document 1 also discloses that the power consumption when executing a neural network model that uses a low-bit weight parameter (1 bit) for computation is lower than the power consumption when executing a neural network model that uses a high-bit weight parameter (16 to 32 bits) represented by floating point. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-103441 [Patent Document 2] Japanese Patent Application Publication No. 2020-74099 [Patent Document 3] Japanese Patent Publication No. 2022-507704 Summary of the Invention [Problem to be solved by the invention]

[0004] When DVFS control is used to reduce power consumption, the time required for neural network model calculations can increase significantly. Although neural network models using low bit-weight parameters can reduce power consumption while suppressing increases in calculation time, there is a problem in that the structure of the neural network model determines the hardware configuration, making it difficult to easily switch between neural network models. [Means for solving the problem]

[0005] According to one embodiment of the present disclosure, there is provided a computing device (100, 100B, 100C, 100D), which includes a computing unit (10) that executes a neural network model to perform a computation process, an acquisition unit (20) that acquires one or more of the power consumption of the computing unit, the temperature of the computing unit, and the availability rate of the computing unit as a determination value, and a switching unit (30, 30B) that can switch the weight of the neural network model used in the computation process between a high bit weight and a low bit weight having a bit number lower than the high bit weight. a storage unit (40) for storing the high bit weights and the low bit weights in advance; The switching unit switches the weight from the high bit weight to the low bit weight when the determination value is equal to or greater than a predetermined first threshold value. According to another aspect of the present disclosure, there is provided a computing device. The computing device includes: a computing unit that executes a neural network model to perform computational processing; an acquisition unit that acquires, as a determination value, one or more of the power consumption of the computing unit, the temperature of the computing unit, and the availability rate of the computing unit; a switching unit that can switch the weight of the neural network model used in the computational processing between a high bit weight and a low bit weight having a bit number lower than the high bit weight; and a storage unit that stores the high bit weight. When the determination value is equal to or greater than a predetermined first threshold, the switching unit switches the weight from the high bit weight to the low bit weight and quantizes the high bit weight to switch it to the low bit weight. According to another aspect of the present disclosure, there is provided a computing device. The computing device includes: a computing unit that executes a neural network model to perform a computation; an acquisition unit that acquires, as a determination value, one or more of the power consumption of the computing unit, the temperature of the computing unit, and the availability rate of the computing unit; and a switching unit that can switch the weight of the neural network model used in the computation between a high bit weight and a low bit weight having a bit number lower than the high bit weight. The computing unit performs, in parallel, a first computation process that executes a first neural network model with the high bit weight and a second computation process that executes a second neural network model with the low bit weight. When the determination value is equal to or greater than a predetermined first threshold, the switching unit switches the weight from the high bit weight to the low bit weight and switches the weight of the first neural network model from the high bit weight to the low bit weight. According to another aspect of the present disclosure, there is provided a computing device. The computing device includes: a computing unit that executes a neural network model to perform computational processing; an acquisition unit that acquires, as a determination value, one or more of the power consumption of the computing unit, the temperature of the computing unit, and an availability rate of the computing unit; and a switching unit that can switch a weight of the neural network model used in the computational processing between a high bit weight and a low bit weight having a bit number lower than the high bit weight. The switching unit switches the weight from the high bit weight to the low bit weight when the determination value is equal to or greater than a predetermined first threshold, and switches the weight from the low bit weight to the high bit weight when the determination value is equal to or less than a predetermined third threshold that is lower than the first threshold.

[0006] According to this type of computing device, the switching unit can switch the weight of the neural network model between high bit weight and low bit weight depending on power consumption, etc. Therefore, computation can normally be performed with a desired accuracy, and when power consumption, etc. increases, power consumption can be reduced while suppressing an increase in the time required for computation. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 2 is an explanatory diagram showing an outline of the configuration of a computing device. [Figure 2]FIG. 10 is an explanatory diagram of a calculation process using a neural network model. [Figure 3] 10 is a flowchart showing an example of a calculation process. [Figure 4] FIG. 10 is an explanatory diagram showing an outline of the configuration of a calculation device in a second embodiment. [Figure 5] 10 is a flowchart showing an example of a calculation process in the second embodiment. [Figure 6] FIG. 10 is an explanatory diagram showing an outline of the configuration of a calculation device in a third embodiment. [Figure 7] FIG. 10 is an explanatory diagram showing an outline of the configuration of a calculation device in a fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] A. First embodiment: As shown in FIG. 1, the arithmetic device 100 includes a calculation unit 10, an acquisition unit 20, a switching unit 30, and a storage unit 40. The arithmetic device 100 is a device that executes a neural network model to perform arithmetic processing. The arithmetic device 100 includes a microcomputer configured with a central processing unit (CPU), RAM, and ROM, and the microcomputer executes a pre-installed program to realize the functions of each of these units. In this embodiment, the arithmetic device 100 is an SoC (System On a Chip). The arithmetic device 100 is mounted on a vehicle, for example, and performs arithmetic processing for image analysis. The arithmetic device 100 may also be implemented on a cloud.

[0009] The calculation unit 10 executes a neural network model to perform calculations. As shown in FIG. 2, the neural network includes an input layer L1, multiple intermediate layers L2, and an output layer L3. The number of intermediate layers L2 can be determined arbitrarily. The input layer L1 is a layer to which information is input. The intermediate layer L2 is a layer that calculates features based on information transmitted from the input layer L1. The output layer L3 is a layer that outputs results based on information transmitted from the intermediate layer L2. Each layer includes multiple nodes. Edges connecting nodes in adjacent layers are weighted. The calculation unit 10 performs calculations including a product-sum operation, for example, by multiplying node N1 by a weight W11, node N2 by a weight W12, and node N3 by a weight W13, and adding these values ​​together to find the value of node N4.

[0010] The storage unit 40 (see FIG. 1) stores weights by which values ​​input to nodes in a neural network model are multiplied. In this embodiment, the storage unit 40 stores a predetermined high bit weight Wh and a predetermined low bit weight Wl, which has a number of bits lower than the number of bits of the high bit weight Wh. In this embodiment, the storage unit 40 stores optimized 32-bit floating-point data as the high bit weight Wh, and stores 4-bit integer data as the low bit weight Wl. Furthermore, the high bit weight Wh and the low bit weight Wl are each optimized.

[0011] The acquisition unit 20 acquires a determination value related to the power consumption of the arithmetic device 100. In this embodiment, the acquisition unit 20 acquires the temperature value of the arithmetic device 100 from the sensor 200 as the determination value. The acquisition unit 20 can estimate that the higher the temperature of the arithmetic device 100, the higher the power consumption of the arithmetic device 100.

[0012] The switching unit 30 switches the weight of the neural network model used in the calculation process. The switching unit 30 can switch the weight of the neural network model between a high-bit weight Wh and a low-bit weight Wl.

[0013] 3 is a process in which the switching unit 30 switches the weights of the neural network model. This process is repeatedly executed when the calculation unit 10 executes calculation processing. In step S100, the acquisition unit 20 acquires a judgment value.

[0014] In step S110, the switching unit 30 determines whether the determination value acquired in step S100 is equal to or greater than a first threshold. If the determination value is equal to or greater than the first threshold, the switching unit 30 proceeds to step S120 and switches the weight from the high bit weight Wh to the low bit weight Wl. On the other hand, if the determination value is less than the first threshold, the switching unit 30 ends the calculation process without switching the weight.

[0015] According to the arithmetic device 100 of the present embodiment described above, the switching unit 30 can switch the weight of the neural network model between a high bit weight Wh and a low bit weight Wl in accordance with power consumption, etc. Therefore, calculations can usually be performed with a desired accuracy, and when power consumption, etc. increases, power consumption can be reduced while suppressing an increase in the time required for calculation.

[0016] Furthermore, since the switching unit 30 switches to an already optimized low bit weight Wl, the accuracy of calculations using the low bit weight Wl can be improved compared to when switching to a non-optimized low bit weight Wl.

[0017] B. Second embodiment: The arithmetic device 100B of the second embodiment shown in Fig. 4 is different from the first embodiment in that the storage unit 40 stores only the high bit weight Wh, but the other configurations are the same. The switching process in the second embodiment shown in Fig. 5 is different from the first embodiment in that the switching unit 30B performs post-quantization on the high bit weight Wh to switch it to a low bit weight in step S125, but the processes of steps S100 and S110 are the same.

[0018] In step S125, the switching unit 30B performs post-quantization on the high bit weight Wh to switch it to a low bit weight. More specifically, the switching unit 30B reduces the number of bits of the mantissa part represented by 23 bits of the 32-bit high bit weight Wh to 7 bits, and switches it to a low bit weight of 16-bit floating-point data.

[0019] According to the arithmetic device 100 of the second embodiment described above, the switching unit 30B performs post-quantization on the high bit weight Wh to switch it to a low bit weight, so that the number of bits of the low bit weight can be determined arbitrarily.

[0020] C. Third embodiment: 6 differs from the first embodiment in that it includes a control unit 50. The third embodiment differs from the first embodiment in that the control unit 50 controls the voltage and clock frequency of the arithmetic device 100.

[0021] When the determination value rises to or exceeds a predetermined second threshold, the control unit 50 controls the voltage and clock frequency of the arithmetic device 100 to be higher than when the determination value is less than the second threshold. The second threshold is a value smaller than the first threshold. That is, the control unit 50 executes DVFS control.

[0022] According to the calculation device 100C of the third embodiment described above, when the judgment value rises above the second threshold, the control unit 50 controls the voltage and clock frequency of the calculation device 100 to be higher than when the judgment value is below the second threshold, thereby reducing power consumption.

[0023] D. Fourth embodiment: 7 differs from the first embodiment in that it includes an estimation unit 60. The fourth embodiment differs from the first embodiment in that the switching unit 30 switches the weight from a high bit weight Wh to a low bit weight Wl even when the amount of calculation estimated by the estimation unit 60 is equal to or greater than a predetermined threshold.

[0024] The estimation unit 60 estimates the amount of calculation required for the requested calculation process.

[0025] In this embodiment, when the amount of calculation is equal to or greater than a predetermined threshold, the switching unit 30 switches the weight of the neural network model from a high-bit weight Wh to a low-bit weight Wl. Note that when the judgment value is equal to or greater than a first threshold and the weight of the neural network model has already been switched from a high-bit weight Wh to a low-bit weight Wl, the switching unit 30 may switch to a weight with an even lower number of bits.

[0026] According to the arithmetic device 100D of the fourth embodiment described above, it is possible to suppress power consumption when performing arithmetic processing that requires a large amount of calculation.

[0027] E. Fifth embodiment: The fifth embodiment differs from the first embodiment in that the calculation unit 10 executes a plurality of neural network models to perform calculation processing in parallel. The configuration of the calculation device 100 of the fifth embodiment is the same as the configuration of the calculation device 100 of the second embodiment, so a description of the configuration of the calculation device 100 will be omitted. The fifth embodiment also has substantially the same effects as the second embodiment.

[0028] The calculation unit 10 performs a first calculation process in parallel, which executes a first neural network model with a high bit weight Wh, and a second calculation process in parallel, which executes a second neural network model with a low bit weight Wl. That is, the calculation unit 10 has a redundant configuration. The calculation device 100 normally uses the calculation result obtained by the calculation unit 10 executing the first neural network model as the calculation result of the calculation unit 10.

[0029] In this embodiment, when the determination value is equal to or greater than the first threshold value, the switching unit 30 switches the weight of the first neural network model, which is the main model, from the high-bit weight Wh to the low-bit weight Wl in step S120 (see FIG. 3).

[0030] F. Other Embodiments: (F1) In the above-described embodiment, the acquisition unit 20 acquires the temperature of the arithmetic device 100 as the judgment value. However, the acquisition unit 20 may acquire one or more of the power consumption of the arithmetic device 100, the temperature of the arithmetic device 100, and the availability of the arithmetic device 10 as the judgment value. The availability of the arithmetic device 10 is, for example, the memory usage rate due to multiply-accumulate (MAC) operations of the arithmetic device 10. When multiple types of values ​​are acquired as the judgment value, a first threshold is set for each type of judgment value.

[0031] (F2) In the above-described embodiment, the switching unit 30 may switch the weight from the low bit weight Wl to the high bit weight Wh when the judgment value falls below the first threshold. Alternatively, the switching unit 30 may switch the weight from the low bit weight Wl to the high bit weight Wh when the judgment value is below a predetermined third threshold that is lower than the first threshold. According to this embodiment, when the judgment value rises or falls near the first threshold, the switching unit 30 can avoid switching the weight more frequently than when the switching unit 30 switches the weight from the low bit weight Wl to the high bit weight Wh when the judgment value falls below the first threshold.

[0032] (F3) In the above-described embodiment, the storage unit 40 is provided in the arithmetic device 100. However, the present invention is not limited to this, and the storage unit 40 may be an external storage area.

[0033] (F4) In the second embodiment described above, the switching unit 30B may determine the number of low-bit-weight bits according to the judgment value. More specifically, the switching unit 30B determines the number of low-bit-weight bits using a table or function prepared in advance. For example, the switching unit 30B reduces the number of low-bit-weight bits as the judgment value increases.

[0034] (F5) In the second embodiment described above, the switching unit 30B reduces the number of bits in the mantissa part of the 32-bit high-bit weight Wh and switches it to a 16-bit low-bit weight. This is not limiting, and the switching unit 30B may perform post-quantization using, for example, a histogram of the high-bit weight Wh. In this case, the storage unit 40 stores the histogram of the high-bit weight Wh. More specifically, the switching unit 30B performs post-quantization using the width of the histogram interval calculated by the following equation (1). Note that the time required for post-quantization by the switching unit 30B is approximately several milliseconds, which does not significantly affect the time required for the calculation processing by the calculation unit 10.

[0035] S=A / 2 (n-1) …(1) Here, S is the width of the histogram interval, A is the difference between the minimum and maximum values ​​of the histogram, and n is the number of low-order bits. n can be set arbitrarily.

[0036] (F6) In the third embodiment described above, the estimation unit 60 controls the voltage and clock frequency of the arithmetic device 100. However, the present invention is not limited to this, and the estimation unit 60 may control at least one of the voltage and clock frequency of the arithmetic device 100.

[0037] (F7) In the fourth embodiment described above, the switching unit 30 may switch the weight of the neural network model from the high bit weight Wh to the low bit weight Wl when the amount of calculation is equal to or greater than a threshold and the control unit 50 cannot execute DVFS control. In this case, the calculation device 100D includes the control unit 50, and the control unit 50 executes DVFS control when the amount of calculation is equal to or greater than a threshold.

[0038] The present disclosure is not limited to the above-described embodiments and can be realized in various configurations without departing from the spirit thereof. For example, the technical features in the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention section can be appropriately replaced or combined to solve the above-described problems or achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0039] 10... Calculation unit, 20... Acquisition unit, 30, 30B... Switching unit, 40... Storage unit, 50... Control unit, 60... Estimation unit, 100, 100B, 100C, 100D... Calculation device, 200... Sensor

Claims

1. A computing device (100, 100B, 100C, 100D), A calculation unit (10) that executes a neural network model and performs calculation processing; an acquisition unit (20) that acquires one or more of the power consumption of the arithmetic unit, the temperature of the arithmetic unit, and the operating rate of the arithmetic unit as a judgment value; a switching unit (30, 30B) capable of switching the weight of the neural network model used in the arithmetic processing between a high bit weight and a low bit weight having a bit number lower than the high bit weight; a storage unit (40) for storing the high bit weight and the low bit weight in advance, The switching unit switches the weight from the high bit weight to the low bit weight when the determination value is equal to or greater than a predetermined first threshold.

2. A computing device, a calculation unit that executes a neural network model and performs calculations; an acquisition unit that acquires, as a determination value, one or more of the power consumption of the arithmetic unit, the temperature of the arithmetic unit, and the availability rate of the arithmetic unit; a switching unit that can switch the weight of the neural network model used in the calculation process between a high bit weight and a low bit weight having a bit number lower than the high bit weight; a storage unit that stores the high bit weight; The switching unit is When the determination value is equal to or greater than a predetermined first threshold, the weight is switched from the high bit weight to the low bit weight; A computing device that performs quantization on the high bit weights and switches to the low bit weights.

3. 3. The computing device according to claim 2, The switching unit determines the number of low-bit-weight bits in accordance with the determination value.

4. The arithmetic device according to claim 1 or 2, further comprising: A computing device comprising a control unit (50) that controls at least one of the voltage and clock frequency of the computing device to be higher when the judgment value rises to or above a second threshold value that is smaller than the first threshold value than when the value is less than the second threshold value.

5. 2. The computing device according to claim 1, further comprising: An estimation unit (60) for estimating the amount of calculation in the calculation processing, The switching unit switches the weight from the high bit weight to the low bit weight when the amount of calculation is equal to or greater than a predetermined threshold.

6. A computing device, a calculation unit that executes a neural network model and performs calculations; an acquisition unit that acquires, as a determination value, one or more of the power consumption of the arithmetic unit, the temperature of the arithmetic unit, and the availability rate of the arithmetic unit; a switching unit that can switch the weight of the neural network model used in the calculation process between a high bit weight and a low bit weight having a number of bits lower than the high bit weight, the calculation unit performs a first calculation process for executing the first neural network model with a high bit weight and a second calculation process for executing the second neural network model with a low bit weight in parallel; The switching unit is When the determination value is equal to or greater than a predetermined first threshold, the weight is switched from the high bit weight to the low bit weight; A computing device that switches the weights of the first neural network model from the high-bit weights to the low-bit weights.

7. A computing device, a calculation unit that executes a neural network model and performs calculations; an acquisition unit that acquires, as a determination value, one or more of the power consumption of the arithmetic unit, the temperature of the arithmetic unit, and the availability rate of the arithmetic unit; a switching unit that can switch the weight of the neural network model used in the calculation process between a high bit weight and a low bit weight having a number of bits lower than the high bit weight, The switching unit is When the determination value is equal to or greater than a predetermined first threshold, the weight is switched from the high bit weight to the low bit weight; When the determination value is equal to or less than a predetermined third threshold value that is lower than the first threshold value, the weight is switched from the low bit weight to the high bit weight.

Images