Range control device and method

Abstract

In the present disclosure, in an arithmetic circuit which extracts, with a range set using an effective bit width, and outputs a result of cumulative addition of product-sum operation results of two data items in a first bit width in which the decimal point position is set, a range control unit: counts each of the number of times the result of cumulative addition in the arithmetic circuit exceeds the upper limit of the range and the number of times the result of cumulative addition falls below the minimum step width for the lower limit of the range; and when one of the counted numbers exceeds a threshold value and the adjustment of the decimal point position cannot resolve this problem of the threshold value being exceeded, expands the effective bit width toward the upper limit side or lower limit side.

Description

Range control device and method

[0001] The disclosed technology relates to a range control device and a range control method.

[0002] Convolutional neural networks (CNNs) are often used in recent image recognition processing. CNNs are a type of neural network consisting of multiple layers, including convolutional layers, pooling layers, and fully connected layers. CNNs perform two-dimensional convolution processing on image data. In two-dimensional convolution processing, a three-dimensional feature map is used as input, and product-sum operations with two-dimensional kernels, which are weighting coefficients, are repeated to perform cumulative sums in the channel direction. A bias is then added to the cumulative sum, and activation function processing is performed to obtain an output feature map. This makes it possible to extract local features from the input feature map.

[0003] YOLO (You Only Look Once) (Non-Patent Document 1), which performs object detection, and DeepLab, which performs semantic segmentation, are algorithms based on CNN. As AI (Artificial Intelligence) technology becomes more widely used in society, it becomes necessary to process these algorithms in real time. On the other hand, edge devices such as drones and surveillance cameras have limited resources, such as power, so it is essential to implement these AI inference algorithms using hardware such as LSIs (Large Scale Integration) and FPGAs (Field Programmable Gate Arrays).

[0004] In dedicated hardware for inference processing using CNN, the calculation circuit dedicated to convolution calculations accounts for the majority of the circuit area and power consumption, so a lightweight method has been proposed to reduce the bit width used for calculations in order to keep the circuit size down.

[0005] Generally, parameter values ​​such as feature maps and kernels used in convolution processing are often handled as 32-bit floating-point numbers. However, since the circuit size required to perform 32-bit floating-point multiply-accumulate operations becomes very large, the circuit size and power consumption can be reduced by converting the values ​​to low-bit quantization and fixed-point numbers. However, when performing operations using low-bit quantization and fixed-point numbers, the data range and quantization step width are fixed. Therefore, the calculation accuracy is lower than when performing operations using 32-bit floating-point numbers, and the inference performance of the CNN deteriorates.

[0006] Therefore, a technique has been proposed in which the decimal point position in the fixed-point representation is controlled for each layer of the CNN to adjust the range of values ​​close to the representation of the parameter values ​​(Non-Patent Document 2).

[0007] Furthermore, a technique has been proposed in which, when sequentially processing each frame included in a video, the value range adjustment for each layer of each frame is dynamically determined from the result of the previous frame, based on the characteristic that the difference in pixel values ​​between the frame to be processed and the previous frame is small (Patent Document 1). The method described in Patent Document 1 provides an upper limit counter and a lower limit counter that count the number of data values ​​above the maximum value and below the minimum value of the quantization step width for the data values ​​of the previous frame. The method described in Patent Document 1 selects value ranges such that the values ​​of these counters do not exceed thresholds, and adjusts the decimal point position to an optimal position for each frame.

[0008] International Publication No. 2022 / 003855

[0009] Joseph Redmon et.al, "YOLOv3: An Incremental Improvement", arXiv:1804.02767v1 [cs.CV] 8 Apr 2018.Zhisheng Li et.al, "Laius: An 8-bit Fixed-point CNN Hardware Inference Engine", 2017 IEEE International Symposium on Parallel and Distributed Processing with Applications and 2017 IEEE International Conference on Ubiquitous Computing and Communications (ISPA / IUCC).

[0010] In conventional algorithms that dynamically control the position of the decimal point, the bit width remains fixed while only the position of the decimal point is changed, so there is a trade-off between the upper limit count and the lower limit count, which can result in a decrease in inference accuracy in some cases.

[0011] The disclosed technology has been developed in consideration of the above points, and aims to control the range of values ​​so as to suppress a decrease in inference accuracy caused by reducing the bit width for operations included in the inference process.

[0012] A first aspect of the present disclosure is a range control device including: a counter that counts the number of times that an accumulated addition result of a product-sum operation of two data in a first bit width in which a decimal point position is set, cuts out and outputs the result in a range set by a second bit width within the first bit width based on the position of the decimal point, and counts the number of times that the accumulated addition result exceeds an upper limit of the range and the number of times that it falls below a minimum step width of a lower limit of the range; a first determination unit that, when either of the numbers counted by the counter exceeds a predetermined threshold, determines whether an estimated value of the number of times counted by the counter will be less than the threshold if the position of the decimal point is shifted toward the upper limit or the lower limit; a second determination unit that, when the first determination unit determines that the estimated value of the number of times is not less than the threshold, determines whether the range should be extended toward the upper limit or the lower limit based on the number of times counted by the counter; and a setting unit that sets the position of the decimal point and the second bit width in the arithmetic circuit based on the determination results of the first determination unit and the second determination unit.

[0013] A second aspect of the present disclosure is a range control method executed by a range control device including a counter, a first determination unit, a second determination unit, and a setting unit, wherein the counter counts the number of times that the cumulative addition result of an arithmetic circuit that extracts and outputs an accumulated addition result of a product-sum operation of two pieces of data in a first bit width in which a decimal point position is set, by a range set in a second bit width within the first bit width based on the position of the decimal point, and the first determination unit determines whether the cumulative addition result exceeds an upper limit of the range and falls below a minimum step width of a lower limit of the range, and exceeds a predetermined threshold, the second determination unit determines whether the estimated value of the number of times counted by the counter will be less than the threshold when the position of the decimal point is shifted to the upper limit side or the lower limit side, and when the first determination unit determines that the estimated value of the number of times is not less than the threshold, the second determination unit determines whether the value range should be expanded to the upper limit side or the lower limit side based on the number of times counted by the counter, and the setting unit sets the position of the decimal point and the second bit width in the arithmetic circuit based on the determination results of the first determination unit and the second determination unit.

[0014] According to the disclosed technology, it is possible to control the range so as to suppress a decrease in inference accuracy caused by reducing the bit width for the calculations included in the inference process.

[0015] FIG. 1 is a diagram for explaining a comparison method. FIG. 2 is a diagram for explaining the comparison method. FIG. 3 is a block diagram showing a hardware configuration of a video inference processing device according to the first and second embodiments. FIG. 4 is a functional block diagram of the video inference processing device. FIG. 5 is a flowchart showing an example of range control processing. FIG. 6 is a flowchart showing an example of range control processing. FIG. 7 is a diagram for explaining expansion of an effective bit width. FIG. 8 is a diagram for explaining expansion of an effective bit width. FIG. 9 is a diagram for explaining a saturation count in the second embodiment.

[0016] An example of an embodiment of the disclosed technology will be described below with reference to the drawings. Note that the same reference numerals are used to designate identical or equivalent components and parts in each drawing. Also, the dimensional proportions in the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0017] <Comparative Method> Before describing the details of each embodiment, the details and problems of a method in which the bit width is fixed and the position of the decimal point is dynamically controlled (hereinafter referred to as the "comparative method") will be described.

[0018] For example, if each parameter of the CNN is an 8-bit fixed-point number including a sign, the comparison method performs a product-sum operation for convolution processing using a maximum of 15 bits of the 16 bits (8 bits x 8 bits), excluding one bit for the sign. The comparison method also accumulates the results of the product-sum operation across all channels and extracts 8 bits as the effective bit width from the 15-bit cumulative addition result to which a bias is added. The comparison method then performs activation function processing on the extracted 8 bits to obtain an 8-bit output feature map. The 8 bits extracted from these 15 bits are determined based on the step width per bit, i.e., the position of the decimal point.

[0019] Figure 1 shows an example of how an effective bit width (8 bits) is extracted using the comparison method. In the example in Figure 1, a total of 8 bits, including the 5th to 11th bits and 1 sign bit, are extracted from the 15-bit cumulative addition result. Of the cumulative addition result, the large values ​​corresponding to the 12th to 14th bits that cannot be expressed in 8 bits are called "upper limit saturation," and the small values ​​corresponding to the 1st to 4th bits are called "lower limit saturation." Upper limit saturation is saturated to the most significant bit, and lower limit saturation is truncated to the least significant bit. The comparison method counts the number of times this saturation and truncation occurs, and adjusts the position of the decimal point for the next frame based on a comparison of the count with a threshold.

[0020] In the comparison method, since only the decimal point position is changed while the effective bit width is fixed, there is a trade-off between the upper limit saturation count and the lower limit saturation count, and therefore, even if the decimal point position is adjusted, it may not be possible to keep the count below the threshold value for either or both the upper and lower limits.

[0021] For example, Fig. 2 shows an example of the upper and lower limit saturation counts before and after the decimal point position is adjusted in the comparison method. The left diagram in Fig. 2 shows the counts where the cumulative sum exceeds the upper and lower limit thresholds before the decimal point position is adjusted, and the right diagram in Fig. 2 shows the counts where the cumulative sum exceeds the upper and lower limit thresholds after the decimal point position is adjusted. As shown in Fig. 2, before adjustment, the upper limit saturation count exceeds the threshold, and the lower limit saturation count is below the threshold.

[0022] In the comparative method, adjusting the decimal point position to prevent one count from exceeding the threshold may cause the other count to exceed the threshold. In the example of Figure 2, adjusting the decimal point position allows the upper saturation count to be kept within the threshold, but the lower saturation count exceeds the threshold.

[0023] Furthermore, as shown in Figure 3, if both counts exceed the threshold before adjusting the decimal point position, adjusting the decimal point position will not bring the counts below the threshold. Specifically, as shown in the upper right diagram of Figure 3, suppose the decimal point position is adjusted to increase the cumulative sum so as to keep the upper saturation count below the threshold. In this case, the lower saturation counter count will increase further beyond the threshold. Similarly, as shown in the lower right diagram of Figure 3, suppose the decimal point position is adjusted to decrease the cumulative sum so as to keep the lower saturation count below the threshold. In this case, the upper saturation counter count will increase further beyond the threshold.

[0024] In the following embodiments, the decimal point position is adjusted for the convolution operation layer for the input video in the same manner as in the comparison method. If at least one of the upper saturation count and the lower saturation count does not fall within the threshold, the effective bit width, i.e., the range width, is expanded by a small number of bits. In the following embodiments, an example is described in which the disclosed technology is applied to a video inference processing device that uses a CNN to perform inference processing for object detection from each frame of video.

[0025] 4 is a block diagram showing the hardware configuration of a visual inference processing device 10 according to Embodiment 1. As shown in FIG. 4, the visual inference processing device 10 includes a range control device 200 and an arithmetic circuit 300.

[0026] The range control device 200 has a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, and an input / output I / F (Interface) 204. Each component is connected to each other via a bus 205 so as to be able to communicate with each other.

[0027] The CPU 201 is a central processing unit that executes various programs and controls each component. That is, the CPU 201 reads programs from the ROM 202 and executes the programs using the RAM 203 as a work area. The CPU 201 controls the above components and performs various arithmetic processing in accordance with the programs stored in the ROM 202. In this embodiment, the ROM 202 stores a range control program, which will be described later.

[0028] The ROM 202 stores various programs and various data. The RAM 203 temporarily stores programs or data as a work area. The input / output I / F 204 is an interface for exchanging data with the arithmetic circuit 300.

[0029] The arithmetic circuit 300 is a hardware circuit configured to perform arithmetic processing corresponding to each of the multiple layers constituting the CNN in accordance with the CNN processing algorithm. The arithmetic circuit 300 includes a product-sum operation circuit 301 that performs a product-sum operation on two pieces of data, and an accumulator circuit 302 that accumulates the product-sum operation results.

[0030] The arithmetic circuit 300 includes registers corresponding to the decimal point position and ranges, which will be described later. The range control device 200 also includes registers corresponding to the upper saturation counter, lower saturation counter, upper threshold, and lower threshold, which will be described later.

[0031] Next, the functional configuration of the visual inference processing device 10 according to the first embodiment will be described. Fig. 5 is a functional block diagram of the visual inference processing device 10. As shown in Fig. 5, the functional configuration of the visual inference processing device 10 includes a calculation unit 30 and a range control unit 20.

[0032] The arithmetic unit 30 further includes, as functional components, a product-sum calculation unit 31 and an accumulative addition unit 32. When the product-sum calculation circuit 301 executes processing, the product-sum calculation circuit 301 functions as the product-sum calculation unit 31, and when the accumulative addition circuit 302 executes processing, the accumulative addition circuit 302 functions as the accumulative addition unit 32. In other words, the arithmetic circuit 300 functions as the arithmetic unit 30.

[0033] Range control unit 20 further includes, as functional components, upper limit saturation counter 21, lower limit saturation counter 22, decimal point position determination unit 23, expansion determination unit 24, reduction determination unit 25, and setting unit 26. Each functional component of range control unit 20 is realized when CPU 201 reads out a range control program stored in ROM 202, expands it in RAM 203, and executes it.

[0034] The product-sum operation unit 31 performs a product-sum operation on two pieces of data with a first bit width (e.g., 16 bits) in which the position of the decimal point is set, and passes the product-sum operation result to the cumulative addition unit 32. Here, the two pieces of data are an input image (feature map) and a kernel used in the convolution operation process.

[0035] The cumulative addition unit 32 accumulates the results of the product-sum operations, and outputs the accumulated result by cutting it out into a range set by a second bit width (e.g., 8 bits) within the first bit width based on the position of the decimal point. The second bit width is the above-mentioned effective bit width. Each time the cumulative addition unit 32 obtains a cumulative addition result, if the cumulative addition result exceeds the upper limit of the range set by the set effective bit width, the cumulative addition unit 32 outputs a signal to the upper limit saturation counter 21 to instruct it to count up. Similarly, if the cumulative addition result is below the minimum step width of the lower limit of the range set by the set effective bit width, the cumulative addition unit 32 outputs a signal to the lower limit saturation counter 22 to instruct it to count up.

[0036] The upper limit saturation counter 21 counts the number of times the cumulative addition result in the cumulative addition unit 32 exceeds the upper limit of the range of the second bit width. The lower limit saturation counter 22 counts the number of times the cumulative addition result in the cumulative addition unit 32 falls below the minimum step width of the range of the second bit width. The minimum step width is the value represented by the least significant bit within the effective bit width. For example, if the effective bit width is 8 bits and the range is −128 to 127, the minimum step width is “1.” In this case, for example, “0.5” cannot be accurately represented and is represented as “0.” Therefore, the lower limit saturation counter 22 regards such values ​​represented as “0” as lower limit saturation and counts them as an index of numerical precision.

[0037] Specifically, each of the upper limit saturation counter 21 and the lower limit saturation counter 22 adds one to its count when it receives a count-up instruction from the cumulative addition unit 32. Each of the upper limit saturation counter 21 and the lower limit saturation counter 22 resets its counter when processing for one frame is completed and the count is read by the decimal point position determination unit 23. This counts the number of times the cumulative addition result exceeds the upper limit (hereinafter also referred to as the "upper limit saturation count") and the number of times it falls below the minimum step width of the lower limit (hereinafter also referred to as the "lower limit saturation count") during one frame. Hereinafter, when the upper limit saturation count and the lower limit saturation count are not distinguished from each other, they are simply referred to as the "count."

[0038] After the calculation unit 30 has completed processing for one frame, the decimal point position determination unit 23 reads the upper limit saturation count number and the lower limit saturation count number from the upper limit saturation counter 21 and the lower limit saturation counter 22, respectively. The decimal point position determination unit 23 determines whether either the upper limit saturation count number or the lower limit saturation count number exceeds a predetermined threshold. If either count number exceeds the predetermined threshold, the decimal point position determination unit 23 determines whether the estimated value of the count number will be less than the threshold if the position of the decimal point set in the first bit width is shifted toward the upper limit or the lower limit.

[0039] Specifically, when the upper saturation count exceeds the upper threshold, the decimal point position determination unit 23 determines a predetermined multiple of the lower saturation count as an estimate of the lower saturation count after decimal point position adjustment. Similarly, when the lower saturation count exceeds the lower threshold, the decimal point position determination unit 23 determines a predetermined multiple of the number of times the upper saturation count exceeds the upper threshold as an estimate of the upper saturation count after decimal point position adjustment.

[0040] For example, the decimal point position determination unit 23 determines whether the upper limit saturation count is greater than the threshold λ 上限 and whether the lower limit saturation count exceeds the threshold λ 下限 The decimal point position determination unit 23 determines whether or not the upper limit saturation count exceeds the threshold λ 上限 If the estimated number of counts is greater than the threshold λ 下限 The decimal point position determination unit 23 determines whether the estimated value of the count number exceeds the threshold λ 下限 If the counts do not exceed the threshold, it is determined that the situation where both counts exceed the threshold can be resolved by adjusting the decimal point position.

[0041] The decimal point position determination unit 23 notifies the reduction determination unit 25 of a determination result indicating that neither count number exceeds the threshold. The decimal point position determination unit 23 also notifies the setting unit 26 of a determination result indicating that adjusting the decimal point position can eliminate the situation where both count numbers exceed the threshold. The decimal point position determination unit 23 also notifies the extension determination unit 24 of a determination result indicating that adjusting the decimal point position cannot eliminate the situation where either count number exceeds the threshold.

[0042] When the decimal point position determination unit 23 notifies the extension determination unit 24 of a determination result indicating that the problem cannot be resolved, the extension determination unit 24 determines whether to extend the value range to the upper limit or the lower limit based on the number of times counted by the upper limit saturation counter 21 and the lower limit saturation counter 22.

[0043] Specifically, the extension determination unit 24 determines whether the upper limit exceedance rate, which is the number of times the cumulative sum exceeds the upper limit threshold, is greater than the lower limit exceedance rate, which is the number of times the cumulative sum falls below the lower limit minimum step width, for the lower limit threshold. If the upper limit exceedance rate is greater than the lower limit exceedance rate, the extension determination unit 24 determines to extend the value range toward the upper limit. Also, if the lower limit exceedance rate is greater than the upper limit exceedance rate, the extension determination unit 24 determines to extend the value range toward the lower limit.

[0044] For example, if the upper saturation count is 8 and the threshold λ 上限 is 5, the lower limit saturation count is 12, and the threshold λ 下限 is 10, the extension determination unit 24 determines the upper limit exceedance rate E 上限 =8 / 5=160%, lower limit excess rate E 下限 = 12 / 10 = 120% and determines that the upper limit exceedance rate is greater. If the extension determination unit 24 determines that the upper limit exceedance rate is greater, it notifies the setting unit 26 of the determination result indicating that the value range should be extended toward the upper limit, and if it determines that the lower limit exceedance rate is greater, it notifies the setting unit 26 of the determination result indicating that the value range should be extended toward the lower limit.

[0045] When the decimal point position determination unit 23 notifies the reduction determination unit 25 of a determination result indicating that none of the count numbers exceeds the threshold, the reduction determination unit 25 determines whether the currently set effective bit width exceeds the reference bit width. For example, if the reference effective bit width is 8 bits and the setting unit 26 has expanded the effective bit width to 9 bits or more as determined by the extension determination unit 24, the reduction determination unit 25 determines that the currently set effective bit width exceeds the reference bit width. In this case, the reduction determination unit 25 determines whether to reduce the range from the upper limit side or the lower limit side. Because the effective bit width continues to increase as the frame processing progresses as determined by the extension determination unit 24, the reduction determination unit 25 functions to improve the efficiency of calculations.

[0046] Specifically, the reduction determination unit 25 calculates an upper limit margin rate for an upper limit threshold, which is expressed as the difference between the upper limit threshold and the number of times the cumulative sum result exceeds the upper limit, and a lower limit margin rate for a lower limit threshold, which is expressed as the difference between the lower limit threshold and the number of times the cumulative sum result falls below the lower limit minimum step width. If the upper limit margin rate is greater than the lower limit margin rate, the reduction determination unit 25 determines to reduce the value range from the upper limit side, and if the lower limit margin rate is greater than the upper limit margin rate, the reduction determination unit 25 determines to reduce the value range from the lower limit side.

[0047] For example, if the upper saturation count is 3 and the threshold 上限 is 5, the lower limit saturation count is 2, and the threshold 下限 is 10, the reduction determination unit 25 上限 = (5-3) / 5 = 40%, lower limit margin rate M 下限 = (10 - 2) / 10 = 80%. In this case, the reduction determination unit 25 determines that the lower limit margin rate is larger. If the reduction determination unit 25 determines that the upper limit margin rate is larger, it notifies the setting unit 26 of the determination result indicating that the value range should be reduced from the upper limit side, and if the reduction determination unit 25 determines that the lower limit margin rate is larger, it notifies the setting unit 26 of the determination result indicating that the value range should be reduced from the lower limit side.

[0048] The setting unit 26 sets the decimal point position and the effective bit width in the arithmetic circuit 300 based on the determination results of the decimal point position determination unit 23 , the extension determination unit 24 , and the reduction determination unit 25 .

[0049] Specifically, it is assumed that the setting unit 26 is notified of a determination result indicating that adjusting the decimal point position can eliminate the situation where both count numbers exceed the threshold value by the decimal point position determination unit 23. In this case, the setting unit 26 shifts the current decimal point position by a predetermined number of bits (e.g., 1 bit) toward the side where the count number exceeds the threshold value, and sets the new decimal point position to that position.

[0050] In addition, when the setting unit 26 is notified by the extension determination unit 24 of a determination result indicating that the value range should be extended to the upper or lower limit, the setting unit 26 sets the value range as a new value range by extending the upper or lower limit of the current value range by a predetermined number of bits (for example, 1 bit).

[0051] In addition, when the setting unit 26 is notified by the reduction determination unit 25 of a determination result indicating that the range is to be reduced from the upper limit or lower limit side, the setting unit 26 sets the range as a new range by reducing the upper limit or lower limit side of the current range by a predetermined number of bits (for example, 1 bit).

[0052] Next, the operation of the visual inference processing device 10 will be described.

[0053] When video is input to the video inference processing device 10, for each frame of the video, the product-sum operation unit 31 performs a product-sum operation on the input video (feature map) and the kernel at a first bit width (e.g., 16 bits) where the decimal point position is set, and passes the product-sum operation result to the cumulative addition unit 32. The cumulative addition unit 32 then accumulates the product-sum operation results, and outputs the cumulative addition result by cutting it out in a range set by an effective bit width (e.g., 8 bits) based on the decimal point position. Furthermore, if the cumulative addition result exceeds the upper limit of the range or falls below the minimum step width of the lower limit of the range, the cumulative addition unit 32 outputs a signal to the upper limit saturation counter 21 and the lower limit saturation counter 22 to instruct them to count up.

[0054] Range control processing is executed in the range control unit 20. Figures 6 and 7 are flowcharts showing an example of the range control processing executed by the range control unit 20. The range control processing is performed by the CPU 201 reading out a range control program from the ROM 202, expanding it into the RAM 203, and executing it.

[0055] In step S10, the CPU 201, functioning as the setting unit 26, sets the variable i, which indicates the frame number of the video, to 0. Next, in step S11, the CPU 201, functioning as the setting unit 26, determines whether i is equal to or less than the final frame number of the video. If i≦the final frame number, the process proceeds to step S12, and if i>the final frame number, the range control process ends.

[0056] Next, in step S11, CPU 201, functioning as decimal point position determination unit 23, reads out the upper limit saturation count number and the lower limit saturation count number from upper limit saturation counter 21 and lower limit saturation counter 22. Next, in step S13, CPU 201, functioning as decimal point position determination unit 23, determines whether the upper limit saturation count number is greater than threshold λ 上限 It is determined whether the upper limit saturation count is greater than the threshold λ. 上限 In this case, the process proceeds to step S14, and the upper limit saturation count number≦threshold λ 上限 In this case, the process proceeds to step S20.

[0057] In step S14, the CPU 201, as the decimal point position determination unit 23, determines whether the lower limit saturation count is equal to or smaller than the threshold λ 下限 It is determined whether the lower limit saturation count is greater than the threshold λ. 下限 In this case, the process proceeds to step S15, and the lower limit saturation count number≦threshold value λ 下限 In this case, the process proceeds to step S18.

[0058] In step S15, the CPU 201, functioning as the extension determination unit 24, determines the upper limit exceedance rate E 上限 and lower limit exceedance rate E 下限 Estimate the upper limit exceedance rate E 上限 is the lower limit exceedance rate E 下限 Determine whether it is greater than E 上限 >E 下限 In this case, the process proceeds to step S16. 上限 ≦E 下限 In this case, the process proceeds to step S17.

[0059] In step S16, the CPU 201, as the setting unit 26, sets a range obtained by expanding the upper limit of the current range, i.e., the MSB (Most Significant Bit) side by one bit, as a new range. In step S17, the CPU 201, as the setting unit 26, sets a range obtained by expanding the lower limit of the current range, i.e., the LSB (Least Significant Bit) side by one bit, as a new range.

[0060] On the other hand, in step S18, the CPU 201 determines in the decimal point position determination unit 23 that the value obtained by multiplying the lower limit saturation count by two is the threshold λ下限 It is determined whether the lower limit saturation count number×2<threshold λ 下限 In this case, the process proceeds to step S19, and the lower limit saturation count number×2≧threshold value λ 下限 In step S19, the CPU 201, functioning as the setting unit 26, sets the position obtained by shifting the current decimal point position by one bit toward the MSB as the new decimal point position. As a result, the value range determined based on the decimal point position is also shifted by one bit toward the MSB.

[0061] On the other hand, in step S20, the CPU 201, as the decimal point position determination unit 23, determines whether the lower limit saturation count number is equal to or smaller than the threshold λ 下限 It is determined whether the lower limit saturation count is greater than the threshold λ. 下限 In this case, the process proceeds to step S21, and the lower limit saturation count number≦threshold value λ 下限 In this case, the process proceeds to step S23 (FIG. 7).

[0062] In step S21, the CPU 201 determines, as the decimal point position determination unit 23, that the upper limit saturation count number multiplied by two is the threshold λ 上限 Determine whether it is smaller than the upper limit saturation count number×2<threshold λ 上限 In this case, the process proceeds to step S22, and the upper limit saturation count number×2≧threshold value λ 上限 In step S22, the CPU 201, functioning as the setting unit 26, sets the position obtained by shifting the current decimal point position by one bit toward the LSB as the new decimal point position. As a result, the value range determined based on the decimal point position is also shifted by one bit toward the LSB.

[0063] On the other hand, in step S23, the CPU 201, functioning as the reduction determination unit 25, determines whether the currently set effective bit width exceeds the reference bit width (for example, 8 bits). If the current effective bit width is greater than 8 bits, the process proceeds to step S24, and if the current effective bit width is less than or equal to 8 bits, the process proceeds to step S29.

[0064] In step S24, the CPU 201, as the reduction determination unit 25, determines the upper limit margin rate M 上限 and the lower limit margin rate M 下限Calculate the upper margin rate M 上限 Or the lower limit margin rate M 下限 It is determined whether or not M exceeds 50%. 上限 or M 下限 If M exceeds 50%, the process proceeds to step S25. 上限 and M 下限 If none of the above exceeds 50%, the process proceeds to step S29.

[0065] In step S25, the CPU 201, functioning as the reduction determination unit 25, determines the upper limit margin rate M 上限 is the lower limit margin rate M 下限 Determine whether it is greater than M 上限 >M 下限 In this case, the process proceeds to step S26. 上限 ≦M 下限 In this case, the process proceeds to step S27.

[0066] In step S26, the CPU 201, as the setting unit 26, sets a new range to a range obtained by subtracting one bit from the MSB side of the current range. In step S27, the CPU 201, as the setting unit 26, sets a new range to a range obtained by subtracting one bit from the LSB side of the current range.

[0067] Next, in step S29, the CPU 201 resets the count numbers of the upper limit saturation counter 21 and the lower limit saturation counter 22. Then, the CPU 201, functioning as the setting unit 26, increments the variable i by 1, and returns to step S11.

[0068] As described above, in the video inference processing device according to the first embodiment, the range control unit, when the cumulative addition result of the multiplication and addition operation of two data in a first bit width in which the decimal point position is set and output in a range set in a second bit width within the first bit width based on the decimal point position, exceeds the upper limit of the range and falls below the minimum step width of the lower limit of the range, determines whether the estimated value of the number of times counted by the counter will be less than the threshold if the decimal point position is shifted toward the upper limit or lower limit, and when it is determined that the estimated value of the number of times is not less than the threshold, determines whether the range should be extended toward the upper limit or lower limit based on the number of times counted by the counter, and sets the decimal point position and effective bit width in the calculation circuit based on the results of each determination.

[0069] Specifically, in this embodiment, as shown in FIG. 2 , when only the upper limit saturation count exceeds the threshold, the range on the upper limit side is widened at the expense of the range on the lower limit side by shifting the range (effective bit width) by adjusting the decimal point position (hereinafter also referred to as “range adjustment”). If this results in both counts falling within the threshold, no further adjustment is required. However, if the range adjustment coarsens the range on the lower limit side, and it is predicted that the lower limit saturation count will increase and exceed the threshold, as shown in FIG. 8 , the range width is widened by expanding the upper limit of the effective bit width without performing range adjustment. Then, the state in which the count exceeds the threshold is alleviated without adjusting the range on the lower limit side.

[0070] The left diagram in Figure 8 shows the upper limit saturation count and the lower limit saturation count before the effective bit width is expanded, and the right diagram in Figure 8 shows the upper limit saturation count and the lower limit saturation count after the effective bit width is expanded. By expanding the effective bit width from x bits to x+1 bits, which is the upper limit expansion by one bit, the upper limit saturation count falls within the threshold. By repeating this process over multiple frames, both counts converge to below the threshold. Similarly, if only the lower limit saturation count exceeds the threshold, the upper limit saturation count resulting from range adjustment is estimated, and then either range adjustment or effective bit width expansion is selected.

[0071] As shown in FIG. 3 , even when both counts exceed the threshold, it is difficult to resolve the state in which the counts exceed the threshold by range adjustment, so the effective bit width is selected to be expanded. In this case, as shown in FIG. 9 , the effective bit width on the upper or lower limit side corresponding to the larger of the upper and lower limits, i.e., the excess rate E, is expanded. As described above, this excess rate E is determined from the ratio of the current count to the threshold. In this case, the upper limit side of the effective bit width is expanded by 1 bit in the current frame, and the same determination is made for the next frame and thereafter, and the effective bit width is further expanded as necessary. In the example of FIG. 9 , the effective bit width is expanded from x bits to x+1 bits, and then further expanded to x+2 bits. By repeating this process, both counts can be converged to or below the threshold.

[0072] In this embodiment, if neither the upper saturation count nor the lower saturation count exceeds a threshold, a determination is made to reduce the effective bit width. If the determination is solely to expand the effective bit width, the effective bit width will only increase as the number of frames increases. Therefore, a determination to reduce the effective bit width is made from the perspective of improving computational efficiency. In this embodiment, if the effective bit width exceeds a standard based on a design value or the like, and the margin rate M to the threshold for both counts does not exceed a predetermined percentage (e.g., 50%), a reduction is made on either the upper or lower limit of the effective bit width. By repeating this process until the final frame, the effective bit width and decimal point position converge to an appropriate value, enabling range control that achieves both suppressed increases in computational cost and high inference accuracy.

[0073] That is, according to this embodiment, it is possible to control the range so as to prevent a decrease in inference accuracy caused by reducing the bit width for the calculations included in the inference process.

[0074] In the above embodiment, when both count numbers exceed the threshold, the exceedance rate is estimated from the ratio between the current count number and the threshold, but this is not limiting. The exceedance rate may be estimated by other methods, such as comparing twice the current count number with the threshold.

[0075] Similarly, the margin rate is not limited to being calculated from the ratio between the current count and the threshold value, but may be calculated using another method, such as comparing 1 / 2 of the current count number with the threshold value.

[0076] Second Embodiment Next, a second embodiment will be described. In the second embodiment, components similar to those of the video inference processing device 10 according to the first embodiment will be assigned the same reference numerals and descriptions thereof will be omitted. Furthermore, components having some of the same functions will be assigned reference numerals having the same last two digits and detailed descriptions thereof will be omitted.

[0077] 5, the video inference processing device 110 according to the second embodiment includes, as its functional components, a calculation unit 30 and a range control unit 120. The range control unit 20 further includes, as its functional components, an upper limit saturation counter 121, a lower limit saturation counter 122, a decimal point position determination unit 23, an extension determination unit 124, a reduction determination unit 125, and a setting unit 26.

[0078] The upper limit saturation counter 121 and the lower limit saturation counter 122 are counters that function not only for the decimal point position of the current frame, but also for the decimal point positions before and after it. Specifically, like the upper limit saturation counter 21 and the lower limit saturation counter 22 of the first embodiment, the upper limit saturation counter 121 and the lower limit saturation counter 122 count the upper limit saturation count number and the lower limit saturation count number based on the decimal point position of the current frame. In addition, the upper limit saturation counter 121 and the lower limit saturation counter 122 simultaneously count the count numbers when the decimal point position is shifted.

[0079] For example, because the value of the cumulative addition result obtained by cumulatively adding 8-bit * 8-bit product-sum operations is 16 bits, it is also possible to simultaneously count saturation count numbers when the decimal point position is shifted, as shown in Fig. 10. In the example of Fig. 10, upper limit saturation counter 121 and lower limit saturation counter 122 can count based on the decimal point position of the current frame, a decimal point position shifted by one bit toward the upper limit, and a decimal point position shifted by one bit toward the lower limit.

[0080] The extension determining unit 124 determines whether to extend the upper limit side or the lower limit side, without estimating the excess rate from the count number at the current decimal point position, as in the extension determining unit 24 of the first embodiment.

[0081] For example, if the upper limit saturation count is 8 at the decimal point position of the current frame and the threshold λ 上限 is 5, the lower limit saturation count is 12, and the threshold 下限 If the value is 10, both counts exceed the threshold, so either the upper or lower limit of the effective bit width is expanded by one bit. In this case, if the current decimal point position is shifted one bit toward the lower limit, the upper limit saturation count becomes 2 and the lower limit saturation count becomes 30. Also, if the current decimal point position is shifted one bit toward the upper limit, the upper limit saturation count becomes 10 and the lower limit saturation count becomes 5. In this case, the upper limit exceedance rate when shifted toward the upper limit is 200%, and the lower limit exceedance rate when shifted toward the lower limit is 300%. Therefore, the expansion determination unit 124 determines that the lower limit saturation count should be relaxed, and determines to expand the lower limit of the effective bit width.

[0082] Similarly, the reduction determination unit 125 determines whether to reduce from the upper limit or the lower limit of the number of effective bits, using the upper limit saturation counter number and the lower limit saturation count number at the decimal point positions before and after the decimal point. For example, suppose the reference effective bit width is 8 bits, but the calculation is performed with 9 bits in the current frame. Also, if the upper limit saturation count number at the decimal point position of the current frame is 2 and the threshold 上限 is 5, the lower limit saturation count is 5 and the threshold 下限is 10, the effective bit width exceeds the standard, and both counts do not exceed the threshold, so the effective bit width is reduced. In this case, if the decimal point position of the current frame is shifted one bit toward the lower limit, the upper limit saturation count becomes 1 and the lower limit saturation count becomes 8. Also, if the decimal point position of the current frame is shifted one bit toward the upper limit, the upper limit saturation count becomes 3 and the lower limit saturation count becomes 2. In this case, if the margin rate is the ratio of the count increased by shifting the decimal point position to the threshold, the upper limit margin rate M 上限 = (5-3) / 5 = 40%, lower limit margin rate M 下限 =(10-8) / 10=20%. Therefore, the reduction determination unit 125 determines that there is a large margin on the upper limit side, and determines to reduce the upper limit side of the effective bit width.

[0083] The hardware configuration of the visual inference processing device 110 according to the second embodiment is similar to the hardware configuration of the visual inference processing device 10 according to the first embodiment shown in FIG. 4, and therefore a description thereof will be omitted.

[0084] Furthermore, the operation of the video inference processing device 110 in the second embodiment will not be explained because the only difference from the range control processing in the first embodiment is the processing of steps S15 and S25 of the range control processing shown in Figures 6 and 7.

[0085] As described above, the video inference processing device according to the second embodiment uses not only the upper limit saturation count and the lower limit saturation count based on the decimal point position of the current frame, but also counts based on decimal points shifted forward or backward. This makes it possible to determine whether to expand or reduce the upper limit or the lower limit without inferring the excess rate or margin rate from the count at the decimal point position of the current frame.

[0086] In addition, the range control process executed by the CPU by reading the software (program) in each of the above embodiments may be executed by various processors other than the CPU. Examples of processors in this case include PLDs (Programmable Logic Devices) whose circuit configuration can be changed after manufacture, such as FPGAs (Field-Programmable Gate Arrays), and dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors having a circuit configuration designed specifically to execute specific processes. Furthermore, the range control process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

[0087] In addition, although the above embodiments have been described with respect to an embodiment in which the range control program is pre-stored (installed) in ROM, the present invention is not limited to this. The program may be provided in a form stored on a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

[0088] The following additional notes are provided regarding the above-described embodiments.

[0089] (Supplementary Item 1) A range control device including: a counter that counts the number of times that an accumulated addition result of two data in a first bit width in which a decimal point position is set, cuts out and outputs the accumulated addition result in a range set by a second bit width within the first bit width based on the position of the decimal point, and the number of times that the accumulated addition result exceeds an upper limit of the range and falls below a minimum step width of a lower limit of the range; a first determination unit that, when either of the numbers counted by the counter exceeds a predetermined threshold, determines whether an estimated value of the number counted by the counter will be less than the threshold when the position of the decimal point is shifted toward the upper limit or the lower limit; a second determination unit that, when it is determined by the first determination unit that the estimated value of the number of times is not less than the threshold, determines whether the range should be extended toward the upper limit or the lower limit based on the number counted by the counter; and a setting unit that sets the position of the decimal point and the second bit width in the arithmetic circuit based on the determination results of the first determination unit and the second determination unit.

[0090] (Supplementary Item 2) The second judgment unit determines to extend the value range toward the lower limit when an upper limit exceedance rate, which is represented by the number of times the cumulative addition result exceeds the upper limit for the upper threshold, is greater than a lower limit exceedance rate, which is represented by the number of times the cumulative addition result falls below the minimum step width of the lower limit for the lower threshold, and determines to extend the value range toward the upper limit when the lower limit exceedance rate is greater than the upper limit exceedance rate; and the setting unit sets the range extended toward the upper limit or the lower limit determined by the second judgment unit as the second bit width in the arithmetic circuit. A value range control device according to Supplementary Item 1.

[0091] (Supplementary Item 3) The range control device according to Supplementary Item 2, wherein the second judgment unit calculates the upper limit exceedance rate and the lower limit exceedance rate based on the number of times the cumulative addition result exceeds the upper limit and the lower limit, respectively, in the range when expanded to either the upper limit side or the lower limit side.

[0092] (Supplementary Item 4) A range control device according to any one of Supplementary Items 1 to 3, further comprising a third judgment unit that judges whether to reduce the range from the upper limit side or the lower limit side when none of the numbers counted by the counters exceeds the threshold value and the currently set second bit width exceeds a reference number of bits, and the setting unit sets the decimal point position and the second bit width in the arithmetic circuit based on the judgment results of the first judgment unit, the second judgment unit, and the third judgment unit.

[0093] (Supplementary Item 5) The third determination unit determines to reduce the value range from the upper limit side when an upper limit margin rate, relative to the upper limit threshold, represented by the difference between the upper limit threshold and the number of times the cumulative addition result exceeds the upper limit, is greater than a lower limit margin rate, relative to the lower limit threshold, represented by the difference between the lower limit threshold and the number of times the cumulative addition result falls below the lower limit minimum step width, and determines to reduce the value range from the lower limit side when the lower limit margin rate is greater than the upper limit margin rate; and the setting unit sets the range reduced from the upper limit side or the lower limit side determined by the third determination unit as the second bit width in the arithmetic circuit. A value range control device according to Supplementary Item 4.

[0094] (Supplementary Item 6) The range control device according to Supplementary Item 5, wherein the third determination unit calculates the upper limit margin rate and the lower limit margin rate based on the number of times the cumulative addition result exceeds the upper limit and the number of times it falls below the minimum step width of the lower limit in the range when reduced from either the upper limit side or the lower limit side.

[0095] (Supplementary Item 7) A range control device according to any one of Supplementary Items 1 to 6, wherein, when either the number of times the cumulative addition result exceeds the upper limit or the number of times it falls below the minimum step width of the lower limit exceeds one of the upper and lower limit thresholds, the first determination unit estimates the number of times to a predetermined multiple of the other of the number of times the cumulative addition result exceeds the upper limit or the number of times it falls below the minimum step width of the lower limit; and when the first determination unit determines that the estimated value of the number of times is less than the other of the upper limit and lower limit thresholds, the setting unit sets the position of the decimal point to a position shifted to the other of the upper limit and the lower limit.

[0096] (Supplementary Item 8) The range control device according to any one of Supplementary Items 1 to 7, wherein the output of the arithmetic circuit is a feature map for each layer of a convolutional neural network.

[0097] (Supplementary Item 9) A range control method is executed by a range control device including a counter, a first judgment unit, a second judgment unit, and a setting unit, wherein the counter counts the number of times that an accumulated addition result of a product-sum operation of two data in a first bit width in which a decimal point position is set, cuts out and outputs the accumulated addition result in a range set by a second bit width within the first bit width based on the decimal point position, and the accumulated addition result of the operation circuit exceeds an upper limit of the range and falls below a minimum step width of a lower limit of the range; when either of the numbers counted by the counter exceeds a predetermined threshold, the first judgment unit judges whether an estimated value of the number of times counted by the counter will be less than the threshold when the position of the decimal point is shifted toward the upper limit or the lower limit; when the first judgment unit judges that the estimated value of the number of times is not less than the threshold, the second judgment unit judges whether to extend the range toward the upper limit or the lower limit based on the number of times counted by the counter; the setting unit sets the decimal point position and the second bit width in the arithmetic circuit based on determination results of the first determination unit and the second determination unit.

[0098] (Supplementary Item 10) A range control program for causing a computer to function as each part of the range control device according to any one of Supplementary Items 1 to 8.

[0099] (Supplementary Item 11) A range control device comprising: a memory; and at least one processor connected to the memory, wherein the processor is configured to: determine whether an estimated value of the number of times counted by a counter that counts the number of times an accumulative addition result of two data in a first bit width in which a decimal point position is set cuts out and outputs the result of a product-sum operation in a range set by a second bit width within the first bit width based on the position of the decimal point, when either of the numbers counted by a counter that counts the number of times the accumulative addition result exceeds an upper limit of the range or the number of times it falls below a minimum step width of a lower limit of the range exceeds a predetermined threshold; determine whether an estimated value of the number of times counted by the counter will be less than the threshold if the position of the decimal point is shifted toward the upper limit or the lower limit, when it is determined that the estimated value of the number of times is not less than the threshold, based on the number counted by the counter; and set the position of the decimal point and the second bit width in the arithmetic circuit based on each determination result.

[0100] (Supplementary Item 12) A non-transitory storage medium storing a program executable by a computer to execute a range control process, wherein the range control process comprises: determining whether an estimated value of the number of times counted by a counter that counts the number of times the cumulative addition result of two pieces of data, in a first bit width in which a decimal point position is set, exceeds an upper limit of the range and the number of times it falls below a minimum step width of a lower limit of the range, when either of the numbers counted by a counter exceeds a predetermined threshold; determining whether an estimated value of the number of times counted by the counter will be less than the threshold if the position of the decimal point is shifted toward the upper limit or the lower limit, when it is determined that the estimated value of the number of times is not less than the threshold; determining whether the range should be extended toward the upper limit or the lower limit based on the number counted by the counter; and setting the position of the decimal point and the second bit width in the arithmetic circuit based on each determination result.

[0101] 10, 110 Video inference processing device 20, 120 Range control unit 21, 121 Upper limit saturation counter 22, 122 Lower limit saturation counter 23 Decimal point position determination unit 24, 124 Extension determination unit 25, 125 Reduction determination unit 26 Setting unit 30 Calculation unit 31 Product-sum calculation unit 32 Accumulation unit 110 Video inference processing device 120 Range control unit 121 Upper limit saturation counter 122 Lower limit saturation counter 124 Extension determination unit 125 Reduction determination unit 200 Range control device 201 CPU 202 ROM 203 RAM 204 Input / output I / F 205 Bus 300 Calculation circuit 301 Product-sum calculation circuit 302 Accumulation circuit

Claims

1. A range control device comprising: a counter that counts the number of times that an accumulative addition result of two data, in a first bit width in which a decimal point position is set, exceeds the upper limit of the range and the number of times that it falls below the minimum step width of the lower limit of the range, in an arithmetic circuit that extracts and outputs the accumulative addition result within a range set by a second bit width based on the position of the decimal point; a first judgment unit that, when either of the numbers counted by the counter exceeds a predetermined threshold, judges whether an estimated value of the number counted by the counter will be less than the threshold if the position of the decimal point is shifted toward the upper limit or the lower limit; a second judgment unit that, when the first judgment unit judges that the estimated value of the number will not be less than the threshold, judges whether to extend the range toward the upper limit or the lower limit based on the number counted by the counter; and a setting unit that sets the position of the decimal point and the second bit width in the arithmetic circuit based on the judgment results of the first judgment unit and the second judgment unit.

2. The range control device according to claim 1, wherein the second judgment unit judges that the range should be extended toward the lower limit if an upper limit exceedance rate, which is represented by the number of times the cumulative sum result exceeds the upper limit for the upper threshold, is greater than a lower limit exceedance rate, which is represented by the number of times the cumulative sum result falls below the minimum step width of the lower limit for the lower threshold, and judges that the range should be extended toward the upper limit if the lower limit exceedance rate is greater than the upper limit exceedance rate; and the setting unit sets the range extended toward the upper limit or the lower limit determined by the second judgment unit as the second bit width in the arithmetic circuit.

3. A range control device as described in claim 1 or claim 2, further comprising a third judgment unit that judges whether to reduce the range from the upper limit side or the lower limit side when none of the numbers counted by the counter exceeds the threshold value and the currently set second bit width exceeds a reference number of bits, and the setting unit sets the decimal point position and the second bit width in the arithmetic circuit based on the judgment results of the first judgment unit, the second judgment unit, and the third judgment unit.

4. A range control method executed by a range control device including a counter, a first judgment unit, a second judgment unit, and a setting unit, wherein the counter counts the number of times that an accumulated addition result of a product-sum operation of two data in a first bit width in which a decimal point position is set, cuts out and outputs the accumulated addition result in a range set in a second bit width within the first bit width based on the decimal point position, and the accumulated addition result in each of the first bit width and the second ... the setting unit sets the decimal point position and the second bit width in the arithmetic circuit based on determination results of the first determination unit and the second determination unit.

Images