Arithmetic device, data compression method, and data compression program

Abstract

To provide a technique capable of suppressing an increase in access time.SOLUTION: Arithmetic devices 100, 100B each include: an arithmetic unit 11 that executes a neural network model to perform arithmetic processing; a cache unit 12 in which data used by the arithmetic unit is stored; a calculation unit 31 that calculates a hit rate, which is a ratio at which the data used by the arithmetic unit is stored in the cache unit; a determination unit 32 that determines a compression rate according to the hit rate and a predetermined target hit rate; and a compression unit 33 that compresses the data stored in the cache unit at the compression rate.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a computing device, a data compression method, and a data compression program. [Background technology]

[0002] Techniques for compressing neural network models are known. Patent Document 1 describes a technique for compressing tensor data of weights of a neural network model using Tucker decomposition. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2022-537738 Summary of the Invention [Problem to be solved by the invention]

[0004] In the computational processing for executing a neural network model, data in the intermediate layer of the neural network model stored in cache memory may be referenced. Because the capacity of cache memory is small, if the data in the intermediate layer is not stored in the cache memory, access time becomes long. Therefore, a technology that can suppress the long access time has been desired. [Means for solving the problem]

[0005] According to one embodiment of the present disclosure, there is provided a computing device (100, 100B), which includes a computing unit (11) that executes a neural network model to perform computational processing, a cache unit (12) that stores data used by the computing unit, a calculation unit (31) that calculates a hit rate, which is the rate at which data used by the computing unit is stored in the cache unit, a determination unit (32) that determines a compression rate according to the hit rate and a predetermined target hit rate, and a compression unit (33) that compresses data stored in the cache unit at the compression rate.

[0006] According to this type of computing device, since compression is performed by determining the compression rate according to the hit rate and the target hit rate, there is a high probability that the hit rate can be approximated to the target hit rate, which makes it possible to prevent the hit rate from decreasing and therefore to prevent the access time from becoming longer. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 2 is an explanatory diagram showing an outline of the configuration of a computing device. [Figure 2] FIG. 10 is an explanatory diagram of a calculation process using a neural network model. [Figure 3] 10 is a flowchart showing an example of a calculation process. [Figure 4] FIG. 10 is an explanatory diagram showing an outline of the configuration of a calculation device in a second embodiment. [Figure 5] 10 is a flowchart showing an example of a calculation process in the second embodiment. [Figure 6] 10 is a flowchart showing an example of a calculation process in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] A. First embodiment: As shown in Fig. 1, the arithmetic device 100 includes a CPU 10, a memory 20, and a control circuit unit 30. In this embodiment, the arithmetic device 100 is a device that executes a neural network model to perform arithmetic processing. The arithmetic device 100 is mounted on, for example, a vehicle and performs arithmetic processing for image analysis. The arithmetic device 100 may also be implemented on a cloud. The CPU 10 executes the programs installed in the memory 20 to realize the functions of the calculation unit 11 and performs calculation processing using data stored in the cache unit 12. The control circuit unit 30 is made up of a microcomputer configured with a CPU, RAM, and ROM, and realizes the functions of the calculation unit 31, determination unit 32, and compression unit 33 and manages the memory 20 by the microcomputer executing the programs pre-installed in the RAM, ROM, and memory 20.

[0009] The calculation unit 11 executes a neural network model to perform calculations. As shown in FIG. 2, the neural network includes an input layer L1, multiple intermediate layers L2, and an output layer L3. The number of intermediate layers L2 can be determined arbitrarily. The input layer L1 is a layer to which information is input. The intermediate layer L2 is a layer that calculates features based on information transmitted from the input layer L1. The output layer L3 is a layer that outputs results based on information transmitted from the intermediate layer L2. Each layer includes multiple nodes. Edges connecting nodes in adjacent layers are weighted. The calculation unit 11 performs calculations including a product-sum operation, for example, to multiply node N1 by a weight W11, multiply node N2 by a weight W12, and multiply node N3 by a weight W13, and then add these values ​​together to find the value of node N4.

[0010] The cache unit 12 (see FIG. 1) stores data used by the calculation unit 11 for calculation processing. More specifically, it temporarily stores weights by which values ​​input to nodes in the input layer, intermediate layer, and output layer of the neural network model are multiplied, and values ​​of nodes in the intermediate layer. The process of storing data used by the calculation unit 11 for calculation processing in the cache unit 12 is also referred to as a "storage step."

[0011] The memory 20 stores the programs executed by the components 10 and 30, weights by which values ​​input to nodes in the input layer, intermediate layer, and output layer in the neural network model are multiplied, and values ​​of nodes in the intermediate layer.

[0012] The calculation unit 31 calculates a hit rate, which is the rate at which data used by the calculation unit 11 is stored in the cache unit 12. The hit rate is highly dependent on the storage capacity of the cache unit 12 relative to the memory 20. Generally, the larger the storage capacity of the cache unit 12, the more data can be stored, and therefore the higher the hit rate. Furthermore, the higher the data compression rate, the more data can be stored in the cache unit 12, and therefore the higher the hit rate.

[0013] The determination unit 32 determines the compression rate of the data to be stored in the cache unit 12 by the operation unit 11 based on the hit rate calculated by the calculation unit 31 and a predetermined target hit rate. In the present disclosure, the compression rate is a rate that indicates how much information volume of the compressed data is reduced from the original data when the data is compressed. Details of the determination of the compression rate will be described later.

[0014] The compression unit 33 compresses the data that the calculation unit 11 stores in the cache unit 12 at the compression rate determined by the determination unit 32. Details of the compression method will be described later.

[0015] 3 is a process in which the determination unit 32 determines the compression rate and the compression unit 33 compresses the data transferred by the calculation unit 11. This process is repeatedly executed when the calculation unit 11 executes calculation processing. The calculation processing is also referred to as a "calculation step."

[0016] In step S100, the calculation unit 31 acquires the amount of data to be transferred by the operation unit 11. This step is also referred to as the "calculation step."

[0017] In step S110, the determination unit 32 determines the compression rate based on the hit rate calculated in step S100 and a predetermined target hit rate. This step is also referred to as the "determination step." In this embodiment, the determination unit 32 determines the compression rate from the target hit rate and the hit rate, for example, using the following formula (1):

[0018] CR=r / ra…(1) Here, CR is the compression ratio, r is the calculated hit ratio, and ra is the target hit ratio.

[0019] In this embodiment, the determination unit 32 determines the rank number of the compressed matrix data from the compression rate. The determination unit 32 determines the rank number of the matrix data using a function or table that defines a predetermined relationship between the compression rate and the rank number. The smaller the rank number of the matrix data, the smaller the data volume, and therefore the higher the compression rate.

[0020] In step S120, the compression unit 33 compresses the data transferred by the calculation unit 11 at the compression rate determined in step S110. This step is also referred to as the "compression step." In this embodiment, the compression unit 33 compresses the data by singular value decomposition (SVD). More specifically, the singular value decomposition is performed so that the rank number of the data approximates the rank number calculated using a function or table that defines a predetermined relationship between the compression rate and the rank number.

[0021] According to the computing device 100 of this embodiment described above, compression is performed by determining the compression rate according to the hit rate and the target hit rate, so there is a high probability that the hit rate can be approximated to the target hit rate. Therefore, a decrease in the hit rate can be suppressed, and computation can be performed using data stored in the cache unit 12 without referring to the memory 20. Therefore, an increase in access time can be suppressed.

[0022] B. Second embodiment: 4 differs from the first embodiment in that it includes a measurement unit 34, but the other configurations are the same. The compression process of the second embodiment shown in FIG. 5 differs from the first embodiment in that it determines a target hit rate according to a target access time in step S103, but the other steps are the same.

[0023] The measurement unit 34 (see FIG. 4) measures the access time required for the calculation unit 11 to acquire data used in calculation processing. In this embodiment, the measurement unit 34 can measure the access time required for the calculation unit 11 to acquire data from the cache unit 12 and the access time required for the calculation unit 11 to acquire data from the memory 20.

[0024] In step S103 (see FIG. 5), the determination unit 32 determines a target hit rate from a predetermined target access time. The determination unit 32 determines the target hit rate from the target access time using, for example, the following formula (2).

[0025] AT = T1 × ra + T2 × ra…(2) Here, AT is the target access time, T1 is the access time required to obtain data from the cache unit 12, ra is the target hit rate, and T2 is the access time required to obtain data from the memory 20.

[0026] According to the arithmetic device 100B of the second embodiment described above, the determination unit 32 determines the target hit rate from a predetermined target access time, and therefore, data can be compressed at a compression rate that allows the access time to approximate the target access time.

[0027] C. Third embodiment: 6 differs from the second embodiment in that step S110 is executed when the access time is equal to or greater than a predetermined threshold access time. The other steps are the same as those of the second embodiment. The configuration of the arithmetic device 100B of the third embodiment is the same as that of the arithmetic device 100B of the second embodiment, and therefore a description of the configuration of the arithmetic device 100B will be omitted.

[0028] In step S105, the determination unit 32 determines whether the access time measured by the measurement unit 34 is equal to or greater than a threshold access time. The threshold access time is a value greater than the target access time. If the access time is equal to or greater than the threshold access time, the determination unit 32 proceeds to processing in step S110. On the other hand, if the access time is less than the threshold access time, the determination unit 32 proceeds to processing in step S115.

[0029] In step S115, the determination unit 32 determines the compression ratio to be a default value. The default compression ratio is, for example, the compression ratio stored in the storage area of ​​the calculation unit 31. That is, if the access time is equal to or greater than the threshold access time, the compression ratio is updated, whereas if the access time is less than the threshold access time, the compression ratio is not updated.

[0030] According to the computing device 100B of the third embodiment described above, when the access time is equal to or greater than a predetermined threshold access time, the determination unit 32 determines the compression rate in accordance with the target hit rate determined from the access time and the target access time, and the hit rate. This makes it possible to prevent the compression rate from being changed frequently.

[0031] D. Other Embodiments: (D1) In the above-described embodiment, the arithmetic device 100 may be an accelerator such as a GPU (Graphics Processing Unit) or an NPU (Neural network Processing Unit).

[0032] (D2) In the above-described embodiment, the calculation unit 11 executes a neural network model to perform calculation processing. However, the calculation unit 11 may execute calculation processing without executing a neural network model. Note that the data handled by the calculation unit 11 is preferably matrix data or tensor data.

[0033] (D3) In the above-described embodiment, the calculation unit 31 calculates the hit rate as the rate at which data used by the calculation unit 11 for calculation processing is stored in the cache unit 12. However, the calculation unit 31 may calculate the hit rate as the rate at which data used by the calculation unit 11 for calculation processing is stored in the memory 20.

[0034] (D4) In the above-described embodiment, the determination unit 32 determines the rank number of the matrix after compression using a function or table that defines a predetermined relationship between the compression rate and the rank number. This is not limiting, and the compression unit 33 may determine the rank number of the matrix after compression using a function or table that defines a predetermined relationship between the compression rate and the rank number.

[0035] (D5) In the above-described embodiment, the compression unit 33 performs compression using singular value decomposition. Compression is not limited to this, and compression using sparsification or an algorithm may also be performed. Furthermore, if the data to be transferred is tensor data, the compression unit 33 can compress it using Tucker decomposition.

[0036] (D6) In the second and third embodiments described above, the determination unit 32 determines the target hit rate from the target access time. However, the present invention is not limited to this, and the compression unit 33 may determine the target hit rate from the target access time.

[0037] The present disclosure is not limited to the above-described embodiments and can be realized in various configurations without departing from the spirit thereof. For example, the technical features in the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention section can be appropriately replaced or combined to solve the above-described problems or achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0038] The control circuitry 30 and techniques described herein may be implemented by a special-purpose computer configured with a processor and memory programmed to perform one or more functions embodied in a computer program. Alternatively, the control circuitry 30 and techniques described herein may be implemented by a special-purpose computer configured with one or more dedicated hardware logic circuits. Alternatively, the control circuitry 30 and techniques described herein may be implemented by one or more special-purpose computers configured with a processor and memory programmed to perform one or more functions in combination with a processor configured with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium. [Explanation of symbols]

[0039] 10...CPU, 11...arithmetic unit, 12...cache unit, 20...memory, 30...control circuit unit, 31...calculation unit, 32...determination unit, 33...compression unit, 34...measurement unit, 100, 100B...arithmetic device

Claims

1. A computing device (100, 100B), A calculation unit (11) that executes a neural network model and performs calculation processing; a cache unit (12) in which data used by the calculation unit is stored; a calculation unit (31) that calculates a hit rate, which is the rate at which data used by the calculation unit is stored in the cache unit; a determination unit (32) that determines a compression rate according to the hit rate and a predetermined target hit rate; a compression unit (33) that compresses data stored in the cache unit at the compression rate.

2. 2. The computing device according to claim 1, further comprising: a measurement unit (34) that measures an access time required for the calculation unit to acquire data used in the calculation process, The target hit rate is determined in accordance with a predetermined target access time.

3. 3. The computing device according to claim 2, The determination unit determines the compression rate in accordance with the hit rate and the target hit rate when the access time is equal to or greater than a predetermined threshold access time.

4. A data compression method executed by a computing device, comprising: a calculation step of executing the neural network model to perform calculations; a storing step of storing data used in the calculation step in a cache unit; a calculation step of calculating a hit rate, which is a rate at which data used in the calculation step is stored in the cache unit; a determining step of determining a compression rate according to the hit rate and a predetermined target hit rate; a compression step of compressing the data stored in the storage step at the compression rate.

5. A data compression program, A calculation function that executes the neural network model and performs calculations; a storage function for storing data used in the calculation function in a cache unit; a calculation function for calculating a hit rate, which is a rate at which data used in the calculation function is stored in the cache unit; a determining function for determining a compression rate according to the hit rate and a predetermined target hit rate; a compression function of compressing the data stored by the storage function at the compression rate; and a compression program that causes a computer to realize the compression function.

Images