Data processing method, data processing device, and data processing program

Abstract

To provide a method for data processing for conducting second calculation processing by using data in the middle of calculation in first calculation processing, the method being capable of reducing a memory bandwidth by reducing the usage amount of a memory holding the data in the middle of calculation in the first calculation processing.SOLUTION: A method for data processing regarding a machine learning model includes: generating compression data by compressing data being calculated in first calculation processing; storing the generated compression data into a memory region; and conducting second calculation processing by using the compression data stored in the memory region. The data being calculated is floating-point number data and reducing the number of bits in the mantissa part of the data being calculated leads to generation of the compression data.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present disclosure relates to a data processing method , data processing device and data processing program .

Background Art

[0002] Generally, in machine learning, in data processing such as model training, intermediate data, which are intermediate results of calculation in forward processing, may be stored in an external memory such as a DRAM (Dynamic Random Access Memory) for backward processing. Then, among the intermediate data stored in the external memory, intermediate data necessary for the calculation of backward processing may be read from the external memory, and the calculation of backward processing is executed.

Summary of Invention

Problem to be Solved by the Invention

[0003] It may be difficult to sufficiently secure a memory bandwidth between a processor and an external memory that is required to store intermediate data generated for such machine learning in an external memory every time.

Means for Solving the Problem

[0004] The data processing method according to an embodiment of the present invention At least one processor performs a forward process in a neural network including at least a first layer and a second layer, generating first intermediate data from the first layer and second intermediate data from the second layer; at least one processor generates compressed data by compressing the first intermediate data; at least one processor performs a backward process on the neural network based on the compressed data and the second intermediate data; at least one processor updates the parameters of the neural network after the backward process; the compressed data is generated by converting the first intermediate data, expressed in a first numeric format, to a second numeric format having fewer bits than the first numeric format, the second numeric format being a floating-point data format; the update is performed using ADAM; and the second intermediate data is used in the backward process without lossy compression. .

Brief Description of Drawings

[0005] [Figure 1] It is a block diagram showing an example of a data processing device according to a first embodiment of the present invention. [Figure 2] It is an explanatory diagram showing an example of neural network training executed by the data processing device shown in FIG. 1. [Figure 3] It is an explanatory diagram showing an example of forward processing in neural network training according to the present embodiment. [Figure 4]This is an explanatory diagram illustrating an example of backward processing and optimization processing in the training of the neural network in this embodiment. [Figure 5] This is an explanatory diagram showing an example of a decision table used in the forward processing of the first embodiment. [Figure 6] Figure 1 is a flowchart illustrating an example of forward processing of a neural network using the data processing device. [Figure 7] This is an explanatory diagram showing an example of a decision table used in the forward processing of a neural network by a data processing device according to a second embodiment of the present invention. [Figure 8] This flowchart shows an example of forward processing of a neural network using a data processing device of the second embodiment. [Figure 9] This is a flowchart showing the continuation of the forward processing in Figure 8. [Figure 10] This block diagram shows an example of the hardware configuration of the data processing device of the embodiment described above. [Modes for carrying out the invention]

[0006] Embodiments of the present invention will be described in detail below with reference to the drawings.

[0007] Figure 1 is a block diagram showing an example of a data processing device in a first embodiment of the present invention. The data processing device 100 shown in Figure 1 has at least one system board 10 including a processor 20 and a plurality of DRAMs (Dynamic Random Access Memory) 50 connected to the processor 20. For example, the data processing device 100 may be a server.

[0008] The processor 20 has multiple arithmetic units 30 and multiple SRAMs (Static Random Access Memory) 40 connected to each of the multiple arithmetic units 30. The processor 20 is connected to the system bus. The processor 20 may be in the form of a chip or in the form of a package. The arithmetic unit 30 is an example of an arithmetic processing unit.

[0009] In this embodiment, the memory bandwidth of SRAM 40 is greater than that of DRAM 50. Therefore, it is preferable to store the data used by the processor 20 for calculations in SRAM 40 if it is possible to store it in SRAM 40. However, if SRAM 40 is built into the processor 20, it may be difficult to store all the data used by the processor 20 in SRAM 40. In this case, the data that cannot be stored in SRAM 40 may be stored in DRAM 50, which has a smaller memory bandwidth.

[0010] The internal memory connected to the arithmetic unit 30 is not limited to SRAM 40, but may be, for example, cache memory. The external memory connected to the processor 20 is not limited to DRAM 50, but may be, for example, MRAM (Magnetoresistive Random Access Memory), HDD (Hard Disk Drive), or SSD (Solid State Drive). SRAM 40 is an example of the first memory, and the memory area allocated to SRAM 40 is an example of the first memory area. DRAM 50 is an example of the second memory, and the memory area allocated to DRAM 50 is an example of the second memory area.

[0011] Thus, the data processing device 100 of this embodiment has multiple types of memory with different memory bandwidths (in this embodiment, SRAM 40 and DRAM 50).

[0012] Furthermore, if an SRAM 40 with sufficient memory capacity can be mounted on the processor 20 or system board 10, the first memory area and the second memory area may be allocated to the SRAM 40.

[0013] The data processing device 100 performs multiple computational processes to train a neural network having multiple layers. One of these computational processes is, for example, forward processing of the neural network, and another is backward processing of the neural network. Furthermore, the computational processes performed by the data processing device 100 are not limited to training the neural network. For example, the data processing device 100 may perform computational processes such as scientific and technical calculations.

[0014] Figure 2 is an explanatory diagram illustrating an example of neural network training performed by the data processing device 100 shown in Figure 1. For example, Figure 2 shows an example of a method for generating a neural network model using machine learning. In the machine learning method for training a neural network having multiple intermediate layers between the input layer and the output layer of this embodiment, forward processing, backward processing, and optimization processing are repeatedly performed multiple times while changing the training data. The data processing device 100 then generates a neural network model based on the training. Forward processing, backward processing, and optimization processing are explained in Figures 3 and 4. Forward processing is an example of a first computation process, and backward processing is an example of a second computation process. Forward processing, backward processing, and optimization processing are examples of a third computation process. In this specification, model generation or neural network generation includes adjusting the model or neural network parameters.

[0015] Figure 3 is an explanatory diagram illustrating an example of forward processing in the training of a neural network in this embodiment. For example, Figure 3 shows an example of forward processing in a method for generating a neural network model using machine learning. In forward processing, data and parameters such as weights are input to the input layer and a predetermined number of intermediate layers. In the input layer, the input data and parameter 1 are processed to generate intermediate data 1. In the intermediate layer following the input layer, intermediate data 1 and parameter 2 are processed to generate intermediate data 2.

[0016] Also in the subsequent intermediate layers, calculation is performed on the intermediate data generated by the immediately preceding intermediate layer and the parameters set for each intermediate layer, and the intermediate data generated by the calculation is output to the next intermediate layer. Note that there may also be intermediate layers that do not use parameters. Examples of the intermediate layers include a convolutional layer, a pooling layer, and a fully connected layer.

[0017] In the present embodiment, the intermediate data generated by the calculation processing of the input layer and the intermediate layers is stored in the SRAM 40 without being compressed. Then, the intermediate layers and output layer that execute calculation processing read uncompressed intermediate data from the SRAM 40 and use it for the calculation processing. By using uncompressed intermediate data in forward processing, the data processing apparatus 100 can execute forward processing without reducing calculation accuracy. The intermediate data is an example of data in the middle of calculation generated by each layer through calculation processing.

[0018] This intermediate data is also used in backward processing described with reference to FIG. 4. In the present embodiment, after being compressed, the intermediate data used in backward processing is stored in the SRAM 40, the DRAM 50, or both the SRAM 40 and the DRAM 50. In backward processing, by compressing intermediate data and then storing it in memory, the memory usage can be reduced. Accordingly, without mounting a higher-speed DRAM 50 or increasing the data bus width, it is possible to sufficiently secure the memory bandwidth between the processor 20 and the DRAM 50 required for storing the intermediate data generated by forward processing in the DRAM 50. In other words, the memory bandwidth required for storing intermediate data in the DRAM 50 can be reduced compared to the memory bandwidth required for storing all intermediate data in the DRAM 50. Consequently, an increase in the system cost of the data processing apparatus 100 can be suppressed.

[0019] For example, the intermediate data used in backward processing may be irreversibly compressed. Irreversible compression has a lower compression cost than lossless compression, and can often achieve a constant compression rate, so the load imposed on the processor 20 by the compression processing can be reduced.

[0020] Furthermore, in the calculation of backward processing that uses intermediate data obtained in forward processing, errors in the intermediate data often only have local effects and do not propagate over a wide range and accumulate. For example, in the backward processing of a convolutional layer, the intermediate data generated by forward processing only affects the gradient of the weights of that convolutional layer.

[0021] Furthermore, the gradient value calculated by backward processing may not require higher accuracy than that required in forward processing. For example, in the weight update of stochastic gradient descent, the gradient value is expected to be smaller than the weight value, so even when the relative error of the gradient is large, the influence on the calculation of backward processing can be reduced. For this reason, even when backward processing is executed using compressed intermediate data, appropriate weights can be calculated.

[0022] The data processing apparatus 100 can perform the above-described operations, for example, by using conversion of floating-point number data. Specifically, the intermediate data may be compressed by executing calculation processing in forward processing using double-precision floating-point number data, and converting the generated intermediate data from double-precision floating-point number data to single-precision floating-point number data. Furthermore, the data processing apparatus 100 may compress the intermediate data by converting single-precision floating-point number data to 8-bit fixed-point number data. This allows easy irreversible compression of intermediate data using existing conversion methods. Furthermore, the data processing apparatus 100 may compress the intermediate data by reducing the number of bits (number of digits) of the mantissa part of floating-point number data.

[0023] The compression ratio of the intermediate data may be set higher as the neural network training progresses. That is, the neural network training shown in Figure 2 may be performed repeatedly while gradually increasing the compression ratio of the intermediate data. For example, the data processing device 100 may calculate the first predetermined number of iterations using single-precision floating-point data, and the next predetermined number of iterations using half-precision floating-point data. The data processing device 100 may then calculate the next 100 iterations using 8-bit fixed-point data. The predetermined number of iterations is, for example, 100 iterations.

[0024] Furthermore, when using floating-point data in forward processing, the data processing device 100 may gradually increase the compression ratio of the intermediate data by sequentially decreasing the number of bits in the mantissa each time a predetermined number of iterations are performed. By gradually increasing the compression ratio of the intermediate data in this way, the memory bandwidth required for transferring the intermediate data can be further reduced, and the increase in the system cost of the data processing device 100 can be further suppressed.

[0025] Furthermore, the data processing device 100 may compress multiple intermediate data files together rather than compressing them one by one. In this case, it may be possible to further increase the compression ratio of the intermediate data, which can contribute to reducing memory bandwidth and system costs.

[0026] In the output layer, the output data is calculated using the intermediate data N generated by the preceding intermediate layer N (the Nth layer). In the output layer, which calculates the error in a classification problem, the output data (solution) is calculated by, for example, using the softmax function as the activation function and cross-entropy as the error function. In the output layer, as explained in Figure 4, the error from the correct answer (loss function) is calculated by comparing the output data with the training data (correct answer data).

[0027] In this way, in forward processing, the input data and parameters are calculated at each layer of the neural network to generate data to be input to the next layer (intermediate data), and the output data is output from the final layer (forward propagation). Forward processing can be used not only for training neural networks but also for inference using neural networks. Forward processing can be represented by computation graphs such as DAG (Directed Acyclic Graph).

[0028] Figure 4 is an explanatory diagram illustrating an example of backward processing and optimization processing in the training of a neural network in this embodiment. For example, Figure 4 shows an example of backward processing in a method for generating a neural network model using machine learning. In backward processing, backpropagation is performed, in which the error is propagated in the reverse order of forward processing. In Figure 4, the sign Δ indicates the error in the data or the error in the parameters. The parameter update process performed in the optimization process is indicated by a dashed arrow.

[0029] First, in backward processing, the error calculation layer (output layer) compares the output data generated in forward processing with the training data, and generates Δintermediate data N, which is the error with respect to the intermediate data N input to the output layer. Δintermediate data N is also the error of the output data output by the Nth intermediate layer.

[0030] Next, in each intermediate layer, starting with the intermediate layer closest to the output layer, the error with respect to the output data (Δintermediate data) and the input data (intermediate data) are calculated, and a Δparameter, which is the error with respect to the parameters of that intermediate layer, is generated. The Δparameter represents the slope of the parameter in a curve that shows the change in error with respect to the change in the parameter. For example, in an intermediate layer adjacent to the input layer, Δintermediate data 2 and intermediate data 1 are calculated to obtain Δparameter 2.

[0031] Furthermore, in each intermediate layer, the error with respect to the output data (Δintermediate data) and the parameters of that intermediate layer are calculated to generate Δintermediate data, which is the error with respect to the input data of that intermediate layer. The error with respect to the input data of that intermediate layer (Δintermediate data) is also the error with respect to the output data of the previous intermediate layer (or input layer). For example, in an intermediate layer adjacent to the input layer, Δintermediate data 2 and parameter 2 are calculated to obtain Δintermediate data 1. Here, the intermediate data is read from, for example, SRAM 40 or DRAM 50 for each layer.

[0032] In the input layer, similar to the intermediate layer, Δintermediate data 1 and the input data are used to calculate Δparameter 1, and then Δintermediate data 1 and parameter 1 are used to calculate Δinput data, which is the error with respect to the input data. Thus, backward processing requires intermediate data, which is the intermediate result of calculations performed by forward processing.

[0033] In the optimization process, the parameters of each intermediate layer and input layer are corrected using the Δ parameter (error gradient) calculated in the backward processing. In other words, the parameters are optimized. Parameter optimization is performed using gradient descent methods such as Momentum-SGD (Stochastic Gradient Descent) or ADAM.

[0034] In this backward processing, the error between the output data and the training data is calculated for the data input to the output layer (the output data of the intermediate layer immediately preceding the output layer). Then, using the calculated data error, the error of the intermediate data is calculated, and using the error of the intermediate data, the error of the parameters is calculated, starting from the output layer (backpropagation). In the parameter update process, the parameters are optimized based on the parameter errors obtained in the backward processing.

[0035] Figure 5 is an explanatory diagram showing an example of a decision table (an example of decision information) used in the forward processing of the first embodiment. For example, the decision tables TBL1(A), TBL1(B), TBL1(C), ... shown in Figure 5 may be allocated to a memory area within the processor 20 (SRAM 40 or registers within the arithmetic unit 30, etc.). Hereafter, when the decision tables TBL1(A), TBL1(B), TBL1(C), ... are described without distinction, they will simply be referred to as decision table TBL1. For example, a decision table TBL1 is provided for each neural network A, B, C, ...

[0036] Each decision table TBL1 in this embodiment has, for each processing target layer, an area for storing an input deletion bit (1 bit), which is an example of information indicating whether or not to delete data, and an area for storing a transfer decision bit (2 bits), which is an example of information indicating the transfer destination. The input deletion bit holds information indicating whether or not to delete the uncompressed target intermediate data input to the processing target layer from the SRAM 40 after the calculation processing of the processing target layer is executed. For example, an input deletion bit of "0" indicates that the uncompressed target intermediate data will not be deleted from the SRAM 40, and an input deletion bit of "1" indicates that the uncompressed target intermediate data will be deleted from the SRAM 40. The input deletion bit is an example of deletion information indicating whether or not to delete uncompressed intermediate data from the SRAM 40.

[0037] In this embodiment, if the input deletion bit is "0", the data processing device 100 continues to retain the uncompressed intermediate data, which is the result of calculations performed in other layers used for the calculation, from the SRAM 40 after the calculation of the target layer has been performed. If the input deletion bit is "1", the data processing device 100 deletes the uncompressed intermediate data, which is the result of calculations performed in other layers used for the calculation, from the SRAM 40 after the calculation of the target layer has been performed.

[0038] The data processing device 100 of this embodiment can reduce the memory capacity of the SRAM 40 built into the processor 20 by deleting uncompressed intermediate data from the SRAM 40 when it is no longer needed for subsequent calculations. Note that uncompressed intermediate data may be used in the calculations of multiple layers. In this case, only the input deletion bit corresponding to the most recently executed layer is set to "1". This prevents the accidental deletion of intermediate data from the SRAM 40, even when common intermediate data is used across multiple layers.

[0039] The transfer determination bit in this embodiment holds information indicating the transfer destination (storage location) of the intermediate data. A transfer determination bit of "00" indicates that the compressed intermediate data will be transferred to the SRAM 40. A transfer determination bit of "01" indicates that the compressed intermediate data will be transferred to the DRAM 50. A transfer determination bit of "10" indicates that the compressed intermediate data will be transferred to both the SRAM 40 and the DRAM 50. The information indicating the transfer destination (storage location) of the intermediate data held in the transfer determination bit is an example of storage location information.

[0040] By providing a transfer determination bit, the data processing device 100 can easily determine the transfer destination of the compressed intermediate data for each layer. Note that if the compressed intermediate data is to be transferred to only one of the SRAM 40 or DRAM 50, that is, if the compressed intermediate data is not transferred to both the SRAM 40 and DRAM 50, the transfer determination bit can be 1 bit. In this case, a "0" in the transfer determination bit indicates a transfer to the SRAM 40, and a "1" in the transfer determination bit indicates a transfer to the DRAM 50.

[0041] Each decision table TBL1 may have input deletion bits and transfer decision bits common to all processing layers. That is, input deletion bits and transfer decision bits may be set for each neural network. Furthermore, at least one of the decision tables TBL1 may hold multiple input deletion bits and multiple transfer decision bits corresponding to at least one of the processing layers. In this case, the multiple input deletion bits and multiple transfer decision bits are set corresponding to each of the multiple data or multiple data sets used in the corresponding processing layer.

[0042] Furthermore, multiple decision tables TBL1 may be provided, each corresponding to a different compression ratio. For example, in the forward processing of neural network A, if the compression ratio is sequentially increased after each predetermined number of iterations, a decision table TBL1(A) may be provided for each compression ratio, and the decision table TBL1(A) corresponding to the number of iterations may be referenced. Alternatively, a compression ratio table (an example of compression ratio information) showing the correspondence between multiple compression ratios and the number of iterations may be provided for each decision table TBL1.

[0043] Figure 6 is a flowchart illustrating an example of forward processing of a neural network by the data processing device 100 shown in Figure 1. In other words, Figure 6 shows an example of a data processing method by the data processing device 100. For example, the processing shown in Figure 6 is realized when the processor 20 of the data processing device 100 executes a data processing program. For example, Figure 6 shows an example of forward processing, which is one of the methods for generating a neural network model using machine learning.

[0044] First, in step S10, the processor 20 transfers input data, such as parameters used in the processing target layer that performs forward processing, to the SRAM 40. In the input layer shown in Figure 3, the processor 20 transfers input data 1 and parameter 1 as input data to the SRAM 40.

[0045] Next, in step S12, the processor 20 performs forward processing using the input data transferred in step S10 to generate intermediate data. Then, in step S14, the processor 20 stores the intermediate data (uncompressed) generated in step S12 in the SRAM 40.

[0046] Next, in step S16, the processor 20 refers to the input deletion bit in the decision table TBL1 and determines whether or not to delete the uncompressed intermediate data input to the processing layer from the SRAM 40. If the input deletion bit is "1", the processor 20 decides to delete the uncompressed intermediate data from the SRAM 40 and proceeds to step S18. If the input deletion bit is "0", the processor 20 decides not to delete the uncompressed intermediate data from the SRAM 40 and proceeds to step S20.

[0047] In step S18, the processor 20 removes the uncompressed intermediate data input to the processing layer from the SRAM 40 and proceeds to step S20. In step S20, the processor 20 compresses the intermediate data calculated by the forward processing of the processing layer and generates compressed data.

[0048] Next, in step S22, the processor 20 refers to the transfer decision bit of the decision table TBL1. If the transfer decision bit is "00", the processor 20 decides to transfer the intermediate data to the SRAM 40 and proceeds to step S24. If the transfer decision bit is "01", the processor 20 decides to transfer the intermediate data to the DRAM 50 and proceeds to step S28. If the transfer decision bit is "10", the processor 20 decides to transfer the intermediate data to both the SRAM 40 and the DRAM 50 and proceeds to step S26.

[0049] In step S24, the processor 20 transfers the intermediate data compressed in step S20 to the SRAM 40 and proceeds to step S30. In step S26, the processor 20 transfers the intermediate data compressed in step S20 to the SRAM 40 and proceeds to step S28.

[0050] In step S28, the processor 20 transfers the compressed intermediate data from step S20 to the DRAM 50 and proceeds to step S30. As a result, the compressed intermediate data can be transferred to at least one of the SRAM 40 and the DRAM 50 according to the value of the transfer determination bit.

[0051] In step S30, if there are any unprocessed layers, the processor 20 returns to step S10 and performs forward processing of the next layer to be processed. If there are no unprocessed layers, i.e., if the forward processing of the neural network is complete, the processor 20 terminates the operation shown in Figure 6.

[0052] In the embodiments described in Figures 1 to 6 above, the amount of memory used to hold the intermediate data generated by forward processing can be reduced. This makes it possible to secure sufficient memory bandwidth between the processor 20 and the DRAM 50 necessary to store the intermediate data generated by forward processing in the DRAM 50 without installing a faster DRAM 50 or widening the data bus width. In other words, the memory bandwidth required to store the intermediate data in the DRAM 50 can be reduced compared to the memory bandwidth required to store all the intermediate data in the DRAM 50. For example, in backward processing, memory usage can be further reduced by compressing the intermediate data before storing it in memory, and the memory bandwidth of the memory storing the intermediate data can be further reduced. This makes it possible to suppress an increase in the system cost of the data processing device 100 without reducing the efficiency and accuracy of backward processing.

[0053] Furthermore, the first and second processes described herein are not limited to forward and backward processing in the training of machine learning models.

[0054] The data processing device 100 reads out the uncompressed intermediate data stored in the SRAM 40 in the intermediate layer and output layer that execute the calculation process, and uses the read uncompressed intermediate data for the calculation process. By using the uncompressed intermediate data for forward processing, forward processing can be performed without reducing the calculation accuracy.

[0055] The data processing device 100 can reduce the memory capacity of the SRAM 40 built into the processor 20 by deleting uncompressed intermediate data from the SRAM 40 when it is no longer needed for subsequent calculations. By providing an input deletion bit for each layer being processed, it is possible to prevent the accidental deletion of intermediate data from the SRAM 40 even when common intermediate data is used by multiple layers.

[0056] By providing a transfer determination bit, the data processing device 100 can easily determine the transfer destination of the compressed intermediate data for each layer.

[0057] Because the intermediate data used in backward processing is irreversibly compressed, the compression cost can be reduced compared to lossless compression, and the compression ratio can be kept constant, thus reducing the load on the processor 20 due to the compression process.

[0058] By representing intermediate data in a floating-point data format and reducing the number of bits (digits) in the mantissa, the intermediate data can be compressed easily and irreversibly.

[0059] By setting the compression ratio of the intermediate data to a higher value as the computational processing of the layers progresses, the memory bandwidth required for transferring the intermediate data can be further reduced, thereby further suppressing the increase in the system cost of the data processing device 100.

[0060] Figure 7 is an explanatory diagram showing an example of a decision table used in the forward processing of a neural network by the data processing device of the second embodiment of the present invention. Detailed explanations of elements similar to those in Figure 5 are omitted.

[0061] The data processing device that references the decision tables TBL2(A), TBL2(B), TBL2(C), ... in Figure 7 has a configuration similar to the data processing device 100 shown in Figure 1. That is, the data processing device in this embodiment has a processor 20 including a plurality of arithmetic units 30 and a plurality of SRAMs 40, and at least one system board 10 including a DRAM 50.

[0062] In the following, when describing decision tables TBL2(A), TBL2(B), TBL2(C), ... without distinction, they will simply be referred to as decision table TBL2. For example, decision table TBL2 is provided for each neural network A, B, C, ..., just like decision table TBL1.

[0063] The decision table TBL2 adds a region for storing compression decision bits to the decision table TBL1 in Figure 5. That is, each decision table TBL2 has a region for storing an input deletion bit (1 bit), a compression decision bit (1 bit), and a transfer decision bit (2 bits) for each processing layer.

[0064] A compression determination bit of "0" indicates that intermediate data will be compressed, while a compression determination bit of "1" indicates that intermediate data will not be compressed. In other words, in this embodiment, it is possible to switch whether or not to compress intermediate data for each processing layer. For example, if the size of the generated intermediate data is large, the compression determination bit is set to "0", and if the size of the generated intermediate data is small, the compression determination bit is set to "1". This makes it possible to suppress the increase in memory bandwidth when the size of the intermediate data is large, and to suppress the increase in the system cost of the data processing device. On the other hand, when the size of the intermediate data is small, the compression cost can be reduced.

[0065] The meaning of each value of the transfer decision bit when the compression decision bit is "0" is the same as the meaning of each value of the transfer decision bit in the decision table TBL1 in Figure 5. That is, a transfer decision bit of "00" indicates that the compressed intermediate data will be transferred to SRAM40. A transfer decision bit of "01" indicates that the compressed intermediate data will be transferred to DRAM50. A transfer decision bit of "10" indicates that the compressed intermediate data will be transferred to both SRAM40 and DRAM50.

[0066] On the other hand, when the compression determination bit is "0", the meaning of each value of the transfer determination bit is as follows: A transfer determination bit of "00" indicates that uncompressed intermediate data will be transferred to DRAM50. A transfer determination bit of "01" indicates that uncompressed intermediate data will not be transferred to DRAM50.

[0067] Uncompressed intermediate data is always transferred to SRAM40. Therefore, if the compression detection bit is "1" and the transfer detection bit is "00", uncompressed intermediate data will be transferred to both SRAM40 and DRAM50. If the compression detection bit is "1" and the transfer detection bit is "01", uncompressed intermediate data will be transferred only to SRAM40.

[0068] In this embodiment, the meaning of the transfer determination bit changes depending on whether or not the intermediate data is compressed using the compression determination bit. That is, the transfer determination bit can be used in the same case whether or not the intermediate data is compressed, thereby suppressing the increase in the size of the determination table TBL2.

[0069] Furthermore, similar to the embodiments described above, each decision table TBL2 may have input deletion bits, compression decision bits, and transfer decision bits common to all processing layers. That is, input deletion bits, compression decision bits, and transfer decision bits may be set for each neural network. In addition, at least one of the decision tables TBL2 may hold multiple input deletion bits, multiple compression decision bits, and multiple transfer decision bits corresponding to at least one of the processing layers. In this case, the multiple input deletion bits, multiple compression decision bits, and multiple transfer decision bits are set corresponding to each of the multiple data or multiple data sets used in the corresponding processing layer.

[0070] Furthermore, multiple decision tables TBL2 may be provided, each corresponding to a different compression ratio. For example, in the forward processing of neural network A, if the compression ratio is sequentially increased every predetermined number of iterations, a decision table TBL2(A) may be provided for each compression ratio, and the decision table TBL2(A) corresponding to the number of iterations may be referenced. Alternatively, a compression ratio table showing the correspondence between multiple compression ratios and the number of iterations may be provided for each decision table TBL2.

[0071] Furthermore, if the compressed intermediate data is to be transferred to only one of the SRAM 40 or DRAM 50, that is, if the compressed intermediate data is not transferred to both SRAM 40 and DRAM 50, the transfer determination bit may be 1 bit. In this case, when the compression determination bit is "0", a transfer determination bit of "0" indicates a transfer to SRAM 40, and a transfer determination bit of "1" indicates a transfer to DRAM 50. When the compression determination bit is "1", a transfer determination bit of "0" indicates a transfer to DRAM 50, and a transfer determination bit of "1" indicates no transfer to DRAM 50.

[0072] Figures 8 and 9 are flowcharts illustrating an example of forward processing of a neural network by a data processing device of the second embodiment. That is, Figures 8 and 9 show an example of a data processing method by the data processing device. For example, the processing shown in Figures 8 and 9 is achieved by the processor 20 of the data processing device executing a data processing program. For example, Figures 8 and 9 show an example of forward processing in a method for generating a neural network model using machine learning. For processing similar to that in Figure 6, the same step numbers are used, and detailed explanations are omitted.

[0073] The process from step S10 to step S18 is the same as the process from step S10 to step S18 in Figure 6. However, after steps S16 and S18, the process proceeds to step S19 instead of step S20.

[0074] In step S19, the processor 20 refers to the compression decision bit in the decision table TBL2 and determines whether or not to compress the intermediate data generated by the forward processing of the layer to be processed. If the processor 20 decides to compress the intermediate data, it moves the process to step S20; if it decides not to compress the intermediate data, it moves the process to step S21.

[0075] Here, the decision of whether or not to compress the data may be determined according to the hardware configuration of the data processing device 100 and the configuration of the neural network. For example, the hardware configuration may be indicated by the storage capacity and memory bandwidth of the SRAM 40, the storage capacity and memory bandwidth of the DRAM 50, and the processing performance of the arithmetic unit 30. For example, the configuration of the neural network may be indicated by the computational procedure of the neural network, or by a computational graph showing the neural network.

[0076] The processing in step S20 is the same as the processing in step S20 in Figure 6. As with the embodiments described above, the data processing device 100 may compress the generated intermediate data by converting it from double-precision floating-point data to single-precision floating-point data. The data processing device 100 may also compress the intermediate data by converting it from single-precision floating-point data to 8-bit fixed-point data. Furthermore, the data processing device 100 may compress the intermediate data by reducing the number of bits (digits) in the mantissa of the floating-point data.

[0077] Furthermore, the compression ratio of the intermediate data may be set higher as the neural network training progresses. The data processing device 100 may compress multiple intermediate data sets together, rather than compressing them one by one.

[0078] After step S20, the process moves to step S22 in Figure 9. In step S21, the processor 20 refers to the transfer decision bit of the decision table TBL2. If the transfer decision bit is "00", the processor 20 moves the process to step S28 in Figure 9, and if the transfer decision bit is "01", the process moves to step S30 in Figure 9.

[0079] The processing from steps S22 to S30 in Figure 9 is the same as the processing from steps S22 to S30 in Figure 6. However, the processing in step S28 is also executed if it is determined by step 21 in Figure 8 that intermediate data should be transferred to the DRAM 50. The processing in step SS30 is also executed if it is determined by step 21 in Figure 8 that intermediate data should not be transferred to the DRAM 50.

[0080] As described above, the embodiments shown in Figures 7 to 9 can also obtain the same effects as those of the embodiments described above. For example, by compressing the intermediate data and then storing it in memory, the amount of memory used can be reduced, and the transfer time of the intermediate data can be shortened. This makes it possible to reduce the memory bandwidth required for the transfer of intermediate data, and thus suppress the increase in the system cost of the data processing device 100.

[0081] Furthermore, in this embodiment, by providing a compression determination bit in the determination table TBL2, the data processing device 100 can switch whether or not to compress intermediate data for each processing layer. This makes it possible to suppress the increase in memory bandwidth when the size of intermediate data is large, and thus suppress the increase in the system cost of the data processing device. On the other hand, when the size of intermediate data is small, the compression cost can be reduced.

[0082] By changing the meaning of the transfer decision bit depending on whether or not the intermediate data is compressed using the compression decision bit, the transfer decision bit can be used interchangeably whether or not the intermediate data is compressed, thereby suppressing the increase in the size of the decision table TBL2.

[0083] In the embodiments described above, some or all of the data processing devices may be composed of hardware, or they may be composed of information processing by software (programs) executed by a CPU (Central Processing Unit) or GPU (Graphics Processing Unit), etc. If the information processing is composed of software, the software that realizes at least some of the functions of each device in the embodiments described above may be stored on a non-temporary storage medium (non-temporary computer-readable medium) such as a flexible disk, CD-ROM (Compact Disc-Read Only Memory), or USB (Universal Serial Bus) memory, and the software information processing may be executed by having a computer read it. Alternatively, the software may be downloaded via a communication network. Furthermore, the information processing may be executed by hardware by implementing the software on a circuit such as an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array).

[0084] The type of storage medium used to store data processing programs and other software is not limited. The storage medium is not limited to removable media such as magnetic disks or optical disks; it may also be a fixed storage medium such as a hard disk or memory. Furthermore, the storage medium may be located inside or outside the computer.

[0085] Figure 10 is a block diagram showing an example of the hardware configuration of the data processing device of the embodiment described above. The hardware configuration of the data processing device 100 in Figure 1 will be described below. The data processing device 100 may be implemented as a computer comprising, for example, a processor 20, a main memory 50 (e.g., DRAM 50), an auxiliary memory 60 (memory), a network interface 70, and a device interface 80, all connected via a bus 90. For example, the processor 20 executes a data processing program, thereby performing the operations described in Figure 6 or Figures 8 to 9.

[0086] The data processing device 100 has one of each component, but it may have multiple identical components. Also, although Figure 10 shows one data processing device 100, the software may be installed on multiple data processing devices 100, and each of the multiple data processing devices 100 may execute the same or different parts of the software. In this case, it may be a distributed computing configuration in which each data processing device 100 communicates via a network interface 70 or the like to execute processing. In other words, the data processing device 100 in the above-described embodiment may be configured as a computer system that realizes its function by having one or more data processing devices 100 execute instructions stored in one or more storage devices. Alternatively, it may be configured so that information transmitted from a terminal is processed by one or more data processing devices 100 located on the cloud, and the processing results are transmitted to the terminal.

[0087] The operations described in the flow chart of Figure 6 and the operations described in the flow charts of Figures 8 and 9 may be executed in parallel using one or more processors 20, or using multiple computers via a network. Furthermore, various calculations may be distributed to multiple arithmetic cores within the processor 20 and executed in parallel. Also, some or all of the processing and means of this disclosure may be executed by at least one of a processor and a storage device located on a cloud that can communicate with the data processing device 100 via a network. Thus, the data processing device 100 in the above-described embodiments may be in the form of parallel computing using one or more computers.

[0088] The processor 20 may be an electronic circuit (processing circuit, processing circuitry, CPU, GPU, FPGA, or ASIC, etc.) including a computer control unit and arithmetic unit. Alternatively, the processor 20 may be a semiconductor device including a dedicated processing circuit. The processor 20 is not limited to an electronic circuit using electronic logic elements, but may also be implemented using an optical circuit with optical logic elements. Furthermore, the processor 20 may include computational functions based on quantum computing.

[0089] The processor 20 may perform calculations based on data and software (programs) input from each of the internal components of the data processing device 100, and may output calculation results and control signals to each of the internal components. The processor 20 may also control each component of the data processing device 100 by executing the OS (Operating System) or applications of the data processing device 100.

[0090] The data processing device 100 in the above-described embodiment may be implemented by one or more processors 20. Here, the processor 20 may refer to one or more electronic circuits arranged on one chip, or one or more electronic circuits arranged on two or more chips or two or more devices. When multiple electronic circuits are used, each electronic circuit may communicate by wire or wireless.

[0091] The main memory 50 (for example, the DRAM 50 in Figure 1) may store instructions executed by the processor 20 and various data, and the information stored in the main memory 50 may be read by the processor 20. The auxiliary memory 60 is a memory device other than the main memory 50. These memory devices refer to any electronic component capable of storing electronic information, and may be semiconductor memory. The semiconductor memory may be either volatile memory or non-volatile memory. In the data processing device 100 in the above-described embodiment, the memory device for storing various data may be implemented by the main memory 50 or the auxiliary memory 60, or it may be implemented by an internal memory such as SRAM 40 built into the processor 20.

[0092] In the above-described embodiment, if the data processing device 100 consists of at least one storage device (memory) and a plurality of processors 20 connected to (coupled with) this at least one storage device (memory), then a plurality of processors 20 may be connected to (coupled with) one storage device (memory), or a single processor 20 may be connected to it. Furthermore, a single processor 20 may be connected to (coupled with) a plurality of storage devices (memories), or a single storage device (memory) may be connected to it. In addition, the configuration may include one in which at least one of the plurality of processors 20 is connected to (coupled with) at least one storage device (memory). Furthermore, this configuration may be realized by the storage devices (memories) and processors 20 included in a plurality of data processing devices 100. Moreover, the configuration may include one in which the storage device (memory) is integrated with the processor 20 (for example, a cache memory including an L1 cache and an L2 cache).

[0093] The network interface 70 is an interface for connecting to the communication network 200 wirelessly or via a wired connection. The network interface 70 can be any appropriate interface, such as one conforming to existing communication standards. Information may be exchanged between the data processing device 100 and an external device 210 connected via the communication network 200 through the network interface 70. The communication network 200 may be a WAN (Wide Area Network), LAN (Local Area Network), PAN (Personal Area Network), or a combination thereof, as long as information is exchanged between the data processing device 100 and the external device 210. An example of a WAN is the Internet; an example of a LAN is IEEE 802.11 or Ethernet®; and an example of a PAN is Bluetooth® or NFC (Near Field Communication).

[0094] The device interface 80 is an interface such as USB that connects directly to the external device 220.

[0095] The external device 220 may be connected to the data processing device 100 via a network, or it may be connected directly to the data processing device 100.

[0096] External device 210 or external device 220 may, for example, be an input device. The input device may be a camera, microphone, motion capture device, various sensors, keyboard, mouse, or touch panel, and will provide the acquired information to the data processing device 100. Alternatively, it may be a device equipped with an input unit, memory, and processor, such as a personal computer, tablet terminal, or smartphone.

[0097] Furthermore, the external device 210 or external device 220 may, for example, be an output device. The output device may be a display device such as an LCD (Liquid Crystal Display), CRT (Cathode Ray Tube), PDP (Plasma Display Panel), or organic EL (Electro Luminescence) panel, or it may be a speaker that outputs sound, etc. It may also be a device equipped with an output unit, memory, and processor, such as a personal computer, tablet terminal, or smartphone.

[0098] Furthermore, the external device 210 or external device 220 may be a storage device (memory). For example, the external device 210 may be network storage, and the external device 220 may be storage such as an HDD. The external device 220, which is a storage device (memory), is an example of a recording medium that can be read by a computer such as the processor 20.

[0099] Furthermore, the external device 210 or external device 220 may be a device having some of the functions of the components of the data processing device 100 in the embodiment described above. In other words, the data processing device 100 may transmit or receive some or all of the processing results of the external device 210 or external device 220.

[0100] Where the expression "at least one of a, b, and c" or "at least one of a, b, or c" (including similar expressions) is used herein, it includes any of a, b, c, ab, ac, bc, or abc. It also includes multiple instances of any one element, such as aa, abb, aabbcc, etc. Furthermore, it includes adding other elements other than the enumerated elements (a, b, and c), such as abcd having d.

[0101] In this specification (including the claims), when expressions such as "using data as input / based on data / according to / in accordance with data" (including similar expressions) are used, unless otherwise specified, this includes cases where the various data themselves are used as input, or where the various data have been processed in some way (e.g., data with added noise, normalized data, features extracted from the data, intermediate representations of the various data, etc.) are used as input. Furthermore, when it is stated that some result is obtained "using data as input / based on data / according to / in accordance with data" (including similar expressions), this includes cases where the result is obtained based solely on the data in question, as well as cases where the result is also influenced by other data, factors, conditions, and / or states other than the data in question. Furthermore, when it is stated that "data is output" (including similar expressions), unless otherwise specified, this includes cases where the various data themselves are used as output, or where the various data have been processed in some way (e.g., data with added noise, normalized data, features extracted from the data, intermediate representations of the various data, etc.) are used as output.

[0102] In this specification (including the claims), the terms “connected” and “coupled” are intended to be non-restrictive terms that include any direct connection / coupling, indirect connection / coupling, electrical connection / coupling, communicative connection / coupling, operational connection / coupling, physical connection / coupling, etc. The terms should be interpreted as appropriate in the context in which they are used, but any form of connection / coupling that is not intentionally or naturally excluded should be interpreted non-restrictively as being included in the terms.

[0103] In this specification (including the claims), when the expression "A configured to B" is used, it may include that the physical structure of element A has a configuration capable of performing operation B, and that the permanent or temporary setting / configuration of element A is configured to actually perform operation B. For example, if element A is a general-purpose processor, it is sufficient that the processor has a hardware configuration capable of performing operation B, and that it is configured to actually perform operation B by the setting of a permanent or temporary program (instruction). Furthermore, if element A is a dedicated processor or dedicated arithmetic circuit, it is sufficient that the circuit structure of the processor is implemented to actually perform operation B, regardless of whether control instructions and data are actually attached.

[0104] Wherever terms meaning "comprising" or "possessing" (e.g., "comprising / including" and "having") are used in this specification (including the claims), they are intended to be open-ended terms, including cases where the subject matter of such terms is not the object of the term. Where the object of such terms meaning "comprising" or "possessing" is an expression that does not specify a quantity or suggests a singular number (an expression with the article "a" or "an"), such expression should be interpreted as not being limited to a specific number.

[0105] In this specification (including the claims), even if expressions such as "one or more" or "at least one" are used in one place, and expressions that do not specify a quantity or suggest a singularity (expressions using the articles a or an) are used in another place, the latter expressions are not intended to mean "one." In general, expressions that do not specify a quantity or suggest a singularity (expressions using the articles a or an) should not necessarily be interpreted as not being limited to a specific number.

[0106] In this specification, if a particular configuration of an embodiment is described as yielding a specific advantage or result, it should be understood that, unless otherwise stated, the same advantage or result can also be obtained from one or more other embodiments having the same configuration. However, it should be understood that the presence or absence of such an advantage or result generally depends on various factors, conditions, and / or states, and that the configuration does not necessarily guarantee that the advantage or result can be obtained. The advantage or result can only be obtained from the configuration described in the embodiment when various factors, conditions, and / or states are met, and the advantage or result cannot necessarily be obtained in the claimed invention that defines the configuration or a similar configuration.

[0107] In this specification (including the claims), when terms such as "maximize" are used, they include finding the global maximum value, finding an approximation of the global maximum value, finding the local maximum value, and finding an approximation of the local maximum value, and should be interpreted appropriately depending on the context in which the term is used. They also include finding approximations of these maximum values ​​probabilistically or heuristically. Similarly, when terms such as "minimize" are used, they include finding the global minimum value, finding an approximation of the global minimum value, finding the local minimum value, and finding an approximation of the local minimum value, and should be interpreted appropriately depending on the context in which the term is used. They also include finding approximations of these minimum values ​​probabilistically or heuristically. Similarly, when terms such as "optimize" are used, they include finding the global optimal value, finding an approximation of the global optimal value, finding the local optimal value, and finding an approximation of the local optimal value, and should be interpreted appropriately depending on the context in which the term is used. They also include finding approximations of these optimal values ​​probabilistically or heuristically.

[0108] In this specification (including the claims), when multiple hardware components perform a predetermined process, each component may cooperate to perform the predetermined process, or some components may perform all of the predetermined process. Alternatively, some components may perform part of the predetermined process, while other components perform the remainder. In this specification (including the claims), when expressions such as "one or more hardware components perform a first calculation process, and the one or more hardware components perform a second calculation process" are used, the hardware component performing the first calculation process and the hardware component performing the second calculation process may be the same or different. In other words, it is sufficient that the hardware component performing the first calculation process and the hardware component performing the second calculation process are included in the one or more hardware components. Hardware may include electronic circuits or devices containing electronic circuits.

[0109] In this specification (including the claims), when multiple memory devices store data, each of the multiple memory devices may store only a portion of the data or the entire data. Furthermore, a configuration in which some of the multiple memory devices store data is also included.

[0110] While embodiments of this disclosure have been described in detail above, this disclosure is not limited to the individual embodiments described above. Various additions, modifications, substitutions, and partial deletions are possible, provided that they do not depart from the conceptual idea and spirit of the present invention derived from the claims and their equivalents. For example, where numerical values ​​or mathematical formulas are used in the description in all of the embodiments described above, they are provided as examples only and are not limited thereto. Also, the order of operations in the embodiments is provided as examples only and is not limited thereto. [Explanation of Symbols]

[0111] 10 System Boards 20 processors 30 Arithmetic unit 40 SRAM 50 DRAM (main memory) 60 Auxiliary storage 70 Network Interfaces 80 device interfaces 90 bus 100 Data Processing Devices 200 Communication Networks 210 External device 220 External device

Claims

1. A neural network comprising at least a first layer and a second layer, wherein a forward process is performed by at least one processor to generate first intermediate data from the first layer and second intermediate data from the second layer, The at least one processor generates compressed data by compressing the first intermediate data, The at least one processor performs backward processing of the neural network based on the compressed data and the second intermediate data. The at least one processor updates the parameters of the neural network after the backward processing has been performed. The compressed data is generated by converting the first intermediate data, which is represented in a first numeric format, into a second numeric format having fewer bits than the first numeric format. The second numeric format is a floating-point number data format, The aforementioned update is performed by using ADAM. The second intermediate data is used in the backward processing without irreversible compression. Data processing method.

2. The compression of the first intermediate data is performed according to the compression settings defined before the execution of the forward process. The data processing method according to claim 1.

3. The at least one processor determines whether to store the generated compressed data in at least one of the first memory or the second memory, The first memory has a different bandwidth than the second memory. The data processing method according to claim 1 or claim 2.

4. The compressed data is stored in both the first memory and the second memory. The data processing method according to claim 3.

5. The first intermediate data and the second intermediate data are stored in the first memory, The compressed data is stored in the second memory, The first memory has a different bandwidth than the second memory. The data processing method according to claim 1 or claim 2.

6. The at least one processor determines whether to delete the second intermediate data, The data processing method according to any one of claims 1 to 5.

7. The at least one processor determines whether to compress the first intermediate data and not compress the second intermediate data. The data processing method according to any one of claims 1 to 6.

8. The compression decision includes determining by at least one processor whether each of the first intermediate data and the second intermediate data is compressed based on decision bits assigned to each of the first and second layers. The data processing method according to claim 7.

9. The compressed data is generated by applying lossy compression to the first intermediate data using at least one processor. The data processing method according to any one of claims 1 to 8.

10. The first numeric format is a floating-point number data format, The compressed data is generated by deleting the number of bits in the mantissa of the first intermediate data. The data processing method according to any one of claims 1 to 9.

11. The data processing method is performed in multiple iterations, The compression rate of the compression changes in at least one of the plurality of iterations. The data processing method according to any one of claims 1 to 10.

12. Performing the backward processing based on the compressed data includes performing the backward processing using data generated from the compressed data. The data processing method according to any one of claims 1 to 11.

13. At least one processor performs forward processing of a neural network to generate intermediate data, The aforementioned at least one processor generates compressed data by compressing the intermediate data, The at least one processor performs backward processing of the neural network based on the compressed data, After the backward processing is performed, the at least one processor performs the parameter update process of the neural network in multiple iterations. The compression rate used for the compression changes in at least one of the multiple iterations. Data processing method.

14. The compressed data is generated by applying lossy compression to the intermediate data using at least one processor. The data processing method according to claim 13.

15. The plurality of iterations, each including the forward processing and the backward processing, are performed while gradually increasing the compression rate of the intermediate data. The data processing method according to claim 13 or claim 14.

16. The data processing method described in any one of Claims 1 to 15 is performed. Data processing device.

17. Cause a computer to execute the data processing method described in any one of Claims 1 to 15. Data processing program.

Images