Data processing method, data processing device and data processing program

Abstract

To provide a method for data processing for conducting second calculation processing by using data in the middle of calculation in first calculation processing, the method being capable of reducing a memory bandwidth by reducing the usage amount of a memory holding the data in the middle of calculation in the first calculation processing.SOLUTION: A method for data processing regarding a machine learning model includes: generating compression data by compressing data being calculated in first calculation processing; storing the generated compression data into a memory region; and conducting second calculation processing by using the compression data stored in the memory region. The data being calculated is floating-point number data and reducing the number of bits in the mantissa part of the data being calculated leads to generation of the compression data.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present disclosure relates to a data processing method.

Background Art

[0002] Generally, regarding machine learning, in data processing such as model training, intermediate data that is an intermediate result of the calculation of forward processing may be stored in an external memory such as DRAM (Dynamic Random Access Memory) for backward processing. And, among the intermediate data stored in the external memory, the intermediate data necessary for the calculation of backward processing may be read from the external memory and the calculation of backward processing may be executed.

Summary of the Invention

Problems to be Solved by the Invention

[0003] It may be difficult to sufficiently secure the memory bandwidth between the processor and the external memory necessary for storing the intermediate data generated for such machine learning in the external memory every time.

Means for Solving the Problems

[0004] The data processing method according to an embodiment of the present invention is a data processing method related to a machine learning model, which compresses data during the calculation of a first calculation process to generate compressed data, stores the generated compressed data in a memory area, uses the compressed data stored in the memory area to execute a second calculation process, the data during the calculation is floating-point data, and the compressed data is generated by reducing the number of bits of the mantissa part of the data during the calculation.

Brief Description of the Drawings

[0005]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0006] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0007] FIG. 1 is a block diagram showing an example of a data processing device according to the first embodiment of the present invention. The data processing device 100 shown in FIG. 1 includes at least one system board 10 including a processor 20 and a plurality of DRAMs (Dynamic Random Access Memories) 50 connected to the processor 20. For example, the data processing device 100 may be a server.

[0008] The processor 20 has a plurality of arithmetic units 30 and a plurality of SRAMs (Static Random Access Memories) 40 respectively connected to the plurality of arithmetic units 30. The processor 20 is connected to the system bus. The processor 20 may be in the form of a chip or in the form of a package. The arithmetic unit 30 is an example of an arithmetic processing unit.

[0009] In this embodiment, the memory bandwidth of the SRAM 40 is larger than the memory bandwidth of the DRAM 50. Therefore, when the data used by the processor 20 for calculation can be stored in the SRAM 40, it is preferably stored in the SRAM 40. However, when the SRAM 40 is built into the processor 20, it may be difficult to store all the data used by the processor 20 in the SRAM 40. In this case, the data that cannot be stored in the SRAM 40 may be stored in the DRAM 50 with a small memory bandwidth.

[0010] Note that the internal memory connected to the arithmetic unit 30 is not limited to the SRAM 40, and may be, for example, a cache memory. The external memory connected to the processor 20 is not limited to the DRAM 50, and may be, for example, an MRAM (Magnetoresistive Random Access Memory), an HDD (Hard Disk Drive), or an SSD (Solid State Drive), etc. The SRAM 40 is an example of the first memory, and the memory area allocated to the SRAM 40 is an example of the first memory area. The DRAM 50 is an example of the second memory, and the memory area allocated to the DRAM 50 is an example of the second memory area.

[0011] Thus, the data processing apparatus 100 of this embodiment has a plurality of types of memories with different memory bandwidths (in this embodiment, the SRAM 40 and the DRAM 50).

[0012] Note that when an SRAM 40 with a sufficient memory capacity can be mounted on the processor 20 or the system board 10, the first memory area and the second memory area may be allocated to the SRAM 40.

[0013] The data processing device 100 performs a plurality of computational processes to execute the training of a neural network having a plurality of layers. One of the computational processes is, for example, the forward process of the neural network, and another one of the computational processes is the backward process of the neural network. Also, the computational processes executed by the data processing device 100 are not limited to the training of the neural network. For example, the data processing device 100 may execute computational processes such as scientific and technological calculations.

[0014] FIG. 2 is an explanatory diagram showing an example of the training of the neural network executed by the data processing device 100 shown in FIG. 1. For example, FIG. 2 shows an example of a method for generating a neural network model by machine learning. In the machine learning for training a neural network having a plurality of intermediate layers between the input layer and the output layer in the present embodiment, the forward process, the backward process, and the optimization process are repeatedly executed a plurality of times while changing the training data. Then, the data processing device 100 generates a neural network model based on the training. The forward process, the backward process, and the optimization process will be described with reference to FIGS. 3 and 4. The forward process is an example of the first computational process, and the backward process is an example of the second computational process. The forward process, the backward process, and the optimization process are examples of the third computational process. Note that in this specification, the generation of a model or the generation of a neural network includes adjusting the model or neural network parameters.

[0015] FIG. 3 is an explanatory diagram showing an example of forward processing in the training of the neural network of the present embodiment. For example, FIG. 3 shows an example of forward processing in a method for generating a neural network model by machine learning. In forward processing, data and parameters such as weights are input to each of the input layer and a predetermined number of intermediate layers. In the input layer, the input data and parameter 1 are calculated to generate intermediate data 1. In the intermediate layer next to the input layer, the intermediate data 1 and parameter 2 are calculated to generate intermediate data 2.

[0016] In subsequent intermediate layers as well, the intermediate data generated by the previous intermediate layer and the parameters set for each intermediate layer are calculated, and the intermediate data generated by the calculation is output to the next intermediate layer. Note that there may be an intermediate layer that does not use parameters. Examples of intermediate layers include a convolutional layer, a pooling layer, a fully connected layer, and the like.

[0017] In the present embodiment, the intermediate data generated by the calculation processing of the input layer and the intermediate layer is stored in the SRAM 40 without being compressed. Then, the intermediate layer and the output layer that execute the calculation processing read the uncompressed intermediate data from the SRAM 40 and use it for the calculation processing. By using the intermediate data of the uncompressed data in the forward processing, the data processing apparatus 100 can execute the forward processing without degrading the calculation accuracy. The intermediate data is an example of the data in the middle of the calculation generated by each layer by the calculation processing.

[0018] This intermediate data is also used in the backward processing described in FIG. 4. In the present embodiment, the intermediate data used in the backward processing is stored in the SRAM 40, the DRAM 50, or both the SRAM 40 and the DRAM 50 after being compressed. In the backward processing, by compressing the intermediate data and then holding it in the memory, the amount of memory used can be reduced. Thereby, without mounting a faster DRAM 50 and without increasing the data bus width, it is possible to sufficiently secure the memory bandwidth between the processor 20 and the DRAM 50 necessary for storing the intermediate data generated by the forward processing in the DRAM 50. In other words, the memory bandwidth required for storing the intermediate data in the DRAM 50 can be reduced compared to the memory bandwidth required for storing all the intermediate data in the DRAM 50. Thereby, an increase in the system cost of the data processing apparatus 100 can be suppressed.

[0019] For example, the intermediate data used in the backward processing may be irreversibly compressed. Irreversible compression has a smaller compression cost compared to reversible compression and may be able to keep the compression rate constant, so the load on the processor 20 due to the compression process can be reduced.

[0020] Also, in the calculation of the backward processing that uses the intermediate data obtained by the forward processing, the error of the intermediate data often only has a local influence and does not propagate and accumulate over a wide range. For example, in the backward processing of the convolutional layer, the intermediate data generated by the forward processing only affects the gradient of the weights of that convolutional layer.

[0021] Also, the value of the gradient calculated by the backward processing may not require as high an accuracy as the forward processing. For example, in the update of the weights by stochastic gradient descent, since the value of the gradient is expected to be smaller than the value of the weights, even when the relative error of the gradient is large, the influence on the calculation of the backward processing can be reduced. Therefore, even when the backward processing is executed using the compressed intermediate data, appropriate weights can be calculated.

[0022] The data processing device 100 can perform the above operations, for example, by using the conversion of floating-point data. Specifically, the calculation process in the forward process can be executed using double-precision floating-point data, and the generated intermediate data can be compressed by converting it from double-precision floating-point data to single-precision floating-point data. Further, the data processing device 100 may compress the intermediate data by converting it from single-precision floating-point data to 8-bit fixed-point decimal data. Thereby, the intermediate data can be easily and irreversibly compressed using an existing conversion method. Furthermore, the data processing device 100 may compress the intermediate data by reducing the number of bits (number of digits) of the mantissa part of the floating-point data.

[0023] Note that the compression rate of the intermediate data may be set higher as the training of the neural network progresses. That is, the training of the neural network shown in FIG. 2 may be repeatedly executed while gradually increasing the compression rate of the intermediate data. For example, the data processing device 100 may calculate the first predetermined number of iterations using single-precision floating-point data, and calculate the next predetermined number of iterations using half-precision floating-point data. The data processing device 100 may further calculate the next 100 iterations using 8-bit fixed-point decimal data. The predetermined number of iterations is, for example, 100 iterations or the like.

[0024] Also, when using floating-point data in the forward process, the data processing device 100 may gradually increase the compression rate of the intermediate data by sequentially reducing the number of bits of the mantissa part every time a predetermined number of iterations is executed. By gradually increasing the compression rate of the intermediate data in this way, the memory bandwidth for transferring the intermediate data can be further suppressed, and an increase in the system cost of the data processing device 100 can be further suppressed.

[0025] Note that the data processing device 100 may compress a plurality of intermediate data together instead of compressing the intermediate data one by one. In this case, the compression rate of the intermediate data may be further increased, which can contribute to suppressing the memory bandwidth and the system cost.

[0026] In the output layer, the output data is calculated using the intermediate data N generated by the intermediate layer N (the Nth layer) immediately before the output layer. In the output layer for calculating the error in the classification problem, for example, the output data (solution) is calculated by using the softmax function as the activation function and the cross entropy as the error function. In the output layer, as described with reference to FIG. 4, the error (loss function) from the correct answer is calculated by comparing the output data with the teacher data (correct data).

[0027] As described above, in the forward process, in each layer of the neural network, the input data and the parameters are calculated, and the data (intermediate data) input to the next layer is calculated, and the output data is output from the final layer (forward propagation). Note that the forward process may be used not only for training the neural network but also for inference using the neural network. The forward process can be represented by a computational graph such as a DAG (Directed Acyclic Graph).

[0028] FIG. 4 is an explanatory diagram showing an example of the backward process and the optimization process in the training of the neural network according to the present embodiment. For example, FIG. 4 shows an example of the backward process in the method for generating a neural network model by machine learning. In the backward process, error backpropagation is performed in which the error is propagated in the reverse order of the forward process. In FIG. 4, the symbol Δ indicates the error of the data or the error of the parameter. The parameter update process performed in the optimization process is indicated by a dashed arrow.

[0029] First, in backward processing, in the layer that calculates the error (output layer), the output data generated in forward processing is compared with the teacher data, and Δ intermediate data N, which is the error with respect to the intermediate data N input to the output layer, is generated. Δ intermediate data N is also the error of the output data output by the Nth intermediate layer.

[0030] Next, in each intermediate layer, starting from the intermediate layer closer to the output layer in order, the error with respect to the output data (Δ intermediate data) and the intermediate data, which is the input data, are calculated, and Δ parameter, which is the error with respect to the parameters of the intermediate layer, is generated. Δ parameter indicates the gradient of the parameter in the curve showing the change in error with respect to the change in the parameter. For example, in the intermediate layer adjacent to the input layer, Δ intermediate data 2 and intermediate data 1 are calculated to obtain Δ parameter 2.

[0031] Also, in each intermediate layer, the error with respect to the output data (Δ intermediate data) and the parameters of the intermediate layer are calculated, and Δ intermediate data, which is the error with respect to the input data of the intermediate layer, is generated. The error (Δ intermediate data) with respect to the input data of the intermediate layer is also the error of the output data of the previous intermediate layer (or input layer). For example, in the intermediate layer adjacent to the input layer, Δ intermediate data 2 and parameter 2 are calculated to obtain Δ intermediate data 1. Here, the intermediate data is read from, for example, SRAM 40 or DRAM 50 for each layer.

[0032] In the input layer as well as in the intermediate layer, Δ intermediate data 1 and the input data are calculated to obtain Δ parameter 1, and Δ intermediate data 1 and parameter 1 are calculated to obtain Δ input data, which is the error with respect to the input data. Thus, in backward processing, the intermediate data, which is the intermediate result of the calculation by forward processing, is required.

[0033] In the optimization process, for each intermediate layer and the input layer, the parameters are corrected using the Δ parameters (error gradients) calculated in the backward process. That is, the parameters are optimized. The optimization of the parameters is performed using a gradient descent method such as Momentum-SGD (Stochastic Gradient Descent) or ADAM.

[0034] In this way, in the backward process, the error of the data input to the output layer (the output data of the intermediate layer immediately before the output layer) from the output data and the teacher data is calculated. Then, the process of calculating the error of the intermediate data using the calculated error of the data and the process of calculating the error of the parameters using the error of the intermediate data are performed in order from the output-side layer (error backpropagation). In the parameter update process, the parameters are optimized based on the error of the parameters obtained in the backward process.

[0035] FIG. 5 is an explanatory diagram showing an example of a determination table (an example of determination information) used in the forward process of the first embodiment. For example, the determination tables TBL1(A), TBL1(B), TBL1(C),... shown in FIG. 5 may be allocated to a storage area (such as SRAM40 or a register in the arithmetic unit 30) in the processor 20. Hereinafter, when explaining the determination tables TBL1(A), TBL1(B), TBL1(C),... without distinction, they are simply referred to as the determination table TBL1. For example, the determination table TBL1 is provided for each of the neural networks A, B, C,....

[0036] Each determination table TBL1 of the present embodiment has, for each processing target layer, an area for storing an input deletion bit (1 bit), which is an example of information indicating whether to delete data, and an area for storing a transfer determination bit (2 bits), which is an example of information indicating a transfer destination. The input deletion bit holds information indicating whether to delete the uncompressed target intermediate data input to the processing target layer from the SRAM40 after executing the calculation process of the processing target layer. For example, "0" of the input deletion bit indicates not to delete the uncompressed target intermediate data from the SRAM40, and "1" of the input deletion bit indicates to delete the uncompressed target intermediate data from the SRAM40. The input deletion bit is an example of deletion information indicating whether to delete the uncompressed intermediate data from the SRAM40.

[0037] When the input deletion bit is "0", the data processing apparatus 100 of the present embodiment continues to hold without deleting the uncompressed intermediate data, which is the calculation processing result in other layers used for the calculation process, from the SRAM40 after executing the calculation process of the processing target layer. Further, when the input deletion bit is "1", the data processing apparatus 100 deletes the uncompressed intermediate data, which is the calculation processing result in other layers used for the calculation process, from the SRAM40 after executing the calculation process of the processing target layer.

[0038] The data processing apparatus 100 of the present embodiment can suppress the memory capacity of the SRAM40 incorporated in the processor 20 by deleting from the SRAM40 when the uncompressed intermediate data becomes unnecessary for subsequent calculation processes. Note that the uncompressed intermediate data may be used for the calculation processes of a plurality of layers. In this case, only the input deletion bit corresponding to the layer executed last is set to "1". Thereby, even when the intermediate data common to a plurality of layers is used, it is possible to prevent the intermediate data from being erroneously deleted from the SRAM40.

[0039] The transfer determination bit of this embodiment holds information indicating the transfer destination (storage destination) of the intermediate data. "00" of the transfer determination bit indicates that the compressed intermediate data is transferred to the SRAM 40. "01" of the transfer determination bit indicates that the compressed intermediate data is transferred to the DRAM 50. "10" of the transfer determination bit indicates that the compressed intermediate data is transferred to both the SRAM 40 and the DRAM 50. The information indicating the transfer destination (storage destination) of the intermediate data held in the transfer determination bit is an example of the storage destination information.

[0040] By providing the transfer determination bit, the data processing apparatus 100 can easily determine the transfer destination of the compressed intermediate data for each layer. When the compressed intermediate data is transferred to only one of the SRAM 40 or the DRAM 50, that is, when the compressed intermediate data is not transferred to both the SRAM 40 and the DRAM 50, the transfer determination bit may be 1 bit. In this case, "0" of the transfer determination bit indicates transfer to the SRAM 40, and "1" of the transfer determination bit indicates transfer to the DRAM 50.

[0041] Each determination table TBL1 may have an input deletion bit and a transfer determination bit common to all processing target layers. That is, the input deletion bit and the transfer determination bit may be set for each neural network. Further, at least one of the determination tables TBL1 may hold a plurality of input deletion bits and a plurality of transfer determination bits corresponding to at least one of the processing target layers. In this case, the plurality of input deletion bits and the plurality of transfer determination bits are set corresponding to each of the plurality of data or the plurality of data groups used in the corresponding processing target layer.

[0042] In addition, a plurality of determination tables TBL1 may be provided corresponding to each of a plurality of compression rates. For example, in the forward processing of the neural network A, when the compression rate is sequentially increased every time a predetermined number of iterations are executed, a determination table TBL1(A) is provided for each compression rate, and the determination table TBL1(A) corresponding to the number of iterations is referred to. Alternatively, a compression rate table (an example of compression rate information) indicating the correspondence between a plurality of compression rates and the number of iterations may be provided corresponding to each determination table TBL1.

[0043] FIG. 6 is a flowchart showing an example of the forward processing of a neural network by the data processing apparatus 100 of FIG. 1. That is, FIG. 6 shows an example of a data processing method by the data processing apparatus 100. For example, the processing shown in FIG. 6 is realized by the processor 20 of the data processing apparatus 100 executing a data processing program. For example, FIG. 6 shows an example of forward processing in a method for generating a neural network model by machine learning.

[0044] First, in step S10, the processor 20 transfers input data such as parameters used in the layer to be processed for forward processing to the SRAM 40. In the input layer shown in FIG. 3, the processor 20 transfers the input data 1 and the parameter 1 to the SRAM 40 as input data.

[0045] Next, in step S12, the processor 20 executes forward processing using the input data transferred in step S10 and generates intermediate data. Next, in step S14, the processor 20 stores the intermediate data (uncompressed) generated in step S12 in the SRAM 40.

[0046] Next, in step S16, the processor 20 refers to the input deletion bit in the determination table TBL1 and determines whether to delete the uncompressed intermediate data input to the layer to be processed from the SRAM 40. When the input deletion bit is "1", the processor 20 decides to delete the uncompressed intermediate data from the SRAM 40 and transfers the process to step S18. When the input deletion bit is "0", the processor 20 decides not to delete the uncompressed intermediate data from the SRAM 40 and transfers the process to step S20.

[0047] In step S18, the processor 20 deletes the uncompressed intermediate data input to the layer to be processed from the SRAM 40 and transfers the process to step S20. In step S20, the processor 20 compresses the intermediate data calculated by the forward processing of the layer to be processed and generates compressed data.

[0048] Next, in step S22, the processor 20 refers to the transfer determination bit in the determination table TBL1. When the transfer determination bit is "00", the processor 20 decides to transfer the intermediate data to the SRAM 40 and transfers the process to step S24. When the transfer determination bit is "01", the processor 20 decides to transfer the intermediate data to the DRAM 50 and transfers the process to step S28. When the transfer determination bit is "10", the processor 20 decides to transfer the intermediate data to both the SRAM 40 and the DRAM 50 and transfers the process to step S26.

[0049] In step S24, the processor 20 transfers the intermediate data compressed in step S20 to the SRAM 40 and transfers the process to step S30. In step S26, the processor 20 transfers the intermediate data compressed in step S20 to the SRAM 40 and transfers the process to step S28.

[0050] In step S28, the processor 20 transfers the intermediate data compressed in step S20 to the DRAM 50 and shifts the process to step S30. As described above, according to the value of the transfer determination bit, the compressed intermediate data can be transferred to at least one of the SRAM 40 and the DRAM 50.

[0051] In step S30, if there is an unprocessed layer, the processor 20 returns the process to step S10 and executes the forward process for the next layer to be processed. If there is no unprocessed layer, that is, if the forward process of the neural network is completed, the processor 20 ends the operation shown in FIG. 6.

[0052] As described above, in the embodiment described with reference to FIGS. 1 to 6, the amount of memory used to hold the intermediate data generated by the forward process can be reduced. As a result, without mounting a faster DRAM 50 and without increasing the data bus width, the memory bandwidth between the processor 20 and the DRAM 50 required to store the intermediate data generated by the forward process in the DRAM 50 can be sufficiently secured. In other words, the memory bandwidth required to store the intermediate data in the DRAM 50 can be reduced compared to the memory bandwidth required to store all the intermediate data in the DRAM 50. For example, in the backward process, by compressing the intermediate data and then holding it in the memory, the amount of memory used can be further reduced, and the memory bandwidth of the memory storing the intermediate data can be further reduced. Thereby, without degrading the efficiency and accuracy of the backward process, an increase in the system cost of the data processing apparatus 100 can be suppressed.

[0053] Note that the first process and the second process in this specification are not limited to the forward process and the backward process in the training of the machine learning model.

[0054] The data processing apparatus 100 reads out intermediate data stored in the non-compressed SRAM 40 in the intermediate layer and the output layer that execute calculation processing, and uses the read non-compressed intermediate data for calculation processing. By using the intermediate data of the non-compressed data in the forward processing, the forward processing can be executed without degrading the calculation accuracy.

[0055] When the non-compressed intermediate data becomes unnecessary for subsequent calculation processing, the data processing apparatus 100 can suppress the memory capacity of the SRAM 40 incorporated in the processor 20 by deleting it from the SRAM 40. By providing an input deletion bit for each layer to be processed, it is possible to prevent the intermediate data from being erroneously deleted from the SRAM 40 even when common intermediate data is used in a plurality of layers.

[0056] By providing a transfer determination bit, the data processing apparatus 100 can easily determine the transfer destination of the compressed intermediate data for each layer.

[0057] Since the intermediate data used in the backward processing is irreversibly compressed, the compression cost can be reduced compared to reversible compression and the compression rate can be made constant, so the load imposed on the processor 20 by the compression processing can be reduced.

[0058] The intermediate data can be easily irreversibly compressed by expressing the intermediate data in the floating-point data format and reducing the number of bits (number of digits) of the mantissa part to compress the intermediate data.

[0059] By setting the compression rate of the intermediate data to be higher as the calculation processing of the layer progresses, the memory bandwidth for transferring the intermediate data can be further suppressed, and an increase in the system cost of the data processing apparatus 100 can be further suppressed.

[0060] FIG. 7 is an explanatory diagram showing an example of a determination table used in the forward processing of a neural network by the data processing apparatus according to the second embodiment of the present invention. For elements similar to those in FIG. 5, detailed descriptions thereof are omitted.

[0061] The data processing apparatus that refers to the determination tables TBL2(A), TBL2(B), TBL2(C),... in FIG. 7 has the same configuration as the data processing apparatus 100 shown in FIG. 1. That is, the data processing apparatus of this embodiment has at least one system board 10 including a processor 20 including a plurality of arithmetic units 30 and a plurality of SRAMs 40, and a DRAM 50.

[0062] Hereinafter, when the determination tables TBL2(A), TBL2(B), TBL2(C),... are described without distinction, they are simply referred to as the determination table TBL2. For example, similar to the determination table TBL1, the determination table TBL2 is provided for each of the neural networks A, B, C,....

[0063] The determination table TBL2 has an area for storing compressed determination bits added to the determination table TBL1 in FIG. 5. That is, each determination table TBL2 has an area for storing an input deletion bit (1 bit), a compressed determination bit (1 bit), and a transfer determination bit (2 bits) for each layer to be processed.

[0064] "0" of the compressed determination bit indicates that the intermediate data is compressed, and "1" of the compressed determination bit indicates that the intermediate data is not compressed. That is, in this embodiment, it is possible to switch the presence or absence of compression of intermediate data for each layer to be processed. For example, when the size of the generated intermediate data is large, the compressed determination bit is set to "0", and when the size of the generated intermediate data is small, the compressed determination bit is set to "1". Thereby, when the size of the intermediate data is large, an increase in the memory bandwidth can be suppressed, and an increase in the system cost of the data processing apparatus can be suppressed. On the other hand, when the size of the intermediate data is small, the compression cost can be reduced.

[0065] When the compression determination bit is "0", the meaning of each value of the transfer determination bit is the same as the meaning of each value of the transfer determination bit in the determination table TBL1 of FIG. 5. That is, "00" of the transfer determination bit indicates that the compressed intermediate data is transferred to the SRAM 40. "01" of the transfer determination bit indicates that the compressed intermediate data is transferred to the DRAM 50. "10" of the transfer determination bit indicates that the compressed intermediate data is transferred to both the SRAM 40 and the DRAM 50.

[0066] On the other hand, when the compression determination bit is "0", the meaning of each value of the transfer determination bit is as follows. "00" of the transfer determination bit indicates that the uncompressed intermediate data is transferred to the DRAM 50. "01" of the transfer determination bit indicates that the uncompressed intermediate data is not transferred to the DRAM 50.

[0067] The uncompressed intermediate data is always transferred to the SRAM 40. For this reason, when the compression determination bit is "1" and the transfer determination bit is "00", the uncompressed intermediate data will be transferred to both the SRAM 40 and the DRAM 50. When the compression determination bit is "1" and the transfer determination bit is "01", the uncompressed intermediate data will be transferred only to the SRAM 40.

[0068] In this embodiment, the meaning of the transfer determination bit changes according to whether the intermediate data is compressed or not by the compression determination bit. That is, the transfer determination bit can be shared between the case of compressing the intermediate data and the case of not compressing it, and an increase in the size of the determination table TBL2 can be suppressed.

[0069] Note that, similar to the above-described embodiments, each determination table TBL2 may have an input deletion bit, a compression determination bit, and a transfer determination bit that are common to all layers to be processed. That is, the input deletion bit, the compression determination bit, and the transfer determination bit may be set for each neural network. Further, at least one of the determination tables TBL2 may hold a plurality of input deletion bits, a plurality of compression determination bits, and a plurality of transfer determination bits corresponding to at least one of the layers to be processed. In this case, the plurality of input deletion bits, the plurality of compression determination bits, and the plurality of transfer determination bits are set corresponding to each of the plurality of data or data groups used in the corresponding layer to be processed.

[0070] Also, a plurality of determination tables TBL2 may be provided corresponding to each of a plurality of compression rates. For example, in the forward processing of neural network A, when the compression rate is sequentially increased every predetermined number of iterations, a determination table TBL2(A) is provided for each compression rate, and the determination table TBL2(A) corresponding to the number of iterations is referred to. Alternatively, a compression rate table indicating the correspondence between a plurality of compression rates and the number of iterations may be provided corresponding to each determination table TBL2.

[0071] Furthermore, when the compressed intermediate data is transferred to only one of SRAM40 or DRAM50, that is, when the compressed intermediate data is not transferred to both SRAM40 and DRAM50, the transfer determination bit may be 1 bit. In this case, when the compression determination bit is "0", "0" of the transfer determination bit indicates transfer to SRAM40, and "1" of the transfer determination bit indicates transfer to DRAM50. When the compression determination bit is "1", "0" of the transfer determination bit indicates transfer to DRAM50, and "1" of the transfer determination bit indicates not transferring to DRAM50.

[0072] FIG. 8 and FIG. 9 are flowcharts showing examples of forward processing of a neural network by the data processing apparatus according to the second embodiment. That is, FIG. 8 and FIG. 9 show an example of a data processing method by the data processing apparatus. For example, the processing shown in FIG. 8 and FIG. 9 is realized by the processor 20 of the data processing apparatus executing a data processing program. For example, FIG. 8 and FIG. 9 show an example of forward processing among methods for generating a neural network model by machine learning. For the same processing as in FIG. 6, the same step numbers are assigned, and detailed description thereof is omitted.

[0073] The processing from step S10 to step S18 is the same as the processing from step S10 to step S18 in FIG. 6. However, after step S16 and step S18, the processing shifts to step S19 instead of step S20.

[0074] In step S19, the processor 20 refers to the compression determination bit of the determination table TBL2 and determines whether to compress the intermediate data generated by the forward processing of the layer to be processed. If the processor 20 decides to compress the intermediate data, the processing shifts to step S20, and if the processor 20 decides not to compress the intermediate data, the processing shifts to step S21.

[0075] Here, the determination of whether to compress may be determined according to the hardware configuration of the data processing apparatus 100 and the configuration of the neural network. For example, the hardware configuration may be indicated by the storage capacity and memory bandwidth of the SRAM 40, the storage capacity and memory bandwidth of the DRAM 50, and the processing performance of the arithmetic unit 30, etc. For example, the configuration of the neural network may be indicated by the calculation procedure of the neural network, or may be indicated by a calculation graph showing the neural network.

[0076] The process of step S20 is the same as the process of step S20 in FIG. 6. Similar to the above-described embodiments, the data processing apparatus 100 may compress the generated intermediate data by converting it from double-precision floating-point data to single-precision floating-point data. The data processing apparatus 100 may compress the intermediate data by converting it from single-precision floating-point data to 8-bit fixed-point decimal data. Further, the data processing apparatus 100 may compress the intermediate data by reducing the number of bits (digits) of the mantissa part of the floating-point data.

[0077] Also, the compression rate of the intermediate data may be set higher as the training of the neural network progresses. The data processing apparatus 100 may compress a plurality of intermediate data together instead of compressing the intermediate data one by one.

[0078] After step S20, the process proceeds to step S22 in FIG. 9. In step S21, the processor 20 refers to the transfer determination bit of the determination table TBL2. When the transfer determination bit is "00", the processor 20 proceeds to step S28 in FIG. 9, and when the transfer determination bit is "01", the processor 20 proceeds to step S30 in FIG. 9.

[0079] The processes from step S22 to step S30 in FIG. 9 are the same as the processes from step S22 to step S30 in FIG. 6. However, the process of step S28 is also executed when it is determined in step 21 of FIG. 8 that the intermediate data is transferred to the DRAM 50. The process of step SS30 is also executed when it is determined in step 21 of FIG. 8 that the intermediate data is not transferred to the DRAM 50.

[0080] Also in the embodiments shown in FIGS. 7 to 9 above, the same effects as those of the above-described embodiments can be obtained. For example, by compressing the intermediate data and then holding it in the memory, the amount of memory used can be reduced, and the transfer time of the intermediate data can be shortened. Thereby, the memory bandwidth for transferring the intermediate data can be suppressed, and an increase in the system cost of the data processing apparatus 100 can be suppressed.

[0081] Furthermore, in this embodiment, by providing a compression determination bit in the determination table TBL2, the data processing apparatus 100 can switch the presence or absence of compression of the intermediate data for each layer to be processed. Thereby, when the size of the intermediate data is large, an increase in the memory bandwidth can be suppressed, and an increase in the system cost of the data processing apparatus can be suppressed. On the other hand, when the size of the intermediate data is small, the compression cost can be reduced.

[0082] By changing the meaning of the transfer determination bit according to the presence or absence of compression of the intermediate data by the compression determination bit, the transfer determination bit can be shared between the case where the intermediate data is compressed and the case where it is not compressed, and an increase in the size of the determination table TBL2 can be suppressed.

[0083] Part or all of the data processing apparatus in the above-described embodiments may be configured by hardware, or may be configured by information processing of software (program) executed by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like. When configured by information processing of software, software that realizes at least some functions of each apparatus in the above-described embodiments is stored in a non-transitory storage medium (non-transitory computer-readable medium) such as a flexible disk, a CD-ROM (Compact Disc-Read Only Memory), or a USB (Universal Serial Bus) memory, and the software may be executed by being read into a computer. Further, the software may be downloaded via a communication network. Furthermore, the information processing may be executed by hardware by implementing the software in a circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0084] The type of storage medium that stores software such as a data processing program is not limited. The storage medium is not limited to removable ones such as magnetic disks or optical disks, and may be a fixed-type storage medium such as a hard disk or a memory. Further, the storage medium may be provided inside the computer or outside the computer.

[0085] FIG. 10 is a block diagram showing an example of the hardware configuration of the data processing apparatus according to the above-described embodiment. Hereinafter, the hardware configuration of the data processing apparatus 100 in FIG. 1 will be described. The data processing apparatus 100 may be realized as a computer including, as an example, a processor 20, a main storage device 50 (e.g., DRAM 50), an auxiliary storage device 60 (memory), a network interface 70, and a device interface 80, which are connected via a bus 90. For example, when the processor 20 executes a data processing program, the operations described with reference to FIGS. 6 or 8 to 9 are executed.

[0086] The data processing apparatus 100 includes one of each component, but may include a plurality of the same components. Further, in FIG. 10, one data processing apparatus 100 is shown, but software may be installed in a plurality of data processing apparatuses 100, and each of the plurality of data processing apparatuses 100 may execute the same or different parts of the software. In this case, it may be a form of distributed computing in which each of the data processing apparatuses 100 communicates via a network interface 70 or the like to execute processing. That is, the data processing apparatus 100 in the above-described embodiment may be configured as a computer system in which one or a plurality of data processing apparatuses 100 execute instructions stored in one or a plurality of storage devices to realize functions. Further, it may be configured such that information transmitted from a terminal is processed by one or a plurality of data processing apparatuses 100 provided on the cloud, and the processing result is transmitted to the terminal.

[0087] The operations described in the flow of FIG. 6 and the operations described in the flows of FIGS. 8 to 9 may be executed in parallel using one or more processors 20 or using a plurality of computers via a network. Also, various operations may be allocated to a plurality of arithmetic cores within the processor 20 and executed in parallel. Also, part or all of the processes, means, etc. of the present disclosure may be executed by at least one of a processor and a storage device provided on a cloud that can communicate with the data processing device 100 via a network. Thus, the data processing device 100 in the above-described embodiment may be in the form of parallel computing using one or a plurality of computers.

[0088] The processor 20 may be an electronic circuit (processing circuit, Processing circuit, Processing circuitry, CPU, GPU, FPGA, or ASIC, etc.) including a control device and an arithmetic device of a computer. Also, the processor 20 may be a semiconductor device including a dedicated processing circuit, etc. The processor 20 is not limited to an electronic circuit using electronic logic elements and may be realized by an optical circuit using optical logic elements. Also, the processor 20 may include an arithmetic function based on quantum computing.

[0089] The processor 20 may perform arithmetic processing based on data and software (programs) input from each device, etc. of the internal configuration of the data processing device 100, and may output arithmetic results and control signals to each device, etc. The processor 20 may control each component constituting the data processing device 100 by executing the OS (Operating System) of the data processing device 100 and applications, etc.

[0090] The data processing apparatus 100 in the above-described embodiment may be implemented by one or more processors 20. Here, the processor 20 may refer to one or more electronic circuits arranged on one chip, or may refer to one or more electronic circuits arranged on two or more chips or two or more devices. When using a plurality of electronic circuits, each electronic circuit may communicate by wire or wirelessly.

[0091] The main storage device 50 (e.g., DRAM 50 in FIG. 1) may store instructions and various data executed by the processor 20, and the information stored in the main storage device 50 may be read by the processor 20. The auxiliary storage device 60 is a storage device other than the main storage device 50. Note that these storage devices mean any electronic components capable of storing electronic information, and may be semiconductor memories. The semiconductor memory may be either a volatile memory or a non-volatile memory. In the data processing apparatus 100 in the above-described embodiment, the storage device for storing various data may be realized by the main storage device 50 or the auxiliary storage device 60, or may be realized by a built-in memory such as SRAM 40 built into the processor 20.

[0092] When the data processing apparatus 100 in the above-described embodiment is composed of at least one storage device (memory) and a plurality of processors 20 connected (coupled) to this at least one storage device (memory), a plurality of processors 20 may be connected (coupled) to one storage device (memory), or one processor 20 may be connected. Also, a plurality of storage devices (memories) may be connected (coupled) to one processor 20, or one storage device (memory) may be connected. Further, at least one of the plurality of processors 20 may include a configuration in which it is connected (coupled) to at least one storage device (memory). Also, this configuration may be realized by the storage devices (memories) and processors 20 included in a plurality of data processing apparatuses 100. Furthermore, a configuration in which the storage device (memory) is integrated with the processor 20 (e.g., a cache memory including L1 cache and L2 cache) may be included.

[0093] The network interface 70 is an interface for connecting to the communication network 200, either wirelessly or by wire. The network interface 70 may use an appropriate interface such as one that conforms to an existing communication standard. Information may be exchanged with an external device 210 connected via the communication network 200 by the network interface 70. Note that the communication network 200 may be any one of a WAN (Wide Area Network), a LAN (Local Area Network), a PAN (Personal Area Network), or a combination thereof, as long as information can be exchanged between the data processing device 100 and the external device 210. Examples of a WAN include the Internet, etc., examples of a LAN include IEEE802.11 and Ethernet (registered trademark), etc., and examples of a PAN include Bluetooth (registered trademark) and NFC (Near Field Communication), etc.

[0094] The device interface 80 is an interface such as USB that directly connects to the external device 220.

[0095] The external device 220 may be connected to the data processing device 100 via a network or may be directly connected to the data processing device 100.

[0096] The external device 210 or the external device 220 may be, for example, an input device. The input device is, for example, a device such as a camera, a microphone, a motion capture, various sensors, a keyboard, a mouse, or a touch panel, and provides the acquired information to the data processing device 100. It may also be a device having an input unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0097] Also, for example, the external device 210 or the external device 220 may be an output device. The output device may be, for example, a display device such as an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube), a PDP (Plasma Display Panel), or an organic EL (Electro Luminescence) panel, or may be a speaker that outputs sound or the like. Further, it may be a device including an output unit, a memory, and a processor, such as a personal computer, a tablet terminal, or a smartphone.

[0098] Also, the external device 210 or the external device 220 may be a storage device (memory). For example, the external device 210 may be a network storage or the like, and the external device 220 may be a storage such as an HDD. The external device 220 that is a storage device (memory) is an example of a recording medium readable by a computer such as the processor 20.

[0099] Also, the external device 210 or the external device 220 may be a device having some functions of the components of the data processing device 100 in the above-described embodiment. That is, the data processing device 100 may transmit or receive part or all of the processing results of the external device 210 or the external device 220.

[0100] In this specification (including claims), when an expression such as "at least one (one) of a, b, and c" or "at least one (one) of a, b, or c" (including similar expressions) is used, it includes any one of a, b, c, a - b, a - c, b - c, or a - b - c. Also, it may include multiple instances of any element, such as a - a, a - b - b, a - a - b - b - c - c, etc. Further, it also includes adding other elements other than the enumerated elements (a, b, and c), such as having d like a - b - c - d.

[0101] In this specification (including the claims), when expressions such as "using data as an input", "using data", "based on data", "in accordance with data", "in response to data", etc. (including similar expressions) are used, unless otherwise specified, it includes cases where various data themselves are used as an input, and cases where something obtained by performing some processing on various data (for example, data with noise added, normalized data, feature quantities extracted from data, intermediate representations of various data, etc.) is used as an input. Also, when it is described that "a certain result is obtained using data as an input", "using data", "based on data", "in accordance with data", "in response to data", etc. (including similar expressions), it includes cases where the result is obtained based only on the said data, and may also include cases where the result is obtained under the influence of other data, factors, conditions, and / or states, etc. other than the said data. Further, when it is described that "data is output" (including similar expressions), unless otherwise specified, it includes cases where various data themselves are used as an output, and cases where something obtained by performing some processing on various data (for example, data with noise added, normalized data, feature quantities extracted from data, intermediate representations of various data, etc.) is output.

[0102] In this specification (including the claims), when the terms "connected" and "coupled" are used, they are intended as non - limiting terms that include any of direct connection / coupling, indirect connection / coupling, electrical connection / coupling, communicative connection / coupling, operative connection / coupling, physical connection / coupling, etc. The said terms should be appropriately interpreted according to the context in which they are used, but connection / coupling forms that are not intentionally or naturally excluded should be interpreted non - limitatively as being included in the said terms.

[0103] In this specification (including the claims), when the expression "A is configured to B" is used, the physical structure of element A has a configuration capable of performing operation B, and the permanent or temporary setting / configuration of element A may include being set to actually perform operation B. For example, when element A is a general-purpose processor, the processor has a hardware configuration capable of performing operation B, and it may be set to actually perform operation B by a permanent or temporary program (instruction) setting. Also, when element A is a dedicated processor or a dedicated arithmetic circuit, etc., regardless of whether control instructions and data are actually attached, the circuit structure of the processor, etc., may be implemented to actually perform operation B.

[0104] In this specification (including the claims), when terms meaning inclusion or possession (e.g., "comprising / including" and "having", etc.) are used, they are intended as open-ended terms, including cases where something other than the object indicated by the object of the term is included or possessed. When the object of these terms meaning inclusion or possession does not specify a quantity or is an expression suggesting a singular number (an expression with "a" or "an" as an article), the expression should be interpreted as not being limited to a specific number.

[0105] In this specification (including the claims), even if an expression such as "one or more" or "at least one" is used in one place, and an expression that does not specify a quantity or implies a singular number (an expression with "a" or "an" as an article) is used in another place, the latter expression is not intended to mean "one". Generally, an expression that does not specify a quantity or implies a singular number (an expression with "a" or "an" as an article) should be construed as not necessarily being limited to a specific number.

[0106] In this specification, when it is described that a specific effect (advantage / result) is obtained for a specific configuration of a certain embodiment, unless there are other reasons, it should be understood that the same effect can also be obtained for one or more other embodiments having the same configuration. However, it should be understood that the presence or absence of the effect generally depends on various factors, conditions, and / or states, etc., and the effect is not necessarily obtained by the configuration. The effect is only obtained by the configuration described in the embodiment when various factors, conditions, and / or states, etc., are satisfied, and the effect is not necessarily obtained in the invention according to the claim that defines the configuration or a similar configuration.

[0107] In this specification (including the claims), when terms such as "maximize" are used, it includes obtaining a global maximum value, obtaining an approximation of the global maximum value, obtaining a local maximum value, and obtaining an approximation of the local maximum value, and should be appropriately interpreted according to the context in which the term is used. It also includes obtaining approximations of these maximum values probabilistically or heuristically. Similarly, when terms such as "minimize" are used, it includes obtaining a global minimum value, obtaining an approximation of the global minimum value, obtaining a local minimum value, and obtaining an approximation of the local minimum value, and should be appropriately interpreted according to the context in which the term is used. It also includes obtaining approximations of these minimum values probabilistically or heuristically. Similarly, when terms such as "optimize" are used, it includes obtaining a global optimum value, obtaining an approximation of the global optimum value, obtaining a local optimum value, and obtaining an approximation of the local optimum value, and should be appropriately interpreted according to the context in which the term is used. It also includes obtaining approximations of these optimum values probabilistically or heuristically.

[0108] In this specification (including the claims), when a plurality of hardware performs a predetermined process, each piece of hardware may cooperate to perform the predetermined process, or some of the hardware may perform all of the predetermined process. Also, some of the hardware may perform a part of the predetermined process and another piece of hardware may perform the remainder of the predetermined process. In this specification (including the claims), when an expression such as "one or more pieces of hardware perform a first computational process and the one or more pieces of hardware perform a second computational process" is used, the hardware that performs the first computational process and the hardware that performs the second computational process may be the same or different. That is, it is sufficient that the hardware that performs the first computational process and the hardware that performs the second computational process are included in the one or more pieces of hardware. Note that the hardware may include an electronic circuit or a device including an electronic circuit, etc.

[0109] In this specification (including the claims), when a plurality of storage devices (memories) store data, each of the plurality of storage devices (memories) may store only a part of the data or may store all of the data. Further, a configuration in which some of the plurality of storage devices (memories) store data may be included.

[0110] As described above in detail with respect to the embodiments of the present disclosure, the present disclosure is not limited to the individual embodiments described above. Various additions, changes, replacements, and partial deletions, etc. are possible without departing from the conceptual ideas and spirit of the present invention derived from the content defined in the claims and their equivalents. For example, in all of the above-described embodiments, when numerical values or mathematical formulas are used in the description, they are shown as examples and are not limited thereto. Also, the order of each operation in the embodiments is shown as an example and is not limited thereto.

Explanation of Reference Numerals

[0111] 10 System board 20 Processor 30 Arithmetic unit 40 SRAM 50 DRAM (main storage device) 60 Auxiliary storage device 70 Network interface 80 Device interface 90 Bus 100 Data processing device 200 Communication network 210 External device 220 External device

Claims

1. A method for generating a neural network including at least a first layer and a second layer, the method comprising: executing, by at least one processor, forward processing to generate first intermediate data from the first layer and to generate second intermediate data from the second layer; compressing, by the at least one processor, the first intermediate data to generate compressed data; performing, by the at least one processor, backward processing of the neural network based on the compressed data and the second intermediate data; updating, by the at least one processor, parameters of the neural network after performing the backward processing; the compressed data is generated by converting the first intermediate data expressed in a first numeric format into a second numeric format having fewer bits than the first numeric format; the second numeric format is a floating-point data format; The update is performed by using ADAM; the second intermediate data is used in the backward processing without lossy compression; Data processing methods.

2. The compression of the first intermediate data is performed according to a compression setting defined before the forwarding process is performed. The data processing method according to claim 1 .

3. determining, by the at least one processor, whether to store the generated compressed data in at least one of a first memory or a second memory; the first memory has a different bandwidth than the second memory; 3. The data processing method according to claim 1.

4. The compressed data is stored in both the first memory and the second memory. The data processing method according to claim 3 .

5. The first intermediate data and the second intermediate data are stored in a first memory; The compressed data is stored in a second memory; the first memory has a different bandwidth than the second memory; 3. The data processing method according to claim 1.

6. determining, by the at least one processor, whether to delete the second intermediate data.

6. The data processing method according to claim 1.

7. The method of claim 6, wherein the at least one processor determines that the first intermediate data is compressed and the second intermediate data is not compressed.

7. The data processing method according to claim 1.

8. The decision regarding compression includes determining, by the at least one processor, whether each of the first intermediate data and the second intermediate data is to be compressed based on a decision bit assigned to each of the first layer and the second layer. The data processing method according to claim 7.

9. Generating the compressed data by applying lossy compression to the first intermediate data by the at least one processor.

9. The data processing method according to claim 1.

10. The first numeric format is a floating-point number data format; the compressed data is generated by deleting the number of bits of the mantissa part of the first intermediate data.

10. The data processing method according to claim 1.

11. The data processing method according to claim 1, wherein the data processing method is performed in multiple iterations, a compression rate of the compression is varied in at least one of the plurality of iterations; The data processing method according to any one of claims 1 to 10.

12. Performing the backward processing based on the compressed data includes performing the backward processing using data generated from the compressed data. The data processing method according to any one of claims 1 to 11.

13. Executing forward processing of a neural network by at least one processor to generate intermediate data; compressing, by the at least one processor, the intermediate data to generate compressed data; performing, by the at least one processor, backward processing of the neural network based on the compressed data; performing, by the at least one processor, a process of updating parameters of the neural network in a plurality of iterations after performing the backward process; a compression rate used for said compression is varied in at least one of said plurality of iterations; Data processing methods.

14. The method of claim 13, wherein the at least one processor generates the compressed data by applying lossy compression to the intermediate data.

14. The data processing method according to claim 13.

15. performing the plurality of iterations each including the forward processing and the backward processing while gradually increasing the compression rate of the intermediate data.

15. The data processing method according to claim 13 or 14.

16. Executing the data processing method according to any one of claims 1 to 15. Data processing device.

17. A method for causing a computer to execute the data processing method according to any one of claims 1 to 15. Data processing program.

Images