Data cache device and program

Abstract

A data cache device for temporarily holding digital data comprising: a data holding unit (101) that holds data; a data input interface unit (102) for inputting data from outside of the data cache device; a data output interface unit (103) for outputting data to outside of the data cache device; a selector unit (104) that selects one or a plurality of paths simultaneously among a path for transferring the data from the data input interface unit to the data holding unit, a path for transferring the data from the data input interface unit to the data output interface unit, and a path for transferring the data from the data holding unit to the data output interface unit; a selector control unit (105) that controls the selector unit; and a read prediction table unit (107) that holds the number of times of planned reading of the data which is read during processing. The selector unit is controlled in accordance with the number of times of reading that is set in the read prediction table unit.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on and claims the benefit of priority from Japanese Application No. 2021-129531, filed on August 6, 2021, the entire contents of which are incorporated herein by reference. [Technical Field]

[0002] The present disclosure relates to a data cache device that temporarily stores digital data. [Background technology]

[0003] For example, a computing device that processes a neural network performs inference by successively executing convolution operations: that is, a convolution operation is performed on the original input data using a predetermined algorithm, and then a convolution operation is performed on the output data using a subsequent algorithm.

[0004] Here, the process of performing a convolution operation on one piece of input data to obtain an output is called a layer. In a neural network, the convolution operation algorithm for each layer is defined, and the network is composed of a combination of these layers.

[0005] To perform a convolution operation, a multiply-and-accumulate operation is performed, where weight values ​​are multiplied by the input data and the results are added. The kernel size for this weight value is determined by the convolution algorithm; for example, in the case of a 3x3 kernel, the weights of nine elements (3 vertical x 3 horizontal) are multiplied and accumulated for one piece of input data. This convolution process is required depending on the elements of the input data, so the convolution operation of a neural network requires the reading of a large amount of weight data. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2018-88256 Summary of the Invention

[0007] If this weight is read from an external read device at the required timing, a huge number of read accesses will occur, resulting in an increase in read power. Furthermore, in many cases, the read time from the external read device may become a bottleneck for the entire process. Patent Document 1, for example, is known as a semiconductor device that can reduce power consumption. In Patent Document 1, power supply to sets in a cache memory that are accessed less frequently is cut off.

[0008] A neural network processing device may be equipped with a cache device. A typical cache device stores data read from an external source in an internal cache device based on temporal or spatial proximity. If the same data needs to be read again, the data is read from the cache device without accessing the external source. This cache device is expected to have the effect of hiding the time delay caused by reading data from the external source. Furthermore, it is possible to reduce power consumption by implementing different methods between the external storage device and the cache device.

[0009] However, in neural network processing, multiple layers may be processed together. That is, input data may be divided into processing units, and multiple layers of processing may be performed consecutively on the divided input data. This allows the output data of one layer to be immediately reused as input data for the next layer, reducing unnecessary memory access and delays.

[0010] After detailed investigation by the inventors, it was found that when performing such processing, the weights for each layer must be frequently switched and complex reading is required, but in this case, temporal proximity and positional proximity are not necessarily guaranteed, resulting in an increase in cache misses, and therefore, the inventors found that conventional cache devices cannot effectively reduce power consumption.

[0011] Therefore, an object of the present disclosure is to provide a data cache device that is effective in reducing power consumption.

[0012] The present disclosure employs the following technical means to solve the above problems. The claims are merely examples showing the correspondence with the specific means described in the embodiments below as one aspect, and do not limit the technical scope of the present invention.

[0013] The data cache device disclosed herein is a data cache device that temporarily holds digital data, and includes a data holding unit that holds data, a data input interface unit for inputting data from outside the data cache device, a data output interface unit for outputting data to outside the data cache device, a selector unit that simultaneously selects one or more of a path for transferring data from the data input interface unit to the data holding unit, a path for transferring data from the data input interface unit to the data output interface unit, and a path for transferring data from the data holding unit to the data output interface unit, a selector control unit that controls the selector unit, and a read prediction table unit that holds the planned number of reads that the data will be read out during processing, and controls the selector unit according to the number of reads set in the read prediction table unit.

[0014] The program disclosed herein is a program for controlling a data cache device that includes a data holding unit that holds data, a data input interface unit for inputting data from outside the data cache device, a data output interface unit for outputting data to outside the data cache device, a path for transferring data from the data input interface unit to the data holding unit, a path for transferring data from the data input interface unit to the data output interface unit, and a selector unit that simultaneously selects one or more of the paths for transferring data from the data holding unit to the data output interface unit, and is a program that causes a computer to function as read prediction table means that holds the planned number of reads that the data will be read at the time of processing, and selector control means that controls the selector unit in accordance with the number of reads set in the read prediction table means.

[0015] Another aspect of the program of the present disclosure is a program for creating a table that sets whether data stored in a data cache device or data from an external storage device is read as data necessary for an arithmetic unit to perform an operation, and causes a computer to function as a read counting means that performs an arithmetic simulation of the arithmetic unit using an arithmetic simulator and counts the number of times each piece of data is read when the operation is performed, and a table creation means that creates the table based on the number of times each piece of data is read by the read counting means. [Brief explanation of the drawings]

[0016] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: [Figure 1] FIG. 1 is a diagram illustrating a configuration of a data cache device according to a first embodiment; [Figure 2] FIG. 2 is a diagram showing the overall configuration of a computing device including a data cache device according to a first embodiment; [Figure 3]FIG. 3 is a flowchart illustrating the operation of the data cache device according to the first embodiment; [Figure 4] FIG. 4 is a diagram illustrating a configuration of a data cache device according to a second embodiment; [Figure 5] FIG. 5 is a diagram illustrating a configuration of a data cache device according to a third embodiment; [Figure 6] FIG. 6 is a flowchart illustrating the operation of the data cache device according to the third embodiment. [Figure 7] FIG. 7 is a diagram illustrating a configuration of a data cache device according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, a data cache device according to an embodiment of the present disclosure will be described with reference to the drawings. (First embodiment) FIG. 1 shows the configuration of a data cache device 1 according to a first embodiment, and FIG. 2 shows the overall configuration including the data cache device 1, an external storage device 2, an arithmetic unit 3, and an arithmetic algorithm control unit 4. As shown in FIG. 2, the data cache device 1 is connected to the external storage device 2, the arithmetic unit 3, and the arithmetic algorithm control unit 4. The data cache device 1 inputs data from the external storage device 2, retains some of the data, and then outputs it to the arithmetic unit 3. The arithmetic algorithm control unit 4 manages the layer parameters and algorithms of the neural network, and controls the data cache device 1 and the arithmetic unit 3 in conjunction with each other.

[0018] Next, the internal configuration of the data cache device 1 of this embodiment will be described. Reference numeral 101 denotes a data storage unit. In this embodiment, this data storage unit 101 is a cache memory that temporarily stores weight data of a neural network. The data storage unit 101 is implemented by an SRAM or a flip-flop circuit. The data storage unit 101 is also called a "local buffer."

[0019] Reference numeral 102 denotes a data input interface unit 102. The data input interface unit 102 is connected to the external storage device 2 and receives data from the external storage device 2. The connection between the data input interface unit 102 and the external storage device 2 may be made via a bus interface or the like, as long as data can be logically acquired. The data input interface unit 102 includes a function as a bus master, and acquires data from the external storage device 2 in response to a request transmitted from the arithmetic unit 3 via a selector unit 104.

[0020] A data output interface 103 is connected to the arithmetic unit 3.

[0021] A selector unit 104 is a circuit that selects the destination of input data from the data input interface unit 102 and the source of requested data from the data output interface unit 103.

[0022] A selector control unit 105 is a unit that controls the selector unit 104, and controls the connection of the selector unit 104 in units of instruction execution cycles. The selector control unit 105 simultaneously selects one or more of a path for transferring data from the data input interface unit 102 to the data holding unit 101, a path for transferring data from the data input interface unit 102 to the data output interface unit 103, and a path for transferring data from the data holding unit 101 to the data output interface unit 103.

[0023] Reference numeral 106 denotes a read counter unit. The read counter unit 106 has a function of counting the number of accesses to the data output interface unit 103. A plurality of counters are stored internally, and reads for each address requested by the arithmetic unit 3 via the data output interface unit 103 are counted independently.

[0024] Reference numeral 107 denotes a read prediction table unit. The read prediction table unit 107 is implemented using an SRAM or a flip-flop circuit, and stores the number of reads at which the weight value in each layer is read. An initial value for the predicted number is written when the system is started, and may be updated at any timing thereafter. Depending on the implementation policy, only the magnitude relationship with a previously set read count threshold may be retained.

[0025] Reference numeral 108 denotes a read prediction table control unit. The read prediction table control unit 108 acquires algorithm information specified by the calculation algorithm control unit 4, reads the predicted number of reads of weight data for the corresponding layer from the read prediction table unit 107, and issues a selector selection signal to the selector control unit 105. In addition, the read prediction table control unit 108 has the function of acquiring read counter information from the read counter unit 106 and updating the value of the read prediction table unit 107 based on instructions from the calculation algorithm control unit 4.

[0026] 3 is a flowchart showing the operation of the data cache device 1. In the example described below, the arithmetic device 3 will be described taking as an example the calculation of moving image processing using a neural network. The arithmetic device 3 performs image processing for each frame that makes up a moving image. More specifically, the frame image or feature map to be processed is divided into multiple processing unit areas, which are called tiles, and an inference process using a neural network is performed for each tile. The inference process performed for each frame is the same.

[0027] In step S001, the read prediction table control unit 108 receives a list of algorithms to be processed from the computation algorithm control unit 4. For example, when multiple layers of a neural network are processed collectively, the information includes the number of layers, the number of kernels in each layer, the number of input data channels, the number of output data channels, and the input activation size. Based on the received information, the read prediction table control unit 108 calculates the required amount of weight to be held and the number of tiles for which the computation will be repeated. Alternatively, the computation algorithm control unit 4 may itself calculate the amount of weight to be held and receive the result.

[0028] The read prediction table control unit 108 determines whether the amount of weight required to be stored is equal to or less than the capacity of the data storage unit 101. If the amount of weight required for processing multiple layers is equal to or less than the capacity of the data storage unit 101 (YES in S001), the read prediction table control unit 108 continues to execute the process of step S002.

[0029] In step S002, the read prediction table control unit 108 receives information on the number of tiles when processing multiple layers from the calculation algorithm control unit 4, and determines whether the number of tiles is two or more.

[0030] At this time, if the number of tiles to be processed is 1 ("No" in S002), there is no advantage in terms of power consumption or performance to transferring the data to the data holding unit 101 and temporarily holding it, so the process proceeds to step S003, where the read prediction table control unit 108 updates the read prediction table unit 107 so that data is transferred directly from the external storage device 2 without going through the data holding unit 101. If the number of tiles is 2 or more ("Yes" in S002), the process proceeds to step S004, where the read prediction table unit 107 is updated with a flag set indicating that the entire area of ​​the prediction table is to be transferred to the data holding unit 101. Note that the order of the determinations in steps S001 and S002 described above may be reversed.

[0031] Next, in step S012, data is transferred from the external storage device 2 in accordance with the read prediction table unit 107. Specifically, when the number of tiles to be processed is 1, the data is transferred directly without going through the data holding unit 101, and when the number of tiles to be processed is 2 or more, the data is transferred from the external storage device 2 to the data holding unit 101.

[0032] On the other hand, if the amount of weight data is equal to or greater than the capacity of the data storage unit 101 in step S001 ("No" in S001), the process proceeds to step S005, where the read prediction table unit 107 is updated so that weight data for layers up to the capacity of the data storage unit 101 is transferred to the data storage unit 101, starting with the data of the layer with the largest number of processed tiles.

[0033] Step S006 is a step in which weight data to be read from the data holding unit 101 is transferred from the external storage device 2 to the data holding unit 101. The read prediction table control unit 108 sends a read request to the data input interface unit 102 for the weight data that needs to be read from the data holding unit 101. Alternatively, the request may be notified via the calculation algorithm control unit 4.

[0034] The data input interface unit 102 transfers the weight data to be newly read from the data holding unit 101 from the external storage device 2 to the data holding unit 101. At this time, the data input interface unit 102 functions as a bus master, generates an address of the external storage device 2 from the weight position information held in the read prediction table, and issues a data transfer request. Alternatively, a DMA controller or the like may be provided between the external storage device 2 and the data input interface unit 102 to provide the same function.

[0035] Step S007 and subsequent steps are steps that occur after processing of the first tile has begun. Step S007 occurs when the calculation algorithm control unit 4 sends a processing start signal to the calculation device 3, and the calculation device 3 sends a request to the data output interface unit 103 to acquire the necessary weight data together with the address. The data output interface unit 103 receives the weight data transfer request together with the address from the calculation device 3. At this time, the data output interface unit 103 transmits the requested address information to the selector control unit 105.

[0036] The selector control unit 105 refers to the read prediction table unit 107, determines whether the requested data is present in the external storage device 2 or is held in the data holding unit 101, and controls the selector unit 104 to connect a path to the required storage area so that the data output interface unit 103 can access the data. The selector control unit 105 notifies the data output interface unit 103 of map information indicating whether the data is present in the external storage device 2 or in the data holding unit 101 as a signal to establish a path.

[0037] The data output interface unit 103 increments the requested transfer data amount of the read counter unit 106 for each piece of information indicating which layer the requested data is weight data for and for each piece of map information returned from the selector control unit 105. Based on the map information, the read counter unit 106 counts and holds the number of times weight data has been read from the data holding unit and the number of times it has been read directly from the data input interface unit.

[0038] In step S008, the calculation device 3 continues to acquire weight data via the data output interface unit 103 until all tiles have been processed. During this time, the data output interface unit 103 continues step S007. If processing of all tiles has been completed in step S008 ("Yes" in S008), the process proceeds to step S009. In the processing up to this point, when one frame is processed, the count data of the number of times weight data has been read out in each layer is counted by the read counter unit 106.

[0039] In step S009, the read prediction table control unit 108 reads out count data from the read counter unit 106 for each layer and compares it with a threshold value previously specified by the calculation algorithm control unit 4. The threshold value is calculated from the amount of power consumed in reading and transferring data. In other words, it is determined based on the amount of power required to transfer data from the external storage device 2 without going through the data holding unit 101 and the amount of power required to transfer data from the external storage device 2 to the data holding unit 101 and read the data from the data holding unit 101.

[0040] If the number of reads is equal to or greater than the threshold, reading the weight data from the data holding unit 101 consumes less power than directly acquiring the data from the external storage device 2. As a result of comparing the number of reads of the weight data with the threshold, the read prediction table control unit 108 sets a flag for the weight data of the layer whose number of reads is equal to or greater than the threshold. In step S010, the flag information set in the above-mentioned step S009 is read and written to the read prediction table unit 107.

[0041] It is also possible to set a flag in the read prediction table unit 107 to read from the data holding unit 101 in descending order of the read count, that is, in descending order of the read amount, without performing threshold evaluation in step S009.

[0042] Step S011 is a step of transferring data to the data holding unit 101 based on the information of the read prediction table unit 107. The read prediction table control unit 108 sends a read request to the data input interface unit 102 for weight data that needs to be newly read from the data holding unit 101. Alternatively, the request may be notified via the calculation algorithm control unit 4. The data input interface unit 102 overwrites the weight data that should newly be read from the data holding unit 101 in the area of ​​the weight data that is now to be read directly from the external storage device 2. Alternatively, step S006 may be executed again based on the newly updated contents of the read prediction table unit 107.

[0043] In step S013, the remaining frames are processed using the data in the read prediction table unit 107 and the data holding unit 101 prepared by the above-described processing.

[0044] In this embodiment, weight data has been described as an example of data to be stored in the data storage unit 101, but it goes without saying that the present disclosure is not limited to weight data and may also be applied to other data, such as activation data or other weight data.

[0045] The configuration of the data cache device 1 of this embodiment has been described above, but the data cache device 1 that performs the above-mentioned control can be realized by storing a program having modules that realize the functions of the selector control unit 105, read counter unit 106, read prediction table unit 107, and read prediction table control unit 108 in a storage device and executing the program with a CPU. Such a program is also included in the scope of the present disclosure.

[0046] When processing the first frame, the data cache device 1 of this embodiment sets a flag in the weight data of a layer whose read count is equal to or greater than a threshold value, and writes the flag to the read prediction table unit 107. Since the weight data is transferred to the data storage unit 101 based on the read prediction table unit 107, the weight data stored in the data storage unit 101 can be used when processing the remaining frames, thereby improving the calculation performance while suppressing power consumption. In addition, when processing the remaining frames, the counting and comparison process of this embodiment may be stopped to reduce the power used for evaluation.

[0047] (Second embodiment) 4 is a diagram showing the configuration of the data cache device 5 according to the second embodiment. The basic configuration of the data cache device 5 according to the second embodiment is the same as that of the data cache device 1 according to the first embodiment, but the data cache device 5 according to the second embodiment differs in that it handles compressed weight data.

[0048] The data cache device 5 is connected to the external storage device 2, the arithmetic device 3, and the arithmetic algorithm control unit 4 described in Fig. 2. The data cache device 5 inputs data from the external storage device 2, stores some of the data, and then outputs it to the arithmetic device 3. The arithmetic algorithm control unit 4 manages the layer parameters and algorithms of the neural network, and controls the data cache device 5 and the arithmetic device 3 in conjunction with each other.

[0049] Next, the internal configuration of the data cache device 5 according to the second embodiment will be described. Reference numeral 501 denotes a data holding unit. This data holding unit 501 is a cache memory that temporarily holds weight data. The data holding unit 501 is implemented by an SRAM or a flip-flop circuit.

[0050] Reference numeral 502 denotes a data input interface unit. The data input interface unit 502 is connected to the external storage device 2 and receives data from the external storage device 2. The connection between the data input interface unit 502 and the external storage device 2 may be established via a bus interface or the like, as long as data can be logically acquired. The data input interface unit 502 includes a function as a bus master, and acquires data from the external storage device 2 in response to a request transmitted from the arithmetic unit 3 via a selector unit 504.

[0051] A data output interface unit 503 is connected to the arithmetic unit 3.

[0052] A selector unit 504 is a circuit that selects the destination of input data from the data input interface unit 502 and the source of requested data from the data output interface unit 503.

[0053] A selector control unit 505 is a unit that controls the selector unit 504, and controls the connection of the selector unit 504 in units of instruction execution cycles.

[0054] Reference numeral 506 denotes a read counter unit. The read counter unit 506 has a function of counting the number of accesses to the data output interface unit 503. A plurality of counters are stored internally, and reads for each address requested by the arithmetic unit 3 via the data output interface unit 503 are counted independently.

[0055] Reference numeral 507 denotes a read prediction table unit. The read prediction table unit 507 is implemented using an SRAM or a flip-flop circuit, and stores the number of times that weight data in each layer is read. An initial value for the predicted number of times is written when the system is started, and the number of times may be updated at any timing thereafter. Depending on the implementation policy, only the magnitude relationship with a previously set read count threshold may be retained.

[0056] Reference numeral 508 denotes a read prediction table control unit. The read prediction table control unit 508 acquires algorithm information designated by the calculation algorithm control unit 4, reads the predicted number of reads of weight data for the corresponding layer from the read prediction table unit 507, and issues a selector selection signal to the selector control unit 505. In addition, the read prediction table control unit 508 has the function of acquiring read counter information from the read counter unit 506 based on instructions from the calculation algorithm control unit 4 and updating the value of the read prediction table unit 507.

[0057] Reference numeral 509 denotes a data decompression unit. The data decompression unit 509 decompresses, by a predetermined method, the compressed weight data transferred based on a data request from the calculation device 3. The compression method is not limited to a specific method, and the algorithm does not affect the effectiveness of the present disclosure, but may be, for example, run-length coding.

[0058] The operation of the data cache device 5 in the second embodiment conforms to the flowchart in Fig. 3. However, in creating the prediction table in steps S002, S003 to S005, calculations must be performed based on the data amount of compressed weight.

[0059] In this embodiment, weight data has been described as an example, but it goes without saying that the present disclosure is not limited to weight data and may also be applied to other data, such as activation data or other weight data.

[0060] Like the data cache device 1 of the first embodiment, the data cache device 5 of the second embodiment can also improve the computing performance while suppressing the amount of power consumption.

[0061] (Third embodiment) 5 is a diagram showing the configuration of a data cache device 6 according to the third embodiment. The data cache device 6 is connected to an external storage device 2, an arithmetic unit 3, and an arithmetic algorithm control unit 4. The data cache device 6 receives data from the external storage device 2, stores some of the data, and then outputs it to the arithmetic unit 3. The arithmetic algorithm control unit 4 manages layer parameters and algorithms of the neural network and controls the arithmetic unit 3.

[0062] A weight allocation program 7 is a program for determining whether weight data is to be allocated to the external storage device 2 or to the data holding unit 601.

[0063] Reference numeral 8 denotes a calculation simulator. The calculation simulator 8 is software that logically simulates the operations of the external storage device 2, the calculation device 3, the calculation algorithm control unit 4, and the data cache device 6.

[0064] Next, the internal configuration of the data cache device 6 of this embodiment will be described. A data storage unit 601 is a cache memory that temporarily stores weight data. The data storage unit 601 is implemented by an SRAM or a flip-flop circuit.

[0065] Reference numeral 602 denotes a data input interface unit. The data input interface unit 602 is connected to the external storage device 2 and receives data from the external storage device 2. The connection between the data input interface unit 602 and the external storage device 2 may be established via a bus interface or the like, as long as data can be logically acquired. The data input interface unit 602 incorporates a function as a bus master, and acquires data from the external storage device 2 in response to a request transmitted from the arithmetic unit 3 via a selector unit 604.

[0066] A data output interface unit 603 is connected to the arithmetic unit 3.

[0067] A selector unit 604 is a circuit that selects the destination of input data from the data input interface unit 602 and the source of requested data from the data output interface unit 603. The selector unit 604 connects a path for data from the data holding unit 601 or to the data input interface unit 602 according to the requested address issued by the data output interface unit 603.

[0068] A selector control unit 605 controls whether data is transferred from the data input interface unit 602 to the data holding unit 601 or whether data is sent in response to a request from the data output interface unit 603.

[0069] A data allocation table 606 is implemented by an SRAM or a flip-flop circuit, and records which layer's weight value is held in the data holding unit 601.

[0070] Next, the internal structure of the weight allocation program 7 will be described. A weight allocation table creating means 701 determines whether to allocate the weight data of each layer to the external storage device 2 or to store it in the data storage unit 601 based on the predicted number of accesses.

[0071] Reference numeral 702 denotes a read count means, which calculates the number of times weight data has been read for each layer. This read count is calculated by having the calculation simulator 8 execute layer processing, and acquiring the number of times each weight data has been read from the calculation simulator 8.

[0072] The weight allocation table creating means 701 and the read counting means 702 are configured as program modules, and their functions are realized by executing the weight allocation program 7 by a computer.

[0073] Next, FIG. 6 is a diagram showing the operation of the data cache device 6 according to the third embodiment. S101 is a step in which the weight placement program 7 uses the calculation simulator 8 to execute a calculation simulation. The weight placement program 7 calculates the number of accesses to the weight data using the calculation simulator 8 based on information such as the algorithm, number of kernels, number of layers, and input data size, which are input information to the calculation algorithm control unit 4. In step S102, the weight placement table creation means 701 executes this process for all tiles and layers, reads out the number of accesses for each layer, and lists it in the counting means 702. After step S102 is completed ("Yes" in step S102), the process proceeds to step S103.

[0074] Step S103 is a step of tallying up the number of weight accesses for each layer. In step S103, the weight allocation table creation means 701 sorts the read counts for each layer held in the read count means 702 in descending order.

[0075] Steps S104 to S106 are steps for creating a weight allocation table. The weight allocation table creation means 701 sets addresses in the allocation table so that data is allocated to the data storage unit 601 in descending order of the number of weight accesses, based on the previously acquired capacity of the data storage unit 601. For example, if the addresses of the data storage unit 601 and the addresses of the external storage device 2 are designed independently, the addresses of the data storage unit 601 can be mapped to the weight area with the greatest number of accesses. The processing of step S104 is performed sequentially in descending order of the number of accesses, and in step S105, it is determined whether the data storage capacity of the data storage unit 601 has become full. If the area of ​​the data storage unit 601 has been used up in step S105 ("Yes" in step S105), the process proceeds to step S106.

[0076] Step S106 is a step for specifying an area to be allocated in the external storage device 2. In step S106, an address in the external storage device 2 is specified for the weight data of the layer that cannot be stored in the data storage unit 601.

[0077] Step S107 is a step in which the created weight allocation table is transferred to the data allocation table unit 606 of the data cache device 6. In step S107, in order to carry out actual processing, the weight allocation program 7 transfers the created weight allocation table information to the data allocation table unit 606. The transfer may involve reading via the data input interface unit 102, or the data allocation table unit 606 may be memory mapped from an external area and transferred using an external master (not shown) or the like.

[0078] Step S108 is a step of transferring weight data to be read from data holding unit 601 according to the transferred data allocation table unit 606 to data holding unit 601. Data input interface unit 602 reads the data allocation table, switches selector unit 604 to data holding unit 601 by selector control unit 605, and transfers weights containing addresses of data holding unit 601 from external storage device 2. This transfer may be performed using data input interface unit 102, or may be performed by connecting an external master to data holding unit 601.

[0079] Step S109 is a step for transferring data during arithmetic processing. In this step, data requested by the arithmetic unit 3 reaches the selector unit 604 via the data output interface unit 603, and is obtained from the data holding unit 601 or the data input interface unit 602. The data output interface unit 603 receives the necessary weight data from the arithmetic unit 3, and generates an address to the area where the weight is stored by referring to the data allocation table. During arithmetic processing, step S109 continues until the processing is completed.

[0080] The data cache device 6 of the third embodiment includes a weight allocation program 7 and an arithmetic simulator 8, and determines whether to allocate weight data to the external storage device 2 or to store it in the data storage unit 601 based on the results of the arithmetic simulation, and updates the data allocation table unit 606 based on the determination result. By allocating weight data according to this data allocation table unit 606, it is possible to improve the arithmetic performance while suppressing power consumption.

[0081] (Fourth embodiment) Fig. 7 is a diagram showing the configuration of a data cache device 9 according to the fourth embodiment. The basic configuration of the data cache device 9 according to the fourth embodiment is the same as that of the data cache device 1 according to the first embodiment, but the data cache device 9 according to the fourth embodiment is different in that it has three output destination arithmetic devices 3. Note that while Fig. 7 shows an example in which there are three output destinations, the number of output destination arithmetic devices 3 is not limited to three, and may be two or four or more.

[0082] The data cache device 9 is connected to the external storage device 2, the arithmetic device 3, and the arithmetic algorithm control unit 4 described in Fig. 2. The data cache device 9 inputs data from the external storage device 2, retains some of the data, and then outputs it to the arithmetic device 3. The arithmetic algorithm control unit 4 manages the layer parameters and algorithms of the neural network, and controls the data cache device 9 and the arithmetic device 3 in conjunction with each other.

[0083] Next, the internal configuration of the data cache device 9 according to the fourth embodiment will be described. A data storage unit 901 is a cache memory that temporarily stores weight data. The data storage unit 901 is implemented by an SRAM or a flip-flop circuit.

[0084] Reference numeral 902 denotes a data input interface unit. The data input interface unit 902 is connected to the external storage device 2 and receives data from the external storage device 2. The connection between the data input interface unit 902 and the external storage device 2 may be established via a bus interface or the like, as long as data can be logically acquired. The data input interface unit 902 incorporates a function as a bus master, and acquires data from the external storage device 2 in response to a request transmitted from the arithmetic unit 3 via a selector unit 904.

[0085] A data output interface unit 903 is connected to three arithmetic units 3.

[0086] A selector unit 904 is a circuit that selects the destination of input data from the data input interface unit 902 and the source of requested data from the data output interface unit 903.

[0087] A selector control unit 905 is a unit that controls the selector unit 904, and controls the connection of the selector unit 904 in units of instruction execution cycles.

[0088] Reference numeral 906 denotes a read counter unit. The read counter unit 906 has a function of counting the number of accesses to the data output interface unit 903. A plurality of counters are stored internally, and via the data output interface unit 903, the reads for each address requested by the arithmetic unit 3 are counted independently for each arithmetic unit 3.

[0089] Reference numeral 907 denotes a read prediction table unit. The read prediction table unit 907 is implemented by an SRAM or a flip-flop circuit, and stores the number of reads at which the weight value in each layer is read. In this embodiment, the number of reads is stored for each arithmetic unit 3. An initial value for the predicted number is written when the system is started, and may be updated at any timing thereafter. Depending on the implementation policy, only the magnitude relationship with a previously set read number threshold may be retained.

[0090] Reference numeral 908 denotes a read prediction table control unit. The read prediction table control unit 908 acquires algorithm information designated by the calculation algorithm control unit 4, reads the predicted number of reads of weight data for the corresponding layer from the read prediction table unit 907, and issues a selector selection signal to the selector control unit 905. In addition, the read prediction table control unit 908 has the function of acquiring read counter information from the read counter unit 906 based on instructions from the calculation algorithm control unit 4, and updating the value of the read prediction table unit 907.

[0091] The operation of the data cache device 9 of the fourth embodiment conforms to the flowchart of Fig. 3. However, in step S007, the data output interface unit 103 increments the requested transfer data amount of the read counter unit 106 for each piece of map information returned from the selector control unit 105, including information on which arithmetic unit the data requested is from and which layer weight the data is assigned to.

[0092] Furthermore, in S009, the number of data reads used when comparing the number of data reads with the threshold value is the number of reads linked to the arithmetic device 3 used in the process. For example, it is assumed that the read prediction table unit 907 stores the number of reads of weight data for each layer in association with each of three arithmetic devices A, B, and C. If the arithmetic devices A and B are used but the arithmetic device C is not used in the process to be performed, the total number of reads linked to the arithmetic devices A and B is compared with the threshold value, and a flag is set for weight data whose read count is equal to or greater than the threshold value.

[0093] In this embodiment, weight data has been described as an example, but it goes without saying that the present disclosure is not limited to weight data and may also be applied to other data, such as activation data or other weight data.

[0094] Like the data cache device 1 of the first embodiment, the data cache device 9 of the fourth embodiment can also improve the computing performance while suppressing the amount of power consumption.

[0095] Furthermore, the data cache device 9 of the fourth embodiment counts the number of reads for each arithmetic unit 3 and stores the number of reads in the read prediction table unit 907, so that it can determine the weight data to be stored in the data holding unit 901 according to the arithmetic unit 3 that will execute the processing to be performed.

[0096] Although the data cache device according to the embodiment of the present disclosure has been described above, the present disclosure is not limited to the above-described embodiment. In the above embodiment, when processing of all tiles is completed ("Yes" in S008 in FIG. 3), a flag is set for data whose read count is equal to or greater than a threshold (S009), the read prediction table is updated (S010), and the data is transferred to the data storage unit based on the updated read prediction table (S011). That is, data that is effective in reducing power consumption when stored in the data storage unit 101 is determined on a frame-by-frame basis, but this unit does not necessarily have to be frame-by-frame; for example, it may be tile-by-tile. That is, when processing of one tile is completed, a flag may be set for data in a layer whose read count is equal to or greater than a threshold. This configuration is particularly effective when the tile size is a neural network, regardless of the layer.

Claims

1. A data cache device that temporarily stores digital data, A data storage unit (101) for storing data; a data input interface unit (102) for inputting data from outside the data cache device; a data output interface unit (103) for outputting data to the outside of the data cache device; a selector unit (104) that simultaneously selects one or more of a path for transferring data from the data input interface unit to the data holding unit, a path for transferring data from the data input interface unit to the data output interface unit, and a path for transferring data from the data holding unit to the data output interface unit; a selector control unit (105) for controlling the selector unit; a read counter unit (106) that counts the number of reads from the data output interface unit; a read prediction table unit (107) for holding the expected number of reads of the data when it is processed; a read prediction table control unit (108) that sets the number of reads according to an algorithm in the read prediction table unit; controlling the selector unit in accordance with the number of reads set in the read prediction table unit; the read prediction table control unit updates the read prediction table unit in accordance with an output of the read counter unit; The read counter unit counts and holds the number of times data is read from the data holding unit and the number of times data is directly read from the data input interface unit.

2. A data cache device that temporarily stores digital data, A data storage unit (101) for storing data; a data input interface unit (102) for inputting data from outside the data cache device; a data output interface unit (103) for outputting data to the outside of the data cache device; a selector unit (104) that simultaneously selects one or more of a path for transferring data from the data input interface unit to the data holding unit, a path for transferring data from the data input interface unit to the data output interface unit, and a path for transferring data from the data holding unit to the data output interface unit; a selector control unit (105) for controlling the selector unit; a read counter unit (106) that counts the number of reads from the data output interface unit; a read prediction table unit (107) for holding the expected number of reads of the data when it is processed; a read prediction table control unit (108) that sets the number of reads according to an algorithm in the read prediction table unit; controlling the selector unit in accordance with the number of reads set in the read prediction table unit; the read counter unit counts the number of reads for each output destination to which the data cache device outputs data; The read prediction table control unit updates the read prediction table unit based on the number of reads for each output destination.

3. 1. A program for controlling a data cache device comprising: a data holding unit for holding data; a data input interface unit for inputting data from outside the data cache device; a data output interface unit for outputting data to outside the data cache device; and a selector unit for simultaneously selecting one or more of a path for transferring data from the data input interface unit to the data holding unit, a path for transferring data from the data input interface unit to the data output interface unit, and a path for transferring data from the data holding unit to the data output interface unit, the program comprising: a read prediction table means for holding a predicted number of reads of the data when the data is processed; a selector control means for controlling the selector unit in accordance with the number of reads set in the read prediction table means; a read counter means for counting the number of reads from the data output interface unit; a read prediction table control means for setting the number of reads in accordance with the algorithm in the read prediction table means and updating the read prediction table means in accordance with the output of the read counter means; a program for causing a computer to function such that the read counter means counts and holds the number of times data is read from the data holding unit and the number of times data is read directly from the data input interface unit;

4. 1. A program for controlling a data cache device comprising: a data holding unit for holding data; a data input interface unit for inputting data from outside the data cache device; a data output interface unit for outputting data to outside the data cache device; and a selector unit for simultaneously selecting one or more of a path for transferring data from the data input interface unit to the data holding unit, a path for transferring data from the data input interface unit to the data output interface unit, and a path for transferring data from the data holding unit to the data output interface unit, the program comprising: a read prediction table means for holding a predicted number of reads of the data when the data is processed; a selector control means for controlling the selector unit in accordance with the number of reads set in the read prediction table means; a read counter means for counting the number of reads from the data output interface unit; a read prediction table control means for setting the number of reads in accordance with the algorithm in the read prediction table means and updating the read prediction table means in accordance with the output of the read counter means; the read counter means counts the number of reads for each output destination to which the data cache device outputs data; a program for causing a computer to function so as to update the read prediction table control means based on the number of reads for each output destination;

5. A program for creating a table for setting whether data stored in a data cache device or data stored in an external storage device is to be read as data required for an arithmetic unit to execute an arithmetic operation, the program comprising: a read counting means (702) for performing a calculation simulation of the calculation device using a calculation simulator and counting the number of times each data is read during calculation execution; a table creating means (701) for creating the table based on the number of times each data item is read by the read count means; It functions as a program for causing a computer to function such that the table creating means creates the table based on the number of times each piece of data is read and the size of a data holding unit in the data cache device;

Images