Semiconductor device and control method

Abstract

This technology provides a way to improve processing speed on the NPU. [Solution] A semiconductor device 1 that operates as a processor for machine learning comprises N (where N is an integer of 1 or more) arithmetic cores 10, 20, 30, 40, a main memory 600, at least one cache memory 810 for the N arithmetic cores, a history recording circuit 820 that stores the history of Read requests issued by the N arithmetic cores on an address-by-address basis of the main memory targeted by the Read request, and a discrimination circuit 830 that determines whether to store the target data indicated by a Write request issued by the N arithmetic cores in the cache memory or the main memory based on the history associated with the address of the target data indicated by the Write request.

Description

Technical Field

[0005] ,

[0001] The present disclosure relates to a semiconductor device and a control method.

Background Art

[0002] Patent Document 1 describes a prefetch technique in which data is read in advance into a cache memory for the purpose of improving the processing performance of arithmetic operations.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Generally, data used in the past is uniformly stored in a cache memory. The inventor of the present disclosure has found that in an NPU (Neural network Processing Unit) capable of efficiently executing processing related to a neural network, it is not efficient to uniformly store data used in the past in a cache memory due to the nature of the data to be handled. Therefore, a technique for improving the processing speed in an NPU is desired.

Means for Solving the Problems

[0005] The present disclosure can be realized in the following forms.

[0006] According to one embodiment of the present disclosure, a semiconductor device (1) that operates as a processor for machine learning is provided. The semiconductor device comprises N (where N is an integer of 1 or more) arithmetic cores (10, 20, 30, 40), a main memory (600), at least one cache memory (810) for the N arithmetic cores, a history recording circuit (820) that stores a history of Read requests issued by the N arithmetic cores in units of addresses of the main memory that are the targets of the Read requests, and a discrimination circuit (830) that determines whether to store the target data indicated by a Write request issued by the N arithmetic cores in the cache memory or the main memory based on the history associated with the address of the target data indicated by the Write request.

[0007] Due to the characteristics of the processing performed by the NPU, it is possible to some extent to differentiate between addresses in main memory that are frequently accessed and those that are not. In the above configuration, data that is determined to be stored in cache memory based on the history of read requests is stored in cache memory. Therefore, data that has been used in the past is not uniformly stored in cache memory, and the cache hit rate is likely to be improved. As a result, the time to access memory can be shortened, and the processing speed on the NPU can be improved.

[0008] In other forms of the present disclosure, a control method is provided for controlling a semiconductor device operating as a processor for machine learning. The semiconductor device comprises at least one arithmetic core, a main memory, at least one cache memory for the arithmetic core, and a specific circuit. The control method includes: a first step of incrementing a count value associated with the address to which a Read request is made by adding a predetermined number each time a Read request is made from the arithmetic core; and a second step of determining, when a Write request is made from the arithmetic core, whether to store the target data indicated by the Write request in the main memory or the cache memory, based on the count value associated with the address to which the target data indicated by the Write request is to be written. The first and second steps are performed by the specific circuit.

[0009] With the above configuration, it is easy to determine whether or not to save the target data indicated by a Write request to the cache memory based on the number of Read requests. [Brief explanation of the drawing]

[0010] [Figure 1] This is an explanatory diagram showing the schematic configuration of a semiconductor device according to an embodiment. [Figure 2] This is an explanatory diagram showing an example of an NPU configuration. [Figure 3] This is a flowchart for processing the recording of history, such as read requests. [Figure 4] This is an explanatory diagram showing an example of how the counter value changes over time. [Figure 5] This is a flowchart of the process executed when a Write request is received. [Figure 6] This is a flowchart of the data writing process to the dedicated cache. [Figure 7] This is an explanatory diagram regarding writing data to a dedicated cache. [Figure 8]This is a flowchart of the process executed when a Read request is received. [Figure 9] This is an explanatory diagram showing an example of how a machine learning model can be structured. [Figure 10] This is an explanatory diagram illustrating an example of processing in a machine learning model. [Modes for carrying out the invention]

[0011] A. Embodiments: As shown in Figure 1, the semiconductor device 1 according to this embodiment performs processing related to neural networks. For example, the semiconductor device 1 operates as a processor for machine learning. The semiconductor device 1 is composed of a System On Chip (SoC).

[0012] The semiconductor device 1 comprises an NPU (Neaural Network Processing Unit) 100, an NPU 200, a CPU (Central Processing Unit) 300, a GPU (Graphics Processing Unit) 400, a main memory 600, a main memory cache 700, a specific circuit section 800, a bus B1, and a bus B2. The main memory 600 is also called the "main memory."

[0013] NPU100 is used for processing related to neural networks. NPU100 may also perform acceleration of processing related to neural networks. Acceleration refers to speeding up the execution speed of processing for a specific purpose. The same applies to NPU200. In this embodiment, NPU100 and NPU200 access the main memory cache 700 via a specific circuit unit 800, which will be described later. NPU100 and NPU200 can exchange data with the specific circuit unit 800 via bus B2. The specific circuit unit 800 can exchange data with the main memory cache 700 via bus B1. In addition, NPU100 and NPU200 can exchange data with each other via bus B2. The following description will focus on the configuration of NPU100, but NPU200 has a similar configuration to NPU100.

[0014] As shown in Figure 2, the NPU 100 comprises a processing core 10, a processing core 20, a processing core 30, a processing core 40, and a control circuit unit 50. The number of processing cores in the NPU 100 is arbitrary. Processing cores 10, 20, 30, and 40 are also referred to as "N processing cores (where N is an integer greater than or equal to 1)". Each of the processing cores 10, 20, 30, and 40 has a MAC (Multiply Accumulation) not shown, which performs multiply-accumulate operations. The number of MACs in each processing core is arbitrary. Note that the NPU 200, CPU 300, and GPU 400 are not shown in Figure 2.

[0015] The arithmetic cores 10, 20, 30, and 40 each have a DMA (Direct Memory Access) controller not shown. In the present embodiment, the DMA controller issues a Read request or a Write request to a specific circuit unit 800 described later in order to execute data transfer. The Read request is a request for instructing reading of data from the main memory 600. The Write request is a request for instructing writing of data to the main memory 600. Further, the DMA controller executes data transfer between other arithmetic cores. The arithmetic cores 20, 30, and 40 also have the same configuration as the arithmetic core 10.

[0016] The control circuit unit 50 performs overall control of the NPU 100. For example, the control circuit unit 50 determines whether to activate each of the arithmetic cores 10, 20, 30, and 40, and supplies power to the arithmetic core determined to be activated so that the arithmetic core can operate. The control circuit unit 50 determines an arithmetic core that processes data among the arithmetic cores 10, 20, 30, and 40. Further, the control circuit unit 50 executes arithmetic processing other than the multiply-accumulate operation.

[0017] As shown in FIG. 1, the CPU 300 is a general-purpose processor. The GPU 400 is a processor specialized for image processing. The CPU 300 and the GPU 400 access the main memory cache 700 via the bus B1.

[0018] The main memory 600 is a storage device that stores data related to the processes executed by the semiconductor device 1 and is the main storage device in the semiconductor device 1. The main memory 600 is constituted by, for example, a DRAM (Dynamic Random Access Memory). The cache 700 for the main memory is arranged between a processor such as the NPU 100 or the CPU 300 and the main memory 600 in order to bridge the difference in processing speed between the processors such as the NPU 100 and the CPU 300 and the read / write speed of the main memory 600. The cache 700 for the main memory is constituted by, for example, an SRAM (Static Random Access Memory). The data in the main memory 600 that has been most recently accessed is primarily stored in the cache 700 for the main memory. The storage of data in the cache 700 for the main memory, data replacement, etc. are executed by a memory controller (not shown).

[0019] Subsequently, a characteristic configuration in the present embodiment will be described. The specific circuit section 800 includes a dedicated cache 810, a history recording circuit 820, and a storage destination determination circuit 830. The dedicated cache 810 may also be referred to as a "cache memory". The storage destination determination circuit 830 may also be referred to as a "determination circuit".

[0020] The dedicated cache 810 is a cache memory for the arithmetic cores of the NPU 100 and the NPU 200. The dedicated cache 810 is composed of, for example, SRAM. The history recording circuit 820 stores the history of Read requests issued by the arithmetic cores of the NPU 100 and the NPU 200 in predetermined address units in the main memory 600. Storing in address units includes storing data for each individual address and storing data for groups containing multiple addresses. The history recording circuit 820 is composed of an integrated circuit such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The storage destination determination circuit 830 determines whether to store the target data indicated by the Write requests issued by the arithmetic cores of the NPU 100 and the NPU 200 in the dedicated cache 810 or the main memory 600, based on the address of the main memory 600 that indicates the target data to be written to, and the history of Read requests issued by the arithmetic cores stored in the history recording circuit 820. In actual processing, data determined to be stored in the main memory 600 is first stored in the main memory cache 700, but after being stored in the main memory cache 700, it is transferred to the main memory 600 at a predetermined timing. The storage destination determination circuit 830 is composed of an integrated circuit such as an FPGA or ASIC. The specific circuit section 800 is also called the "specific circuit".

[0021] The following describes the processing performed by the history recording circuit 820. The history recording circuit 820 has a counter CT corresponding to each group into which the storage area of ​​the main memory 600 has been divided in advance by region. For example, the storage area of ​​the main memory 600 is divided into a predetermined number of addresses, such as every 100 addresses. For example, if it is divided into 100 groups, the history recording circuit 820 has 100 counter CTs. The initial value of each counter CT is set to zero. The counter CT is used to count the number of Read requests, etc. That is, the value of the counter CT represents the memory access history (history). Ideally, it would be desirable to record the memory access history for each address, but it is anticipated that the processing load would become too high if the memory access history were recorded for each address. For this reason, in this embodiment, the storage area of ​​the main memory 600 is divided into a predetermined number of addresses. The value of the counter CT is also called the "count value".

[0022] Figure 3 shows a flowchart of the process for recording the history of Read requests, etc. The process shown in Figure 3 is started when the sleep state of NPU100 or NPU200 is released and the reset of NPU100 or NPU200 is released. In step S101, the history recording circuit 820 determines whether or not a Read request has been issued by any of the multiple arithmetic cores 10 of NPU100 and / or the multiple arithmetic cores 10 of NPU200. The origin of the Read request is not considered. If a Read request has been issued (step S101; YES), the history recording circuit 820 executes the process in step S102. If a Read request has not been issued (step S101; NO), the history recording circuit 820 executes the process in step S103.

[0023] In step S102, the history recording circuit 820 increments the value of the counter CT associated with the group to which the source address specified by the Read request belongs. In this embodiment, the value of the counter CT is increased by 2 each time. Figure 4 shows an example of the change in the value of the counter CT. For example, the value of the counter CT is increased by 2 each time a Read request is made at time t2 and time t3. The process in step S102 is also called the "first step".

[0024] In step S103 shown in Figure 3, the history recording circuit 820 determines whether any of the multiple processing cores in the NPU 100 and the NPU 200 has issued a Write request. If a Write request is issued (step S103; YES), the history recording circuit 820 executes the process in step S104. If no Write request is issued (step S103; NO), the history recording circuit 820 executes the process in step S105.

[0025] In step S104, the history recording circuit 820 decrements the counter CT associated with the group to which the destination address specified by the Write request belongs. Here, the value of counter CT is decreased by 1. The value deducted from the value of counter CT when a Write request is issued is also called the "second value". Note that if the value of counter CT is zero, no decrement is performed. In the example shown in Figure 4, the value of counter CT is not decremented at the time of the first Write request at time t1, but the value of counter CT is decremented at the time of the second and subsequent Write requests, such as at time t7.

[0026] In step S105 shown in Figure 3, the history recording circuit 820 determines whether a certain time FI has elapsed. The certain time FI represents a predetermined period of time, for example, 500 milliseconds. The history recording circuit 820 determines whether a certain time FI has elapsed from the time t0 when the process shown in Figure 3 started, or whether a further certain time FI has elapsed since the previous certain time FI elapsed. If a certain time has elapsed (step S105; YES), the history recording circuit 820 executes the process in step S106. If a certain time has not elapsed (step S105; NO), the process in step S107 is executed.

[0027] In step S106, the history recording circuit 820 decrements the values ​​of all counter CTs. In this embodiment, the value of each counter CT is decreased by 3. In the example shown in Figure 4, a certain amount of time FI has elapsed at time t5 and time t11, so the value of each counter CT is decreased by 3. The value that is subtracted from the value of the counter CT each time a certain amount of time FI has elapsed is also called the "first value". Note that no decrement is performed on counter CTs whose value is zero.

[0028] In step S107 shown in Figure 3, the history recording circuit 820 determines whether or not to terminate the process. The process shown in Figure 3 is terminated if the sleep state of NPU 100 or NPU 200 is turned on, or if NPU 100 or NPU 200 is reset. If the process is not terminated (step S107; NO), the history recording circuit 820 executes the process from step S101 onwards again. If the process is terminated (step S107; YES), the process shown in Figure 3 is terminated. Note that all counter CTs may be reset when the process is terminated.

[0029] The history recording circuit 820 performs the process shown in Figure 3, adding a value corresponding to the number of times a Read request has been issued for each address group, thereby recording the trend of Read requests as a history.

[0030] Next, we will explain the process executed by the storage destination determination circuit 830 when it receives a Write request from any of the processing cores. When either the NPU100 or NPU200 processing core issues a Write request, the process shown in Figure 5 begins. As shown in Figure 5, in step S201, the storage destination determination circuit 830 reads the value of counter CT corresponding to the group to which the destination address specified in the Write request belongs. In step S202, the storage destination determination circuit 830 assigns points to the group using the value of counter CT. The points are used to determine the storage destination of the target data specified in the Write request.

[0031] In step S203, the storage destination determination circuit 830 determines whether the score is above a predetermined threshold. If the score is above the threshold (step S203; YES), the storage destination determination circuit 830 executes the process in step S204. If the score is below the threshold (step S203; NO), the storage destination determination circuit 830 executes the process in step S205. The process in step S203 for determining the storage destination is also called the "second step".

[0032] In step S204, the destination determination circuit 830 writes the target data specified by the Write request to the dedicated cache 810. A score above a threshold indicates that there is a high tendency for Read requests to be issued for the group to which the destination address specified by the Write request belongs, i.e., that the reference frequency is high. By storing data at such addresses in the dedicated cache 810, it is not necessary to access main memory 600 when a Read request is issued. Details of the writing process to the dedicated cache 810 will be described later. Also, if there is no free space in the dedicated cache 810, data that meets predetermined conditions for freeing up space in the dedicated cache 810 will be written back to main memory 600. For example, data from the address group with the smallest counter CT value is selected as the target for writing back.

[0033] In step S205, the destination determination circuit 830 transfers the target data specified by the Write request to the main memory cache 700 to the destination address in the main memory 600 specified by the Write request. After the target data is written to the main memory cache 700, the target data is transferred to the main memory 600. A score below the threshold indicates that there is a low tendency for Read requests to be issued for the group to which the destination address specified by the Write request belongs, i.e., the reference frequency is low. Since it is considered that there is little need to store data at such addresses in the dedicated cache 810, the data is transferred to the main memory 600. Since the data transfer to the main memory 600 is a known technique, a detailed explanation of the process is omitted. After step S204 or step S205 is executed, the process shown in Figure 5 is completed.

[0034] In this embodiment, it is possible to easily determine whether or not to save the target data indicated by a Write request in the dedicated cache 810 based on the points assigned according to the number of Read requests issued.

[0035] Next, we will explain the data saving process to the dedicated cache 810 in step S204 of Figure 5. As shown in Figure 6, in step S301, the storage destination determination circuit 830 writes the target data specified by the write request to the dedicated cache 810. Here, in the dedicated cache, data is managed in predetermined units, similar to a general cache memory.

[0036] Generally, cache memory loads data from main memory in predetermined data units. For example, data from multiple consecutive addresses, starting with a specified address, is loaded into the cache memory in predetermined data units. This data unit is called a cache line or simply a line. Address information from main memory is also stored in the cache memory. Address information stored in the cache memory is called a cache tag or simply a tag. Multiple cache lines are called a set.

[0037] In step S302, the storage destination determination circuit 830 determines whether the target data written to the dedicated cache in step S301 is the same size as the cache line. If the target data written to the dedicated cache is the same size as the cache line (step S302; YES), the process shown in Figure 6 is completed. If the target data written to the dedicated cache is not the same size as the cache line (step S302; NO), the storage destination determination circuit 830 executes the process in step S303.

[0038] For example, suppose that in main memory 600, each byte is assigned an address. Also, suppose that the size of one cache line is 8 bytes. As shown in Figure 7, suppose that the destination specified in the Write request is a non-contiguous address such as 0x00, 0x01, 0x03, 0x05, and 0x07. Also, suppose that the target data is written to the dedicated cache 810 based on the score assigned to the group to which the destination address belongs. In this case, the data stored at 0x02, 0x04, and 0x06 in main memory 600 is insufficient in the cache line formed by the data written to the dedicated cache 810. In this case, in step S302, the storage destination determination circuit 830 determines that the written target data is not the same size as the cache line.

[0039] In step S303 shown in Figure 6, the storage destination determination circuit 830 issues a Read request to the main memory cache 700 to request data for the corresponding cache line. This is to write any missing data from the dedicated cache 810 to the dedicated cache 810. Data is requested on a cache line basis. If the main memory cache 700 does not contain the corresponding data, the data is written from the main memory 600 to the main memory cache 700, and then the data is transferred from the main memory cache 700 to the storage destination determination circuit 830.

[0040] In step S304, the storage destination determination circuit 830 writes the cache lines transferred from the main memory cache 700 that correspond to the missing data to the dedicated cache 810. In the example shown in Figure 7, the data stored in 0x02, 0x04, and 0x06 in the main memory 600 is written to the dedicated cache 810. Therefore, as shown in Figure 7, all data is written to the line. This completes the data saving process to the dedicated cache in step S204 of Figure 5. At this point, the data in the source main memory 600 does not match the data in the cache line, but when certain conditions are met, the cache line will be written back to the main memory 600. For example, when the cache line becomes a cache line with low usage frequency, the cache line is deleted from the dedicated cache 810. At this timing, the cache line will be written back to the main memory 600. A known algorithm can be used to manage the cache memory.

[0041] Next, referring to Figure 8, we will explain the process executed by the storage destination determination circuit 830 when it receives a Read request from any of the processing cores. Note that the process shown in Figure 8 is the same as a normal memory access to the cache memory. When either the NPU100 or NPU200 processing core issues a Read request, the process shown in Figure 8 begins. In step S401, the storage destination determination circuit 830 determines whether the data at the source address specified by the Read request is in the dedicated cache 810. When the data at the source address specified by the Read request is in the cache memory, it is called a cache hit. In the case of a cache hit, access to the main memory 600 is not necessary. When the data at the source address specified by the Read request is not in the dedicated cache 810, it is called a cache miss or cache miss hit.

[0042] If a cache hit occurs (step S401; YES), the storage destination determination circuit 830 executes the process in step S404. If a cache hit does not occur, i.e., if a cache miss occurs (step S401; NO), the storage destination determination circuit 830 executes the process in step S402.

[0043] In step S402, the destination determination circuit 830 requests the data for the source address specified by the Read request from the main memory cache 700 in order to write the data for the source address specified by the Read request to the dedicated cache 810. In response, data is transferred from the main memory cache 700 to the destination determination circuit 830. If the data is not found in the main memory cache 700, the data is written from the main memory 600 to the main memory cache 700, and then the data is transferred from the main memory cache 700 to the destination determination circuit 830. In step S403, the destination determination circuit 830 writes the transferred cache line to the dedicated cache 810.

[0044] In step S404, the storage destination determination circuit 830 reads the data at the source address specified by the Read request from the dedicated cache 810 and sends the read data to the processing core that sent the Read request. After that, the process shown in Figure 8 is completed.

[0045] Traditionally, the NPU accessed main memory 600 or main memory cache 700 via NPU-specific buses B2 and B1. As mentioned above, other devices such as CPU300 and GPU400 also access main memory 600 or main memory cache 700 via bus B1. This increased traffic on bus B1, leading to congestion. This resulted in data transfer delays, affecting the processing performance of NPU100, NPU200, CPU300, and GPU400.

[0046] Here, based on the characteristics of the processing performed by the NPU, it is possible to some extent to separate the addresses in main memory 600 into those that are frequently accessed and those that are not. As mentioned above, semiconductor device 1 operates as a processor for machine learning. For example, NPU100 and NPU200 perform machine learning processing to create a CNN (Convolutional Neural Network), which is one type of machine learning model.

[0047] Figure 9 shows an example of the configuration of a machine learning model M, which is a CNN. Figure 10 shows an explanatory diagram of an example of processing in machine learning model M. Here, we will explain an example in which an image is input to machine learning model M. Machine learning model M includes an input layer L10, an intermediate layer L20, and an output layer L30. The intermediate layer L20 has multiple convolutional layers that extract features of the input data IM. The parameter values ​​for each layer are examples.

[0048] The hidden layer L20 includes the first convolutional layer L21 and the second convolutional layer L22. In the convolutional layers of a CNN, the kernel is applied to the data output from the previous layer while moving it, and features of the data are extracted. The distance the kernel is moved is called the stride. The kernel is the coefficient matrix used to perform the convolution operation. The kernel is sometimes called a filter.

[0049] The first convolutional layer L21 processes data output from the input layer L10. In the illustrated example, the input data IM is input to the first convolutional layer L21. The input data IM has a size of 128 × 128 pixels. The input data IM is a grayscale image and contains information for one channel. The first convolutional layer L21 performs convolution on the input data IM using 64 kernels. The size of all 64 kernels is 3 × 3. The stride of all 64 kernels is 1. Therefore, each time the convolution process for each kernel is executed, the input data IM is read from the main memory cache 700 or main memory 600. In this case, it is expected that the frequency of memory access to the area where the input data IM is stored will be high.

[0050] The processing target of the second convolutional layer L22 is the intermediate data MD1 output from the first convolutional layer L21. In the illustrated example, the first convolutional layer L21 outputs intermediate data MD1 with a size of 128 × 128 pixels and 64 channels. In the second convolutional layer L22, convolution processing is performed on the intermediate data MD1 using 128 kernels. The size of all 128 kernels is 3 × 3. The stride of all 128 kernels is 2. Therefore, each time the convolution processing of each kernel is executed, the intermediate data MD1 will be read from the main memory cache 700 or main memory 600. In this case, it is expected that the frequency of memory access to the area where the intermediate data MD1 is stored will be high.

[0051] Intermediate data MD2 is output from the second convolutional layer L22. In the illustrated example, intermediate data MD2 with a size of 64 x 64 pixels and 128 channels is output from the second convolutional layer L22.

[0052] Figure 10 shows an explanatory diagram illustrating an example of a skip connection configuration. The first convolutional layer L21 outputs intermediate data MD1. Intermediate data MD1 is input to the second convolutional layer L22. The second convolutional layer L22 outputs intermediate data MD2. Here, the output of the second convolutional layer L22 is added to the input to the second convolutional layer L22, i.e., intermediate data MD1, via the skip connection. Therefore, the third convolutional layer L23 receives the sum of the intermediate data MD2, which is the output of the second convolutional layer L22, and the intermediate data MD1, which is the input to the second convolutional layer L22. In this case, it is expected that the frequency of memory access to the area where the intermediate data MD1 is stored will be higher compared to when there is no skip connection.

[0053] Thus, given the characteristics of the processing performed by the NPU, it is thought that it is possible to some extent to separate frequently accessed addresses from less frequently accessed addresses within main memory 600.

[0054] According to this embodiment, the data stored in the dedicated cache 810 is data that has been determined to be stored in the cache memory based on the history of read requests. Compared to an embodiment in which recently accessed data is uniformly stored in the cache memory, there is a higher possibility of improving the cache hit rate. Therefore, the memory access time can be shortened and the processing speed on the NPU can be improved.

[0055] In this embodiment, it is possible to easily determine whether or not to save the target data indicated by a Write request to the cache memory based on a score representing the memory access history. By saving frequently accessed data in the dedicated cache 810, there is no need to wait for data transfer from the main memory 600 when a Read request is issued from the NPU. Therefore, the overall processing speed of the semiconductor device 1 is improved.

[0056] If the value of counter CT were simply incremented each time a read request was made, it is expected that data at addresses that were frequently accessed in the past and data at addresses that have been frequently accessed recently would have the same score. Data at addresses that have been frequently accessed within the time interval including the present moment are considered likely to be accessed again from the present moment onward. Data at addresses that have been infrequently accessed within the time interval including the present moment are considered unlikely to be accessed again from the present moment onward. Therefore, to avoid the former and the latter having the same score, the value of counter CT is decremented after a certain period of time has elapsed. In this way, the recent trend of memory access can be reflected in the score.

[0057] When a write request is issued from the NPU, the data targeted by the write request is updated. If a counter CT is provided for each address, then the counter CT can be reset. However, in this embodiment, a counter CT is provided for each group. When the data targeted by the write request is updated, it means that at least some of the addresses in the group to which the address of the data targeted by the write request belongs have been overwritten. Therefore, instead of resetting the counter CT, a certain value is deducted from the value of the counter CT. In this way, the recent trend of memory access can be reflected in the score.

[0058] In this embodiment, the history recording circuit 820 and the storage destination determination circuit 830 are configured using integrated circuits such as FPGAs and ASICs. The functions of determining whether or not to write to the dedicated cache 810 and performing the writing process are implemented in hardware. Therefore, compared to a configuration in which these functions are controlled by software, it is possible to achieve faster processing, reduced power consumption, and reduced implementation difficulty.

[0059] B. Other embodiments: (B1) The method of scoring is not limited to the method described in the above embodiment. As shown in Figure 2, if the NPU 100 has two or more processing cores, scores may be assigned using the priority set in advance for each processing core. When scoring in step S202 of Figure 5, the score may be obtained by adding a value that is larger for each processing core that issued the Write request to the value of counter CT associated with the group to which the address to be written specified by the Write request belongs, with the value being larger for each processing core that issued the Write request. Each processing core is set to one of the following priority levels: "highest priority", "priority", or "normal". For example, if the priority of the processing core that issued the Write request is set to "highest priority", 2 is added to the value of counter CT. If the priority of the processing core that issued the Write request is set to "priority", 1 is added to the value of counter CT. If the priority of the processing core that issued the Write request is set to "normal", no value is added to the value of counter CT.

[0060] Therefore, when each processing core has a priority assigned to it, it becomes easier to use the score as an indicator to correctly evaluate the address read history.

[0061] (B2) Furthermore, points may be assigned according to the burst length indicated by the Write request. When the DMA controller transfers data in burst mode, the value of counter CT associated with the group to which the address to be written specified by the Write request belongs may be increased by a value corresponding to the burst length indicated by the Write request, and this increase may be used as the score. The increased value is set to be larger the larger the burst length. The burst length indicates the data unit when transferring data in burst mode. For example, suppose semiconductor device 1 supports three types of burst lengths. If the burst length indicated by the Write request is the largest burst length, 2 is added to the value of counter CT. If the burst length indicated by the Write request is the next largest burst length, 1 is added to the value of counter CT. If the burst length indicated by the Write request is the smallest burst length, no value is added to the value of counter CT. Burst mode is also called burst transfer mode. The value added according to the burst length is also called the "third value".

[0062] When transferring data with a large burst length to the main memory 600, it is expected that the bandwidth of bus B1 will become limited. Therefore, it is preferable from the viewpoint of controlling the bandwidth of bus B1 to store data with a large burst length in a dedicated cache 810. In the above embodiment, since data with a large burst length can be stored in the cache memory, it is possible to avoid putting pressure on the bandwidth of bus B1.

[0063] (B3) Scoring may be performed by combining other embodiments (B1) and (B2).

[0064] (B4) When determining the destination to write the target data specified by the Write request, the threshold value used for comparison with the score may be changed according to the operating status of the multiple processing cores provided by the NPU. When the processing load on the NPU is low, it is expected that the utilization rate of some processing cores will decrease. When the utilization rate of the processing cores is low, it is expected that the utilization rate of the dedicated cache 810 will decrease. In such cases, even if the threshold value is increased, it is expected that processing can continue without flushing out the data in the dedicated cache 810. By changing the threshold value according to the operating status of the processing cores, the dedicated cache 810 can be used efficiently.

[0065] (B5) Read and write requests are issued by the arithmetic core while the arithmetic core is performing machine learning-related processing, and the history recording circuit and discrimination circuit may be configured to operate to process Read and write requests while the arithmetic core is performing machine learning-related processing. Machine learning-related processing is, for example, the process of performing machine learning using training data to generate a machine learning model. In machine learning-related processing, it is possible to sort addresses in main memory into those that are accessed frequently and those that are not, so the score can be used as an indicator to correctly evaluate the address read history.

[0066] (B6) In the embodiment, an example was described in which the counter CT is counted down each time a certain period of time has elapsed. However, it is not necessary to count down the counter CT each time a certain period of time has elapsed.

[0067] (B7) In the embodiment, an example was described in which the counter CT is counted down when a Write request is issued from the computing core. However, it is not necessary to count down the counter CT when a Write request is issued.

[0068] (B8) In the embodiment described, an example was given in which the semiconductor device 1 is equipped with one dedicated cache 810, but the embodiment is not limited to this. For example, if the semiconductor device 1 is equipped with multiple clusters, a dedicated cache may be provided for each cluster. A cluster is a collection of computing cores.

[0069] (B9) In the embodiment described, an example was given in which NPU100 and NPU200 have the same configuration, but NPU100 and NPU200 may have different configurations. For example, NPU100 and NPU200 may perform data transfer without using a DMA controller.

[0070] (B10) The semiconductor device according to this disclosure is not limited to the example of the semiconductor device 1 of the embodiment, and may have one or more arithmetic cores, main memory, at least one cache memory, history recording circuit, and discrimination circuit. Therefore, the semiconductor device 1 may not have all or part of the CPU 300, GPU 400, and main memory cache 700. The semiconductor device 1 may have only one of the NPU 100 and NPU 200. Also, the semiconductor device 1 may have three or more NPUs.

[0071] This disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from its spirit. For example, the technical features in the embodiments corresponding to the technical features in each form described in the summary of the invention can be replaced or combined as appropriate in order to solve some or all of the above-described problems, or to achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate. [Explanation of symbols]

[0072] 1...Semiconductor device, 10, 20, 30, 40...Arithmetic core, 50...Control circuit unit, 100, 200...NPU, 300...CPU, 400...GPU, 600...Main memory, 700...Cache for main memory, 800...Specific circuit unit, 810...Dedicated cache, 820...History recording circuit, 830...Storage destination determination circuit, B1, B2...Bus, CT...Counter.

Claims

1. A semiconductor device that operates as a processor for machine learning, N processing cores (where N is an integer greater than or equal to 1), Main memory and At least one cache memory for the N computing cores, A history recording circuit that stores the history of Read requests issued by the N processing cores on an address-by-address basis in the main memory that is the target of the Read request, A discrimination circuit that determines whether to store the target data indicated by the Write requests issued by the N processing cores in the cache memory or the main memory, based on the history associated with the address of the target data indicated by the Write requests, Equipped with, Semiconductor equipment.

2. A semiconductor device according to claim 1, The history recording circuit increments the count value associated with the address targeted by the Read request by adding a predetermined number each time the Read request is issued. The discrimination circuit determines that if the score based on the count value associated with the address to which the target data indicated by the Write request is to be written is equal to or greater than a predetermined threshold, the target data indicated by the Write request should not be saved to the main memory but to the cache memory. Semiconductor equipment.

3. A semiconductor device according to claim 2, The history recording circuit counts down the count value by deducting a predetermined first value each time a certain amount of time has elapsed. Semiconductor equipment.

4. A semiconductor device according to claim 3, When a Write request is issued, the history recording circuit counts down the count value by subtracting a predetermined second value from the count value associated with the address indicating the area to be written specified by the Write request. Semiconductor equipment.

5. A semiconductor device according to any one of claims 2 to 4, N is an integer greater than or equal to 2, Each of the N processing cores has a predetermined priority. When a Write request is issued, the discrimination circuit adds a value that increases with the priority set for one of the N processing cores that issued the Write request to the score associated with the address indicating the area to be written as specified by the Write request. Semiconductor equipment.

6. A semiconductor device according to claim 5, When a Write request is issued, the discrimination circuit adds a predetermined third value, according to the burst length indicated by the Write request, to the score associated with the address indicating the area to be written specified by the Write request. Semiconductor equipment.

7. A semiconductor device according to claim 6, While the aforementioned computing core is performing machine learning-related processing, The computing core issues the Read request and the Write request, The history recording circuit stores the history of the Read requests. The discrimination circuit determines, based on the history, whether to store the target data indicated by the Write request in the cache memory or the main memory. Semiconductor equipment.

8. A semiconductor device according to claim 7, The threshold is changed according to the operating status of two or more of the computing cores. Semiconductor equipment.

9. A control method for controlling a semiconductor device that operates as a processor for machine learning, The semiconductor device comprises at least one arithmetic core, a main memory, at least one cache memory for the arithmetic core, and a specific circuit. The first step involves incrementing a predetermined number each time a Read request is issued from the computing core, thereby increasing the count value associated with the address targeted by the Read request. When a Write request is issued from the computing core, a second step is to determine whether to store the target data indicated by the Write request in the main memory or the cache memory, based on the count value associated with the address to which the target data indicated by the Write request is to be written. Includes, The first and second steps are performed by the specific circuit. Control method.

Images