Processing system

Abstract

In a processing system with multiple processors, a shared cache memory is allocated. [Solution] The processing systems 100 and 100a that process tasks include a plurality of processors 10 and 20, a shared cache memory 30 shared by the plurality of processors, a resource manager 50 that allocates, for each task to be executed, an execution processor from among the plurality of processors to execute the task, and a memory area to be used in the shared cache memory, and a first profile storage unit 60 that stores a cache partitioning profile 61 in which area-related information, which is information about the memory area to be used, is set for each type of task. The resource manager uses the area-related information according to the type of task to be executed in the cache partitioning profile to perform the allocation.

Description

Technical Field

[0001] This disclosure relates to a processing system.

Background Art

[0002] Conventionally, various systems have been considered as processing systems for processing tasks. The processing system of Patent Document 1 implements two types of memories having different latency characteristics, and allocates a memory having high latency characteristics to a task (application) with a low cache hit rate, and allocates a memory having low latency characteristics to a task with a high cache hit rate.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, processing systems equipped with multiple types of processors such as GPUs (Graphics Processing Units) and NPUs (Neural network Processing Units) in addition to CPUs (Central Processing Units) have been used. Regarding the mode of sharing cache memory in such a multi-processor configuration, it has not been sufficiently studied. For example, since the processing system of Patent Document 1 is a technology premised on a single processor in the first place, the task allocation in the case of sharing cache memory by multiple processors has not been studied at all. Therefore, in a processing system equipped with multiple processors, a technology for allocating shared cache memory is desired.

Means for Solving the Problems

[0005] As one embodiment of this disclosure, a processing system (100, 100a) for processing tasks is provided. This processing system comprises a plurality of processors (10, 20), a shared cache memory (30) shared by the plurality of processors, a resource manager (50) that performs the allocation of an execution processor from among the plurality of processors to execute a task, and a used memory area which is a memory area to be used in the shared cache memory, for a task to be executed, and a first profile storage unit (60) that stores a cache partitioning profile (61) in which area-related information, which is information about the used memory area, is set for each type of task. The resource manager performs the allocation using the area-related information according to the type of task to be executed in the cache partitioning profile.

[0006] This type of processing system includes a resource manager that performs the allocation of execution processors and memory areas, and a first profile storage unit that stores cache partitioning profiles in which area-related information is set for each type of task. The resource manager uses the area-related information corresponding to the type of task to be executed in the cache partitioning profile to perform the allocation, so that it can allocate memory areas in the shared cache memory corresponding to the type of task to be executed for the task to be executed. [Brief explanation of the drawing]

[0007] [Figure 1] This block diagram shows a schematic configuration of a processing system as one embodiment of the present disclosure. [Figure 2] This is an explanatory diagram showing an example of the settings for a cache partitioning profile. [Figure 3] This is an explanatory diagram showing an example of the settings for a cache hit rate profile. [Figure 4] This is a flowchart showing the procedure for resource allocation processing in the first embodiment. [Figure 5]This is an explanatory diagram showing the allocation pattern determined as a result of the resource allocation process in the first embodiment. [Figure 6] This block diagram shows the schematic configuration of the processing system according to the second embodiment. [Figure 7] This is a flowchart showing the procedure for resource allocation processing in the second embodiment. [Modes for carrying out the invention]

[0008] A. First Embodiment: A1. System configuration: The processing system 100 shown in Figure 1 processes the input tasks. Examples of such tasks include tasks performed in applications that process image data and tasks performed in machine learning-based object recognition applications.

[0009] The processing system 100 comprises a first processor 10, a second processor 20, a shared cache memory 30, a first memory 41, a second memory 42, a resource manager 50, a first profile storage unit 60, and a second profile storage unit 70. In the processing system 100, each of the above-mentioned components is configured to communicate with each other via an internal bus schematically shown by thick arrows in Figure 1. In this embodiment, the processing system 100 is configured as a monolithic SOC (System On Chip). Alternatively, the processing system 100 may be configured as a so-called chiplet in which each component is mounted on an interposer.

[0010] The first processor 10 and the second processor 20 are configured to handle the same tasks. Examples of the two processors 10 and 20 include a CPU, a GPU, and an NPU.

[0011] The shared cache memory 30 is memory that is used by the two processors 10 and 20 as secondary or tertiary cache memory when executing tasks. The shared cache memory 30 is shared by the two processors 10 and 20 as a so-called last-level cache (LLC).

[0012] The first memory 41 and the second memory 42 are memories used by the two processors 10 and 20 when executing tasks. The two memories 41 and 42 are, for example, high-bandwidth memory (HBM) and DRAM (Dynamic Random Access Memory). In this embodiment, the second memory 42 has lower latency than the first memory 41. In other words, the first memory 41 has higher latency than the second memory 42. The second memory 42 is composed of, for example, low-latency high-bandwidth memory (HBM). The first memory 41 is composed of, for example, normal HBM.

[0013] The resource manager 50, by executing the resource allocation process described later, allocates the processor, memory, and memory area (hereinafter referred to as "used memory area") to be used in the shared cache memory 30 for the input task.

[0014] The first profile storage unit 60 stores the cache partitioning profile 61. The cache partitioning profile 61 contains information about the memory area used in the shared cache memory 30 (hereinafter referred to as "area-related information") for each type of task to be executed.

[0015] As shown in Figure 2, the cache partitioning profile 61 of this embodiment has information regarding the size of the memory area used (hereinafter referred to as size-related information) set for each type of task. Specifically, for example, "Task A" has a large memory area used size set as size-related information. Similarly, for example, "Task B" has a small memory area used size set as size-related information, and "Task C" also has a large memory area used size set as size-related information. In this embodiment, a large memory area used size corresponds to 60% of the memory size of the shared cache memory 30, and a small memory area used size corresponds to 40% of the memory size of the shared cache memory 30. Such settings are determined by, for example, conducting experiments in which the memory area used size in the shared cache memory 30 is set in various patterns, and the processing speed (time required to complete the application) is measured when a task is executed in each pattern, and then setting the memory area used size in the pattern that results in the highest processing speed. The 60% and 40% mentioned above are merely examples of experimental results. Note that the size-related information may be any information relating to the size of the memory area used, such as "60%" or "40%", instead of the aforementioned "large" or "small".

[0016] Although not shown in Figure 2, in this embodiment, the processor that executes a task (hereinafter referred to as the "execution processor") is pre-configured according to size-related information (size of the memory area used). Specifically, the first processor 10 is configured for "large" (60%) memory area sizes. Conversely, the second processor 20 is configured for "small" (40%) memory area sizes. This configuration is determined by conducting experiments in advance for each type of task and each memory area size, in which the first processor 10 and the second processor 20 each execute the task, and the processor that achieves higher processing speed is identified and configured accordingly. The fact that the processing system 100 is configured for "large" memory area sizes and the second processor 20 is configured for "small" memory area sizes is merely one example of experimental results.

[0017] As described above, in the cache split profile 61, size-related information and information regarding the execution processor are set for each type of task. Therefore, in the present embodiment, the area-related information includes the size-related information and the information regarding the execution processor.

[0018] The second profile storage unit 70 shown in FIG. 1 stores a cache hit rate profile 71. In the cache hit rate profile 71, information regarding the level of the cache hit rate (hereinafter referred to as "cache hit rate-related information") is set for each type of task to be executed.

[0019] As shown in FIG. 3, in the cache hit rate profile 71 of the present embodiment, a cache hit rate is set for each type of task. Specifically, for example, a cache hit rate of "80%" is set for "Task A". Also, for example, a cache hit rate of "92%" is set for "Task B", and a cache hit rate of "56%" is set for "Task C". Note that the cache rate-related information may be any information regarding the level of the cache hit rate, such as information indicating "high" when the cache hit rate is equal to or higher than a predetermined threshold and "low" when the cache hit rate is less than the threshold, instead of the above-described specific cache hit rates. Such cache hit rate-related information may be set in advance by conducting an experiment to measure the cache hit rate for each type of task and obtaining the average value of the cache hit rate or the like.

[0020] A2. Resource Allocation Process: The resource allocation process shown in FIG. 4 is executed by the resource manager 50 when an application is executed and an interrupt for task execution is input. In step S105, the resource manager 50 determines whether the task to be executed is latency-sensitive. "Latency-sensitive" means that the latency has a great impact on the processing speed. Specifically, in the present embodiment, the resource manager 50 refers to the cache hit rate profile 71, and when the cache hit rate of the task is lower than a predetermined threshold, it determines that the latency has a great impact on the processing speed (that is, it is sensitive). This is because when the cache hit rate is low, memory access occurs frequently, and the latency of that memory access has a great impact on the processing speed. For example, when the above-mentioned threshold is "70%", for tasks A and B shown in FIG. 3, it is determined that they are "not latency-sensitive". On the other hand, for task C, it is determined that it is "latency-sensitive". Hereinafter, "step S" will be simply described as "S".

[0021] When it is determined that the task is latency-sensitive (S105: YES), the resource manager 50 allocates the second memory 42 with low latency for the execution of the task (S115). On the other hand, when it is determined that the task is not latency-sensitive (S105: NO), the resource manager 50 allocates the first memory 41 with high latency for the execution of the task (S110).

[0022] After the completion of S110 or S115, the resource manager 50 identifies the processing loads of the first processor 10 and the second processor 20, and determines whether such loads are below a preset threshold (S120).

[0023] If the processing load is not below the threshold, in other words, if it is determined that the processing load is greater than the threshold (S120: NO), the resource manager 50 determines whether the priority of the task to be executed is higher than the task currently running on the processor with the lowest load among the two processors 10 and 20 (S155). If it is determined that the priority of the task to be executed is higher (S155: YES), the resource manager 50 performs preemption of the running task (S160). On the other hand, if it is determined that the priority of the task to be executed is not higher (S155: NO), the resource manager 50 registers the task to be executed in the queue (S165). If the processing load of both processors 10 and 20 is greater than the threshold, it is difficult to execute a new task. However, even in such cases, if the priority of the task to be executed is higher than the task currently running on the processor with the lower processing load, preemption of the running task is performed, allowing the input task to be executed on that processor. Furthermore, if a task to be executed has a low priority, it is added to a queue, allowing the task to be executed when the processing load is low, without unintentionally stopping ongoing tasks.

[0024] In S120 described above, if it is determined that the processing load is below a threshold (S120: YES), the resource manager 50 determines whether the task to be executed is sensitive to the cache size (S125). "Sensitive to cache size" means that the size of the memory area used by the shared cache memory 30 has a significant impact on the processing speed. Specifically, in this embodiment, the resource manager 50 refers to the cache partitioning profile 61 and determines that the task is "sensitive to cache size" if the size of the memory area used by the task is "large". On the other hand, it determines that the task is "not sensitive to cache size" if the size of the memory area used by the task is "small". Therefore, for example, tasks A and C shown in Figure 2 are determined to be "sensitive to cache size". On the other hand, task B is determined to be "not sensitive to cache size".

[0025] If it is determined that the task is sensitive to cache size (S125:YES), the resource manager 50 allocates a predetermined large area in the shared cache memory 30 to the task, that is, a predetermined area where the memory size is 60%, and also assigns the first processor 10, which is a processor corresponding to a "large" size of memory area used, as the execution processor (S130).

[0026] On the other hand, if it is determined that the task is not sensitive to the cache size (S125: NO), the resource manager 50 allocates a predetermined small area in the shared cache memory 30 to the task, that is, a predetermined area where the memory size is 40%, and also allocates the second processor 20, which is the processor corresponding to the "small" size of the memory area used, as the execution processor (S135).

[0027] After the completion of S130 or S135, the resource manager 50 determines whether the task to be executed has a higher priority than the tasks currently running on the assigned processor (S140). If it is determined that the task to be executed has a higher priority (S140: YES), the resource manager 50 performs preemption of the running tasks (S145). On the other hand, if it is determined that the task to be executed does not have a higher priority (S140: NO), the resource manager 50 registers the task to be executed in a queue (S150). If the task to be executed has a higher priority than the tasks currently running on the assigned processor, preemption of the running tasks is performed, allowing the input task to be executed on that processor. If the task to be executed has a lower priority, it is registered in a queue, allowing the input task to be executed when the processing load is low without unnecessarily stopping tasks currently running on the assigned processor.

[0028] After the completion of steps S145, S150, S160, or S165 described above, the resource allocation process terminates. As shown in Figure 5, as a result of the resource allocation process described above, the allocation pattern of the two processors 10, 20, the shared cache memory 30, and the two memory locations 41, 42 when processing the incoming task will be one of the first to fourth patterns. By executing the resource allocation process described above, the processing system 100 selects an appropriate pattern from these allocation patterns depending on whether the task is latency sensitive or cache size sensitive.

[0029] According to the processing system 100 of the first embodiment described above, it includes a resource manager 50 that performs allocation between the execution processor and the memory area to be used, and a first profile storage unit 60 that stores a cache partitioning profile 61 in which area-related information is set for each type of task. The resource manager 50 performs the allocation using the area-related information corresponding to the type of task to be executed in the cache partitioning profile 61, so that it can allocate a memory area in the shared cache memory 30 corresponding to the type of task to be executed for the task to be executed.

[0030] Furthermore, the resource manager 50 uses information about the size of the memory area to be used according to the type of task to be executed to identify the memory area to be used and the processor to be executed, so that it can allocate an appropriate size of memory area and processor to the task to be executed according to the type of task to be executed.

[0031] Furthermore, when the cache hit rate is relatively low depending on the type of task to be executed, the second memory 42, which has relatively low latency, is allocated to the task to be executed, thus preventing an excessive decrease in the processing speed of task execution even when the cache hit rate is low. Also, when the cache hit rate is relatively high depending on the type of task to be executed, the first memory 41, which has relatively high latency, is allocated to the task to be executed, thus reducing costs compared to a configuration where all memory is low latency memory.

[0032] Furthermore, when the processing load of the two processors 10 and 20 is below a threshold, the system allocates either the first memory 41 or the second memory 42 to the task to be executed. This avoids the unnecessary allocation of the first memory 41 or the second memory 42 when both processors 10 and 20 are under heavy load and the task cannot be executed. Additionally, if the processing load of the two processors 10 and 20 is greater than a threshold, and the priority of the task to be executed is higher than the priority of a task currently running on the processor with the lowest processing load among the two processors 10 and 20, the system preempts the task currently running on that processor, increasing the likelihood that the task to be executed will be executed on the processor with the lowest load. Furthermore, if the load of the two processors is greater than a threshold, and the priority of the task to be executed is less than or equal to the priority of a task currently running on the processor with the lowest processing load among the multiple processors, the system saves the task to be executed to a queue. This avoids the unnecessary initiation of execution when there is a high probability that the task cannot be executed.

[0033] Furthermore, after an execution processor is allocated, the resource manager 50 preempts the task currently running on the execution processor if the priority of the task to be executed is higher than the priority of the task currently running on the execution processor, thereby increasing the likelihood that the task to be executed will be executed on the execution processor. Also, if the priority of the task to be executed is lower than or equal to the priority of the task currently running on the execution processor, the task to be executed is saved to a queue, thus avoiding the unnecessary initiation of execution when there is a high probability that the task to be executed cannot be executed.

[0034] B. Second Embodiment: The processing system 100a of the second embodiment shown in Figure 6 differs from the processing system 100 of the first embodiment shown in Figure 1 in that the first memory 41, the second profile storage unit 70, and the cache hit rate profile 71 are omitted. The other components of the processing system 100a are the same as those of the processing system 100, so the same components are denoted by the same reference numerals, and their detailed descriptions are omitted.

[0035] In the processing system 100a of the second embodiment, only the second memory 42 is provided as memory. Therefore, regardless of the type of task, the second memory 42 is allocated to the task to be executed.

[0036] The task assignment process of the second embodiment shown in Figure 7 differs from the task assignment process of the first embodiment shown in Figure 4 in that steps S105 to S115 are omitted. The other steps in the task assignment process of the second embodiment are the same as those of the first embodiment, so the same reference numerals are used for the same steps, and their detailed explanation is omitted.

[0037] In the second embodiment, steps S105 to S115 are not executed. This is because, as described above, the second memory 42 is allocated to the task to be executed regardless of the type of task.

[0038] The processing system 100a of the second embodiment described above has the same effects as the processing system 100 of the first embodiment.

[0039] C. Other embodiments: (C1) In each of the above embodiments, the execution processor was pre-configured in the cache partitioning profile 61 according to the size of the memory area used, but the disclosure is not limited thereto. In addition to the cache partitioning profile 61, a configuration may be provided which has a profile that associates the size of the memory area used with the execution processor.

[0040] (C2) In each of the above embodiments, S120, S155 to S165 may be omitted. Similarly, in each embodiment, S140 to S150 may be performed.

[0041] (C3) In each of the above embodiments, the number of processors was two, but it may be any number of three or more. Similarly, the number of memory may also be any number of three or more. In such a configuration, for example, if the number of memory is three and the latency sensitivity of each memory is three levels, the cache hit rate may be divided into three levels and memory having the corresponding latency sensitivity may be allocated to the task of each level.

[0042] This disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from its spirit. For example, the technical features in each embodiment corresponding to the technical features in the embodiments described in the summary of the invention can be replaced or combined as appropriate in order to solve some or all of the above-mentioned problems, or to achieve some or all of the above-mentioned effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate. [Explanation of symbols]

[0043] 10, 20… Processor, 30… Shared cache memory, 50… Resource manager, 60… First profile storage unit, 61… Cache partitioning profile, 100, 100a… Processing system

Claims

1. A processing system (100, 100a) for processing tasks, Multiple processors (10, 20), A shared cache memory (30) shared by the aforementioned multiple processors, A resource manager (50) that performs the allocation of an execution processor to execute the task among the plurality of processors, and a memory area to be used in the shared cache memory, A first profile storage unit (60) stores a cache partitioning profile (61) in which area-related information, which is information about the memory area used, is set for each type of task, Equipped with, The resource manager, in the cache partitioning profile, uses the area-related information corresponding to the type of task to be executed to perform the allocation. Processing system.

2. In the processing system described in claim 1, The region-related information includes size-related information relating to the size of the memory area used, which is set for each type of task, and information relating to the execution processor corresponding to the memory area used among the plurality of processors. The resource manager is a processing system that uses the size-related information corresponding to the type of task to be executed to identify the memory area to be used and the execution processor.

3. A processing system according to claim 1, Two memories distinct from the shared cache memory, comprising a first memory (41) and a second memory (42) having lower latency than the first memory, A second profile storage unit (70) stores a cache hit rate profile (71) which stores cache hit rate-related information, which is information regarding the high or low cache hit rate in the shared cache memory, for each type of task. Furthermore, The aforementioned resource manager is Using the cache hit rate profile, if the cache hit rate is low for the type of task to be executed, the first memory among the first and second memories is allocated to the task. A processing system that, when the cache hit rate is high for the type of task to be executed, allocates the second memory of the two memory units to the task.

4. In the processing system described in claim 3, The aforementioned resource manager is (i) Identify the processing load of the multiple processors, (ii) If the identified processing load is below a threshold, the allocation of the first memory or the second memory to the task to be executed is performed. (iii) If the identified processing load is greater than the threshold, and the priority of the task to be executed is higher than the priority of a task currently running on the processor with the lowest load among the multiple processors, preemption of the task currently running on that processor shall be performed. (iv) A processing system that, if the identified processing load is greater than the threshold, and the priority of the task to be executed is less than or equal to the priority of a task currently running on the processor with the lowest processing load among the plurality of processors, stores the task to be executed in a queue.

5. In the processing system described in claim 4, The resource manager, after the execution processor is allocated, If the priority of the task to be executed is higher than the priority of a task currently running on the execution processor, preemption of the task currently running on the execution processor is performed. A processing system that, if the priority of a task to be executed is less than or equal to the priority of a task currently being executed by the execution processor, saves the task to be executed to a queue.

Images