Reduction acceleration program, reduction acceleration method and information processing device

Abstract

To enable processing including reduction to be accelerated.SOLUTION: A processing part 12 extracts a plurality of tasks having a data dependence relation from a source program. The processing part 12 respectively classifies the plurality of tasks into a first type of tasks including reduction operation to be subjected to parallel processing by a plurality of subtasks and a second type of tasks other than the first type of tasks on the basis of the statement of each of the plurality of tasks. The processing part 12 sets priority information showing the priority of executing the first type of tasks is increased more than the second type of tasks in the source program. The processing part 12 inserts a parallelization instruction sentence showing that respective subtasks are divided to undergo parallel processing into the source program for the plurality of respective subtasks in the first type of tasks.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a high-speed reduction program, a high-speed reduction method, and an information processing device. [Background technology]

[0002] In computer software development, developers often use high-level programming languages ​​such as C. A source program written in a high-level language is converted by a compiler into an object program that can be executed by a processor.

[0003] For example, a parallelizing compiler that translates a source program into an object program for a parallel computer has been proposed. The proposed parallelizing compiler detects reduction syntax from a source program, which is an operation that obtains a single result from array data based on operators that are commutative and associative and of the same kind. The proposed parallelizing compiler converts the detected reduction into a loop that can be executed in parallel. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 9-319722 Summary of the Invention [Problem to be solved by the invention]

[0005] A source program may describe multiple tasks including operations such as reduction. Tasks including reduction operations may be described in advance using a predetermined syntax so that the tasks are processed in parallel when executed. In addition, in order to control parallelization between tasks, data dependencies between tasks may be described in advance in the source program, such as the output of one task becoming the input of another task. When a compiler generates an object program according to these descriptions in the source program, it may not be possible to sufficiently speed up processing based on the object program.

[0006] In one aspect, the present invention aims to increase the speed of processing including reduction. [Means for solving the problem]

[0007] In one embodiment, a reduction acceleration program is provided. The reduction acceleration program causes a computer to execute the following processes. The computer extracts a plurality of tasks having data dependencies from a source program. The computer classifies each of the plurality of tasks into a first type of task including a reduction operation to be processed in parallel by a plurality of subtasks and a second type of task other than the first type of task based on a statement of each of the plurality of tasks. The computer sets priority information in the source program indicating that the execution of the first type of task is to be given a higher priority than the second type of task. The computer inserts, for each of a plurality of subtasks in the first type of task, a parallelization directive indicating that the subtask is to be divided and processed in parallel into the source program.

[0008] Also, in one embodiment, a reduction acceleration program is provided. The computer causes the computer to execute the following process. The computer extracts a plurality of tasks having data dependencies from a source program. The computer classifies each of the plurality of tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task, based on a statement of each of the plurality of tasks. The computer sets priority information in the source program indicating that the execution priority of the first type of task is to be increased over the second type of task.

[0009] In one embodiment, a reduction acceleration program is provided. A computer extracts a plurality of tasks having data dependencies from a source program. The computer classifies each of the plurality of tasks into a first type of task including a reduction operation to be processed in parallel by a plurality of subtasks and a second type of task other than the first type of task based on a statement of each of the plurality of tasks. The computer inserts a parallelization directive into the source program for each of the plurality of subtasks in the first type of task, the parallelization directive indicating that the subtask is to be divided and processed in parallel.

[0010] According to another aspect, there is provided a method for accelerating reduction executed by a computer, and an information processing device having a storage unit and a processing unit. Effect of the Invention

[0011] On the one hand, it can speed up processing involving reduction. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an information processing apparatus according to a first embodiment. [Diagram 2] 13A and 13B are diagrams illustrating an example of adding priority information and parallelization directives. [Diagram 3] FIG. 13 illustrates an example of speeding up processing. [Figure 4] FIG. 11 illustrates an example of hardware of an information processing apparatus according to a second embodiment. [Diagram 5] FIG. 2 is a diagram illustrating an example of functions of an information processing device. [Figure 6] FIG. 13 is a diagram illustrating an example of a description of task parallelism with data dependency. [Figure 7] FIG. 13 is a diagram illustrating an example of a description of a reduction operation. [Figure 8] FIG. 13 is a diagram illustrating an example of an execution of a reduction operation. [Figure 9] FIG. 10 is a diagram for explaining an example of detection of a reduction task in a source program. [Figure 10] 13 is a flowchart illustrating an example of processing by the information processing device. [Figure 11] FIG. 2 illustrates an example of a source program. [Figure 12] FIG. 13 is a diagram illustrating an example of classification of reduction tasks and other tasks. [Figure 13] FIG. 13 is a diagram illustrating an example of priority setting. [Figure 14] FIG. 13 is a diagram illustrating an example of inserting a parallelization directive. [Figure 15] FIG. 11 is a diagram illustrating a first example of speeding up processing. [Figure 16] FIG. 11 is a diagram illustrating a second example of speeding up processing. [Figure 17] FIG. 11 is a diagram illustrating a third example of speeding up processing. [Figure 18] FIG. 13 is a diagram illustrating a fourth example of speeding up processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The present embodiment will be described below with reference to the drawings. [First embodiment] A first embodiment will be described.

[0014] FIG. 1 is a diagram illustrating an information processing apparatus according to a first embodiment. The information processing device 10 performs preprocessing on a source program. In the preprocessing, the source program is modified. The information processing device 10 may generate an object program based on the preprocessed source program. The information processing device 10 may function as a compiler that performs the preprocessing and generates the object program. The information processing device 10 may also be referred to as a computer. The information processing device 10 has a memory unit 11 and a processing unit 12.

[0015] The storage unit 11 may be a volatile semiconductor memory such as a random access memory (RAM), or may be a non-volatile storage such as a hard disk drive (HDD) or a flash memory. The processing unit 12 is, for example, a processor such as a central processing unit (CPU), a graphics processing unit (GPU), or a digital signal processor (DSP). However, the processing unit 12 may include an electronic circuit for a specific purpose such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The processor executes a program stored in a memory such as a RAM (which may be the storage unit 11). A set of multiple processors may be called a "multiprocessor" or simply a "processor."

[0016] Here, the source program includes a description of a plurality of tasks and a description of data dependencies between the tasks. A task is an execution unit of processing assigned to a thread. A thread is a unit of computing resources generated corresponding to each of a plurality of processor cores included in a node that executes an object program. Two or more tasks that have a dependency relationship are executed in an order according to the dependency relationship. Two or more tasks that do not have a dependency relationship can be executed in parallel. The source program also includes a description that enables parallel execution of a task that includes a reduction operation. The source program 20 is an example of a source program using OpenMP (registered trademark), including a description of a task that includes a reduction operation and a description of data dependencies between tasks. An example of the programming language is C.

[0017] The source program 20 includes descriptions of a task "task0" that executes reduction, and other tasks "task1", ​​"task2", .... In the following description, for example, a task with the task name "task0" will be written simply as task0.

[0018] Each task is explicitly defined by a task directive (#pragma omp task). In addition, for each task, input / output processing (in, out) within that task is described in the depend clause of the task directive, creating data dependencies between tasks. Data dependencies include flow dependencies, reverse dependencies, and output dependencies. A flow dependency is a dependency from out to in (out->in), in other words, a dependency in which data output by one task becomes the input of another task. An reverse dependency is a dependency from in to out (in->out). An output dependency is a dependency from out to out (out->out). When the target program is executed, each task is scheduled by the runtime based on the dependency relationships, and is executed on each thread.

[0019] Additionally, task0 includes a reduction operation that aggregates the elements of an array into one. Tiling is applied to task0, and one tile is one task. In the example of source program 20, task0 becomes N / TILE tasks. An integer value of 2 or more is set to the variable TILE of source program 20 by the user. Each of the N / TILE tasks corresponding to task0 is represented as a subtask of task0. The inoutset in the depend clause of task0 represents the dependency relationship between task0 and other in / out tasks.

[0020] In the description example of source program 20, there is no dependency between task0 and task2, so they are executed in parallel. Also, task1 is a function that uses the result of task0, and there is a flow dependency with task0. Therefore, task1 is executed after task0 is executed. In this way, the processing result of a task that performs a reduction operation is often used by other tasks.

[0021] The processing unit 12 processes the source program 20 as follows so as to enable efficient execution of the processes described in the source program 20. First, the processing unit 12 extracts a plurality of tasks having data dependency relationships from the source program 20. For example, the processing unit 12 can extract a plurality of tasks having data dependency relationships based on the depend clause of each task.

[0022] Next, the processing unit 12 classifies each of the extracted tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task, based on the statement of each of the extracted tasks. The first type of task may be called a reduction task.

[0023] For example, the processing unit 12 searches for a statement that matches a predetermined condition corresponding to a reduction from the description of each task in the source program 20, and classifies a task that includes a statement that matches the condition into a first type of task. The processing unit 12 classifies a task that does not include a statement that matches the condition into a second type of task.

[0024] More specifically, the predetermined condition is both the following condition (1) and condition (2). Condition (1) is that the statement is within an OpenMP task and within a loop contained within the OpenMP task.

[0025] Condition (2) is that the statement matching condition (1) includes both a read and a write to a scalar variable or an array accessed by a variable other than the loop variable. The statements of task1 and task2 do not satisfy condition (1), so the processing unit 12 classifies task1 and task2 as the second type of task.

[0026] Additionally, task0 includes three commented statements: stmt0, stmt1, and stmt2. The statement in stmt0 does not meet condition (1) because it is in an OpenMP task but is not a statement in an inner loop.

[0027] The statement in stmt1 satisfies both conditions (1) and (2). The statement in stmt1 is a statement in a loop contained within an OpenMP task. The statement in stmt1 also contains read and write operations on the a_local variable, which is a scalar variable. Therefore, the statement in stmt1 is the target reduction.

[0028] The statement in stmt2 does not meet condition (1) because it is in an OpenMP task but is not a statement in an inner loop. Therefore, the processing unit 12 classifies task0 as a first type of task.

[0029] Then, the processing unit 12 sets priority information in the source program 20 indicating that the execution priority of the first type task is to be higher than that of the second type task. Furthermore, the processing unit 12 inserts a parallelization directive into the source program 20 indicating that each of a plurality of subtasks in the first type task is to be divided and processed in parallel for each subtask. The processing unit 12 may insert the parallelization directive into the source program 20 before setting the priority information.

[0030] FIG. 2 is a diagram showing an example of adding priority information and parallelization directives. The source program 20a is obtained by adding priority information and parallelization directives to the source program 20. The source program 20a is stored in the storage unit 11.

[0031] For example, the processing unit 12 generates a source program 20a by adding priority clauses 21, 21a, 21b, . . . of task directives and a parallelization directive 22 to the source program 20 for each task.

[0032] The priority clauses 21, 21a, 21b, ... correspond to priority information. In one example, the priority clause 21 indicates a priority of "10" for task0. The priority clause 21a indicates a priority of "5" for task1. The priority clause 21b indicates a priority of "5" for task2. The larger the priority value, the higher the execution priority. A task with a higher priority will be executed before a task with a lower priority. In other words, task0, which is a first type of task, will be executed before task1 and task2, which are second type tasks.

[0033] The parallelization directive 22 indicates that each of the multiple subtasks of task0 is to be divided and processed in parallel. That is, according to the source program 20, task0 is processed in parallel using multiple subtasks, but according to the source program 20a, each subtask is further divided into smaller tasks and processed in parallel. Note that the parallelization directive 22 is only an example, and an instruction to divide and parallelize one subtask may be expressed by a parallelization directive written in a different way.

[0034] According to the source program 20, the unit of assignment to one thread for task0 is one subtask. On the other hand, according to the source program 20a, the unit of assignment to one thread for task0 is one of multiple tasks obtained by further dividing one subtask. The number of subtasks to be divided into is determined in advance by the user.

[0035] For example, task0 can be called the parent task, a subtask of task0 can be called the child task, and a task obtained by further dividing the subtask can be called the grandchild task. Since the task (grandchild task) that is parallelized by the parallelization directive c21 is a task that does not have a dependency description, synchronization is performed within the dependent task (task0).

[0036] The processing unit 12 generates an object program based on the source program 20a. The object program is executed by a node having a plurality of processor cores. The information processing device 10 may execute the object program.

[0037] In this way, according to the information processing device 10, a plurality of tasks having data dependencies are extracted from the source program 20. Based on the statements of each of the plurality of tasks, each of the plurality of tasks is classified into a first type of task including a reduction operation to be processed in parallel by a plurality of subtasks, and a second type of task other than the first type of task. Priority information indicating that the execution priority of the first type of task is to be higher than that of the second type of task is set in the source program 20. For each of the plurality of subtasks in the first type of task, a parallelization directive indicating that the subtask is to be divided and processed in parallel is inserted into the source program 20.

[0038] This enables the information processing device 10 to speed up the processing including reduction. Specifically, this is as follows. FIG. 3 is a diagram illustrating an example of speeding up processing.

[0039] As an example, the number of threads to which tasks can be assigned is assumed to be four. Four threads 0 to 3 are shown in Fig. 3. Also, the source programs 20 and 20a include tasks 3 and 4, which are classified as a second type of task, in addition to tasks 0 to 2. Tasks 3 and 4 are assumed to have no dependency relationship with other tasks.

[0040] Task sequences 30 and 31 exemplify the order in which tasks are executed by each thread. The direction from left to right in task sequences 30 and 31 is the positive direction of time. In task sequences 30 and 31, a rectangle with a task name written therein indicates the period during which the task with that task name is executed. However, in task sequence 31, the task names of tasks that are subtasks of task0 are omitted. Also, it is assumed that the number of subtasks is 6. Furthermore, it is assumed that the number of divisions when subtasks are divided is 2.

[0041] The shaded rectangle at the beginning of each task execution period represents the overhead incurred when executing a task with data dependency. This overhead is incurred by the processes required to execute a task with data dependency, such as task generation, generation of dependencies between tasks, checking the dependencies and executing a task only if the dependencies are satisfied.

[0042] Task sequence 30 is a comparative example, and shows the execution status of each task during execution of an object program generated based on source program 20. When tasks 2 to 4, which are tasks other than reduction, are executed first as in the example of task sequence 30, a situation occurs in which many threads are not executing any tasks due to the dependency relationship of task 0 -> task 1.

[0043] The task sequence 31 shows the execution status of each task during execution of the object program generated based on the source program 20a. As described above, according to the example of the source program 20a, the reduction task, task0, is executed with priority over the other tasks 1 to 4. In addition, although there is a flow dependency of task0->task1, there is no dependency between the other tasks, so that combinations of tasks other than the combination of tasks 0 and 1 can be executed in parallel.

[0044] In this case, as shown in task sequence 31, the reduction task is executed with priority, and tasks can be executed efficiently in many threads while satisfying the dependency relationship of task0->task1. Also, by subdividing the subtasks of task0, other free threads can be effectively utilized. This reduces the time required to complete the execution of all tasks from task0 to 4. For example, the time from the start of execution of the first task in task sequence 31 to the completion of execution of the last task (task1, 2) is shorter than the time from the start of execution of the first task in task sequence 30 to the completion of execution of the last task (task1). In this way, the processing including reduction is accelerated.

[0045] [Second embodiment] Next, a second embodiment will be described. FIG. 4 illustrates an example of hardware of an information processing apparatus according to the second embodiment.

[0046] The information processing device 100 has a processor 101, a RAM 102, a HDD 103, a GPU 104, an input interface 105, a medium reader 106, and a communication interface 107. These units of the information processing device 100 are connected to a bus inside the information processing device 100. The processor 101 corresponds to the processing unit 12 of the first embodiment. The RAM 102 or the HDD 103 corresponds to the storage unit 11 of the first embodiment.

[0047] The processor 101 is an arithmetic device that executes program instructions. The processor 101 is, for example, a CPU. The processor 101 loads at least a part of the programs and data stored in the HDD 103 into the RAM 102 and executes the programs. The processor 101 includes cores (processor cores) 101a, 101b, . . . The information processing device 100 may also have multiple processors. A set of multiple processors may be called a "multiprocessor" or simply a "processor."

[0048] The RAM 102 is a volatile semiconductor memory that temporarily stores programs executed by the processor 101 and data used in calculations by the processor 101. Note that the information processing device 100 may include a type of memory other than a RAM, or may include multiple memories.

[0049] The HDD 103 is a non-volatile storage device that stores software programs such as an OS (Operating System), middleware, and application software, as well as data. The information processing device 100 may include other types of storage devices, such as a flash memory or an SSD (Solid State Drive), or may include multiple non-volatile storage devices.

[0050] The GPU 104 outputs an image to a display 111 connected to the information processing device 100 in accordance with an instruction from the processor 101. As the display 111, any type of display can be used, such as a CRT (Cathode Ray Tube) display, a liquid crystal display (LCD: Liquid Crystal Display), a plasma display, or an organic EL (OEL: Organic Electro-Luminescence) display.

[0051] The input interface 105 acquires an input signal from an input device 112 connected to the information processing device 100, and outputs the signal to the processor 101. The input device 112 may be a pointing device such as a mouse, a touch panel, a touch pad, or a trackball, a keyboard, a remote controller, or a button switch. In addition, a plurality of types of input devices may be connected to the information processing device 100.

[0052] The medium reader 106 is a reading device that reads programs and data recorded on a recording medium 113. For example, a magnetic disk, an optical disk, a magneto-optical disk (MO: Magneto-Optical disk), a semiconductor memory, etc. can be used as the recording medium 113. Magnetic disks include flexible disks (FD: Flexible Disks) and HDDs. Optical disks include compact discs (CDs) and digital versatile discs (DVDs).

[0053] The medium reader 106 copies, for example, the program or data read from the recording medium 113 to another recording medium such as the RAM 102 or the HDD 103. The read program is executed, for example, by the processor 101. Note that the recording medium 113 may be a portable recording medium, and may be used for distributing the program or data. Also, the recording medium 113 and the HDD 103 may be referred to as computer-readable recording media.

[0054] The communication interface 107 is connected to a network 114 and communicates with other information processing devices via the network 114. The communication interface 107 may be a wired communication interface connected to a wired communication device such as a switch or a router, or a wireless communication interface connected to a wireless communication device such as a base station or an access point.

[0055] The information processing device 100 functions as, for example, a compiler for a High-Performance Computing (HPC) system. The information processing device 100 may be an HPC system or may be a part of an HPC system.

[0056] FIG. 5 is a diagram illustrating an example of functions of the information processing device. The information processing device 100 includes a storage unit 120 and a compiler 130. The storage unit 120 uses storage areas of the RAM 102 and the HDD 103. The compiler 130 is realized by the processor 101 executing a compiler program stored in the RAM 102.

[0057] The storage unit 120 stores a source program to be compiled and an object program generated based on the source program. For example, OpenMP is used to describe parallel processing in the source program. As an example, the programming language is C.

[0058] The compiler 130 compiles a source program to generate an object program, and includes a task classification unit 131, a priority setting unit 132, an intra-task parallelism application unit 133, and an object program generation unit 134.

[0059] The task classification unit 131 extracts a plurality of tasks having data dependencies from a source program. Based on the statements of each task, the task classification unit 131 detects a task that includes a reduction operation that is processed in parallel by a plurality of subtasks, that is, a reduction task, among the plurality of tasks. The task classification unit 131 classifies each task included in the source program into a reduction task and other tasks.

[0060] The priority setting unit 132 sets, in the source program, priority information indicating that the execution priority of the reduction task is to be higher than that of other tasks. The intra-task parallelization application unit 133 applies intra-task parallelism to the reduction task. Specifically, the intra-task parallelization application unit 133 inserts, for each of a plurality of subtasks in the reduction task, a parallelization directive that indicates that the subtask is to be divided and processed in parallel, into the source program.

[0061] The object program generation unit 134 generates an object program based on the source program edited by the priority setting unit 132 and the intra-task parallelism application unit 133, and stores the object program in the storage unit 120. The object program may be executed by the processor 101, or may be executed by another computer having a multi-core processor.

[0062] FIG. 6 is a diagram showing an example of a description of task parallelism with data dependency. Each task is explicitly defined by the OpenMP task directive (#pragma omp task). In addition, data dependencies between tasks are created by describing the input / output processing (in, out) within each task in the depend clause of the task directive. As mentioned above, data dependencies include flow dependencies (out->in), reverse dependencies (in->out), and output dependencies (out->out). When the target program is executed, each task is scheduled by the runtime based on the dependencies, and is executed on each thread.

[0063] For example, source program 41 includes a description of task parallelism with data dependency. In the example of source program 41, there is no dependency between task0 and task1, so they are executed in parallel. On the other hand, task2 has a flow dependency with tasks0 and 1. Therefore, task2 is executed after tasks0 and 1 are executed.

[0064] FIG. 7 is a diagram showing an example of a description of a reduction operation. Source program 42 includes a description of task0 that includes a reduction operation that aggregates the elements of an array into one. Tiling is applied to task0, and one tile is one task. A task corresponding to one tile corresponds to a subtask of task0. In the example of source program 42, task0 is processed in parallel by N / TILE subtasks. In addition, inoutset in the depend clause of task0 represents a dependency relationship between task0 and other in / out tasks. In the example of source program 42, the inoutset task can be executed in parallel, and has a dependency relationship with other in / out tasks. In the example of source program 42, task1 is executed after all of task0 that executes reduction, i.e., all subtasks of task0, have been executed.

[0065] FIG. 8 is a diagram illustrating an example of the execution of a reduction operation. As in the example of FIG. 7, when task1 depends on task0, task0 is processed in parallel by multiple subtasks, resulting in a many-to-one dependency relationship. Task flow 50 illustrates a many-to-one dependency relationship between task0 and task1. In this way, the subsequent task (task1) cannot be executed until the entire task that executes reduction (task0) is completed. For this reason, the more subsequent tasks there are, the easier it is to increase speed by prioritizing execution of the task that executes reduction.

[0066] Next, an example of detection of a reduction task by the task classification unit 131 will be described. FIG. 9 is a diagram for explaining an example of detection of a reduction task in a source program. A source program 43 is a portion of task0 in the source program 42. In the source program 43, comments stmt0, stmt1, and stmt2 are added to each statement so that the statements to be searched for by the reduction task can be easily understood.

[0067] The task classification unit 131 searches for a statement that matches a predetermined condition corresponding to reduction from the description of each task in the source program 43, and classifies a task that includes a statement that matches the condition as a reduction task. The processing unit 12 classifies a task that does not include a statement that matches the condition as a task other than a reduction task, i.e., as other tasks. The predetermined conditions are both the above-mentioned conditions (1) and (2). Conditions (1) and (2) are recited as follows:

[0068] Condition (1) is that the statement is within an OpenMP task and within a loop contained within the OpenMP task. Condition (2) is that the statement matching condition (1) includes both a read and a write to a scalar variable or an array accessed by a variable other than the loop variable.

[0069] The statement in stmt0 does not meet condition (1) because it is in an OpenMP task but is not a statement in an inner loop. The statement in stmt1 satisfies both conditions (1) and (2). The statement in stmt1 is a statement in a loop contained within an OpenMP task. The statement in stmt1 also contains read and write operations on the a_local variable, which is a scalar variable. Therefore, the statement in stmt1 is the target reduction.

[0070] The statement in stmt2 does not meet condition (1) because it is in an OpenMP task but is not a statement in an inner loop. Since task 0 includes a statement that satisfies both the condition (1) and the condition (2), the task classification unit 131 classifies task 0 as a reduction task.

[0071] 7, the statement of task1 does not satisfy condition (1), so the task classification unit 131 classifies task1 as another task.

[0072] Next, the processing procedure of the information processing device 100 will be described. FIG. 10 is a flowchart illustrating an example of processing by the information processing device. (S10) The task classification unit 131 detects tasks with data dependencies by analyzing the source program. As described above, the task classification unit 131 can identify the dependency relationships between tasks based on the description of the depend clause of each task.

[0073] (S11) The task classification unit 131 searches for tasks included in the source program in the order of appearance. (S12) The task classification unit 131 determines whether the task is a reduction task. The determination of whether the task is a reduction task is performed based on whether the task includes a statement that satisfies the above-mentioned conditions (1) and (2). If the task is a reduction task, the process proceeds to step S13. If the task is not a reduction task, the process proceeds to step S15.

[0074] (S13) The priority setting unit 132 sets the priority of the reduction task for the relevant task in the source program. The priority of the reduction task is higher than the priority of the other tasks.

[0075] (S14) The intra-task parallelization application unit 133 applies task parallelism within a dependent task. That is, the intra-task parallelization application unit 133 inserts a parallelization directive into the source program for each of a plurality of subtasks in the corresponding task (reduction task), which indicates that the subtask is to be further divided and processed in parallel. Then, the process proceeds to step S16.

[0076] (S15) The priority setting unit 132 sets the priority of other tasks in relation to the relevant task in the source program. Then, the process proceeds to step S16. (S16) The task classification unit 131 determines whether or not all tasks in the source program have been searched. If all tasks have been searched, the process ends. If there are tasks that have not been searched, the process proceeds to step S12, and the procedure from step S12 onwards is performed on the next searched task.

[0077] Note that steps S13 and S14 may be executed in the reverse order. Furthermore, the object program generation unit 134 generates an object program based on the source program after the priority setting unit 132 has set priorities and the intra-task parallelization application unit 133 has inserted parallelization directives.

[0078] Next, a specific example of processing the source program by the compiler 130 will be described. FIG. 11 is a diagram illustrating an example of a source program. The source program 60 is stored in the storage unit 120. The source program 60 describes the following five types of tasks, task0 to task4. Task0 is a reduction task. Task1 is a function that uses the result of task0. Task2 to task4 are functions that have no dependency at all.

[0079] The dependency between tasks described in the source program 60 is task0->task1, that is, a flow dependency in which task1 uses the result of task0. In addition, among tasks 0 to 4, only task 0 is generated multiple times, resulting in N / TILE tasks. Each of the N / TILE tasks corresponding to task 0 is referred to as a subtask of task 0.

[0080] FIG. 12 is a diagram showing an example of classification of reduction tasks and other tasks. The task classification unit 131 detects tasks with data dependency, task0 to task4, from task directives having depend clauses included in the source program 60.

[0081] The task classification unit 131 searches for each detected task in the order of appearance, and classifies the tasks into two types: tasks that perform reduction (reduction tasks) and tasks that perform other calculations (other tasks). The task classification unit 131 detects reduction tasks using the method illustrated in Fig. 9. In the example of the source program 60, the task classification unit 131 classifies task0 as a reduction task, and tasks1 to 4 as other tasks.

[0082] 12A illustrates a code 61 of a portion of a source program 60 that corresponds to task 0, which is classified as a reduction task. FIG 12B illustrates a code 62 of a portion of a source program 60 that corresponds to tasks 1 to 4, which are classified as other tasks.

[0083] The priority setting unit 132 sets priorities for the reduction task and other tasks. FIG. 13 is a diagram illustrating an example of priority setting.

[0084] 13A illustrates an example of a code 63 for defining a priority level set in the source program 60. The priority setting unit 132 sets the code 63 in the source program 60. The code 63 defines the priority levels to be set in the reduction task and other tasks.

[0085] Fig. 13B illustrates an example of code 61a when a priority is set for the reduction task code 61. Fig. 13C illustrates an example of code 62a when a priority is set for the other task code 62.

[0086] The priority setting unit 132 adds a priority clause c11 in the code 61a to the task command for task0. The priority setting unit 132 also adds a priority clause c12 in the code 62a to the task command for task 1. The priority setting unit 132 also adds a priority clause c13 in the code 62a to the task command for task 2. The priority setting unit 132 also adds priority clauses to tasks 3 and 4, similar to tasks 1 and 2.

[0087] The larger the number set in the priority clause, the higher the priority. In the examples of Codes 63, 61, and 62a, the execution priority of the reduction task is higher than the execution priority of other tasks.

[0088] Furthermore, the intra-task parallelization application unit 133 inserts into the source program 60 parallelization directives for further dividing the subtasks in the reduction task and processing them in parallel. FIG. 14 is a diagram showing an example of inserting a parallelization directive.

[0089] Code 61b illustrates an example of code 61b when a parallelization directive c21 is inserted into code 61 of the reduction task. The intra-task parallelization application unit 133 inserts a parallelization directive c21 at a position corresponding to a subtask of the reduction. The parallelization directive c21 indicates that the innermost loop of task0 (corresponding to a subtask) is to be divided into multiple tasks and processed in parallel. The parallelization directive c21 causes each of the multiple subtasks (multiple child tasks) of task0 (parent task) to be further subdivided into multiple tasks (grandchild tasks) and processed in parallel. The number of divisions of one subtask (child task), i.e., the number of grandchild tasks for one child task, is determined in advance by the user.

[0090] Incidentally, since the task (grandchild task) parallelized by the parallelization directive c21 is a task without a dependency description, synchronization is performed within the dependent task (within task0). In this way, the intra-task parallelization application unit 133 further applies task parallelism within a task with data dependency that performs reduction with set priority. In this case, the inner task is synchronized within the task with data dependency. On the other hand, the outer task with data dependency is synchronized between tasks. By using partial task parallelism with little overhead, it is possible to achieve higher speeds compared to the case where all tasks with data dependency are executed.

[0091] Next, an example of speeding up processing by the above-mentioned priority setting and parallelization directive insertion will be described. FIG. 15 is a diagram illustrating a first example of speeding up processing. In the following description, the number of threads to which tasks can be assigned is assumed to be 4 as an example. Task sequences 70 and 71 show an example of the execution of tasks 0 to 4 in the source program 60 by four threads 0 to 3. The direction from left to right in the task sequences 70 and 71 is the positive direction of time. In the task sequences 70 and 71, a rectangle with a task name written therein indicates the period during which the task with that task name is executed. A shaded rectangle added to the beginning of a rectangle representing the period during which each task is executed indicates the overhead when executing a task with data dependency. In addition, the number of subtasks corresponding to task 0 is assumed to be 6. The expressions of each task in the task sequences 70 and 71 are the same in the subsequent figures.

[0092] A task sequence 70 is a comparative example, and shows the execution status of each task during execution of an object program generated without editing the source program 60 by the priority setting unit 132 and the intra-task parallelism application unit 133. When tasks 2 to 4, which are tasks other than reduction, are executed first as in the example of task sequence 30, a situation arises in which many threads are not executing any tasks due to the dependency relationship of task 0 -> task 1.

[0093] The task sequence 71 shows the execution status of each task during execution of the object program generated by editing the source program 60 by the priority setting unit 132. In the example of the task sequence 71, editing by the intra-task parallelism application unit 133 has not been performed.

[0094] In this case, as represented by the task sequence 71, the reduction task, task0, is executed with priority over the other tasks 1 to 4. Also, although there is a flow dependency of task0->task1, there is no dependency between the other tasks, and therefore combinations of tasks other than the combination of tasks 0 and 1 can be executed in parallel. In this way, the compiler 130 sets the priority so that the reduction task is executed with priority over other tasks, thereby making it possible to execute tasks more efficiently with more threads than in the comparative example represented by the task sequence 70, thereby speeding up processing.

[0095] FIG. 16 is a diagram illustrating a second example of speeding up the processing. Task sequences 80 and 81 show an example of the execution of task0 by four threads 0 to 3. In the example of Fig. 16, the source program to be compiled corresponds to the source program 42 in Fig. 7. For example, task1, which has a flow-dependency relationship with task0, is not shown in the figure. When task1 is executed, task1 is executed after the execution of task0 is completed.

[0096] Task sequence 80 is a comparative example, and shows the execution status of task0 during execution of an object program generated without editing source program 42 by priority setting unit 132 and intra-task parallelism application unit 133. The number of subtasks corresponding to task0 is six, whereas four threads are executed. Therefore, while four subtasks are being executed, the remaining two tasks are waiting to be executed.

[0097] The task sequence 81 shows the execution status of task0 during execution of the object program generated by editing the source program 42 by the intra-task parallelism application unit 133. In the example of Fig. 16, editing by the priority setting unit 132 does not have to be performed. The division number when subdividing the subtasks is assumed to be 2. In the task sequence 81, the task names are omitted for tasks obtained by subdividing the subtasks of task0.

[0098] As shown in task sequence 81, subdividing the subtasks makes it possible to effectively utilize other free threads and increase the speed. In this case, since each subtask of task0 (which has 6 subtasks) is divided into two, threads 0 to 3 distribute and execute a total of 12 tasks for task0 in parallel.

[0099] Note that because of the overhead involved in dividing and executing subtasks, the execution time per task (grandchild task) assigned to one thread will be longer than if the division were not performed. Even with this in mind, subdividing subtasks can still speed up processing.

[0100] Furthermore, when task1 is executed, task1 is executed after all of task0 has been executed. The length of the execution period of task1 is the same in both task sequences 80 and 81. Comparing task sequences 80 and 81, task1 is completed sooner when task1 is executed after all of task0 has been executed in task sequence 81 than when task1 is executed after all of task0 has been executed in task sequence 80.

[0101] FIG. 17 is a diagram illustrating a third example of speeding up the processing. Task sequences 70 and 72 show an example of execution of tasks 0 to 4 in the source program 60 by four threads 0 to 3. The task sequence 70 is a comparative example and is the same as the task sequence 70 in FIG.

[0102] The task sequence 72 shows the execution status of each task during execution of the object program generated by editing the source program 60 by both the priority setting unit 132 and the intra-task parallelism application unit 133. Note that in the task sequence 72, the task names are omitted for tasks that are subtasks of task0.

[0103] As shown in task sequence 72, the reduction task is executed with priority, and tasks can be executed efficiently in many threads while satisfying the dependency relationship of task0->task1. Also, by subdividing the subtasks of task0, other free threads can be effectively utilized. This reduces the time required to complete the execution of all tasks from task0 to 4. For example, the time from the start of execution of the first task in task sequence 72 to the completion of execution of the last task (task1, 2) is shorter than the time from the start of execution of the first task in task sequence 72 to the completion of execution of the last task (task1). In this way, processing including reduction is accelerated.

[0104] FIG. 18 is a diagram illustrating a fourth example of speeding up the processing. Task sequences 71 and 72 show an example of execution of tasks 0 to 4 in the source program 60 by four threads 0 to 3. The task sequence 71 is the same as the task sequence 71 in Fig. 15. The task sequence 72 is the same as the task sequence 72 in Fig. 17.

[0105] The time from the start of execution of the first task in the task sequence 72 to the completion of execution of the last task (task 1, 2) is shorter than the time from the start of execution of the first task in the task sequence 71 to the completion of execution of the last task (task 1, 3).

[0106] In this way, editing the source program 60 by both the priority setting unit 132 and the intra-task parallelism application unit 133 can speed up processing compared to editing by only the priority setting unit 132. However, the compiler 130 may use only one of the priority setting unit 132 and the intra-task parallelism application unit 133.

[0107] One of the programming techniques in the HPC field is task parallelism with data dependencies. When a task is executed, a thread corresponding to a core in a computing node is generated, and the task is executed on that thread. The runtime schedules tasks based on the data dependencies of the tasks described in the source program by the user, and synchronizes between tasks instead of synchronizing the entire system, allowing threads to be used efficiently and program execution to be accelerated.

[0108] Task parallelism with data dependencies has a larger overhead than general task parallelism. In general task parallelism, only task creation and task execution are performed. On the other hand, in task parallelism with data dependence, the processing is complicated, involving task creation, dependency creation, dependency check, and execution only if the dependency is satisfied. However, in the case of general task parallelism, the overhead is less than in task parallelism with data dependence because there is no dependency check, but the synchronization cost is higher than in task parallelism with data dependence because of the global synchronization.

[0109] Therefore, the problem is how to further speed up reduction operations implemented using task parallelism with data dependencies. The information processing device 100 converts the reduction operation of a task-parallel program with data dependency into a description that assigns priority and further parallelizes tasks within the task with data dependency. As a result, the information processing device 100 can speed up the execution order of the task with data dependency that performs reduction, and can speed up the processing of the task with data dependency by combining it with task parallelism that has little overhead, thereby improving the execution performance of the program.

[0110] As described above, the information processing device 100 executes the following processes. The task classification unit 131 extracts a plurality of tasks having data dependencies from a source program. Based on a statement of each of the plurality of tasks, the task classification unit 131 classifies each of the plurality of tasks into a first type task including a reduction operation to be processed in parallel by a plurality of subtasks, and a second type task other than the first type task. The priority setting unit 132 sets priority information in the source program indicating that the execution priority of the first type task is to be higher than that of the second type task. The intra-task parallelization application unit 133 inserts a parallelization directive into the source program for each of a plurality of subtasks in the first type task, indicating that the subtask is to be divided and processed in parallel.

[0111] This enables the information processing device 100 to speed up processing including reduction. The reduction task described above corresponds to a first type of task. Tasks other than the reduction task, that is, other tasks, correspond to a second type of task.

[0112] For example, the task classification unit 131 extracts a plurality of tasks based on a task instruction described in a source program. The task instruction includes a depend clause that indicates a data dependency relationship between tasks. Therefore, the information processing device 100 can efficiently extract a plurality of tasks having a data dependency relationship by referring to the task instruction.

[0113] Furthermore, the data dependency between tasks includes a relationship in which a processing result of a first task classified as a first type of task is used in a second task classified as a second type of task. This allows the information processing device 100 to efficiently use multiple threads that execute multiple tasks while maintaining the data dependency between the first task and the second task, thereby speeding up processing.

[0114] The priority setting unit 132 adds a priority clause to each task command of a plurality of tasks described in a source program according to the classification of the task, thereby enabling the information processing device 100 to easily set the priority of the execution of each task.

[0115] Furthermore, the object program generation unit 134 generates an object program based on a source program including priority information set by the priority setting unit 132 and parallelization directives inserted by the intra-task parallelism application unit 133. In this way, the task classification unit 131, the priority setting unit 132 and the intra-task parallelism application unit 133 are suitable as part of the functions of the compiler 130. However, the functions of the object program generation unit 134 may be provided by another computer. In that case, the information processing device 100 may transmit the source program preprocessed by the task classification unit 131, the priority setting unit 132 and the intra-task parallelism application unit 133 to the other computer.

[0116] Furthermore, the information processing device 100 can process the source program using only either the priority setting unit 132 or the intra-task parallelism application unit 133, thereby achieving faster processing.

[0117] The information processing of the first embodiment can be realized by causing the processing unit 12 to execute a program. The information processing of the second embodiment can be realized by causing the processor 101 to execute a program. The program can be recorded in a computer-readable recording medium 113.

[0118] For example, the program can be distributed by distributing recording medium 113 on which the program is recorded. Alternatively, the program may be stored in another computer and distributed via a network. The computer may store (install) the program recorded in recording medium 113 or a program received from another computer in a storage device such as RAM 102 or HDD 103, and read and execute the program from the storage device.

[0119] The following supplementary notes are further disclosed regarding the above-described embodiments including the first and second embodiments. (Appendix 1) To the computer, Extracting a plurality of tasks having data dependencies from a source program; Based on the statement of each of the plurality of tasks, classifying each of the plurality of tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task; setting priority information in the source program, the priority information indicating that the execution of the first type task is to be given a higher priority than the execution of the second type task; inserting into the source program a parallelization directive that indicates, for each of the plurality of subtasks in the first type task, that the subtask is to be divided and processed in parallel; A reduction acceleration program that performs the processing.

[0120] (Note 2) In the extraction of the plurality of tasks, the plurality of tasks are extracted based on a task instruction written in the source program. 2. The reduction acceleration program according to claim 1, which causes the computer to execute the process.

[0121] (Supplementary Note 3) The data dependency includes a relationship in which a processing result of a first task classified as a first type of task is used in a second task classified as a second type of task. The reduction acceleration program described in Appendix 1.

[0122] (Note 4) In setting the priority information, a priority clause corresponding to a classification destination of the task is added to a task instruction of each of the plurality of tasks described in the source program. 2. The reduction acceleration program according to claim 1, which causes the computer to execute the process.

[0123] (Supplementary Note 5) generating an object program based on the source program including the priority information and the parallelization directive; 2. The reduction acceleration program according to claim 1, which causes the computer to execute the process.

[0124] (Appendix 6) The computer, Extracting a plurality of tasks having data dependencies from a source program; Based on the statement of each of the plurality of tasks, classifying each of the plurality of tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task; setting priority information in the source program, the priority information indicating that the execution of the first type task is to be given a higher priority than the execution of the second type task; inserting into the source program a parallelization directive that indicates, for each of the plurality of subtasks in the first type task, that the subtask is to be divided and processed in parallel; Reduction speedup method.

[0125] (Appendix 7) A storage unit for storing a source program; a processing unit that extracts a plurality of tasks having data dependency from the source program, classifies each of the plurality of tasks into a first type task including a reduction operation to be processed in parallel by a plurality of subtasks and a second type task other than the first type task based on a statement of each of the plurality of tasks, sets priority information in the source program indicating that the execution priority of the first type task is to be higher than that of the second type task, and inserts into the source program, for each of the plurality of subtasks in the first type task, a parallelization directive indicating that the subtask is to be divided and processed in parallel; An information processing device having the above configuration.

[0126] (Appendix 8) To the computer, Extracting a plurality of tasks having data dependencies from a source program; Based on the statement of each of the plurality of tasks, classifying each of the plurality of tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task; setting priority information in the source program, the priority information indicating that the execution priority of the first type task is to be higher than that of the second type task; A reduction acceleration program that performs the processing.

[0127] (Appendix 9) The computer, Extracting a plurality of tasks having data dependencies from a source program; Based on the statement of each of the plurality of tasks, classifying each of the plurality of tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task; setting priority information in the source program, the priority information indicating that the execution priority of the first type task is to be higher than that of the second type task; Reduction speedup method.

[0128] (Appendix 10) A storage unit for storing a source program; a processing unit that extracts a plurality of tasks having data dependencies from the source program, classifies each of the plurality of tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task based on a statement of each of the plurality of tasks, and sets priority information in the source program indicating that the execution priority of the first type of task is to be higher than that of the second type of task; An information processing device having the above configuration.

[0129] (Appendix 11) To the computer, Extracting a plurality of tasks having data dependencies from a source program; Based on the statement of each of the plurality of tasks, classifying each of the plurality of tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task; inserting into the source program a parallelization directive that indicates, for each of the plurality of subtasks in the first type task, that the subtask is to be divided and processed in parallel; A reduction acceleration program that performs the processing.

[0130] (Appendix 12) The computer: Extracting a plurality of tasks having data dependencies from a source program; Based on the statement of each of the plurality of tasks, classifying each of the plurality of tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task; inserting into the source program a parallelization directive that indicates, for each of the plurality of subtasks in the first type task, that the subtask is to be divided and processed in parallel; Reduction speedup method.

[0131] (Appendix 13) A storage unit for storing a source program; a processing unit that extracts a plurality of tasks having data dependencies from the source program, classifies each of the plurality of tasks into a first type task including a reduction operation to be processed in parallel by a plurality of subtasks and a second type task other than the first type task based on a statement of each of the plurality of tasks, and inserts into the source program a parallelization directive indicating that each of the plurality of subtasks in the first type task is to be divided and processed in parallel; An information processing device having the above configuration. [Explanation of symbols]

[0132] 10. Information processing device 11 Storage section 12 Processing section 20 Source Program

Claims

1. On the computer, Extracting a plurality of tasks having data dependencies from a source program; Based on the statement of each of the plurality of tasks, classifying each of the plurality of tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task; setting priority information in the source program, the priority information indicating that the execution of the first type task is to be given a higher priority than the execution of the second type task; inserting into the source program a parallelization directive that indicates, for each of the plurality of subtasks in the first type task, that the subtask is to be divided and processed in parallel; A reduction acceleration program that performs the processing.

2. In the extraction of the plurality of tasks, the plurality of tasks are extracted based on a task instruction written in the source program; 2. The reduction speed-up program according to claim 1, which causes the computer to execute a process.

3. the data dependency includes a relationship in which a processing result of a first task classified into the first type of task is used in a second task classified into the second type of task; 2. The reduction speed-up program according to claim 1.

4. In setting the priority information, a priority clause corresponding to a classification destination of the task is added to a task instruction of each of the plurality of tasks described in the source program.

2. The reduction speed-up program according to claim 1, which causes the computer to execute a process.

5. generating an object program based on the source program including the priority information and the parallelization directive; 2. The reduction speed-up program according to claim 1, which causes the computer to execute a process.

6. The computer Extracting a plurality of tasks having data dependencies from a source program; Based on the statement of each of the plurality of tasks, classifying each of the plurality of tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task; setting priority information in the source program, the priority information indicating that the execution of the first type task is to be given a higher priority than the execution of the second type task; inserting into the source program a parallelization directive that indicates, for each of the plurality of subtasks in the first type task, that the subtask is to be divided and processed in parallel; Reduction speedup method.

7. A storage unit for storing a source program; a processing unit that extracts a plurality of tasks having data dependency from the source program, classifies each of the plurality of tasks into a first type task including a reduction operation to be processed in parallel by a plurality of subtasks and a second type task other than the first type task based on a statement of each of the plurality of tasks, sets priority information in the source program indicating that the execution priority of the first type task is to be higher than that of the second type task, and inserts into the source program, for each of the plurality of subtasks in the first type task, a parallelization directive indicating that the subtask is to be divided and processed in parallel; An information processing device having the above configuration.

8. On the computer, Extracting a plurality of tasks having data dependencies from a source program; Based on the statement of each of the plurality of tasks, classifying each of the plurality of tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task; setting priority information in the source program, the priority information indicating that the execution priority of the first type task is to be higher than that of the second type task; A reduction acceleration program that performs the processing.

9. On the computer, Extracting a plurality of tasks having data dependencies from a source program; Based on the statement of each of the plurality of tasks, classifying each of the plurality of tasks into a first type of task including a reduction operation that is processed in parallel by a plurality of subtasks and a second type of task other than the first type of task; inserting into the source program a parallelization directive that indicates, for each of the plurality of subtasks in the first type task, that the subtask is to be divided and processed in parallel; A reduction acceleration program that performs the processing.

Images