Grid computing method and management device

Abstract

To reduce a waiting time until job completion, in grid computing.SOLUTION: A client 11 can communicate with a server 12 via a network. The server 12 simultaneously transmits the same job to a plurality of clients 11 (1), and when first receiving a job execution result from any one of the plurality of clients 11 (2), transmits a job execution stop command to the other clients 11 (3), and adopts the received execution result as the job execution result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology disclosed herein relates to grid computing. [Background technology]

[0002] Grid computing is a technology that links multiple computers via a network and uses them as a high-performance parallel system as a whole. In particular, it is a method of distributing processing across a wide area or by bundling various types of computers together via the Internet.

[0003] Non-Patent Document 1 discloses a study on executing the same job multiple times in grid computing in order to detect malicious alteration of the client's computation results. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Matsuda et al., "Quantitative Evaluation of the Impact of Bad Guys' Collusion on Confidentiality and Reliability in Grid Computing," IPSJ 82nd National Convention, 2020 Summary of the Invention [Problem to be solved by the invention]

[0005] In grid computing, jobs are executed by multiple clients that are not under strict management and have different performance levels. As a result, the job success rate is low and the job processing time varies widely, resulting in the problem of long waiting times until the job is successful and completed.

[0006] The technology disclosed herein aims to reduce the waiting time until a job is completed in grid computing. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the technology disclosed herein provides a method for performing grid computing in a computer system including a server and a plurality of clients that can communicate with the server via a network, the method comprising: the server is connected via a network to a database that stores at least client information, the client information including the number of slots for each client and the status of each slot; The server By referring to the client information, a slot in the "waiting for allocation" state is reserved; The same job, Have a reserved slot The job is simultaneously transmitted to a plurality of clients, and when the execution result of the job is first received from any one of the plurality of clients, an instruction to stop execution of the job is transmitted to the other clients among the plurality of clients, and the received execution result is adopted as the execution result of the job.

[0008] With this configuration, the same job can be executed simultaneously by multiple clients, reducing the failure rate of the job. Furthermore, when one client first completes the job and sends the execution results to the server, this execution result is adopted, and the execution of the job by other clients is stopped. This reduces the waiting time until the job is completed.

[0009] Furthermore, in the grid computing method, the server may determine the number of concurrent execution trials based on the congestion status of the grid indicated by the number and percentage of slots in the "allocation waiting" state and the priority of the job, and reserve a number of slots in the "allocation waiting" state equivalent to the number of concurrent execution trials.

[0010] In order to solve the above-mentioned problems, the technology disclosed herein provides a management device that executes grid computing using a plurality of clients that can communicate via a network, the management device comprising: The system is connected via a network to a database that stores at least client information, the client information including the number of slots for each client and the status of each slot, and reserves slots in the "waiting for allocation" state by referring to the client information; The same job, Have a reserved slot The job is simultaneously transmitted to a plurality of clients, and when the execution result of the job is first received from any one of the plurality of clients, an instruction to stop execution of the job is transmitted to the other clients among the plurality of clients, and the received execution result is adopted as the execution result of the job.

[0011] With this configuration, the same job can be executed simultaneously by multiple clients, reducing the failure rate of the job. Furthermore, when one client first completes the job and sends the execution results to the management device, the execution results are adopted and the execution of the jobs by other clients is stopped. This reduces the waiting time until the job is completed.

[0012] In addition, the management device may determine the number of concurrent execution trials based on the grid congestion status indicated by the number and percentage of slots in the ``allocation waiting'' state and the priority of the job, and reserve a number of slots in the ``allocation waiting'' state equivalent to the number of concurrent execution trials. [Effects of the Invention]

[0013] As described above, the technology disclosed herein can reduce the waiting time until a job is completed in grid computing. [Brief explanation of the drawings]

[0014] [Figure 1] Conceptual diagram of grid computing according to the present disclosure [Figure 2] System configuration example in the embodiment [Figure 3] (a) to (c) are examples of information stored in the database. [Figure 4] Client slot state diagram [Figure 5] Job state transition diagram [Figure 6] Trial state transition diagram [Figure 7] A flowchart showing an example of a process in which a server causes a client to execute a job. [Figure 8] What to do if a "success" signal is received [Figure 9] What to do if you receive a "fail" signal [Figure 10] What to do if you receive a "stop" signal [Figure 11] Processing when "Waiting for output file transfer" signal is received [Figure 12] An example of the information held in the database for each job [Figure 13]An example of the information held in the database for each client DETAILED DESCRIPTION OF THE INVENTION

[0015] Exemplary embodiments will now be described in detail with reference to the drawings.

[0016] The definitions of terms used in this application are as follows: "trial," "number of concurrently executed trials," "slot," and "number of slots" are terms unique to this application. "Job": A series of processes that process input files based on commands and obtain output files. A log file is also generated at the same time, showing the status along the way and the cause of failure. The priority indicates whether the job is important or not. "Trial": A single execution of a job. In the case of simultaneous multiple execution of a job, multiple trials are executed simultaneously. "Number of concurrent trials": The number of trials for one job. The value can be 1. "Slot": A unit responsible for executing a job on a client. "Number of slots": The maximum number of jobs that can be executed by a client. This is determined by the memory, processing speed, and capacity of the recording disk. "Client": A computer on which a job is executed. As long as it can execute a job, it can be a fixed-position device such as a personal computer, or a mobile device such as a smartphone or an in-vehicle MPU (Media Processing Unit). "Server": A computer that manages jobs and trials, manages clients, and manages input files, output files, and log files. The server is an example of a management device in the present disclosure. "Storage": The recording disk on which input files, output files, and log files are recorded and the computer that controls it. "Database": A computer that maintains information about jobs, trials, clients, input files, output files, and log files in a tabular format. A relational database is used. "Network": Servers and clients, servers and databases, and servers and storage are connected by a communication network such as the Internet. Files, commands, instructions, etc. are exchanged via the network. "User": A person or computer requesting a job to be performed.

[0017] FIG. 1 is a conceptual diagram of grid computing according to the present disclosure. As shown in FIG. 1, multiple clients 11 are scattered across the planet. Each client 11 is a node with computing power, such as an MPU (Media Processing Unit) mounted on an automobile. Each client 11 can communicate with a server 12 via a network. The server 12 causes each client 11 to execute a job.

[0018] In the present disclosure, the server 12 causes multiple clients 11 to simultaneously execute the same job. When a client 11 completes the job first, the execution result of that client 11 is adopted, and execution of the same job by all other clients 11 is stopped. That is, in FIG. 1, the server 12 assigns the same job to multiple clients 11 (clients 1, 2, and 3 in FIG. 1) and causes them to start execution (1). The client 11 that completes the job first (client 1 in FIG. 1) returns the execution result to the server 12 (2). Upon receiving the execution result, the server 12 sends a job stop command to the other clients 11 (clients 2 and 3 in FIG. 1) (3).

[0019] This allows the same job to be executed simultaneously by multiple clients 11, thereby reducing the failure rate of the job. Furthermore, even if the processing time varies among the clients 11, the execution result of the client 11 with the shortest processing time can be adopted. Therefore, the waiting time until the job is completed can be reduced.

[0020] Fig. 2 shows an example of a system configuration in an embodiment. In Fig. 2, a server 12 is connected to each client 11, a database 21, a storage 22, and a user 31 via a network. In Fig. 2, each client 11 has multiple slots, with client 1 having three slots and clients 2 to 4 each having two slots. The database 21 stores data on the congestion status of the network, client information, trial information, and job information.

[0021] 3 shows an example of information stored in the database 21. The client information shown in FIG. 3(a) includes the number of slots and slot information of the client 11. For each slot that the client 11 has, the slot information includes the status of the slot and the ID of the trial that the slot is executing.

[0022] The trial information shown in FIG. 3(b) includes the state of the trial, log file information, the ID of the job of the trial, and the ID of the slot in which the trial is being executed.

[0023] 3(c) includes the priority of the job, the job content, the number of concurrent execution trials of the job, the job status, and the trial ID of the job. The job content includes a command, input file information, and output file information.

[0024] Figure 4 is a state transition diagram of the slot of the client 11. As shown in Figure 4, the slot of the client 11 takes the states of "not executable," "waiting for allocation," and "executing." When the slot becomes executable, the client 11 sends that information to the server 12. At this time, the slot state transitions from "not executable" to "waiting for allocation" (1). When the slot becomes executable, the client 11 sends that information to the server 12. At this time, the slot state transitions from "waiting for allocation" to "not executable" (2). The client 11 is assigned a trial by the server 12. At this time, the slot state transitions from "waiting for allocation" to "executing" (3). When the trial of the client 11 is successful, aborted, or failed, at this time, the slot state transitions from "executing" to "waiting for allocation" (4).

[0025] Figure 5 is a job state transition diagram. As shown in Figure 5, a job can take the following states: "Waiting for allocation," "Executing," "Output file transfer," "Waiting for completion," "Success," and "Failure." When the server 12 determines which client 11 will execute the trial and the first execution begins, the job state transitions from "Waiting for allocation" to "Executing" (1). If one trial is successfully executed, the job state transitions from "Executing" to "Output file transfer" (2). On the other hand, if all trial executions fail, the job state transitions from "Executing" to "Failure" (3). If an output file transfer fails but there are still trials in progress, the job state transitions from "Output file transfer" to "Executing" (4). If all trial output file transfers fail, the job state transitions from "Output file transfer" to "Failure" (5). If one trial succeeds in transferring the output file, the remaining trials (if any) are ordered to be aborted, and the job state transitions from "output file transfer" to "waiting for completion" (6). As soon as the remaining trials are aborted, the job state transitions from "waiting for completion" to "success" (7). If there are no remaining trials, the job state transitions directly from "output file transfer" to "success" (8).

[0026] Figure 6 is a state transition diagram for a trial. As shown in Figure 6, a trial can take the following states: "Waiting for Allocation," "Executing," "Log File Transfer," "Waiting for Output File Transfer," "Output File Transfer," "Success," "Aborted," and "Failed." When a client is assigned and the trial is ready for execution, it transitions from "Waiting for Allocation" to "Executing." If the execution is successful, it transitions to "Log File Transfer," and if the log file transfer is successful, it transitions to "Waiting for Output File Transfer." When a transfer command is received from the server, the output file transfer is executed and it transitions to "Output File Transfer." If the output file transfer is successful, it transitions to "Success." When an abort command is received from the server 12 while in the "Waiting for Allocation," "Executing," "Log File Transfer," or "Waiting for Output File Transfer" states, it transitions to "Aborted." When in the "Executing," "Log File Transfer," or "Output File Transfer" states, if the execution, log file transfer, or output file transfer fails, it transitions to "Failed."

[0027] 7 to 11 are flowcharts showing examples of processing in which the server 12 causes the client 11 to execute a job. Note that, for the sake of simplicity, the processing of one job by the server 12 is shown here, but in reality, processing is performed for multiple jobs simultaneously.

[0028] In FIG. 7, the server 12 receives a job execution request from the user 31 (S11). The job is assigned a priority. The server 12 registers the job in the database 21 and sets the job status to "waiting for allocation" (S12). The server 12 determines the number of concurrent execution trials based on the grid congestion status and the job priority (S13). Here, the grid congestion status refers to the number and percentage of slots in the "waiting for allocation" status among the slots of the client 11.

[0029] The server 12 reserves slots in the "waiting for allocation" state in a number corresponding to the number of concurrently executed trials, and updates the status of the reserved slots to "executing" (S14). The server 12 prepares trials in a number corresponding to the number of concurrently executed trials, and registers the status as "waiting for allocation" in the database 21 (S15). The server 12 updates the status of the job to "executing" (S16). Then, the server 12 distributes the trials of the job to the reserved slots, updates the status of the slots to "executing", and updates the status of the trials to "executing" (S17).

[0030] Thereafter, the server 12 continuously receives signals sent by each trial, specifically, "success," "failure," "cancellation," and "output file transfer waiting" (S18). The processing when "success," "failure," "cancellation," and "output file transfer waiting" are received is shown in Figs. 8 to 11, respectively.

[0031] FIG. 8 shows the process when a "success" signal is received. The server 12 updates the slot status to "waiting for allocation" (S21) and updates the trial status to "success" (S22). The server 12 refers to the database 21 to determine whether there are other trials whose status is "executing" or "waiting for output trial transfer" (S23). If there are other trials whose status is "executing" or "waiting for output trial transfer", the server 12 sends a stop command to the other trials (S24) and updates the job status to "waiting for completion". Then, the process returns to S18 in FIG. 7. On the other hand, if there are no other trials whose status is "executing" or "waiting for output trial transfer", the server 12 updates the job status to "success" (S26) and ends the process.

[0032] FIG. 9 shows the process when a "failure" signal is received. The server 12 updates the slot status to "allocation waiting" (S31) and determines whether the trial status is "file transfer" (S32). If the trial status is "file transfer," the server 12 updates the trial status to "failed" (S33) and determines whether there is a trial in "output file transfer waiting" (S34). If there is a trial in "output file transfer waiting," the server 12 selects one trial in "output file transfer waiting" (S35), updates the status of the selected trial to "file transfer" (S36), and sends an output file transfer command to the selected trial (S37). Then, the process returns to S18 in FIG. 7. If there is no trial in "output file transfer waiting," the server 12 determines whether there is a trial in "execution" (S38). If there is a trial in "execution," the server 12 updates the job status to "execution" (S39). Then, the process returns to S18 in FIG. 7. On the other hand, if there is no trial with a status of "executing", the server 12 updates the job status to "failed" (S3A) and ends the process.

[0033] In S32, if the trial status is not "file transfer", the server 12 updates the trial status to "failed" (S3B) and determines whether or not there are any trials of "execute" or "waiting for output file transfer" (S3C). If there are any trials of "execute" or "waiting for output file transfer", the process returns to S18 in FIG. 7. On the other hand, if there are no trials of "execute" or "waiting for output file transfer", the server 12 determines whether or not there are any other successful trials (S3D). If there are other successful trials, the server 12 updates the job status to "success" (S3E) and ends processing. On the other hand, if there are no other successful trials, the server 12 updates the job status to "failed" (S3F) and ends processing.

[0034] FIG. 10 shows the process when a "cancel" signal is received. The server 12 updates the slot status to "waiting for allocation" (S41) and updates the trial status to "cancel" (S42). The server 12 then determines whether there are other trials of "execute" or "waiting for output file transfer" (S43). If there are other trials of "execute" or "waiting for output file transfer", the process returns to S18 in FIG. 7. On the other hand, if there are no other trials of "execute" or "waiting for output file transfer", the process updates the job status to "success" (S44) and ends the process.

[0035] 11 shows the processing when a "waiting for output file transfer" signal is received. The server 12 updates the trial status to "waiting for output file transfer" (S51) and determines whether the job status is "output file transfer" (S52). If the job status is not "output file transfer", the server 12 updates the job status to "output file transfer" (S53), updates the trial status to "output file transfer" (S54), and sends an output file transfer command to the trial (S55). On the other hand, if the job status is "output file transfer", the process returns to S18 in FIG. 7.

[0036] That is, when the server 12 receives a "success" for a trial, it sends a command to each slot to abort the other trials. When a trial is "succeeded," "aborted," or "failed," the server 12 updates the status of that slot to "waiting for allocation." When one trial is "succeeded" and all other trials have reached either "aborted" or "failed," the server 12 determines that the job has succeeded and updates the job status to "success." When all trials have reached "aborted" or "failed," the server 12 determines that the job has failed and updates the job status to "failed."

[0037] Furthermore, when a trial becomes "waiting for output file transfer" and the other trials are not in "output file transfer" status, the server 12 updates the trial status to "output file transfer" and sends an output file transfer command. When a trial becomes "waiting for output transfer" and the other trials are in "output file transfer" status, the server 12 maintains the trial status as "waiting for output file transfer." When a trial fails to transfer the output file, the server 12 sends an output file transfer command if there is a trial in "waiting for output file transfer" status.

[0038] 12 shows an example of information about each job stored in the database 12. FIG. 13 shows an example of information about each client stored in the database 12.

[0039] As described above, in this embodiment, the server 12 causes multiple clients 11 to simultaneously execute multiple trials. When the server 12 receives a "success" signal from one trial, it sends a command to the client 11 to stop the other trials. This allows the trials to be executed in the shortest possible time, reducing the waiting time until the job is completed.

[0040] In the above-described embodiment, in order to reduce the amount of network transfer, exclusive control is performed in the process of transferring the output file to the storage 22. However, the present disclosure is not limited to this.

[0041] The above-described embodiments are merely examples and should not be construed as limiting the scope of the present disclosure. The scope of the present disclosure is defined by the claims, and all modifications and variations that fall within the scope of the claims equivalents are within the scope of the present disclosure. [Industrial Applicability]

[0042] The technology disclosed herein can reduce the waiting time until a job is completed in grid computing, and is therefore useful for realizing grid computing that uses an MPU mounted on an automobile, for example. [Explanation of symbols]

[0043] 11 Client 12 Server (management device)

Claims

1. 1. A method for performing grid computing in a computer system including a server and a plurality of clients that can communicate with the server via a network, comprising: the server is connected via a network to a database that stores at least client information, the client information including the number of slots for each client and the status of each slot; The server By referring to the client information, a slot in the "waiting for allocation" state is reserved; Sending the same job simultaneously to multiple clients that have reserved slots; When the execution result of the job is received for the first time from any one of the plurality of clients, an instruction to stop execution of the job is transmitted to the other clients among the plurality of clients; The received execution result is adopted as the execution result of the job.

1. A grid computing method comprising:

2. 2. The grid computing method according to claim 1, The server Determine the number of concurrent execution trials based on the grid congestion status indicated by the number and percentage of slots in the "waiting for allocation" state and the priority of the job. Reserve slots in the "waiting for allocation" state for the number of concurrent trials 1. A grid computing method comprising:

3. A management device that executes grid computing using a plurality of clients that can communicate via a network, The system is connected via a network to a database that stores at least client information, the client information including the number of slots for each client and the status of each slot; By referring to the client information, a slot in the "waiting for allocation" state is reserved; Sending the same job simultaneously to multiple clients that have reserved slots; When the execution result of the job is received for the first time from any one of the plurality of clients, an instruction to stop execution of the job is transmitted to the other clients among the plurality of clients; The received execution result is adopted as the execution result of the job. A management device characterized by:

4. 4. The management device according to claim 3, Determine the number of concurrent execution trials based on the grid congestion status indicated by the number and percentage of slots in the "waiting for allocation" state and the priority of the job. Reserve slots in the "waiting for allocation" state for the number of concurrent trials A management device characterized by:

Images