Distributed processing system, distributed processing method, and program

Abstract

A distributed processing system (100) comprises: at least one master node (10); and a plurality of worker nodes (30) each having a computation resource for executing processing in accordance with an instruction from the master node. The master node has a device information collection unit (13) that collects device information of each worker node, a device information transmission unit (15) that transmits, to at least any of the plurality of worker nodes, device information of the other worker nodes, and a processing allocation unit (17) that allocates processing to any of the plurality of worker nodes. Each worker node has a processing execution unit (60) that executes processing allocated from the master node, and a processing sharing request unit (36) that, when the processing allocated from the master node increases too much, requests sharing of the processing to another worker node having the same kind of computation resource (protection region (61), FPGA (62), or the like) on the basis of the device information of the other worker nodes.

Description

[Technical Field]

[0001] The present invention relates to a distributed processing system, a distributed processing method, and a program. [Background technology]

[0002] Conventionally, there is a distributed processing system in which a master node collects equipment information, such as the presence or absence of a protected area (enclave), from multiple worker nodes in advance, and when the master node receives a processing instruction from a user, it selects which worker node will execute the processing based on the collected equipment information and assigns the processing (see, for example, non-patent document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Vaucher, S. et al. “SGX-Aware Container Orchestration for Heterogeneous Clusters.” 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS) (2018): 730-741. Summary of the Invention [Problem to be solved by the invention]

[0004] In a distributed processing system, the master node may allocate not only processes that require calculations in the protected area, but also processes that require calculations in hardware such as FPGAs (field programmable gate arrays).In this case, the master node collects device information from multiple worker nodes, such as the protected area and the presence or absence of hardware such as FPGAs, but this can cause the following problems:

[0005] The first challenge is that when there is a lot of processing that requires both a protected area and an FPGA, the processing is concentrated on the worker node that has both a protected area and an FPGA.

[0006] The second issue is that the master node keeps track of device information such as protected areas and FPGAs, and selects worker nodes to execute processing based on this information. However, since device information is not expected to be shared between worker nodes, the authenticity of devices (computing resources) such as FPGAs and protected areas cannot be confirmed between worker nodes.

[0007] The present invention has been made to solve the above-mentioned problems, and its main objective is to provide a distributed processing system, a distributed processing method, and a program that can improve the imbalance in the allocation of processing to multiple worker nodes. [Means for solving the problem]

[0008] The distributed processing system according to the present invention comprises at least one master node and a plurality of worker nodes each having a computing resource for executing a process in accordance with an instruction from the master node, and the master node stores device information of each worker node. As a service application, information on whether or not there is a protected area where data independence is guaranteed and whether or not the worker node has hardware equipment is included. a device information collection unit that collects the Based on the collected device information of each worker node, a protection domain and a hardware device having the same type of computing resources are selected. Device information of other worker nodes , each worker node with the same type of computing resources The master node has an equipment information sending unit that sends equipment information to the master node, and a processing allocation unit that allocates processing to one of the plurality of worker nodes, and the worker node has a processing execution unit that executes the processing allocated by the master node, and a processing allocation request unit that, when the number of processing allocated by the master node becomes too large, requests other worker nodes that have the same type of computing resources to share the processing based on the equipment information of the other worker nodes. [Effects of the Invention]

[0009] According to the present invention, it is possible to improve the imbalance in allocation of processing to a plurality of worker nodes. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic configuration diagram of a distributed processing system according to an embodiment. [Figure 2] FIG. 10 is a diagram illustrating the operation of the distributed processing system according to the embodiment when collecting device information. [Figure 3] FIG. 10 is a diagram illustrating an operation when processing is allocated in the distributed processing system according to the embodiment. [Figure 4] FIG. 10 is an explanatory diagram illustrating address confirmation between worker nodes in the distributed processing system according to the embodiment. [Figure 5] FIG. 10 is an explanatory diagram illustrating the process of collecting device information in the distributed processing system according to the embodiment. [Figure 6] FIG. 10 is a sequence diagram illustrating a device information collection process in the distributed processing system according to the embodiment. [Figure 7] FIG. 10 is an explanatory diagram illustrating device information sharing between worker nodes in the distributed processing system according to the embodiment. [Figure 8] FIG. 10 is an explanatory diagram illustrating device information sharing between worker nodes in the distributed processing system according to the embodiment. [Figure 9] FIG. 10 is a sequence diagram illustrating device information sharing between worker nodes in the distributed processing system according to the embodiment. [Figure 10] FIG. 10 is an explanatory diagram of processing allocation in the distributed processing system according to the embodiment. [Figure 11] FIG. 10 is a sequence diagram illustrating a process allocation process in the distributed processing system according to the embodiment. [Figure 12] FIG. 1 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of a master node and a worker node according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention (hereinafter referred to as "the present embodiment") will be described in detail with reference to the drawings. Note that each drawing is merely a schematic illustration to allow a sufficient understanding of the present invention. Therefore, the present invention is not limited to the illustrated examples. Furthermore, in each drawing, common or similar components are given the same reference numerals, and redundant explanations thereof will be omitted.

[0012] This embodiment is intended to provide a distributed processing system that distributes and utilizes security resources such as processing in a protected area (enclave), processing in hardware devices such as FPGAs (field programmable gate arrays), and keys among multiple host computers.

[0013] <Configuration of distributed processing system> The configuration of a distributed processing system according to this embodiment will be described below with reference to Fig. 1. Fig. 1 is a schematic diagram of a distributed processing system 100 according to this embodiment.

[0014] As shown in Fig. 1, a distributed processing system 100 according to this embodiment includes at least one master node 10 and multiple worker nodes 30A, 30B, and 30C. Here, three worker nodes, 30A, 30B, and 30C, are used as an example of the worker nodes. The master node 10 is communicatively connected to each of the worker nodes 30A, 30B, and 30C via a network (not shown). The worker nodes 30A, 30B, and 30C are also communicatively connected to each other via a network (not shown).

[0015] The master node 10 is a server that instructs the worker nodes 30A, 30B, and 30C to execute processes. The master node 10 includes a control unit 11 and a storage unit 21.

[0016] The control unit 11 is realized by a central processing unit (CPU) (not shown) of the master node 10 executing a control program AP10 pre-stored in a storage unit 21. The control unit 11 further functions as an authentication information sending unit 12, a device information collecting unit 13, a device information checking unit 14, a device information sending unit 15, an instruction receiving unit 16, a processing allocation unit 17, and a processing result receiving unit 18.

[0017] The authentication information sending unit 12 is means for sending information (authentication information) used for authentication in the worker nodes 30A to 30C to the worker nodes 30A to 30C. The device information collector 13 is a means for collecting device information of each of the worker nodes 30A to 30C. Here, the device information will be described as information representing the configuration of the process execution unit 60 of the worker nodes 30A to 30C. The device information checking unit 14 is a means for checking the device information of the worker nodes 30A to 30C. The device information sending unit 15 is a means for sending device information of other worker nodes to the worker nodes 30A to 30C having the same type of protection means or FPGA based on the collected device information (collected device information 26). Note that this is just an example, and the device information sending unit 15 may send device information of other worker nodes to each of the worker nodes 30A to 30C, and is not limited thereto. The instruction receiving unit 16 is a means for receiving a processing execution instruction from the outside (for example, a terminal device operated by a user). The process allocation unit 17 is a means for allocating a process, for which an external process execution instruction has been received, to one of the plurality of worker nodes 30A to 30C. The processing result receiving unit 18 is a means for receiving processing results from each of the worker nodes 30A to 30C.

[0018] The storage unit 21 stores an ID 22, a private key 23, a public key 24, certificate information 25, collection device information 26, and a control program AP10.

[0019] ID22 is number information unique to the master node 10. The private key 23 is key information used when decrypting encrypted data. The private key 23 is embedded, for example, at the time of manufacturing, and is kept secret from other devices. The public key 24 is key information used when encrypting communications. The public key 24 is information paired with the private key 23 and is used to decrypt information encrypted by the private key 23. This public key 24 is made public to other devices. The certificate information 25 is issued by a trusted third party and is information that guarantees the authenticity of the worker node. The collected device information 26 is device information collected by the master node 10 from each worker node 30 . The control program AP10 is a program for causing a computer to function as the master node 10.

[0020] The worker node 30A is a server that executes processing in accordance with instructions from the master node 10. The worker node 30A includes a control unit 31, a storage unit 41, and a processing execution unit 60. Although not shown in the figure, the worker nodes 30B and 30C are configured in the same manner as the worker node 30A.

[0021] The control unit 31 is realized by a CPU (not shown) of the worker node 30A executing a control program AP30 pre-stored in the storage unit 41. The control unit 31 functions as an authentication information notification unit 32, a processing reception unit 33, a device information notification unit 34, a device information confirmation unit 35, a processing allocation request unit 36, and a processing sending unit 37.

[0022] The authentication information notification unit 32 is a means for notifying the result of authentication by the worker node 30A. The process receiving unit 33 is a means for receiving processes from the master node 10 and other worker nodes 30B and 30C. The device information notification unit 34 is a means for notifying the master node 10 and the other worker nodes 30B and 30C of its own device information. The device information confirmation unit 35 is a means for confirming the device information of the other worker nodes 30B and 30C. The processing allocation request unit 36 ​​is a means for requesting other worker nodes having the same type of computing resources to share the processing when the amount of processing allocated by the master node 10 becomes too large, based on the equipment information of the other worker nodes 30B, 30C sent from the master node 10. The process sending unit 37 is a means for sending information related to the process. The information related to the process includes, for example, a result of executing the process assigned by the master node 10 (completion process), a request (processing allocation request) when a part of the process assigned by the master node 10 is to be shared with other worker nodes 30B, 30C, and a notification (processing allocation request notification) to the master node 10 when a part of the process is to be shared with other worker nodes 30B, 30C.

[0023] The storage unit 41 stores an ID 42, a private key 43, a public key 44, its own device information 45, device information 46 of other worker nodes, and the control program AP30.

[0024] The ID 42 is number information unique to the worker node 30 . The private key 43 is key information used when decrypting encrypted data. The private key 43 is embedded, for example, at the time of manufacturing, and is kept secret from other devices. The public key 44 is key information used to encrypt communications. The public key 24 is made public to other devices. The device information 45 is information about the device itself. The device information 46 is device information of other worker nodes. The control program AP30 is a program for causing a computer to function as a worker node 30.

[0025] The process execution unit 60 is a calculation unit that executes processes assigned by the master node 10. The process execution unit 60 executes, for example, processes that depend on hardware devices (hereinafter, may be referred to as "device-dependent processes").

[0026] The worker nodes 30A to 30C have the same configurations for the control unit 31 and the storage unit 41, but differ in the configuration of the process execution unit 60. FPGA62 The worker node 30B has a protected area 61 and an FPGA 62 in the process execution unit 60. The worker node 30C has a protected area 61 It has the following characteristics.

[0027] Here, a "protected area" refers to an area that is separated in software by system management functions such as an OS (Operating System), where communication from service applications outside the protected area is possible only via a specific API (application programming interface), and where the independence of the internal data is guaranteed.

[0028] An "FPGA (field programmable gate array)" is a type of PLD (programmable logic device) that allows the structure of logic circuits to be changed and redefined. FPGAs can implement any logic circuit depending on the application using a hardware description language (HDL). In the fields of audio and image signal processing and encryption, FPGAs can sometimes achieve calculation speeds 10 to 20 times faster than when performing the same processing on a general-purpose CPU.

[0029] The worker node 30 may be configured to have other computational resources in the processing execution unit 60 instead of or in addition to the protected area 61 and the FPGA 62. Examples of other computational resources include a GPU (Graphics Processing Unit). A GPU is a unit that performs computational processing required for image rendering such as 3D graphics. A GPU can sometimes achieve computational speeds several times to 100 times or more faster than when performing similar processing using a general-purpose CPU.

[0030] <Outline of distributed processing system operation> An overview of the operation of the distributed processing system will be described below with reference to Figures 2 and 3. Figure 2 is an explanatory diagram of the operation of the distributed processing system 100 when collecting device information. Figure 3 is an explanatory diagram of the operation of the distributed processing system 100 when allocating processing.

[0031] As shown in FIGS. 2 and 3, in this embodiment, the master node 10 and each worker node 30 perform the following processing. (1) As shown in FIG. 2 , first, the master node 10 performs device authentication of each worker node 30 and collects device information 74 of the processing execution unit 60, such as the presence or absence of a protected area 61 and the presence or absence of an FPGA 62. At this time, the authentication information sending unit 12 of the master node 10 sends a random number 71 to each worker node 30. In response to this, each worker node 30 sends a signature 72 for the random number 71, a public key 73, and device information 74. The master node 10 performs device authentication of each worker node 30 by receiving the signature 72, public key 73, and device information 74 from the worker node 30. The master node 10 registers the collected device information 74 of each worker node 30 in the collected device information 26.

[0032] (2) Next, the master node 10 sends, to each worker node 30, device information 46 of other worker nodes having the same type of computing resources, based on the collected device information 26. Each worker node 30 stores the device information 46 of other worker nodes sent from the master node 10 in the storage unit 41.

[0033] (3) Next, the master node 10 receives a process execution instruction from the outside (for example, a terminal device operated by a user) at any timing. Then, as shown in Fig. 3, the master node 10 assigns the process to a worker node 30 that has computational resources such as a protected area 61 and an FPGA 62 and is capable of executing the process instructed by the process execution instruction. Here, the explanation is given assuming that the master node 10 sends an assignment process 81 to a worker node 30B that has both a protected area 61 and an FPGA 62 in its process execution unit 60.

[0034] (4) Next, when the number of processes allocated to a certain worker node 30 increases too much, the worker node 30 sends part of the processes to another worker node having the same type of computing resources. Here, it is assumed that the worker node 30B has completed the execution of the process in the FPGA 62 but has not yet completed the execution of the process in the protection domain 61, and requests the worker node 30C, which has the protection domain 61, to share the uncompleted process. In this case, the worker node 30B sends a request for sharing the uncompleted process (processing sharing request 84) to the worker node 30C. The processing sharing request 84 includes information about the completed process 82 (e.g., the execution result of the process in the FPGA 62) and information about the uncompleted process 83 (e.g., the content of the uncompleted process in the protection domain 61). The worker node 30B also sends a notification (processing sharing request notification 85) to the master node 10 indicating that it has requested the worker node 30C to share the uncompleted process. This allows the master node 10 to recognize that the execution result of the process assigned to the worker node 30B is sent from the worker node 30C.

[0035] (5) Next, the worker node 30 (here, worker node 30C) that has been requested to share the uncompleted process executes the requested process, and when the execution of the requested process is completed, sends a completion allocation process 87 to the master node 10. The completion allocation process 87 includes information about the completion process 82 executed by worker node 30B and information about the completion allocation request process 86 executed by worker node 30C. The information about the completion process 82 is, for example, information about the execution result of the process in the FPGA 62 of worker node 30B. The information about the completion allocation request process 86 is, for example, information about the execution result of the process in the protection area 61 of worker node 30C itself. The master node 10 that has received the completion allocation process 87 sends the process execution result to the user's terminal device (the sender of the process execution instruction) based on the completion allocation process 87.

[0036] In this distributed processing system 100, the master node 10 performs prior device authentication of the worker nodes 30A, 30B, and 30C, prior sharing of device information among the worker nodes 30A, 30B, and 30C, and information aggregation at the master node 10. This allows the distributed processing system 100 to verify the authenticity of the protected area 61 and FPGA 62 in the worker node 30 to which the processing is to be divided. Furthermore, when processing is biased toward a worker node 30 with abundant functions (worker node 30B in the illustrated example), that is, when too much processing is allocated from the master node 10 to the worker node 30B, the distributed processing system 100 can allocate part of the processing to a worker node with fewer functions (worker node 30C in the illustrated example). As a result, the distributed processing system 100 can improve bias in processing allocation among multiple worker nodes 30.

[0037] <Example of distributed processing system operation> Specific examples of the operation of the distributed processing system will be described below with reference to Figures 4 to 11. Each figure will focus on the components of the master node 10 and worker nodes 30 that operate in each operation. Note that the description here will be given assuming that there are three worker nodes 30A, 30B, and 30C. However, the number of worker nodes 30 is not limited to three.

[0038] 4, the distributed processing system 100 first checks addresses among the worker nodes 30A, 30B, and 30C in advance.

[0039] In the example shown in FIG. 4, the master node 10 is assigned an IP address of "192.168.10.100." The worker node 30A is assigned an IP address of "192.168.10.2." The worker node 30B is assigned an IP address of "192.168.10.3." The worker node 30C is assigned an IP address of "192.168.10.4."

[0040] The distributed processing system 100 operates as follows when verifying addresses between the worker nodes 30A, 30B, and 30C. (1) The distributed processing system 100 allows a group of worker nodes with computing resources to join a specific multicast address (IP address). (2) The master node 10 sends a request to each worker node 30 to confirm the existence of a worker node 30 that can provide computing resources to the multicast address. (3) The worker nodes 30A, 30B, and 30C send communication information (such as IP addresses) to the master node 10 to notify the master node 10 of their presence.

[0041] As shown in FIG. 5, in the distributed processing system 100, the master node 10 first authenticates the worker nodes 30A, 30B, and 30C with the device of the processing unit 60 and collects device information from the processing unit 60. FIG. 5 is an explanatory diagram of the distributed processing system 100 collecting device information. In the example shown in FIG. 5, the worker nodes 30A, 30B, and 30C store their assigned ID information and certificate information issued by a trusted third party in the storage unit 41 (FIG. 1). The certificate information includes ID information, public key information, private key information, subject information, issuer information, and expiration date information. The device information collection unit 13 of the master node 10 sends a request to the worker nodes 30A, 30B, and 30C to send device information, and collects the device information from the worker nodes 30A, 30B, and 30C. In response to the request, the worker nodes 30A, 30B, and 30C send their device information to the master node 10. Then, the device information collector 13 of the master node 10 collects the device information of the worker nodes 30A, 30B, and 30C and registers it in the device information 26 (FIG. 1).

[0042] 6, the distributed processing system 100 operates as follows when collecting device information: Fig. 6 is a sequence diagram of the distributed processing system 100 when collecting device information.

[0043] When collecting device information, the master node 10 performs device authentication of the process execution units 60 of the worker nodes 30A, 30B, and 30C and collects the device information. At that time, as shown in Fig. 6, the master node 10 checks the device information of the worker nodes 30A, 30B, and 30C. Here, the explanation will focus on the case where the master node 10 checks the device information of the worker node 30A.

[0044] The master node 10 sends a random number to the worker node 30A (step S105). In response to this, the worker node 30A signs the random number using private key information stored in itself, which is the source of the input value (step S110).

[0045] After step S110, the worker node 30A sends the signature and public key information to the master node 10 (step S115). This public key information includes device information of the process execution unit 60 of the worker node 30A.

[0046] After step S115, the master node 10 signs a random number using the public key information received from the worker node 30A, and verifies whether the signature matches the signature received from the worker node 30A to confirm that the worker node 30A is a trustworthy party (step S120). In other words, the master node 10 performs device authentication of the processing execution unit 60 of the worker node 30A using a challenge-response method. Hereinafter, the processing from step S105 to step S120 is referred to as step S130. By the processing of step S130, the master node 10 confirms the device information of the worker node 30A.

[0047] Thereafter, the distributed processing system 100 performs the same processes of steps S131 and S132 as in step S130 on the other worker nodes 30B and 30C, thereby allowing the master node 10 to check the device information of the worker nodes 30B and 30C.

[0048] As shown in Figures 7 and 8, after collecting device information, the distributed processing system 100 shares device information among the worker nodes 30A, 30B, and 30C. Figure 7 is an explanatory diagram of device information sharing among the worker nodes 30A, 30B, and 30C of the distributed processing system 100. Figure 8 is an explanatory diagram of device information sharing among the worker nodes 30A, 30B, and 30C of the distributed processing system 100.

[0049] As shown in Figures 7 and 8, the equipment information sending unit 15 of the master node 10 sends equipment information of other worker nodes to a worker node 30 having the same type of computing resources based on the collected equipment information (collected equipment information 26 (Figure 1)).

[0050] At this time, the device information sending unit 15 of the master node 10 sends device information of the FPGA 62 of the worker node 30B to the worker node 30A which has the FPGA 62 as a computational resource. Also, the device information sending unit 15 of the master node 10 sends device information of the FPGA 62 of the worker node 30A and device information of the protected area 61 of the worker node 30C to the worker node 30B which has the protected area 61 and the FPGA 62 as computational resources. Also, the device information sending unit 15 of the master node 10 sends device information of the protected area 61 of the worker node 30B to the worker node 30C which has the protected area 61 as a computational resource. This allows the distributed processing system 100 to minimize the amount of device information sent and complete the sending of device information in a short time.

[0051] 9, the distributed processing system 100 operates as follows when device information is shared among the worker nodes 30A, 30B, and 30C. Fig. 9 is a sequence diagram when device information is shared among the worker nodes of the distributed processing system 100.

[0052] When device information is shared among the worker nodes 30A, 30B, and 30C, the device information collection unit 13 of the master node 10 sends a request to send device information to the worker node 30A, and in response, the worker node 30A notifies the master node 10 of the device information of the worker node 30A (step S205a). Similarly, the master node 10 sends a request to send device information to the worker node 30B, and the worker node 30B notifies the master node 10 of the device information of the worker node 30B (step S205b). Similarly, the master node 10 sends a request to send device information to the worker node 30C, and the worker node 30C notifies the master node 10 of the device information of the worker node 30C (step S205c). As a result, the device information of the worker nodes 30A, 30B, and 30C is collected, as shown by the device information collection unit 13 in FIG. 7.

[0053] Next, the device information sending unit 15 of the master node 10 sends device information of the FPGA 62 of the worker node 30B to the worker node 30A, which has the FPGA 62 as a computing resource, and has the worker node 30A confirm the device information (step S210a). The device information sending unit 15 of the master node 10 also sends device information of the FPGA 62 of the worker node 30A and device information of the protected area 61 of the worker node 30C to the worker node 30B, which has the protected area 61 and the FPGA 62 as computing resources, and has the worker node 30B confirm the device information (step S210b). The device information sending unit 15 of the master node 10 also sends device information of the protected area 61 of the worker node 30B to the worker node 30C, which has the protected area 61 as a computing resource, and has the worker node 30C confirm the device information (step S210c). By performing these processes, the distributed processing system 100 enables cooperation between worker nodes 30 having the same type of computing resources.

[0054] 10, the distributed processing system 100 performs processing allocation when it receives a processing execution instruction from the outside (for example, a terminal device operated by a user) at any timing. Fig. 10 is an explanatory diagram of processing allocation in the distributed processing system 100.

[0055] The instruction receiving unit 16 of the distributed processing system 100 receives a processing execution instruction from the outside (for example, a terminal device operated by a user) at any timing. In response, the processing allocation unit 17 of the distributed processing system 100 allocates the processing to a worker node 30 having the computational resources capable of executing the processing instructed in the processing execution instruction. Here, a case will be described in which the master node 10 sends an allocation processing 81 to a worker node 30B having a protection domain 61 and an FPGA 62 as computational resources. The allocation processing 81 corresponds to the processing instructed in the processing execution instruction. Here, the allocation processing 81 will be described as including a processing 81a in the protection domain and a processing 81b in the FPGA.

[0056] In the worker node 30B, the process receiving unit 33 receives the allocation process 81 sent from the master node 10 and causes the process executing unit 60 to execute the allocation process 81. In the process executing unit 60, the protected area 61 executes the process 81a in the protected area included in the allocation process 81, and the FPGA 62 executes the process 81b in the FPGA included in the allocation process 81.

[0057] Here, if the number of allocation processes 81 sent from the master node 10 to the worker node 30B increases too much, the worker node 30B requests another worker node that can provide computing resources to share some of the processes. At this time, the device information confirmation unit 35 of the worker node 30B confirms the computing resources of the other worker nodes based on the device information 46 (FIGS. 1 and 2) of the other worker nodes that has been shared in advance, and selects another worker node that can provide computing resources. Here, it is assumed that the worker node 30B has completed execution of processing in the FPGA 62 but has not yet completed execution of processing in the protection domain 61, and requests the worker node 30C, which has the protection domain 61, to share the uncompleted processing. Therefore, it is assumed that the worker node 30B selects the worker node 30C as another worker node that can provide computing resources.

[0058] In this case, the processing allocation request unit 36 ​​in the worker node 30B sends a processing allocation request 84 to the worker node 30C. The processing allocation request 84 requests other worker nodes to allocate the uncompleted processing. The processing allocation request 84 includes information about the completed processing 82 (e.g., the execution result of the processing in the FPGA 62, etc.) and information about the uncompleted processing 83 (e.g., the content of the uncompleted processing in the protection area 61, etc.).

[0059] Furthermore, in worker node 30B, the processing sending unit 37 sends a processing share request notification 85 to the master node 10. The processing share request notification 85 notifies the master node 10 that another worker node has been requested to share the uncompleted processing. In the master node 10, the processing result receiving unit 18 receives the processing share request notification 85. This enables the master node 10 to recognize that the execution result of the processing assigned to worker node 30B will be sent from worker node 30C.

[0060] The worker node 30 (here, worker node 30C) that has been requested to share the uncompleted process executes the requested uncompleted process 83 (processing in the protected area) and, upon completion of the requested process, sends a completion assignment process 87 to the master node 10. The completion assignment process 87 is the execution result of the assignment process 81 sent from the master node 10 to the worker node 30B. The completion assignment process 87 includes information about the completion process 82 executed by the worker node 30B and information about the completion process 88 executed by the worker node 30C. The information about the completion process 82 is, for example, information about the execution result of the process in the FPGA 62 of the worker node 30B. The information about the completion process 88 is, for example, information about the execution result of the process in the protected area 61 of the worker node 30C itself. The master node 10 that has received the completion assignment process 87 sends the execution result of the process to the user's terminal device (the sender of the process execution instruction) based on the completion assignment process 87.

[0061] 11, the distributed processing system 100 operates as follows when allocating work: Fig. 11 is a sequence diagram of the distributed processing system 100 when allocating work.

[0062] During process allocation, the instruction receiving unit 16 of the master node 10 receives a "process requiring a protected area" from the user's terminal device at any timing (step S305a).

[0063] After step S305a, the process allocation unit 17 of the master node 10 allocates the "process requiring a protection area" to the worker node 30C having the protection area 61 (step S306a). In the worker node 30C, the process acceptance unit 33 accepts the "process requiring a protection area" (step S310c), and the protection area 61 executes the "process requiring a protection area" (step S311c).

[0064] Furthermore, during process allocation, the instruction receiving unit 16 of the master node 10 receives a "process requiring an FPGA" from the user's terminal device at any timing (step S305b). The process allocation unit 17 of the master node 10 allocates the "process requiring an FPGA" to the worker node 30A having the FPGA 62 (step S306b). In the worker node 30A, the process receiving unit 33 receives the "process requiring an FPGA" (step S310a), and the FPGA 62 executes the "process requiring an FPGA" (step S311a).

[0065] During task allocation, the instruction receiving unit 16 of the master node 10 receives a "process requiring a protected area and / or an FPGA" from the user's terminal device at any timing (step S305c). The task allocation unit 17 of the master node 10 allocates the "process requiring a protected area and / or an FPGA" to the worker node 30B, which has the protected area 61 and the FPGA 62 (step S306c). In the worker node 30B, the task receiving unit 33 receives the "process requiring a protected area and / or an FPGA" (step S310b), and the FPGA 62 executes the "process requiring an FPGA" (step S311b). However, in this description, it is assumed that the number of allocated tasks 81 sent from the master node 10 to the worker node 30B has increased too much, and therefore only the process using the FPGA 62 is executed in step S311b. In other words, in this description, it is assumed that the execution of the process in the FPGA 62 has been completed, but the execution of the process in the protected area 61 has not yet been completed.

[0066] After step S311b, the device information confirmation unit 35 of the worker node 30B confirms the computing resources of the other worker nodes (here, the device information of the protected area of ​​the worker node 30C) based on the device information 46 (FIGS. 1 and 2) of the other worker nodes that has been shared in advance (step S320), and selects another worker node (here, the worker node 30C) that can provide the computing resources. In other words, when dividing the device-dependent processing, if the memory used in the protected area or the memory used in the FPGA is full, each worker node 30 allocates the uncompleted processing using the memory of another worker node. Note that the allocation destination of the uncompleted processing (the destination of the processing division) may be determined based on the priority information, which indicates the priority, sent by the master node 10 in advance to each worker node 30. The priority may be determined, for example, in descending order of capacity that can be confirmed in the device information.

[0067] After step S320, in worker node 30B, the processing share request unit 36 ​​sends a processing share request 84 to worker node 30C (step S325). The processing share request 84 includes authentication information for worker node 30B. If the capacity of the share destination of the uncompleted processing (the destination to which the processing is divided) is full, the processing waits for execution in another worker node. In addition, the processing sending unit 37 sends a processing share request notification 85 to the master node 10 (step S330).

[0068] After step S325, in the worker node 30C, the device information confirmation unit 35 confirms the authentication information of the worker node 30B included in the processing allocation request 84 (step S326c), and if the authentication information is confirmed, executes the uncompleted processing (processing using the protected area 61) requested in the processing allocation request 84 (step S327c). When the execution of the requested uncompleted processing is completed, the processing sending unit 37 sends the completed allocation processing 87 to the master node 10 (step S328c).

[0069] <Hardware configuration> The master node 10 and the worker nodes 30 of the distributed processing system 100 according to this embodiment are realized by, for example, a computer 900 configured as shown in Fig. 12. Fig. 12 is a hardware configuration diagram showing an example of the computer 900 that realizes the functions of the master node 10 and the worker nodes 30 according to this embodiment. The computer 900 has a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM 903, an HDD (Hard Disk Drive) 904, an input / output I / F (Interface) 905, a communication I / F 906, and a media I / F 907.

[0070] The CPU 901 operates based on a program stored in the ROM 902 or the HDD 904, and performs control by the control units 11 and 31 (FIG. 1). The ROM 902 stores a boot program executed by the CPU 901 when the computer 900 is started up, programs related to the hardware of the computer 900, and the like.

[0071] The CPU 901 controls an input device 910 such as a mouse or keyboard, and an output device 911 such as a display or printer, via an input / output I / F 905. The CPU 901 acquires data from the input device 910 via the input / output I / F 905, and outputs generated data to the output device 911. The input / output I / F 905 corresponds to the input unit and output unit of the master node 10 and the worker node 30.

[0072] The HDD 904 stores programs executed by the CPU 901 and data used by the programs. The communication I / F 906 receives data from other devices via a communication network (e.g., NW (Network) 920) and outputs the data to the CPU 901, and also transmits data generated by the CPU 901 to other devices via the communication network. The communication I / F 906 corresponds to the communication unit between the master node 10 and the worker node 30.

[0073] The media I / F 907 reads a program or data stored in the recording medium 912 and outputs it to the CPU 901 via the RAM 903. The CPU 901 loads a program related to a target process from the recording medium 912 onto the RAM 903 via the media I / F 907, and executes the loaded program. The recording medium 912 is an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto Optical Disk), a magnetic recording medium, a semiconductor memory, or the like.

[0074] For example, when a computer 900 functions as the master node 10 and worker node 30 of the present invention, a CPU 901 of the computer 900 executes a program loaded onto a RAM 903 to realize the functions of the master node 10 and worker node 30. Furthermore, data stored in the RAM 903 is stored in an HDD 904. The CPU 901 reads and executes a program related to a target process from a recording medium 912. Alternatively, the CPU 901 may read a program related to a target process from another device via a communication network (NW 920).

[0075] <Effects> The effects of the distributed processing system 100 according to the present invention will be described below. (1) As shown in Fig. 1, a distributed processing system 100 according to this embodiment includes at least one master node 10 and multiple worker nodes 30A, 30B, and 30C each having computational resources for executing processing in accordance with instructions from the master node 10. The master node 10 includes an equipment information collection unit 13 that collects equipment information about each of the worker nodes 30A, 30B, and 30C, an equipment information sending unit 15 that sends equipment information about other worker nodes to at least one of the multiple worker nodes 30A, 30B, and 30C, and a process allocation unit 17 that allocates processing to one of the multiple worker nodes 30A, 30B, and 30C. The worker node 30 is characterized by having a processing execution unit 60 that executes processing assigned by the master node 10, and a processing allocation request unit 36 ​​that, when the number of processes assigned by the master node 10 becomes too large, requests other worker nodes having the same type of computing resources (protected area 61, FPGA 62, etc.) to share the processing based on the equipment information of the other worker nodes.

[0076] In this way, the distributed processing system 100 according to the present invention performs prior device authentication of the worker nodes 30A, 30B, and 30C at the master node 10, shares prior device information among the worker nodes 30A, 30B, and 30C, and aggregates information at the master node 10. This allows the distributed processing system 100 to verify the authenticity of the protected area 61 and FPGA 62 within the worker node 30 to which the processing is to be divided. Furthermore, when processing is biased toward a worker node 30 with a wide range of functions (in the illustrated example, worker node 30B), that is, when too much processing is allocated from the master node 10 to the worker node 30B, the distributed processing system 100 can allocate part of the processing to a worker node with fewer functions (in the illustrated example, worker node 30C). As a result, the distributed processing system 100 can improve bias in processing allocation among multiple worker nodes 30.

[0077] (2) As shown in FIG. 5, in the distributed processing system 100 of (1), the worker nodes 30A, 30B, and 30C have a memory unit 41 that stores certificates and key information, and the master node 10 has a memory unit 21 that stores the certificates and key information stored by the worker nodes 30A, 30B, and 30C.

[0078] By doing this, the distributed processing system 100 can perform prior equipment authentication of the worker nodes 30A, 30B, and 30C at the master node 10, prior sharing of equipment information between the worker nodes 30A, 30B, and 30C, and aggregation of information at the master node 10.

[0079] (3) As shown in Figures 2 and 5, in the distributed processing system 100 of (1), the equipment information of other worker nodes sent from the master node 10 to each worker node 30 may include information on the computational resources possessed by the other worker nodes and information on the protection areas possessed by the other worker nodes.

[0080] In this way, the distributed processing system 100 can request other worker nodes to share part of the processing assigned by the master node 10 by sharing device information in advance among the worker nodes 30A, 30B, and 30C.

[0081] (4) As shown in FIG. 5, in the distributed processing system 100 of (1), the worker node 30 stores the ID, public key, and private key assigned to the computing resources it owns in the memory unit 41 as equipment information, and may further have an equipment information verification unit 35 that verifies the authenticity of other worker nodes based on the equipment information of the other worker nodes.

[0082] In this way, the distributed processing system 100 can verify the authenticity of other worker nodes.

[0083] (5) In the distributed processing system 100 of (1), the processing allocation unit 17 of the master node 10 may allocate processing so that the processing is completed on each worker node 30 based on the equipment information of each worker node 30 collected by the equipment information collection unit 13 in accordance with processing execution instructions received from outside.

[0084] In this way, the distributed processing system 100 allows the master node 10 to allocate processing so that processing is completed by each worker node 30.

[0085] (6) In the distributed processing system 100 of (1), when the amount of processing assigned by the master node 10 becomes too large, the worker node 30 may verify the authenticity of the other worker nodes based on the certificates contained in the equipment information of the other worker nodes, and may request other worker nodes with similar computing resources to share the processing.

[0086] In this way, the distributed processing system 100 can request other worker nodes to share part of the processing assigned by the master node 10.

[0087] The present invention is not limited to the above-described embodiments, and many modifications can be made by a person having ordinary skill in the art within the technical concept of the present invention.

[0088] For example, the processing execution unit 60 is not limited to the protected area 61 and the FPGA 62, but may be other computing resources such as a GPU. [Explanation of symbols]

[0089] 10 master nodes (servers) 11 Control section 12 Authentication Information Sending Department 13 Equipment Information Collection Department 14 Device information confirmation section 15. Equipment Information Sending Department 16 Instruction Reception Department 17 Processing allocation unit 18 Processing result receiving section 21 Memory section 22 ID 23 Private key 24 Public Key 25 Certificate Information 26. Collected Device Information 30, 30A, 30B, 30C Worker nodes (servers) 31 Control Unit 32 Authentication information notification section 33 Processing Reception Department 34 Device information notification section 35 Device information confirmation section 36 Processing Assignment Request Department 37 Processing and Sending Department 41 Storage section 42 ID 43 Private key 44 Public Key 45 Device information 46 Device information 60 Processing execution unit 61 Protection Area (Computational Resources) 62 FPGA (computing resources) 71 Random Numbers 72 signatures 73 Public Key 74 Device information 81 Distribution processing 81a Processing in a protected area 81b FPGA processing 82 Completion processing (processing on FPGA) 83 Incomplete processing (processing in protected area) 84 Processing allocation request 85 Processing allocation request notice 86 Completion sharing request processing (processing in the protected area) 87 Completed allocation process 88 Completion processing (processing in protected area) 100 Distributed Processing System 900 Computers 901 CPU 902 ROM 903 RAM 904 HDD 905 Input / Output Interface 906 Communication I / F 907 Media I / F 910 Input Device 911 Output Device 912 Recording Media 920 NW AP10 control program AP30 control program

Claims

1. At least one masternode; a plurality of worker nodes each having a computing resource for executing a process in accordance with an instruction from the master node; The master node: a device information collection unit that collects, as device information of each worker node, information on whether or not there is a protected area in which data independence from the service application is guaranteed and whether or not the worker node has hardware devices; a device information sending unit that sends device information of other worker nodes having the same type of computing resources in the protection domain and the hardware device to each worker node having the same type of computing resources based on the collected device information of each worker node; a process allocation unit that allocates a process to any one of the plurality of worker nodes, The worker node: a processing execution unit that executes the processing assigned by the master node; a processing allocation request unit that requests other worker nodes having the same type of computing resources to share the processing when the number of processes allocated from the master node becomes too large based on the device information of the other worker nodes. A distributed processing system characterized by:

2. 2. The distributed processing system according to claim 1, The worker node has a storage unit that stores certificates and key information, The master node has a storage unit that stores the certificate and key information stored in the worker node. A distributed processing system characterized by:

3. 2. The distributed processing system according to claim 1, The worker node: The device information includes an ID, a public key, and a private key assigned to the computing resource of the device itself, and the device information is stored in a storage unit. The device information verification unit further includes a device information verification unit that verifies the authenticity of the other worker node based on the device information of the other worker node. A distributed processing system characterized by:

4. 2. The distributed processing system according to claim 1, The process allocation unit of the master node allocates processes so that the processes are completed at each worker node based on the device information of each worker node collected by the device information collection unit in accordance with a process execution instruction received from an external device. A distributed processing system characterized by:

5. 2. The distributed processing system according to claim 1, When the number of processes allocated by the master node becomes too large, the worker node verifies the authenticity of the other worker nodes based on the certificates included in the device information of the other worker nodes, and requests other worker nodes having the same type of computing resources to share the processing. A distributed processing system characterized by:

6. A distributed processing method for distributing and executing processing among a plurality of worker nodes in accordance with instructions from a master node, The master node collects, as device information of each worker node, information on whether or not there is a protected area in which data independence from a service application is guaranteed and whether or not the worker node has hardware devices; The master node sends device information of other worker nodes having the same type of computing resources in the protection domain and the hardware device to each worker node having the same type of computing resources based on the collected device information of each worker node; The master node assigns processing to any one of the plurality of worker nodes; a step in which the worker node executes the processing assigned by the master node; When the number of processes allocated by the master node becomes too large, the worker node requests other worker nodes having the same type of computing resources to share the processes based on the device information of the other worker nodes. A distributed processing method characterized by:

7. A program for causing a computer to function as a master node that instructs a worker node to execute a process, The computer A procedure for collecting, as device information for each worker node, information on whether or not there is a protected area in which data independence from the service application is guaranteed and whether or not the worker node has hardware devices; a step of sending device information of other worker nodes having the same type of computing resources in the protection domain and the hardware device to each worker node having the same type of computing resources based on the collected device information of each worker node; A program that executes the procedure for allocating processing to an arbitrary worker node.

Images