Universal distributed computing platform

Abstract

To provide a universal distributed computing platform which utilizes potential computational resources existing in various places.SOLUTION: In one embodiment, a method includes a step of requesting an operation instruction message and operation execution input to a system comprising a plurality of participant devices and a computing resource intermediating between a client device and the participant devices. An instruction set is converted to computation instructions which can be executed by the participant devices participating in computation processing of a platform, and the computation instructions are transmitted to the participant devices, and the participant devices execute the computation instructions. Results of the computation instructions on the participant devices are combined after transmitted to the platform, and processing of transmitting a final computation result to the client device is performed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a universal distributed computing platform (hereinafter referred to as the platform) that utilizes potential computing resources present in various locations. [Background technology]

[0002] In recent years, the performance of consumer devices, including mobile computing, has increased dramatically, but many users are not making full use of their capabilities. In contrast, the computing resources required by companies, corporations, and organizations are increasing year by year, creating room for utilizing potential computing resources.

[0003] Although there are already private distributed computing projects such as those described in Non-Patent Document 1, they are all specialized in performing calculations in specialized fields and have not yet been established as general-purpose computing platforms. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 11-250027 [Non-Patent Document 1] https: / / foldingathome.org Summary of the Invention [Problem to be solved by the invention]

[0005] One of the factors in the previous section is the limitation of participating machines. Currently existing platforms are all targeted at x86 or compatible architectures, and do not natively support the ARM-based architecture, which is currently the mainstream of mobile computing processors. This makes it difficult for people to participate in the platform.

[0006] Another issue is the lack of platform uniformity, which requires participants to install different client applications on their machines for each project, which can be a deterrent to participation in the platform.

[0007] Another problem is the lack of appropriate incentives for participants. Previous private distributed computing projects, such as those described in Non-Patent Document 1, have relied primarily on volunteer participants and lacked appropriate rewards. This is an essential element in creating a long-term sustainable platform. [Means for solving the problem]

[0008] In general, one innovative aspect of the subject matter described herein is an information processing system comprising a plurality of participant devices and a computing resource intermediating between a trustor device and the participant devices, the information processing system comprising: receiving a calculation instruction message provided by a trustor device, the calculation instruction message submitting a calculation to the system; receiving operation execution input data from the entrustor device corresponding to the operation command message provided by the entrustor device; converting the set of instructions into computational instructions executable on participant devices participating in the computing process of the platform; sending a computation command message to the participant device; executing, at the participant device, the received executable computational instructions; sending, on the participant device, a result message of the computational instruction to the platform; and collecting result messages of said computational instructions from the participant devices; combining the results of the calculations; sending the combined calculation result to the entrustor device; The present invention can be embodied in an information processing system that processes the above.

[0009] These and other implementations may optionally include one or more of the following features.

[0010] In some implementations, receiving a certification command message accompanying the computation request from the trustor device to attest to the validity of the computation result, and when collecting result messages of the computation command from the participant devices or during the step of combining the computation results, verifying the result of the computation performed on the participant device based on the certification command message from the trustor device.

[0011] In some implementations, the method includes dividing a computation request from a client device into a serially computable area and a parallel computable area, converting the request into an intermediate language, and merging the results when a result message of the computation command is received.

[0012] In some implementations, the step of converting into computational instructions executable on the participant device includes converting the computational instructions from the platform so that transmission, execution, and response of the computational instructions from the platform to the participant device are completed over HTTP and HTTPS protocols, and converting the computational instructions so that they can be executed in parallel on multi-core processing units in the participant device.

[0013] In some implementations, execution of the computational instructions on the participant devices includes encryption and decryption steps using secret sharing techniques to keep the data encrypted while the computational instructions are executed.

[0014] In some implementations, the method includes a step of sending a retry calculation command message to the same participant device or another participant device for a calculation command message that is determined to be fraudulent as a result of verifying the calculation results performed by the participant device.

[0015] In some implementations, the method includes collecting information including the computing capabilities and network connection status of the participant devices prior to the execution of the computation by the participant devices, and selecting and transmitting a computation message from the trustor that is deemed to have the most appropriate computation time and message transmission time when allocating the computation message to the participant devices.

[0016] In some implementations, a mechanism is included in which a reward is collected from the trustor according to the number of computational instructions executed by the participant device, and the reward is given to the participant device.

[0017] Some implementations include a mechanism that allows users to select the calculation content according to their needs based on information such as the calculation content and purpose disclosed by the client.

[0018] Other embodiments of this aspect include corresponding systems, apparatus, and computer programs configured to perform the actions of the methods encoded on computer storage devices.

[0019] The subject matter described herein can be implemented in particular embodiments to realize one or more of the following advantages. [Effects of the Invention]

[0020] The proposed structure eliminates the traditional limitation on participating machines by converting computational instructions into executable instructions on participant devices, more specifically, into JavaScript or WASM exchanged over HTTP or HTTPS, allowing many machines with Internet connectivity to participate.

[0021] This will allow for more effective use of potential computing resources, and by making effective use of power-efficient processors, it will eliminate the need for large-scale facilities and cooling mechanisms that have been used to operate traditional data centers and servers, which will lead to a reduction in the consumption of scarce resources and greenhouse gas emissions.

[0022] In addition, by converting and dividing the calculation request from the client device into an intermediate language, the calculations that the client can entrust become extremely diverse regardless of the programming language or software operation, thereby improving the situation where participants have to install different client applications on their own machines for each project.

[0023] Furthermore, in response to a calculation request from a client device, the system includes a step of receiving and verifying a proof command message that proves the legitimacy of the calculation result, encryption of the calculation data using secret sharing technology, and a retry mechanism in the event of an invalid result, making it possible to significantly improve the legitimacy and security of calculations, which were not taken into consideration in conventional platforms.

[0024] In addition, participants will be given rewards based on the number of calculation commands executed, and these will be collected from calculation consignors, providing an incentive for participants to continue participating in the platform.

[0025] These efforts will lead to the emergence of more computational contractors, encouraging the formation of a market and an increase in participants, which will boost the overall effect of inventions.

[0026] Various features and advantages of the above-described subject matter are described below in conjunction with the drawings. Further features and advantages will be apparent from the subject matter described in the specification and claims. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is an architecture diagram of a universal distributed computing platform. [Figure 2] FIG. 10 is a diagram illustrating the conversion of a computation command from a trustor device into serial computation and parallel computation via an intermediate language. [Figure 3] A diagram showing the division of calculation commands from a trustor device and the integration of calculation results from participants. [Figure 4]FIG. 10 is a diagram showing a certification command from a trustor device and its execution form. [Figure 5] 10 is a diagram showing encryption of input data from a trustor device using secret sharing and decryption processing when integrating calculation results from participants. FIG. [Figure 6] FIG. 10 is a diagram showing a flow for retrying an abnormal response from a participant. [Figure 7] This is a diagram showing the flow of rewards for participants, entrusters, and platform operators. [Figure 8] FIG. 10 illustrates a flow diagram for collecting participant device information and optimizing the delivery of computational instructions. DETAILED DESCRIPTION OF THE INVENTION

[0028] Like reference numbers and designations in the various drawings indicate like elements.

[0029] Generally, this specification describes systems and techniques for operating a universal computing platform that can execute client-provided computational instructions, including parallel operations, at lower cost, lower emissions, and higher performance than typical data centers.

[0030] The process begins with a client providing a computation request, which in some form is characterized by being written in a high-level language such as C, Java, or Python, involving extensive, large-scale parallel computation, and making it easy to prove the correctness of the computation results.

[0031] First, the reason for writing a high-level language is that when writing a computational program, using a language dedicated to parallel computing is generally a costly choice for most people, and it is not possible to take advantage of the rich library ecosystem that is specific to the language.

[0032] Next, extensive parallel computing capability is required. One advantage of this system over competing data centers equipped with general-purpose servers is that it can provide massive parallel computing capabilities at low cost. Therefore, to maximize the performance of this system, the commissioned calculations must have a certain level of parallel computing capability.

[0033] In addition, the correctness of the calculation results for delegated computation must be provable. This is because there is no way to prove, by itself, that the calculation was performed correctly on the participant's terminal. Furthermore, since the proof calculation cannot be performed on the participant's terminal, it must be performed in steps that are sufficiently small compared to the original calculation instructions. As calculations that satisfy these conditions, most NP problems, and more specifically combinatorial optimization problems, are thought to be the main targets of computation for this system.

[0034] The next step after receiving a computation from a client may be to translate it into an intermediate language for more efficient execution. The exact nature of this language does not affect the system, but it must be at least Turing complete, have a computation section and a proof section, and have a toolchain capable of compiling from a high-level language.

[0035] First, we prove that a Turing-complete implementation enables computation equivalent to the entrusted computation. Furthermore, due to the system constraints mentioned above, computation commands must always be accompanied by proof commands, and this must be guaranteed at the language level. Furthermore, due to the constraints mentioned above, computation requests from the client are made in a general high-level language, so a toolchain is required to convert this into the corresponding intermediate language.

[0036] This intermediate language will be tentatively called Bruck hereafter.

[0037] The computational instructions converted to Bruck are then divided into instruction units in a step called sharding, and then divided into serial and parallel operation parts at the largest unit that can be combined.

[0038] A serial operation part is a sequence of instructions in which the operands of an instruction depend on the previous term. These cannot be divided into parallel instructions, so the calculations are executed consecutively on the same participant terminal.

[0039] The parallel computation part is a set of instructions whose operands depend on anything other than the previous term. These are distributed to an appropriate number of participant devices according to the division number.

[0040] These computational instructions require input data. The first input data provided by the consignor is combined with the first sharded area to form a computation unit called a chunk. Once the computation of one chunk is completed, the result of that chunk becomes available, allowing the next chunk to be generated.

[0041] In certain cases, to comply with the privacy regulations of the trustor, it is necessary to prevent the disclosure of computation data to participants. For this reason, some implementations add a step of encrypting chunks using secret sharing techniques. In this case, one chunk is divided into n shares using a secret sharing technique such as Shamir's secret sharing. Note that the algorithms used for secret sharing are not limited to those listed here.

[0042] The calculation result of the encrypted data is recovered by collecting responses from the n participant devices, recorded as the chunk result, and the process returns to the normal processing flow.

[0043] After generating the chunks, they must be converted into computational instructions that can be executed on the participant's terminal.

[0044] In one form, in order to lower the participation conditions for participant terminals as much as possible, the minimum requirement is that the participant terminal must be able to operate a web browser that can communicate via HTTP or HTTPS protocol, and the data is converted into a language that can be executed on the browser.

[0045] A more modern solution involves using the latest modern browsers to implement a conversion process into WASM, a language designed to run in parallel on multi-core processing units, thereby improving overall system performance.

[0046] The calculation instructions converted into a format that can be executed on the participant terminals are distributed to participants who are online and ready, and the calculations are executed.

[0047] In this case, in some cases, selectivity may be provided as to which participant device the distribution process is to be executed for.

[0048] For example, it may include an algorithm that collects information in advance, such as the stability of radio wave conditions, communication speed, processor power efficiency, and memory capacity, and selects the most appropriate device for the computational command that is about to be delivered.

[0049] The client can also specify the above conditions, which makes it possible to satisfy the client's emission control requirements, for example, by setting conditions such as only allowing participant devices with a certain level of power efficiency to be used for calculations.

[0050] The results of the computational instructions executed by the participant terminals are sent back to the platform.

[0051] The platform persists the received calculation result and uses it as the data to generate the next chunk.

[0052] If there is no response from the client for a certain period of time or if an abnormal response is confirmed, the participant terminal to which the data is to be delivered will be reallocated and the data will be retried.

[0053] When the results accumulate to a provable computation unit, the proof command is executed on the platform to prove the correctness of the computation result.

[0054] If the responses are confirmed to be valid, the results are persisted and combined to become the final output result of the job (commissioned computation).

[0055] If the validity is not proven at this time, the chunk results are discarded, and the participant terminals to which the chunks are to be delivered are reallocated and the process is retried.

[0056] In one form, participants are rewarded for their computational effort, an incentive that can be redeemed for money or other services, thereby ensuring the sustainability of the platform.

[0057] In addition, in this form, negative rewards are given for invalid responses, disconnections, etc., and participation in the platform is suspended for negative rewards above a certain level, which increases the soundness and reliability of the platform.

[0058] While this specification contains many specific implementation details, these should not be considered limitations on the scope of any invention or what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, while features may be described above as working in a particular combination, and may even be initially claimed as such, one or more features of a claimed combination may in some cases be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of a subcombination.

[0059] Similarly, while operations are shown in a particular order in the figures, this should not be understood as requiring such operations to be performed in the particular order shown, or in sequential order, or that all of the operations shown be performed to achieve a desired result. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the division of various system components in the above-described embodiments should not be understood as requiring such division in all embodiments, and it should be understood that the described program components and systems generally can be integrated together in a single software product or packaged in multiple software products.

[0060] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown or sequential order to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous. [Explanation of symbols]

[0061] 100 Platform 200 Consignor Reception Site 300 Calculation command receiving server 400 Precompiler 500 Sharda 600 Chunk Splitter 650 Encryptor 700 Post-Compiler 800 Computational Instruction Distribution Server 900 Participant Registration Site 1000 Calculation result receiving server 1050 Decryptor 1100 Chunk Merger 1200 Shard Merger 1300 Finalizer 1400 Reward Calculation Server 1500 Timeout Trigger 1600 Device information collection server 1700 Device Information Integrated Database 5000 General high-level languages ​​(C, C++, Java, Python, etc.) written directly by the client 5050 Orders for calculation and certification from trustees 5100 An intermediate language that is converted by a precompiler for efficient internal use 5200 Executable Abstract Syntax Trees Partitioned into Serial and Parallel Operations 5250 Executable Abstract Syntax Tree divided into serial and parallel operations, including proof instruction sections 5300 Calculation unit extracted from 5200 5350 Computational instructions extracted from 5250 and converted to executable on participant devices 5400 Computational instructions executable on participant devices converted from 5300 5400 results run on 5500 participant devices 5600 5500 results combined to give 5200 results 5700: The result of the computation on the participant device, whose correctness has been proven by the proof command from 5250. Certification Orders Extracted from 5800 5050 and 5250 5900 Calculation execution instructions including input data encrypted by secret sharing techniques such as additive secret sharing 6000 5900 Calculation results calculated on the participant's device while the input data remains encrypted 6100 is the calculation result of decrypting the calculation result 5500 based on secret sharing technology. 6200 Obtaining a Certification Order 6300 Retry signal due to timeout and incorrect computation result proof 6400, 5500, or 6000 for participant terminal and proof section complexity

Claims

1. An information processing system comprising a plurality of participant devices and a computing resource that mediates between a trustor device and the participant devices, receiving a calculation instruction message provided by a trustor device, the calculation instruction message submitting a calculation to the system; receiving operation execution input data from the entrustor device corresponding to the operation command message provided by the entrustor device; converting the set of instructions into computational instructions executable on participant devices participating in the computing process of the platform; sending a computation command message to the participant device; executing, at the participant device, the received executable computational instructions; sending, on the participant device, a result message of the computational instruction to the platform; and collecting result messages of said computational instructions from the participant devices; combining the results of the calculations; sending the combined calculation result to the entrustor device; An information processing system that processes.

2. receiving a certification command message accompanying the calculation request from the entrustor device, the certification command message certifying the validity of the calculation result; 2. The method of claim 1, further comprising verifying the results of the computations performed on the participant devices based on a certification command message from the trustor device when collecting the computation command result messages from the participant devices or in the step of combining the computation results.

3. The method of claim 1 , further comprising the step of dividing a computation request from the client device into a serially computable area and a parallel computable area, and merging the results upon receiving a result message of the computation instruction.

4. The method of claim 3 , further comprising the step of converting computation messages from the trustor device into an intermediate language for converting the computation messages into serial computation and parallel computable steps.

5. 2. The method of claim 1, wherein the step of converting into computing instructions executable on the participant device further comprises converting the computing instructions from the platform so that transmission, execution, and response of the computing instructions from the platform to the participant device are completed over HTTP and HTTPS protocols.

6. The method of claim 1 , wherein the step of converting into computational instructions executable on the participant device further comprises converting for execution in parallel on multi-core processing units in the participant device.

7. The method of claim 1 , further comprising encryption and decryption steps for executing the computational instructions on the participant devices while keeping the data encrypted using secret sharing techniques.

8. The method of claim 1 , further comprising the step of sending a retry calculation command message to the same participant device or another participant device when a calculation command message is determined to be fraudulent as a result of verifying the calculation execution result by the participant device.

9. The method of claim 1 further comprising the step of collecting information including the computing capabilities and network connection status of the participant devices prior to the execution of the computation by the participant devices, and selecting and transmitting a computation message from the trustor to a participant device that is deemed to have the most appropriate computation time and message transmission time.

10. The method according to claim 1 , further comprising a mechanism for collecting a reward from a trustor according to the number of executions of computational instructions by the participant device and giving the reward to the participant device.

11. The method of claim 1 , further comprising a mechanism for penalizing improper responses or disconnections to computations performed by the participant devices.

12. The method according to claim 1, further comprising a mechanism for enabling the participant to select calculation contents according to needs based on information such as calculation contents and calculation purpose disclosed by the client.

Images