System and method for secure distributed processing across a network of heterogeneous processing nodes

Abstract

A method for distributed processing includes receiving a job packet at a command center, the command center including a processor, a network interface, and a memory. The method includes determining a value of a dimension of the job packet; determining a total processing cost of the job packet based on a predetermined rule applied to the value of the determined dimension of the job packet; and identifying one or more available member devices that are communicatively connected to the command center via a network interface. In addition, the method includes the following operations: dividing the job packet into one or more threads based on at least one of the determined value of the dimension, the total processing cost, or the available member devices, assigning threads of the one or more threads to member devices, and sending the assigned threads to a secure processing environment of the member device via the network interface.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application Serial No. 62 / 637,267, filed on March 1, 2018. This provisional application is incorporated herein by reference as if reproduced in its entirety below. Technical Field

[0003] The present disclosure relates generally to distributed computing and the Internet of Things (IoT). More specifically, the present disclosure relates to systems and methods for secure distributed processing across a network of heterogeneous processing nodes. Background Art

[0004] The growth of processor-enabled control of non-computing devices, sometimes referred to as the "Internet of Things" (IoT), in both public environments (e.g., streetlights, traffic signals, electric meters) and homes (e.g., Internet-connected lights, home automation systems, media streaming platforms) provides vast, untapped opportunities for the collection and processing of data at a scale and level of granularity that was previously unimaginable, at least in part. For example, a controller used in some IoT home automation systems may spend a significant amount of time monitoring and collecting temperatures inside and outside a house during the course of its intended operation. While monitoring and logging temperature data does not typically require a large amount of computation and, therefore, does not require a powerful processor, an IoT controller may still have significant processing and storage resources to support occasional computationally intensive operations, such as voice recognition.

[0005] A modern home may have multiple devices with similar sensor technology and processing capabilities as the home automation controller described above. In summary, these devices can potentially be used to collect large amounts of highly granular data about their environment (e.g., block-by-block temperature data), as well as unused processing power to analyze and process such data. In addition to the possibility of conducting data science and mining data at scale, the ability to coordinate and unify the processing power of a cloud or "fog" of small networked processors provides new opportunities to extend the useful life of older devices (such as older personal computers, tablets, and phones), where the processing and data collection resources of these older devices may no longer be suitable for supporting applications on subsequent generations of the same devices, but can still be effectively used as part of a larger processing network. Further applications of secure distributed processing across a network of heterogeneous processing nodes include, but are not limited to, providing CPU power to support proof-of-work-based systems to verify transactions recorded in a distributed ledger.

[0006] Realizing this vast amount of unused computing power and sensor data presents significant technical challenges, including but not limited to orchestration problems (e.g., distributing processing tasks among many small processors) and security issues, such as protecting the privacy of users and preventing the abuse of a device's sensors and processing power, such as by secretly recording conversations within the house or co-opting a device's processing resources and network connectivity (e.g., turning a device into a "spambot").

[0007] The embodiments disclosed and claimed herein address these technical challenges by providing systems and methods for secure distributed processing across a network of heterogeneous processing nodes. Summary of the invention

[0008] The present disclosure provides systems and methods for secure distributed processing across a network of heterogeneous processing nodes.

[0009] In a first embodiment, a method for distributed processing includes receiving a job bundle at a command center, wherein the command center includes a processor, a network interface, and a memory. In addition, the method includes: determining a value of a dimension of the job bundle; determining a total processing cost of the job bundle based on a predetermined rule applied to the value of the determined dimension of the job bundle. Further operations include identifying one or more available member devices that are communicatively connected to the command center via a network interface, and dividing the job bundle into one or more threads based on at least one of the determined value of the dimension, the total processing cost, or the available member devices. In addition, the method includes the following operations: allocating a thread of the one or more threads to a member device, and sending the allocated thread to a secure processing environment of the member device via the network interface.

[0010] In a second embodiment, a method for distributed processing includes: receiving a thread from a command center via a network interface at a member device including a processor and a memory; receiving control parameters of the thread from the command center via the network interface; and processing the thread based on the control parameters in a secure processing environment of the member device.

[0011] In a third embodiment, a non-transitory computer-readable medium includes program code that, when executed by a processor, causes a command center to receive a job package at the command center, the command center including a processor, a network interface, and a memory. The non-transitory computer-readable medium also includes program code that, when executed by the processor, causes the command center to: determine a value of a dimension of the job package; determine a total processing cost of the job package based on a predetermined rule applied to the value of the determined dimension of the job package; identify one or more available member devices communicatively connected to the command center via a network interface; divide the job package into one or more threads based on at least one of the determined value of the dimension, the total processing cost, or the available member devices; assign a thread in the one or more threads to a member device; and send the assigned thread to a secure processing environment of the member device via the network interface.

[0012] In a fourth embodiment, a command center includes a processor, a network interface, and a memory containing instructions, which when executed by the processor causes the command center to receive a job package. The instructions, when executed by the processor, further cause the command center to: determine a value of a dimension of the job package; determine a total processing cost of the job package based on a predetermined rule applied to the value of the determined dimension of the job package; identify one or more available member devices that are communicatively connected to the command center via the network interface; divide the job package into one or more threads based on at least one of the determined value of the dimension, the total processing cost, or the available member devices; assign a thread of the one or more threads to a member device; and send the assigned thread to a secure processing environment of the member device via the network interface.

[0013] In a fifth embodiment, a member device includes a network interface, a processor, and a memory containing instructions, which when executed by the processor causes the member device to receive a thread from a command center via the network interface; receive control parameters of the thread from the command center; and process the thread based on the control parameters in a secure processing environment of the member device.

[0014] Other technical features will be apparent to those skilled in the art from the following figures, description and claims.

[0015] Before carrying out the following specific embodiments, it may be advantageous to set forth the definitions of certain words and phrases used throughout this patent document. The term "coupling" and its derivatives refer to any direct or indirect communication between two or more elements, whether or not these elements are in physical contact with each other. The terms "send", "receive" and "transmit" and their derivatives cover both direct and indirect communication. The terms "include" and "include" and their derivatives refer to including but not limited to. The term "or" is inclusive, meaning and / or. The phrase "associated with..." and its derivatives refer to including, being included, interconnected with..., including, being included, being connected to or connected with..., being coupled to or coupled with..., being communicable with..., collaborating with..., interlaced, juxtaposed, approaching, being bound to or bound with..., having the property of..., having a relationship with..., etc. The term "controller" refers to any device, system or part thereof that controls at least one operation. Such a controller can be implemented in hardware or a combination of hardware and software and / or firmware. The functions associated with any particular controller can be centralized or distributed, whether locally or remotely. The phrase "at least one of" when used with a list of items means that different combinations of one or more of the listed items may be used, and that only one of the items in the list may be needed. For example, "at least one of A, B, and C" includes any of the following combinations: A, B, C, A and B, A and C, B and C, and A, B, and C.

[0016] In addition, the various functions described below can be implemented or supported by one or more computer programs, each of which is formed by a computer-readable program code and embodied in a computer-readable medium. The terms "application" and "program" refer to one or more computer programs, software components, instruction sets, processes, functions, objects, classes, instances, related data or a part thereof suitable for being implemented in a suitable computer-readable program code. The phrase "computer-readable program code" includes any type of computer code, including source code, object code and executable code. The phrase "computer-readable medium" includes any type of medium that can be accessed by a computer, such as a read-only memory (ROM), a random access memory (RAM), a hard drive, a compact disc (CD), a digital video disc (DVD) or any other type of memory. "Non-transient" computer-readable media does not include wired, wireless, optical or other communication links that transmit transient electrical signals or other signals. Non-transient computer-readable media include media in which data can be stored permanently and media in which data can be stored and rewritten later, such as rewritable optical discs or erasable memory devices.

[0017] Definitions for certain other words and terms are provided throughout this patent document. Those of ordinary skill in the art should understand that in many, if not most, instances, such definitions apply to prior, as well as future uses of such defined words and phrases. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] For a more complete understanding of the present disclosure and its advantages, reference is now made to the following description taken in conjunction with the accompanying drawings, in which:

[0019] Figure 1 illustrates an example of a network context including a command center and heterogeneous processing nodes according to certain embodiments of the present disclosure;

[0020] Figure 2 illustrates an example of a command center according to certain embodiments of the present disclosure;

[0021] Figure 3 illustrates an example of a member device according to certain embodiments of the present disclosure;

[0022] Figure 4 illustrates the layers of an I / O stack implemented in a member device according to certain embodiments of the present disclosure;

[0023] Figure 5 illustrates the operation of a method for secure distributed processing across a network of heterogeneous processing nodes at a command center in accordance with certain embodiments of the present disclosure; and

[0024] Figure 6 Operations of a method for secure distributed processing across a network of heterogeneous processing nodes at member devices are illustrated in accordance with certain embodiments of the present disclosure. DETAILED DESCRIPTION

[0025] Discussed below Figures 1 to 6 The various embodiments used to describe the principles of the present disclosure in this patent document are merely examples and should not be construed in any way to limit the scope of the present disclosure. Those skilled in the art will appreciate that the principles of the present disclosure can be implemented in any suitably arranged wireless communication system.

[0026] Figure 1 An example of a network context 100 including a command center and heterogeneous processing nodes is illustrated in accordance with certain embodiments of the present disclosure.

[0027] According to certain embodiments, network context 100 includes command center 105, one or more machines 110a, 110b, 110c, and 110d providing an interface layer, one or more networks 115, and heterogeneous processing nodes 120a, 120b, 120c, 120d, and 120e.

[0028] exist Figure 1 In a non-limiting example, command center 105 is a management server, such as a Hewlett-Packard Proliant server embodied on a single server rack. According to certain embodiments, command center 105 includes program code that, when executed by one of the cores of a processor of command center 105, causes it to provide an interface layer for connecting to processing nodes 120a-e over a network 115 (such as the Internet). As implied by the presence of machines 110a-d, a variety of architectures for implementing command center 105 are possible and within the scope of the present disclosure. According to some embodiments, command center 105 is a single server, and machines 110a-d include virtual machines executed on the server. According to other embodiments, command center 105 may include multiple physical servers, wherein each of machines 110a-d is implemented on its own server. According to still other embodiments, command center 105 and machines 110a-d are implemented in a cloud across various machines. Many embodiments in which the command center includes a network interface and processing and memory resources for implementing the appropriate interface layer and performing the operations of the disclosed method are possible and within the scope of the present disclosure.

[0029] According to some embodiments, network 115 is a wired network that connects command center 105 to each of heterogeneous processing nodes 120a, 120c, and 120d. According to other embodiments, network 115 is a wireless network, such as a Wi-Fi or 3G wireless network. Network 115 carries communications using context-appropriate network protocols, including but not limited to HTTP, Aeron, Message Queuing Telemetry Transport (MQTT), NanoIP, ROLL (Routing on Low Power and Lossy Networks), uIP, and UDP. Other communication protocols are also possible and within the scope of the present disclosure.

[0030] According to some embodiments, one or more networks of heterogeneous processing nodes 120a-120e are connected to command center 105 via network 115. According to some embodiments, the processing nodes may be directly connected to network 115, such as Figure 1 Alternatively, according to other embodiments, a processing node may be indirectly connected to network 115 via another processing node, such as Figure 1 According to yet other embodiments, the processing nodes may be connected to the network 115 either directly or indirectly, such as Figure 1 For purposes of this disclosure, a processing node connected directly or indirectly to command center 105 via network 115 comprises a “member device” of a processing network under the control of command center 105 .

[0031] According to certain embodiments, the member devices at each of the processing nodes 120a-120e are heterogeneous at least in terms of their processing and job handling capabilities. Aspects of the heterogeneity between the member devices at the processing nodes 120a-120e include, but are not limited to, processing speed, available RAM, power requirements, available storage, type of network connection (e.g., Wi-Fi connection, 3G connection, Gigabit Ethernet), available sensors (e.g., microphone, thermometer, camera, barometer), processor capabilities (e.g., floating point processing capabilities, true random number generation capabilities), and the number of processor cores. According to certain embodiments, the command center can classify member devices based on one or more of these attributes. For example, for the purpose of the command center's job orchestration logic, a device with resources similar to certain generations of Raspberry Pi (e.g., a clock speed of 1GHz and 1GB of storage) will be classified as a "10X" device. As a further example, for purposes of the command center's job orchestration logic, a device with resources similar to certain generations of the Amazon Echo (256MB of RAM, 4GB of storage, and multi-band wireless connectivity) may be classified as a "100X" device.

[0032] According to some embodiments, the heterogeneity of the member devices at a processing node includes additional dimensions, such as their availability to perform work at different times and the processing resources available at different times. Thus, for example, in contrast to a data center where management logic assumes that the processing power of a machine potentially available to process a job is not time-dependent, certain embodiments as disclosed and claimed may operate in a context where the underlying capacity of available processing devices is more fluid and may change, for example, when devices are added or removed from the network or when devices are required to devote computing resources to tasks other than those orchestrated by the command center.

[0033] Although Figure 1 One example of a network context comprising a command center and heterogeneous processing nodes is illustrated, but other embodiments with more or less decentralized architectures are possible. Furthermore, according to some embodiments, the command center and the processing nodes may be embodied on the same device.

[0034] Figure 2 An example of command center 200 is illustrated in accordance with certain embodiments of the present disclosure.

[0035] exist Figure 2In a non-limiting example, command center 200 is implemented on a server and includes memory 205, processor 210, job scheduler 215, and network interface 230. In addition, command center 200 is configured to implement an interface layer 220, which includes one or more instances of an application programming interface (API) shown as 225a-225d, through which command center 200 communicates with networked member devices (e.g., Figure 1 The member devices at the processing nodes 120a-120e in the system interact.

[0036] According to certain embodiments, memory 205 includes non-volatile memory containing program code that, when executed by processor 210, causes the command center to receive job packets via network interface 230, partition the job packets into threads based on one or more dimensions of the determined job packets, assign the threads to available member devices, and send each assigned thread to a secure processing environment on each member device.

[0037] In addition, according to some embodiments, memory 205 includes a library of rules and capability information for member devices acting as processing nodes. According to some embodiments, memory 205 includes a library of rules that maps dimensions of job packages (e.g., a specified deadline for completing a processing task), networking requirements (e.g., are member devices required to send and receive data over a 3G network?), the amount of data to be processed, data parameters to be collected (such as the number of sensors to be used or the time interval at which data is to be collected), the availability of parallel processing, load sharing constraints, processor or sensor requirements (e.g., clock speed, whether a floating point processor or true random number generation is required) to processing cost values.

[0038] According to some embodiments, memory 205 may contain one or more libraries of data specifying the processing capabilities of member devices. In some embodiments, the library of member device capabilities may be maintained at a high level of granularity, where the data for each member device includes, but is not limited to, information about device-specific processor and memory resources. Alternatively, according to other embodiments, the library of device capability information includes a dictionary of device identifiers and a classification determined for the device (e.g., "10X" or "100X"). Many variations are possible and within the scope of the present disclosure.

[0039] In some embodiments, processor 210 is a central processing unit (CPU) chip located in a server. According to other embodiments, command center 200 can be implemented across multiple servers, or as part of a cloud computing system. In such an embodiment, processor 210 includes multiple separate processors operating across dynamically changeable racks (cast) of server machines. According to other embodiments, command center 200 can be implemented on a virtual machine, and processor 210 is a virtual CPU, which includes the assignment of processor resources of (one or more) physical machines that implement the virtual machine.

[0040] In certain embodiments, command center 200 includes a job scheduler 215. According to some embodiments, job scheduler 215 receives information about the availability and usage of member devices at processing nodes via network interface 230. Job scheduler 215 also receives information about currently assigned threads and received job packets from processor 210. In some embodiments, the information about currently assigned threads and received job packets can be further processed and re-expressed as one or more node and command center level metrics of system load. Furthermore, in some embodiments, member devices at processing nodes can periodically send updates about their current processing load and / or job status information to the command center. Figure 2 In a non-limiting example, job scheduler 215 operates to allocate threads among member devices based in part on current usage data received from member devices and expected usage based on information about queued job packets received from memory 205 and processor 210. According to some embodiments, job scheduler 215 dynamically allocates job packets to available member devices based on the availability and capabilities of the devices. In addition, Figure 2 In a non-limiting example, job scheduler 215 implements predetermined rules that are applied, for example, to the determined parameters of the received job packages and the stored information about the capabilities of the member devices at the processing nodes to allocate threads in a manner that optimizes the values ​​of one or more metrics of processing performance. Metrics of processing performance whose values ​​may be maximized at job scheduler 215 include, but are not limited to, job package execution time, power consumption at the processing nodes, redundancy between processing nodes (to increase the success rate of probabilistic execution of threads), and utilization of processing nodes (e.g., it may be desirable to allocate threads of a job package to a minimum number of processing nodes in order to keep other nodes reserved for other job packages).

[0041] According to some embodiments, the assigned thread is transferred from the command center 200 to the member device, and the returned result is provided from the member device to the command center via the interface layer 220. Figure 2In a non-limiting example, the interface layer 220 includes a network interface 230 and application programming interface (API) instances 225a-225d, through which the command center 200 communicates with networked member devices (e.g., Figure 1 The member devices at the processing nodes 120a-120e in the system interact with each other.

[0042] In some embodiments, network interface 230 operates to connect command center 200 to one or more networks (e.g., Figure 1 The network interface 230 may be interconnected with the network 115 in the network. Depending on the embodiment, the network interface 230 may have a network address represented as a node ID, a port number, or an IP address. According to some embodiments, the network interface 230 is implemented as hardware, such as a network interface card (NIC). Alternatively, the network interface 230 may be implemented as software, such as an instance of the java.net.NetworkInterface class. In addition, according to some embodiments, the network interface 230 supports communication through multiple protocols, such as TCP / IP and wireless protocols, such as 3G or Bluetooth.

[0043] exist Figure 2 In a non-limiting example, interface layer 220 includes one or more application programming interfaces (APIs) 225a-225d for establishing communications between command center 200 and member devices at processing nodes. According to some embodiments, each of APIs 225a-225d may designate portions of memory 205 as reserved for input and output of job packets that have been allocated and issued for processing.

[0044] According to some embodiments, each of the APIs 225a-225d maps a set of preset processing functions associated with the performance of the thread to be processed (e.g., to perform a hash function). In some embodiments, the preset processing functions can be represented in an object-oriented format and include methods and properties of the APIs 225a-225d. For example, in some embodiments, the preset functions can be mapped to API objects of an IoT-specific operating system (such as the Conera platform developed by V2COM).

[0045] According to some embodiments, preset processing functions may be mapped to device or operating system specific commands. For example, according to some embodiments, the API may map a command center level function "hash" to a Java method "hashCode". Alternatively, according to other embodiments, each API of API 225a-225d may map preset functions to a command set suitable for the processing capabilities of a particular category of member devices (e.g., the "10X" and "100X" devices discussed above). Alternatively, in some embodiments, there may be only a single API that maps preset functions to commands across the entire operating system range and processing capabilities found in member devices at a processing node. According to yet other embodiments, individual APIs or API sets in API 225a-225d may provide a mapping of command center level preset functions to different data models to support communication across a wide range of networking environments and member device capabilities. For example, in Figure 2 In a non-limiting example, APIs 225a and 225b may provide a representational state transfer (REST) ​​API associated with a JavaScript Object Notation (JSON) data model, while APIs 225c and 225d may provide a simple object access protocol (SOAP) API associated with an extensible markup language (XML) data model. Many variations are possible and are within the intended scope of the present disclosure.

[0046] Figure 3 An example of a member device 300 according to certain embodiments of the present disclosure is illustrated. According to some embodiments, the member device 300 is another "smart" type of device, such as a controller for a home appliance or other household appliance. According to other embodiments, the member device 300 is a smart phone or tablet computer. According to still other embodiments, the member device 300 is a networked meter, or a development board attached to a point of presence of a public facility (e.g., a street light, switch, traffic light, or utility pole). According to still other embodiments, the member device 300 can be a gateway device, such as an Internet of Things (IoT) gateway device. Examples of gateway devices that can be used as member devices 300 include, but are not limited to, Neuron C SmartConcentrator and Dendrion Power Smart Submetering Unit from V2COM, and many embodiments are possible and within the scope of the present disclosure.

[0047] According to some embodiments, member device 300 includes processor 305. Figure 3 In a non-limiting example, processor 305 is a multi-function processor having one or more cores (as opposed to, for example, a function-specific ASIC) that is capable of executing program code stored in non-transitory memory 310. Figure 3 In a non-limiting example, processor 305 includes or is coupled to clock 307, the speed of which provides a measure of the processing power of processor 305. In some embodiments, processor 305 has features such as, for example, but not limited to, floating point processing, true random number generation, or smart caching that are not found in processors of other member devices.

[0048] According to some embodiments, the memory 310 contains program code that, when executed by the processor 305, causes the processor to perform the functions of the member device 300 of which it is a part. Returning to the non-limiting example of a controller for a home automation system discussed herein, the memory 310 contains instructions that, when executed by the processor 305, causes it to perform functions such as reading and writing temperature data to the memory 310, controlling the operation of the input / output device 320 to send control signals for devices within the home (e.g., smart light bulbs, electronically controlled heaters and air conditioners, and electronically controlled curtains). In addition, areas of the memory 310 may be allocated for data storage and storage of additional program code, including one or more applications 315.

[0049] According to some embodiments, application 315 includes program code or software that can be written to memory 310 and read and executed by processor 305. Figure 3 In a non-limiting example, applications 315 include applications associated with core functionality of a member device (e.g., home automation applications, such as applications for automatically controlling Internet-connected light bulbs) and virtual computers hosted on member device 300.

[0050] exist Figure 3 In a non-limiting example, member device 300 has input / output device 320. According to some embodiments, input / output device 300 includes a display (such as an LED display), a keyboard, and a mouse. According to other embodiments, I / O device 320 is a set of input / output pins that are connected to pins of a processor or controller (such as processor 305).

[0051] exist Figure 3 In a non-limiting example, the member device 300 has an operating system (OS) 325 that supports basic functions of the member device 300, including but not limited to executing applications 315, receiving data from sensors 330, and sending and receiving data via network interface 325. According to some embodiments, OS 325 can be a general-purpose operating system, such as Android or iOS. According to other embodiments, OS 325 can be a proprietary operating system, such as Fire OS. According to other embodiments, OS 325 is an IoT-oriented operating system, such as Conera of V2COM.

[0052] According to some embodiments, member device 300 includes sensor 330. Figure 3 In non-limiting examples, sensors 330 include, but are not limited to, cameras, microphones, thermometers, air pressure sensors, light meters, energy monitors, water meters, and rain sensors.

[0053] In some embodiments, the network interface 325 operates to connect the member device 300 to one or more networks (e.g., Figure 1 The network interface 325 may be interconnected with the network 115 in the network. Depending on the embodiment, the network interface 325 may have a network address represented as a node ID, a port number, or an IP address. According to some embodiments, the network interface 325 is implemented as hardware, such as a network interface card (NIC). Alternatively, the network interface 325 may be implemented as software, such as an instance of the java.net.NetworkInterface class. In addition, according to some embodiments, the network interface 225 supports communication through multiple protocols, such as TCP / IP and wireless protocols, such as 3G or Bluetooth.

[0054] Figure 4 Illustrated are layers of an input / output ("I / O") stack implemented in a member device according to certain embodiments of the present disclosure.

[0055] exist Figure 4 In the non-limiting example of , two different member devices 400 and 405 are shown. Figure 4 In a non-limiting example, the processing architecture of each member device 400 is depicted as a hierarchy of abstract layers, sometimes collectively referred to as an “I / O stack.” According to some embodiments, hardware layers 410 and 415 are the base level of the I / O stack of member devices 400 and 405 .

[0056] In some embodiments, each of hardware layers 410 and 415 includes a processor and memory of a memory device (e.g., Figure 3 ), and components of the member device under the direct control of the processor of the member device (e.g., Figure 3 ) as shown in FIG. 3 .

[0057] According to some embodiments, operating systems or OS layers 420 and 425 comprise the next level of the I / O stack of member devices 400 and 405. In some embodiments, OS layers 420 and 425 correspond to the operating systems (e.g., Figure 3In addition, in some embodiments, the OS on each member device is the native or originally provided OS of the member device. For example, if member device 400 is Smartphone, then according to some embodiments, operating system 420 is the "iOS" operating system. Similarly, if member device 405 is reading device, then according to some embodiments, operating system 425 is a "Fire" operating system. Alternatively, according to some embodiments, the operating system at layers 420 and 425 is a specially adapted operating system that replaces or enhances the native operating system of the member device, such as where the native operating system of the member device only supports low-level communications (such as, at the physical layer in the OSI model). Examples of suitable operating systems include, but are not limited to, Conera IoT OS developed by V2COM. According to some embodiments, the alternative operating system can enable member devices to communicate at a higher level within the OSI model and send data in a format beyond the raw bit stream.

[0058] According to some embodiments, virtual machine layers 430 and 435 comprise the next layer in the I / O stack of member devices 400 and 405. According to some embodiments, virtual machine layers 430 and 435 are implemented as Java virtual machines, Azure virtual machines, or as components of a Kubernetes system. Figure 4 In a non-limiting example, each of virtual machine layers 430 and 435 operates to provide a secure container that separates data, memory, and processing resources to perform secure distributed processing across heterogeneous processing nodes. Thus, each of virtual machine layers 430 and 435 separates the execution environment and data used to process the assigned thread from the processes associated with the native or originally intended functionality of the member device. For example, where member device 400 is a network-connected home automation controller, processes associated with the functionality of the member device as a home automation controller, such as an interface to a web-based control application, cannot access data associated with the assigned thread being processed at the member device.

[0059] According to some embodiments, frameworks 440 and 445 comprise the next layer in the I / O stack of member devices 400 and 405. According to some embodiments, frameworks 440 and 445 provide for communication between member devices and between member devices and a command center (e.g., Figure 2 The access point for networked communication between the command center 200 shown in FIG. Figure 4 In the non-limiting example of FIG. 4 , framework layer 440 of member device 400 is shown connected to framework layer 445 of member device 405 .

[0060] According to some embodiments, each of framework layers 440 and 445 may implement a system-specific protocol or use commands (such as in an interface layer (e.g., Figure 2 In such an embodiment, the system-specific protocols and commands may define separate data and control channels for communication between heterogeneous nodes and the command center of the distributed processing network. According to some embodiments, the control channel provided at one of the framework layers 440 and 445 may be used to send part or all of the allocated threads to the member devices. According to other embodiments, the data channel provided at one of the framework layers 440 and 445 may be used to send part or all of the allocated threads to the member devices. In addition, according to some embodiments, the member devices send the threads that have completed processing to the command center via the data channel or the control channel or a combination thereof.

[0061] exist Figure 4 In a non-limiting example, by implementing a system-specific protocol or using commands or API-specific methods such as preset functions of the interface layer of the command center, the frameworks 440 and 445 further strengthen the security of processes and data executed within the virtual machine layers 430 and 435 and upper layers. For example, by responding only to commands and inputs expressed according to preset commands of the command center, the frameworks 440 and 445 may be less susceptible to spoof attacks or requests expressed according to the command set of the operating systems 420 and 425.

[0062] According to certain embodiments, secure processing environments ("SPEs") 450 and 455 comprise the next layer in the I / O stack of member devices 400 and 405. According to certain embodiments, each of secure processing environments 450 and 455 provides a simulation of a trusted execution environment or "secure world" provided by certain physical processors within each of virtual machines 430 and 435. According to certain embodiments, threads 460, 465, 470, and 475 allocated and sent by a command center are processed within each of SPEs 450 and 455. In some embodiments, multiple job threads are executed within a single SPE provided in a member device. According to other embodiments, multiple virtual machines are run on a member device, and each virtual machine provides its own SPE within which a job thread executes. Many variations are possible and within the intended scope of the present disclosure.

[0063] Depending on, but not limited to, the control parameters received from the command center and the underlying processing capabilities of the hardware layers 410 and 415, the threads in each of the SPEs 450 and 455 can be processed in parallel or individually. In addition, in some embodiments, each thread in the SPE is a thread of a different job package received and distributed by the command center. Alternatively, in other embodiments, multiple threads belonging to the same job package are processed in the SPE of a specific member device. In some embodiments, a separate SPE is provided for each thread by the virtual machine and the framework layer. According to such an embodiment, the proprietary data associated with a thread is not available for any other thread, thereby enhancing security.

[0064] refer to Figure 4 The described embodiments are illustrative and are not intended to limit the scope of the present disclosure. Other embodiments not shown in FIG. 4 are possible and are contemplated as a part of the present disclosure. For example, according to certain embodiments, a member device such as member device 400 can implement multiple virtual machines. In addition, each virtual machine can have its own secure processing environment. In addition, according to other embodiments, multiple layers can be combined or implemented as part of a single abstract layer. For example, in some implementations of the Conera operating system utilizing V2COM, Conera OS provides a framework layer and an operating system layer.

[0065] Figure 5 A method 500 for secure distributed processing across a network of heterogeneous processing nodes at a command center is illustrated in accordance with certain embodiments of the present disclosure. Figure 5 In a non-limiting example, job packets are received at a command center, partitioned into threads, and the threads are assigned to member devices. Further, in this illustrative example, threads are assigned according to predetermined rules to optimize the value of one or more metrics of processing performance, such as the time required to process a job packet, or the overall probability of successful processing if probabilistic execution is implemented.

[0066] According to certain embodiments, method 500 includes operation 505, in which a job package is received at a command center. In some embodiments, the job package is a request for a processing task to be performed by a group of networked member devices and is received via an interface provided by the command center. In some cases, the interface for receiving the job package can be a web interface, where parameters are set by a visitor to the website. In other cases, the interface can receive the job package from another machine or process without human intervention. According to certain embodiments, the job package is received as a set of data on which one or more processing operations (e.g., hashing, verification, or optical character recognition) are performed together with a set of processing parameters (e.g., sensor requirements for member devices, deadlines for processing tasks, or fault or interrupt processing parameters). According to other embodiments, the job package can contain executable code and can also be received as an object including methods and properties. According to certain other embodiments, the job package is received via a network (such as, network 115) via a network interface (e.g., Figure 2 ) is received as a request for data collection and processing according to certain operating parameters. Examples of such embodiments include, but are not limited to, ongoing requests to collect and analyze temperature data collected across a group of member devices (e.g., gateways, meters, or controllers located at premises or utility points of presence) covering a particular geographic area. Figure 5 In a non-limiting example, a job package is received in a structured format, wherein the data to be processed conforms to requirements enforced by the interface (such as, the data to be analyzed is received in .SQL format). Other embodiments are also possible, wherein the data to be processed is unstructured (such as, columns of a columnar database), and only the parameters defining the processing to be performed on the data conform to the format expected by the command center. Many variations are possible and are within the intended scope of the present disclosure.

[0067] In some embodiments, the job package received at the command center at operation 505 may have a job package that is configured according to an API (e.g., Figure 2 In one non-limiting example, a job package may be developed based on an API provided with Conera IoT OS.

[0068] In addition, in some embodiments, the job packets received at the command center can also include information defining shared objects and common memory areas within the job packets. For example, some processing operations may require memory resources that exceed the memory resources provided at any one processing node. In this case, the command center may make some of its own memory available as shared memory resources to process the job packets. In some embodiments, the memory available as a shared resource can be an "input" or "output" memory provided by each API of the command center.

[0069] In some cases, the job bundle may be initially structured (or partitioned by the job orchestrator at operation 525) in a manner that allows for probabilistic execution and optimizes the value of one or more metrics of processing performance. In the context of the present disclosure, probabilistic execution encompasses the concept of assigning certain constituent threads of the job bundle to multiple processing nodes in anticipation that one or more processing nodes will fail or be unable to execute their assigned threads. The number of processing nodes to which the threads have been assigned corresponds to the total probability that the threads will be successfully executed by at least one processing node.

[0070] In some embodiments, threads of a job bundle that can be assigned for probabilistic execution can be initially expressed in a job bundle with reference to an idempotent method of a command center API. In the context of the present disclosure, an idempotent method includes operations that provide the same output for a given input regardless of the time or place (e.g., different processing nodes) of execution. According to some embodiments, as part of operation 530, threads of a job bundle expressed with reference to an idempotent method are assigned to multiple processing nodes for concurrent probabilistic execution. Figure 5 In a non-limiting example, all job threads, including the job package itself, are cancelable. In other embodiments, threads that are only assigned for probabilistic execution are each individually cancelable and can be canceled in response to a signal from the command center indicating that the thread has been successfully executed at a different node, or in response to conditions at the processing node (e.g., resource-intensive tasks associated with the node's native functionality, such as speech recognition).

[0071] According to certain embodiments, method 500 proceeds to operation 510, where the command center determines one or more values ​​of the dimensions of the received job packet. In some embodiments, the dimensions of the received job packet determined are the amount of computing horsepower required to process the job packet (e.g., expressed in terms of a number of FLOPS), the memory requirements associated with processing the job packet, and the availability requirements for processing the job packet (e.g., the minimum number of member devices required for successful probabilistic execution, or alternatively, the number of member devices that need to remain free to process or receive other job packets). Other examples of dimensions of job packets include, but are not limited to, memory usage associated with processing the job packet, load sharing constraints between available member devices, processor requirements associated with processing the job packet (e.g., whether a floating point processor or true random number generation is required), sensor requirements (e.g., a job packet that needs to collect temperature data requires some member devices to have a thermometer), deadlines for job packets, interrupt or failure protocols (e.g., ignoring threads that failed to complete on member devices), or networking requirements (e.g., a job packet that needs to analyze wireless base station signal strength requires some member devices to have, for example, 3G connectivity). Additional dimensions of the job packets determined at operation 510 include further analysis of memory requirements associated with processing the job packets, such as the relative amounts of volatile and persistent memory required at one or more processing nodes. Still other dimensions of the job packets that may be determined at operation 510 include a determination as to whether execution memory for the job packets may be shared among multiple processing nodes, and rules for sharing memory resources among processing nodes. By sharing memory resources across processing nodes, certain embodiments according to the present disclosure achieve a degree of biomimicry in their operation because, just as the human brain consolidates memories in specific areas of the brain during rapid eye movement ("REM") sleep, systems according to the present disclosure may similarly consolidate operational memory for executing job packets.

[0072] According to certain embodiments, the determination of the dimensions of the job packet may be performed by analyzing metadata provided as part of a request from an interface at which the job packet is initially received by a command center. For example, when submitting a job for processing, a client of the interface may also specify the dimensions of the job packet and their values. Alternatively, the determination of the values ​​of the dimensions of the job packet may be performed by a rules engine at the command center that applies one or more predetermined rules to analyze the data and instructions of the job packet.

[0073] According to certain embodiments, method 500 then proceeds to operation 515, where the command center determines a total processing cost for processing the job package. In some embodiments, the total processing cost is a static analysis performed by applying one or more predetermined rules to one or more values ​​of a dimension of the job package (e.g., the dimension of the job package determined at operation 510). For example, if at operation 510, the number of member devices required to process the task and the classification of the capabilities of the member devices (e.g., "10X", "100X" devices, etc.) are dimensions of the job package, then in the event that the command center determines that at least five (5) "10X" member devices are required to process the job, a predetermined rule specifying the number of required devices multiplied by the device classification is applied. In this non-limiting example, the job package is determined to have a cost of fifty (50) units.

[0074] According to some embodiments, the total processing cost determined at operation 515 may be used as a basis for a monetary cost to be charged to the entity that submitted the job request. Alternatively, according to other embodiments, such as those in which a large amount of computational work operates as a proxy for scarcity or is used to enforce aspects of security of job packages (e.g., calculation of a large hash), the determination of the total processing cost at operation 515 provides a gating function to exclude fraudulent job packages or job packages that do not meet a desired value for the total processing cost or a desired value for a specified dimension.

[0075] According to certain embodiments, method 500 then proceeds to operation 520, where the command center identifies one or more available member devices that are communicatively connected to the command center via a network interface. Figure 5 In a non-limiting example, member devices that are communicatively connected to the command center via a network send "heartbeat" messages at predetermined intervals to inform the command center of their connectivity and availability. Based on the received "heartbeat" messages, a registry of job packages currently being processed, and information about job packages in the processing queue, the command center is able to determine which member devices are available. According to other embodiments, the command center performs identification of available devices based on member devices that meet defined constraints on the job packages (such as processing capabilities).

[0076] In some embodiments, method 500 then proceeds to operation 525, wherein the control center divides the job packets into threads for processing by available member devices. According to some embodiments, the job packets are divided into threads based on the values ​​of one or more dimensions of the determined job packets. For example, if the job packet needs to process a total of 10 gigabytes of data, and each available member device has only 1GB of available storage, the command center can divide the job packets into ten or more threads based on the memory resources and data volume of the available devices as the dimensions of the determined job packets. According to other embodiments, the job packets are divided into threads based on the total processing cost of the determined job packets or according to the member devices that can be used to process the jobs. According to other embodiments, the command center divides the job packets into threads based on other factors (such as, the deadline for completing the task).

[0077] exist Figure 5 In a non-limiting example, the command center also generates a command set for the member devices as part of dividing the job packets into threads at operation 525. According to some embodiments, the commands for the member devices may be generated according to an API (e.g., Figure 2 225a-225d) or by a framework on a member device that handles the job thread (e.g., Figure 4 In addition, according to certain embodiments of the present disclosure, the division of the job package into threads at operation 525 may include encrypting sensitive or user-specific data to be distributed to member devices as part of the thread for processing. According to such embodiments, the data of the entity submitting the job is protected from possible theft or intrusion of the member device of the thread processing the job package.

[0078] According to some embodiments, at operation 530, each determined thread is assigned to an available member device. Figure 5 In a non-limiting example, such allocation includes: a) specifically assigning each of the determined threads to a member device; b) creating or updating a registry or data structure that records the allocation of job packages to each thread of a particular member device. According to some embodiments, the data structure is maintained at the command center. According to other embodiments, the data structure that records the assignment or allocation of job threads to member devices can be maintained across multiple machines as a safeguard against failure or loss of a registry copy at the command center.

[0079] According to some embodiments, at operation 535, the command center sends each assigned thread to a secure processing environment of a member device (e.g., Figure 5In some embodiments, the allocated threads are sent to the network via the network interface of the command center and then to the framework on each member device. In other embodiments, including but not limited to embodiments using the V2COM Conera API at the command center, the allocated threads are sent through the control channel provided by the API.

[0080] In addition, at operation 535, some embodiments send one or more control parameters, such as a fault / interruption parameter corresponding to each of the assigned threads or a deadline for completing the processing task. In addition, in some embodiments, as part of sending the assigned threads to the SPEs on the available member devices, the command center also sends a copy of the command center's public key for encryption, so that after completing the processing of the threads, the member devices can encrypt their processing results before sending the completed threads back to the command center.

[0081] According to certain embodiments, threads for execution may be sent and triggered separately for execution. For example, at operation 535, the command center may send threads to member devices via a control channel provided by an API at the control center. The threads may then be received at the member devices and preloaded into the secure processing environment of the member devices. After a predetermined condition is met (e.g., a confirmation that a threshold number of threads have been successfully received and loaded into the SPE of the member device is received), the command center may then issue a call stack, thereby triggering the execution of each received thread.

[0082] Figure 6 Illustrated are operations of a method 600 for secure distributed processing across a network of heterogeneous processing nodes at member devices, in accordance with certain embodiments of the present disclosure.

[0083] According to some embodiments, method 600 begins at operation 605, where a member device receives an assigned thread from a command center via a network. According to other embodiments, a member device may receive an assigned thread, such as via a connection with another member device (e.g., Figure 4 440 and 445) indirectly receive the assigned threads from the command center. Depending on but not limited to the processing power and availability of the member devices, each member device may receive only a single thread of the job package from the command center, or the member device may receive multiple threads of the job package from the command center.

[0084] According to some embodiments, each member device receives one or more control parameters associated with processing a thread from the command center at operation 610. Depending on the embodiment, the control parameters may specify at least one of: a deadline for completion of the assigned thread, a fail / abort instruction for processing the assigned thread, or an encryption / decryption instruction for sending the thread to the command center upon completion.

[0085] In some embodiments, at operation 615, the member device processes the assigned thread in the secure processing environment of the member device according to the control parameters received from the control center. In addition, at operation 620, the member device provides the processed thread to the command center via the network. According to some embodiments, as part of sending the completed job thread back to the command center, the member device can sign or encrypt the completed thread using the public key of the command center, or some other trust indicator that the command center received the processed thread from the member device to which the thread was assigned.

[0086] exist Figure 5 In a non-limiting example, application layer security provided by a secure processing environment of a member device is implemented in an I / O stack (e.g., Figure 4 The security of the system is backstopped by the security in the top layer of the I / O stack (shown in FIG. 1 ). Therefore, although the network to which the member devices are connected may be secure, and the member devices themselves may not be secure, the implementation of the secure processing environment at the member devices still allows the system of distributed processing nodes to be secure. According to certain embodiments, because the secure processing environment of each member device is inaccessible to other processes executed on the member devices, the security of the system for distributed processing is enforced. In addition, in some embodiments, the security of the system can be further enforced by encrypting the content entering and exiting the SPE and the SPE only accepting the content using the command set of the API of the command center.

[0087] Nothing in this application should be construed as implying that any particular element, step, or function is essential to be included in the scope of the claims. The scope of the patented subject matter is limited solely by the claims. Furthermore, all claims are not intended to invoke 35 U.S.C. §112(f) unless the exact words "means for..." are followed by a participle.

Claims

1. A method for distributed processing, the method comprising: receiving a job packet at a command center including a processor, a network interface, and a memory; determining values ​​of dimensions of the job bundle, wherein the dimensions of the job bundle include at least one of: memory usage, clock speed of a processor, whether threads of the job bundle are to be processed in parallel, load sharing constraints among available member devices, presence of floating point processors, required sensors, deadlines of the job bundle, processing interruption parameters, availability of true random number generation, or parameters associated with networking capabilities; determining a total processing cost for the job package based on predetermined rules applied to the determined values ​​of the dimensions of the job package; identifying one or more available member devices communicatively connected to the command center via the network interface; partitioning the job packets into one or more threads based on at least one of the values ​​of the determined dimensions, the total processing cost, or the available member devices; assigning threads from the one or more threads to member devices; as well as The allocated thread is sent to the secure processing environment of the member device via the network interface.

2. The method of claim 1, wherein the assigned thread is sent via a control channel between the command center and the member device.

3. The method of claim 1, further comprising: assigning a second thread of the one or more threads to a second member device; as well as The allocated second thread is sent to the secure processing environment of the second member device via the network interface. The method of claim 1 , wherein the threads are allocated to execute in parallel.

5. The method of claim 1, further comprising: The completed assigned threads are received at the command center from the member devices via the network interface.

6. A method for distributed processing, the method comprising: receiving threads from a command center at a member device including a processor and a memory via a network interface, wherein the threads are one or more threads into which the job packet is divided based on a value of a determined dimension of the job packet, a total processing cost of the job packet, or at least one of available member devices, wherein the dimension of the job packet includes at least one of: memory usage, a clock speed of a processor, whether threads of the job packet are to be processed in parallel, a load sharing constraint between available member devices, the presence of floating point processors, required sensors, a deadline for the job packet, a processing interruption parameter, the availability of true random number generation, or a parameter associated with networking capabilities; receiving control parameters of the thread from the command center via the network interface; and The thread is processed based on the control parameters in a secure processing environment of the member device.

7. The method of claim 6, wherein the secure processing environment is provided by a framework operating on a Java virtual machine implemented on the member device.

8. The method of claim 6, further comprising: receiving a second thread from the command center via the network interface, receiving a control parameter of a second thread from the command center via the network interface; as well as processing the thread based on control parameters of the second thread in a second secure processing environment of the member device; Wherein processes and memory of the member device associated with the secure processing environment are isolated from processes and memory associated with a second secure processing environment.

9. The method of claim 6, wherein processes and memory associated with the secure processing environment of the member device are isolated from processes and memory associated with applications running on the member device.

10. The method of claim 7, wherein the framework on the member device is communicatively connected to the framework on a second member device.

11. The method of claim 6, wherein the control parameter is associated with at least one of a schedule for processing the thread, an interrupt protocol for the thread, or a fault protocol for the thread.

12. A non-transitory computer readable medium comprising program code which, when executed by a processor, causes a command center to: receiving a job packet at the command center, the command center comprising the processor, the network interface, and the memory; determining values ​​of dimensions of the job bundle, wherein the dimensions of the job bundle include at least one of: memory usage, clock speed of a processor, whether threads of the job bundle are to be processed in parallel, load sharing constraints among available member devices, presence of floating point processors, required sensors, deadlines of the job bundle, processing interruption parameters, availability of true random number generation, or parameters associated with networking capabilities; determining a total processing cost for the job package based on predetermined rules applied to the determined values ​​of the dimensions of the job package; identifying one or more available member devices communicatively connected to the command center via the network interface; partitioning the job packets into one or more threads based on at least one of the values ​​of the determined dimensions, the total processing cost, or the available member devices; assigning threads from the one or more threads to member devices; as well as The allocated thread is sent to the secure processing environment of the member device via the network interface.

13. The non-transitory computer readable medium of claim 12, wherein the assigned thread is sent via a control channel between the command center and the member device.

14. The non-transitory computer readable medium of claim 12, further comprising program code that, when executed by the processor, causes the command center to: assigning a second thread of the one or more threads to a second member device; and The allocated second thread is sent to the secure processing environment of the second member device via the network interface.

15. The non-transitory computer readable medium of claim 12, wherein the threads are allocated to execute in parallel.

16. The non-transitory computer readable medium of claim 12, comprising program code that, when executed by the processor, causes the command center to: The completed assigned threads are received from the member devices via the network interface.

17. A command center comprising: processor; Network interface; as well as a memory comprising instructions that, when executed by the processor, cause the command center to: Receive the work package, determining values ​​of dimensions of the job bundle, wherein the dimensions of the job bundle include at least one of: memory usage, clock speed of a processor, whether threads of the job bundle are to be processed in parallel, load sharing constraints between available member devices, presence of floating point processors, required sensors, deadlines for the job bundle, processing interruption parameters, availability of true random number generation, or parameters associated with networking capabilities, determining a total processing cost for the job package based on predetermined rules applied to the determined values ​​of the dimensions of the job package; identifying one or more available member devices communicatively connected to the command center via the network interface, dividing the job packet into one or more threads based on at least one of the value of the determined dimension, the total processing cost, or the available member devices, assigning threads of the one or more threads to member devices, and The allocated thread is sent to the secure processing environment of the member device via the network interface.

18. A member device, comprising: Network interface; processor; as well as a memory, the memory comprising instructions, which, when executed by the processor, cause the member device to: receiving threads from a command center via the network interface, wherein the threads are one or more threads into which the job packet is divided based on at least one of a value of a determined dimension of the job packet, a total processing cost of the job packet, or available member devices, wherein the dimension of the job packet comprises at least one of: memory usage, a clock speed of a processor, whether threads of the job packet are to be processed in parallel, a load sharing constraint between available member devices, the presence of floating point processors, required sensors, a deadline for the job packet, a processing interruption parameter, the availability of true random number generation, or a parameter associated with networking capabilities, receiving control parameters of the thread from the command center via the network interface, and The thread is processed based on the control parameters in a secure processing environment of the member device.

Images