Method and system for data processing
By parsing and compiling query expressions in edge computing devices and central server systems, rapid and seamless software function deployment and management of IoT systems are achieved, solving the problems of functional flexibility and scalability, and supporting ease of use for non-professional users.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- STREAM ANALYZE SWEDEN AB
- Filing Date
- 2024-02-16
- Publication Date
- 2026-04-21
AI Technical Summary
Existing IoT systems lack functional flexibility, have limited performance and scalability during initial design, deployment, development, and maintenance, and require professional technicians for system deployment and maintenance.
The system employs edge computing devices and a central server to generate execution plans by parsing query expressions and compiling them into platform-independent assembly code. Interpreted software functions on the edge devices execute both compiled and uncompiled parts, enabling seamless deployment and dynamic management of software functions.
It enables rapid and seamless deployment and management of software functions in heterogeneous hardware environments, supports ease of use for non-professional users, improves the system's functional flexibility and scalability, and meets the performance requirements of large-scale IoT systems.
Smart Images

Figure CN121901271A_ABST
Abstract
Description
[0001] This application is a divisional application of patent application No. 202480021052.9, filed on September 23, 2025, entitled "Method and System for Data Processing".
[0002] This invention relates to methods and systems for data processing. Specifically, the methods and systems of this invention relate to data processing in an Internet of Things (IoT) environment, which includes multiple physically separated edge devices and may include one or more central devices, all of which are interconnected for digital communications such as those over the Internet.
[0003] More specifically, the present invention relates to a method for distributing and deploying software functionality on such systems.
[0004] More specifically, the present invention relates to the processing of data collected by such edge devices, and to the generation of processed data useful for automatically determining and performing various actions.
[0005] In IoT applications, it is known to use more or less autonomous edge devices that can serve as sensors and / or participants in IoT systems. Such edge devices can be hardware-based and also more or less software-based. For example, such edge devices may be equipped with dedicated hardware that implements certain logic, or are generally programmable, running systems such as FreeRTOS or Android. © The operating system.
[0006] Typically, once such a system is installed along with an edge device, the installation is often relatively static. For example, in many conventional systems, activities such as installing add-ons, testing, simulating, and changing functionality are time-consuming. Even when parameters or similar tools can be used to configure the edge device, its functionality may be limited to the use cases envisioned by the installer.
[0007] Therefore, one problem with conventional distributed IoT systems is the lack of functional flexibility in initial design, deployment, and further development and updates. This is especially true when using existing and legacy hardware.
[0008] Given the large amounts of measurement data that many of these systems need to process, another issue is performance and scalability. Real-time requirements are often challenging, and the available hardware resources at each edge component are typically limited.
[0009] Another issue is maintenance, including testing and simulating new or hypothetical operating scenarios.
[0010] Another issue is usability. There is a desire for a system that can be deployed, used, developed, and maintained across a wide range of applications without requiring in-depth knowledge of its programming or technical details.
[0011] Therefore, there is a desire to provide a large-scale, scalable IoT system that includes multiple edge devices and may include one or more central units, offering improved flexibility in functionality during design, deployment, further development and maintenance, and is easy to deploy.
[0012] SE 2050998-0 A discloses a general configuration for a large-scale distributed system capable of achieving at least one of the above objectives.
[0013] However, there is a desire to achieve a way to distribute software functionality across such systems that allows for seamless and rapid deployment and use of such functionality across any sub-part of such systems, even in the presence of heterogeneous prerequisites such as hardware, operating systems, etc.
[0014] Such systems and methods for distributing and deploying software functions should also be able to distribute and deploy software functions dynamically and interactively in real time by non-programmer users, for example in the general manner described in the previous Swedish patent application mentioned above.
[0015] The present invention solves at least some of the problems mentioned above.
[0016] Therefore, embodiments of the present invention relate to a system comprising a plurality of edge computing devices and a first central server, each such edge computing device and each such central server further comprising: a memory; a central processing unit (CPU); and a digital communication interface arranged to allow digital communication over a digital communication network. Each of the edge computing devices may also include a sensor. The central server may be arranged to accept queries from a querying party and via the digital communication interface, the result of which includes at least one stream of data objects, at least one of which is an endless stream of objects calculated based on data measured by the sensors and provided continuously or intermittently over time and communicated via the digital communication interface, the endless stream having no defined endpoints at the time of the query.
[0017] The central server can be configured to perform the following steps: a) parse the query to generate a parsed query expression; b) generate an execution plan corresponding to the parsed query expression; and c) compile the at least part of the execution plan by converting at least part of the execution plan into corresponding platform-independent assembly code, thereby obtaining a at least partially compiled execution plan.
[0018] At least one of the edge computing devices can be configured to subsequently receive at least a partially compiled execution plan via the digital communication interface.
[0019] The at least one edge computing device in the edge computing device may be configured with an interpreted software function that is configured to execute on the CPU of the edge computing device and to run both the compiled and uncompiled portions of the at least partially compiled execution plan by means of an interpretation of at least a portion of the at least partially compiled execution plan.
[0020] Each edge computing device may be configured to, when running the at least partially compiled execution plan, produce a first result of the at least partially compiled execution plan and communicate the first result via the digital communication interface.
[0021] The at least partially compiled execution plan can define the calculations to be performed based on measurements from the sensor.
[0022] The edge computing device under consideration can be arranged to perform the computation as part of the explanation.
[0023] The query can be defined according to a query language with a predetermined syntax, which is arranged to define a query whose result includes at least one data stream and allows the query to be defined based on calculations to be performed based on measurements from the sensor.
[0024] The query language can be an object-oriented query language.
[0025] Parsing can include resolving a query into an equivalent abstract syntax tree, such as resolving it into an S-expression.
[0026] Parsing can be a representation of one or more of the following types: declarative; object-oriented; based on predicate logic, such as using multi-directional external functions; strongly typed; and relational.
[0027] This representation can be extended by providing primitives for user-defined predicates, such as providing multidirectional external function aspects with different specific implementations depending on the known / unknown state of the external function parameters.
[0028] The compilation process may include a central server transcribing at least a portion of the assembly code into platform-specific assembly code.
[0029] Any uncompiled portion of the execution plan can be executed by the interpreter software function of the edge computing device interpreting the considered portion of the execution plan; while any compiled portion of the execution plan can be executed by the interpreter software function of the edge computing device directly initiating its execution through the CPU of the edge computing device.
[0030] During the compilation process, there may be cases where only the arithmetic portion of the execution plan, or at least the arithmetic portion, will be compiled.
[0031] The central server can be configured to store platform-independent assembly code for later reuse.
[0032] The central server can be configured to transcribe the stored platform-independent assembly code into platform-specific code for different edge computing devices.
[0033] Each sensor and / or edge computing device can be referenced using a global namespace or set of attributes according to the syntax described.
[0034] Each edge computing device may be configured to discard at least some, preferably all, of the measurement data measured by the sensor, rather than storing the measurement data in the memory, after the measurement data has been communicated through the digital communication interface and / or a piece of information such as the result calculated based on the measurement data.
[0035] The system may also include a distributed or monolithic central server, with each edge computing device having a client role relative to the central server, wherein all communication between the edge computing devices can then be conducted via the central server.
[0036] The interpreter software function may include an external application programming interface (API) configured to allow external computer code to call expressions using the query language and / or configured to allow external computer code to call expressions using the query language.
[0037] The interpreter software functions may include machine learning functions arranged to define, train, and execute machine learning models such as neural networks.
[0038] The memory of the edge computing device may include definitions of query references and trained machine learning functions interpreted by interpreting software functions executed on the edge computing device.
[0039] The interpreted software function under consideration can be configured to train the defined machine learning function based on the results of the query.
[0040] Embodiments of the present invention relate to a method for collecting data in a system comprising a plurality of edge computing devices and a first central server, each such computing device and each such central server further comprising: a memory; a central processing unit (CPU); and a digital communication interface arranged to allow digital communication over a digital communication network; each of the edge computing devices further comprising a sensor.
[0041] The method may include the following steps:
[0042] a) The central server receives a query from the querying party and via the digital communication interface, the result of which includes at least one stream of data objects, at least one of which is an endless stream of objects, the endless stream being calculated based on data measured by the sensor and provided continuously or intermittently over time and communicated via the digital communication interface, the endless stream having no defined endpoints at the time of the query;
[0043] b) The central server parses the query to generate a parsed query expression;
[0044] c) The central server generates an execution plan corresponding to the parsed query expression, the execution plan defining the computations to be performed based on measurements from the sensors;
[0045] d) The central server compiles said at least part of the execution plan by translating at least a portion of the execution plan into corresponding platform-independent assembly code;
[0046] e) At least one of the edge computing devices receives, via the digital communication interface, at least a partially compiled execution plan; and
[0047] f) The interpretation software function of at least one edge computing device in the edge computing device is executed on the CPU of the edge computing device and runs both the compiled and uncompiled portions of the at least partially compiled execution plan to produce a first result of the at least partially compiled execution plan, the interpretation including the execution of the computation.
[0048] In the following, the invention will be described in detail with reference to exemplary embodiments and accompanying drawings, wherein:
[0049] Figure 1 A client device according to the present invention is shown;
[0050] Figure 2 This is an overview of a system according to the invention suitable for carrying out the method according to the invention;
[0051] Figure 3 This is an overview of the domain name server database configuration;
[0052] Figure 4a This is the first overview explaining the architecture of software functions;
[0053] Figure 4b This is the second overview explaining the architecture of software functions, showing a more... Figure 4a More details on the core functionality of the explained software functions;
[0054] Figure 5This illustrates a combination of edge computing devices and a central server;
[0055] Figure 6 This is a flowchart illustrating the first method;
[0056] Figure 7 This is a flowchart illustrating the second method;
[0057] Figure 8 This is a flowchart illustrating the third method;
[0058] Figure 9 This is a flowchart illustrating the fourth method;
[0059] Figure 10 The information flow in the system is illustrated;
[0060] Figure 11a An exemplary full-engine edge computing device is illustrated; and
[0061] Figures 11b to 11d First, second and third exemplary thin engine edge computing devices are illustrated respectively.
[0062] In the accompanying drawings, the same reference numerals are used to denote the same or corresponding parts.
[0063] This invention relates to, for example Figure 2 System 100, generally shown in the figure, includes Figure 1 Several edge computing devices (EDCs) of the general type shown are 110, 120, 130, and 140.
[0064] Each such edge computing device (EDC) is a piece of computing hardware that includes corresponding sensors (S), memory (M), a central processing unit (CPU), and a digital communication interface (IF).
[0065] The sensor S can be any type of sensor arranged to measure parameters at the edge computing device EDC under consideration. Parameters can represent physical properties at or near the edge computing device EDC, where the sensors can be: light sensors; camera sensors; temperature sensors; sound sensors; electrical or magnetic sensors, such as current, voltage, impedance, or magnetic field sensors; orientation sensors, such as gyroscopes or accelerometers; pressure sensors; chemical sensors, such as pH sensors or sensors for sensing specific solid, liquid, and / or gaseous chemicals; and so on. Parameters can also represent logical properties that can be read by the edge computing device EDC, such as settable doors, buttons, logical states delivered by auxiliary devices, and so on.
[0066] In this context, it is important to arrange the sensor S to sense specific conditions at the “edge” location of the edge computing device EDC. The computing device being an “edge” device means it is physically located at the location relevant to the sensed conditions. System 100 typically includes multiple such edge computing device EDCs, each located at a corresponding physical location, and the physical locations of different edge computing device EDCs are typically different. This means that system 100 covers multiple such physical locations where such measurements can be performed by the edge computing device EDC. Thus, data sensed from multiple different physical locations can be cascaded by one or more central servers CS associated with the edge computing devices in the manner described herein to form an overall view of some types of situations involving the multiple physical locations.
[0067] Each central server CS may or may not include sensors S of the type described herein.
[0068] The sensor S may include an analog-to-digital converter to convert sensed analog values (such as temperature) into a digital representation that can be fed to the CPU.
[0069] Of course, each edge computing device (ECD) may include more than one sensor S of the same and / or different types. All such sensors are connected to a memory M via a suitable digital communication bus, so that computer software functions executing on the CPU can access the measurements received from the sensors S in question.
[0070] Similarly, each edge computing device (ECD) may also include one or more actuators A, such as optical, mobile, electric, or magnetic actuators, as appropriate. All such actuators A are connected to the memory M via the digital communication bus, such that computer software functions executing on the CPU can apply actuation at the actuator A in question.
[0071] The memory M can be a RAM memory, and preferably a digital memory interconnected with both the sensor S and the CPU.
[0072] The CPU can be any suitable CPU, such as a single-core or multi-core CPU.
[0073] Edge computing devices (EDCs) can be physically independent programmable devices. They can be general-purpose programmable devices or programmable devices with hardware limited to a specific purpose. In this context, it is important that they can be programmed using machine code that can be fed into memory M from an external source and executed by the CPU. For example, an edge computing device (EDC) could be a programmable sensor or a PC laptop computer.
[0074] Edge computing devices (EDCs) may include an operating system configured to execute on a CPU and provide higher-level services to computer software executing on the CPU within an environment provided by the operating system. However, such an operating system is not required in all embodiments of the invention.
[0075] The digital communication interface (IF) can be any suitable digital wired or wireless interface arranged to allow the edge computing device (ECD) to communicate with external devices. For example, the interface could be an internet interface, such as a socket; or a serial interface.
[0076] Specifically, such as Figure 2 As shown, the interface IF of each edge computing device ECD allows it to conduct digital communication on a digital communication network NW such as the Internet, and all system 100 devices are connected to this digital communication network NW for communication.
[0077] Specifically, one or more central servers (CS, NS) can be connected to the network NW to communicate with edge computing devices (ECDs). Each edge computing device (ECD) is connected to at least one (such as exactly one) such central server (CS). The central server (CS) can in turn be connected in one or more hierarchical structures, such as in a tree structure, where leaf (or client) central server (CS) is connected to parent (or server) central server (CS).
[0078] Figure 2 A specific type of central server is also shown, namely the namespace server NS (see below).
[0079] A server being "centralized" means that it defines logically centralized functionality, and in this sense, it is accessible from a single, well-defined logical location. For example, a central server can be a conventional, standalone server, taking the form of a single piece of hardware running computer software. However, each central server can also be embodied as a distributed logical server, such as a defined server function accessible via an internet "cloud"-like function. Therefore, several such central servers (CS) can actually run partially or entirely on a common hardware platform, depending on the circumstances. Such design options are chosen from, for example, the scalability, performance, security, and resilience of system 100.
[0080] The system 100 of the present invention can be made highly scalable and can include at least 1,000 (such as at least 10,000 or even at least 100,000) connected edge computing devices (ECDs) that are connected to the same system 100 in such a way that they can all communicate directly or indirectly with each other on the network NW as described herein.
[0081] Each central server (CS) can serve at least 10 edge computing devices (ECDs), such as at least 100 edge computing devices (ECDs). Each central server (CS) can also serve up to 10,000 edge computing devices (ECDs), such as up to 1,000 edge computing devices (ECDs).
[0082] As will become apparent from the description of this invention, each edge computing device (ECD) may also refer to other edge computing devices (ECDs) on the network (NW). Therefore, data can flow in a tree structure formed by system 100, which includes one or more layers of edge computing devices (ECDs) forming leaves and nodes adjacent to the leaves in the tree structure, and also includes one or more layers of central servers (CSs) located away from such leaves.
[0083] It should also be remembered that the interpreter software function ES described herein can be executed on both the edge computing device ECD and the central server CS, thus enabling both devices with their own sensors S and cascaded data streams, rather than more centrally located devices that process data measured using their own sensors, to handle queries of the type described herein.
[0084] Therefore, according to the present invention, each edge computing device (ECD) is equipped with a corresponding interpreted software function (ES), such as Figure 1 The diagram is presented in a simplified manner. The interpreted software function ES is arranged to execute on the corresponding CPU of the edge computing device ECD under consideration, such as executing locally on the edge computing device ECD, or executing completely locally on the edge computing device ECD.
[0085] As used herein, "local execution" means that the software code embodying the software function ES is directly loaded into and executed on the CPU of the edge computing device ECD, without any logic or computation occurring on the CPU of any external device. It should be noted that this does not preclude the communication of data to and / or from the edge computing device ECD in an informational sense, which can be used as the basis for executing the software function ES. For example, the numerical working data product of one edge computing device ECD can be consumed by another edge computing device ECD for further computation therein. However, "local execution" of software excludes distributed execution, such as in the sense that different computations to be performed by the software function ES are executed in different threads on different, disjoint hardware devices. The inventors anticipate that individual edge computing device ECDs can be arranged to share some software function ES functionality, even in distributed execution environments involving a subset of such edge computing device ECDs. This could include one edge computing device ECD delegating a specifically defined computation (particularly queries related to sensors of such invoked edge computing device ECDs) to other edge computing devices. Generally speaking, the present invention offers the greatest advantage in a hardware environment in which CPU-intensive computations are performed as far as possible on the leaf (at each edge computing device ECD), thereby utilizing the CPU resources of all connected devices.
[0086] A software function ES is an interpreted software function. This means that it can accept information with a predetermined format, process the information step-by-step in a specific order, and interpret or execute one piece of information at a time. Such information processed by the software function ES is referred to herein as "computer code" and is received by the edge computing device under consideration (ECD) via its digital communication interface IF and stored in its memory M. The computer code further includes definitions, statements, and / or instructions that the interpreted software function ES parses, processes, and executes. In this sense, the traditional Java Virtual Machine is an interpreted software function, although it is configured to interpret Java code rather than the computer code according to the present invention. It is important to note the distinction between "interpretation" and "execution," where the latter means loading binary machine code instructions onto the CPU for direct execution, while the former means that the interpreter interprets the computer code and thus instructs the CPU. In other words, an interpreted language is a language containing at least some aspects that are not represented in the natively directly executable machine language of the CPU used, but rather need to be decoded by an interpreter in some way for the CPU to execute.
[0087] According to the present invention, the received, stored, and interpreted computer code is formatted according to a query language having a predetermined syntax. Here, "query language" is a language that allows users to define queries on a specific dataset. For example, regular SQL is a query language, although not according to the present invention.
[0088] That is, the syntax of the query language is arranged to define queries whose results are data streams. As used herein, a data “stream” is a set of data segments with a time dimension, preferably a set of continuously generated data segments, and possibly even a set of real-time data segments. Therefore, each such data segment may be associated with or include timing information, such as the time of measurement, the time of transmission, the time of reception, etc. For example, an edge computing device ECD may include a clock that can be arranged to provide the time associated with each sensor S measurement, such as before the considered measurement is processed by the software function ES.
[0089] In some embodiments of the invention, each edge computing device (ECD) is arranged to produce the result of at least one query defined by the computer code when the computer code is interpreted using the software function (ES). As mentioned, the result may be a data stream. The edge computing device (ECD) is then further arranged to transmit the considered result to a querying device, such as a different edge computing device (ECD) or a central server (CS), via its digital communication interface (IF). Generally, the querying device can be any device external to the edge computing device (ECD) that submits the considered query to the edge computing device (ECD) via its digital communication interface (IF) and then receives a response to the query. In some cases, depending on the detailed system configuration, the response may also be returned to an entity different from the entity that submitted the query.
[0090] Specifically, the syntax can be arranged to allow general queries, and more specifically, queries defined hereafter based on calculations to be performed based on measurements from sensors S of the edge computing device ECD under consideration. Thus, the computer code of the query language format interpreted by software functions can include appropriately formatted references to the specific sensors S of the edge computing device ECD, thereby interpreting the reading of the current value of the considered sensor S and using that read current value in one or more calculations based on the result of determining the query outcome.
[0091] In some implementations, the edge computing device under consideration (ECD) (an edge computing device on which software functions ES are executed) is arranged to perform the computation as part of the interpretation.
[0092] The system 100 according to the invention can be designed to be a self-sufficient system, meaning it does not depend on other external systems for its operation. Nevertheless, it can be designed to provide robust scalability mechanisms for tight integration with software on many different hardware platforms. A key aspect of this is the interpreted software function ES executed on each edge computing device (ECD). That is, the interpreted software function ES can be downloaded to each individual edge computing device (ECD) and installed to execute on the corresponding CPU of the edge computing device (ECD) under consideration. Once downloaded and installed, the software function ES of each such edge computing device (ECD) can interpret the computer code, provided that the computer code follows the explicitly defined syntax.
[0093] In practice, software functions can be ported (converted) to be executable on many different hardware / software environments (such as different processor architectures and / or different operating systems). In this case, it is preferable to interpret the software function ES, which is particularly suitable for each type of such hardware / software environment, but provides corresponding (or even identical) interpretation capabilities regarding the syntax.
[0094] The resulting agnosticism regarding the environment only requires that the software function ES support certain predefined functionalities, simplifying the interpretation of the software function ES so that it can run on edge computing devices (ECDs) with very limited local hardware resources. On the other hand, the number of such small edge computing devices (ECDs) can be very large, with each ECD running such simplified interpreted software function ESs.
[0095] Simultaneously, system 100 achieves large-scale scaling by running numerous instances of the interpreted software function ES in parallel on different devices. Specifically, system 100 can be scaled to run the interpreted software function ES in numerous replicas on large multi-core systems, clusters, and the cloud.
[0096] These simplified and extended concepts will now be briefly explained and illustrated.
[0097] Regarding the first simplification, the inventors have successfully configured the interpreted software function ES to implement the required "kernel" software functionality, which can then be simplified to run directly on small devices with limited or no operating system support. By porting the interpreted software function ES to various such resource-constrained environments, the system 100 of the present invention can be made virtually independent of the hardware, operating system, and communication infrastructure used by the interpreted software function ES running on each corresponding edge computing device ECD implementing such predefined minimal kernel software functionality. Therefore, each such interpreted software function ES can run completely independently on its corresponding supporting device or computer, thereby forming the edge computing device ECD according to the present invention.
[0098] For certain types of hardware, the inventors have devised kernel software functionalities that do not require operating system support and can run on bare metal. In fact, it has been demonstrated that the minimum possible configuration requirements for kernel system K are less than 100K of RAM (transient memory) and only 200K of persistent non-transient memory, such as flash memory.
[0099] Kernel software functions can be included as part of the interpreter software function ES. For example, kernel software functions can provide the necessary interpretation capabilities, including syntax parsing, for the core parts of the well-defined query language, while the rest of the software function ES (including non-required functions, etc.) can add more advanced functionality.
[0100] Generally speaking, since different edge computing devices in the edge computing device ECD may include different hardware configurations, the interpreter software function ES may be particularly suitable for the corresponding hardware configuration of each type of different edge computing device ECD, while the syntax may be the same for each edge computing device in the different edge computing device ECD.
[0101] In some implementation schemes, such as Figure 1 As shown, it is designed to provide tight integration between the interpreted software function ES and other (external to system 100) software running on the same edge computing device ECD. The interpreted software function ES can run as an embedded engine within other embedded software systems ESS.
[0102] Regarding extensions, we go to Figure 2 The diagram illustrates how a very large number of such simplified edge computing devices (ECDs) can be interconnected to form a single distributed system 100 containing many different edge computing devices (ECDs) and a central server (CS).
[0103] To achieve this, each edge computing device (ECD) can be managed by a specific central server (CS) that can itself run as a cloud service on the network (NW).
[0104] As mentioned above, the interpreted software function ES can be designed with the necessary kernel functionalities, including computer code query syntax interpretation and interface IF communication support. For very limited hardware environments, only this kernel can constitute the entire interpreted software function ES. Then, depending on the hardware limitations of the edge computing device ECD on which the software function ES is to be executed and depending on the general system configuration, various non-essential additional functionalities can be added.
[0105] In some implementations, each sensor S and / or edge computing device ECD can be referenced using a global namespace or attribute set according to the syntax described above. For example, each edge computing device ECD can be assigned a unique number or name, or a unique network NW address can be used as a unique identifier for each edge computing device ECD. Each sensor S of the edge computing device ECD can similarly use the same or different identification criteria for addressing. A simple example of such a naming standard is "[ECD_NO].[SENSOR_NO]". Preferably, according to the naming standard, sensors of the same type can be represented using the same subname.
[0106] exist Figure 2 In the example shown, each edge computing device (ECD) of system 100 registers its globally unique identity with a corresponding central server (CS), which: runs on some servers connected to the network NW; resides in a container or PC; or even as a separate process on the same computer on which the edge computing device (ECD) executes the interpreted software function ES. In the latter case, the central server (CS) and the edge computing device (ECD) actually run on the same hardware, but are logically and functionally separate. Each central server (CS) can serve several edge computing device (ECD) to implement the tree structure of data flow in system 100. However, in general, both the edge computing device (ECD) and the central server (CS) discussed herein can be implemented as physically separate, independent hardware entities.
[0107] exist Figure 2 In this system, there are two central servers (CS) running in the network NW, and four different edge computing devices (ECDs) are registered.
[0108] The system may also include a central namespace server (NS) where a database (DB) is deployed. The database (DB) contains information defining the name of each edge computing device (ECD) within the edge computing devices. The database (DB) may be located in or connected to the namespace server (NS).
[0109] The database DB may include information about the network address location of each edge computing device in the edge computing device ECD. The database DB may also include metadata information about each edge computing device ECD, such as what type of sensor S is available at each such edge computing device ECD, what type and / or version of the interpreter software function ES is executed on each edge computing device ECD, and the numerical attributes (such as the units used) used when each edge computing device ECD delivers measurements. This metadata can then be used to determine the parameter values used in the query preprocessing performed by some edge computing device ECDs.
[0110] Generally speaking, the namespace server (NS) itself can be a central server (CS) of the type described above, which has additional namespace information processing functions and is configured to serve the 100 participants of the system using namespace-related information services and requests.
[0111] All edge computing devices (ECDs) are interconnected via the central server (CS) and at least one such namespace server (NS, if used) through a network (NW).
[0112] As mentioned, different instances of the interpreted software function ES running on different computers can communicate with a central server CS running on the same or some other computers. Each edge computing device ECD can communicate with the central server CS to which it is registered. The central server CS of the services under consideration can retain some metadata about each edge computing device ECD it serves and can forward data and streams to other central server CSs, such as for distribution to edge computing device ECDs served by other such central server CSs.
[0113] Generally speaking, all connected devices, specifically edge computing devices (ECDs), can operate independently of each other. Specifically, an edge computing ECD does not need to be continuously connected to its central server (CS), as long as the ECD registers with the CS. Actual digital communication of object streams between the ECD and the CS can only begin and be completed when needed. When there is no ongoing communication activity, the edge client can operate autonomously.
[0114] This functionality can be implemented as part of the digital communication interface (IF), minimizing network NW traffic and thus allowing for large-scale scalability. Of course, heartbeats and similar keep-alive signals can be transmitted between devices to keep registered connections active. However, even this type of periodic communication may not be necessary, as offline edge computing devices (ECDs) will simply not respond to queries made to them.
[0115] Therefore, system 100 includes a set of interconnected peer devices, which in turn include multiple devices selected from the following list:
[0116] • On some network-connected devices, the interpreted software function ES runs as embedded software;
[0117] • Dedicated edge computing devices (ECDs) that run interpreted software functions (ES);
[0118] • Central server CS; and
[0119] • Namespace server NS.
[0120] In practice, system 100 may include a large number of such interconnected peer devices, particularly a very large number of edge computing devices (ECDs). Therefore, it is desirable to be able to scale the number of peers to handle an extremely large number of edge computing devices (ECDs), ranging from tens of thousands to billions. This can be achieved by expanding the number of edge computing devices (ECDs) registered in each central server (CS) to thousands of ECDs registered in each CS, and by defining a hierarchical structure with several hierarchical levels as described above.
[0121] Even if part of system 100 stops, system 100 of such interconnected peers should continue. For this purpose, the database DB of the domain name server NS can be a wrapped database using its own regular JDBC (Java Database Connectivity) wrapper, such as... Figure 3 An example of encapsulating a relational database.
[0122] In this specific example, the wrapper could be a plug-in that interprets software functions ES, enabling access to any relational database using a standard database interface such as JDBC. The database DB could maintain identifiers for all peers in system 100, as well as other metadata such as which equipment is accessible through each edge computing device ECD, which sensors S they access, etc.
[0123] It should be noted that a namespace server NS of the present invention type without a database backend can also be implemented, and in this case, such a namespace server can still maintain its database. However, to scale to millions of interconnected peers in system 100, a wrapped database DB as described is preferably used.
[0124] It should also be understood that the namespace server NS and / or its database DB can be implemented as defined server functions that execute in a distributed hardware environment.
[0125] As mentioned above, the Central Server (CS) and Namespace Server (NS) can be configured to run on many different kinds of hardware configurations and in different software environments. In a simple case, they can each run on a regular PC. In a more scalable configuration, each Central Server (CS) can run in a separate container (such as Docker, see www.docker.com), and the Namespace Server (NS) along with the relational database (DB) can run on a dedicated cluster. Leveraging this scalability, if the number of edge computing devices (ECDs) registered to each Central Server (CS) is 1000 and there are two levels of Central Servers (CS), up to approximately 10... 9There are 10 peers. The database (DB) will not be a bottleneck in this configuration because a limited amount of metadata for each peer can be stored there.
[0126] Generally, interpreter software functions (SAs) provide general data analysis and inference capabilities to the edge computing devices (ECDs) on which they operate. This functionality may typically be implemented in the same manner on different port versions of the interpreter software function (SA), making the interpreter software function (SA) independent of the specific hardware / software environment in which it executes. Generally, the functionality of the interpreter software function (SA) described herein may be implemented in a way that makes it independent of the hardware, operating system, and communication infrastructure of the device on which it operates. In an exemplary implementation, the kernel of the interpreter software function (ES) may be written in the C99 computer language.
[0127] Figure 4a This illustrates the main components of ES, the interpreter of such software functions.
[0128] The core of the interpreter software function Elasticsearch is the aforementioned kernel. It provides the general capabilities required for real-time data stream analysis. This includes a compute engine, data stream processor, and inference engine. It may also include a main-memory local database, EDB (see [link to EDB]). Figure 1 This can be an object-oriented database and can also be included in the main storage M of the edge computing device ECD under consideration. Using such a natively provided database, specifically an object-oriented database, in the edge computing device ECD enables the use of powerful query processors and optimizers, such as those that can store and manage analytical models and edge computing device ECD metadata.
[0129] Furthermore, the object-oriented data model describing the metadata attributes of the edge computing device ECD under consideration (such as information about the type or attributes of the edge computing device ECD sensor S, the unit of measurement used by the edge computing device ECD, the hardware specifications or attributes of the edge computing device ECD, etc.) enables requests from the edge computing device ECD for information about edge computing device ECD attributes, names, and measurement conversions (the so-called "mediator"). Such requests can be made using the query language described above, and the response to such a query depends on the specific metadata information stored in the queried edge computing device ECD or even in other edge computing device ECDs subqueried in the general type of query cascade described herein.
[0130] The kernel includes the data stream interpretation functionality. In an exemplary embodiment, the kernel is tightly integrated with an interpreter for executing plans on a digital data stream known as SLOG (Streaming Logic), and also tightly integrated with an interpreter for CommonLisp, known as aLisp (built in the Lisp family of conventional computer programming languages). Indeed, the inventors have found that constructing at least the kernel portion of an interpreted software function ES in a recursive manner and using a computer programming language such as Lisp provides highly efficient processing of continuous query types handled by the system 100 of the present invention as described herein.
[0131] Figure 4a The upward arrow indicates the data flow. The corresponding interpreter software functions ES of one or more sensors S and / or one or more other edge computing devices ECD generate the incoming source data streams, which are then injected... Figure 4a The kernel shown transforms these incoming digital data streams into one or more new digital object streams for output from the edge computing device ECD under consideration.
[0132] Such source data streams can be implemented as kernel access to the raw sensor S-interface on an edge computing device ECD, with the interpreter software function ES running on that edge computing device ECD. Source streams can also be data streams generated by a corresponding interpreter software function ES running on another connected edge computing device ECD, received via a network NW such as using a TCP communication protocol. For example, such source streams can be communicated in response to successive queries from the interpreter software function ES considering the reception and consumption of the source stream.
[0133] Similarly, the resulting object data stream can be transmitted to other central servers (CS) and edge computing devices (ECDs) using communication infrastructure (such as TCP) provided by the network (NW). This allows for the configuration of very large distributed systems 100 of the type of peers of this invention, where such peers generate object data streams for other peers to consume as source data streams. On the edge computing device (ECD), object stream data elements of the computed or received object data stream can be directly transmitted to actuator A installed on the ECD under consideration, thereby activating the actuation of actuator A.
[0134] Generally speaking, the system 100 and method described in this paper are specifically designed for simple interactive development and deployment of applications that process and analyze real-time data streams in distributed and mobile environments, enabling online, real-time interactive analysis, processing and aggregation of streams generated by sensors S on edge computing devices ECDs.
[0135] Edge computing devices can be, for example, Android units, PC desktop computers, Raspberry Pi, or MCUs (microcontroller units) such as Mango OH Red or ARM M4. As mentioned above, the interpreted software function ES, and specifically the kernel, can be designed to have a very small footprint (the inventors have managed to reduce the total memory footprint of the fully functional interpreted software function in the test environment to a range from about 20kB to about 5MB, depending on the configuration), and is essentially hardware- and operating system-independent, as well as completely independent of any third-party software.
[0136] In each edge computing device (ECD), the combination of a corresponding main memory M-database (EDB), a software-implemented computing engine, a software-implemented data stream processor, and a software-implemented inference engine allows for the direct use of edge analytics on the ECD. This contrasts with many conventional approaches, where all measurement data is first uploaded from the edge device to a central server, where all data processing is centrally executed. This approach achieves a dramatic reduction in data volume by processing the data stream already present in the ECD. Thus, aggregation analyses, such as swarm analytics, are only required on the connected set of ECDs at a centralized or semi-centralized level (e.g., at different aggregation levels within the tree-structured data stream).
[0137] Thus, the interpreter function ES has filtering capabilities; in other words, it is configured to filter (discard) data from the available source data stream to produce an output object data stream containing less data per unit of time than the source data stream.
[0138] In some configurations, the interpreter software function (ES) can also function as a data stream server, running on edge computing devices (ECDs) or a central server (CS). In this data stream server role, the interpreter software function (ES) is deployed to collect data from one or more connected edge computing devices (ECDs) and combine / process the data as object data streams, forwarding them to other peers. For example, whenever an analysis model in some edge computing device (ECD) detects a strong vibration, an object data stream containing the vibration spectrum and the geolocation of the considered edge computing device (ECD) can be sent to a stream server running on the central server (CS), which is configured to receive similar information from multiple differently connected edge computing device (ECD) devices. If the stream server simultaneously receives many such streams from edge computing device (ECD) devices geographically close to each other, it may indicate an earthquake. The stream server can also forward the received and potentially processed data to other connected system peers (100) or external systems, such as for permanent central storage or batch analysis.
[0139] As described above, the interpreter software function ES is configured to interpret computer code formatted according to a well-defined query language syntax. Such a query language syntax can be selected to provide interactive and high-level, user-oriented analysis of the available source data stream.
[0140] Specifically, the query language can be a high-level, object-oriented query language. This syntax can allow for different, non-adversely related execution paths of conditional logic (such as "select-from-where" clauses), which are then interpreted and executed by the interpreter software function ES. Queries can be at least partially declarative, as their interpretation and execution result only in a search of the databases within each corresponding edge computing device (ECD) and the execution of any computations on those databases, without updating those databases within the ECD or changing the state of the device. However, it is preferred that the queries be at least partially non-declarative (see below).
[0141] In some implementations, the query language can be designed to allow queries to have adverse effects, such as updating a database, signaling actuators that change device state, or sending messages to other edge computing device ECDs when such queries are loaded into the memory M of an edge computing device ECD on which an interpreting software function ES runs and executed by the interpreting software function ES that interprets the considered query. In other words, when the interpreting software function ES executes on the considered edge computing device ECD and then interprets the loaded computer code, as a result of the interpretation, programming embodied in the computer code is executed, which may include responses to states and / or adverse effects.
[0142] Another term for the "declarative" nature of the language is that it is "non-procedural." Therefore, this query language is preferably not non-procedural, but at least partially procedural. Providing the query language as a declarative / non-procedural query language allows users to specify, in an intuitive way, the expected results with respect to the output data, rather than precisely specifying what the edge computing device (ECD) should do in terms of computation to achieve that expected result. However, simultaneously defining the query language to have certain non-declarative / procedural elements allows the query to have adverse effects and include stateful functionality. When processing streaming data, such procedural / non-declarative elements of the query language provide the possibility of extracting information from state-changing sensors.
[0143] The following is an example of a declaratively defined object-oriented query that returns an object stream from a given stream of sensor readings from an accelerometer sensor S:
[0144] select stream of absolute(shakes)
[0145] from vector of number shakes,
[0146] where shakes from accelerometer()
[0147] The query receives a set of acceleration readings because the object is a 3D vector of shakes from the sensor accessed via the function `accelerometer`. It returns a stream of single numbers as the absolute values of the shakes object. The function `accelerometer()` here is stateful because it returns a new value each time it is called. Interpreting software functions (ES) is generally aware that the query contains calls to stateful functions and is arranged to take this into account when parsing and interpreting queries. This provides more efficient handling of this type of query. Whenever the accelerometer emits a new vector, the query calculates and emits its absolute value. Functions can also be expressed via queries. For example, the `absolute` function can be defined as...
[0148] create function absolute(Vector of Number v) -> Number
[0149] as select sqrt(sum(v .^ 2))
[0150] The query `select sqrt(sum(v .^ 2))` takes a numeric vector object `v` as a parameter and returns its absolute value.
[0151] In some implementations, computer code is formatted according to an object-oriented query language, allowing users to specify computations, filtering, and / or transformations of the data stream at each connected edge computing device (ECD). In other words, the current query language can be an object-oriented query language.
[0152] Object-oriented query languages are languages that support queries where variables are bound to any kind of object (e.g., numbers, vectors, matrices, strings, records, key-value stores, etc.). Objects can even represent entire edge computing devices (ECDs), enabling queries on potentially large sets of the ECD and its internal database (EDB). In the code example above, the variables `shakes` and `v` are bound to a stream and a vector, respectively. This contrasts with relational query languages such as SQL, where variables in a query (the SELECT statement) must be bound to only rows in a table. Such object-oriented query languages allow for the filtering and transformation of any kind of data object.
[0153] This query language can also contain procedural (stateful) functions, where the state of the database EDB or the edge computing device ECD itself is updated by calling the function. The query processor interpreting the software functions in Elasticsearch must then consider that the order in which stateful functions are called within the query is important, as it will change the results. For example, a database EDB access after a state change will produce different results than before.
[0154] It should be noted that, for example, procedural statements exist in SQL, allowing manipulation of state and variables in the aforementioned sense. However, this is not permitted in SQL queries (i.e., select-from-where statements). Therefore, a key difference between this query language and, for example, SQL, is that this query language allows the use of variables and / or stateful functions defined using the query language and its syntax within queries.
[0155] The results of object-oriented queries can also be object streams. For example, such computations and filtering of real-time streaming data can be defined as mathematical formulas and expressions using such an object-oriented query language, referred to in this paper as a "stream model." Such a stream model is a collection of definitions of mathematical functions, filters, and other expressions defined on a data stream, such as measurement results. Using an object-oriented query language, models can be specified at a very high level without requiring deep programming knowledge from the user. Instead of writing detailed programs expressing how the model should be executed, it is possible to simply specify in the stream model what to do with respect to the final result and the extent to which that final result depends on any intermediate computation results. Users do not need to worry about the details of how to efficiently implement algorithms that integrate a large number of edge computing devices (ECDs) and central servers (CSs) to achieve common goals in data collection and processing.
[0156] In some implementations, the object stream in a function that returns a stream is stateful, for example, through sources originating from an edge computing device (ECD) or environmental sensors (S), such as the accelerometer mentioned above. Therefore, the query processor interpreting the software function ES can be arranged to account for the adverse effects of such stateful stream functions when optimizing and executing queries. Furthermore, the order of objects in the object data stream produced by such queries can also be important, and thus the functions used or employed by such queries become stateful.
[0157] In some implementations, computer code may be provided to the edge computing device ECD as plaintext (as opposed to compiled / object / machine code) via the interface IF. However, in some implementations, the computer code may be pre-compiled and provided to the edge computing device ECD in non-plaintext form, compiled into the binary instruction format of the edge computing device ECD (which can then be performed on the central server CS or elsewhere where more CPU / memory resources are available than on the edge computing device ECD in question). In the latter case, an interpreted software function ES may be provided with a simpler version of the pre-compiled computer code configured to read and execute it. This results in a simpler interpreted software function ES. However, in some implementations, at least one (such as at least several), substantially all, or possibly even all, edge computing device ECDs are still characterized by corresponding interpreted software function ESs arranged to interpret the plaintext computer code of the stated syntax. Whether each particular edge computing device ECD accepts plaintext computer code can be specified in the metadata of the edge computing device ECD in question or the edge computing device ECD type, and the central server CS can be arranged to check this metadata setting and selectively compile or not compile the computer code distributed to each edge computing device ECD based on such metadata settings.
[0158] As mentioned above, an object data stream can be a stream of data objects, such as a stream of data objects in which each data object represents the value of a corresponding tensor, which in turn can represent, for example, the current physical state of a particular local environment sensed using one or more sensors S.
[0159] However, data objects can be of any type, ranging from simple alphanumeric information such as measurement values (INT, LONG, CHAR, etc.) to more complex data structures (ARRAY, LIST, SET, BAG, RECORD, etc.) according to a predefined syntax. Preferably, such data structures are defined as part of the syntax of the query language. In, for example, an embedded software system ESS, a data object can also be a reference to a complex data object or a pointer, such as a reference to a data object or a callback function.
[0160] In some implementations, each of the objects is handled by a callback function, such as a callback function of an embedded software system ESS and / or as part of the interface IF, and executed in a defined central server CS, such as in a client application within the central server CS or on an edge computing device ECD. The objects under consideration may include pointers, addresses, or other identifiers of the callback functions.
[0161] In some implementations, the object data stream is an endless stream of objects, which are computed continuously or intermittently over time by the interpreter software function ES of the edge computing device ECD and delivered to the query receiver via the interface IF. The object stream can be communicated via the digital communication interface IF, for example, using callback functions, when it becomes available after the computation, or in batches, as appropriate. As an endless stream of objects, the computation and delivery (e.g., as calls to callback functions) can continue until something stops it, such as a request to stop the delivery of query results or the edge computing device ECD in question going offline or being interrupted. Therefore, as used herein, the term "endless stream" is intended to indicate that the data object stream has no defined endpoint at the time of querying, but is arranged to produce results, for example, via callback functions, which continue to be generated (and, where applicable, one or more corresponding callback functions are called), until some condition arises that causes generation to stop. In other words, this condition may be at least partially unpredictable when the endless stream is initiated, requiring the activation of some kind of completion mechanism to stop the endless stream.
[0162] In some implementations, the syntax allows for different conditional logic-related execution paths that are interpreted and executed by the interpreter software function ES. For example, a query defined according to the computer code may include IF statements, WHILE loops, and similar items.
[0163] Therefore, a query defined using the query language can be defined as continuously resulting in the computation and / or filtering of measurements in the source data stream available at the edge computing device ECD that receives the considered query, and delivering an endless data stream as a result. In this document, such a query is referred to as a “continuous query.” In some embodiments, the inventive system 100 using a query language that allows such continuous queries allows users to interactively specify continuous queries to continuously analyze measurements flowing through the edge computing device ECD and the central server CS in real time. As mentioned, the result of a continuous query is a real-time (endless) object stream of processed / filtered measurements. The results can then be distributed to the consumer via special system callback functions, such as for display to the user.
[0164] In contrast to discontinuous queries, continuous queries deliver a data stream as a response that depends not only on the state of the queried database at the time the query is made, but can also change over time as the internal state of the queried database changes. This is, for example, true for continuous queries made to an edge computing device (ECD) with an internal database (EDB), the internal state of which may affect the value of the data stream generated in response to the continuous query or as the latest readings from sensors are continuously accessed.
[0165] As an example illustrating this aspect, a continuous query can be defined to return a location vector stream of a specific edge computing device's EDC, measured every second at all times when the device's EDC is sufficiently close to a given geolocation. In this example, the calculation involving the comparison between the measured geolocation and the predetermined geolocation is performed locally on the edge computing device's EDC, and then the edge computing device sends the processed data as a continuous data stream to the query peer.
[0166] Both the flow model and user data can be stored in each individual edge computing device (ECD), in its object-oriented in-memory database (EDB), and similarly on each central server (CS). (Generally, any central server CS can run an interpreter function (ES) and may also include such an object-oriented database EDB used by the interpreter function ES running on the central server CS.) Since data flow processing at each edge computing device (ECD) typically involves matching rapidly flowing stream objects in real-time with data in the local database EDB, the fact that the database EDB is an object-oriented database allows the edge computing device (ECD) to be designed to be computationally fast relative to its CPU capabilities due to efficient data representation and processing. Alternatively, the object-oriented aspect of the database EDB makes it particularly well-suited for efficiently processing the objects that constitute the data flow, the primary task of which is to process the data flow. For example, to determine if the spectrum of a measured vibration might later damage the sensing edge computing device (ECD) due to material fatigue, the frequency measured by the vibration sensor S on the edge computing device (ECD) under consideration can be matched with the local object-oriented database EDB of the resonant frequency of the edge computing device's own EDB at the edge computing device (EDC).
[0167] A key aspect of interpreting software functions (ES) is that they can be designed to allow the combination of object streams from several different edge computing devices (ECDs). In other words, interpreting ES can support the interpretation of "fusion queries," which can be defined using the query language and are defined to reference several different available source streams. The interpreting ES is then configured to computationally combine these available object streams to produce a specific output object stream when interpreting such fusion queries. An example of such fusion queries is a sequential query, designed such that when interpreted on a specific edge computing device (ECD), it causes that ECD to observe when several other edge computing devices (EDCs) in a specific geographic area simultaneously detect strong vibrations. Then, when the fusion query produces a predetermined result, an alert is issued to the user, possibly providing a real-time visualization of the maximum or average amplitude of the observed vibrations. The user can then interactively send new queries to the affected edge computing device (ECD) in real-time to find out the details of the cause.
[0168] The query language, and specifically its syntax, allows queries to be referenced by the first (requesting) edge computing device 110 (see [link]). Figure 2 The information received from the second (responding) edge computing device 120. This type of reference utilizes the aforementioned global namespace and can specifically use the services of the namespace server NS to locate the responding edge computing device 120 on the network NW. This connection can then be mediated by one or more intermediate central servers CS between the requesting edge computing device ECD and the responding edge computing device ECD. To achieve this, it is preferable that the query language is an object-oriented query language as described above, according to which variables can be bound to different kinds of edge computing device ECD objects, and subqueries can be expressed for edge computing device ECD objects.
[0169] Therefore, the first query received by the first edge computing device 110 may include a reference to the second edge computing device 120. When interpreting the first query, as a result of the reference to the second edge computing device 120, the interpreter software function ES running on the first edge computing device 110 will subsequently submit a second query to the second edge computing device 120, requesting specific information specified by the first query. Then, when interpreting the second query, the interpreter software function ES running on the second edge computing device 120 will return an object stream to the first edge computing device 110, which will be used by the interpreter software function ES running on the first edge computing device 110 to compute the result of the first query, which takes the form of an object stream returned to the entity that submitted the first query to the first edge computing device 110. In other words, the interpreter software function ES executing on the first edge computing device 110 can be arranged to cause the first edge computing device 110 to query the information from the second edge computing device 120, as a result of a query referenced by the second edge computing device 120 as defined in computer code.
[0170] As an alternative to or supplement to the second edge computing device 120, references in queries made to the first edge computing device 110 may point to a specific sensor S included in the second edge computing device 120, such as sensor S being identified and accessible via the global namespace.
[0171] A query made to the first edge computing device 110 can be a sequential query, and a query made to the second edge computing device 120 as a result of a previous query may be the same. However, the two queries can be sequential or discontinuous, and independent of each other, depending on the actual information sought and how the query is defined.
[0172] In addition, system 100 may also include a third edge computing device 130, which may be of the same type as the second edge computing device 120 in terms of the type of sensor S available in the device, but is not necessarily required to be.
[0173] The interpreter software function ES running on the first edge computing device 110 can then be configured to make corresponding queries to both the second edge computing device 120 and the third edge computing device 130 as a result of namespace references in the query made to the first edge computing device 110. The interpreter software function ES running on the third edge computing device 130 can then be configured to generate a resulting object stream in response to a query received from the first edge computing device 110, and to transmit the object stream to the first edge computing device 110 via the digital interface IF of the first edge computing device 110.
[0174] Then, the interpreted software function ES running on the first edge computing device 110 can be arranged to perform its computation as defined in the query made to the first edge computing device 110, using both the object stream received from the second edge computing device 120 and the object stream received from the third edge computing device 130.
[0175] Generally speaking, the corresponding result of the query performed by the first edge computing device 110 may include a corresponding endless object stream received by the first edge computing device 110 from the second edge computing device 120 or the third edge computing device 130, which is continuously or intermittently calculated by the second edge computing device 120 or the third edge computing device 130 over time and transmitted to the first edge computing device 110 via the digital communication interface IF of the first edge computing device 110.
[0176] The interpreter software function ES, executed on the first edge computing device 110, is then arranged to cause the first edge computing device 110 to propose the considered query as the result of the query received by the first edge computing device 110.
[0177] It should be noted that the first query may include a second and a third query as part of its computer code definition, or enable the interpreter software function ES executing on the first edge computing device 110 to formulate the second and third queries for communication to the computer code of the second edge computing device 120 and the third edge computing device 130. The corresponding interpreter software function ES executing on the second edge computing device 120 and the third edge computing device 130 will then interpret the queries defined by the second and third computer code, respectively, and as a result, deliver the corresponding data streams to the first edge computing device 110. Then, depending on the definition of the first query, the second and / or third queries may subsequently be defined in a corresponding manner, referencing the fourth and subsequent edge computing devices ECS and / or sensors. Since the query language can support complex logic and / or contain stateful functions as described above, such subsequent queries can be dynamically defined on each interpreter edge computing device ECD, for example, based on parameters describing the local network NW adjacent to the considered edge computing device ECD or other updated parameter data. In this way, a single query made to the first edge computing device 110 can trigger an automatically evolving query cascade, potentially involving a large number of other edge computing device ECDs that collect, process, and transmit data streams that ultimately reach the first edge computing device 110 for processing. This also provides users with a very powerful way to automatically deploy distributed logic to complex systems of edge computing device ECDs by essentially defining the desired results in the first query using the necessary specificity. For example, the first query can define any other edge computing device ECDs to be involved, such as a specific geographic area of interest and / or a specific type of sensor S to be used, based on parameter data defined in the first query. The interpretation of the query can then perform the actual selection of auxiliary edge computing device ECDs based on such parameter values and current conditions.
[0178] These mechanisms also enable the local computing power of each edge computing device (ECD) to be maximized in true distributed computing, while still providing a robust, flexible, and user-friendly system 100 that can be customized and updated instantly and in real time. For example, if a user wants to see the impact of an updated query, the updated query can be simply pushed to the first edge computing device 110, which will immediately begin interpreting the updated query, including updated queries for the second edge computing device 120 and the third edge computing device 130, and so on, as needed, ultimately producing an updated object stream returned to the querying user. This updated object stream will then typically be more or less immediately available, or at least fast enough, to allow users to perform such deployments as part of iterative development, where the updated object stream constitutes feedback to the design process, including the first query definition revised through several iterations.
[0179] With a similar objective, each edge computing device (ECD) can be arranged to filter (discard) at least some, preferably substantially all or even all, of such measurement data after it has been conveyed via a digital communication interface IF, such as based on an interpreted query, to the sensors S of the edge computing device (ECD) under consideration, and / or a message calculated based on the measurement data, rather than storing such measurement data in its memory M. In other words, each edge computing device (ECD) can perform measurements, perform query-defined calculations based on the measurement data, and transmit the measurement data and / or the results of said calculations to the requesting party, and then clear the measurement data from its memory M. In this way, efficient data flow can be achieved throughout the system 100 without clogging the individual memories M of the individual edge computing device (ECDs).
[0180] As mentioned, the type of query described herein can refer to a specific edge computing device ECD and / or a specific sensor S of such an edge computing device ECD. However, the interpreter software function ES of a specific edge computing device ECD can also refer to a specific object stream generated within another edge computing device ECD, such as in the form of results from internal computation, or more generally, to a stream of preprocessed or raw measurement data from a specific sensor S included in another edge computing device ECD.
[0181] Specifically, the interpreter software function ES of the first edge computing device 110 can be arranged to accept subscription requests from the Alpha edge computing device 140 for a stream of objects generated by a single query processed by the interpreter software function ES of the first edge computing device 110, in a context where the Alpha edge computing device 140 has not made a considered query to the first edge computing device 110. For example, the first edge computing device 110 can calculate an endless stream of vibration measurements from analyses of various other edge computing device ECDs, as the result of a specific query defined within the first edge computing device 110 or made to the first edge computing device 110 from some other peer entity. The Alpha edge computing device 140 can then publish a subscription to the resulting endless stream by requesting such a subscription via the interface IF of the first edge computing device 110. Such a subscription can be continuous or time-limited, and can, of course, be canceled by the Alpha edge computing device 140 at any time.
[0182] As described above, each edge computing device (ECD) has a relationship with a specific central server (CS), and the two communicate with each other. Typically, it is preferred that this relationship be a client-server type relationship, where the edge computing device (ECD) acts as the client and the central server (CS) acts as the server. This is advantageous from a security perspective and also allows for handling edge computing devices with limited capabilities or resources.
[0183] Therefore, each edge computing device (ECD) in the aforementioned edge computing device can have a client role associated with a specific corresponding central server (CS) in the aforementioned central server (CS). In this context, having a client role means that the central server (CS) cannot connect to such an edge computing device (ECD) before a communication connection has been established between the edge computing device (ECD) and the central server (CS). This means that the central server (CS) is not allowed to contact the edge computing device (ECD). In other words, the central server (CS) does not have the capability to establish a communication connection with the edge computing device (ECD) at the initiative of the central server (CS), or is not granted the necessary access credentials, at least not enough to establish a communication connection capable of sending or receiving computer code of the type described herein. On the other hand, the edge computing device (ECD) can establish a communication connection with the central server (CS) using credentials such as login credentials, PKI key credentials, or the like. Therefore, communication between the edge computing device (ECD) and its central server (CS) is always initiated by the edge computing device (ECD) before a digital communication connection is established between them, and never by the other way around.
[0184] After such digital communication is established between the edge computing device ECD and its central service server CS to realize the client / server relationship, both the edge computing device ECD and the central server CS can communicate with their counterparts in the client / server relationship using digital communication protocols that may be conventional in themselves.
[0185] As described above, in some implementations, all communication between individual edge computing devices (ECDs), such as communication with other edge computing devices (ECDs) and the interactive GUI described below, occurs via at least one central server (CS), thereby disallowing direct communication between the individual edge computing devices (ECDs). In some cases, at least one central server (CS) serving an edge computing device (ECD) needs to communicate with one or more intermediate central servers (CSs, such as higher-level central servers (CSs) in the tree structure described above) to reach a central server (CS) serving another edge computing device (ECD).
[0186] Using this type of network topology, a secure system is implemented, which also eliminates the need to provide server functionality to the edge computing device (ECD), thus saving valuable storage space.
[0187] As described above, in some embodiments, an interpreter software function ES running on the first edge computing device 110 is arranged to query the second edge computing device 120, whereby the interpreter software function ES running on the second edge computing device 120 is arranged to generate a data object stream in response and transmit the stream to the first edge computing device 110 via the digital communication interface IF of the first edge computing device 110. The interpreter software function ES of the first edge computing device 110 is then arranged to perform computations using the received data stream to compute a specific result.
[0188] In this scenario, the interpreter software function ES of the first edge computing device 110 can be arranged to perform preprocessing on the data object stream received from the second edge computing device 120. This preprocessing then yields a preprocessed data object stream, which forms the stream ultimately used in the computation performed by the first edge computing device 110. Specifically, the preprocessing operation can be arranged to transform the received data object stream such that the data contained therein conforms to a predefined global data ontology.
[0189] Generally, such preprocessing may include at least one of the following: mapping a name standard (which is local to the edge computing device ECD) to a global name standard (such as the System 100 global name standard); unit of measurement conversion; defined data format conversion; and applying the results of measurement data calibration to the measured values.
[0190] A simple example is a case where the second edge computing device 120 internally uses a unit of measurement different from the correct unit of measurement according to the global data ontology. However, preprocessing could also be more refined, such as statistically processing the measurement data received in the object stream from the second edge computing device 120 to remove statistical outliers, and so on. In other examples, the binary representation of the received data can be transformed to fit the global data ontology. For example, a signed four-byte integer value can be transformed into an unsigned four-byte integer value.
[0191] As used in this article, the term "data ontology" refers to a system of definitions and / or rules for measurement data, specifying how measurement data is represented in terms of units of measurement, statistical and calibration considerations, binary representation, etc.
[0192] For example, fusion queries (as described above) require that the integrated data streams be comparable, even if the object streams involved may represent the same or similar data in different ways. For instance, the second edge computing device 120 may represent temperature in Fahrenheit, while the third edge computing device 130 uses Celsius. To enable the combination of such heterogeneous data streams from different edge computing device ECDs, an interpreter software function ES, specifically an interpreter software function ES executed on the first edge computing device 110, can be arranged to allow the mediation model to be defined as queries and functions, harmonizing them by transforming the arriving heterogeneous object streams into a common model (global data ontology). Such mediation models can be locally defined in any edge computing device ECD that forms the stream server that integrates data streams from different other edge computing device ECDs. In addition to the examples provided above, such mediation can also include mapping the local names of sensors S to well-known nomenclature and calibration of local measurements.
[0193] Therefore, in the above scenario, where the first edge computing device 110 also queries the third edge computing device 130, the interpreter software function ES of the first edge computing device 110 can be arranged to now perform another preprocessing on the data object stream received from the third edge computing device 130. Similar to the preprocessing of data received from the second edge computing device 120, this additional preprocessing can produce a preprocessed stream of data used in the computation performed by the first edge computing device 110, rather than the data actually received from the third edge computing device 130. Corresponding to the preprocessing previously described, this preprocessing can also be arranged to transform the data stream received from the third edge computing device 130 such that the data conforms to the global data ontology.
[0194] Each of these preprocessing activities can perform the preprocessing under consideration using defined parameter values. These parameter values can differ for different preprocessing operations and specifically between data received from different edge computing device ECDs. They can be of the general types described above, including the units of measurement used, etc. Using such parameters, which can be defined globally or locally for individual edge computing device ECDs or for such defined types of edge computing device ECDs, and which can be provided by one or more central servers CS and / or stored in individual edge computing device ECDs, a common data ontology can be automatically assigned throughout System 100, even if System 100 contains many different types of different edge computing device ECDs, and users do not need to worry about these aspects when defining their queries.
[0195] In some implementations, the preprocessing is performed based on metadata about the second edge computing device 120; or metadata about an edge computing device ECD of a specific defined type to which the second edge computing device 120 belongs, from which the preprocessed data stream under consideration is received. This metadata can then be defined via the digital communication interface IF of the first edge computing device 110. In other words, information required to perform the preprocessing under consideration (e.g., the preprocessed parameters) is communicated through the digital communication interface IF of the first edge computing device 110. For example, the first edge computing device 110 may query its central server CS based on the global namespace identification of the second edge computing device 120, and then use the received parameters in the preprocessing of the received data stream.
[0196] In some implementations, the digital communication interface IF of the first edge computing device 110 may include at least one encapsulation mechanism arranged to transform the received data stream from an external data format into a data format internal to the query language. In other words, the second edge computing device 120 and / or the third edge computing device 130 deliver the data stream to the first computing device 110 using a data format (such as a defined data structure or binary representation) that is not based on the global data ontology and / or not internal to the query language. The encapsulation mechanism of the first edge computing device 110 can then transform the received data and encapsulate it into a data format directly acceptable to the interpreted software function ES running on the first edge computing device 110.
[0197] The data format being “internal” to the query language means that it is based on data provided as part of the definition of the query language and is used directly by the interpreter software function ES without further conversion.
[0198] It should be understood that corresponding mechanisms can be applied when the first edge computing device 110 receives data from a source external to system 100, or when the first edge computing device 110 receives data from a source within system 100 that does not constitute the edge computing device ECD itself. It is understood that the query language may support query definitions based on data collected from such "external" sources. Then, corresponding wrappers can be defined for such data sources, which are arranged to transform the received data into a corresponding query language internal data representation. This principle may be specifically applied to such received data streams.
[0199] Therefore, the wrapper functionality described above can take the form of an API that enables mapping as incoming data stream objects arrive, allowing them to be injected into the interpreter software function ES kernel so that the accessed data stream can be used in sequential queries defined using the query language. The wrapper itself can be defined as a query language function that returns a stream of objects from the encapsulated data source. System 100 may include a library of predefined wrappers to interoperate with public data infrastructures such as relational databases via JDBC and data processing systems via Kafka, Azure IoT Hub, or MQTT. Using an infrastructure with wrappers, such new wrappers can be easily developed and deployed on the fly as new requirements emerge.
[0200] To enable collaboration between the interpreted software function ES and peripheral computer code (such as computer code not formed by the query language but executed on the same edge computing device ECD as the interpreted software function ES), the interpreted software function ES may include an external application programming interface (API) configured to allow expressions of this query language to call such external computer code and / or configured to allow external computer code to call expressions employing the query language. In this context, "external computer code" is intended to refer to computer code that is not part of the interpreted software function ES and is not computer code based on the query language, such as other software running on the same edge computing device hardware or other hardware digitally communicating with the edge computing device ECD under consideration.
[0201] For example, system 100 may include a library of predefined query language functions for performing various specific tasks, such as mathematical / statistical computations, object stream filtering and transformation, signal processing, model and data management, etc. This library may be stored in one or more central servers (CS) or bound to interpreter software functions (ES) in each or at least several edge computing devices (ECDs). The function library can be modular because it can be extended to meet new user needs, and it is arranged so that users can instantly define and deploy new user functions by simply pushing updated library information to the relevant device ECDs.
[0202] However, existing algorithms and codebases can be implemented in other programming languages, or for other reasons are not directly compatible with interpreted software functions (ES). Such existing code can then be used as "external" query language functions using the programming language-specific APIs provided by interpreted software functions (see...). Figure 4aInsert system 100. Then, such external functions can be used transparently in queries and expressions defined using this query language. For example, in cases where the interpreted software function ES is implemented in Lisp, and where it is desirable to use code in the C programming language as part of the computation of query results in an edge computing device ECS, such C-language-specific APIs can be used, allowing the interpreted software function ES to directly call the C-language-implemented code, resulting in the execution of the corresponding C code as a result of the interpretation and processing of the query executed by the interpreted software function ES.
[0203] like Figure 4a As shown, using the concepts of external functions and stream wrappers, it can be explained that software functions ES can be arranged to be highly scalable, in the sense that many different kinds of plugins can be added without changing other parts of the system 100.
[0204] "Analysis Model" Figure 4a An analysis model is a model that specifies the transformation, filtering, computation, and inference of a source data stream to produce an object data stream as a result. Such analysis models can be specified by the user without requiring deep programming skills or detailed knowledge of the internal workings of the interpreter software function ES kernel. Furthermore, such analysis models can be defined using the same object-oriented query language used to define queries of this invention type (using the syntax described above). Therefore, an analysis model can be defined as a set of query language functions and / or sequential query definitions pushed to the edge computing device ECD via the interface IF and stored in the local database EDB. The analysis model can then be used in queries proposed to the edge computing device ECD via the API of the interpreted software function ES running on the ECD under consideration.
[0205] Still referencing Figure 4a "External functions" are functions implemented in any conventional programming language (such as C, Lisp, or Java) to implement external algorithms, such as numerical, statistical, and / or inference algorithms. Such external functions can be used as plug-ins, referenced in queries of the type defined in this invention using the query language. Using the external function API of the interpreted software functions ES, such functions can be directly referenced and accessed from the interpreted software functions ES via query language references without any other porting or modification. Such external functions can be pre-compiled and loaded into local memory M during installation or later (such as when needed). Specifically, such external function algorithms can be used in analysis models of the type described above to filter and transform incoming data streams into exported object streams.
[0206] Furthermore, external functions can be permitted to access functionality provided by interpreted software functions (ES), thereby allowing the addition of very powerful capabilities to interpreted software functions (ES) via such external functions, such as access to the file system, operating system calls, inference engines, or complex database managers that form part of the kernel functionality. The external function API can also include mappings of external language data structures to query language data structures, enabling direct access to data without data conversion. For example, based on a single mapping definition for simple and complex data types (included in the API), C language data structures can be directly mapped to their corresponding query language data structures.
[0207] To access the incoming data stream in a sequential query, the data stream wrapper can be implemented partially (as an external function) or entirely using functions defined by the query language. For example, the query language could be configured with a standard sensor interface for a common sensor S, provided as part of the interpreter software function ES. Only one such data stream wrapper needs to be implemented for each incoming data stream; once implemented for a specific stream type, all such streams can be queried using a sequential query of the type of this invention. This data stream wrapper can then be defined as a sequential query that returns a stream of object data. Such queries may be defined as functions, thus accepting arguments, such as the identity of the stream it encapsulates.
[0208] Data stream wrappers need to physically access external data streams and convert each of the arriving data stream elements into a suitable data format for efficient and flexible processing by the interpreter software function (ES). Different streams typically use different data structures to represent their elements; therefore, data stream wrappers of this type usually convert such external data representations into a format already supported by the interpreter software function (ES). However, in some cases, binary data representations can be directly promoted to the interpreter software function (ES) without any data transformation. This can be achieved by mapping such binary data representations to an internal binary data format particularly suitable for corresponding to the known binary data format output by the sensor S under consideration.
[0209] Interpreted software functions (ES) can be deployed with built-in libraries that include built-in data stream wrappers for common infrastructures such as Kafka, Azure IoT Hub, MQTT, CVS, and JSON Streaming. Furthermore, additional wrappers can be easily downloaded to each edge computing device (ECD) as needed, thus forming part of the immediately effective interpreted software function (ES).
[0210] Typically, the data stream originating from sensor S will be endless (as described above). However, in some cases, the data stream can also be finite. For example, there might be a special JDBC data stream wrapper available that processes finite results from an SQL query, which are passed as wrapper function parameters to a relational database via JDBC. This wrapper can then be used to maintain peer metadata in the domain name server NS.
[0211] As described above and as Figure 4a As shown, interpreted software functions (ES) can also be embedded in the software environment that exists on the hardware on which the interpreted software function ES executes. In this way, the embedded application or system can access the object data stream generated through the continuous query API provided by the interpreted software function ES. The embedded application or system can run in the same process and address space as the interpreted software function ES, such as when running an embedded interpreted software function ES on an edge computing device (ECD) with limited hardware resources. In another example, an interpreted software function ES running on a specific edge computing device (ECD) can act as a client to a central server (CS) running on some other computer or cluster communicating via TCP or some other communication infrastructure. For example, there might be such an embedded interface defined for a public infrastructure such as Kafka, MQTT, or Azure EventHub.
[0212] Figure 4b The hierarchical structure of the components in an example of an edge computing device (ECD) of the type of this invention is illustrated. Deeper layers are independent of the upper layers. The "local database" is the local master database (EDB) existing in each edge computing device (ECD) as described above. The streaming model and temporary data are stored in this EDB. The local database EDB is managed by a subsystem denoted as "saStorage" via the interface "sa_storage.h".
[0213] At the top of "saStorage", there are two separate interpreters ("SLOG" and "aLisp"). The module "Lisp-SLOG API" is the link between these interpreters, allowing "aLisp" to be called from "SLOG" and vice versa.
[0214] "aLisp" is an interpreter for a subset of "CommonLisp" (the standard dialect of the Lisp programming language itself), extended by functions (i.e., "sa_kernel") required to implement the upper application layer in edge computing devices (ECDs). "CommonLisp" is a standard functional programming language where all functions return a finite number of objects of different types as results. The objects returned from functions are stored in main memory (EDB), which becomes problematic when the results are too large. Note that the Lisp code itself is data stored in the main database (EDB).
[0215] On the other hand, "SLOG" is a data stream interpreter that expresses execution plans in a language similar to programming languages such as Prolog. The "SLOG" operator does not return a single object; instead, the result is a handle to a stream of objects. The calling application sends callbacks to "SLOG," which are then applied to the elements of the resulting object stream. Therefore, the "SLOG" operator is a so-called generator, unlike functions in "aLisp." External object-oriented query language ("OSQL") functions can be implemented as external "SLOG" operators. External "SLOG" operators (i.e., external query language functions) return the resulting object stream by iteratively calling callback functions in the interpreter (IS) as arguments.
[0216] External programs (such as those within the embedded software system ESS) can invoke the edge computing device ECD kernel via the API "CQ API". The invoking application can execute as a separate process on the same computer, or from a different peer or even an external entity via a suitable communication system such as TCP. The invoking application can also be in the form of one or more application threads. Thus, the kernel guarantees thread safety. Elements of the object stream can be passed to the ESS by interpreting the software IS that calls callback functions within the embedded software system ESS.
[0217] Refer again Figure 2 In some embodiments of the invention, system 100 also includes an interactive graphical user interface (GUI) that allows a user of system 100 to visually view computer code of the type of the invention formatted according to the syntax. The viewed computer code is computer code stored in several different edge computing device ECDs, which uses the syntax to define several different queries using the query language. These different queries may typically include interactions between requesting edge computing device 110 and responding edge computing devices 120, 130, defined by the queries considered above, and also in a cascading / tree configuration of complex query definitions of the type discussed.
[0218] However, even though the GUI can be arranged to display the computer code as alphanumeric text, in the implementation discussed here, the GUI is arranged to use a visual symbol system to visualize the computer code.
[0219] That is, the query language can be isomorphic to such a visual symbol system, meaning that any query language computer code that follows the syntax will have a visual corresponding representation that can be visualized in the interactive GUI. This visual symbol system can be based on graphical primitives that have a well-defined one-to-one mapping to the corresponding representation in the computer code and are graphically interconnected in a way that represents how entities, data, computations, and queries are interconnected (as defined by one or more queries defined by the computer code), such as using lines or arrows.
[0220] Isomorphisms can be arranged such that the visual representation is more abstract than the actual computer code, thereby hiding certain details of the computer code, such as the specific computations performed on specific data. However, it is preferable that the isomorphism is complete, meaning that the computer code can be fully represented by the visual symbol system, and that the computer code and the visual representation can be converted to each other without losing any information.
[0221] Furthermore, the interactive GUI can be configured to allow the user to graphically and interactively (e.g., using a computer screen and computer mouse; or a touchscreen) directly use the visual symbol system to modify / correct / define one or more queries. For example, a user can drag and drop individual edge computing device ECDs to define requesting edge computing device ECDs and responding edge computing device ECDs, or can use graphical primitives in the interactive GUI to add queries, query parameters, and logic. It is noted here that, due to the isomorphism, such changes will always perfectly correspond to changes in the graphically represented alphanumeric computer code.
[0222] Then, as a result of such changes, the interactive GUI is configured to communicate the updated computer code to any relevant edge computing device ECDs, thereby reflecting any updates made to the visual representation. This may mean: providing updated queries to one or more edge computing device ECDs; instructing one or more edge computing device ECDs that previously loaded sequential queries are no longer active; and / or providing updated queries to one or more edge computing device ECDs. This provision is carried out in the manner generally described herein, using a corresponding central server CS to automatically push the considered updates to the considered edge computing device ECDs. Once updated, the relevant edge computing device ECDs can typically begin using (interpreting) the loaded queries immediately. Especially in cases where such visual definitions and loaded queries are sequential queries of the type described above, this will result in changes to the interactive GUI leading to the immediate and automatic full deployment of computer logic across potentially numerous related edge computing device ECDs.
[0223] System 100 can be configured to continuously provide the interpreted software function ES with an updated state of state and information flow to an interactive GUI (such as via its corresponding central server CS), such that the interactive GUI is updated in real time or near real time with respect to the current state of query programming and information flow in System 100.
[0224] In this way, the interactive GUI will be dynamically updated to display the changed information flow based on query initiation, execution, and cessation. This means that when a user uses the interactive GUI to change the visual representation of computer code, these changes are immediately translated into corresponding computer code and pushed to the relevant edge computing device (ECD) (possibly after the computer code is automatically compiled for a very thin edge computing device ECD). The pushed query information then generates various modified information flows, which are immediately reflected in the graphical GUI by feedback information from the considered edge computing device ECD for the user to view.
[0225] An interactive GUI can be configured to request computer code stored therein from each of the associated edge computing devices (ECDs), and in response to such request, provide the visual view based on the computer code received from the edge computing device (ECD).
[0226] Figure 6 An example of a method for collecting data in system 100 is shown.
[0227] The method begins in the first step.
[0228] In subsequent steps, corresponding interpreter software functions of the general type discussed herein are provided for at least the first edge computing device 110 and the second edge computing device 120 among the plurality of edge computing devices (ECDs) included in system 100. These interpreter software functions are arranged to execute on the CPU of the edge computing device (ECD) under consideration and interpret computer code of the type discussed herein, which is received via the digital communication interface (IF) of the edge computing device (ECD) under consideration and stored in the memory (M) of the edge computing device (ECD). The computer code is based on a query language of the type of this invention with a predetermined syntax, which is then arranged to define queries whose results are data streams.
[0229] In a subsequent step, the first edge computing device 110 in the edge computing device ECD provides the second edge computing device 10 with computer code of the type that uses the syntax to define at least one query via the digital communication interface of the second edge computing device 120 in the edge computing device ECD.
[0230] In subsequent steps, the second edge computing device interprets the received computer code, which includes the second edge computing device 120 performing calculations based on measurements from the sensor S of the second edge computing device, and the query being defined according to the calculation to be performed.
[0231] In a subsequent step, the second edge computing device generates the result of the at least one query.
[0232] In subsequent steps, the second edge computing device transmits the result to the first edge computing device 110 via the digital communication interface IF of the second edge computing device 120.
[0233] It should be understood that, in this embodiment example and in other embodiment examples, communication between edge computing devices (ECDs) can typically be conducted via the respective digital communication interface (IF) of each edge computing device in the edge computing device (ECD) involved in the communication under consideration, and also via any involved intermediate central server (CS).
[0234] The method ends in the subsequent steps.
[0235] Figure 7 A method for collecting data in system 100 is illustrated. Similarly, system 100 includes at least a first edge computing device 110 and a second edge computing device 120.
[0236] The method begins in the first step.
[0237] In subsequent steps, corresponding interpreter software functions ES of the present invention type are provided for at least the first edge computing device 110 and the second edge computing device 120. These interpreter software functions are arranged to execute on the CPU of the edge computing device ECD under consideration and interpret computer code of the present invention type, which is received via the digital communication interface IF of the edge computing device ECD under consideration and stored in the memory M of the edge computing device ECD under consideration according to a query language of the present invention type with a predetermined syntax, the syntax being arranged to define a query whose result is a data stream.
[0238] In a subsequent step, the first interpreter software function ES of the type executed on the first edge computing device 110 submits a first query of the type to the second edge computing device 120.
[0239] In a subsequent step, in response to the first query received by the second edge computing device 120, a second interpretation software function ES of the type described above, executed on the second edge computing device 120, generates a second data stream (the term "second data stream" simply refers to the data stream generated by the "second" edge computing device 120). The second edge computing device 120 transmits the second data stream back to the first edge computing device 110 via the digital communication interface IF of the first edge computing device ECD.
[0240] In subsequent steps, the first interpreter software function ES performs preprocessing on the second data stream (referred to as "second" preprocessing, since it is performed on the "second" data stream), thereby producing a preprocessed second data stream used in the first computation. This second preprocessing transforms the second data stream such that it conforms to a predefined global data ontology of the type described herein.
[0241] In a subsequent step, the first interpreter software function ES uses the preprocessed second data stream to perform a first computation to compute a first result.
[0242] The method ends in the subsequent steps.
[0243] Figure 8 A method for collecting data in a system 100 comprising several edge computing devices (ECDs) of the present invention is illustrated.
[0244] The method begins in the first step.
[0245] In subsequent steps, a corresponding interpreter software function ES of the type described herein is provided for each of the plurality of edge computing devices (ECDs). This interpreter software function is arranged to execute on the CPU of the considered edge computing device (ECD) and interpret computer code received via the digital communication interface IF of the considered edge computing device (ECD) and stored in the memory M of the considered edge computing device (ECD) according to a query language of the type of this invention with a predetermined syntax. This syntax is arranged to define queries transmitted from the corresponding requesting edge computing device (ECD) to one or more corresponding responding edge computing devices (ECDs) based on corresponding calculations to be performed by the considered responding edge computing device (ECD) based on measurements from the corresponding sensor S of the responding edge computing device (ECD). The result of these queries is a data stream returned to the requesting edge computing device (ECD). The query language is isomorphic to a visual symbol system of this type and employs the above-described manner.
[0246] In subsequent steps, an interactive graphical user interface (GUI) of the type described above is provided, allowing the user of system 100 to visually view computer code stored in several different edge computing devices in the edge computing device ECD using the syntax and to define several queries using the query language, including the relationship between the requesting edge computing device ECD and the responding edge computing device ECD defined by the considered query using the visual symbol system.
[0247] In subsequent steps, the user uses the interactive GUI to graphically and interactively change one or more queries in the query using the visual symbol system.
[0248] In subsequent steps, the interactive GUI thus automatically transmits the updated computer code of this type to any relevant edge computing device (ECD), thereby reflecting the change to the query.
[0249] The method ends in the subsequent steps.
[0250] These methods and / or other aspects of the methods described herein can generally be freely combined.
[0251] The processing of queries in the system 100 of the type described herein will be described in more detail below.
[0252] Generally, queries are processed by transforming and converting them into what is known as an execution plan. An execution plan is an intermediate procedure that, for a given query or function definition, explicitly specifies how to process the query to achieve a response. Specifically, the execution plan specifies a particular sequence of steps used to access data, such as data in a database, to process the query.
[0253] The aspects specified by the execution plan can include the optimization order and / or selected strategies for accessing internal and external algorithms and / or data flows. To tune query performance, execution plans can be configured to be inspectable by query tuning experts and therefore expressed at a high level of human readability. However, they are not designed for actually programming query algorithms (i.e., writing programs); instead, execution plans can be represented as data structures in the main memory of the edge computing device (ECD). They can also be graphically presented to the user in the aforementioned GUI.
[0254] In the system 100 of this invention, the execution plan can express streaming computation; in other words, it is a description of high-performance computing of numerical algorithms applied to a potentially endless (and possibly continuous) stream of data flowing through the system 100. In the example of the system 100 described herein, the internal execution plan language expressing the execution plan can be a language called SLOG (Streaming Logic). SLOG is a very simple yet powerful procedural algebra used to represent executable procedural code on streaming data to implement optimized OSQL. The internal execution plan language (such as SLOG) used to define the execution plan in the system 100 can have one, any combination of, or all of the following properties:
[0255] Streaming: The internal execution plan language can be configured to generate potentially endless streams of bindings from variables to different kinds of objects (including strings, numbers, vectors, arrays, and even other streams), instead of producing complete data objects as in regular programming languages. This is achieved by generating bindings to stream elements and passing them to the application via callback functions, rather than constructing complete data structures as the result of queries.
[0256] Filtering: The internal execution language can be configured to provide powerful logical filtering of data objects. This can be achieved by providing predicate-based logic operators, as in Datalog or SQL.
[0257] Computation: The internal execution language can be configured to perform high-performance numerical computations via assembly instructions that are at least partially compilable to the relevant hardware. This can be achieved by providing the possibility of declaring variables and functions used in the query language to use basic hardware-oriented data types and instructions, such as functions on numbers, arrays, or strings.
[0258] Simplicity: The internal execution plan language can be defined using a small number of primitives, such as fewer than 10 operators, which are well-suited for efficient filtering and computation of data streams, and at least some of these operators (specifically numerical operators) can be easily compiled into corresponding hardware-supported assembly instructions. In the specific case of SLOG, it can be described by just five operators, which are used to define increasingly complex combinations of operators involving both streaming logic filtering and numerical computation.
[0259] Abstract: As mentioned, the Internal Execution Plan Language (IOPL) can be configured to prevent users from writing computer programs in it; instead, IOPL programs can be configured to be represented as data structures in the internal main memory database within an edge computing device (ECD). Such execution plans can be generated by the query optimizer based on the query under consideration. For example, when an OSQL query or function definition is perceived as slow, an advanced user can examine the generated SLOG algebraic expressions to identify any poor optimization decisions, allowing for query reformulation or instructing the query optimizer on how to improve the plan.
[0260] Scalable: The internal execution language can be scalable, allowing new types of filtering and computation to be added to the data stream without altering the system core. To this end, the internal execution plan language can be configured to support the general types of external functions described above. Such external functions might be implemented in some traditional, conventional external programming language, for example, by extending SLOG with new operators. Programmers can then use the API of the considered edge computing device (ECD) to perform computations based on the bound variables of the tuples of new variable bindings issued iteratively so far, bound by the external function, thereby generating a stream of variables bound to objects. The API can be configured to include the possibility of influencing the query optimizer, for example, by providing the cost and size of the results produced by the external function to guide the query optimizer to reorder and transform operators, thereby optimizing execution speed and data requirements.
[0261] Embeddable: The internal execution plan language can be configured to allow functions expressed in the internal execution plan language to be called from a conventional programming language used by the embedded software system ESS. The results are then returned as an object stream via a system call to a callback function in the embedded software system ESS expressed in the conventional programming language. This callback function accesses objects bound to variables generated from the object stream. In the API, for each tuple of bound variables in the returned object stream, a function in the caller is invoked from the interpreter software function ES of the edge computing device ECD.
[0262] As detailed below, a fragment of a single execution plan (e.g., each) can always be compiled into binary machine-specific instructions, while any remaining part of the internal execution plan language program can be interpreted.
[0263] Starting with a general viewpoint and as mentioned above... Figure 2 As described, system 100 can be highly distributed, comprising a federation of several central server CSs managing a potentially very large number of edge computing device ECDs. By forming a hierarchical structure of central server CSs with registered edge computing device ECDs, system 100 can be scaled to handle a large number of edge computing device ECDs. This is in Figure 5 The example provided is shown in the document.
[0264] Each edge computing device (ECDA) (and possibly each central server (CS)) can be one of several possible types. A “full-engine” device (“query processor”) includes all the software modules required for query processing; while a “thin-engine” device (“query executor”) may only run execution plans that have already been generated (processed) by the full-engine device. In this document, the term “software IS” is used to refer to such software modules configured to run query code in the manner described herein.
[0265] Because small edge computing devices (ECDs) may have limited resources, such as at most 1MB of RAM, or at most 512KB of RAM, or at most 256KB of RAM, or at most 128KB of RAM, or even at most 64KB of RAM, thin engine devices, such as those without user data stored in a local database and without a query optimizer, are more suitable for applications like... Figure 11b , Figure 11c , Figure 11dThe device is shown. The RAM can be volatile (non-persistent) RAM memory, excluding any non-volatile (persistent) memory such as flash memory. Small edge computing devices (ECDs) may also have limited connectivity. Therefore, they may not be able to process queries themselves and need to be configured as thin engines. Each such thin-engine edge computing device (ECD) can then be configured to execute only queries that have already been optimized and compiled on a full-engine device (such as on different edge computing device ECDs or a central server CS as a full-engine type device). It should be noted that the streaming architecture of operators used in the execution plan, such as in SLOG, significantly limits the amount of memory required compared to materializing large query results; large query results are generally not feasible in small devices with limited memory. In some implementations, the central server CS is always a full engine, while the edge computing device ECD can be either a full engine or a thin engine. For thin-engine edge computing device ECDs, query processing can be performed on the central server CS where they are registered. A full-engine device can be implemented using up to 100MB of RAM, or up to 50MB of RAM, or up to 20MB of RAM, or up to 10MB of RAM, or even less than 5MB of RAM, such as Figure 11a The device is generally shown in the figure, wherein the corresponding definition of "RAM" is as described above.
[0266] Nevertheless, it should be understood that the various central server CS and edge computing devices ECD can run different or the same interpretive software functions ES between any two entities in terms of detail, scope, or functionality.
[0267] As will be described below, queries formulated in this query language can be translated into corresponding execution plans. These execution plans can then be compiled into device-independent and / or device-dependent binary code. Depending on the hardware prerequisites on each type of edge computing device (ECD), the interpreted software functions (ES) executed on the ECD under consideration can include various levels of functionality.
[0268] In the simplest case, for a thin-engine edge computing device (ECD) with very limited hardware specifications, the interpreter software (IS) can be configured to interpret and execute pre-compiled, device-independent binary code, thus requiring the translation of each such binary instruction into the corresponding device-specific machine code instruction. Alternatively, there may be thin-engine edge computing devices (ECDs) capable only of executing pre-compiled device-specific binary code; in this case, system 100 will also include edge computing devices (ECDs) configured to interpret code rather than simply execute it.
[0269] For thin-engine or full-engine edge computing devices (ECDs) with more robust hardware specifications, the interpreter software IS can interpret and execute uncompiled execution plans in addition to interpreting device-independent binary code of that type.
[0270] For full-engine edge computing devices (ECDs) with even more powerful hardware specifications, the interpreter software IS can interpret and run query language queries of the present invention type, in addition to interpreting device-independent binary code and uncompiled execution plans.
[0271] System 100 may include one or more of the above-described types of thin engine and full engine edge computing devices (ECDs).
[0272] Figures 11a to 11d An overview of various alternative options is provided.
[0273] Figure 11aSelected portions of an exemplary full-engine edge computing device (ECD) are illustrated. As described above, the edge computing device (ECD) includes an interface IF, through which the edge computing device can receive queries, uncompiled execution plans, machine-specific assembler code, and / or machine-independent assembler code from peer edge computing device (ECD) and / or central server (CS). After processing the query (such as generating a compiled or uncompiled execution plan from the query), the edge computing device (ECD) can transmit the results of such processing to one or more peer edge computing device (ECD) and / or central server (CS). The edge computing device (ECD) includes an interpreter software function ES, which in turn includes the interpreter software IS. The interpreter software IS further includes a module 201 arranged to interpret uncompiled query language code of the type of this invention and may be arranged to generate compiled and / or uncompiled execution plan code of the type of this invention. Thus, module 201 can interpret, run, and may at least partially translate and / or compile incoming queries. The interpreter software IS also includes a module 202 arranged to interpret and run uncompiled execution plan code (such as SLOG code). The interpreter software IS also includes module 203, which is configured to interpret and run platform-independent assembler code (such as SLAP code, see below). The interpreter software IS also includes module 204, which is configured to execute platform-specific assembler code, such as by invoking a loader. Since the code defining the query to be run by the edge computing device ECD can include elements from each code abstraction layer in the code abstraction layer (query language code, execution plan code, machine-independent assembler code, machine-specific assembler code), modules 201, 202, 203, and 204 can communicate with each other so that one or more correct parts of the considered query are processed by the correct module. Specifically, module 201 can be configured to push the execution plan to module 202 for processing; while module 202 can push the assembler code to modules 203 and / or 204 for processing. Any artifacts (such as execution plans or compiled portions) can be distributed back to other edge computing devices ECD and / or the central server CS via interface IF.
[0274] Figure 11b An example of a first exemplary thin engine client edge computing device (ECD) is illustrated, which, instead of excluding module 201, is arranged to use modules 202, 203, and 204 as described above to handle execution plans and machine-independent / machine-specific assembler code. Any resulting compiled assembler code can be distributed back to other edge computing devices (ECDs) and / or the central server (CS) via interface IF. Otherwise, Figure 11b Edge computing devices (ECDs) will be compatible with Figure 11a It works in the same way as edge computing devices (ECDs).
[0275] Figure 11cA second exemplary thin engine client edge computing device (ECD) is illustrated, which corresponds to Figure 11b The thin engine client edge computing device ECD, but excluding module 202. Figure 11c The client-side edge computing device ECD can therefore interpret and run the assembler code in modules 203 and 204, but does not perform any compilation whose results can be distributed back.
[0276] Figure 11d Similar to Figure 11c However, a third exemplary thin engine client edge computing device (ECD) that does not include module 203 is also illustrated.
[0277] It's possible that all queries and models cannot be, or are not, fully compiled into binary instructions. For example, for this reason, in some implementations, the corresponding interpreter software IS for all, at least most, or at least some of the edge computing devices in an edge computing device ECD is arranged to interpret execution plans that are at least not fully compiled, in order to interpret execution plan fragments that have not yet been translated into binary code. In such cases, the execution plan compiler (query processor) identifies code fragments in the execution plan that it can compile, then generates binary code for those compiled execution plan fragments, leaving the rest to the interpreter. Therefore, in such cases, the execution plan may include internal execution plan language code and pointers to binary code fragments.
[0278] Alternatively, in some implementations, all, at least some, or at least one of the interpreter software in the interpreter software IS of the edge computing device ECD may not include a query processor. Instead, a query executor may be provided for assembly code (such as SLAP code, see again below) to interpret any compiled, machine-independent assembly code that forms part of a query passed to the device under consideration.
[0279] In both alternatives, some (but not all) of the thin engine edge computing devices (ECDs) of System 100 may also be completely uninterpreted software functions (ES), and instead be configured to receive queries only in the form of compiled platform-specific binary code invoked by the loader.
[0280] The primary objective of the system 100 described herein is to achieve full user interactivity, such as through the GUI. All optimizations and compilation should be performed without delay after the query or function definitions have been defined.
[0281] To achieve this, it is important to minimize compilation time so as to guarantee that defined or updated queries are executed immediately without causing any noticeable delay to the operating user.
[0282] Furthermore, since memory on thin engines is typically very limited, the size of the generated code should be minimized. Given the generally slower processing speed of such thin engines, the generated code should also be as efficient as possible.
[0283] The compilation and optimization of queries should also take into account that the edge computing devices ECD and the central server CS in System 100 (the consortium) may have different hardware and / or operating system architectures. That is, the architecture of System 100 should be able to handle the distribution of code to all different kinds of peers in the consortium, and therefore must be able to deliver code for different architectures around the consortium under consideration.
[0284] Furthermore, some edge computing devices (ECDs) and / or central server (CSs) may lack certain functionalities. For example, some devices may not support binary code generation (this is the case, for example, for OSX devices). System 100 can then include features to compensate for such missing functionalities in different edge computing device ECDs and / or central server (CSs).
[0285] Figure 9 The steps of a method for query processing in system 100, which is configured to perform the steps, are illustrated. As described above, the method is also used to distribute and deploy software functionality on system 100.
[0286] Figure 10 The information flow and process are illustrated from the perspective of the action entity (the software module of the interpreting software function ES of the central server CS) and the data operated by the action entity.
[0287] It should be noted that Figure 9 All intermediate representations of the information indicated in the middle are verifiable for adjustment experts.
[0288] The method begins in the first step.
[0289] In subsequent steps, if this has not already been performed, the interpreted software function ES of the current type can be provided to one or more central servers CS and / or one or more edge computing devices ECD.
[0290] In subsequent steps, the central server CS accepts queries of the type of this invention from the querying party and via the digital communication interface IF. The querying party can be a different central server CS, an edge computing device ECD, or any external entity that uses the API of the central server CS to submit the query. As described above, the query is defined such that the result of the query under consideration is a stream of data objects, wherein at least one stream is an endless stream of objects computed based on data measured by the sensors S of the edge computing device ECD, as described above. The results can then be provided continuously or intermittently over time and communicated via the digital communication interface IF, as will be described below.
[0291] Specifically, the received query can be defined according to a query language of the present invention type with a predetermined syntax, which is arranged to define a query whose result is a data stream and allows the query to be defined based on a calculation to be performed based on the measurement value from the sensor S.
[0292] The query language can be OSQL (object-oriented) query language.
[0293] To illustrate the principles of the invention, the following example is provided in the form of an OSQL function, the task of which is to find a stream containing prime numbers less than n:
[0294] create function primes(Integer n)->Stream of Integer m
[0295] as select m
[0296] where notany(select factor
[0297] from Integer factor
[0298] where mod(m,factor) = 0
[0299] and factor in range(2,sqrt(m)))
[0300] and m in range(2,n)
[0301] Note that the select expression in the body does not specify the order in which functions and filters should be evaluated; it only states the expected properties of the resulting stream of integers. An optimized executable for calculating the prime number stream is automatically generated by the query processor.
[0302] Furthermore, OSQL functions can be defined based on variables of any domain (such as integers, vectors, matrices, strings, etc.), which is beneficial for defining queries and functions involving numerical computation.
[0303] OSQL queries are considered anonymous (lambda) functions without arguments, and are executed immediately after being defined. For example, the following OSQL query (defined as finding a stream of numbers between 1 and 10 whose square roots are less than 10):
[0304] select Stream of i from Integer i
[0305] where i in range(1,10) and sqrt(i) < 10
[0306] This can be handled by generating the following anonymous lambda function:
[0307] create function lambda()->Stream of Integer
[0308] as select i from Integer i
[0309] where i in range(1,10) and sqrt(i) < 10,
[0310] Then lambda() is called immediately.
[0311] In subsequent steps, the central server CS's interpreter software function ES parses the received query to generate a parsed query expression.
[0312] Queries received by the central server CS can be in text (plaintext) format, or can be easily transformed into such a format by, for example, unpacking and / or decrypting the query.
[0313] Therefore, the first step in query processing is to parse the query into an equivalent syntax from text for further internal processing. Such an equivalent can be a so-called abstract syntax tree, such as an S-expression.
[0314] An abstract syntax tree (AST) is typically a data tree representation of the abstract syntactic structure of text (such as source code) written in a formal language. Each node in the tree represents a construct that appears in the text.
[0315] S-expressions (or "symbolic expressions") are typically expressions that use similar named symbols to represent nested list (tree structure) data. In Lisp's common bracketed syntax, an S-expression can be defined as:
[0316] 1. Atom, or
[0317] 2. An expression of the form (x .y), where x and y are S expressions.
[0318] More specifically, an S-expression can be viewed as a Lisp program that can be evaluated immediately. Evaluation of an S-expression can result in the execution of at least some (such as all) of the subsequent query processing steps described below to evaluate the initially received query. The use of S-expressions enables the representation of OSQL in Lisp code, which can then be executed immediately or transmitted to other peer devices for remote evaluation (i.e., sending executable software code from a central server CS to a remote computer entity for subsequent execution, and subsequently returning the execution result to the central server CS).
[0319] The S-expression used for (equivalent to) primes() is as follows:
[0320] (create-function primes ((integer n)) ((stream of ((integer m))))
[0321] as (m)
[0322] where(and (notany (select (factor))
[0323] foreach((integer factor))
[0324] where (and (= (mod m factor) 0)
[0325] (= (in (range 2 (sqrt m)))factor))))
[0326] (= (in (range 2 n)) m)))
[0327] In subsequent steps, the parsed query can be transformed into an equivalent query in a declarative object-oriented query representation based on predicate logic by the interpreter function ES of the central server CS. A "declarative" representation is a representation that defines the logic of computation without specifying its control flow. A "predicate" is a logical symbol representing an attribute or relation. For example, in a first-order formula P(a), the symbol P is the predicate applicable to each constant a. Similarly, in the formula R(a,b), R is the predicate applicable to each constant a and b. Predicates can be interpreted as relations. For example, in the standard semantics of first-order logic, the formula R(a,b) would be true in interpretation if the entities represented by a and b represent the relation represented by R.
[0328] In some implementations, declarative object-oriented query representations can well classify various objects, such as tensors, functions, data generators, etc. "Strongly classified" languages have strict classification rules that allow the compiler to determine the type of an expression at compile time, in contrast to "weakly classified" (or "late-binding") languages (such as Lisp and Python) that must determine the type of each object at runtime. The inventors have successfully used a declarative object-oriented query representation that extends Datalog (a declarative logic programming language that is a subset of Prolog). However, Datalog's bottom-up query processing is not suitable for data streaming because it materializes large data objects to produce complete query results, while data streaming produces variable-bound streams of small transient objects passed to callback functions in the application.
[0329] An exemplary extension of traditional languages such as Datalog is a primitive for defining user-defined predicates based on multi-directional foreign functions (i.e., functions that can be called with several different inverse functions depending on what arguments and results are bound or not bound, called binding patterns, in the form of external OSQL functions defined in some external programming language such as Lisp, Lisp, or Java). See T. Risch et al., “Representing Matrices Using Multi-DirectionalForeign Functions,” which is disclosed in Gray et al., “Functional Approach to Computing with Data” (Springer, ISBN 3-540-00375-4, 2004). For example, a call to the function sqrt(Real x)->Real y in the query select sqrt(4) requires that y be calculated as the square root of x=4, i.e., x is bound and y is not bound. However, in the query select x from Number x where sqrt(x)=2, y=2 is bound and x is calculated as y. 2 In the query `select sqrt(4)=2`, both x and y are bound, and the query optimizer will subsequently select test 2. 2 The execution plan for =4 is cheaper than the test 2, which is the square root of 4.
[0330] In some implementations, declarative object-oriented query representations may require that the outer function definition also correspond to the inverse outer function definition, or support such a definition.
[0331] Therefore, when using multi-directional external functions, programmers can choose not only to implement the function in the selected external language, but also to implement it in its inverse. For example, the inverse of the external function sqrt(Real x) -> Real r is r. 2 This can significantly improve the performance of queries using external functions because it allows the query optimizer to choose between the function and its inverse to bind variables in subsequent steps of defining the execution plan. By defining the inverse of the function, the query optimizer has more options for optimizing the query because it can bind more variables by applying such inverses.
[0332] Therefore, declarative query representations can include such multidirectional (or at least bidirectional) functions.
[0333] In some implementations, declarative object-oriented query representations may require or support the use of associated cost functions (cost models) for defining external functions.
[0334] The inventors have successfully used the query language ObjectLog, which is extended with streams and tensors, for the purposes of this invention (see Risch et al., “Main Memory Oriented Optimization of OOQueries using Typed Datalog with Foreign Predicates”, EEE Transactions on Knowledge and Data Engineering, Vol. 4, No. 6, December 1992).
[0335] In the transformation to equivalent queries described above, the interpreter software function ES of the central server CS can use type inference (generally, "inference" refers to the automatic detection of the type of an expression in a formal language) to determine the types of variables appearing in the parsed query. The internal use of the predicate-based logical representations of the types described above enables powerful logic-based transformations in the subsequent definition of the execution plan. Such transformations can be used to simplify and optimize logical query expressions to produce smaller and faster equivalent expressions. However, predicate representations alone may not be sufficient for efficient query execution; therefore, subsequent query processing steps (defining the execution plan) can be supplemented with information from the initial query, along with data statistics and knowledge about the functions (such as statistics about their execution cost, result size, and the percentage of possible input variables that produce non-empty results) to automatically select a good strategy for the execution plan.
[0336] The ObjectLog defined in the above function is represented as follows:
[0337] predicate primes (Integer n, Integer m)
[0338] locals Stream of Integer v7
[0339] where makestream((lambda (Integer m1, Integer factor)
[0340] locals Real v3,Integer v6
[0341] where mod(m1, factor, 0)
[0342] and sqrt(m1, v3)
[0343] and cast_integer(v3, v6)
[0344] and range(2, v6, factor)),
[0345] m, v7)
[0346] and notany(v7)
[0347] and range(2, n, m);
[0348] ObjectLog is a relational language (a programming language where programming logic consists of relations and outputs are computed based on the applied queries), in which predicates p(v1,…,v) are defined. n ), without specifying which variables v i What are the inputs and which variables v? i This is the output. Therefore, ObjectLog is a purely logical representation of the query definition, well-suited for logical transformations and simplifications (to define the execution plan, see below) prior to subsequent query optimization steps. In contrast, OSQL is both a relational and functional query language, where OSQL functions are defined based on declarative queries (“select” expressions), with the function's input and output indicated by the -> symbol in the function header. In OSQL, functions are defined with logical directions and produce results based on the bound variables. This is not the case in ObjectLog relations where no variables are specified as bound. To run an ObjectLog program from a given OSQL query, additional information about the variables bound in the OSQL query is required. This additional information may also include the inverse of the function and the cost model of the outer function. It should be recognized that ObjectLog is one possible example of a useful relational language and other relational languages with one or more of the properties described herein with respect to ObjectLog can be used instead.
[0349] As a simple example of the ObjectLog representation of a parsed query, suppose it is defined as the following query that finds the square root of 4 for integers between 1 and 1,000,000:
[0350] select Stream of x from Integer x where x in range(1,1000000) andsqrt(x)=4
[0351] Without knowing the inverse of y=sqrt(x) is x=y 2 In this case, the entity processing the query would have to test whether sqrt(x) = 4 for a million values of x. Knowing the inverse, this could be changed to performing a single multiplication. That's more than a million times faster.
[0352] In the prime number example described above, the ObjectLog compiler can perform type checking and expand nested function calls into union predicates. Furthermore, it can generate bindings for the variable v7 to a stream of integers represented by anonymous lambda predicates applied to parameters m and v7. Anonymous lambda predicates and functions are not specific to ObjectLog but can be used in other examples. Note that the test predicate mod(n, factor, 0) requires variables n and factor to be known, and the call to makestream(pred, arg, b) to define the stream of integers b requires binding m to the set of possible bindings of v in pred(arg, v), making the predicate definition unexecutable in its presented form. However, the query optimizer will produce an executable in the form of an execution plan.
[0353] That is, in subsequent steps, the interpreter software function ES of the central server CS can generate an execution plan corresponding to the parsed (and possibly compiled) query expression described above, whereby the execution plan defines at least one computation to be performed based on measurements from the sensor S of the edge computing device ECD. It should be understood that this definition can be explicit (e.g., explicitly referencing the sensor S and / or the edge computing device ECD under consideration) or implicit (i.e., the identity of the sensor S is inferred from other information in the query, or is provided externally to the central server CS and / or the edge computing device ECD, or is dynamically determined on-the-fly at runtime based on current parameter values).
[0354] For this purpose, SLOG (as characterized above) can be used, or an alternative internal execution plan language with one or more of the attributes mentioned above can be used.
[0355] As mentioned above, non-procedural, logic-based query representations do not specify which algorithm (i.e., external function) is used or in what order the elements of the result stream or set are produced. Therefore, in this case, and in others, responding to the query or invoking a function requires the query optimizer of the central server CS (Service Controller) to select a good execution strategy.
[0356] Cost-based query optimization (the practice of determining the most efficient way to execute a given query by considering a set of possible query plans) is useful for the purposes of this invention to generate execution plans based on data statistics and other knowledge about the operators in the query and the data stored in a memory database on a central server CS. Query optimization is used to generate efficient and executable execution plans. This can be very important because a poor execution plan can be thousands of times slower than the optimal execution plan, or even fail to execute.
[0357] As described above, the execution plan can be expressed using the internal execution plan language, specifically SLOG for OSQL queries. Therefore, for a given OSQL function, the query optimizer of the interpreter function ES in the central server CS generates an optimized stream generator in SLOG that implements the function under consideration. Optimization can be guided by a query cost model that estimates the cost of calling the stream generator based on the optimization metrics of the operators and the algorithms used therein. The stream generator is a predicate extended using binding annotations that indicate what variables are the inputs and outputs in the predicate, respectively. When executed, the stream generator produces a binding stream for the output variables, assuming the input variables are known. For example, the function `primes(Integer n)->Stream of Integer m` will produce a binding stream of variables `m` up to prime numbers less than `n`.
[0358] SLOG is an example of a very simple yet powerful language for representing OSQL execution plans as stream generators. It has only five built-in operators: "call" (calling an external stream generator), "funcall" (external function call), "and" (intersection), "or" (union), and "or!" (conditional union). In addition to these five system operators, users can implement stream generators in various external programming languages via an API. The system can include and support libraries of predefined stream generators that can be used directly by programmers.
[0359] Below are some examples of SLOG algebra expressions generated for some exemplary OSQL queries.
[0360] The following stream generators are generated by the query optimizer for the function `primes()`:
[0361] primes-+(Integer n, Integer m)<-
[0362] locals (Bag of Integer v6)
[0363] call(range--+,2,n,m)
[0364] and funcall(makebag,(lambda-+(Integer m1, Integer factor)<-
[0365] locals(Real v2, Integer v5)
[0366] funcall(sqrt,m1,v2)
[0367] and funcall(cast_integer,v2,v5)
[0368] and call(range--+,2,v5,factor)
[0369] and funcall(mod,m1,factor,0)), m, v6)
[0370] and funcall(notany,v6)
[0371] Here, we test the `funcall(notany, v6)` function calling the external function `notany(v6)`. The function succeeds when it returns true if `bag v6` is empty.
[0372] Here, the stream generator in the SLOG execution plan is generated by annotating the previous ObjectLog predicate with binding patterns, which indicate that the parameter n must be bound to make it applicable and produce a bound stream to m. The binding pattern is a string where "-" indicates that the variable must be bound to some value, and "+" indicates that the variable will be bound by calling the stream generator. Generally, the SLOG stream generator produces zero, one, or several bindings to an unbound variable for a given bound variable. For example, the call range--+(2,v5,factor) (when v5 is bound to 3) generates a bound object stream of the variable factor to each of the integers 2 and 3. A common case is that the stream generator produces a single value called an outer function. For example, the outer function sqrt(x) calculates the positive square root of x, and the outer function call funcall(sqrt,m1,v2) binds v2 to 2 (if m1 is bound to 4).
[0373] Queries can generate anonymous (lambda) stream generators, for example:
[0374] lambda+(Integer i)<-
[0375] call(range--+,1, 10, i) and
[0376] funcall(less,i,5)
[0377] Therefore, as a result of this step, an execution plan is established, which is equivalent to receiving a query (a query defined using the query language) and defining the efficient way to produce a response (result) to the query.
[0378] In subsequent steps, the at least part of the execution plan, which is part of the interpreter software function ES of the central server CS as the internal language compiler, can then be compiled by converting at least a portion of the resulting execution plan into corresponding platform-independent assembly code (an "assembler" as defined in the assembler language).
[0379] It is important to note that the term "assembler language" refers to a symbolic machine code language that typically uses one statement per machine instruction. However, assemblers can be defined in a platform-independent manner, independent of the specific hardware and / or operating system requirements or features that may exist on the edge computing device (ECD) that executes the corresponding binary machine code (see below). Assemblers can be defined as binary or plaintext code. Assemblers can be converted into binary code for a specific computer architecture by a binary code generator designed for that specific architecture.
[0380] However, in some implementations, at least a portion of the resulting execution plan is not compiled, but is instead preserved as non-assembly, non-binary code to be interpreted by the interpreter software function ES of the edge computing device ECD.
[0381] As a result of compiling into assembler code, the performance of the execution plan generated by the query optimizer in the general manner described above is further improved. Continuing with the example of SLOG as the internal execution plan language, the SLOG compiler translates the SLOG stream generator of the above type into an assembly language called SLAP (SLog Assembly Program).
[0382] Generally, machine-independent assembly languages can be used as assembler languages. SLAP is an example of such machine-independent assembly languages. When the execution plan is compiled into such an assembly language, further code generation optimizations can be applied to improve execution speed and minimize code size, such as removing unnecessary jumps, optimizing register allocation, peephole optimization, and dead code elimination.
[0383] In some implementations, where no binary code generator is implemented for some computer architecture, or if some binary code is generated for a certain architecture (such as OSX), an abstract code interpreter (such as SLAP) interpreter for interpreting software IS can be used to interpret the assembler.
[0384] In some implementations, the internal execution plan language is a type that the interpreter functions of the edge computing device (ECD) can directly interpret when the internal execution plan language statements are not compiled. For example, this is the case in the exemplary system 100 described herein, which uses SLOG as the internal execution plan language and SLAP as the assembler language.
[0385] Therefore, the execution plan defined using the internal execution plan language does not need to be fully compiled into assembly language, as it can also be directly interpreted by the interpreted software function ES. Thus, the compiler can generate (e.g., SLAP) assembly language code snippets for (e.g., SLOG) expressions that it can compile (e.g., compile to SLAP), while the interpreted software function ES processes the remaining (e.g., SLOG) code that cannot be further compiled or whose compilation is deemed unworthy of interpretation.
[0386] For example, the compiler can be configured not to compile: objects whose object type cannot be statically determined; and / or operations that the compiler does not support, such as string manipulation, iteration on packages and streams, data transformations, etc., depending on the situation. Generally, costly operations that are not supported by the compiled machine instructions (such as FFT (Fast Fourier Transform), image recognition, and advanced numerical array computations) are not worthwhile. Therefore, such operators are instead implemented as interpreted calls to regular external functions.
[0387] Continuing with the example primes above, the SLOG code with the SLAP code snippet is shown below:
[0388] primes-+(Integer n, Integer m)<-
[0389] locals (Bag of Integer v6)
[0390] call(slap--+, <code1>, n, m)
[0391] and funcall(makebag,(lambda-+(Integer m1,Integer factor)<-
[0392] call(slap--+, <code2>, m1, factor)), m, v6)
[0393] and funcall(notany(v6))
[0394] Because the SLOG code in the body of the inner anonymous (lambda) stream generator only contains defined types (integers and real numbers) and only arithmetic operations, it can be fully compiled into... <code2>The SLAP code in the file. <code1>The SLAP code implementation generates a loop of integers between 2 and n.
[0395] In subsequent steps, the binary code assembler portion of the interpreter software function ES of the central server CS can transcribe at least a portion of the assembler into platform-specific binary code. This transcription process can consist entirely of simple line-by-line translation, which converts the assembly statements of the resulting platform-independent assembler into corresponding platform-specific binary statements for the specific hardware / operating system platform on which the execution plan is intended to be executed.
[0396] Therefore, the binary code assembler (such as the SLAP assembler) of the interpreter function ES of the central server CS can generate binary code (statements) for one or more target architectures supported by System 100, thus generating code packages for different architectures. Both platform-specific code packages and platform-independent (SLAP) code can be stored in the database of the central server CS for later reuse. To generate platform-specific binary code for different types of edge computing devices (ECDs), a specific binary assembler (such as a specific SLAP binary assembler) can be made available for each supported target architecture. The code package can also contain several types of assembled binary machine code, which is then invoked (initiated by) at runtime from the interpreter on the target machine (the edge computing device ECD under consideration).
[0397] Machine-specific binary code can be assembled into the target machine architecture during runtime and stored in a database on the central server (CS), thus directly relating to when the code is (to be) transmitted to the edge computing device (ECD) for query processing. If the code executes in the central server (CS), it can be immediately assembled into the central server (CS)'s native machine code.
[0398] As an alternative to binary code, binary code assemblers can also generate platform-specific assembler code. In this case, such assembly code can be efficiently bytecode interpreted in edge computing devices (ECDs) whose machine architecture does not allow just-in-time compilation. This is the case for OSX, for example.
[0399] Continuing with the SLOG / SLAP example, the following are exemplary statements using different platform-specific formats (except for the first line, which shows the corresponding platform-independent SLAP statements):
[0400] dadd D2,D3; SLAP assembly instruction
[0401] addsd xmm2,xmm3; Intel x86-64 assembly instruction
[0402] fadd D2,D3; ARM Aarch64 assembly instruction
[0403] A binary code assembler can allocate the available registers of the target architecture, place the remaining operands on the stack (SLAP typically has more registers than the physical device), and transcribe each instruction one-to-one.
[0404] Any part of the execution plan that is not compiled into assembly code and / or not transcribed into platform-specific assembly / binary code can be interpreted by the edge computing device ECD that interprets the software function ES. Conversely, any compiled and transcribed binary code parts of the execution plan can be executed directly by the processor of the edge computing device ECD.
[0405] When the entire execution plan can be compiled into the assembler, there's no need for an execution plan interpreter on the edge computing device ECD, saving storage space (both RAM and flash) on the ECD. In such cases, the central server CS can determine whether a particular query definition requires an execution plan language interpreter function to be installed on the edge computing device ECD to run a given query or call a function, and if so, initiate the automatic download and installation of such a function on the ECD in question (if it's not already installed).
[0406] In some implementations, at least one (such as several) edge computing device (ECD) in system 100 is capable of interpreting internal execution plan language code and executing at least one of compiled assembly code and transcribed binary code. Furthermore, in some implementations, at least a portion of the execution plan definition is compiled, while at least a portion of the execution plan is not compiled.
[0407] In subsequent steps, at least one edge computing device in the edge computing device ECD can subsequently receive, via its digital communication interface IF, at least partially compiled execution plans generated as described above.
[0408] In a subsequent step, the interpretation software function ES of the considered edge computing device is executed on the CPU of at least one edge computing device in the edge computing device ECD, running both compiled and uncompiled portions of the at least partially compiled execution plan, as appropriate, to produce a first result of the at least partially compiled execution plan. Interpretation includes performing the above calculations based on measured sensor S data.
[0409] As clearly seen above, this operation can be performed: on a single edge computing device (ECD); in parallel on several edge computing device (ECD) systems; and / or in a multi-level set of edge computing device (ECD) systems that report to each other according to a tree structure of such ECDs. Typically, this will be defined directly in the parsed query.
[0410] As illustrated and explained above, any uncompiled portion of the execution plan is executed by the interpreter software function ES of the edge computing device ECD interpreting the considered portion of the execution plan; while any compiled portion of the execution plan is executed by the interpreter software function ES of the edge computing device ECD directly initiating its execution through the CPU of the edge computing device ECD.
[0411] In subsequent steps, the edge computing device ECD transmits the results of the execution plan (first result, followed by the endless stream of objects as described above) to the central server CS via its digital communication interface IF.
[0412] In subsequent steps, the central server (CS) responds to receiving the result from the edge computing device (ECD) by transmitting the query result, or a secondary result calculated based on the result received from the edge computing device (ECD), to the considered querying party. This thus constitutes a response to the submitted query.
[0413] In an alternative implementation, the edge computing device (ECD) can transmit results to different entities, such as different central servers (CS) or different edge computing devices, as can be specified (or implied) as part of the query. Correspondingly, when the central server (CS) receives the results, it can provide the results (or secondary results) to entities different from the query entity.
[0414] System 100 can be configured such that any system peer-to-peer edge computing device (ECD) and / or central server (CS) receiving a query from any entity can request immediate and complete query processing of such received queries by its parent central server (CS). The parent central server (CS) then performs the steps described above in response to such a request to generate an execution plan that is at least partially compiled and sent back to the requesting child entity. This transmitted execution plan may subsequently contain only platform-specific binary code (and any uncompiled portions of the execution plan) specific to the requesting child entity's hardware and / or operating system. It may also contain platform-independent binary code as described above. The parent central server (CS) can be configured to store both the compiled and uncompiled portions of the execution plan in its database for later use by edge computing devices (ECDs) with similar or different configurations.
[0415] The system 100 of this invention can typically be designed to minimize unnecessary work and keep the thin engine (edge computing device ECD) as simple as possible. This can be achieved while ensuring the full functionality of the entire engine. Furthermore, the thin engine can be operated using a variety of functions by transmitting queries for optimization and compilation to some other peers in the consortium, generating execution plans and / or binary code for execution on the peers.
[0416] Therefore, the internal language compiler can be configured to compile only, or at least the arithmetic portion of the internal language execution plan, leaving the remaining predicates as is for the interpreted software function ES to be interpreted at the edge computing device ECD. Thus, the internal language compiler can be configured to replace the compiled predicates with a CODE object for each pair of adjacent predicates.
[0417] In some implementations, the internal language compiler can be configured to compile the execution plan into a platform-independent assembler, and then store the assembler in a database such as a central server (CS). Then, when an additional edge computing device (ECD) is invoked to process the same query or a portion of a query, the stored assembler can be rapidly transcribed by the same or different central server (CS) into binary code specific to any hardware and / or operating system platform of the additional edge computing device (ECD). The stored assembler (such as SLAP code) may be stored in a shared manner within System 100, such as being shared between peer central server (CS) for reuse at the System 100 level. This avoids the internal language compiler having to compile the same execution plan multiple times for different machine architectures; transcription is typically much faster than compilation. The platform-independent assembler can also be interpreted on one or more edge computing devices (ECDs), as described above.
[0418] Furthermore, using the above method, the central server CS can be configured to transcribe platform-independent assemblers into platform-specific machine code only when necessary, and then retain (cache) the resulting platform-specific binary code for later reuse.
[0419] Since transcription is typically a very inexpensive operation, it can be configured to be executed on the fly, such as even when the execution plan is being sent to a peer-to-peer central server (CS) or an edge computing device (ECD) as part of it.
[0420] Furthermore, platform-independent assembly languages such as SLAP code can be easily made portable and efficient in terms of code size and interpretation time, and can be easily shared between peer-to-peer central servers (CS) for reuse and deployment at various edge computing devices (ECDs).
[0421] In cases where platform-independent assemblers are interpreted by the interpreter software function ES at the edge computing device ECD (which is also possible in some implementations), such interpretation is generally faster than the interpretation of the internal execution plan language, at least when it represents several predicates.
[0422] In some implementations, the assembler contains one or more special instructions arranged to invoke the interpreter when called, targeting internal language code that cannot be or will not be converted into binary code. For example, the interpreter may need to be invoked when processing a new stream. This allows for the elimination of some or all of the interpreter used to execute the language on small edge computing devices (ECDs).
[0423] Therefore, the interpreted software function ES of each edge computing device ECD can be configured to interpret the internal execution plan language and / or assembly language, depending on the situation.
[0424] Therefore, a full engine can be provided and process code (corresponding to a query or part of a query) defined in an internal execution plan language, assembly language, and / or platform-specific binary code, and each of these instances can then be processed by its natively installed and executed interpreted software function ES. Conversely, a thin engine can be configured to always receive only the corresponding platform-specific binary code. In the latter case, an interpreter for the execution language is not required.
[0425] All parsed, compiled, optimized, and transcribed code can be cached and / or shared across different central server (CS) instances for reuse. For example, when adding a new target architecture for an edge computing device (ECD), implementing a transcriber backend for that new target architecture is straightforward. Since all code optimizations are done at a higher level (when the execution plan is defined), transcription can be performed centrally and then automatically propagated to all relevant devices, such as on-demand propagation when needed.
[0426] As mentioned, thin engine edge computing devices (ECDs) can be configured to not include query processing algorithms, and their interpreted software functions in Elasticsearch can therefore be undefined and simply execute optimized and compiled execution plans, such as... Figure 10 As shown (the interpreted software function ES then includes only the internal execution plan language interpreter and downstream components, or in some implementations only the downstream components). In the SLOG example discussed above, the internal execution plan language interpreter can be a SLOG interpreter that is available on a thin-engine edge computing device ECD and configured to handle SLOG primitives for which the SLOG compiler cannot generate assembly code. For thin-engine edge computing device ECDs, the execution plan typically includes pointers to binary code segments that are invoked by the internal execution plan language interpreter when encountered in the execution plan; therefore, such thin-engine edge computing device ECDs can also handle such pointer-based binary code included in the interpreted execution plan.
[0427] There may be computer architectures that do not implement a binary code generator (such as the absence of a SLAP binary code generator), or computers that, for platform reasons, do not allow dynamic generation of binary code, for example, in OSX devices. To address this, System 100 may also include an assembler (such as a SLAP) bytecode interpreter, as part of an interpreter software function ES on at least some edge computing devices (ECDs), which interprets assembler instructions on the computer, where the assembler language cannot be transcribed into binary code. This also... Figure 10 As shown in the diagram. In this case, the execution plan may include pointers to code segments represented as platform-independent assembly language instructions, for which the internal execution plan language interpreter may invoke the assembler interpreter that interprets the software function ES.
[0428] The preferred embodiments have been described above. However, it will be apparent to those skilled in the art that many modifications can be made to the disclosed embodiments without departing from the basic spirit of the invention.
[0429] For example, edge computing devices (ECDs) can take many different forms in terms of both hardware and software platforms. Because interpreted software functions (ESs) are easily portable and can potentially operate in a way that is largely independent of their environment details (much like Java code), they can be deployed and operated on any general-purpose programmable hardware connected to computer networks such as the Internet in almost any of the forms described above. The same system 100 can include many different types of edge computing devices (ECDs) without having to specifically consider factors other than the hardware limitations of each individual edge computing device (ECD).
[0430] In some implementations, the device is connected to the computer network only intermittently, in which case the scheduled installation is delayed until the device is connected to the network later.
[0431] In some implementations, the device can only connect to the internet once the (compiled) execution plan is installed on it. The execution plan then runs autonomously without reconnecting to the network.
[0432] The above description outlines many different implementation schemes. Generally, all implementation schemes are freely combinable, provided there are no other specifications and they are compatible. This should occur frequently.
[0433] For example, edge computing devices (EDCs) and / or central servers (CSs) can query each other in a tree structure, as described herein. In such cases, each pair of querying entities and queried entities can act as the corresponding central server (CS) associated with the respective edge computing device (EDC) in terms of parsing, compiling, optimizing, and transcribing the query code, also as described herein. Thus, such queries can occur across multiple layers of devices.
[0434] Therefore, the present invention is not limited to the described embodiments, but may vary within the scope of the appended claims.
Claims
1. A system (100) comprising a plurality of edge computing devices (ECDs) and a first central server (CS), each such edge computing device (ECD) and each such central server (CS) further comprising: Memory (M); Central Processing Unit (CPU); and A digital communication interface (IF) is configured to allow digital communication over a digital communication network (NW). Each edge computing device in the aforementioned edge computing device (ECD) also includes: Sensor (S), The central server (CS) is configured to accept queries from the querying party and via the digital communication interface (IF), the results of which include at least one stream of data objects, the at least one stream being calculated based on data measured by the sensor (S) and provided continuously or intermittently over time and communicated via the digital communication interface (IF). The central server (CS) is characterized in that, in response to receiving the query, it performs the following steps: a) Parse the query to produce a parsed query expression, which serves as a corresponding at least partially declarative, strongly typed, object-oriented, predicate-logic query representation; b) Generate an execution plan corresponding to the parsed query expression, the generation including a logic-based transformation and a cost-based query optimization of the parsed query expression; c) Determine a first portion of the execution plan, wherein the first portion is deemed uncompileable or its compilation is deemed unworthy; and d) Obtaining at least a partially compiled execution plan by compiling the second part of the execution plan, instead of the first part, into corresponding assembly code, wherein the compilation includes determining the expression type in the execution plan, and the compiler determines at compile time that the type is a strongly typed expression. At least one edge computing device in the edge computing device (ECD) is arranged to subsequently receive the at least partially compiled execution plan via the digital communication interface (IF), and run the at least partially compiled execution plan to produce a first result for the at least partially compiled execution plan, and transmit the first result via its digital communication interface (IF). The at least partially compiled execution plan defines the calculations to be performed based on measurements from the sensor (S), and in... The edge computing device (ECD) under consideration is arranged to perform the computation as part of the interpretation.
2. The system (100) of claim 1, wherein the query is defined according to a query language having a predetermined syntax arranged to define a query whose result includes at least one data stream and allows the query to be defined based on a calculation to be performed based on a measurement from the sensor (S).
3. The system (100) according to claim 1 or 2, wherein the query language is an object-oriented query language.
4. The system (100) according to any of the preceding claims, wherein the parsing includes parsing the query into an equivalent abstract syntax tree, such as parsing into an S-expression.
5. The system (100) according to any of the preceding claims, wherein the parsing becomes a representation employing multidirectional external functions and / or relational types.
6. The system (100) of claim 5, wherein the representation can be extended by providing primitives for user-defined predicates, for example, providing multi-directional external functions with different specific implementations based on known / unknown states that depend on the parameters of the external function.
7. The system (100) according to any of the preceding claims, wherein step c includes the central server (CS) transcribing at least a portion of the assembly code into platform-independent and / or platform-specific assembly code.
8. The system (100) according to any of the preceding claims, wherein any uncompiled portion of the execution plan is executed by the interpreter software function (ES) of the edge computing device (ECD) interpreting the considered portion of the execution plan; and any compiled portion of the execution plan is executed by the interpreter software function (ES) of the edge computing device (ECD) directly initiating its execution via the CPU of the edge computing device (ECD).
9. The system (100) according to any of the preceding claims, wherein in step c, only the arithmetic portion or at least the arithmetic portion of the execution plan is compiled.
10. The system (100) according to any of the preceding claims, wherein the central server (CS) is configured to store the platform-independent assembly code for later reuse.
11. The system (100) of claim 10, wherein the central server (CS) is configured to transcribe stored platform-independent assembly code into platform-specific code for different edge computing devices (ECDs).
12. The system (100) according to any of the preceding claims, wherein each sensor (S) and / or edge computing device (ECD) is capable of being referenced using a global namespace or attribute set according to the syntax.
13. The system (100) according to any of the preceding claims, wherein each edge computing device (ECD) is arranged to discard at least some, preferably all, of the measurement data after the measurement data measured by the sensor (S) and / or a piece of information such as the result calculated based on the measurement data has been communicated via the digital communication interface (IF), rather than storing the measurement data in the memory (M).
14. The system (100) according to any of the preceding claims, wherein the system (100) further comprises a distributed or monolithic central server (CS), each of the edge computing devices (ECDs) having a client role relative to the central server, and wherein all communication between the various edge computing devices (ECDs) is performed via the central server (CS).
15. The system (100) according to any of the preceding claims, wherein the interpreted software function (ES) includes an external application programming interface (API) arranged to allow external computer code to call expressions using the query language and / or arranged to allow external computer code to call expressions using the query language.
16. The system (100) according to any of the preceding claims, wherein the interpreter software function (ES) includes machine learning functions arranged to define, train, and execute machine learning models such as neural networks.
17. The system (100) of claim 16, wherein the memory (M) of the edge computing device (ECD) includes machine learning functions for defining and training the query references as interpreted by the interpreted software function (ES) executed on the edge computing device (ECD).
18. The system (100) of claim 16 or 17, wherein the interpreted software function (ES) of the consideration is arranged to train the defined machine learning function based on the result of the query.
19. A method for collecting data in a system (100), the system (100) comprising a plurality of edge computing devices (ECDs) and a first central server (CS), each such computing device (ECD) and each such central server (CS) further comprising: Memory (M); Central Processing Unit (CPU); The method includes a digital communication interface (IF) configured to allow digital communication over a digital communication network (NW), and each edge computing device (ECD) further includes a sensor (S). The method is characterized by the following steps: a) The central server (CS) accepts queries from the querying party and via the digital communication interface (IF), the result of which includes at least one stream of data objects, the at least one stream being calculated based on data measured by the sensor (S) and being provided continuously or intermittently over time and communicated via the digital communication interface (IF); b) The central server (CS), in response to receiving the query, parses the query to generate a parsed query expression as a corresponding at least partially declarative, strongly typed, object-oriented, predicate logic query representation; generates an execution plan corresponding to the parsed query expression, the generation including logic-based transformations and cost-based query optimizations of the parsed query expression, the execution plan defining computations to be performed based on measurements from the sensor (S); determines a first portion of the execution plan, the first portion being deemed uncompileable or unworthy of compilation; and compiles the second portion of the execution plan, instead of the first portion, by converting the second portion of the execution plan into corresponding assembly code, thereby obtaining an at least partially compiled execution plan, the compilation including determining the expression type in the execution plan, the compiler determining at compile time that the type is a strongly typed expression; c) At least one of the edge computing devices (ECDs) receives the at least partially compiled execution plan via the digital communication interface (IF); and d) The software function (ES) of at least one edge computing device in the edge computing device (ECD) is executed on the CPU of the edge computing device (ECD) and runs the at least partially compiled execution plan to produce a first result for the at least partially compiled execution plan, interpreting the execution including the computation.