An assembled computing system and method based on node orchestration

Abstract

The application discloses an assembled computing system and method based on node arrangement, which comprises a data processing layer, an assembled computing service layer and a visual arrangement layer; the data processing layer communicates with the service layer through an internal cache calling interface, comprises a data source layer, a unified modeling module and a model cache layer, and shields heterogeneous data differences through standardized naming specifications; the assembled computing service layer communicates with the arrangement layer through a RESTful interface, contains processor node management, link management, an execution module and an MVEL expression engine, realizes link persistence, multiple reuse and expression analysis execution, and provides a visual interface; a user constructs a computing link by dragging various processor nodes, each node port has a fixed data format, and only the same format ports are allowed to be connected; the node supports general parameter configuration, and specific instances are constructed through different parameter values. The application realizes zero-code construction of a complex data processing link, reduces data access complexity, and improves development efficiency.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an assembly-type computing system and method based on node orchestration. Background Technology

[0002] In fields such as power, energy, and industrial control, data typically originates from multiple heterogeneous systems, including SCADA, EMS, and MIS systems. However, the diverse data structures and inconsistent naming conventions of these systems lead to difficulties in data integration and reuse. Traditional data processing methods mainly include two approaches. The first approach uses traditional programming methods to develop data processing programs. Developers need to write specific code for each business scenario, use SQL statements to query the database, and use programming languages ​​such as Java and Python to implement business logic. The second approach uses ETL tools such as Kettle. While these tools provide visual data flow design pages, which lowers the technical threshold to some extent, they are mainly designed for data warehouse scenarios. Both approaches suffer from drawbacks such as high technical barriers, low development efficiency, poor flexibility, poor reusability, and complex data access. Furthermore, the requirement for specialized programming knowledge makes it inaccessible to non-technical personnel. Writing and debugging code takes a significant amount of time, and code modifications are necessary when requirements change, leading to high maintenance costs. Code reuse is difficult between different projects, and heterogeneous data lacks a unified access method and has weak expression access capabilities.

[0003] To address the issue of weak expression access capabilities, the MVEL expression language was introduced. It is widely used in rule engines and data processing, supporting rich syntax such as attribute access, method calls, conditional judgments, and mathematical operations to meet complex data processing needs. However, using MVEL directly requires a certain level of programming knowledge, which makes the learning cost high for non-technical personnel and limits its widespread application among business users. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a modular computing system and method based on node orchestration, which overcomes the shortcomings of existing technologies such as high technical threshold for data processing, low development efficiency, poor flexibility, and complex data access, and realizes the construction of low-code or even zero-code data processing processes.

[0005] Technical solution: The modular computing system based on node orchestration described in this invention includes:

[0006] The data processing layer includes a data source layer, a unified modeling module, and a model cache layer. The data source layer provides raw data input. The unified modeling module is used to perform unified modeling on the raw data provided by the data source layer and define a standardized data model. The model cache layer is used to store and manage the data modeled by the unified modeling module.

[0007] The modular computing service layer includes a processor node management module, a link management module, an execution module, and an MVEL expression engine. The processor node management module is used to manage and expand processor node types; the link management module is used for link management operations; the execution module is used to execute the computing process according to the orchestrated links and obtain the computing results; the MVEL expression engine is used to parse and execute expression access; the modular computing service layer directly obtains relevant model data from the data processing layer through internal caching.

[0008] The visual orchestration layer includes a visual orchestration interface, processor nodes, computation link construction, and computation result acquisition. The visual orchestration interface is used to display and connect each processor node. The processor nodes perform corresponding logical processing to construct conditional judgment logic. The visual orchestration layer obtains processor node information, link information, and executes computation links and obtains results through a RESTful interface.

[0009] Furthermore, the visual orchestration interface includes a node library panel, a canvas area, and a configuration panel; the node library panel is used to display information about all current processor nodes; the canvas area allows users to build computational chains by dragging and dropping; and the configuration panel allows users to configure parameters and edit MVEL expressions for selected nodes.

[0010] Furthermore, when parsing an expression, the MVEL expression engine first obtains the corresponding data object through the model cache layer, then performs evaluation operations sequentially according to the expression's syntax tree, and finally obtains the result of logical processing by each processor node.

[0011] Furthermore, the visual orchestration interface supports drag-and-drop operations, allowing users to connect nodes by dragging and dropping. When connecting nodes, the visual orchestration interface only displays connectable ports with matching data formats, and blocks or prompts for mismatched ports.

[0012] Furthermore, the processor nodes include condition builder nodes, finder nodes, filter nodes, general calculator nodes, accumulator nodes, average calculator nodes, grouper nodes, paginator nodes, merger nodes, and template generator nodes.

[0013] Furthermore, each processor node has multiple input ports and multiple output ports, each port having a predefined data format type.

[0014] Furthermore, the predefined data format types include model classes, basic data classes, and grouping classes.

[0015] Furthermore, the execution module executes the computational logic of each processor node in sequence: it obtains the input data required by the current node from the model cache layer through the MVEL expression engine, then executes the processing logic of the processor node, and finally passes the calculation result to the downstream node for processing and calculation in sequence.

[0016] An assembly-based computing method based on node orchestration includes the following steps:

[0017] The provided raw data is modeled in a unified manner, a standardized data model is defined, and the data after unified modeling is stored and managed.

[0018] Manage and expand processor node types; select processor nodes through a visual interface and drag and drop to connect them to build computing links; manage computing links and configure the parameters and input / output relationships of each node; each processor node accesses data in the cache through expression parsing and execution.

[0019] The computation process of each node is executed according to the orchestrated link sequence to obtain the computation results. Processor node information, link information, execution of computation links and obtaining results are obtained through RESTful interface.

[0020] Furthermore, the connection of processor nodes follows the port type matching rule, that is, a connection is only allowed when the output port data format of the upstream node is consistent with the input port data format of the downstream node; during the connection process, the visual orchestration interface automatically hides or prompts ports with mismatched data formats.

[0021] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: 1. The unified modeling module eliminates the model inconsistency problem of heterogeneous systems. Users do not need to understand the underlying data structure; they can access any data simply by using standardized naming conventions. 2. Non-technical personnel can build data processing flows through a visual interface without writing code or learning complex expression syntax. Business personnel with simple training can independently build data processing links by simply inputting simple expressions on the required processor nodes and connecting different processor nodes. They can then obtain data through the execution engine to complete complex data processing logic, reducing dependence on the development team and improving development efficiency. 3. Data format security: Through a port type matching mechanism, the format compatibility of data transmission between nodes is ensured. Incompatible connections can be detected and blocked during the orchestration stage, avoiding runtime issues caused by... 4. High processor reuse: Through a parameterized construction mechanism, processor nodes of the same type can generate instances with different processing logic by configuring different parameter values. This eliminates the need to develop new processor nodes for each business scenario, improving processor reuse and development efficiency. 5. Persistent link reuse: Orchestrated computing links can be persistently stored and repeatedly executed. Users can reuse saved links without repeated orchestration, improving data processing efficiency. At the same time, link version management and sharing functions facilitate team collaboration. 6. Zero-code processor extension: Through a visual configuration interface, users can customize new processor nodes and add them to the node library without writing code. This lowers the technical threshold for processor extension, allowing non-technical personnel to flexibly customize processors according to business needs. Attached Figure Description

[0022] Figure 1 This is a diagram showing the overall architecture of the assembled computing engine of the present invention;

[0023] Figure 2 This is a flowchart illustrating the unified modeling and data caching process of the present invention;

[0024] Figure 3 This is a schematic diagram of data access in the model cache layer of the present invention;

[0025] Figure 4 This is a schematic diagram illustrating the working principle of the MVEL expression engine of the present invention.

[0026] Figure 5 A schematic diagram of the visual layout interface of this invention;

[0027] Figure 6 This is a diagram illustrating the core attributes of the processor node of the present invention;

[0028] Figure 7 The configured JSON template. Detailed Implementation

[0029] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0030] like Figure 1 As shown, the modular computing system based on node orchestration described in this invention adopts a three-layer architecture design, including a visual orchestration layer, a modular computing service layer, and a data processing layer. The layers communicate with each other through standardized interfaces.

[0031] The data processing layer is responsible for the access, standardized modeling, and caching of heterogeneous data, including the data source layer, the unified modeling module, and the model cache layer. The data source layer provides raw data input; the unified modeling module is used to perform unified modeling on the raw data provided by the data source layer and define a standardized data model; the model cache layer is used to store and manage the data modeled by the unified modeling module, enabling unified access to the data.

[0032] Data source layer: Supports access to various heterogeneous data sources, including relational databases, NoSQL databases, real-time data streams, and API interfaces. It shields the technical differences between different data sources through a unified data access interface.

[0033] like Figure 2 As shown, the unified modeling module is responsible for extracting data from various data sources (relational databases, NoSQL databases, real-time data streams, API interfaces, etc.) and performing standardized modeling according to predefined model specifications. The modeling process automatically handles operations such as field mapping, data type conversion, and default value filling, unifying heterogeneous data from different systems into a standardized data model structure. After modeling is complete, the data is loaded into the model cache layer. The model cache layer uses memory caching technology, supporting real-time data updates and batch refreshes to ensure data timeliness.

[0034] The unified modeling module defines conventional naming rules, such as: St represents a power plant, Voltage represents the voltage level, Acline represents an AC line, p represents active power, q represents reactive power, name represents the name, type represents the type, value represents the value, and status represents the status.

[0035] like Figure 3 As shown, the model cache layer provides a unified data access interface for each processor node. Each processor node accesses data in the model cache layer through MVEL expressions. The model cache layer locates the corresponding data object based on the field path in the expression and returns the result. This unified data access method allows each processor node to obtain the required data simply by using standardized expressions, without needing to concern itself with the storage details of the underlying data.

[0036] The modular computing service layer is the core middleware connecting the visualization orchestration layer and the data processing layer. It communicates with the visualization orchestration layer via a RESTful interface and with the data processing layer through internal cached calls, including:

[0037] Processor Node Management Module: Responsible for saving, modifying, and querying processor nodes. It manages processor node type definitions, port configurations, parameter templates, and MVEL expression templates. It supports the registration and management of predefined and user-defined processor nodes, manages and extends processor node types, and allows custom nodes to access the model cache via MVEL expressions.

[0038] The Link Management module manages links, including saving, modifying, and querying them. It persistently stores user-orchestrated computational links in a link library, supporting multiple loading and repeated execution of links. Users can directly call saved links for data processing without re-orchestrating. The module also supports link version management, import / export, and sharing. Furthermore, it describes the type, parameter configuration, and location information of each processor node using a `nodes` array and the connection relationships between nodes using an `edges` array, storing these in a JSON serialized format in the link library. The frontend can then reconstruct the link graph by parsing the JSON data to enable multiple loading and repeated execution of links.

[0039] The execution module executes the computation process according to the orchestrated chain and obtains the results. It starts when the user clicks the run button, performing topological sorting according to the directed acyclic graph structure of the computation chain to determine the execution order of each node. The execution engine executes the computation logic of each node sequentially: first, it retrieves the required input data from the model cache layer through the MVEL expression engine; then, it executes the node's processing logic; and finally, it passes the computation result to the downstream node. For cases where there are branches in the chain, the execution engine selects the corresponding execution path based on the condition constructor's judgment result. The execution module also supports breakpoint debugging and single-step execution functions, facilitating the troubleshooting of expression execution problems.

[0040] like Figure 4 As shown, the MVEL expression engine integrates the MVEL 2.x expression engine, responsible for expression parsing and data retrieval. It accesses data in the model cache layer through standardized naming conventions and supports expression functionalities such as field access, nested attribute traversal, conditional judgment, mathematical operations, function calls, and collection operations. The MVEL expression engine is tightly integrated with the model cache layer, allowing users to access cached model data simply by writing MVEL expressions.

[0041] The MVEL expression engine supports various expression types, including: field access (e.g., St.name to retrieve the substation name), nested access (e.g., Acline.St.Voltage to retrieve the voltage level of the substation to which the AC line belongs), conditional judgments (e.g., St.votageType≥220 to determine the substation voltage level), and mathematical operations, such as... Calculate the average active power of two lines, function calls such as getVoltageNameById(id) to get the corresponding voltage level name, and collection operations such as AclineList.size() to get the number of AC lines.

[0042] When parsing an expression, the MVEL expression engine first obtains the corresponding data object through the model cache layer, then performs evaluation operations sequentially according to the expression's syntax tree, and finally returns the calculation result.

[0043] The visual orchestration layer includes: a visual orchestration interface, processor nodes, computation chain construction, and computation result acquisition.

[0044] Visual orchestration interface: Provides a graphical user interface, including three sub-components: a node library panel, a canvas area, and a configuration panel. For example... Figure 5 As shown, the left side of the interface is the node library panel, displaying all available processor node types; the middle is the canvas area; the visual orchestration interface supports drag-and-drop operations, allowing users to connect nodes by dragging and dropping; when connecting nodes, the visual orchestration interface only displays connectable ports with matching data formats, masking or prompting for mismatched ports. Users can drag and drop nodes from the node library onto the canvas; the right side is the configuration panel, displaying the configurable parameters of a selected node. Users connect the input and output ports of each node through connection operations to build a complete computing chain.

[0045] The visual orchestration interface also offers processor customization, allowing users to create new processor nodes through visual configuration. Users can define the processor's name, input / output ports and their data formats, processing logic (supporting definition via MVEL expressions or visual logic combinations), and general parameter configuration items. Once a custom processor node is defined, the system automatically saves it to the node library. Users can then find the custom processor in the library and drag and drop it onto the canvas for orchestration, just like using predefined processors. This feature enables non-technical users to flexibly expand processor types according to business needs without writing code.

[0046] Processor Nodes: A variety of predefined processor node types are provided, including condition builder, finder, filter, general calculator, accumulator, average calculator, grouper, paginator, merger, and template generator. Each processor node has multiple input and output ports, each corresponding to a fixed data format. Only ports with the same data format are allowed to connect to each other to ensure format compatibility during data transfer between nodes. Each processor node also has general parameter configuration options, allowing users to construct specific processor instances by configuring different parameter values.

[0047] Each processor node supports multiple input ports and multiple output ports. Each port has a predefined data format type, including model classes, basic data classes, and grouping classes. During node connection, the system automatically verifies the compatibility of the port data formats. A connection is only established if the data format of the upstream node's output port matches the data format of the downstream node's input port. This port type matching mechanism ensures format compatibility of data transmitted between nodes, avoiding processing errors caused by data format mismatches.

[0048] like Figure 6 As shown, each processor node provides common parameter configuration items. Users can configure parameter values ​​according to specific business needs, thereby constructing instances of the same type of processor node with different processing logic. The core attributes of each processor node are described below. Each processor node has the following attributes:

[0049] Unique Identifier (nodeCode): Each processor node instance has a globally unique code used to uniquely identify the node in the link;

[0050] Node Type (nodeType): Each processor node has a unique type identifier used to distinguish different types of processors, such as "Conditioner" for condition constructor, "Calculator" for calculator, and "Classifier" for grouper, etc.

[0051] Node description: Each processor node has a descriptive name to help users understand the purpose of the node;

[0052] Input Ports: Each processor node has zero or more input ports, each with a port code (portCode), port name (portName), and data type (dataType).

[0053] Output Ports: Each processor node has zero or more output ports, and each port also has a port code (portCode), port name (portName), and data type (dataType).

[0054] Parameter configuration (params): Each processor node has configurable parameter items, and users can customize the specific behavior of the node by configuring parameter values.

[0055] The following explains how different processor nodes work:

[0056] The Condition Constructor node is used to build the internal model conditions required by the finder. Users input MVEL condition expressions through the configuration panel. At runtime, the Condition Constructor node performs condition checks on the input data and distributes the data to different output ports based on the results. For example, a user can configure the condition expression as St.Voltage ≥220. The Condition Constructor node will output data from substations with voltage levels greater than or equal to 220kV to the "Condition Met" port, and output other data to the "Condition Not Met" port.

[0057] The finder node is used to retrieve target model data from the model cache layer. Users can find the corresponding device model by specifying the name of the device model they want to query, and can also specify the query range to filter the device models. The finder traverses the matching data objects in the model cache and returns the results. Users can configure the search expression to Acti, and the finder will find the DC line model from the model cache.

[0058] The filter node is used to filter data based on set conditions. The filter node provides a condition expression input box and a threshold parameter input box. After the user sets the filtering conditions, the filter evaluates each input data item during runtime, only sending data that meets the conditions to the output port. For example, if the user sets the voltage level threshold parameter to 220kV, this filter instance will automatically filter out data from devices with voltage levels no higher than 220kV, retaining only data from devices with voltage levels higher than 220kV.

[0059] The general-purpose calculator node is used to perform general MVEL expression calculations. Users can input any MVEL mathematical expression through the configuration panel, supporting mathematical operations such as addition, subtraction, multiplication, division, and modulo (%). The general-purpose calculator node obtains the required input data from the model cache layer, performs the expression calculation, and outputs the result. Users can configure the calculation expression to Bus.v / Bus.upWarning, and the general-purpose calculator will calculate the bus voltage over-limit rate.

[0060] The accumulator node is used to sum specified numerical fields in the input dataset. The user configures the field name to be accumulated. The accumulator iterates through all records in the input dataset, sums the values ​​of that field, and finally outputs the accumulated result. The user-configured expression is `Aclineend.p`, and the general-purpose calculator will calculate the active power imbalance of the AC line.

[0061] The Average Calculator node is used to calculate the arithmetic mean of a specified numerical field in the input dataset. The user configures the name of the field for which the average should be calculated. The Average Calculator iterates through all records in the input dataset, calculates the average of that field, and outputs the result. If the user configures the calculated field as `Acline.voltageType`, the Average Calculator will calculate the average voltage level for all AC lines.

[0062] The grouper node is used to group data according to a specified field. The user configures the grouping field, and the grouper divides the input dataset into multiple groups based on the different values ​​of that field. Each group is output as an independent dataset to downstream nodes. The grouper supports multi-level grouping, meaning it can first group by a first field, and then further group by a second field within each group; for example, the user can first group by substation voltage level, and then further group by equipment type within each voltage level group.

[0063] The paginator node is used to perform pagination on data. Users configure the number of records per page and the current page number. The paginator extracts a subset of the input dataset according to the specified page number and outputs pagination metadata such as the total number of records, total number of pages, and current page number. The paginator is suitable for large datasets, avoiding loading all data at once and improving system response speed.

[0064] Merger nodes are used to merge data from multiple data sources. A merger node has multiple input ports that can receive data streams from different upstream nodes and merge multiple data streams into a single output according to a preset merging strategy (such as concatenating by field, associating by key values, etc.). For example, a user can connect the output ports of multiple calculators to two input ports of the merger, and the merger will combine the calculated data into a single result set.

[0065] The template generator node is used to generate structured output content based on a JSON template. Users define the JSON template through a configuration panel, which supports nested field access and function calls. The template uses key-value pairs, where the key is the output field name and the value is an MVEL expression or function call, which can be configured as follows: Figure 7 The JSON template shown.

[0066] The template generator node parses the JSON template at runtime, executes the corresponding MVEL expression or function call for each field, supports multi-level nested field access (such as Acline.StartSt.name), and assembles the results into a JSON object for output. The frontend can flexibly configure template fields according to business needs, and customize the output format without modifying the backend code.

[0067] Computational Link Construction: Users select the desired nodes from the processor nodes, drag and drop them onto the canvas area through the visual orchestration interface, and connect the ports of each node according to the data processing logic to build a complete computing link.

[0068] Calculation result acquisition: After the calculation chain is completed, the calculation results are displayed through a visual orchestration interface, supporting result viewing, export, and persistent storage.

[0069] The visual orchestration layer and the modular computing service layer communicate through a RESTful interface. The visual orchestration layer calls the processor node management, link management and execution modules of the modular computing service layer through the RESTful interface, and the modular computing service layer returns the execution results and status information through the RESTful interface.

[0070] This invention provides an assembly-based computing method based on node orchestration, comprising the following steps:

[0071] The provided raw data is modeled in a unified manner, a standardized data model is defined, and the data after unified modeling is stored and managed.

[0072] Manage and expand processor node types; select processor nodes through a visual interface and drag and drop to connect them to build computing links; manage computing links and configure the parameters and input / output relationships of each node; each processor node accesses data in the cache through expression parsing and execution.

[0073] The computation process of each node is executed according to the orchestrated link sequence to obtain the computation results. Processor node information, link information, execution of computation links and obtaining results are obtained through RESTful interface.

[0074] The connection of processor nodes follows the port type matching rule, that is, a connection is only allowed when the output port data format of the upstream node is consistent with the input port data format of the downstream node; during the connection process, the visual orchestration interface automatically hides or prompts ports with mismatched data formats.

Claims

1. A modular computing system based on node orchestration, characterized in that, include: The data processing layer includes the data source layer, the unified modeling module, and the model caching layer; The data source layer provides the raw data input; The unified modeling module is used to perform unified modeling on the raw data provided by the data source layer and define a standardized data model; the model cache layer is used to store and manage the data modeled by the unified modeling module. The modular computing service layer includes a processor node management module, a link management module, an execution module, and an MVEL expression engine; The processor node management module is used to manage and expand processor node types; the link management module is used for link management operations. The execution module is used to execute the computation process according to the orchestrated links and obtain the computation results; The MVEL expression engine is used to parse and execute expression access; The modular computing service layer obtains relevant model data directly from the data processing layer through an internal cache; The visual orchestration layer includes a visual orchestration interface, processor nodes, computation link construction, and computation result acquisition; the visual orchestration interface is used to display and connect the various processor nodes. The processor node performs the corresponding logical processing to construct the condition judgment logic; The visual orchestration layer obtains processor node information, link information, and execution computation links through RESTful interfaces and retrieves the results.

2. The modular computing system according to claim 1, characterized in that, The visual orchestration interface includes a node library panel, a canvas area, and a configuration panel. The node library panel displays information about all current processor nodes. The canvas area allows users to build computational chains by dragging and dropping. The configuration panel allows users to configure parameters and edit MVEL expressions for selected nodes.

3. The modular computing system according to claim 1, characterized in that, When parsing an expression, the MVEL expression engine first obtains the corresponding data object through the model cache layer, then performs evaluation operations sequentially according to the expression's syntax tree, and finally obtains the result of the logical processing of each processor node.

4. The modular computing system according to claim 1, characterized in that, The visual orchestration interface supports drag-and-drop operations, allowing users to connect nodes by dragging and dropping. When connecting nodes, the visual orchestration interface only displays connectable ports with matching data formats, and blocks or prompts for mismatched ports.

5. The modular computing system according to claim 1, characterized in that, Processor nodes include condition builder nodes, finder nodes, filter nodes, general calculator nodes, accumulator nodes, average calculator nodes, grouper nodes, paginator nodes, merger nodes, and template generator nodes.

6. The modular computing system according to claim 1, characterized in that, Each processor node has multiple input ports and multiple output ports, and each port has a predefined data format type.

7. The modular computing system according to claim 1, characterized in that, Predefined data format types include model classes, basic data classes, and grouping classes.

8. The modular computing system according to claim 1, characterized in that, The execution module executes the computational logic of each processor node in sequence: it obtains the input data required by the current node from the model cache layer through the MVEL expression engine, then executes the processing logic of the processor node, and finally passes the calculation result to the downstream node for processing and calculation in sequence.

9. A node-arrangement-based assembly-type computing method, characterized in that, Includes the following steps: The provided raw data is modeled in a unified manner, a standardized data model is defined, and the data after unified modeling is stored and managed. Manage and expand processor node types; select processor nodes through a visual interface and drag and drop to connect them to build computing links; manage computing links and configure the parameters and input / output relationships of each node; each processor node accesses data in the cache through expression parsing and execution. The computation process of each node is executed according to the orchestrated link sequence to obtain the computation results. Processor node information, link information, execution of computation links and obtaining results are obtained through RESTful interface.

10. The assembly-type calculation method according to claim 9, characterized in that, Processor node connections follow port type matching rules, meaning a connection is only allowed when the output port data format of the upstream node matches the input port data format of the downstream node. During the connection process, the visual orchestration interface automatically hides or prompts for ports with mismatched data formats.

Images