A workflow visualization, orchestration, and execution method based on structured data template serialization

By using a unified structured data model and differentiated rendering technology, the complexity of data transformation and identifier conflicts in process automation workflows are resolved, thereby improving the maintainability of the system and the user experience.

CN123115209APending Publication Date: 2026-11-17JIAZHIYUN BIG DATA TECHNOLOGY (SHANGHAI) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611141071.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-30
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies for visual orchestration of process automation workflows suffer from problems such as high data transformation complexity, high coupling between visual styles and node types, and identifier conflicts, leading to increased system complexity and inconsistency.

Method used

A unified structured data model is used to describe the visual presentation, storage format and execution logic of the workflow. Differentiated rendering driven by node type and cubic Bézier curve edge rendering are used, combined with batch identifier remapping to solve the loose coupling management of visual style and node type and identifier conflicts in template installation.

Benefits of technology

It simplifies data transformation in automated workflows, loosely couples visual styles and node types, and eliminates identifier conflicts, thereby improving system maintainability and user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123115209A_ABST
    Figure CN123115209A_ABST
Patent Text Reader

Abstract

This invention discloses a workflow visualization orchestration and execution method based on structured data template serialization, belonging to the field of process automation technology. The method defines a unified structured data format for node and edge objects. Nodes are drawn as rhombuses based on conditional judgment types, large rounded rectangles for start / end types, and small rounded rectangles for other types. Edges are rendered using cubic Bézier curves with the vertical inflection point coordinates between the bottom of the source node and the top of the target node as control points. The node array and edge array are stored as two independent fields in a single database record of the workflow through JSON serialization. During template installation, the template record is deserialized, and all nodes and edges are traversed to regenerate unique identifiers. The source and target references in the edges are replaced using a mapping table from old identifiers to new identifiers, ensuring consistent reference relationships before creating a new workflow record. This invention drives the entire chain operation from canvas rendering and single-table persistence to template installation through a unified structured data model, eliminating data format conversions between stages and achieving batch remapping of node and edge identifiers during template installation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of process automation technology, specifically relating to a workflow visualization orchestration and execution method, system, terminal device, and computer-readable storage medium based on structured data template serialization. Background Technology

[0002] Visual orchestration of automated workflows typically involves three stages: users drag and drop nodes onto a canvas, connect nodes to determine their execution order, and configure parameters for each node via a properties panel. The data generated in these three stages needs to be persistently stored for later loading, editing, and execution. Existing solutions use different data representations for each stage—canvas rendering uses screen coordinates and graphical style objects, database storage uses relational tables and foreign key relationships, and the workflow execution engine uses an in-memory object-oriented model. The conversion between these three representations increases system complexity and can easily introduce data inconsistencies during the conversion process. Furthermore, in template marketplace scenarios, users need to "install" publicly available workflows created by others into their own accounts as new workflows. Existing solutions require copying multiple related records (workflow table, node table, edge table, parameter table, etc.) at the database level, involving numerous steps and making it easy to miss related data. Moreover, after installation, the node and edge identifiers in the new workflow are completely identical to the original identifiers in the template. When two users simultaneously use the same template and attempt to perform operations such as exporting, identifier conflicts may occur. In existing solutions, the binding relationship between the visual style of nodes and connections on the canvas and the node type is usually hard-coded in each node rendering function. Adding a new node type requires modifying multiple rendering logic branches simultaneously. Regarding edge rendering, existing solutions mostly use straight lines or simple quadratic Bézier curves for connection. When the vertical distance between nodes is large, the lines will be steep and vertical, resulting in a visually unsmooth appearance. Summary of the Invention

[0003] The technical problems to be solved by this invention are: in the field of process automation, how to use a unified structured data model to describe the visual presentation, storage form and execution logic of workflow, and eliminate the data conversion gap between each link; how to solve the loose coupling management of visual style and node type through node type-driven differentiated rendering; and how to solve the identifier conflict problem after template installation through batch identifier remapping. To address the aforementioned technical problems, this invention provides a workflow visualization orchestration and execution method based on structured data template serialization. This method uses a unified structured data format for node and edge objects throughout the entire process of canvas rendering, database persistence, and template copying, allowing each layer to directly use the same data structure without format conversion. First, two core structured data formats are defined. Node objects contain id (a unique string identifier), type (one of 22 predefined types), label (a Chinese string description label), x and y (two-dimensional numeric coordinates), and config (Record).<string, unknown> (Dictionary type configuration parameters). The edge object contains id (a unique string identifier), source (a string identifier for the source node), target (a string identifier for the target node), and type (four enumeration values: default, condition_true, condition_false, and error). Second, the canvas rendering module directly consumes the node and edge arrays, determining the drawing method based on the node object's `type` field. Conditional node types are drawn as a rhombus with an 80-pixel diagonal—its four vertices are located 40 pixels directly above, to the right, below, and to the left of the center. Start and end node types are drawn as large rounded rectangles with a 32-pixel radius. The remaining 20 types are drawn as small rounded rectangles with a 12-pixel radius. Default colors for different node types are centrally managed through a color map—this map uses the `NodeType` enumeration value as the key and the HEX color string as the value. When a new node type needs to be added, simply add an entry to the enumeration list, add the corresponding color value to the color map, and add the rounded radius value to the size map; the rendering logic itself requires no modification. Third, the edge rendering uses a cubic Bézier curve with the vertical inflection point's ordinate as the control point's ordinate. The starting point is the bottom center of the source node, the vertical inflection point's ordinate is the arithmetic mean of the source node's bottom ordinate and the target node's top ordinate, and the ending point is the top center of the target node. The four control points of the cubic Bézier curve are set as follows: P0 = starting point, P1 = (starting point.x, inflection point.y), P2 = (ending point.x, inflection point.y), P3 = ending point. Because the ordinates of two control points are equal, the curve exhibits a natural S-shaped arc transition at the vertical bend rather than a sharp angle. Fourth, in the data persistence step, the node array and edge array are each serialized into strings using JSON.stringify(), and written to the database as a single record along with the two TEXT fields, nodes_json and edges_json, and the workflow metadata fields. The complete workflow requires only one database record. Fifth, the template installation operation is a key innovation of this invention: After reading records from the template table and deserializing them, the node array is traversed, and for each node, `crypto.randomUUID()` is called to generate a new identifier to replace the original `id`, establishing an old-new mapping table of "old identifier → new identifier". Similarly, the edge array is traversed, generating a new identifier for each edge to replace the original `id`, and then the old-new mapping table is used to find the corresponding new identifier to replace the `source` field based on the original value of the edge's `source`, and to find the corresponding new identifier to replace the `target` field based on the original value of the `target`. Finally, the replaced node array and edge array are serialized and written to a new workflow record. This step ensures that all internal references in the installed workflow are completely self-consistent and do not conflict with any identifiers in the original template. This invention utilizes a unified structured data model to ensure that canvas rendering, database storage, and template installation all operate on the same set of data, eliminating the complexity and inconsistency risks associated with format conversion. Differentiated rhombus / rounded rectangle rendering driven by node types enables loose coupling between visual style and node type management. The use of cubic Bézier curves with vertical inflection point ordinates for control point design creates an immersive edge rendering experience. Batch identifier remapping and the integration of old and new mapping tables ensure a complete and consistent migration of multi-object reference relationships during template installation. Attached Figure Description Figure 1 This is the overall flowchart of the workflow visualization orchestration and execution method based on structured data template serialization in this embodiment of the invention. The meanings of the markings in the diagram are as follows: 101- Define the structured data format for node and edge objects. 102- Render nodes of different types (rhombus, large rounded rectangle, or small rounded rectangle) according to their type. 103- Render directed edges using a cubic Bézier curve with the vertical inflection point from the bottom of the source node to the top of the target node as the control point, and draw arrows. 104- Users add and move nodes by dragging and dropping, and create edges by connecting them. 105- Serialize the node and edge arrays into JSON strings and store them as two fields in the database workflow table. 106- Template installation operation: Read the template record and deserialize the node and edge arrays. 107- Traverse the node and edge arrays, regenerate identifiers for each object, establish an old-new mapping table, and replace edge references. 108- Create a new workflow record and update the template download count. Detailed Implementation To make the objectives, technical solutions, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings. like Figure 1As shown in mark 101, the system's data model layer defines two core structured data types. The NodeType enumeration contains 22 enumeration values: start, end, open_app, click, input_text, key_press, scroll, ai_judge, condition, loop, api_call, tts_speak, send_message, wait, screenshot, clipboard, file_operation, voice_input, ring_gesture, sub_workflow, data_extract, and web_request. In the WorkflowNode interface, id is a string, type is NodeType, label is a string, x and y are numbers, and config is Record.<string, unknown> In the WorkflowEdge interface, id is a string, source and target are strings, label is an optional string, and type is EdgeType (default / condition_true / condition_false / error). like Figure 1 As shown in mark 102, after receiving the node array, the node drawing function first looks up the color map table according to the node type (e.g., start takes green #52c41a, condition takes orange #fa8c16, end takes red #f5222d, click takes blue #1677ff), and fills with the corresponding color at 30% opacity and strokes with a 1.5-pixel line width. For each node, save() is used to save the canvas state, a shadow is set, and then the corresponding geometric drawing method is called. Conditional nodes: Draw a rhombus path (x+40,y)→(x+80,y+40)→(x+40,y+80)→(x,y+40)→closed, fill and stroke. Start / end nodes: Draw a rounded rectangle (rounded corner radius 32 pixels), the rectangular area is from (x,y) to (x+180,y+64), and the four corners are rounded using the Canvas arcTo method. Other types of nodes: Draw a rounded rectangle (rounded corner radius 12 pixels). After drawing, remove the shadow effect and draw the icon and text label on the node frame. like Figure 1As shown in mark 103, the cubic Bézier curve of the edge uses the Canvas's bezierCurveTo(cp1x,cp1y, cp2x, cp2y, endX, endY) method. Here, cp1x = start_x, cp1y = (start_y + end_y) / 2, cp2x = end_x, cp2y = (start_y + end_y) / 2, endX = end_x, and endY = end_y. This control point design causes the curve to bend slowly in the vertical direction, smoothly returning to the end point after passing the inflection point. A triangular arrow is drawn at the end point: if the end point is to the right of the start point, the arrow's vertex is at the end point, and the two bottom vertices are respectively 8 pixels to the left of the end point and 4 pixels vertically offset upwards and downwards; if the end point is to the left of the start point, the arrow direction is reversed. like Figure 1 As shown in mark 104, users can complete the arrangement by dragging and dropping to add nodes, dragging and dropping to move nodes, and connecting lines. like Figure 1 As shown in mark 105, the current node array and edge array are serialized using JSON.stringify during saving. The SQL of the INSERT statement contains the following fields: id (UUID), name, description, category, trigger_type, nodes_json (TEXT), edges_json (TEXT), user_id, created_at, updated_at, usage_count=0, success_count=0, fail_count=0, avg_duration_ms=0. like Figure 1As shown in marks 106 to 108, the complete template installation process is as follows: The user selects a template from the template marketplace and clicks "Install" → The frontend sends a POST request to / workflows / install, carrying the templateId and a custom name → The backend selects nodes_json and edges_json from the database → JSON.parse deserialization → Initializes a Map<string, string> The `idMap` function iterates through the node array `nodes`: for each node, `oldId = node.id`, `newId = crypto.randomUUID()`, `idMap.set(oldId, newId)`, and `node.id = newId`. Iterates through the edge array `edges`: for each edge, `oldId = edge.id`, `newId = crypto.randomUUID()`, `edge.id = newId`, `edge.source = idMap.get(edge.source)`, and `edge.target = idMap.get(edge.target)`. A new workflow record is created: `name` is the user-specified name or template name + "(installation)", `user_id` is the target user ID, and `nodes_json` and `edges_json` are reserialized and stored. The template table is then updated: `SET downloads = downloads + 1 WHERE id = templateId`. A 200 OK response is returned. During the identifier remapping process, if the original identifier corresponding to the source or target of a certain edge is not found in the mapping table, it indicates that there is an anomaly in the template data (the references of nodes and edges are inconsistent), and the system directly throws an exception. This mapping verification mechanism can also be used to automatically detect the integrity of the template data. The above description is merely an illustration of specific embodiments of the present invention and is not intended to limit the scope of the invention. Those skilled in the art can make various modifications and improvements without departing from the spirit and scope of the invention, and all such modifications and improvements fall within the protection scope of the present invention.

Claims

1. A workflow visualization orchestration and execution method based on structured data template serialization, characterized in that, include: The structure of a workflow node object is defined as follows: the node identifier field is a string type, the node type field is one of 22 predefined types, the node description label field is a string type, the two-dimensional horizontal axis field is a numeric type, the two-dimensional vertical axis field is a numeric type, and the configuration parameter field is a dictionary type with string keys to arbitrary values; The structure of a workflow edge object is defined as follows: the edge identifier field is a string type, the source node identifier field is a string type, the target node identifier field is a string type, the edge type field is a default type, the conditional true branch type, the conditional false branch type, and the error type are four enumerated values; When rendering nodes on the canvas, the drawing method depends on the node type: when the node type is a conditional judgment type, a rhombus with a diagonal length of 80 pixels is drawn centered on the node coordinates; when the node type is a start type or end type, a rounded rectangle with a width of 180 pixels, a height of 64 pixels, and a corner radius of 32 pixels is drawn; when the node type is one of the other 20 types, a rounded rectangle with a width of 180 pixels, a height of 64 pixels, and a corner radius of 12 pixels is drawn; all node frames are filled with the semi-transparent color corresponding to the node type and outlined with lines of that color. The default colors for different node types are centrally managed through a color mapping table. When rendering an edge on the canvas, the curve starts at the bottom center of the source node, the arithmetic mean of the ordinate of the curve's starting point and the ordinate of the top center of the target node is used as the vertical inflection point ordinate, and the curve ends at the top center of the target node. A cubic Bézier curve is used to connect the starting point and the ending point, with the curve's starting point as the first control point and the vertical inflection point ordinate as the first control point's ordinate, and the curve's ending point as the second control point and the vertical inflection point ordinate as the second control point's ordinate. At the ending point, a triangle-filled arrow pointing into the target node is drawn. The array of all node objects and the array of all edge objects in the workflow are serialized in JSON string format. The serialized node array string and edge array string are stored as two independent fields along with the workflow name, description, category, trigger type and user identifier in a workflow record in the database. Provides template installation operation: Receives template identifier and target user identifier, reads template record from database template table according to the template identifier, and deserializes the node array string and edge array string stored in the template record into node object array and edge object array; Iterate through each node object in the node object array, regenerate a globally unique identifier for each node object to replace the original identifier, and establish an identifier mapping relationship from the original identifier to the new identifier; iterate through each edge object in the edge object array, regenerate a globally unique identifier for each edge object to replace the original identifier, and replace the source node identifier field and target node identifier field of each edge object with the new identifier according to the identifier mapping relationship, to ensure that the identifier reference relationship of all nodes and edges in the workflow after installation is consistent; Create a new workflow record, serialize the regenerated identifier and replace the referenced array of node objects and edge objects, and store them in the workflow table; The method is executed by the terminal device processor.

2. The method according to claim 1, characterized in that, The 22 predefined node types include: start type, end type, application open type, element click type, text input type, key operation type, page scrolling type, judgment type, condition judgment type, loop type, interface call type, speech synthesis broadcast type, message sending type, waiting type, screenshot type, clipboard type, file operation type, voice input type, wearable device gesture type, sub-workflow type, data extraction type, and network request type.

3. The method according to claim 1, characterized in that, The ordinate of the first control point of the cubic Bézier curve is equal to the ordinate of the vertical inflection point and the abscissa is equal to the abscissa of the starting point of the curve. The ordinate of the second control point is equal to the ordinate of the vertical inflection point and the abscissa is equal to the abscissa of the ending point of the curve.

4. The method according to claim 1, characterized in that, The template installation operation also includes: after reading the template record and before creating a new workflow record, incrementing the download count field value corresponding to the template record by 1; when creating a new workflow record, initializing the four statistical fields of the new workflow's start usage count, success count, failure count, and average time in milliseconds to 0.

5. The method according to claim 1, characterized in that, It also includes workflow copying operations: read the node array and edge array of the existing workflow record, regenerate the identifier for each node object and edge object and replace the source target and target reference in the edge, create a workflow record belonging to the new user identifier, and append the copy suffix to the original name of the new workflow.

6. A workflow visualization orchestration and execution system based on structured data template serialization, characterized in that, include: The structured definition module is used to define a unified structured data format for workflow node objects and edge objects. Node objects contain identifiers, types, labels, coordinates, and a dictionary of configuration parameters, while edge objects contain identifiers, source node identifiers, target node identifiers, and edge types. The canvas rendering module is used to draw nodes on the canvas according to node type: rhombus (conditional judgment type), rounded rectangle with a corner radius of 32 pixels (start / end type), or rounded rectangle with a corner radius of 12 pixels (other types). It also renders directed edges and end arrows using cubic Bézier curves with the vertical inflection point ordinate as the control point ordinate. The drag-and-drop arrangement module allows users to add nodes from the component library to the canvas by dragging and dropping, move node positions by dragging and dropping, and create edges between two nodes by connecting lines. The serialization and persistence module is used to serialize the node array and edge array into JSON strings and store them as two independent fields in the database workflow table. The template installation module is used to read template records, traverse the deserialized node and edge arrays to regenerate identifiers for each object, establish and apply identifier mapping relationships to replace source and target references in edges, create new workflow records, and copy node and edge content.

7. The system according to claim 6, characterized in that, In the canvas rendering module, condition judgment type nodes are drawn as rhombuses with a diagonal length of 80 pixels, start and end type nodes are drawn as rounded rectangles with a corner radius of 32 pixels, and other type nodes are drawn as rounded rectangles with a corner radius of 12 pixels. The edge rendering is a cubic Bézier curve that connects the source node's bottom center as the starting point, the target node's top center as the ending point, and the arithmetic mean of the starting point's ordinate and the ending point's ordinate as the control point's ordinate.

8. The system according to claim 6, characterized in that, The template installation module increments the template download count field by 1 before creating a new workflow record, and initializes the statistics field of the new workflow to 0; it also includes a workflow copying module, which reads an existing workflow, regenerates identifiers for all its nodes and edges, replaces references, and then creates a copy belonging to the new user.

9. A terminal device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.