An extensible low-code scenario engine establishment method and engine

Abstract

This invention proposes a method and engine for building a scalable low-code scenario engine, belonging to the field of code platform engines. In this method, heterogeneous tasks are uniformly abstracted into event nodes with the same metadata structure, and corresponding metadata descriptions are defined. When the front-end orchestration canvas needs to add an event node, the configuration form corresponding to the metadata description of that task type is obtained. The configuration form data is assembled into a JSON configuration object that conforms to the metadata description definition, and the JSON configuration object is saved to a database and a Redis cache. When the task processor executes an event node, the corresponding JSON configuration object is read. Based on the predefined task type information, the configuration parameters required to execute the task are dynamically extracted from the corresponding JSON configuration form. Based on the configuration parameters required for that task type, the corresponding task is executed, improving the scalability and adaptability of the low-code scenario engine.

Description

Technical Field

[0001] This invention relates to the field of code platform engines, and in particular to a method and engine for building a scalable low-code scenario engine. Background Technology

[0002] Currently, low-code platforms have limited capabilities for building business automation scenarios (such as "delaying the execution of a data analysis task 5 minutes after receiving a Kafka message"). Typically, scheduled tasks (Quartz) and message listening (Kafka) are two independent systems that need to be hard-coded together. When adding new task types (such as listening to new message queues or integrating with new batch processing frameworks), the source code must be modified and the application redeployed, making "no-code" or "low-code" extensions impossible.

[0003] Furthermore, Quartz's scheduling configuration is based on Java API or XML, making it difficult for business users to understand and use. In low-code scenarios with multiple tenants and applications, Quartz's default global thread pool can lead to issues such as thread starvation and task crosstalk. Additionally, inconsistencies between the task orchestration canvas and the underlying scheduler state can easily result in "ghost tasks."

[0004] Therefore, the code platform engine in related technologies has the following problems: (1) Poor scalability: Adding new task types (such as new MQ types, new HTTP triggering methods) requires modifying the source code and re-releasing the version, and cannot be dynamically expanded.

[0005] (2) Event source fragmentation: Scheduled tasks and message events (Kafka) are two independent triggering mechanisms that cannot be orchestrated within the same module.

[0006] (3) Low level of visualization: CRON expressions, message filtering rules, etc. are not user-friendly for business personnel.

[0007] To address this problem, the present invention provides a method and engine for building a scalable low-code scenario engine, thereby solving the aforementioned issues. Summary of the Invention

[0008] To address the problems existing in the prior art, this invention innovatively proposes a scalable low-code scene engine creation method and engine, effectively solving the problem of low scalability and adaptability of low-code scene engines caused by existing technologies, and effectively improving the scalability and adaptability of low-code scene engines.

[0009] The first aspect of this invention provides a method for building a scalable low-code scenario engine, comprising: Heterogeneous tasks are uniformly abstracted into event nodes with the same metadata structure, and a corresponding metadata description JSON Schema is defined for each type of heterogeneous task. The metadata description JSON Schema is used to describe the structure and data type of the configuration items corresponding to each type of heterogeneous task. When the front-end orchestration canvas needs to add an event node of a certain heterogeneous task type, it obtains the configuration form corresponding to the metadata description JSON Schema of the task type, assembles the configuration form data into a JSON configuration object that conforms to the definition of the metadata description JSON Schema, and saves the JSON configuration object to the database and Redis cache; When the task processor reaches an event node, it reads the corresponding JSON configuration object, dynamically extracts the configuration parameters required to execute the task from the corresponding JSON configuration form based on the predefined task type information, and executes the corresponding task based on the configuration parameters required for that task type.

[0010] Optionally, the event node includes: a heterogeneous task triggering data source, heterogeneous task processing judgment logic, and execution actions after the heterogeneous task event is triggered.

[0011] Optionally, after the front-end orchestration canvas needs to add event nodes of a certain heterogeneous task type, it further includes: the front-end orchestration canvas performs unified preprocessing on all types of event nodes based on the event node type; the preprocessing includes: Provide a consistent graphical representation of event nodes for different types of tasks; Use connection operations to define the dependencies and execution order between different event nodes.

[0012] Optionally, when the task type is a Kafka subscription task, the metadata description JSON Schema includes the gene code, business identifier, data JSON list, and user ID; when the task type is a Quartz scheduled task, the metadata description JSON Schema includes the scheduling rule cron expression, application code, execution data script, execution condition route, and API interface address.

[0013] Furthermore, when the task type is a Kafka subscription task, the configuration form includes the task identifier, task judgment conditions, the action to be executed when the task judgment conditions are met, and the API call parameters when the task is executed; when the task type is a Quartz scheduled task, the configuration form includes the scheduling rules, data script execution, the jsonPath expression for judging each piece of data in the data script, and the API call parameters.

[0014] Optionally, it also includes: standardizing the definition of the interaction interfaces between the various core components in the engine; specifically including: The engine exposes a unified entry point to the outside world, and each heterogeneous task type calls the unified entry point to enter the engine's processing flow. Different heterogeneous task processors process tasks of their respective types. After data extraction, a data list in a unified format is obtained. Then, each data item in the data list is traversed, condition judgments are performed, and the onTrue or onFalse action is executed based on the judgment result. Different heterogeneous task processors call the same common API call logic, parse the API metadata in ApiConfig, replace the field values ​​in the current data item with the API metadata, construct and send the HTTP request, and return the response result.

[0015] Optionally, it also includes: establishing a multi-tenant isolation mechanism, specifically including: Create separate instances for each heterogeneous task type for each tenant; Allocate a separate thread pool for each tenant; Configure an independent data source connection for each tenant; Isolate all running resources by using the tenant ID namespace to avoid task interference.

[0016] Furthermore, the tenant ID namespace includes tenant ID, task type, and version number, so as to uniquely identify all underlying resources and enable different versions of the same scenario for the same tenant to run in parallel without interfering with each other.

[0017] Optionally, it also includes: establishing a state consistency guarantee mechanism, specifically including: The engine generates a globally unique version number for every modification to each scene, and this version number is used throughout the entire technology stack. The version update process is triggered by generating a globally unique version number. After the version update process is completed, the canvas state is modified to make the canvas consistent with the underlying execution engine state; if the new version runs abnormally, the old version configuration is retained and the reverse replacement is automatically performed to roll back to the previous version state.

[0018] The second aspect of this invention provides a scalable low-code scene engine, which is built upon the scalable low-code scene engine building method described in the first aspect of this invention.

[0019] The technical solution adopted in this invention has the following technical effects: 1. In this invention, heterogeneous tasks are uniformly abstracted into event nodes with the same metadata structure. A corresponding metadata description JSON Schema is defined for each heterogeneous task. The metadata description JSON Schema is used to describe the structure and data type of the configuration items corresponding to each heterogeneous task. When the front-end orchestration canvas needs to add an event node of a certain heterogeneous task type, the configuration form corresponding to the metadata description JSON Schema of that task type is obtained. The configuration form data is assembled into a JSON configuration object that conforms to the definition of the metadata description JSON Schema, and the JSON configuration object is saved to the database and Redis cache. When the task processor executes the event node, the corresponding JSON configuration object is read. According to the predefined task type information, the configuration parameters required to execute the task are dynamically extracted from the corresponding JSON configuration form. Based on the configuration parameters required for the task type, the corresponding task is executed. This effectively solves the problem of low scalability and adaptability of the low-code scenario engine caused by the existing technology, and effectively improves the scalability and adaptability of the low-code scenario engine.

[0020] 2. The event nodes in the technical solution of this invention include: heterogeneous task triggering data source, heterogeneous task processing judgment logic, and execution actions after the heterogeneous task event is triggered; when the front-end orchestration canvas needs to add an event node of a certain heterogeneous task type, the front-end orchestration canvas performs unified preprocessing on all types of event nodes based on the event node type; this allows the addition of new task types to only require the development of corresponding processors, without modifying the engine source code or restarting the engine, achieving true scalability; different task types are seamlessly orchestrated in different canvases, supporting complex business scenarios (such as message delay processing, timed batch processing after message aggregation).

[0021] 3. The technical solution of this invention provides a consistent graphical representation of event nodes corresponding to different types of tasks; the same connection operation is used to define the dependencies and execution order between different event nodes, so that business personnel do not need to understand CRON or Kafaka API, and can complete the configuration of complex scenarios through orchestration.

[0022] 4. The technical solution of this invention innovatively introduces a multi-tenant isolation mechanism, which creates an independent instance for each heterogeneous task type for each tenant; allocates an independent thread pool for each tenant; configures an independent data source connection for each tenant; and isolates all running resources through the tenant ID namespace to avoid task crosstalk, thread starvation and task interference, thereby improving the stability of the engine.

[0023] 5. The technical solution of this invention establishes a state consistency guarantee mechanism. The engine generates a globally unique version number for each modification of each scene, and this version number runs through the entire technology stack. Based on the generation of a globally unique version number, the version update process is triggered. After the version update process is completed, the canvas state is modified to make the canvas consistent with the underlying execution engine state. If the new version runs abnormally, the old version configuration is retained, and the reverse replacement is automatically performed to roll back to the previous version state, ensuring that the canvas is consistent with the scheduler and listener states and eliminating ghost tasks.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating the method of Embodiment 1 in the present invention; Figure 2 This is a schematic diagram of the engine architecture in the method of Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the Kafka message processing flow in the method of Embodiment 1 of the present invention; Figure 4 This is a schematic diagram of the timed task processing flow in the method of Embodiment 1 of the present invention; Figure 5 This is a schematic diagram of the unified API call execution flow in the method of Embodiment 1 of the present invention; Figure 6 This is a schematic diagram of the multi-tenant isolation mechanism in Embodiment 1 of the present invention. Figure 7 This is a schematic diagram of the state consistency guarantee process in the method of Embodiment 1 of the present invention. Detailed Implementation

[0027] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific embodiments and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the invention. To simplify the disclosure of the invention, components and arrangements of specific examples are described below. Furthermore, reference numerals and / or letters may be repeated in different examples. This repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. It should be noted that the components illustrated in the drawings are not necessarily drawn to scale. Descriptions of well-known components, processing techniques, and processes are omitted in this invention to avoid unnecessarily limiting the invention.

[0028] Example 1 like Figure 1 As shown, this invention provides a method for building a scalable low-code scenario engine, including: S1, abstract heterogeneous tasks into event nodes with the same metadata structure, and define a corresponding metadata description JSON Schema for each heterogeneous task. The metadata description JSON Schema is used to describe the structure and data type of the configuration items corresponding to each heterogeneous task. S2, when the front-end orchestration canvas needs to add an event node of a certain heterogeneous task type, obtain the configuration form corresponding to the metadata description JSON Schema of the task type, assemble the configuration form data into a JSON configuration object that conforms to the definition of the metadata description JSON Schema, and save the JSON configuration object to the database and Redis cache; S3: When the task processor reaches an event node, it reads the corresponding JSON configuration object, dynamically extracts the configuration parameters required to execute the task from the corresponding JSON configuration form based on the predefined task type information, and executes the corresponding task based on the configuration parameters required for that task type.

[0029] In step S1, the event node includes: heterogeneous task triggering data source, heterogeneous task processing judgment logic, and execution action after the heterogeneous task event is triggered.

[0030] In traditional solutions, heterogeneous tasks with different trigger sources, such as Kafka messages and Quartz scheduled tasks, have completely different programming models and configuration methods: Kafka messages require configuring Topics, Consumer Groups, deserializers, etc.

[0031] Quartz scheduled tasks require configuring Job classes, Trigger expressions, JobDataMap, etc.

[0032] This solution uses a task metadata description abstraction mechanism to unify these heterogeneous task sources into event nodes with the same metadata structure. (It retains the independent configuration structure (each JSON schema) for each task type; it doesn't convert all tasks to the same JSON format. As seen in the Kafka node and scheduled task node configurations, an event node consists of three parts: 1. Triggering data source (where the event comes from): 1.1 Kafka node: from the data field of a Kafka message; 1.2 Scheduled task node: from the query results of an executed data script (such as SQL). 2. Judgment logic (whether the event should be processed): 2.1 Uniform judgment condition: judge the jsonPath expression of each data record; 2.2 If the result is true, execute if the condition is met; otherwise, execute if the condition is not met. 3. Execution action (what to do after the event is triggered): 3.1 Currently, it mainly involves no action or calling an API; 3.2 When calling an API, the API executor uniformly handles the request address, header, parameters, etc.). Each event node is represented internally by a standardized JSON configuration object, containing common fields such as type (task type), config (type-specific configuration), and next (subsequent actions).

[0033] The core foundation of this engine is the task meta-description abstraction mechanism. This mechanism achieves dynamic expansion of task types and unified visual configuration through a "define once, reuse multiple parts" approach. The metadata description JSON Schema is as follows: Predefined task JSON Schema: For each task type supported by the engine (such as Kafka subscription, scheduled tasks, etc.), the developers predefine its corresponding JSON Schema. This JSON Schema is a metadata description of the task configuration items; it does not contain specific business values, but rather defines the configuration structure, data types, and other information.

[0034] Metadata defines the structure and type of task configuration items, but does not contain specific business values. When the task type is a Kafka subscription task, the metadata description JSON Schema includes: {"appGene": "gene code","businessFlag": "business identifier","data": ["data JSON list"],"userId": "user ID"}. When the task type is a Quartz scheduled task, the metadata description JSON Schema includes: {"cronExpression":"cron expression","dataScript":"application code, data script code","condition":true,"onFalse":{"action":0, "api":"api interface"},"onTrue":{"action":0, "api":"api interface"}}.

[0035] Taking the "kafka subscription" task as an example, a simplified example of its JSON schema is as follows: { "appGene": "Gene Code", "businessFlag": "Business Identifier", "data": ["JSON list of data"] ], "userId": "User ID"; } In step S2, the event node drives the dynamic generation of the visual UI: When the front-end orchestration canvas needs to add a "Kafka Subscription" event node, the front-end component will display a configuration form corresponding to the JSON schema for that task type, such as a text input box with a "Business Identifier" label. Business users do not need to understand the underlying technology; they only need to fill in or select business rules in the intuitive form. A compliant JSON configuration is generated: after the business user completes the form, the front-end automatically assembles the form data into a JSON object that conforms to the schema definition. This JSON object is the completed configuration for that task node and will be saved to the database and Redis cache.

[0036] Each task type (such as Kafka subscription and scheduled tasks) has its own predefined JSON schema to describe the structure and data type of its configuration items. After the user completes the configuration on the front-end orchestration canvas, the engine generates a JSON configuration object that conforms to this schema. This object fully contains the task's "conditions" (jsonPath expression) and the actions to be executed when the conditions are met / not met (such as calling an API). On the core engine side, neither the Kafka processor nor the scheduled task processor directly parses the specific business fields (such as businessFlag, cron, etc.) after obtaining the JSON configuration. Instead, they uniformly enter a standard execution flow unrelated to the configuration content: first, the processor obtains the list of data to be processed (List). <Map<String, Object> Then, it iterates through each piece of data, determining whether to execute the "execute if condition is met" or "execute if condition is not met" branch based on the "judgment conditions" in the configuration, and finally calls the unified API executor (doSmartScenarioApiRequest) to complete the HTTP request. Throughout this process, the engine relies only on the two general structures of "condition" and "action" agreed upon in the JSON configuration, completely unaware of the specific configuration fields specific to each task type. Therefore, no matter how many new task types are added, as long as their configuration follows the "condition-action" paradigm, the engine's core execution logic requires no modification, thus achieving complete decoupling from specific task configurations.

[0037] In the future, new task types can be added as needed, such as action triggers. The configuration panel for orchestration is basically the same as the existing panel. The backend business logic only needs to develop the data processing part. The rest can directly reuse the existing methods without modifying the engine source code to complete the addition of new task types.

[0038] Because of the unified event node model in step S1, the front-end orchestration canvas uses the same set of drag-and-drop, connection, and configuration logic to handle all types of task nodes (i.e., preprocessing): (1) Provide a consistent graphical representation (rectangular node + input / output port) for timer event nodes and Kafka listener event nodes.

[0039] (2) Use the same connection operation to define the dependencies and execution order between nodes.

[0040] (3) Use the same property panel (dynamically generated by reading JSON Schema) to configure parameters for different types of nodes.

[0041] Without a unified event source model, the front end would need to develop a separate set of UI components and interaction logic for each task type, making it impossible to achieve drag-and-drop layout or low-code ease of use.

[0042] In step S3, when the engine's task processor (such as a Kafka message processor) reaches this node, it reads the corresponding JSON configuration. The API executor or task processor internally contains a schema-driven parser that dynamically extracts the key parameters required for task execution from the JSON configuration based on predefined type information. For example, the parser might read the value of `businessFlag` as a business routing identifier. In this way, the engine's core execution logic is completely decoupled from the specific task configuration.

[0043] like Figure 2 The diagram shown is a flowchart of the overall architecture of the engine of this invention, including: Visual orchestration layer: Business users configure Kafka nodes (event nodes) and timed nodes (event nodes) through orchestration, and the engine automatically generates orchestration content in JSON format.

[0044] The orchestration of the Kafka nodes is as follows: Business Identifier (Task Identifier): A unique identifier for the node. It must be manually entered and must be consistent with the Kafka producer. Task condition: Execute when the result of the jsonPath expression for each piece of data is true; the default value is true if not configured. When conditions are met, an action will be executed: this includes two options, no action or API call. No action means no operation will be performed, while API call requires configuring the following API parameters: Request URL: GET / POST / PUT / DELETE + API URL; For example: ${apiServer} / front / lowcode / editor / v1 / businessData / update; Request headers: For example, Authorization: ${BdAuthorization}; Header request parameters: The parameter types are either JSON or form; Execute if conditions are not met: Same as when conditions are met; The scheduling of the scheduled task nodes is as follows: Timing rules: Write a cron expression (optional), for example: "0 0 / 1" ?"; Querying data (data script execution): To invoke a data script, select Application -> Data Script Template to execute it and query the results of the data script execution. Conditional statement (the jsonPath expression to be evaluated for each piece of data in the data script): Execute the conditional statement for each piece of data if the result is true. The default value is true if not configured. Execute when conditions are met: This includes two options: no action or API call. No action means no operation is performed, while API call requires configuring the following API parameters: Request URL: GET / POST / PUT / DELETE + API URL; For example: ${apiServer} / front / lowcode / editor / v1 / businessData / update; Request headers: For example, Authorization: ${BdAuthorization}; Header request parameters: The parameter types are either JSON or form; Execute if conditions are not met: Same as when conditions are met; Core Engine Layer: The task engine calls the corresponding task processor according to the scenario content. Each scenario needs to select the task type (1: scheduled task, 2: Kafka subscription) when it is created. All processors share a unified API executor.

[0045] Infrastructure layer: Provides basic capabilities such as task types and message queues, and a unified entry point and type routing mechanism for core components.

[0046] To achieve scalability and modular decoupling, this engine has standardized the interaction interfaces between the various core components.

[0047] That is, the scalable low-code scenario engine establishment method of this invention also includes: Step S4: Standardize the definition of the interaction interfaces between the various core components in the engine; 1. The engine exposes a unified entry point method, insert(TaskContext context). Whether it's a Kafka message arrival, a Quartz timed trigger, or any new task type added in the future (such as RabbitMQ, file monitoring, etc.), this insert method will ultimately be called to enter the engine's core processing flow. The TaskContext object encapsulates the task's tenant ID, scenario ID, version number, and task type identifier.

[0048] 2. Differentiated Processor Layer: Each processor method focuses on handling the differentiated logic of its specific data source.

[0049] `executeSmartScenario(context)`: A scheduled task processor. Its core responsibility is to parse data scripts (such as SQL scripts or Groovy scripts) from the configuration, execute the scripts, and obtain a list of data to be processed.

[0050] `processBusinessDataSync(context)`: A Kafka subscription processor. Its core responsibility is to extract the `data` field (which can be a single JSON object or a JSON array) from the Kafka message body and use it as a list of data to be processed.

[0051] Future processor expansion: When adding a new task type, simply add the corresponding processor method (such as processRabbitMqData) and register it in the type-processor mapping table. The core engine framework requires no modification.

[0052] Regardless of the processor used, after data extraction, a data list in a uniform format will be obtained. <Map<String, Object> >), and then enter a unified processing loop: traverse each data item in the data list, perform conditional judgments, and decide whether to execute the onTrue or onFalse action based on the judgment result.

[0053] 3. Unified API Execution Layer: doSmartScenarioApiRequest Method: When a processor needs to execute an API call (whether in the onTrue or onFalse branch), all processors call the same public method: doSmartScenarioApiRequest(ApiConfig apiConfig, Map<String, Object> variables).

[0054] This method is responsible for providing general API call logic: Parse the API metadata (URL, request method, request headers, request body template) in ApiConfig; Replace variables (i.e., field values ​​in the current data item) with variable placeholders (such as ${userId}) in the URL, request header, and request body. Build and send an HTTP request; Return the response result.

[0055] like Figure 3 As shown, Figure 3 This illustrates the Kafka message processing flow: 1. A Kafka message arrives, triggering the handler; 2. Query the Redis cache to obtain the scenario configuration; 3. When a cache miss occurs, query the MySQL database; 4. Write the query results to the Redis cache; 5. Retrieve the list of data from the message; 6. Iterate through each data item; (1) Execution condition judgment; (2) Select the onTrue or onFalse action based on the condition result; (3) If an API call is required, it should be handled by the API executor; 7. Process the next data item in a loop; 8. Kafka message description; (1) The default topic of Kafka is ${env}_lowcode_smart, where ${env} is divided into dev, test, and prod according to different environments; (2) The key in Kafka is the customer ID. If it is not sent, no logic will be executed. (3) Example of a Kafka message body: { "userId":"", / / Required; "businessFlag":"supplierUpdate", / / Required, business identifier, must be unique for each customer; "appGene":"", / / Optional, after filling it in, the system will check whether the gene code matches; "data":{"a":"1","b":"2"} / / or [{},{}], required; } (4) The data processing objects in the scenario are all the data parameters in the message body, rather than the entire Kafka message body; (5) If data is an array, then each data item in the array will trigger the execution action.

[0056] like Figure 4 As shown, Figure 4 The scheduled task processing flow is shown below: 1. Quartz triggers tasks periodically according to CRON expressions; 2. Query MySQL to obtain the scenario configuration; 3. Parse the dataScript data script in the configuration; 4. Execute the data script to obtain the data list; 5. Iterate through each data item; (1) Execution condition judgment; (2) Select the onTrue or onFalse action based on the conditions; (3) If an API call is required, it should be handled by the API executor; 6. Process the next data item in the loop.

[0057] like Figure 5 As shown, Figure 5 This demonstrates the unified API call execution flow: 1. The task processor calls the API executor; 2. The API executor parses the API configuration; 3. Construct variable mappings based on scenario types; 4. Process URLs; 5. Construct the HTTP request object; 6. Process request headers and replace variables; 7. Process the request body or query parameters and replace variables; 8. Send an HTTP request to the target API; 9. Return the response result to the task processor.

[0058] Preferably, in order to solve the problem that traditional CRON expressions are not user-friendly for business personnel, this engine provides a set of visual programming and reverse parsing mechanisms.

[0059] 1. Visual orchestration (Natural Language → Parametric Model): When configuring scheduled tasks on the orchestration canvas, the system can directly provide a CRON expression input box, and also offers a multi-level selection panel that maps the various components of CRON (seconds, minutes, hours, days, months, weeks) to natural language options that business users can understand. For example: Execution frequency: Provides radio buttons for "Daily", "Weekly", "Monthly", and "Custom".

[0060] Specific dates within a cycle: If "Monthly" is selected, a calendar selection box for the current month will be displayed; if "Weekly" is selected, checkboxes for "Monday" through "Sunday" will be displayed.

[0061] Execution scope: Provides options such as "execute before a specified date" and "execute N times and then end".

[0062] 2. Reverse analysis (parameter model → CRON expression): When the user confirms and saves the configuration, the front-end conversion function calls a "CRON expression generator". This generator internally maintains a mapping rule table, assigning the user-selected parameter model (e.g., 12:00:00 every day (enumerated value 0) to "0 0 12"). The question mark (?) corresponds to "0 0 12 1" on the 1st of each month at 12:00:00 (enumeration value 1). (e.g., ?) precisely maps enumeration values ​​to CRON expressions conforming to the Quartz specification, with each cron expression and enumeration value pre-set.

[0063] Through this mechanism, the system hides the complexity of CRON expressions at the user interface layer, while retaining robust scheduling capabilities internally. When it is necessary to display or modify existing scheduled tasks, the system can perform the reverse operation: parsing the stored CRON expressions back into the parameter model described above, and automatically selecting the options on the interface, ensuring intuitive and consistent configuration.

[0064] Furthermore, the scalable low-code scenario engine creation method of the present invention also includes: Step S5: Establish a multi-tenant isolation mechanism.

[0065] Specifically, such as Figure 6 As shown, step S5 specifically includes: Create separate instances for each heterogeneous task type for each tenant (create separate Quartz Scheduler instances for each tenant; create separate Kafka Consumer instances for each tenant (different consumer groups, etc.); Allocate a separate thread pool for each tenant; Configure an independent data source connection for each tenant; Isolate all running resources by using the tenant ID namespace to avoid task interference.

[0066] Traditional multi-tenant isolation typically occurs only at the application or database layer (e.g., shared thread pools, different tenants using different data sources). This solution further de-emphasizes isolation (down to the task type instance level): independent runtime resources are created for each task type instance of each tenant (e.g., tenant A's Kafka consumer, tenant B's Quartz scheduler). This fine-grained isolation enables: (1) When a tenant adds a new task type (such as a scheduled task), the new task instance has an independent thread pool and consumer group, and will not squeeze the resources of the existing task types of the tenant or other tenants.

[0067] (2) When a tenant’s task type fails (such as a scheduled task), the failure is restricted to the isolation domain of the task type instance and will not affect other task types of the same tenant, let alone other tenants.

[0068] This solution uniquely identifies all underlying resources using a three-level namespace (e.g., scheduler_tenantA_QUARTZ_v1, consumerGroup_tenantB_KAFKA_v2) consisting of tenant ID, task type, and version number. This namespace design ensures that: (1) Adding any number of task types will not cause resource naming conflicts.

[0069] (2) Different versions of the same scenario for the same tenant can run in parallel (during the transition period of state switching) without interfering with each other.

[0070] This deep integration of scalability and isolation resolves the core dilemma faced by multi-tenant low-code platforms: supporting rapid integration of personalized task types from different tenants while ensuring that tenants do not interfere with each other and tasks do not disrupt each other. This synergistic effect cannot be achieved by simply stacking independent multi-tenant technologies or independent extension technologies.

[0071] In existing low-code platforms or task scheduling systems, the lack of a state synchronization mechanism between the visual orchestration canvas and the underlying execution engine (such as the Quartz scheduler or Kafka listener) poses a serious technical vulnerability. Specifically: 1. Scheduler State Residual: When a business user modifies or deletes a scheduled task on the canvas, the Quartz scheduler still retains the old version of the trigger and job detail. These "zombie" triggers will continue to execute at the scheduled time, calling invalid business logic, causing data corruption or duplicate processing.

[0072] 2. Listener state conflict: When modifying a Kafka subscription, the old Kafka Consumer thread is not automatically shut down. If both the old and new consumers are listening to the same Consumer Group or Topic simultaneously, it can lead to duplicate message consumption or partition rebalancing issues.

[0073] 3. Inconsistency between cache and database: After the canvas is modified, the scene configuration in the database has been updated, but the old configuration is still retained in the Redis cache or memory, causing different instances of the same scene to execute different versions of logic.

[0074] The aforementioned ghost task problem is prevalent in traditional task scheduling systems, but a systematic solution has long been lacking. The industry typically uses a coarse-grained approach of stopping the service, modifying the configuration, and finally restarting the service to solve this problem. However, this leads to business interruption and fails to meet the core requirements of low-code platform engines for hot updates and zero downtime.

[0075] To address the aforementioned ghost task problem, the scalable low-code scenario engine establishment method of this invention also includes: S6, establishing a state consistency guarantee mechanism.

[0076] Specifically, step S6 includes: The engine generates a globally unique version number for every modification to each scene, and this version number is used throughout the entire technology stack. The version update process is triggered by generating a globally unique version number. After the version update process is completed, the canvas state is modified to make the canvas consistent with the underlying execution engine state; if the new version runs abnormally, the old version configuration is retained and the reverse replacement is automatically performed to roll back to the previous version state.

[0077] Specifically, such as Figure 7 As shown, Figure 7 The flowchart of the state consistency guarantee mechanism is shown: 1. Business personnel modify the visualization canvas; 2. The system generates a new version number, v2; 3. Save the canvas configuration to the database and update Redis; 4. Trigger the version update process; 5. Stop the old version of Kafka consumers; 6. Confirm that the consumer has stopped; 7. Start the new version of the Kafka consumer; 8. Remove older versions of Quartz triggers; 9. Added a new version of Quartz triggers; 10. Return to update complete, ensuring the canvas and underlying execution engine are consistent to prevent ghost missions; 11. Mechanism Explanation: Versioned Atomic Replacement Mechanism.

[0078] The state consistency guarantee mechanism proposed in this invention is based on the core design idea of ​​"version number-driven atomic replacement," which specifically includes the following design features: (1) End-to-end version number binding: The engine generates a globally unique version number (e.g., v1, v2, v3) for every modification to each scene. This version number is used throughout the entire technology stack. Storage layer: Both the database and Redis cache scenarios include a version number field in their configuration. Scheduling layer: Both Quartz's JobDataMap and Kafka Consumer's subscription tags record version numbers; Execution layer: The version number triggered is recorded in the task execution log for problem tracing.

[0079] (2) Status confirmation and rollback mechanism: The replacement operation is not "initiated and completed instantly," but includes a status confirmation step. As seen in step 6 of the flowchart, "Confirm the consumer has stopped," the system actively verifies whether the old task has truly been cleared. If the new version malfunctions, the system can automatically perform a reverse replacement based on the retained old version configuration, rolling back to state v1 to prevent the fault from spreading.

[0080] Deep integration with Quartz / Kafka features: Quartz's Scheduler interface provides two separate methods: scheduleJob (add) and unscheduleJob (remove), which are combined into an "atomic operation unit" by this mechanism.

[0081] Kafka consumers are closed through a separate consumer.close() method, and new and old consumers are isolated by using different group.ids. This mechanism generates a unique consumer group identifier (such as group_{sceneId}_v2) using the version number to ensure that new and old consumers do not interfere with each other.

[0082] It should be noted that, in addition to Quartz, the task scheduling framework in the technical solution of this invention can use distributed scheduling frameworks such as XXL-JOB; in addition to Kafka, message middleware can support RabbitMQ, etc., only requiring the development of corresponding "task slots"; in addition to reverse parsing configuration, the visual implementation method can also use graphical drag-and-drop to generate configuration rules; in addition to independent instances, the isolation mechanism can also achieve resource isolation through thread pool isolation, task queue isolation, etc.

[0083] In this invention, heterogeneous tasks are uniformly abstracted into event nodes with the same metadata structure. A corresponding metadata description JSON Schema is defined for each heterogeneous task. This JSON Schema describes the structure and data type of the configuration items for each heterogeneous task. When the front-end orchestration canvas needs to add an event node for a specific heterogeneous task type, the configuration form corresponding to the metadata description JSON Schema for that task type is obtained. The configuration form data is assembled into a JSON configuration object that conforms to the definition of the metadata description JSON Schema, and the JSON configuration object is saved to the database and Redis cache. When the task processor executes an event node, it reads the corresponding JSON configuration object, dynamically extracts the configuration parameters required for task execution from the corresponding JSON configuration form based on the predefined task type information, and executes the corresponding task based on the configuration parameters required for that task type. This effectively solves the problem of low scalability and adaptability of low-code scene engines caused by existing technologies, and effectively improves the scalability and adaptability of low-code scene engines.

[0084] The event nodes in this invention include: heterogeneous task triggering data source, heterogeneous task processing judgment logic, and execution actions after the heterogeneous task event is triggered. When the front-end orchestration canvas needs to add an event node of a certain heterogeneous task type, the front-end orchestration canvas performs unified preprocessing on all types of event nodes based on the event node type. This allows the addition of new task types to only require the development of corresponding processors, without modifying the engine source code or restarting the engine, achieving true scalability. Different task types can be seamlessly orchestrated in different canvases, supporting complex business scenarios (such as message delay processing and timed batch processing after message aggregation).

[0085] The technical solution of this invention provides a consistent graphical representation of event nodes corresponding to different types of tasks; it uses the same connection operation to define the dependencies and execution order between different event nodes, so that business personnel do not need to understand CRON or Kafaka API, and can complete the configuration of complex scenarios through orchestration.

[0086] The technical solution of this invention innovatively introduces a multi-tenant isolation mechanism, which creates an independent instance for each heterogeneous task type for each tenant; allocates an independent thread pool for each tenant; configures an independent data source connection for each tenant; and isolates all running resources through the tenant ID namespace, thereby avoiding task crosstalk, thread starvation and task interference, and improving the stability of the engine.

[0087] The technical solution of this invention establishes a state consistency guarantee mechanism. The engine generates a globally unique version number for each modification of each scene, and this version number runs through the entire technology stack. Based on the generation of a globally unique version number, a version update process is triggered. After the version update process is completed, the canvas state is modified to make the canvas consistent with the underlying execution engine state. If the new version runs abnormally, the old version configuration is retained and the reverse replacement is automatically performed to roll back to the previous version state, ensuring that the canvas is consistent with the scheduler and listener states and eliminating ghost tasks.

[0088] Example 2 The technical solution of the present invention also provides a scalable low-code scene engine, which is based on a scalable low-code scene engine establishment method in Embodiment 1.

[0089] In this invention, heterogeneous tasks are uniformly abstracted into event nodes with the same metadata structure. A corresponding metadata description JSON Schema is defined for each heterogeneous task. This JSON Schema describes the structure and data type of the configuration items for each heterogeneous task. When the front-end orchestration canvas needs to add an event node for a specific heterogeneous task type, the configuration form corresponding to the metadata description JSON Schema for that task type is obtained. The configuration form data is assembled into a JSON configuration object that conforms to the definition of the metadata description JSON Schema, and the JSON configuration object is saved to the database and Redis cache. When the task processor executes an event node, it reads the corresponding JSON configuration object, dynamically extracts the configuration parameters required for task execution from the corresponding JSON configuration form based on the predefined task type information, and executes the corresponding task based on the configuration parameters required for that task type. This effectively solves the problem of low scalability and adaptability of low-code scene engines caused by existing technologies, and effectively improves the scalability and adaptability of low-code scene engines.

[0090] The event nodes in this invention include: heterogeneous task triggering data source, heterogeneous task processing judgment logic, and execution actions after the heterogeneous task event is triggered. When the front-end orchestration canvas needs to add an event node of a certain heterogeneous task type, the front-end orchestration canvas performs unified preprocessing on all types of event nodes based on the event node type. This allows the addition of new task types to only require the development of corresponding processors, without modifying the engine source code or restarting the engine, achieving true scalability. Different task types can be seamlessly orchestrated in different canvases, supporting complex business scenarios (such as message delay processing and timed batch processing after message aggregation).

[0091] The technical solution of this invention provides a consistent graphical representation of event nodes corresponding to different types of tasks; it uses the same connection operation to define the dependencies and execution order between different event nodes, so that business personnel do not need to understand CRON or Kafaka API, and can complete the configuration of complex scenarios through orchestration.

[0092] The technical solution of this invention innovatively introduces a multi-tenant isolation mechanism, which creates an independent instance for each heterogeneous task type for each tenant; allocates an independent thread pool for each tenant; configures an independent data source connection for each tenant; and isolates all running resources through the tenant ID namespace, thereby avoiding task crosstalk, thread starvation and task interference, and improving the stability of the engine.

[0093] The technical solution of this invention establishes a state consistency guarantee mechanism. The engine generates a globally unique version number for each modification of each scene, and this version number runs through the entire technology stack. Based on the generation of a globally unique version number, a version update process is triggered. After the version update process is completed, the canvas state is modified to make the canvas consistent with the underlying execution engine state. If the new version runs abnormally, the old version configuration is retained and the reverse replacement is automatically performed to roll back to the previous version state, ensuring that the canvas is consistent with the scheduler and listener states and eliminating ghost tasks.

[0094] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for building a scalable low-code scenario engine, characterized in that, include: Heterogeneous tasks are uniformly abstracted into event nodes with the same metadata structure, and a corresponding metadata description JSON Schema is defined for each type of heterogeneous task. The metadata description JSON Schema is used to describe the structure and data type of the configuration items corresponding to each type of heterogeneous task. When the front-end orchestration canvas needs to add an event node of a certain heterogeneous task type, it obtains the configuration form corresponding to the metadata description JSON Schema of the task type, assembles the configuration form data into a JSON configuration object that conforms to the definition of the metadata description JSON Schema, and saves the JSON configuration object to the database and Redis cache; When the task processor reaches an event node, it reads the corresponding JSON configuration object, dynamically extracts the configuration parameters required to execute the task from the corresponding JSON configuration form based on the predefined task type information, and executes the corresponding task based on the configuration parameters required for that task type.

2. The method for establishing a scalable low-code scene engine according to claim 1, characterized in that, The event nodes include: heterogeneous task triggering data source, heterogeneous task processing judgment logic, and execution actions after the heterogeneous task event is triggered.

3. The method for establishing a scalable low-code scene engine according to claim 1, characterized in that, When the front-end orchestration canvas needs to add an event node of a heterogeneous task type, the process also includes: the front-end orchestration canvas performs unified preprocessing on all types of event nodes based on the event node type; the preprocessing includes: Provide a consistent graphical representation of event nodes for different types of tasks; Use connection operations to define the dependencies and execution order between different event nodes.

4. A method for establishing a scalable low-code scene engine according to claim 1, characterized in that, When the task type is a Kafka subscription task, the metadata description JSON Schema includes the gene code, business identifier, data JSON list, and user ID; when the task type is a Quartz scheduled task, the metadata description JSON Schema includes the scheduling rule cron expression, application code, execution data script, execution condition route, and API interface address.

5. The method for establishing a scalable low-code scene engine according to claim 4, characterized in that, When the task type is a Kafka subscription task, the configuration form includes the task identifier, task judgment conditions, the action to be executed when the task judgment conditions are met, and the API call parameters when the task is executed; when the task type is a Quartz scheduled task, the configuration form includes the scheduling rules, data script execution, the jsonPath expression for judging each piece of data in the data script, and the API call parameters.

6. The method for establishing a scalable low-code scene engine according to claim 1, characterized in that, This also includes: standardizing the definition of the interaction interfaces between the various core components in the engine; specifically including: The engine exposes a unified entry point to the outside world, and each heterogeneous task type calls the unified entry point to enter the engine's processing flow. Different heterogeneous task processors process tasks of their respective types. After data extraction, a data list in a unified format is obtained. Then, each data item in the data list is traversed, condition judgments are performed, and the onTrue or onFalse action is executed based on the judgment result. Different heterogeneous task processors call the same common API call logic, parse the API metadata in ApiConfig, replace the field values ​​in the current data item with the API metadata, construct and send the HTTP request, and return the response result.

7. The method for establishing a scalable low-code scene engine according to claim 1, characterized in that, This also includes: establishing a multi-tenant isolation mechanism, specifically including: Create separate instances for each heterogeneous task type for each tenant; Allocate a separate thread pool for each tenant; Configure an independent data source connection for each tenant; Isolate all running resources by using the tenant ID namespace to avoid task interference.

8. The method for establishing a scalable low-code scene engine according to claim 7, characterized in that, The tenant ID namespace includes tenant ID, task type, and version number, so as to uniquely identify all underlying resources and enable different versions of the same scenario for the same tenant to run in parallel without interfering with each other.

9. The method for establishing a scalable low-code scene engine according to claim 1, characterized in that, It also includes: establishing a state consistency guarantee mechanism, specifically including: The engine generates a globally unique version number for every modification to each scene, and this version number is used throughout the entire technology stack. The version update process is triggered by generating a globally unique version number. After the version update process is completed, the canvas state is modified to make the canvas consistent with the underlying execution engine state; if the new version runs abnormally, the old version configuration is retained and the reverse replacement is automatically performed to roll back to the previous version state.

10. A scalable low-code scenario engine, characterized in that, This method is based on any one of the scalable low-code scenario engine creation methods described in claims 1-9.

Images