Modularized code processing method and system based on dynamic plug-in discovery in Agent research and development

Abstract

The invention provides a modular code processing method and system based on dynamic plug-in discovery in Agent research and development, and relates to the technical field of computers. The method comprises the following steps: in an Agent system initialization stage, obtaining functional module metadata in a current operating environment, dynamically loading processing nodes according to the functional module metadata, and forming a candidate node set; performing type consistency verification on the data connection relationship among the candidate nodes to obtain an effective node set; constructing and analyzing a node dependency relationship graph based on the dependency relationship between the effective nodes, and generating a hierarchical execution plan; and scheduling the effective nodes according to the hierarchical execution plan, and executing the effective nodes in parallel in the same execution hierarchy. According to the method and the device, the dependence of module management on manual configuration can be reduced under the condition that the number of the function modules is increased or the function modules are updated frequently, and the modules are more consistent and predictable in the aspects of collaboration and execution organization, so that the scale expansion and operation requirements in a complex task scene are met.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and to a modular code processing method and system based on dynamic plug-in discovery in agent development. Background Technology

[0002] As the agent-based R&D model continues to evolve, related software systems are increasingly characterized by a growing number of modules, frequent feature updates, and flexible feature combinations. Agents, typically running as software entities or systems within computer systems, are applied in AI-assisted R&D scenarios to support the execution of R&D tasks. To improve system scalability and reusability, modular approaches are commonly used to organize code, encapsulating different functions into relatively independent modules. These modules are then loaded and used during system operation to meet the demands of complex R&D tasks.

[0003] However, existing modular code processing solutions still have certain limitations in agent development scenarios. On the one hand, some technical solutions rely on pre-configuration or manual maintenance to register and organize modules during functional module management. When the number of functional modules increases or the module update frequency increases, the configuration process is prone to omissions, redundancy, or inconsistencies, and it is difficult to reflect the addition or change of modules in a timely manner, thus affecting the system's maintenance efficiency. On the other hand, in the process of multiple functional modules working together, modules usually need to interact through data connection relationships. However, there is a lack of unified standards for handling data connection relationships between modules in related technologies. The matching relationship between modules often relies on the agreement made by developers during the design phase. When the module interface is adjusted or the module source is inconsistent, connection mismatch problems are prone to occur. In addition, existing technologies mostly adopt fixed or preset processing methods in terms of module execution organization, lacking the ability to flexibly adjust the execution association between modules. When the module scale is large or the task complexity is high, it can easily affect the overall execution efficiency.

[0004] Therefore, existing modular code processing solutions in Agent development scenarios still generally suffer from problems such as insufficient flexibility in module management, limited adaptability of inter-module collaboration, and difficulty in meeting the needs of complex applications in terms of execution organization. There is still room for further improvement in the face of the continuous expansion of functional module scale and the continuous increase in task complexity.

[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] This disclosure provides a modular code processing method and system based on dynamic plugin discovery in Agent development, in order to solve the problems of insufficient flexibility in module management, limited adaptability of inter-module collaboration relationships, and difficulty in meeting the needs of complex applications in related technologies.

[0007] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure.

[0008] According to a first aspect of this disclosure, a modular code processing method based on dynamic plugin discovery in agent development is provided, including: During the Agent system initialization phase, the functional module metadata of the current operating environment is obtained, the processing nodes are dynamically loaded according to the functional module metadata, and the processing nodes are registered as a candidate node set. Perform type consistency verification on the data connection relationships between each candidate node in the candidate node set to obtain the valid node set; Based on the dependencies between the valid nodes in the set of valid nodes, a node dependency graph is constructed, and the node dependency graph is analyzed and processed to generate a hierarchical execution plan. Each effective node is scheduled according to the hierarchical execution plan, and each effective node is executed in parallel within the same execution level.

[0009] In one exemplary embodiment of this disclosure, the step of dynamically loading processing nodes according to the functional module metadata and registering the processing nodes as a candidate node set includes: Read node entry information from the metadata of the functional modules; Based on the node class path indicated by the node entry information, import the node class corresponding to the node class path; Verify whether the node class conforms to the preset processing node interface specification, and dynamically load the node class to generate a processing node when the verification is successful, and register the processing node to the candidate node set.

[0010] In an exemplary embodiment of this disclosure, before performing type consistency verification on the data connection relationships between candidate nodes in the candidate node set, the method further includes: The candidate nodes in the candidate node set are classified and stored according to the namespace to which the candidate nodes belong; In the case where multiple candidate nodes have the same node identifier, the target candidate node that will not be replaced is determined according to the priority order corresponding to the namespace.

[0011] In one exemplary embodiment of this disclosure, the method further includes: If a candidate node in a low-priority namespace is detected to have the same node identifier as a candidate node in a high-priority namespace, the registration of the candidate node in the low-priority namespace is prevented, and registration error information corresponding to the candidate node in the low-priority namespace is returned.

[0012] In one exemplary embodiment of this disclosure, the method further includes: Obtain node reference information and determine whether the node reference information contains a namespace identifier; If the node reference information includes a namespace identifier, the target candidate node corresponding to the node reference information is determined based on the combination of the namespace identifier and the node identifier.

[0013] In one exemplary embodiment of this disclosure, the step of performing type consistency verification on the data connection relationships between candidate nodes in the candidate node set to obtain a valid node set includes: For each candidate node in the candidate node set, read the input type pattern and output type pattern declared by each candidate node; Based on the input type pattern and output type pattern, static type verification is performed on the data connection relationship between each candidate node, and the effective node set is constructed according to the candidate nodes corresponding to the data connection relationships that pass the verification.

[0014] In one exemplary embodiment of this disclosure, the method further includes: Dynamic type validation is performed on the input data passed to the valid node to determine whether the input data satisfies the input type pattern declared by the valid node; If the verification fails, the current execution flow of the valid node is terminated, and the dependent nodes of the valid node are prevented from entering the scheduling execution flow.

[0015] In one exemplary embodiment of this disclosure, constructing a node dependency graph based on the dependencies between valid nodes in the set of valid nodes includes: Identify the data dependencies between the valid nodes in the set of valid nodes; Based on the data dependencies, a node dependency graph is constructed with valid nodes as vertices and dependencies as edges.

[0016] In one exemplary embodiment of this disclosure, after constructing the node dependency graph, the method further includes: Perform a graph traversal operation on the node dependency graph, and record the node identifiers on each traversal path during the graph traversal process; If a duplicate node identifier is detected in any of the traversal paths, the valid node with the duplicate node identifier is determined to be a circular dependency node, and the dependency path information containing the circular dependency node is output.

[0017] In one exemplary embodiment of this disclosure, the step of analyzing and processing the node dependency graph to generate a hierarchical execution plan includes: If no duplicate node identifiers are detected in any of the traversal paths, the dependency level of each valid node in the node dependency graph is determined based on the node dependency graph; the dependency level is used to characterize the sequential relationship of each valid node in the execution order. Based on the dependency hierarchy, valid nodes with the same dependency hierarchy are divided into the same execution hierarchy, and a hierarchical execution plan consisting of multiple execution hierarchies is generated according to the order of the dependency hierarchy.

[0018] In one exemplary embodiment of this disclosure, scheduling each of the effective nodes according to the hierarchical execution plan and performing parallel execution on each of the effective nodes within the same execution level includes: According to the order of each execution level in the hierarchical execution plan, the target execution level to be executed is determined sequentially; For each valid node in the target execution level, the corresponding node execution operation is triggered, and each valid node is scheduled to be executed in parallel within the same target execution level. After each valid node in the target execution level has completed its execution, the process switches to the next execution level to continue execution.

[0019] In an exemplary embodiment of this disclosure, the parallel scheduling and execution of each of the effective nodes within the same target execution level includes: Obtain the task feature information corresponding to each of the effective nodes; Based on the task characteristic information, each effective node is assigned to a different concurrent execution resource, and each effective node is executed through the concurrent execution resource.

[0020] According to a second aspect of this disclosure, a modular code processing system based on dynamic plugin discovery in agent development is provided, comprising: The node discovery module is used to obtain the functional module metadata in the current running environment during the Agent system initialization phase, dynamically load processing nodes according to the functional module metadata, and register the processing nodes as a candidate node set. The type verification module is used to verify the type consistency of the data connection relationship between each candidate node in the candidate node set, and to obtain the valid node set. The dependency analysis module is used to construct a node dependency graph based on the dependency relationships between each valid node in the set of valid nodes, and to analyze and process the node dependency graph to generate a hierarchical execution plan. The parallel scheduling module is used to schedule each of the effective nodes according to the hierarchical execution plan and to perform parallel execution on each of the effective nodes within the same execution level.

[0021] According to a third aspect of this disclosure, a computer-readable storage medium is provided, on which computer program code instructions are stored, which, when invoked by a processor, execute the method as described in the above embodiments.

[0022] As can be seen from the above technical solution, this disclosure possesses at least one of the following advantages and positive effects: This disclosure forms a candidate node set based on the functional module metadata in the runtime environment during the Agent system initialization phase, enabling the availability of functional modules to be updated as the runtime environment changes. This reduces the need for manual configuration when the number of functional modules increases or updates are frequent, making the module management process more adaptable to dynamically changing R&D scenarios.

[0023] Based on this, by performing type consistency verification on the data connection relationships between candidate nodes and forming a valid node set, all nodes participating in subsequent processing meet the predetermined data interaction constraints. This reduces the possibility of mismatched nodes entering the collaborative process when module interfaces change or module sources are diverse, making the module collaboration process more stable and controllable.

[0024] Furthermore, by organizing the execution order of nodes based on the dependencies between effective nodes, the node execution process can be arranged according to the actual relationships. This makes the execution organization more in line with actual processing needs when the module size increases or the task complexity increases, thereby enhancing the system's adaptability to complex application scenarios.

[0025] Furthermore, by scheduling nodes at the same execution level in parallel based on the above execution organization, multiple nodes can participate in the processing process simultaneously when there are no dependency constraints. This alleviates the efficiency limitations caused by serial execution in scenarios with a large number of modules or a large processing load, and enables the overall processing process to have higher execution efficiency while maintaining the rationality of the execution order.

[0026] In summary, this disclosure enables the modular code processing process to better adapt to the needs of continuously expanding functional module scale and increasing task complexity in Agent development scenarios in terms of module management, module collaboration, and execution organization. Without increasing the additional maintenance burden, it combines reduced manual configuration, type consistency verification, dependency-based execution organization, and parallel scheduling at the same level. This ensures that even with frequent additions and subtractions of functional modules and dynamic changes in combination relationships, the execution path and execution results of the modular code processing process remain consistent and predictable, thereby improving the system's operational stability and engineering reliability in complex Agent development scenarios. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A system architecture diagram is shown that can be applied to the modular code processing method based on dynamic plugin discovery in Agent development in the embodiments of this disclosure.

[0029] Figure 2 A flowchart illustrating a modular code processing method based on dynamic plugin discovery in Agent development is shown in an embodiment of this disclosure.

[0030] Figure 3 An architecture diagram of a modular code processing system based on dynamic plugin discovery in Agent development is shown in an embodiment of this disclosure.

[0031] Figure 4 A schematic diagram of a process for registering a processing node as a candidate node set is shown in an embodiment of this disclosure.

[0032] Figure 5 A schematic diagram of an automatic node discovery and registration process is shown in an embodiment of this disclosure.

[0033] Figure 6 A schematic diagram of a namespace resolution process according to an embodiment of this disclosure is shown.

[0034] Figure 7 A flowchart illustrating an overlay protection mechanism according to an embodiment of this disclosure is shown.

[0035] Figure 8 A flowchart illustrating one type of security mechanism in an embodiment of this disclosure is shown.

[0036] Figure 9 A flowchart illustrating a dependency analysis and topology sorting process in an embodiment of this disclosure is shown.

[0037] Figure 10 A flowchart illustrating a hierarchical parallel execution process according to an embodiment of this disclosure is shown.

[0038] Figure 11 A flowchart illustrating another hierarchical parallel execution process in an embodiment of this disclosure is shown.

[0039] Figure 12 A flowchart illustrating a concurrency strategy selection logic in an embodiment of this disclosure is shown.

[0040] Figure 13 A block diagram of a modular code processing system based on dynamic plugin discovery in agent development is shown in an embodiment of this disclosure.

[0041] Figure 14 A schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present disclosure is shown.

[0042] Figure 15 A schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure is shown. Detailed Implementation

[0043] In this disclosure, the terms "first" and "second" are used for description only and do not indicate relative importance or imply the number of technical features. Therefore, the features referred to as "first" or "second" may explicitly or implicitly include at least one of those features. "A plurality of" means at least two, unless otherwise expressly defined.

[0044] First, the relevant terms used in the exemplary embodiments of this disclosure will be explained: Entry Points: A pluggable mechanism for registering and discovering functional components, allowing functional modules to declare processing nodes that can be recognized and loaded by external systems through metadata during the installation or deployment phase. In this embodiment, the entry point mechanism is used to achieve automatic discovery and dynamic loading of processing nodes without modifying the main program code.

[0045] Namespace: An identifier system used to logically distinguish and organize processing nodes with the same name. By assigning different namespace identifiers to processing nodes, the origin of nodes can be distinguished when node identifiers are the same, and conflicts can be resolved in conjunction with preset priority rules.

[0046] DAG (Directed Acyclic Graph): A graph data structure in which there are no directed cycles, where all edges are directed and there is no path from any node back to that node by traversing several edges. In this embodiment, a DAG is used to represent the dependencies between processing nodes, thereby describing the constraints on the order of node execution.

[0047] Topological sorting is a method for sorting nodes in a directed acyclic graph, such that for any dependency in the graph, the preceding node appears before the subsequent node in the sorted result. In this embodiment, topological sorting is used to determine the legal execution order of processing nodes.

[0048] Kahn's Algorithm is an algorithm for topological sorting that maintains the in-degree of nodes and gradually removes nodes with an in-degree of zero to complete the sorting process. In this embodiment, Kahn's Algorithm can be used to process the node dependency graph to generate a hierarchical execution plan.

[0049] Depth-First Search (DFS) is a graph traversal algorithm that traverses a graph structure by delving deeper and deeper along a single path, backtracking as necessary. In this embodiment, DFS is used to traverse the node dependency graph to detect the existence of circular dependencies.

[0050] BaseNode: A basic interface specification for processing nodes defined in this embodiment of the disclosure, used to constrain the basic structure and behavior that processing nodes should possess. Through this interface specification, processing nodes can uniformly declare their input and output type information and execute corresponding processing logic, thereby enabling processing nodes from different sources to be loaded, scheduled, and managed within the same execution framework.

[0051] Layer-based execution: an execution method that groups processing nodes based on node dependencies, where processing nodes with the same dependency level are divided into the same execution level and can be executed in parallel within the same execution level.

[0052] Override Protection: A control strategy to prevent processing nodes from being replaced by nodes with the same name that do not conform to priority rules during registration or loading. In this embodiment, namespace priority is used to restrict the overriding behavior of nodes in lower-priority namespaces over nodes in higher-priority namespaces.

[0053] Type Schema: A declaration method used to describe data structure type constraints, characterizing the type requirements of input and output data for processing nodes. In this embodiment, each processing node declares a corresponding type schema to provide a basis for subsequent type verification.

[0054] Thread Pool / Process Pool: A scheduling method for unified management of concurrent execution resources. It reduces the overhead of frequently creating and destroying execution units by assigning multiple execution tasks to pre-created sets of threads or processes. In this embodiment, concurrent execution resources are used to support the parallel execution needs of multiple processing nodes within the same execution level in a hierarchical execution plan.

[0055] In existing agent development and related plug-in software systems, functions are typically organized in a modular fashion, allowing for system capabilities to be extended through plug-ins or feature packages. However, these technologies still have significant shortcomings in module discovery and management mechanisms. Some systems lack the ability to dynamically discover functional modules; these modules often need to be registered during the compilation or system configuration phase, and their registration information is usually fixed in the system through configuration files or hard-coded methods. When a user installs a new plug-in or feature package, the system cannot automatically recognize the newly added functional module, requiring the user to manually modify the configuration file or regenerate the registration information through a specific interface, followed by a system restart for the configuration to take effect. This approach not only increases the operational complexity of the function extension process but also makes the system prone to startup failures or functional abnormalities due to configuration errors, thus limiting the system's scalability and usability to some extent.

[0056] On the other hand, related technologies lack a unified and reasonable solution for handling naming conflicts among functional modules. In practical applications, functional modules from different sources (such as built-in system modules, third-party extension modules, and user-defined modules) may have the same module name. Some systems adopt a simple loading order overriding strategy, where the later-loaded module directly replaces the earlier-loaded module with the same name. This approach may, under certain circumstances, lead to the unexpected replacement of core system functions, resulting in unpredictable operational results. Other systems avoid conflicts by enforcing globally unique naming, but this method imposes high restrictions on user-defined extensions and is difficult to adapt to the actual needs of an open plugin ecosystem. Due to the lack of conflict resolution mechanisms based on namespaces or priority rules, existing systems struggle to achieve a balance between stability and flexibility.

[0057] Furthermore, related technologies generally lack the ability to automatically analyze node dependencies in task execution organization, which limits system execution efficiency. In execution flows containing multiple processing tasks, related systems typically execute tasks sequentially according to a preset order. Even if some tasks do not have direct dependencies, it is difficult to automatically identify the possibility of parallel execution. If parallel processing is required, it often relies on developers to manually analyze task dependencies and implement parallel control and synchronization through additional configuration or code. This process is complex and prone to errors. Therefore, in scenarios with large task scales or complex task relationships, existing systems struggle to fully utilize computing resources, and their overall execution efficiency needs further improvement.

[0058] To address one or more of the aforementioned technical issues, this disclosure proposes a modular code processing method based on dynamic plugin discovery in Agent development. By dynamically loading processing nodes based on functional module metadata during the system initialization phase and verifying the data connection relationships between nodes before execution, the nodes participating in the execution have consistent data interaction constraints. Simultaneously, a hierarchical execution plan is generated based on node dependencies, and nodes are scheduled to execute in parallel within the same level, making module management and execution organization more adaptable to application scenarios with expanded functional module scale and increased task complexity.

[0059] Figure 1 A system architecture diagram is shown that can be applied to the modular code processing method based on dynamic plugin discovery in Agent development in the embodiments of this disclosure.

[0060] like Figure 1 As shown, system architecture 100 may include one or more terminal devices such as smartphone 101, portable computer 102, and desktop computer 103, network 104, and server 105. Network 104 is used as a medium to provide a communication link between the terminal devices and server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables. Terminal devices may be electronic devices with data processing capabilities, used to initiate task requests to the system, configure processing flows, or trigger the execution of code processing tasks, etc., including but not limited to the aforementioned smartphone 101, portable computer 102, and desktop computer 103.

[0061] Server 105 is used to execute modular code processing methods based on dynamic plugin discovery in Agent development. Internally, it can deploy program modules for acquiring functional module metadata, dynamically loading processing nodes, building node dependencies, and scheduling the execution of processing nodes. Server 105 can be a single server or a server cluster consisting of multiple servers to meet the execution needs of code processing tasks of different scales. Of course, terminal devices can also be used to execute modular code processing methods based on dynamic plugin discovery in Agent development; this disclosure does not limit this use.

[0062] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0063] This disclosure also provides a modular code processing method for agent development based on dynamic plugin discovery. (See references) Figure 2 As shown, the method may include the following steps S210 to S240: Step S210: During the Agent system initialization phase, obtain the functional module metadata in the current running environment, dynamically load the processing nodes according to the functional module metadata, and register the processing nodes as a candidate node set. Step S220: Perform type consistency verification on the data connection relationships between candidate nodes in the candidate node set to obtain the valid node set; Step S230: Based on the dependency relationships between each valid node in the valid node set, construct a node dependency graph, analyze and process the node dependency graph, and generate a hierarchical execution plan; Step S240: Schedule each effective node according to the hierarchical execution plan, and perform parallel execution on each effective node within the same execution level.

[0064] The modular code processing method based on dynamic plugin discovery in Agent development provided by the exemplary implementation of this disclosure reduces the involvement of manual configuration by forming a candidate node set based on the functional module metadata in the runtime environment during the Agent system initialization phase; and forms a valid node set by performing type consistency verification on the data connection relationships between candidate nodes, reducing the possibility of mismatched nodes participating in collaborative processing; further, it organizes the node execution order based on the dependency relationships between valid nodes and performs parallel scheduling within the same execution level, making the execution process more in line with actual processing needs when the module size increases or the task complexity increases. Thus, the modular code processing process is more adaptable to the scale expansion needs of Agent development scenarios in terms of module management, module collaboration and execution organization, and improves the system's support capability for complex development tasks without increasing the additional maintenance burden.

[0065] refer to Figure 3 The diagram illustrates a modular code processing system based on dynamic plugin discovery in agent development. This system employs a layered architecture, comprising, from bottom to top, an automatic node discovery layer, a namespace resolution layer, a dependency analysis layer, and a parallel execution scheduling layer. The functional layers interact through predefined interfaces. When an upper-layer functional layer calls services provided by a lower-layer functional layer, it does not need to be aware of the specific implementation details of the lower layer. This results in a highly modular system structure, achieving a high-cohesion, low-coupling design.

[0066] The system can interact with the user layer during operation. The user layer handles operation requests from plugin developers and system users, such as installing functional modules, submitting task configurations, and triggering modular code processing tasks. For example, Figure 3 The user layer shown includes two types of entry points: "plugin developers publishing feature packages" and "system users defining task flows." Plugin developers provide node class implementations to the system by publishing and installing feature packages. System users define task flows and submit task configurations, triggering modular code execution by passing task-related information to the system's internal processing flow.

[0067] In this system, the node auto-discovery layer scans the functional modules in the runtime environment during system initialization and dynamically loads processing nodes based on the functional module metadata. The namespace resolution layer organizes and resolves conflicts at the namespace level for the loaded processing nodes to determine candidate nodes for subsequent processing. The dependency analysis layer analyzes the dependencies between candidate nodes and generates the node execution hierarchy accordingly. The parallel execution scheduling layer schedules and controls the execution process of valid nodes based on the generated hierarchical execution plan.

[0068] It should be noted that the system operation process includes an initialization phase and an execution phase. The initialization phase is used to discover and register processing nodes, while the execution phase is used to perform dependency analysis and parallel scheduling based on task configuration.

[0069] Below, we will combine Figure 3 The system architecture diagram shown illustrates in detail the modular code processing method based on dynamic plugin discovery in Agent development in this example embodiment, further explaining the specific implementation process of the method in the above system architecture.

[0070] In step S210, during the Agent system initialization phase, the functional module metadata in the current operating environment is obtained, the processing nodes are dynamically loaded based on the functional module metadata, and the processing nodes are registered as a candidate node set.

[0071] Depend on Figure 3 As can be seen, after the function package is installed, the system enters the node auto-discovery layer. In the node auto-discovery layer, the current runtime environment is first scanned; that is, the installed packages are scanned to obtain the corresponding function module metadata. This function module metadata describes the processing node information within the function module that can be recognized and invoked by the system, including but not limited to the processing node's identification information, entry point description, and basic information related to the loading of the processing node.

[0072] After obtaining the functional module metadata, the Entry Points are further read to retrieve the node entry information from the functional module metadata. Based on the node entry information, dynamically loaded classes are executed, and the interface specifications of the loaded node classes are validated. Upon successful validation, the corresponding processing node is registered in the namespace, thus forming candidate node resources and providing the upper layer with the ability to query node classes. In this way, available processing nodes in the current runtime environment can be automatically aggregated during the initialization phase, ensuring that the candidate node set reflects the actual available functional modules in the runtime environment.

[0073] In this step, the candidate node set is used to store the processing nodes obtained through dynamic loading, and serves as the basic data source for subsequent node selection, dependency analysis, and execution scheduling. By completing the dynamic loading and registration of processing nodes during the system initialization phase, the reliance on pre-configured or manually maintained node registration information can be avoided, thereby improving the system's adaptability in scenarios where functional modules are added or updated.

[0074] For example, refer to Figure 4 As shown, processing nodes can be registered as candidate node sets according to steps S410 to S430: Step S410: Read node entry information from the functional module metadata.

[0075] Functional module metadata describes the functional modules installed in the runtime environment. It records the entry descriptions of the processing nodes within the functional module that can be recognized by the system. The node entry information in the functional module metadata indicates the location and loading method of the processing node. For example, the node entry information may include basic information such as node identifier, namespace identifier, and node classpath used to locate the node implementation. By reading the node entry information, the system can obtain the entry basis required for subsequently importing node implementations.

[0076] Step S420: Based on the node class path indicated by the node entry information, import the node class corresponding to the node class path.

[0077] A node classpath is used to uniquely indicate the location or identifier of the code containing the node implementation. The system can perform an import operation based on the node classpath to obtain the node class corresponding to that classpath. The import operation can be understood as loading the implementation corresponding to the node class into a context accessible to the currently running process, enabling the system to access the node class's type declaration information and its executable logic. It should be noted that the import operation can be implemented using dynamic loading methods common in the art, and this disclosure does not limit it to this method.

[0078] Step S430: Verify whether the node class conforms to the preset processing node interface specification, and dynamically load the node class to generate the processing node when the verification is successful, and register the processing node to the candidate node set.

[0079] The processing node interface specification is used to constrain the basic structural requirements that processing nodes should meet, so that processing nodes from different sources can be scheduled within a unified framework. For example, the processing node interface specification may require node classes to have necessary elements such as preset input type pattern declarations, output type pattern declarations, and node execution entry points.

[0080] The system can perform interface compliance verification on the node class imported in step S320. The verification can include checking whether the node class inherits a preset base class or implements a preset interface, whether it contains type pattern declarations for input and output constraints, and whether it has a callable execution function.

[0081] If the verification passes, the system dynamically loads the node class to generate a processing node instance and registers the processing node instance to the candidate node set. This allows for subsequent type consistency checks on the connection relationships between candidate nodes and the selection of a valid node set. If the verification fails, the corresponding node class is excluded from the candidate node set, and information related to the verification failure is recorded for later investigation.

[0082] Through steps S410 to S430 above, the system can complete the discovery, import, verification and registration of processing nodes based on the functional module metadata during the initialization phase, so that the candidate node set reflects the range of processing nodes available in the current operating environment, thereby providing a foundation for subsequent node connection verification, dependency analysis and execution scheduling.

[0083] Further, refer to Figure 5 The diagram illustrates a process for automatic node discovery and registration. After system startup, the system queries the package manager to obtain information on installed packages in the current runtime environment, thus forming a list of installed packages. Subsequently, the installed packages are traversed, and the metadata of the corresponding functional modules for each package is read during the traversal.

[0084] Based on the read functional module metadata, entry points are extracted, that is, node entry information is obtained from the functional module metadata to determine the discoverable nodes provided by the functional module. The extracted node entry information is then filtered and located, such as locating the entry points of groups, to determine the set of entry points that may correspond to the processing nodes.

[0085] After locating the entry point, each entry point is traversed, and node parsing and loading operations are performed for each entry point. Specifically, during the traversal, firstly, the entry point name is parsed to obtain the node identifier and namespace information corresponding to the entry point. For example, for `@common / git_diff`, `@common` represents the namespace to which the node belongs, used to characterize the node's source or priority category, and `git_diff` is the node identifier, used to identify the specific processing node under this namespace. Next, the classpath indicated by the entry point is parsed, and the corresponding node class is dynamically imported into the runtime environment based on the classpath.

[0086] After importing the node class, the system further determines whether the node class inherits from the preset base node type, BaseNode. If the node class does not inherit from BaseNode, the system will not load the node as an available processing node, but will skip it and record a warning message to indicate that the processing node does not conform to the preset processing node interface specification. If the node class inherits from BaseNode, the system will continue to perform subsequent verification processing on the node.

[0087] After the node class inheritance relationship is validated, it is further determined whether the node declares an input type pattern and an output type pattern. If the node class does not declare an input type pattern and an output type pattern, the process is skipped and a warning message is logged. If the node class correctly declares an input type pattern and an output type pattern, the node is registered in the corresponding namespace to complete the registration and categorized storage of the nodes.

[0088] After the above traversal process has been completed at all entry points, the system completes the construction of the node registry, thereby forming a set of candidate nodes for use in subsequent namespace resolution, dependency analysis, and execution scheduling phases.

[0089] Through the above process, the system can automatically discover available processing nodes based on the metadata of installed function packages during the initialization phase, and complete node registration while meeting interface specifications and type declaration requirements, providing basic node resources for subsequent modular code processing. This process can be completed automatically at system startup without manual configuration, thus supporting plug-and-play expansion of functional modules and providing stable basic node resources for subsequent modular code processing.

[0090] Additionally, it should be noted that the functional packages used to generate processing nodes are managed as plugins in this disclosure. By moving the plugin registration capability forward to the system initialization phase, plugins no longer exist as independent, passively triggered execution units, but rather participate in the initialization and construction process as part of the overall system structure. Based on this design, plugin namespace resolution, type validation, and dependency analysis can be completed collaboratively during the initialization phase and continuously constrained by the initialization results during subsequent operation, forming a structured, strongly coupled closed-loop management mechanism.

[0091] Therefore, this disclosure can detect problems such as plugin conflicts, type mismatches, or missing dependencies in advance before the system runs, reduce the risk of anomalies during the runtime, improve the consistency and maintainability of plugin management, and avoid problems such as the fragmentation of plugin lifecycle and unpredictable system behavior caused by simply using the Entry Points mechanism, thereby significantly improving the overall stability and scalability of the system.

[0092] In step S220, the data connection relationships between each candidate node in the candidate node set are checked for type consistency to obtain the valid node set.

[0093] In the exemplary implementation of this disclosure, the candidate node set includes multiple processing nodes obtained through dynamic loading. Each processing node declares an input type pattern and an output type pattern corresponding to its execution logic. The input type pattern and output type pattern are used to characterize the type constraints of the processing node during the data interaction process.

[0094] For example, the potential data connection relationships between candidate nodes can be verified based on the input and output type patterns declared by each processing node in the candidate node set. For instance, a matching judgment can be made between the output type pattern of one processing node and the input type pattern of another processing node to determine whether the data connection relationship meets the consistency requirements at the type level. Specifically, the type consistency check is used to determine whether the data transmitted from the upstream processing node to the downstream processing node conforms to the constraints of the downstream processing node on the input data type.

[0095] For example, a processing node used to analyze text content can declare its input type pattern as a text data type and its output type pattern as a structured analysis result type. Correspondingly, the data connection between the upstream processing node and the upstream processing node will only pass the type consistency check if the upstream processing node's output type pattern matches the text data type.

[0096] After completing the above verifications, candidate nodes are filtered based on the type consistency verification results. Candidate nodes that participate in at least one data connection relationship that passes the type consistency verification are identified as valid nodes, thus constructing a set of valid nodes. This set of valid nodes represents the range of processing nodes that meet the collaborative processing conditions at the data type level and serves as the basis for subsequently constructing the node dependency graph and generating the execution plan.

[0097] This step verifies the type consistency of the data connection relationships between candidate nodes, which can eliminate candidate nodes that do not meet the type constraints before the execution process is organized. This ensures that the nodes participating in the scheduling and execution have a consistent basis in terms of data interaction, thereby improving the stability and controllability of the overall processing flow.

[0098] In some example implementations, before performing type consistency checks on the data connection relationships between candidate nodes in the candidate node set, namespace-level organization and conflict resolution can also be performed on the candidate nodes.

[0099] Specifically, the candidate nodes in the candidate node set are first classified and stored according to their namespaces. The namespace is used to represent the source or logical affiliation of the candidate node. By classifying the candidate nodes according to their namespaces, it is possible to distinguish candidate nodes from different sources but with the same node identifier in the candidate node set.

[0100] After completing the categorized storage, a node identifier conflict check is performed on the candidate node set. If multiple candidate nodes with the same node identifier are detected, the candidate nodes are compared according to the priority order of their namespaces to determine the target candidate node that will not be replaced. The namespace priority order indicates the priority relationship between different namespaces, ensuring that a candidate node conforming to the expected rules is selected when a naming conflict occurs.

[0101] Additionally, if a candidate node in a low-priority namespace is detected to have the same node identifier as a candidate node in a high-priority namespace, the candidate node in the low-priority namespace is prevented from registering, and registration error information corresponding to the candidate node in the low-priority namespace is returned.

[0102] Therefore, candidate nodes in low-priority namespaces cannot overwrite or replace candidate nodes in high-priority namespaces in the system, thus preventing non-core functional packages or untrusted extensions from interfering with the system's core nodes. This control process occurs during system initialization or plugin discovery, completing the judgment and blocking before nodes enter the actual execution or scheduling process, ensuring that conflicting nodes do not meet the conditions for taking effect.

[0103] Therefore, the namespace priority mechanism in this disclosure is not merely a simple naming convention, but rather a constraint in the plugin registration process that participates in system decision-making, binding node registration behavior to the overall system structure, security boundaries, and stability requirements. As a result, while supporting plug-and-play plugin extensions, the system effectively ensures the integrity and consistency of core capabilities, reduces the risk of system anomalies caused by extension conflicts or malicious replacements, and improves the stability and security of system operation.

[0104] By using the above method, conflicts between candidate nodes with the same name can be resolved before the candidate nodes enter the type consistency verification process, thereby avoiding the problem of uncertainty or instability of the candidate node set due to node identifier conflicts.

[0105] By performing the namespace classification and priority judgment steps described above before type consistency verification, it can be ensured that the candidate nodes participating in subsequent data connection verification have a unique basis at the node identifier level, providing a stable candidate range for the subsequent construction of a valid node set.

[0106] In the namespace resolution process, in order to accurately determine the processing node to be used when there are multiple candidate nodes, the node reference information is first obtained and then parsed to determine whether the node reference information contains a namespace identifier.

[0107] Specifically, the process obtains node reference information that indicates the processing node to be used, and parses this information to determine whether it contains a namespace identifier. The node reference information indicates the candidate node to be used in subsequent processing flows, and can be generated from task configuration, processing flow description, or system call requests. For example, the node reference information can be "@namespace / node", or it can be namespace.node, namespace / node_name, etc., as long as it can uniquely and explicitly locate the corresponding target node implementation during the parsing phase, thereby avoiding naming conflicts caused by nodes with the same name in different namespaces or different modules.

[0108] This disclosure does not limit the specific representation of node reference information, but emphasizes that it has a fully defined semantic characteristic, that is, the target node can be directly determined through node reference information without relying on default namespace priority or implicit resolution rules.

[0109] When the node reference information includes a namespace identifier, candidate nodes are no longer matched solely based on the node identifier. Instead, the set of candidate nodes is located based on the combination of the namespace identifier and the node identifier, thereby determining the target candidate node corresponding to the node reference information. By using both the namespace identifier and the node identifier as the basis for determining candidate nodes, the required candidate node can be directly located even when candidate nodes with the same name exist.

[0110] By explicitly specifying the namespace identifier in the node reference information through the above steps, the target candidate node can be determined directly during the node resolution stage without having to execute the default conflict resolution process based on the namespace priority order. This makes the selection process of candidate nodes more explicit and controllable, and is suitable for application scenarios that require precise reference to specific nodes.

[0111] Still referencing Figure 3 The architecture diagram shown is for Figure 3 The namespace resolution layer first resolves task configurations or node references, and then performs a priority search by combining node identifiers and namespace information. If nodes with the same name exist, the system selects based on namespace priority rules, such as priority 3 → priority 2 → priority 1, while simultaneously performing an overwrite protection check to prevent lower-priority nodes from replacing higher-priority nodes. After resolution, the system determines the target node that will actually participate in subsequent processing and provides the query node class to the upper layer.

[0112] refer to Figure 6The diagram illustrates a namespace resolution process. When the system receives a request to execute a node, it first obtains the corresponding node reference information, such as "@namespace / node". It then determines whether the node reference information contains a separator used to distinguish between the namespace and the node identifier, such as " / ". If the node reference information contains a separator, it splits the node reference information into two parts: the namespace name and the node name.

[0113] After parsing, the system further checks if the parsed namespace exists. If the namespace does not exist, an error message indicating that the namespace does not exist is returned. If the namespace exists, the system continues to check if a node corresponding to the node name exists within that namespace. If the node does not exist, an error message indicating that the node does not exist is returned; if the node exists, the corresponding node class is directly returned as the target node for subsequent processing.

[0114] When the node reference information does not contain a separator, namespace resolution is not performed on the node reference. Instead, the search order is initialized according to a pre-defined namespace priority, meaning that the existence of a node corresponding to the node identifier is checked in each namespace sequentially. If a corresponding node is found in the current namespace, the node class is returned as the target node. If no corresponding node is found in the current namespace, the search continues to check if there is a next namespace to search. If all namespaces have been searched and no corresponding node is found, an error message indicating that the node does not exist is returned.

[0115] pass Figure 6 The process shown can determine the target node according to a unified resolution rule, regardless of whether the node reference contains namespace information or not. It supports both direct resolution of nodes with explicitly specified namespaces and automatic search based on priority order when no namespace is specified, thereby providing accurate node resolution results for subsequent node dependency analysis and execution scheduling.

[0116] refer to Figure 7 The diagram illustrates a flowchart of an overlay protection mechanism. Figure 7 In the process, when the system attempts to register a node, it first obtains the node identifier, node class, and corresponding namespace information of the node to be registered, and then initiates a node registration request. Subsequently, it iterates through the node information currently existing in each namespace to check if there is a registered node with the same node identifier as the node to be registered.

[0117] During the traversal, it is checked whether a node with the same name is found. If no node with the same name is detected, the node registration operation is performed directly, and the registration process is completed. If a node with the same name is detected, it is further checked whether the node with the same name already exists in a preset high-priority namespace (such as @common).

[0118] If a node with the same name exists in a higher-priority namespace, registration of that node as an overwrite is refused, and an exception message indicating that overwriting is not allowed is returned to prevent a node in a lower-priority namespace from replacing a node in a higher-priority namespace. If a node with the same name does not exist in a higher-priority namespace, overwrite registration is allowed, and a corresponding warning message is logged upon completion of the node registration to indicate that node overwriting has occurred.

[0119] By overriding the protection process, the system can effectively manage nodes with the same name during the node registration phase. Under controlled conditions, it allows overriding of non-high-priority nodes while ensuring that high-priority nodes are not accidentally replaced, thus achieving a balance between system stability and scalability.

[0120] In some example implementations, references Figure 4 As shown, type consistency verification can be performed according to steps S410 and S420: Step S410: For each candidate node in the candidate node set, read the input type mode and output type mode declared by each candidate node.

[0121] Step S420: Based on the input type pattern and the output type pattern, perform static type verification on the data connection relationship between each candidate node, and construct a valid node set according to the candidate nodes corresponding to the verified data connection relationships.

[0122] In this implementation, each candidate node corresponds to a processing node, which declares its input type pattern and output type pattern corresponding to its execution logic when registered as a candidate node. The input type pattern characterizes the constraints on data type and data structure required by the candidate node when receiving data, while the output type pattern characterizes the type and structure of the data generated by the candidate node when outputting data. The system obtains the type basis for determining whether the data connection relationship between candidate nodes meets the type consistency requirement by reading the input type pattern and output type pattern.

[0123] The system iterates through the possible data connections between candidate nodes and matches the output type patterns of upstream candidate nodes with the input type patterns of downstream candidate nodes to determine whether the data connection meets the consistency condition at the type level. Static type validation is performed before the actual execution of nodes, and is used to identify data connections that do not meet type constraints in advance during the execution process organization phase.

[0124] After completing static type validation, the system filters candidate nodes based on the validation results and identifies candidate nodes that participate in at least one data connection relationship that passes static type validation as valid nodes, thus constructing a set of valid nodes. This set of valid nodes represents the range of candidate nodes that satisfy data interaction constraints at the type level and serves as the basis for subsequently constructing the node dependency graph and generating the execution plan.

[0125] By executing steps S410 and S420, the data connection relationship between candidate nodes can be judged for type consistency without actually running the processing nodes. This eliminates candidate nodes with mismatched types before the execution phase, making the subsequent processing flow more stable and controllable.

[0126] In the example implementation of this disclosure, static type validation is used to initially screen the data connection relationships between candidate nodes. However, since static validation only judges based on the type pattern declared by the node, there may still be cases where the data passed in during actual operation is inconsistent with the declared type. Based on this, dynamic type validation of the input data of valid nodes is further introduced.

[0127] Specifically, when a valid node is scheduled for execution, the input data to be passed to the valid node is obtained, and dynamic type validation is performed on the input data based on the input type pattern declared by the valid node during the registration or loading phase. Dynamic type validation is used to determine during runtime whether the actual type or data structure of the input data conforms to the input type requirements of the corresponding valid node.

[0128] If the dynamic type validation result indicates that the input data meets the input type pattern, the input data is allowed to enter the valid node and the corresponding processing logic continues to be executed. If the dynamic type validation result indicates that the input data does not meet the input type pattern, the input data can be prevented from entering the valid node, and the validation result can be processed accordingly, such as exception marking, error recording, or status updating. This disclosure does not limit the processing methods.

[0129] Furthermore, in the event of a dynamic type validation failure, the current execution flow of the valid node is terminated, and its dependent nodes are prevented from entering the scheduled execution flow. That is, in this case, no default processing result is returned to the valid node, and downstream nodes that depend on the output of the valid node are not triggered or continue to execute, thereby preventing data that does not conform to type constraints from propagating in the modular execution chain.

[0130] By using a phase-based control approach, dynamic type validation results are used to control the execution and scheduling process of nodes, ensuring that the actual execution process of a node conforms to its declared type constraints. This avoids processing exceptions or uncertain execution results caused by input type mismatches, thereby improving the stability and controllability of the modular code processing flow during the runtime phase.

[0131] refer to Figure 8 The diagram illustrates a type safety mechanism. The process can be divided into two interconnected phases: a loading phase and an execution phase. The loading phase performs static type constraint checks before node execution, while the execution phase dynamically verifies and controls the actual input data during node operation.

[0132] At the start of the loading phase, the input and output type patterns declared by the processing nodes are read first. Based on the input and output type patterns, a data flow mapping relationship is established between nodes to describe the correspondence between the output of upstream nodes and the input of downstream nodes.

[0133] After the data flow mapping is constructed, static type validation is performed on the data connection relationships to determine whether the data transmission between nodes meets the preset consistency constraints at the type level. If the static type validation fails, a static type error is identified, and the subsequent execution process is terminated to prevent nodes with mismatched types from entering the execution phase. If the static type validation passes, the processing flow is allowed to enter the execution phase.

[0134] During the execution phase, the actual input data to be passed to the node is first constructed based on the input requirements of the current node and the output results or initial input data of the upstream node. Then, dynamic type validation is performed on the actual input data to determine whether it conforms to the input type pattern declared by the current node.

[0135] When the dynamic type validation passes, the execution logic of the corresponding processing node is triggered. After the node finishes execution, the node output result is recorded, and the current processing context is updated for subsequent nodes to continue using.

[0136] When dynamic type validation fails, the processing node is not executed directly. Instead, the exception is handled according to a preset processing strategy. The processing strategy may include stopping execution, skipping the current node, or ignoring the exception and continuing with subsequent processing. After the exception handling is completed, the processing context state is also updated.

[0137] After each node is processed, it is determined whether there are still any unexecuted processing nodes. If there are still unexecuted nodes, the process returns to the input construction step and continues to the next node; otherwise, the current processing flow ends.

[0138] By combining the loading and execution phases, potential type inconsistencies can be detected in advance through static type validation before node execution, and dynamic type validation can constrain and control the actual input data during node execution. This ensures the continuity of the execution process while improving the security and controllability of the modular code processing.

[0139] In step S230, a node dependency graph is constructed based on the dependency relationships between each valid node in the set of valid nodes, and the node dependency graph is analyzed and processed to generate a hierarchical execution plan.

[0140] After determining the set of valid nodes, the data dependencies between them are identified. Data dependencies characterize the constraints between different valid nodes in terms of data transmission or execution order. For example, the execution of a certain valid node requires the output of a preceding valid node as input.

[0141] Optionally, a node dependency graph can be constructed based on data dependencies. Valid nodes serve as nodes (vertices) in the graph, and the dependencies between nodes serve as directed connections (edges), thus constructing the node dependency graph.

[0142] After constructing the node dependency graph, it is analyzed to determine the relative positions of each valid node within the overall execution flow. Through this analysis, the system can identify the sequential dependencies between valid nodes and determine the dependency level for each node. The dependency level characterizes the sequential relationship of valid nodes in the execution order.

[0143] Based on this, valid nodes are grouped according to their dependency levels. Valid nodes with the same dependency level are grouped into the same execution level, and a hierarchical execution plan consisting of multiple execution levels is generated according to the order of dependency levels. It can be understood that the hierarchical execution plan describes the execution order of valid nodes and the range of nodes that can be executed in parallel, and serves as the basis for subsequent node scheduling and execution.

[0144] This disclosure, by constructing a node dependency graph and generating a hierarchical execution plan, ensures that the execution organization of effective nodes is consistent with their actual dependencies, thereby providing a clear and controllable execution structure for subsequent execution phases.

[0145] In some example implementations, after constructing the node dependency graph, a traversal analysis can be performed on the node dependency graph to detect whether there are circular dependencies.

[0146] Specifically, a graph traversal operation is performed on the node dependency graph, and the node identifiers on each traversal path are recorded during the traversal. The traversal path is used to represent the access sequence formed by starting from a certain valid node and progressively visiting other valid nodes along the dependencies in the node dependency graph.

[0147] During the traversal, the node identifiers on the traversal path are checked. If a duplicate node identifier is detected in any traversal path, it indicates that a circular dependency relationship has formed between the corresponding valid nodes in the traversal path. At this time, the valid nodes with duplicate node identifiers are identified as circular dependency nodes, and the dependency path information containing the circular dependency nodes is output to indicate the node relationship that causes the circular dependency.

[0148] By performing traversal analysis after the node dependency graph is constructed, circular dependencies between nodes can be identified before generating hierarchical execution plans, thereby avoiding problems such as the inability to generate subsequent execution plans or abnormal execution processes caused by circular dependencies.

[0149] Conversely, if it is confirmed that there are no duplicate node identifiers in each traversal path, that is, there are no circular dependencies in the node dependency graph, the execution relationship of the effective nodes is further analyzed based on the node dependency graph to generate a hierarchical execution plan.

[0150] This embodiment allows for the organization of valid nodes into a hierarchical execution structure, ensuring that the execution order of nodes is consistent with their dependencies, while providing a clear hierarchical division for subsequent parallel execution.

[0151] refer to Figure 3 As shown, for Figure 3 The dependency analysis layer first constructs a dependency graph, generating a node dependency structure based on the data dependencies between nodes. Then, loop detection is performed to check for circular dependencies among the nodes. If no circular dependencies are found, topology sorting is further performed, and a hierarchical execution plan is generated accordingly.

[0152] refer to Figure 9The diagram illustrates a process for dependency analysis and topology sorting. In this process, the system first receives a list of nodes input by the task configuration, where each node represents a processing node to be executed. After obtaining the node list, a corresponding node dependency graph is constructed based on the declared data dependencies between the processing nodes, and an in-degree table is simultaneously generated to record the in-degree information of the nodes, providing the basic data structure support for subsequent dependency analysis.

[0153] After constructing the node dependency graph, a depth-first search is performed on it to detect circular dependencies. During the depth-first search, the set of visited nodes (visited) and the stack structure rec_stack corresponding to the current recursive path are maintained to record node identification information on the traversal path. When a node is detected to reappear in the current traversal path during the traversal, a circular dependency is determined to exist in the node dependency graph, i.e., a cycle is found.

[0154] If a circular dependency is detected, the subsequent execution plan generation process is stopped, a circular dependency error is thrown, and dependency path information containing the circular dependency node is output to help locate abnormal situations in the dependency configuration.

[0155] If no circular dependencies are detected during the depth-first search, it indicates that the node dependency graph satisfies the structural properties of a directed acyclic graph. In this case, based on the node dependency graph, the Kahn algorithm is used to hierarchically partition each processing node. Specifically, the dependency level of each node in the execution order is determined by progressively removing nodes with an in-degree of zero and updating the in-degree of their successor nodes.

[0156] After topology sorting is completed, processing nodes with the same dependency level are grouped into the same execution level based on the determined dependency level, thereby generating a hierarchical execution plan consisting of multiple execution levels. Finally, the hierarchical execution plan is output for use in the subsequent parallel execution scheduling phase.

[0157] pass Figure 9 The process shown can automatically generate a hierarchical execution plan that satisfies dependency constraints while ensuring the correctness of node dependencies, thus providing a foundation for safe parallel execution within the same execution level.

[0158] In step S240, each effective node is scheduled according to the hierarchical execution plan, and each effective node is executed in parallel within the same execution level.

[0159] During the execution phase, the system uses a hierarchical execution plan as the scheduling basis, organizes the execution flow of effective nodes according to the order of execution levels, and performs parallel processing on effective nodes within the same execution level, provided that dependency constraints are met.

[0160] This approach enables coordinated scheduling of the execution of multiple valid nodes while ensuring consistency between the execution order and node dependencies. This allows the execution process to both adhere to dependency constraints and enable parallel execution. The scheduling process spans all execution levels within the hierarchical execution plan until all valid nodes have been executed.

[0161] For example, refer to Figure 10 As shown, step S240 can be further derived from steps S1010 and S1020: Step S1010: Determine the target execution level to be executed in sequence according to the order of each execution level in the hierarchical execution plan.

[0162] In this implementation, the hierarchical execution plan has divided each valid node into multiple execution levels according to its dependencies, and the sequential relationship between each execution level is clearly defined. During the execution phase, the system sequentially traverses each execution level according to the hierarchical execution plan to determine the target execution level that needs to be executed.

[0163] Step S1020: For each valid node in the target execution level, trigger the corresponding node execution operation, and perform parallel scheduling and execution of each valid node within the same target execution level. After each valid node in the target execution level has completed its execution, switch to the next execution level to continue execution.

[0164] Since there are no dependency constraints between valid nodes within the same target execution level, the system can simultaneously schedule multiple valid nodes to execute their corresponding processing logic, provided that the dependency conditions are met. After each valid node in the target execution level has completed its execution, the system switches to the next execution level and continues to schedule and execute valid nodes in the next execution level in the same manner, until all execution levels in the hierarchical execution plan have completed their execution.

[0165] This embodiment can transform a hierarchical execution plan into an actual execution process, ensuring that the execution order of effective nodes remains consistent with their dependencies, while achieving parallel execution within the same execution level, thereby completing the orderly scheduling and execution of effective nodes.

[0166] It should be noted that in step S1020, when multiple effective nodes are scheduled to execute in parallel within the same target execution level, concurrent execution resources can be allocated based on the task characteristic information of each effective node.

[0167] For example, when scheduling effective nodes in the target execution level, task characteristic information corresponding to each effective node can be obtained. Task characteristic information characterizes the computational characteristics or resource requirements of effective nodes during execution. This information is determined by the feature descriptions provided by the node during the declaration phase and the execution context information obtained during the runtime phase. For example, task characteristic information may include features reflecting the computational complexity of the node, or features reflecting the node's I / O (input / output) access behavior, such as computationally intensive features, data access frequency features, or external resource dependencies. Then, based on the task characteristic information, the effective nodes within the target execution level are analyzed, and combined with the hierarchical position of the target execution level in the directed acyclic graph, each effective node is allocated to different concurrent execution resources.

[0168] After the allocation of concurrent execution resources is completed, the concurrent execution resources are used to execute each effective node in parallel, so that multiple effective nodes within the same target execution level can participate in the processing at the same time.

[0169] In this way, the allocation process of concurrent execution resources is bound to the node's declared characteristics, runtime context, and hierarchical execution structure of the directed acyclic graph, enabling concurrent scheduling behavior to change dynamically with node characteristics and execution environment without requiring the user to explicitly specify the concurrency strategy.

[0170] It should be noted that concurrent execution resources can be configured according to actual implementation needs, and their specific form and scheduling method do not constitute a limitation on this disclosure.

[0171] refer to Figure 3 As shown, for Figure 3 The parallel execution scheduling layer parses the user-submitted task configuration and generates an execution plan based on the parsing results. During execution, it selects the specific concurrent execution method according to the concurrency strategy and schedules node execution through process pool management or thread pool management. After each node completes execution, it collects and synchronizes the execution results and returns them to the upper-layer process or the user side.

[0172] refer to Figure 11 As shown, a flowchart illustrating a hierarchical parallel execution process is presented. At the beginning of the execution phase, the hierarchical execution plan generated by the dependency analysis phase is received first. The hierarchical execution plan consists of multiple execution levels arranged in dependency order.

[0173] Upon receiving the hierarchical execution plan, the runtime environment required for parallel execution is initialized, including initializing thread pool or process pool resources for concurrent scheduling and result sets for storing execution results. After initialization, the execution plan is traversed layer by layer according to the order of execution levels in the hierarchical execution plan.

[0174] When processing a specific execution level, the system prepares the corresponding node input data for each valid node within that level, and parses and organizes the parameters required for node execution. Subsequently, based on the task characteristics of each node, the system selects appropriate execution pool resources to determine the specific parallel execution method for each node.

[0175] After selecting execution resources, the execution tasks corresponding to each node are submitted to the selected execution pool for execution. Simultaneously, the corresponding asynchronous execution result objects are recorded to facilitate unified management of the execution status of parallel tasks. Within this execution level, the system waits for the completion of asynchronous execution result objects corresponding to all nodes.

[0176] When a task execution failure is detected within this execution level, the corresponding execution error message is output, and the subsequent execution process is terminated to prevent the error from propagating to later levels. When no task execution failure is detected, the execution results of each node in this level are recorded, and the execution log is updated synchronously.

[0177] After processing the results of the current execution level, it is determined whether there is a next execution level. If there is a next execution level, the process switches to the next execution level to continue the above process; if there is no subsequent execution level, it is determined that all nodes at all levels have completed execution, and the overall execution result is returned, ending the current parallel scheduling process.

[0178] pass Figure 11 The processing method shown can schedule and execute multiple nodes in the same execution level in parallel while satisfying the node dependency constraints, and uniformly control and handle abnormal situations during execution, thereby improving overall execution efficiency and operational stability while ensuring the correctness of the execution order.

[0179] refer to Figure 12 The diagram illustrates a flowchart of a concurrency strategy selection logic. Specifically, it determines whether a node explicitly declares its type or whether its execution characteristics are clearly defined through node configuration, type annotations, or other means. If the node type cannot be directly obtained, key feature information related to the node's functionality is further extracted from the node declaration or task configuration to serve as the basis for subsequent resource matching.

[0180] After obtaining the task characteristic information of a node, the first step is to determine whether the node matches the characteristics of a CPU-intensive task. For example, when a node involves complex syntax tree analysis such as AST analysis, rule reasoning, or high-complexity computational operations, the node is identified as a CPU-intensive node and assigned to the process pool for execution, so as to make full use of multi-core processor resources and avoid the impact of global interpreter lock on execution efficiency.

[0181] When a node does not meet the characteristics of a CPU-intensive task, it is further determined whether the node matches the characteristics of an I / O-intensive task. For example, when a node mainly performs I / O-related tasks such as reading and writing files like Git files, accessing version control systems, network communication, or calling Large Language Models (LLMs), the node is identified as an I / O-intensive node and assigned to a thread pool for execution, in order to reduce thread switching overhead and improve resource utilization during I / O waiting periods.

[0182] If neither of the above two types of features is explicitly matched, a preset default execution strategy is adopted, and the node is scheduled for execution using a thread pool to ensure that the node can participate in the execution process smoothly without additional configuration.

[0183] Through the above-mentioned execution resource selection mechanism based on node characteristics, the system can dynamically select appropriate concurrent execution resources according to the actual execution characteristics of different nodes during the parallel scheduling phase, thereby improving overall scheduling efficiency and system resource utilization while ensuring execution correctness.

[0184] It should be noted that, in the specific implementation process, this disclosure is not limited to the specific technical means used in the foregoing embodiments, and the relevant technical means may be appropriately adjusted or replaced according to the actual application environment.

[0185] Regarding node discovery mechanisms, in addition to registering and discovering functional modules through Entry Points, automatic node discovery can also be achieved using a file system-based scanning method. For example, within a pre-defined directory path, module files conforming to naming rules are recursively scanned, and the corresponding node classes are dynamically loaded during the scanning process. The namespace to which a node belongs can be distinguished by the directory hierarchy. This method does not rely on metadata information provided by the package manager, making it suitable for runtime environments with limited plugin management. Its implementation still achieves automatic node registration and does not affect the fundamental design objective of this disclosure to reduce manual configuration through automatic discovery.

[0186] In dependency analysis, besides using the Kahn algorithm to process the node dependency graph, a depth-first traversal approach can also be used to analyze node dependencies and generate the execution order. While different analysis algorithms differ in their specific implementation processes, they all effectively analyze node dependencies and determine the execution hierarchy of nodes accordingly.

[0187] In terms of parallel execution scheduling, in addition to using thread pools or process pools for concurrent scheduling of nodes, coroutine scheduling can be adopted based on the characteristics of node tasks, or a distributed execution mechanism can be introduced to support task scheduling across computing nodes. The selection of the above concurrent execution methods only involves different implementation forms of execution resource management strategies and does not affect the overall design concept of parallel scheduling based on node characteristics disclosed in this publication.

[0188] Regarding type validation, in addition to using a specific data validation framework to validate node input and output data, runtime type checking mechanisms or other data validation tools can also be combined to validate node data types. Replacing different validation tools is an implementation-level change and does not affect the technical effectiveness of ensuring data consistency between nodes through type validation.

[0189] Regarding namespace design, the multi-level namespace structure shown in the aforementioned embodiments is merely illustrative. In actual implementation, the namespace hierarchy can be expanded or adjusted according to the system scale or node origin to achieve more granular priority management. The processing method still distinguishes and selects nodes with the same name based on namespace priority.

[0190] In summary, the aforementioned alternative implementations or extensions only involve different choices of specific technical means, and none of them change the core technical ideas of this disclosure in terms of automatic node discovery, namespace conflict resolution, dependency analysis, and parallel execution scheduling.

[0191] In this exemplary embodiment, a modular code processing system based on dynamic plugin discovery is also provided for agent development. (See references) Figure 13 As shown, the modular code processing system 1300 based on dynamic plugin discovery in Agent development includes a node discovery module 1310, a type verification module 1320, a dependency analysis module 1330, and a parallel scheduling module 1340, wherein: The node discovery module 1310 is used to obtain the functional module metadata in the current running environment during the Agent system initialization phase, dynamically load the processing nodes according to the functional module metadata, and register the processing nodes as a candidate node set. The type verification module 1320 is used to verify the type consistency of the data connection relationship between each candidate node in the candidate node set to obtain the valid node set. The dependency analysis module 1330 is used to construct a node dependency graph based on the dependency relationships between each valid node in the set of valid nodes, and to analyze and process the node dependency graph to generate a hierarchical execution plan. The parallel scheduling module 1340 is used to schedule each effective node according to the hierarchical execution plan and to perform parallel execution on each effective node within the same execution level.

[0192] The specific details of each module in the modular code processing system based on dynamic plugin discovery in the above Agent development have been described in detail in the corresponding Agent development method based on dynamic plugin discovery, so they will not be repeated here.

[0193] Exemplary embodiments of this disclosure also provide an electronic device capable of implementing the above-described method. Referring below... Figure 14 To describe an electronic device 1400 according to such an exemplary embodiment of the present disclosure. Figure 14 The electronic device 1400 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0194] like Figure 14 As shown, the electronic device 1400 can be represented as a general-purpose computing device. The components of the electronic device 1400 may include, but are not limited to: at least one processing unit 1410, at least one storage unit 1420, a bus 1430 connecting different system components (including storage unit 1420 and processing unit 1410), and a display unit 1440.

[0195] Storage unit 1420 stores program code that can be executed by processing unit 1410, causing processing unit 1410 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, processing unit 1410 can perform... Figure 2 The methods and steps in the text.

[0196] Storage unit 1420 may include readable media in the form of volatile storage units, such as random access memory (RAM) 1421 and / or cache memory (Cache) 1422, and may further include read-only memory (ROM) 1423.

[0197] Storage unit 1420 may also include a program / utility 1424 having a set (at least one) of program modules 1425, such program modules 1425 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0198] Bus 1430 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0199] Electronic device 1400 can also communicate with one or more external devices 1470 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1400, and / or any device that enables electronic device 1400 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1450. Furthermore, electronic device 1400 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1460. As shown, network adapter 1460 communicates with other modules of electronic device 1400 via bus 1430. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0200] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0201] refer to Figure 15 As shown, a program product 1500 for implementing the above-described method according to an embodiment of the present disclosure is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0202] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0203] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0204] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0205] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C#, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0206] Finally, the above preferred embodiments are only used to illustrate the technical solutions of this application and are not restrictive. Although this application has been described in detail, those skilled in the art should understand that changes in form and detail can be made without departing from the scope defined by the claims of this application. The dimensions in the drawings are not related to the specific physical object, and the physical object dimensions can be arbitrarily changed.

Claims

1. A modular code processing method for agent development based on dynamic plugin discovery, characterized in that, include: During the Agent system initialization phase, the functional module metadata of the current operating environment is obtained, the processing nodes are dynamically loaded according to the functional module metadata, and the processing nodes are registered as a candidate node set. Perform type consistency verification on the data connection relationships between each candidate node in the candidate node set to obtain the valid node set; Based on the dependencies between the valid nodes in the set of valid nodes, a node dependency graph is constructed, and the node dependency graph is analyzed and processed to generate a hierarchical execution plan. Each effective node is scheduled according to the hierarchical execution plan, and each effective node is executed in parallel within the same execution level.

2. The modular code processing method based on dynamic plugin discovery in Agent development according to claim 1, characterized in that, The step of dynamically loading processing nodes based on the functional module metadata and registering the processing nodes as a candidate node set includes: Read node entry information from the metadata of the functional modules; Based on the node class path indicated by the node entry information, import the node class corresponding to the node class path; Verify whether the node class conforms to the preset processing node interface specification, and dynamically load the node class to generate a processing node when the verification is successful, and register the processing node to the candidate node set.

3. The modular code processing method based on dynamic plugin discovery in Agent development according to claim 1, characterized in that, Before performing type consistency verification on the data connection relationships between candidate nodes in the candidate node set, the method further includes: The candidate nodes in the candidate node set are classified and stored according to the namespace to which the candidate nodes belong; In the case where multiple candidate nodes have the same node identifier, the target candidate node that will not be replaced is determined according to the priority order corresponding to the namespace.

4. The modular code processing method based on dynamic plugin discovery in Agent development according to claim 3, characterized in that, The method further includes: If a candidate node in a low-priority namespace is detected to have the same node identifier as a candidate node in a high-priority namespace, the registration of the candidate node in the low-priority namespace is prevented, and registration error information corresponding to the candidate node in the low-priority namespace is returned.

5. The modular code processing method based on dynamic plugin discovery in Agent development according to claim 3, characterized in that, The method further includes: Obtain node reference information and determine whether the node reference information contains a namespace identifier; If the node reference information includes a namespace identifier, the target candidate node corresponding to the node reference information is determined based on the combination of the namespace identifier and the node identifier.

6. The modular code processing method based on dynamic plugin discovery in Agent development according to claim 1, characterized in that, The step of performing type consistency verification on the data connection relationships between candidate nodes in the candidate node set to obtain a valid node set includes: For each candidate node in the candidate node set, read the input type mode and output type mode declared by each candidate node; Based on the input type pattern and output type pattern, static type verification is performed on the data connection relationship between each candidate node, and the effective node set is constructed according to the candidate nodes corresponding to the data connection relationships that pass the verification.

7. The modular code processing method based on dynamic plugin discovery in Agent development according to claim 6, characterized in that, The method further includes: Dynamic type validation is performed on the input data passed to the valid node to determine whether the input data satisfies the input type pattern declared by the valid node; If the verification fails, the current execution flow of the valid node is terminated, and the dependent nodes of the valid node are prevented from entering the scheduling execution flow.

8. The modular code processing method based on dynamic plugin discovery in Agent development according to claim 1, characterized in that, The step of constructing a node dependency graph based on the dependency relationships between valid nodes in the set of valid nodes includes: Identify the data dependencies between the valid nodes in the set of valid nodes; Based on the data dependencies, a node dependency graph is constructed with valid nodes as vertices and dependencies as edges.

9. The modular code processing method based on dynamic plugin discovery in Agent development according to claim 1, characterized in that, After constructing the node dependency graph, the method further includes: Perform a graph traversal operation on the node dependency graph, and record the node identifiers on each traversal path during the graph traversal process; If a duplicate node identifier is detected in any of the traversal paths, the valid node with the duplicate node identifier is determined to be a circular dependency node, and the dependency path information containing the circular dependency node is output.

10. The modular code processing method based on dynamic plugin discovery in Agent development according to claim 9, characterized in that, The step of analyzing and processing the node dependency graph to generate a hierarchical execution plan includes: If no duplicate node identifiers are detected in any of the traversal paths, the dependency level of each valid node in the node dependency graph is determined based on the node dependency graph; the dependency level is used to characterize the sequential relationship of each valid node in the execution order. Based on the dependency hierarchy, valid nodes with the same dependency hierarchy are divided into the same execution hierarchy, and a hierarchical execution plan consisting of multiple execution hierarchies is generated according to the order of the dependency hierarchy.

11. The modular code processing method based on dynamic plugin discovery in Agent development according to claim 1, characterized in that, The step of scheduling each of the effective nodes according to the hierarchical execution plan and performing parallel execution on each of the effective nodes within the same execution level includes: According to the order of each execution level in the hierarchical execution plan, the target execution level to be executed is determined sequentially; For each valid node in the target execution level, the corresponding node execution operation is triggered, and each valid node is scheduled to be executed in parallel within the same target execution level. After each valid node in the target execution level has completed its execution, the process switches to the next execution level to continue execution.

12. The modular code processing method based on dynamic plugin discovery in Agent development according to claim 11, characterized in that, The parallel scheduling and execution of each effective node within the same target execution level includes: Obtain the task feature information corresponding to each of the effective nodes; Based on the task characteristic information, each effective node is assigned to a different concurrent execution resource, and each effective node is executed through the concurrent execution resource.

13. A modular code processing system for agent development based on dynamic plugin discovery, characterized in that, include: The node discovery module is used to obtain the functional module metadata in the current running environment during the Agent system initialization phase, dynamically load processing nodes according to the functional module metadata, and register the processing nodes as a candidate node set. The type verification module is used to verify the type consistency of the data connection relationship between each candidate node in the candidate node set, and to obtain the valid node set. The dependency analysis module is used to construct a node dependency graph based on the dependency relationships between each valid node in the set of valid nodes, and to analyze and process the node dependency graph to generate a hierarchical execution plan. The parallel scheduling module is used to schedule each of the effective nodes according to the hierarchical execution plan and to perform parallel execution on each of the effective nodes within the same execution level.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program code instructions, which, when invoked by a processor, execute the method as described in any one of claims 1 to 12.

Images