A visual heterogeneous data stream operator packaging method and system

Abstract

The application relates to the technical field of software engineering, in particular to a visual heterogeneous data flow operator packaging method and system. An encapsulation intermediate layer is constructed between a bottom layer graph scheduling framework and a user-defined business operator layer, a variable parameter template is used to derive a template node class, input and output type and quantity are determined in a compiling period through input and output type parameter packages, and input synchronization mechanism and output distribution mechanism are automatically configured in a node initialization stage according to the same; in a running period, heterogeneous data is transmitted in the framework layer through a unified type-erased data carrier, is restored to strong type data in the template node, and the calculation result is packaged into a unified data carrier. The application eliminates the need for manually writing glue code such as port registration, queue creation and synchronization distribution, realizes plug-and-play of the operator and decoupling of the framework business, simultaneously prepositions port type checking to the compiling period, and balances development efficiency, type safety and execution performance.

Description

Technical Field

[0001] This invention relates to the field of software engineering technology, and in particular to a method and system for encapsulating visual heterogeneous data flow operators. Background Technology

[0002] In real-time data processing fields such as industrial machine vision, autonomous driving, and intelligent quality inspection, software systems typically need to process heterogeneous data streams from multiple sensors (such as cameras, LiDAR, encoders, etc.) and execute a series of operators (such as object detection, image segmentation, feature extraction, coordinate transformation, etc.) sequentially according to a predefined computational flow. The data dependencies between these operators form a directed acyclic graph (DAG), and the scheduling engine is responsible for driving the execution of each operator according to the topological order of the graph, thereby completing a complete computational pipeline from raw data to the final decision result.

[0003] Currently, to enable graph scheduling frameworks to uniformly manage heterogeneous operators, existing technologies typically employ two approaches. Approach 1 is a unified data container approach: The framework layer defines a generic data container type (such as void*, std::any, or a key-value dictionary). All operator input and output ports use this generic type for data transfer. Before executing business logic, operator developers must manually extract data from the container and perform type casting to restore the strongly typed data required by the business layer. Approach 2 is a hard-coded port approach: The framework requires each operator to explicitly declare the number of its input ports and the data type of each port, as well as the number of its output ports and the data type of each port during definition. The scheduling engine registers ports and routes data based on these declarations. When a new operator type needs to be integrated, developers must write specific adaptation code for each input-output combination to complete data packaging and unpacking.

[0004] However, in typical vision applications, a processing node often needs to wait for data from multiple upstream nodes to be available before triggering computation (e.g., image data and camera parameter data need to be strictly aligned in time series). Existing technologies lack built-in data synchronization mechanisms, requiring developers to maintain multiple queues and implement synchronization logic themselves. Even if some solutions provide synchronization methods, their synchronization logic is often mixed with business logic, reducing code readability and maintainability. Secondly, output port management is complex and system scalability is poor. When operators need to support multi-input synchronization or multi-output broadcasting, developers must manually write a large amount of repetitive glue code to complete queue synchronization, data copying and distribution, and port binding. When the number of output ports or data types change, the operator definition needs to be modified and the entire system recompiled, resulting in high system coupling, poor scalability, and the risk of runtime crashes due to type mismatches. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method and system for encapsulating visual heterogeneous data stream operators.

[0006] In a first aspect, embodiments of the present invention provide a method for encapsulating visual heterogeneous data stream operators. This method is used to construct an encapsulation intermediate layer between an underlying graph scheduling framework and a user-defined business operator layer. The method includes: Construct a node base class, which provides a port management interface for unified management of the input and output ports of operators; A variable-parameter template is used, with a node base class as the parent class, to derive a template node class, and a business calculation interface is set in the template node class. The template parameters of the template node class include an input type parameter package and an output type parameter package to determine the input type, output type, number of inputs, and number of outputs of the operator at compile time. The template node class contains a business calculation interface, where the input parameters of the business calculation interface correspond one-to-one with each input type in the input type parameter package, and the output parameters correspond one-to-one with each output type in the output type parameter package. During the node initialization phase, an input synchronization mechanism is automatically configured based on the number of operator inputs determined at compile time, so that node execution is triggered when all input data is ready; and an output distribution mechanism is automatically configured based on the number of outputs. In response to the node being triggered to execute, the unified type erased data carrier is obtained from the input synchronization mechanism, the data carrier is restored to the corresponding strongly typed input data according to the input type parameter package, the business computing interface is called to process the strongly typed input data, and the processing result is encapsulated into a unified type erased data carrier; The encapsulated data carrier is transmitted to the corresponding downstream port through the output distribution mechanism.

[0007] In conjunction with the first aspect, the steps for constructing the node base class include: Define a port query interface within the node base class. The port query interface includes an input port query function and an output port query function, which are used to return the communication node pointer of the corresponding input port and the communication node pointer of the corresponding output port according to the index, respectively. Define a port information maintenance structure within the node base class. The port information maintenance structure is used to record the name and type identifier string of each port. Define an array of physical carriers for output ports within the node base class. This array is used to store handles that point to the output distribution channels.

[0008] In conjunction with the first aspect, the steps for deriving template node classes using variable parameter templates, with the node base class as the parent class, include: By using the input type parameter package and the output type parameter package as variable template parameters of the template node class, the template node class can obtain all input types and all output types of the operator at compile time; The number of types in the input type parameter pack and the output type parameter pack are counted using compile-time operators respectively. The number of types in the input type parameter package is used as the number of input ports of the operator, and the number of types in the output type parameter package is used as the number of output ports of the operator. In the template node class, the business calculation interface is set. The input parameters of the business calculation interface correspond one-to-one with the types in the input type parameter package, and the output parameters correspond one-to-one with the types in the output type parameter package.

[0009] In conjunction with the first aspect, based on the number of operator inputs determined at compile time, an input synchronization mechanism is automatically configured to trigger node execution steps when all input data is ready, including: Based on the number of inputs, a separate message queue is created for each input port. The message queue is used to buffer the data received from the upstream node. Create a composite synchronization node and connect each message queue to its respective input port; The combined synchronization node listens to the status of the head of each message queue. When there are valid data frames at the head of each message queue, the combined synchronization node merges the valid data frames at the head of each message queue into a combined data unit and outputs it to trigger the node to execute. The combined data unit contains multiple data items arranged in the order of the input ports, with each data item corresponding to one input port.

[0010] In conjunction with the first aspect, the steps for automatically configuring the output distribution mechanism based on the output quantity determined at compile time include: Based on the number of outputs, multiple output distribution channels are constructed, and each of the multiple output distribution channels corresponds one-to-one with the multiple output ports of the current node; For each output distribution channel, connect the output distribution channel to the input port of the corresponding downstream node to establish a data path from the current node to the downstream node.

[0011] In conjunction with the first aspect, in response to a node being triggered for execution, the steps of obtaining a uniformly typed erased data carrier from the input synchronization mechanism, restoring the data carrier to the corresponding strongly typed input data based on the input type parameter packet, calling the business computing interface to process the strongly typed input data, and encapsulating the processing result into a uniformly typed erased data carrier include: Each data item is extracted from the combined data unit output by the input synchronization mechanism. The data item is a data carrier that is erased in a uniform type. Based on the order of the types in the input type parameter package, perform safe downcasting on each data item to restore it to its corresponding strongly typed input data. Based on the output type parameter package, create output object instances for each output type, and store each output object instance into a collection in order of output port; Call the business computing interface, pass strongly typed input data to the business computing interface, and write the processing result generated by the business computing interface to the output object instance; Each output object instance is encapsulated into a data carrier of a uniform type for erasure.

[0012] In conjunction with the first aspect, the steps of transmitting data to the corresponding downstream nodes through each output distribution channel include: Generate a corresponding compile-time index sequence based on the number of output ports of the current node; Using folded expressions, each data carrier is distributed to the output distribution channel corresponding to each output port of the current node according to the compile-time index sequence. The folded expression is expanded into multiple independent distribution operations during compilation, and each distribution operation corresponds to one output port of the current node. Each output distribution channel transmits the received data carrier to the input port of the connected downstream node.

[0013] Secondly, this application also provides a visual heterogeneous data stream operator encapsulation system, the system including a scheduling engine layer, an encapsulation intermediate layer, and a business operator layer; the encapsulation intermediate layer includes: The node base class module provides a port management interface to uniformly manage the input and output ports of operators; The template derivation module is used to derive template node classes from the node base class module using variable parameter templates, and to set up business calculation interfaces in the template node classes. The template parameters of the template node classes include input type parameter packages and output type parameter packages to determine the input type, output type, number of inputs, and number of outputs of the operator at compile time. The input parameters of the business calculation interface correspond one-to-one with each input type in the input type parameter package, and the output parameters correspond one-to-one with each output type in the output type parameter package. The synchronization configuration module is used to automatically configure the input synchronization mechanism during the node initialization phase, based on the number of input ports determined at compile time by the input type parameter package; the input synchronization mechanism is used to trigger node execution when all input data is ready; and to automatically configure the output distribution mechanism based on the number of outputs; The transmission encapsulation module is used to respond to the node being triggered to execute, obtain the data carrier with unified type erasure from the input synchronization mechanism, restore the data carrier to the corresponding strongly typed input data according to the input type parameter packet, call the business computing interface to process the strongly typed input data, and encapsulate the processing result into a data carrier with unified type erasure. The distribution configuration module is used to transmit the encapsulated data carrier to the corresponding downstream port through the output distribution mechanism.

[0014] In conjunction with the second aspect, the synchronous configuration module includes: The queue creation unit is used to create an independent message queue for each input port based on the number of inputs during the node initialization phase. The message queue is used to buffer data received from the upstream node. The combined synchronization unit is used to create a combined synchronization node, connecting each message queue to its respective input port. The combined synchronization node listens to the status of the head of each message queue. When there are valid data frames at the head of each message queue, it merges the valid data frames at the head of each message queue into a combined data unit for output, thereby triggering node execution. The combined data unit contains multiple data items arranged in the order of the input ports, with each data item corresponding to one input port.

[0015] In conjunction with the second aspect, the template-derived module also includes: The template parameter configuration unit is used to use the input type parameter package and the output type parameter package as variable template parameters of the template node class, so that the template node class can obtain all input types and all output types of the operator at compile time; The type statistics unit module is used to count the number of types in the input type parameter package and the output type parameter package respectively by using compile-time operators; The port number determination unit is used to determine the number of types of the input type parameter packet as the number of input ports of the operator, and the number of types of the output type parameter packet as the number of output ports of the operator. The interface setting unit is used to set the business calculation interface in the template node class. The input parameters of the business calculation interface correspond one-to-one with each input type in the input type parameter package, and the output parameters correspond one-to-one with each output type in the output type parameter package.

[0016] The embodiments of this invention bring the following beneficial effects: This application provides a visual heterogeneous data flow operator encapsulation method and system. This method is used to build an encapsulation intermediate layer between the underlying graph scheduling framework and the user-defined business operator layer. The method includes: constructing a node base class, which provides a port management interface for unified management of the operator's input and output ports; deriving a template node class from the node base class using a variable parameter template, and setting a business calculation interface in the template node class; wherein, the template parameters of the template node class include an input type parameter package and an output type parameter package to determine the operator's input type, output type, number of inputs, and number of outputs at compile time; the input form parameters of the business calculation interface are input... Each input type in the type parameter package corresponds one-to-one with the output type in the output type parameter package. During node initialization, an input synchronization mechanism is automatically configured based on the number of operator inputs to trigger node execution when all input data is ready. Similarly, an output distribution mechanism is automatically configured based on the number of outputs. In response to a node being triggered, a unified type-erased data carrier is obtained from the input synchronization mechanism. Based on the input type parameter package, the data carrier is restored to the corresponding strongly typed input data. The business computing interface is called to process the strongly typed input data, and the processing result is encapsulated into a unified type-erased data carrier. The encapsulated data carrier is then transmitted to the corresponding downstream port through the output distribution mechanism.

[0017] This invention constructs an encapsulated intermediate layer between the underlying graph scheduling framework and user-defined business operators. It utilizes variable parameter templates to derive template node classes, allowing the input and output type parameter packages to determine the input and output types and quantities of operators at compile time. Based on this, input synchronization and output distribution mechanisms are automatically configured during node initialization. At runtime, heterogeneous data is transmitted at the framework layer using a data carrier with unified type erasure, and then restored to strongly typed data within the template node. Finally, the calculation results are encapsulated into a unified data carrier. This invention eliminates the need for developers to manually write glue code for port registration, queue creation, and synchronization distribution logic, achieving plug-and-play functionality of operators and decoupling from framework business logic. Furthermore, by moving port type verification to compile time, it avoids runtime type conversion risks, balancing development efficiency, type safety, and execution performance.

[0018] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0020] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 A flowchart of a visual heterogeneous data stream operator encapsulation method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the encapsulation intermediate layer in a visual heterogeneous data stream operator encapsulation system provided in an embodiment of the present invention; Figure 3 This is an overall architecture diagram of a visual heterogeneous data stream operator encapsulation system provided in an embodiment of the present invention.

[0022] Figure label: 10 - Node base class module, 20 - Template derivation module, 30 - Synchronization configuration module, 40 - Distribution configuration module, 50 - Transmission encapsulation module. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] To facilitate understanding of this embodiment, the technical terms used in this application will be briefly introduced below.

[0025] Graph scheduling refers to the mechanism of decomposing complex computational tasks into multiple nodes (operators), connecting them with directed edges to form a graph, and scheduling the execution of nodes according to data dependencies. TBB (Threading Building Blocks) refers to the C++ library developed by Intel that supports parallel programming. This application relies on its flow_graph module for task concurrency and data flow scheduling. Type erasure refers to a programming technique that uses base class pointers (such as StreamMsg in this application, i.e., std::shared_ptr) uniformly in the underlying framework.<Message Base> This allows data to be transmitted using [a specific method / method], thereby masking the differences in data types across different business applications. Template metaprogramming refers to the technique of using C++ templates to perform computations or generate code at compile time. This application uses variadic templates to achieve adaptation of input and output of arbitrary length and type. Fold expression is a feature introduced in C++17 used to expand variable parameter packages at compile time. This application utilizes it to implement a message delivery mechanism for variable-length output ports.

[0026] After introducing the technical terms used in this application, the application scenarios and design concepts of the embodiments of this application will be briefly described below.

[0027] In real-time data processing scenarios such as industrial vision and autonomous driving, different operators require different types and quantities of input and output data, which leads to difficulties in operator access, complex synchronization logic, and poor system scalability.

[0028] Based on this, this application provides a method and system for encapsulating visual heterogeneous data stream operators.

[0029] Example 1 This application provides a method for encapsulating visual heterogeneous data flow operators. This method is used to build an encapsulation intermediate layer between the underlying graph scheduling framework and the user-defined business operator layer. For example... Figure 1 As shown, the method includes: S110, construct the node base class. The node base class provides a port management interface for unified management of the input and output ports of operators.

[0030] S120: Using a variable parameter template with a node base class as the parent class, a template node class is derived, and a business calculation interface is set in the template node class. The template parameters of the template node class include an input type parameter package and an output type parameter package to determine the input type, output type, number of inputs, and number of outputs of the operator at compile time. The input parameters of the business calculation interface correspond one-to-one with each input type in the input type parameter package, and the output parameters correspond one-to-one with each output type in the output type parameter package.

[0031] S130, during the node initialization phase, an input synchronization mechanism is automatically configured based on the number of operator inputs to trigger node execution when all input data is ready; and an output distribution mechanism is automatically configured based on the number of outputs.

[0032] S140, in response to the node being triggered to execute, obtains the data carrier with unified type erasure from the input synchronization mechanism, restores the data carrier to the corresponding strongly typed input data according to the input type parameter package, calls the business computing interface to process the strongly typed input data, and encapsulates the processing result into a data carrier with unified type erasure.

[0033] S150 transmits the encapsulated data carrier to the corresponding downstream port through the output distribution mechanism.

[0034] This invention constructs an encapsulation middleware layer between the underlying scheduling framework and business operators. It utilizes variable parameter templates to derive template node classes, enabling the input and output type parameter packages to determine the input and output types and quantities of operators at compile time. Based on this, it automatically configures input synchronization and output distribution mechanisms during node initialization. At runtime, the data carrier is downcast to strongly typed data before invoking the business calculation interface and transmitting the data to downstream nodes through the output distribution mechanism. This invention eliminates the need for developers to manually write glue code for port registration, queue creation, and synchronization distribution logic, achieving plug-and-play functionality for operators and decoupling from the framework's business logic. Furthermore, by moving port type verification to compile time, it avoids runtime type conversion risks, balancing development efficiency, type safety, and execution performance.

[0035] In this embodiment, the underlying graph scheduling framework (such as flow_graph based on TBB) is the physical infrastructure supporting the entire visual data processing pipeline. Its core functions include: managing the global directed acyclic graph topology, maintaining data dependencies between nodes, driving nodes to execute according to a preset topological order, and handling concurrent task scheduling and physical routing of data flows. Taking TBB's flow_graph module as an example, this framework provides basic building blocks such as graph, join_node, broadcast_node, and function_node, as well as underlying parallel mechanisms such as message queues and task stealing. However, to maintain versatility and high performance, the number and type of inputs and outputs of its nodes must be determined at compile time, and it cannot perceive the semantics and structure of specific business data. It only cares about how data is moved from one node to the next, not the business meaning of the data itself. This results in the native framework being unable to directly support the flexible access of business operators with variable numbers and heterogeneous types of inputs and outputs.

[0036] User-defined business operator layers are execution units that carry specific visual algorithm logic, such as object detection operators, image segmentation operators, and coordinate transformation operators. These operators are implemented by algorithm engineers or application developers. Their core characteristic is a strong dependency on data types; for example, image processing operators need to receive Image type data and output BoxList type data, while coordinate transformation operators need to receive CameraParam and PointCloud types and output TransformResult type data. Operator developers need to program in a strongly typed environment to fully utilize the compiler's type checking mechanism, ensuring code correctness and maintainability, while also gaining IDE support such as code completion and refactoring. However, operator developers are often not proficient in, and should not be concerned with, the details of the underlying scheduling framework, such as how to create queues, how to attach synchronization nodes, and how to register broadcast channels.

[0037] In this embodiment, the encapsulation middleware layer is built between the underlying graph scheduling framework and the user-defined business operator layer. This middleware layer provides derived base classes (such as TwoInputNodeBase) based on variable parameter templates to the business operators.<In1, In2, Out1, Out2> This allows operator developers to complete business logic development in a strongly typed environment simply by declaring input / output type parameter packages and overriding the compute interface. It shields the scheduling framework from differences in specific data types by using a unified type to erase the data carrier StreamMsg, enabling the scheduling engine to recognize only the unified carrier type without needing to understand the specific structure of the business data. The middle layer also handles the automatic configuration of port management, multi-input synchronization, and multi-output distribution. It automates all the glue code that developers previously had to write manually, such as port registration, queue creation, synchronization node mounting, and broadcast node instantiation, by having the framework complete all of this during compilation and node initialization. This allows operators to be integrated into the scheduling engine in a plug-and-play manner without modifying any core framework code.

[0038] In practical applications, user-defined business operator layers inherit the template node classes provided by the encapsulation middleware layer and rewrite the calculation interface. The encapsulation middleware layer derives the input and output types and quantities of operators at compile time based on the template parameters. Accordingly, during the node initialization phase, it automatically schedules basic components such as join_node and broadcast_node of the underlying graph scheduling framework to build the corresponding synchronization and distribution mechanisms. At runtime, data is processed by type erasure and restoration by the encapsulation middleware layer, flows in a unified carrier at the framework layer, and participates in calculations as strongly typed data at the business layer.

[0039] In conjunction with the first aspect, step S110 involves constructing a node base class, specifically including: S111 defines a port query interface within the node base class. The port query interface includes an input port query function and an output port query function, which are used to return the communication node pointers of the corresponding input port and the corresponding output port based on the index, respectively.

[0040] S112, Define a port information maintenance structure within the node base class. The port information maintenance structure is used to record the name and type identifier string of each port.

[0041] S113, Define an array of physical carriers for output ports within the node base class. The array of physical carriers for output ports is used to store handles pointing to output distribution channels.

[0042] Building a node base class essentially establishes a unified type entry point and port management foundation for the entire encapsulation middleware layer. This node base class does not directly carry business logic, but rather abstracts the interaction between operators and the scheduling engine into reading from input ports and writing to output ports by defining a standardized set of port management interfaces.

[0043] To achieve unified data flow from the perspective of the scheduling engine, it is first necessary to define a globally unified data flow type, StreamMsg, which is essentially a polymorphic smart pointer (i.e., std::shared_ptr). <messagebase>MessageBase is the common base class for all business data, thereby enabling automatic lifecycle management of data and type erasure of business data types at the framework level, ensuring that the scheduling engine only needs to recognize the unified carrier type and does not need to perceive the structural differences of specific business data.

[0044] Based on the determined StreamMsg type, the node base class further constructs the following port management structure: In step S111, a port query interface is defined within the node base class. This port query interface includes an input port query function (get_in_port(size_t index)) and an output port query function (get_out_port(size_t index)), which are used to return the communication node pointers for the corresponding input port and the corresponding output port, respectively, based on the index. These communication node pointers point to the physical communication nodes in the scheduling engine layer that are actually responsible for receiving or sending data, thereby providing a unified port access entry for the scheduling engine. This allows the scheduling engine to complete port addressing and data delivery without needing to be aware of the specific operator type information.

[0045] In step S112, the node base class defines a port information maintenance structure, which internally maintains two std::vectors. <portinfo>The vectors input_port_ and output_port_ record the names and type identifier strings of each input port and output port, respectively. The purpose of this structure is to provide queryable metadata for each port, which can be used by external modules for port traversal, debugging output, or runtime type verification. This makes up for the loss of some type information after type erasure and also provides metadata support for graphical connection configuration and port matching.

[0046] In step S113, the physical carrier array of the output ports is defined within the node base class, which is a std::vector maintained by the derived class. <std::unique_ptr<tbb::flow::broadcast_node <streammsg>The `out_ports` array is a dynamic array used to store smart pointer handles pointing to output distribution channels. This array is dynamically instantiated during the node initialization phase and bound to the output ports to ensure that each output port has an independent physical distribution channel, thereby enabling independent routing and concurrent broadcasting of multiple output data.

[0047] Through the construction of steps S111 to S113 above, and in conjunction with the type erasure capability of the StreamMsg global data carrier, the node base class completely unifies the data type erasure carrying, port query interface, port metadata maintenance, and output physical storage management, providing an inheritable port management skeleton for upper-level template derived classes.

[0048] In conjunction with the first aspect, step S120 utilizes a variable parameter template to derive a template node class from the node base class, specifically including: S121, the input type parameter package and the output type parameter package are used as variable template parameters of the template node class, so that the template node class can obtain all input types and all output types of the operator at compile time.

[0049] S122 uses compile-time operators to count the number of types in the input type parameter pack and the output type parameter pack, respectively.

[0050] S123, take the number of types in the input type parameter package as the number of input ports of the operator, and take the number of types in the output type parameter package as the number of output ports of the operator.

[0051] S124. Set up the business calculation interface in the template node class. The input parameters of the business calculation interface correspond one-to-one with the types in the input type parameter package, and the output parameters correspond one-to-one with the types in the output type parameter package.

[0052] In step S120, C++ template metaprogramming technology is used to completely place the operator's type information in the compile-time processing, providing a dual basis of type and quantity for the automatic configuration of subsequent synchronization and dispatch mechanisms. Specifically: In step S121, the input type parameter package and the output type parameter package are used as variable template parameters of the template node class, enabling the template node class to obtain all input types and all output types of the operator at compile time. Variable parameter templates are one of the core technologies of C++ template metaprogramming, allowing templates to accept any number and type of template parameters. This invention introduces this technology into the operator encapsulation intermediate layer, making the template parameter list of the template node class consist of two independent variable parameter packages, where Inputs... represents the set of data types for all input ports of the operator, and Outputs... represents the set of data types for all output ports of the operator. Through this declaration method, the operator obtains complete type description information at compile time, that is, how many types of input data the operator needs to receive, what those types are, and how many types of output data it needs to produce, and what those types are. This type information is completely preserved at compile time, laying the data foundation for subsequent type safety checks and automatic port configuration, completely eliminating the drawbacks of relying on runtime type identification or manual type registration in traditional solutions.

[0053] In step S122, the number of types in the input type parameter package and the output type parameter package are counted using compile-time operators. C++ provides the compile-time constant expression operator `sizeof...`, which accepts a parameter package as an operand and returns the number of type parameters in that parameter package at compile time; the result is a compile-time constant. This invention applies the `sizeof...` operator to the input type parameter package `Inputs...` and the output type parameter package `Outputs...` respectively, obtaining two compile-time constant values ​​representing the number of input types and the number of output types for the operator. Since `sizeof...` is evaluated at compile time, its result does not incur any runtime overhead, and changes in the number of types in the parameter packages do not increase runtime branching or loop iterations.

[0054] In step S123, the number of types in the input type parameter package is taken as the number of input ports of the operator, and the number of types in the output type parameter package is taken as the number of output ports of the operator. Since each input port carries a specific type of input data, and each output port outputs a specific type of output data, the number of input types is the number of input ports N for the operator, and the number of output types is the number of output ports M for the operator. These two values ​​are determined as constants at compile time and are used to guide the allocation and construction of port physical carriers in subsequent steps. Specifically, the number of input ports determines how many independent message queues need to be instantiated and how many input channels the combined synchronization node needs to listen to; the number of output ports determines how many broadcast nodes need to be instantiated as output distribution channels. This mapping between types and ports establishes an effective correspondence between operator declarations and framework configurations.

[0055] In step S124, a business computation interface is set in the template node class. The input formal parameters of the business computation interface correspond one-to-one with the types in the input type parameter package, and the output formal parameters correspond one-to-one with the types in the output type parameter package. In this embodiment, a business computation interface `compute` is declared in the template node class for users to override. The parameter list of this interface is completely determined by the template parameter package: the first half consists of constant references or smart pointers of various types in the input type parameter package (such as `const In1&`, `const In2&`, ...), serving as strongly typed input parameters for receiving upstream data; the second half consists of references or smart pointers of various types in the output type parameter package (such as `Out1&`, `Out2&`, ...), serving as strongly typed output parameters carrying the computation results. When users override this `compute` interface in their derived classes, they directly obtain a strongly typed parameter list that strictly corresponds to the input type parameter package and the output type parameter package. This ensures the type safety of the business computation interface during compilation and allows algorithm developers to obtain code completion and type checking support from the IDE, greatly improving the development experience and code correctness.

[0056] Through the compile-time derivation and interface setting in steps S121 to S124 above, the template node class completely determines all the type information of the operator (input type, output type, number of inputs, number of outputs) during compile time, and presents this information in the business calculation interface in the form of a strongly typed parameter list. This provides a quantitative basis for the configuration of the synchronization mechanism and distribution mechanism during the node initialization phase, and also provides a type basis for data recovery and calculation calls during the runtime phase.

[0057] In conjunction with the first aspect, step S130 includes: S131, during the node initialization phase, an independent message queue is created for each input port based on the number of inputs. The message queue is used to buffer the data received from the upstream node.

[0058] S132, create a composite synchronization node and connect each message queue to each input port of the composite synchronization node.

[0059] S133, the combined synchronization node listens to the status of the head of each message queue. When there are valid data frames at the head of each message queue, the combined synchronization node merges the valid data frames at the head of each message queue into a combined data unit for output, thereby triggering the node to execute.

[0060] The combined data unit contains multiple data items arranged in the order of the input ports, with each data item corresponding to one input port.

[0061] In step S130, based on the number of inputs N determined at compile time, an input synchronization mechanism is automatically configured during the node initialization phase to trigger node execution when all input data is ready. This step follows from step S120, which uses sizeof...(Inputs) to derive the number of input ports N at compile time, transforming the compile-time determined type count into a runtime-usable physical synchronization structure. Specifically: In step S131, during the node initialization phase, an independent message queue is created for each input port based on the number of inputs. This message queue is used to buffer data received from upstream nodes. Specifically, based on the number N of input ports determined at compile time, the system performs N queue instantiation operations during node initialization, allocating an independent first-in-first-out (FIFO) message queue for each input port. The physical carriers of these message queues are provided by the underlying graph scheduling framework (such as the queues maintained internally by join_node in TBB), and each queue is specifically used to receive data from its corresponding upstream node. When upstream heterogeneous nodes push different types of StreamMsg into their respective input queues, these queues independently buffer data from different upstream nodes. Since the data generation frequency of each upstream node may differ (e.g., the image frame rate is higher than the parameter update rate), the existence of message queues allows multiple data streams of varying speeds to queue orderly in their respective queues, awaiting subsequent synchronization alignment operations. The length of each queue is unlimited, thus preventing data backlog from blocking upstream node production. The number of message queue instantiations is uniquely determined by the N value at compile time. Developers do not need to manually specify the number of queues, nor do they need to dynamically determine the number of input ports at runtime. Thus, the size of the queue storage structure can be determined at compile time.

[0062] In step S132, a combined synchronization node is created, and each message queue is connected to its respective input port. Specifically, after creating N independent message queues, the system further instantiates a combined synchronization node (such as `join_node` in TBB), where the number of input channels matches the number of input ports N. This combined synchronization node can be a combined node provided by the underlying graph scheduling framework (such as `tbb::flow::join_node` in TBB, with the strategy set to `tbb::flow::queueing`), leveraging its multi-input single-output characteristic to achieve synchronous convergence of multiple data streams. Subsequently, the N message queues created in step S131 are connected one by one to the N input ports of the combined synchronization node, making the output of each message queue the data source for the corresponding input channel of the combined synchronization node. Through this connection, the combined synchronization node gains access to the status of the head of each message queue, continuously checking whether there are valid data frames waiting to be processed at the head of each input queue. The number of input channels of the combined synchronization node is driven by the N value determined at compile time. When the N value changes, the number of input channels of the combined synchronization node is automatically adjusted without manual intervention.

[0063] In step S133, the combination synchronization node monitors the head status of each message queue. When a valid data frame exists at the head of each message queue, the combination synchronization node merges the valid data frames at the head of each message queue into a single combined data unit for output, triggering node execution. The combined data unit contains multiple data items arranged in order of input ports, with each data item corresponding to an input port. Specifically, during operation, the combination synchronization node continuously monitors the head status of all N message queues and executes strict synchronization judgment logic: the combination synchronization node determines that the synchronization condition is met only when a valid data frame exists at the head of each message queue (i.e., each of the N queues has a piece of data to be processed at its head).

[0064] At this point, the merging synchronization node extracts the corresponding data frames from the head of each message queue, merges and packages these data frames into a combined data unit (such as a tuple in TBB) according to the input port order (from port 0 to port N-1), and then outputs the combined data unit to the downstream computing node, triggering the execution of that node. For example, in a target detection scenario, this mechanism ensures that image data and corresponding camera intrinsic parameter data acquired at the same time arrive at the computing node synchronously, preventing "the Nth frame image and the N-1th frame intrinsic parameter" from participating in the same calculation due to inconsistent data frame rates. The data items in the combined data unit are arranged according to the input port order, consistent with the arrangement order of the input parameters in the business computing interface described in S124 of S120, thereby ensuring that the type and order of data always match when received from the physical port, packaged by the merging synchronization node, and finally enters the user computing interface, without the need for additional data rearrangement or mapping operations. The combined data unit contains multiple data items arranged according to the input port order, with each data item corresponding to an input port.

[0065] Furthermore, after the combined data unit is output to the downstream computing node, it enters the processing node. The system extracts each StreamMsg from the combined data unit and executes std::dynamic_pointer_cast.<In_X> A safe downcast restores each data carrier to its corresponding strongly typed input data. If the resulting smart pointer is null, it indicates a severe mismatch in data types between upstream and downstream connections. The system proactively intercepts the error and returns an empty `continue_msg` to discard the exception frame, ensuring the system does not crash during runtime. This dynamic type-safe conversion mechanism ensures that data entering the user's computing interface is strictly matched in type, further guaranteeing runtime type safety.

[0066] In conjunction with the first aspect, step S130 automatically configures the output distribution mechanism based on the output quantity determined at compile time, specifically including: S134. Based on the number of outputs, construct multiple output distribution channels, with each of the multiple output distribution channels corresponding one-to-one with the multiple output ports of the current node.

[0067] S135, for each output distribution channel, connect the output distribution channel to the input port of the corresponding downstream node to establish a data path from the current node to the downstream node.

[0068] In step S130, based on the number of outputs determined at compile time, an output distribution mechanism is automatically configured to transmit each output data generated by the node to its corresponding downstream port. This step follows up on step S120, where the number of output ports, or the number of outputs M, is derived at compile time using sizeof...(Outputs). It transforms the number of output types determined at compile time into a physical distribution structure usable at runtime, providing a channel foundation for independent routing and concurrent distribution of multiple heterogeneous output results.

[0069] In step S134, based on the number of output ports (i.e., the number of outputs M) determined at compile time, the system performs M channel instantiation operations during the node initialization phase to construct an independent output distribution channel for each output port. Each output distribution channel can connect to the input port of a single downstream node, or it can connect to the input ports of multiple downstream nodes simultaneously through a broadcast mechanism, realizing one-to-many data distribution. In this embodiment, in the node initialization function (init_impl), the specific number M of output template parameters is first parsed at compile time using the operator consttexpr size_t num_outs = sizeof...(Outs); Then, a loop of number M is executed. In each loop, a low-level broadcast node (tbb::flow::broadcast_node) is dynamically instantiated by calling the memory allocation interface. <streammsg>), and push its smart pointer into the out_ports_ array.

[0070] Therefore, each output port is equipped with an independent data conveyor belt, and each output distribution channel acts as an independent physical distribution carrier, specifically responsible for transmitting a specific type of output data downstream. Each output distribution channel is independent and does not interfere with others, allowing data from different output ports to be distributed in parallel. Each output distribution channel itself has data broadcasting capabilities, enabling the simultaneous replication and distribution of a single received data carrier to the input ports of multiple downstream nodes, supporting scenarios where the same output of an operator is consumed concurrently by multiple downstream operators. The configuration of the construction action and the synchronization mechanism in S130 are completed synchronously during the node initialization phase, requiring no manual intervention from the developer. It is entirely driven by the M value determined at compile time. When the number of types in the output type parameter package changes, the number of instantiated output distribution channels is automatically adjusted synchronously, without requiring any modification to the framework code or rewriting of the port binding logic.

[0071] In step S135, for each output distribution channel, the output distribution channel is connected to the input port of the corresponding downstream node to establish a data path from the current node to the downstream node. Specifically, after instantiating M output distribution channels and storing them in the output port array, the system connects each output distribution channel to the input port of the corresponding downstream node according to the connection relationship defined by the user through the graphical interface or configuration file. For example, when using the TBB scheduling engine, if the native node type is used, the number of inputs and outputs of the node must be defined when it is created. This will prevent the pipeline from being freely orchestrated (if worker C is defined to have only one output when it is created, it is impossible to connect worker C's output to workers E and F). "This embodiment dynamically generates a variable number of output distribution channels, allowing nodes in the pipeline to be manually connected via the editing interface before generation. It dynamically parses and creates node instances and the number of node inputs and outputs. Through the connection operation in step S135, each logical output port of the current node establishes a one-to-one or one-to-many data path with the corresponding input port of the downstream node via the corresponding output distribution channel. Each output distribution channel transmits the output data generated by the current node to the input port of the connected downstream node, achieving independent routing and parallel distribution of multiple output data. This allows the pipeline topology to be flexibly adjusted according to business needs without recompiling or modifying operator code. Furthermore, after each output distribution channel is connected during node initialization, the entire data path remains unchanged during operation, without introducing additional runtime routing overhead, ensuring high efficiency in data flow."

[0072] Through the configuration of steps S134 and S135 above, the output distribution mechanism is fully constructed in the node initialization phase, providing a physical channel foundation for the multi-path independent distribution of result data in the running phase, and together with the input synchronization mechanism configured in step S130, it constitutes a complete port management framework for the encapsulation intermediate layer.

[0073] This embodiment completes the construction of the encapsulation middleware layer through steps S110 to S130: the node base class provides the port management skeleton, the template derived class derives the input and output types and quantities at compile time, the synchronization configuration module allocates an independent message queue for each input port and attaches a combined synchronization node during the node initialization phase, and the distribution configuration module dynamically instantiates broadcast nodes corresponding to the number of output ports during the node initialization phase. These three components work together to form the basic architecture of a complete data processing chain from multi-input synchronization to business computation and then to multi-output distribution.

[0074] In conjunction with the first aspect, step S140 includes: S141, extract each data item from the combined data unit output by the input synchronization mechanism. The data item is a data carrier of uniform type erasure.

[0075] When the combined synchronization node determines in S133 that there are valid data frames at the head of all input queues and merges these data frames into a combined data unit for output, this combined data unit is passed to the downstream computing node, triggering the node's business execution process. First, it is necessary to extract each data item from the combined data unit. As previously stated, this combined data unit contains multiple data items arranged in the order of the input ports. Each data item corresponds to an input port; that is, the first data item corresponds to the data type In1 of the first input port, the second data item corresponds to the data type In2 of the second input port, and so on. Since the data is transmitted at the framework layer in the form of a uniformly type-erased data carrier StreamMsg, the scheduling framework only needs to recognize the uniform carrier type and does not need to perceive the structural differences of the specific business data. However, when the data enters the business computing interface, operator developers need to program in a strongly typed environment to ensure type safety and obtain IDE support. Therefore, the system extracts each data item from the combined data unit output by the combined synchronization node in the order of the input ports. These data items all exist at the framework layer in the form of a uniformly type-erased data carrier, preparing for subsequent type recovery.

[0076] S142, according to the order of the types in the input type parameter package, perform safe downcasting on each data item to restore it to its corresponding strongly typed input data.

[0077] The system performs a safe downcast on each extracted data item, restoring each StreamMsg to its corresponding strongly typed input data, such as Image or CameraParam. This downcast employs a safety mechanism: if the resulting smart pointer is null, it indicates a severe mismatch in data types between upstream and downstream connections. The system proactively intercepts this type error and discards the exception frame, ensuring the system does not crash due to type mismatch. This dynamic type-safe conversion mechanism guarantees the type correctness of runtime data, ensuring strict type matching of data entering the user's computation interface. Since the data items in the composite data unit are arranged according to the input port order, perfectly matching the order of input parameters in the business computation interface, the extracted and converted input data can be directly passed to the corresponding input parameters of the business computation interface in sequence, without additional data rearrangement or mapping operations. The specific implementation of this safe downcast can use C++'s std::dynamic_pointer_cast mechanism, performing a type-safe downcast on the polymorphic smart pointer at runtime. If the conversion fails, a null pointer is returned, allowing the system to determine type matching and handle appropriate exceptions.

[0078] S143, create output object instances for each output type according to the output type parameter package, and store each output object instance into the collection in the order of the output ports.

[0079] After the input data has been restored to its original type and before the business calculation interface is invoked, the system needs to pre-allocate storage space for each output type to hold the processing results generated by the business calculation interface. Based on the output type parameter package, the system pre-declares an empty tuple containing M expected output smart pointers using a tuple unpacking mechanism. It then unpacks the pointers in the tuple one by one using `std::apply` and, combined with comma expressions, executes `std::make_shared` on each type in the output type parameter package. <out>The `()` function allocates strongly typed physical memory space on the heap for all output types, generates corresponding smart pointers, and stores them in tuples. This action occurs before business computation, ensuring that all output object instances are ready to receive computation results when the business computation interface is called. This completely eliminates the memory management burden on developers and reduces runtime risks caused by memory allocation failures or memory leaks. Each output object instance is stored in the set according to the output port order; that is, the first output object instance corresponds to the Out1 type of the first output port, the second corresponds to the Out2 type of the second output port, and so on, perfectly matching the order of output parameters in the business computation interface. This pre-allocation of output object instances ensures that the output results are already in the pre-allocated objects after the user computation interface is executed, eliminating the need for additional memory allocation and data copying operations, further improving runtime efficiency.

[0080] S144, call the business calculation interface, pass the strongly typed input data to the business calculation interface, and write the processing result generated by the business calculation interface to the output object instance.

[0081] The system uses the strongly typed input data obtained by downcasting in S142 as actual parameters, passing them to the business calculation interface in the order of the input parameters. Simultaneously, it passes the M pre-created output object instances from S143 to their corresponding output parameters in sequence, and then calls the calculation interface to execute the business algorithm logic. The list of parameters received by the calculation interface overridden by the user in its derived class is completely consistent with the interface declared in S124. The input side consists of strongly typed input data corresponding one-to-one with each type in the input type parameter package, and the output side consists of output object instances corresponding one-to-one with each type in the output type parameter package. Within this interface, the user focuses on the implementation of the business algorithm, directly manipulating the passed strongly typed input data for calculation and writing the calculation results directly into the passed output object instance. Because all input data and output objects are strongly typed, users can obtain type checking and IDE support during the coding process, greatly improving development efficiency and code correctness. In this step, data type security is jointly ensured by S142 and S143: the input data has been safely downcast and verified in S142, and the output objects have been pre-assigned to the correct type in S143, making the type environment inside the user computing interface completely safe and eliminating the risk of runtime crashes due to type misalignment. In S145, each output object instance is encapsulated into a data carrier with unified type erasure.

[0082] After the user-defined business calculation interface is executed, valid calculation results are written to each output object instance. At this point, these strongly typed result data need to be converted back into a data carrier that can be uniformly processed by the framework layer, so that they can be distributed to downstream nodes through the data flow mechanism of the scheduling engine. The system first encapsulates each output object instance as a StreamMsg, that is, by upcasting the strongly typed output object to a uniformly type-erased data carrier, the framework layer only needs to recognize the uniform carrier type when the data leaves the current node and enters the scheduling framework for flow, without needing to perceive the structural differences of the specific business data. After encapsulation, the system generates a corresponding compile-time index sequence based on the number of output ports, and distributes each output data carrier to the corresponding output distribution channel using a folded expression. This folded expression is directly expanded into multiple independent distribution operations at compile time, each distribution operation corresponding to an output port. When the number of output ports is M, the compiler generates M independent distribution statements at compile time, and each distribution statement sends an output data carrier through the corresponding output distribution channel, completely eliminating the performance overhead of for loops and the risk of array out-of-bounds errors during runtime. After receiving the data carrier, each output distribution channel transmits the data carrier to the input port of its respective downstream node according to the connection relationship established in S142. Since each output distribution channel is independent of each other, the transmission process of each output data can be carried out in parallel without interference, realizing the parallel distribution of multi-channel heterogeneous computing results.

[0083] Thus, S141 to S145 constitute a complete runtime data flow loop from node triggering to result encapsulation: input data is extracted from composite data units, with each data item serving as a uniformly type-erased data carrier; a safe downcast is performed according to the order of the input type parameter package, restoring the input data to a strongly typed state; output objects are pre-allocated according to the output type parameter package before computation; the computation interface executes business logic in a strongly typed safe environment; and the computation result, after being encapsulated into a uniformly type-erased data carrier, is prepared to be transmitted to downstream nodes through the output distribution mechanism. This series of steps, together with the input synchronization mechanism configured in S130 and the output distribution mechanism configured in S150, forms a complete data processing link for the encapsulation middle layer. This allows developers to completely disregard the underlying framework details such as queue management, type erasure and restoration, and output object allocation, and only need to rewrite the business computation interface in a strongly typed environment to complete operator access.

[0084] In conjunction with the first aspect, step S150 includes: S151, Generate the corresponding compile-time index sequence based on the number of output ports of the current node.

[0085] S152 uses a folded expression to distribute each output data carrier to the output distribution channel corresponding to each output port of the current node according to the compile-time index sequence. The folded expression is expanded into multiple independent distribution operations during compilation, and each distribution operation corresponds to one output port of the current node. S153, each output distribution channel transmits the received data carrier to the input port of the connected downstream node.

[0086] In step S150, each output object instance is encapsulated into a uniformly type-erased data carrier and transmitted to the corresponding downstream node through each output distribution channel. Specifically, after the user-defined business calculation interface in step S140 is executed, valid calculation results have been written into each output object instance. At this time, these strongly typed result data need to be converted back into a data carrier that can be uniformly processed by the framework layer so that they can be distributed to downstream nodes through the data flow mechanism of the scheduling engine. The system first encapsulates each output object instance as a StreamMsg, that is, by upcasting the strongly typed output object to a uniformly type-erased data carrier, the framework layer only needs to recognize the uniform carrier type when the data leaves the current node and enters the scheduling framework flow, without needing to perceive the structural differences of the specific business data. After encapsulation, the system executes steps S151 to S153 to complete the zero-overhead parallel distribution of multiple output data.

[0087] In S151, a corresponding compile-time index sequence is generated based on the number of output ports of the current node. Specifically, the system uses std::index_sequence to determine the number M of output ports determined at compile time.<Index...> A compile-time index sequence from 0 to M-1 is generated. This index sequence is determined at compile time, providing compile-time index traversal capability for the independent distribution of each output port in subsequent folded expressions. Since the index sequence is generated entirely at compile time, it does not consume any runtime resources and does not introduce runtime overhead for loop variable incrementing or boundary checks. Each index value in this index sequence uniquely corresponds to an output port, with index 0 corresponding to the first output port, index 1 corresponding to the second output port, and so on up to index M-1 corresponding to the Mth output port—ensuring a one-to-one correspondence between each output data carrier and each output distribution channel during the distribution process.

[0088] In S152, a folded expression is used to distribute each output data carrier to its corresponding output distribution channel. The folded expression is expanded into multiple independent distribution operations at compile time, each corresponding to an output port. Specifically, the system combines the compile-time index sequence generated in S151 with a set storing M output object instances, and uses C++17 folded expressions to perform distribution operations on each output data carrier. This folded expression is directly expanded into M independent distribution statements at compile time, and each distribution statement sends an output data carrier through its corresponding output distribution channel (i.e., the broadcast node corresponding to the physical carrier array of the output port). Since the number of distribution operations M is known at compile time, the compiler can fully expand the loop without retaining any runtime loop control structures, completely eliminating the performance overhead of runtime for loops and the risk of array out-of-bounds errors. For example, when the number of output ports is 3, the compiler generates 3 independent distribution statements at compile time, pushing the three output data carriers to the corresponding three broadcast nodes. After receiving the data carrier, each output distribution channel, according to the connection relationship established in step S142, transmits the data carrier to the input port of its respective connected downstream node. Since each output distribution channel is independent of the others, the transmission process of each output data can be carried out in parallel without interference, thus realizing the parallel distribution of multi-channel heterogeneous computing results.

[0089] Since each output distribution channel has established a connection with the input port of the downstream node according to the user-defined connection relationship in step S135, when the output data carrier is distributed to the corresponding output distribution channel, the output distribution channel, as an independent physical distribution carrier, transmits the received data carrier to the input port of the connected downstream node via a pre-established data path in S153. Each output distribution channel can connect to the input port of a single downstream node, or it can connect to the input ports of multiple downstream nodes simultaneously through a broadcast mechanism, realizing one-to-many data distribution. Since each output distribution channel is independent of each other, the transmission process of each output data can be carried out in parallel without interference, realizing the parallel distribution of multiple heterogeneous computing results. After receiving the data carrier, each downstream node repeats the above-described recovery and calculation process within its respective encapsulation intermediate layer, thereby completing the data-driven execution of the entire pipeline.

[0090] Thus, steps S140-S150 constitute a complete runtime data flow loop from the triggering of node execution to the completion of result distribution. Input data is extracted from combined data units and downcast to a strongly typed form; output objects are pre-allocated before computation; the computation interface executes business logic in a strongly typed and safe environment; and the computation results are encapsulated and distributed to downstream nodes in parallel with zero overhead. Together with the input synchronization mechanism configured in step S130 and the output distribution mechanism configured in step S140, these steps form a complete data processing link for the encapsulation intermediate layer.

[0091] The following section uses a typical target detection operator application scenario, taking a dual-input dual-output (2 In, 2 Out) example, to illustrate the complete implementation process of the technical solution of this invention.

[0092] Suppose we need to connect to an object detection operator that takes both an "RGB image" and "camera parameters" as input. After the object detection algorithm calculates the data, it outputs both a "processed image" and a "box list".

[0093] At compile time, operator developers write the following code declaration: class Object Detection Node : public Two Input Node Base<Image,Camera Param, Processed Image, Box List> At this point, the compiler packages the Image and Camera Param into an input type parameter package (Inputs...), and the Processed Image and Box List into an output type parameter package (Outputs...). The number of input ports is 2 using sizeof...(Inputs), and the number of output ports is 2 using sizeof...(Outputs). Both of these values ​​are compile-time constants and do not incur any runtime overhead.

[0094] The compiler also generates the business computation interface compute(const Image&, const Camera Param&, Processed Image&, Box List&), whose input parameters and output parameters correspond one-to-one. When users override this interface in derived classes, they can directly obtain a strongly typed parameter list, thus achieving type safety at compile time.

[0095] Through the above compile-time derivation, the complete type information of the operator—input types are Image and CameraParam, output types are Processed Image and Box List, the number of input ports is 2, and the number of output ports is 2—is determined at compile time. This provides a complete basis for the automatic configuration of the synchronization and distribution mechanisms in the subsequent node initialization phase.

[0096] After the program starts, the framework calls the operator's initialization function and enters the node initialization phase.

[0097] Input synchronization mechanism configuration: Based on the two input ports determined at compile time, the system creates an independent message queue for each input port. Both message queues are first-in-first-out (FIFO) queues with no length limit, used to buffer data from upstream image nodes and camera parameter nodes, respectively. Subsequently, the system creates a combined synchronization node (e.g., tbb::flow::join_node in TBB, with the strategy set to tbb::flow::queueing), with two input channels, connecting the two message queues to the two input ports of the combined synchronization node one by one. During operation, the combined synchronization node continuously monitors the head status of both message queues. Only when both message queues have valid data frames at their heads is the node extracted and packaged into a single combined data unit (e.g., a tuple in TBB) for output, triggering execution by downstream computing nodes. This mechanism ensures strict alignment of the two data streams over time.

[0098] Configure the output distribution mechanism: Based on the number of output ports determined at compile time (2), the system performs two loop instantiation operations during the node initialization phase. Each time, it calls the memory allocation interface to dynamically instantiate a low-level broadcast node (such as tbb::flow::broadcast_node in TBB). <streammsg>), and push its smart pointer into the output port physical carrier array (i.e., the std::vector maintained by the derived class in the node base class). <std::unique_ptr<tbb::flow::broadcast_node <streammsg>The `out_ports_` array serves as the actual physical layer broadcast carrier for sending data outwards. The two broadcast nodes correspond to the "Processed Image" and "Box List" outputs, respectively. Subsequently, based on the connection relationships defined by the user through the graphical interface or configuration file, the system connects the two broadcast nodes to the input ports of their respective downstream nodes, establishing a data path from the current node to the downstream node.

[0099] With the above initialization configuration, the input synchronization structure (2 independent message queues + 1 combined synchronization node) and output distribution structure (2 broadcast node instances + 2 data paths) of the operator are fully constructed during the node initialization phase. The pre-constructed structure can be used directly when each frame of data flows, without the need for repeated configuration.

[0100] When the pipeline begins processing data, the complete workflow of single-frame data derivation is as follows: Step 1 (corresponding to the behavior during steps S131~S133): The upstream image node and camera parameter node encapsulate the strongly typed Image data and Camera Parameter data into a uniformly typed erased data carrier StreamMsg (i.e., std::shared_ptr). <messagebase>The data is sent to the two input ports of the target detection operator. Each data point enters its corresponding independent message queue and waits. If the image frame rate is faster than the intrinsic parameter update rate, the combined synchronization node temporarily stores the image frame in the corresponding queue until both the Nth frame image and the Nth frame intrinsic parameter arrive at the head of their respective queues. At this point, the combined synchronization node determines that the synchronization condition is met, extracts and packages the two data points into a combined data unit (Tuple) containing two StreamMsgs, and outputs it to the downstream computing node, triggering the node to execute. This process ensures strict alignment of the two data streams in the time series, preventing the "Nth frame image and the (N-1th frame intrinsic parameter)" from participating in the same calculation.

[0101] Step 2 (corresponding to steps S141~S142): After receiving the combined data unit, the computing node extracts two data carriers from the tuple in the order of the input ports, and executes std::dynamic_pointer_cast respectively. and std::dynamic_pointer_cast <cameraparam>A safe downcast is performed to restore strongly typed Image and CameraParam objects, obtaining the strongly typed input data corresponding to each input port. If the resulting smart pointer is null, it indicates a serious data mismatch between the upstream and downstream connections. The system proactively intercepts this type error and discards the abnormal frame to ensure that the system does not crash due to type mismatch during runtime.

[0102] Step 3 (corresponding to step S143): Before calling the user computation interface, the system uses the tuple unpacking mechanism (std::apply) of the C++ standard library to pre-declare an empty tuple (typed_outs) containing two smart pointers expected to be output. It then unpacks the pointers in the tuple one by one using std::apply and executes the expression ((ptrs = std::make_shared...)) using comma expressions. <outs>The `()`, `...)` method allocates strongly typed physical memory space on the heap for the two output types, Processed Image and Box List, and stores the two output object instances into the collection in the order of their output ports. This step ensures that all output objects are ready when the user computation interface is called, and can directly receive the computation results, completely relieving the developer of the memory management burden.

[0103] Step 4 (corresponding to S144): The system calls the compute interface, passing the strongly typed Image and CameraParam data recovered in Step 2 as input arguments to the compute interface, and sequentially passing the ProcessedImage and Box List output object instances created in Step 3 to the corresponding output parameters. Within this interface, developers can focus on implementing object detection algorithms (such as feature extraction, object recognition, bounding box regression, etc.), directly writing the detection results into the passed-in ProcessedImage and Box List objects. The entire computation process is completed in a strongly type-safe environment, providing developers with IDE support for code completion, type checking, and refactoring, fundamentally eliminating the runtime crash risk caused by type mismatches in traditional solutions.

[0104] Step 5 (corresponding to S145): After the calculation is complete, the system calls the auxiliary dispatch function (send_outputs) to encapsulate the Processed Image object and the Box List object into StreamMsg data carriers (i.e., by upcasting the strongly typed output objects to uniformly typed erased data carriers). After encapsulation, the system uses std::index_sequence<Index...> A compile-time index sequence from 0 to 1 is generated, and the C++17 folding expression is applied to push the two output data carriers into their respective output distribution channels. This folding expression is directly expanded into two independent distribution statements at compile time: the first statement pushes the encapsulated ProcessedImage to the first broadcast node via out_ports_[0]->try_put(), and the second statement pushes the encapsulated BoxList to the second broadcast node via out_ports_[1]->try_put(), completely eliminating the performance overhead of for loops and the risk of array out-of-bounds errors during runtime. After receiving the data carriers, each broadcast node, based on the connection relationships established during the initialization phase, transmits the data carriers to the input ports of their respective connected downstream nodes, completing the parallel distribution of multi-path heterogeneous computation results.

[0105] As seen in the complete lifecycle described above, this system hides all the tedious and error-prone framework-level operations—including low-level queue synchronization, type erasure packaging and decomposition, dynamic port generation, and compile-time instruction dispatching—within an encapsulated intermediate layer throughout their entire lifecycle. Operator developers only need to use a single line of class declaration code (class ObjectDetectionNode :public Two Input Node Base).<Image, Camera Param, Processed Image, Box List> By specifying the input and output types and overriding the strongly typed compute interface, operator access can be completed. There is no need to worry about the details of creating the underlying message queue, mounting the composite synchronization node, instantiating the broadcast node, encapsulating and disassembling the data carrier, and allocating the memory of the output object. This achieves plug-and-play and seamless access of visual operators, while ensuring compile-time type safety and zero runtime performance.

[0106] Secondly, this application also provides a visual heterogeneous data stream operator encapsulation system, the system including a scheduling engine layer, an encapsulation intermediate layer, and a business operator layer; combined with Figure 2 As shown, the encapsulation intermediate layer includes: node base class module 10, template derivation module 20, synchronization configuration module 30, transmission encapsulation module 50, and distribution configuration module 40.

[0107] The node base class module 10 is used to provide a port management interface to uniformly manage the input and output ports of operators.

[0108] The template derivation module 20 is used to derive a template node class from the node base class module 10 using a variable parameter template, and to set up a business calculation interface in the template node class. The template parameters of the template node class include an input type parameter package and an output type parameter package to determine the input type, output type, number of inputs and number of outputs of the operator at compile time. The input parameters of the business calculation interface correspond one-to-one with each input type in the input type parameter package, and the output parameters correspond one-to-one with each output type in the output type parameter package.

[0109] The synchronization configuration module 30 is used to automatically configure an input synchronization mechanism during the node initialization phase, based on the number of operator inputs determined at compile time, so as to trigger node execution when all input data is ready; and to automatically configure an output distribution mechanism based on the number of outputs.

[0110] The transmission encapsulation module 50 is used to respond to the node being triggered to execute, obtain the data carrier with unified type erasure from the input synchronization mechanism, restore the data carrier to the corresponding strongly typed input data according to the input type parameter packet, call the business computing interface to process the strongly typed input data, and encapsulate the processing result into a data carrier with unified type erasure.

[0111] The distribution configuration module 40 is used to automatically configure the output distribution mechanism based on the number of outputs determined during compilation, so as to transmit each output data generated by the node to the corresponding downstream port.

[0112] The visual heterogeneous data flow operator encapsulation system provided in this application constructs an encapsulation intermediate layer located between the underlying graph scheduling framework and user-defined business operators. The system is divided into three layers from top to bottom: the upper layer is the scheduling engine layer, the encapsulation intermediate layer, and the lower layer is the business operator layer.

[0113] The upper layer is the scheduling engine layer, which is implemented based on the underlying graph scheduling framework (such as TBB's flow_graph). It includes basic building blocks such as graph, join_node, broadcast_node, and function_node. It is responsible for managing the global directed acyclic graph, maintaining the data dependencies between nodes, and driving each node to execute concurrently according to the preset topology. This layer only recognizes the data carrier StreamMsg with uniform type erasure and does not need to be aware of the type and structure of specific business data.

[0114] Encapsulation intermediate layer (combined) Figure 3 The operator encapsulation and scheduling middleware layer shown contains a unified abstract base class NodeBase and its derived multi-input template classes (such as Two Input NodeBase).<In1, In2, Out1, Out2> This layer is responsible for shielding the differences in underlying data types, handling queue synchronization and alignment for multiple input data, dynamic type conversion, and automatic distribution of multiple outputs. Node Base provides a port lookup interface (containing virtual functions get_in_port and get_out_port, which return pointers to the communication nodes for the corresponding input and output ports, respectively) and a port information maintenance structure (internally maintaining a std::vector). <portinfo>`input_port_` and `output_port_` record the names and type identifier strings of each input and output port, respectively, and the send port carrier (a `std::vector` maintained by the derived class) is defined in the base class. <std::unique_ptr<tbb::flow::broadcast_node <streammsg>The `out_ports_` array serves as the actual physical layer broadcast carrier for sending data outwards. Template-derived classes utilize variable-parameter templates to determine the input and output types and quantities of operators at compile time, and automatically configure input synchronization mechanisms (allocating an independent FIFO queue for each input port and attaching a `join_node`, merging outputs to trigger node execution when all inputs are ready) and output distribution mechanisms (dynamically instantiating M `broadcast_node`s and storing them in the `out_ports_` array as the physical distribution carrier for output data) during node initialization.

[0115] Combination Figure 3 As shown, the lower layer is the business operator layer, containing user-defined specific operators (such as the ObjectDetection Node). Users only need to inherit the template class of the middle layer and override the compute interface, focusing on strongly typed data processing logic, without needing to worry about framework details such as underlying queue management, synchronization mounting, and broadcast instantiation. The data flow path between layers is as follows: after the upstream node generates strongly typed data, it is upcast to StreamMsg and enters the scheduling engine layer for flow; after flowing into the middle layer, it is synchronized by the FIFO queue of join_node and safely downcast by dynamic_pointer_cast, restoring it to strongly typed data and triggering the compute interface execution; after the calculation is completed, the result data is encapsulated as StreamMsg and distributed to downstream nodes through the broadcast_node in the out_ports_ array.

[0116] Therefore, the underlying scheduling framework focuses on efficient data flow and task scheduling, the business operators focus on strongly typed algorithm logic implementation, the node base class module provides a unified port management skeleton, the template derivation module completes type inference and quantity statistics at compile time, the synchronization configuration module and the distribution configuration module complete the automatic construction of the input-side synchronization structure and the output-side distribution structure, respectively, and the encapsulation middle layer serves as an adaptation bridge between the underlying framework and the business operators, enabling the two to evolve independently and without mutual constraints, thus achieving complete decoupling of the system.

[0117] In conjunction with the second aspect, the synchronization configuration module 30 includes: a queue creation unit and a combined synchronization unit.

[0118] The queue creation unit is used during the node initialization phase to create an independent message queue for each input port based on the number of inputs. The message queue is used to buffer data received from upstream nodes.

[0119] The combined synchronization unit is used to create a combined synchronization node, connecting each message queue to its respective input port. The combined synchronization node listens to the status of the head of each message queue. When there are valid data frames at the head of each message queue, it merges the valid data frames at the head of each message queue into a combined data unit for output, thereby triggering node execution.

[0120] The combined data unit contains multiple data items arranged in the order of the input ports, with each data item corresponding to one input port.

[0121] The synchronization configuration module 30 is the core functional unit responsible for implementing automatic synchronization of multi-channel input data. It consists of two parts: a queue creation unit and a combination synchronization unit. The two work together to automatically configure the input synchronization mechanism based on the number of inputs N determined at compile time during the node initialization phase. The queue creation unit first creates an independent message queue for each input port based on the number of inputs N determined at compile time during the node initialization phase. Each message queue is a first-in-first-out (FIFO) queue with no length limit, used to buffer data from different upstream nodes, avoiding data backlog or loss caused by different data generation rates of upstream nodes. The combination synchronization unit then creates a combination synchronization node, connecting each message queue to each input port of the combination synchronization node. This allows the combination synchronization node to continuously monitor the status of the head of each message queue. Only when there are valid data frames at the head of each message queue will the combination synchronization node retrieve the valid data frames from the head of each queue and merge them into a single combined data unit for output, triggering execution of downstream nodes. This synchronization strategy ensures that each set of input data obtained by downstream nodes is strictly aligned in time series. Multiple data items are arranged in the combined data unit according to the input port order, with each data item corresponding to an input port. This guarantees that each set of data participating in the computation by downstream nodes is strictly matched in time, effectively solving the data misalignment problem caused by inconsistent frame rates in multi-source heterogeneous data streams. Through the seamless cooperation of the queue creation unit and the combined synchronization unit, the synchronization configuration module 30 encapsulates the complete configuration process of the input synchronization mechanism within the module. This eliminates the need for developers to concern themselves with the details of creating the underlying queues and attaching synchronization nodes; they only need to provide the input type parameter package when declaring the operator, and the framework can automatically complete the synchronization configuration of multiple input data streams.

[0122] In conjunction with the second aspect, the template derivation module 20 also includes: a template parameter configuration unit, a type statistics unit, a port quantity determination unit, and an interface setting unit.

[0123] The template parameter configuration unit is used to treat the input type parameter package and the output type parameter package as variable template parameters of the template node class, enabling the template node class to obtain all input types and all output types of the operator at compile time. Variadic templates allow the template to accept any number and type of template parameters. This application introduces this feature into the operator encapsulation intermediate layer, making the template parameter list of the template node class consist of two independent variable parameter packages: the input type parameter package represents the set of data types for all input ports of the operator, and the output type parameter package represents the set of data types for all output ports of the operator. Through this configuration, the operator obtains complete type description information at compile time, i.e., how many types of input data the operator needs to receive, what those types are, and how many types of output data it needs to produce, and what those types are, thus overcoming the drawbacks of traditional solutions that rely on runtime type identification or manual type registration.

[0124] The type statistics unit is used to count the number of types in the input and output type parameter packages using compile-time operators. C++ provides the compile-time constant expression operator `sizeof...`, which accepts a parameter package as an operand and returns the number of type parameters in that package at compile time; the result is a compile-time constant. This application applies the `sizeof...` operator to the input and output type parameter packages respectively, obtaining two compile-time constant values, representing the number of input and output types of the operator, respectively. This statistical process is completed at compile time, and its result is used as a compile-time constant for subsequent port number determination, without incurring any runtime overhead, and without increasing runtime branching or loop iterations due to changes in the number of types in the parameter packages.

[0125] The port quantity determination unit is used to determine the number of input type parameter packets as the number of input ports for the operator, and the number of output type parameter packets as the number of output ports for the operator. Since each input port carries a specific type of input data, and each output port outputs a specific type of output data, the number of input types equals the number of input ports for the operator, and the number of output types equals the number of output ports for the operator. These two values ​​are determined as constants at compile time and are used to guide the allocation and construction of port physical carriers in subsequent steps: the number of input ports determines how many independent message queues need to be instantiated, and how many input channels the combined synchronization node needs to listen to; the number of output ports determines how many broadcast nodes need to be instantiated as output distribution channels. Through the mapping relationship between type and port quantity, an automatic correspondence is formed between operator declarations and framework configurations. Developers declare the input and output types, and the port quantity automatically adapts without any manual configuration.

[0126] The interface setting unit is used to set the business computation interface in the template node class. The input formal parameters of the business computation interface correspond one-to-one with each input type in the input type parameter package, and the output formal parameters correspond one-to-one with each output type in the output type parameter package. This application declares a business computation interface `compute` in the template node class for user overriding. The parameter list of this interface is entirely determined by the template parameter package: the first half consists of constant references or smart pointers (such as `const In1&`, `const In2&`, ...) of each type in the input type parameter package, serving as strongly typed input parameters for receiving upstream data; the second half consists of references or smart pointers (such as `Out1&`, `Out2&`, ...) of each type in the output type parameter package, serving as strongly typed output parameters carrying the computation results. When users override this `compute` interface in their derived classes, they directly obtain a strongly typed parameter list that strictly corresponds to the input and output type parameter packages. This ensures the type safety of the business computation interface at compile time and allows algorithm developers to obtain code completion and type checking support from the IDE, significantly improving the development experience and code correctness.

[0127] Through the collaborative work of the four units mentioned above, the template derivation module 20 completes the entire chain from type information extraction to business interface declaration: the template parameter configuration unit obtains type information, the type statistics unit counts the number of types, the port quantity determination unit maps the number of types to the number of ports, and the interface setting unit converts the type parameter package into a strongly typed business calculation interface. Developers only need to specify the input type parameter package and the output type parameter package when deriving the template node class to automatically obtain the correct number of input and output ports and a strongly typed business calculation interface that strictly corresponds to the type parameter package. Complete type safety is guaranteed at compile time, eliminating the need for manual port registration or writing type conversion code.

[0128] Thirdly, embodiments of this application provide a readable storage medium storing computer program instructions, which are read and executed by a processor to perform the above-described method.

[0129] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0130] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0131] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0133] Finally, it should be noted that the above embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.< / streammsg> < / portinfo> < / outs> < / cameraparam> < / messagebase> < / streammsg> < / streammsg> < / out> < / streammsg> < / streammsg> < / portinfo> < / messagebase>

Claims

1. A method for encapsulating visual heterogeneous data stream operators, characterized in that, The method is used to build an encapsulation intermediate layer between the underlying graph scheduling framework and the user-defined business operator layer. The method includes: Construct a node base class, which provides a port management interface for unified management of the input and output ports of operators; A variable parameter template is used to derive a template node class from the node base class, and a business calculation interface is set in the template node class. The template parameters of the template node class include an input type parameter package and an output type parameter package to determine the input type, output type, number of inputs, and number of outputs of the operator at compile time. The input type parameters of the business calculation interface correspond one-to-one with each input type in the input type parameter package, and the output type parameters correspond one-to-one with each output type in the output type parameter package. During the node initialization phase, an input synchronization mechanism is automatically configured based on the number of inputs to the operator, so as to trigger node execution when all input data is ready; and an output distribution mechanism is automatically configured based on the number of outputs. In response to the node being triggered to execute, a unified type erased data carrier is obtained from the input synchronization mechanism, the data carrier is restored to the corresponding strongly typed input data according to the input type parameter package, the business computing interface is called to process the strongly typed input data, and the processing result is encapsulated into a unified type erased data carrier; The encapsulated data carrier is transmitted to the corresponding downstream node through the output distribution mechanism.

2. The method according to claim 1, characterized in that, The steps to construct a node base class include: A port query interface is defined within the node base class. The port query interface includes an input port query function and an output port query function, which are used to return the communication node pointer of the corresponding input port and the communication node pointer of the corresponding output port according to the index, respectively. A port information maintenance structure is defined within the node base class. This port information maintenance structure is used to record the name and type identifier string of each port. An array of physical carriers for output ports is defined within the node base class. This array is used to store handles that point to the output distribution channels.

3. The method according to claim 1, characterized in that, The steps of deriving a template node class using a variable parameter template with the node base class as the parent class include: The input type parameter package and the output type parameter package are used as variable template parameters of the template node class, so that the template node class can obtain all input types and all output types of the operator at compile time; The number of types in the input type parameter package and the output type parameter package are counted using compile-time operators, respectively. The number of types in the input type parameter package is taken as the number of input ports of the operator, and the number of types in the output type parameter package is taken as the number of output ports of the operator; A business calculation interface is set in the template node class. The input form of the business calculation interface corresponds one-to-one with each type in the input type parameter package, and the output form corresponds one-to-one with each type in the output type parameter package.

4. The method according to claim 1, characterized in that, Based on the number of inputs to the operator determined at compile time, an input synchronization mechanism is automatically configured to trigger steps for node execution when all input data is ready, including: Based on the number of inputs, an independent message queue is created for each input port, and the message queue is used to buffer the data received from the upstream node; Create a combined synchronization node and connect each of the aforementioned message queues to the respective input ports of the combined synchronization node; The combined synchronization node monitors the head status of each message queue. When there are valid data frames at the head of each message queue, the combined synchronization node merges the valid data frames at the head of each message queue into a combined data unit and outputs it to trigger node execution. The combined data unit contains multiple data items arranged in the order of the input ports, with each data item corresponding to one input port.

5. The method according to claim 1, characterized in that, The steps for automatically configuring the output distribution mechanism based on the number of outputs determined at compile time include: Based on the number of outputs, multiple output distribution channels are constructed, and each of the multiple output distribution channels corresponds one-to-one with multiple output ports of the current node; For each output distribution channel, the output distribution channel is connected to the input port of the corresponding downstream node to establish a data path from the current node to the downstream node.

6. The method according to claim 4, characterized in that, The steps of responding to a node being triggered to execute, obtaining a uniformly typed erased data carrier from the input synchronization mechanism, restoring the data carrier to the corresponding strongly typed input data according to the input type parameter packet, calling the business computing interface to process the strongly typed input data, and encapsulating the processing result into a uniformly typed erased data carrier include: Each data item is extracted from the combined data unit output by the input synchronization mechanism, and the data item is a data carrier of uniform type erasure. Based on the order of the types in the input type parameter package, perform a safe downcasting on each data item to restore it to its corresponding strongly typed input data. Based on the output type parameter package, create output object instances for each output type, and store each output object instance into a collection in order of output port. The business calculation interface is invoked, the strongly typed input data is passed to the business calculation interface, and the processing result generated by the business calculation interface is written to the output object instance; Each of the output object instances is encapsulated into a data carrier of a uniform type for erasure.

7. The method according to claim 5, characterized in that, The step of transmitting the encapsulated data carrier to the corresponding downstream node through the output distribution mechanism includes: Generate a corresponding compile-time index sequence based on the number of output ports of the current node; Using folded expressions, each data carrier is distributed to the output distribution channel corresponding to each output port of the current node according to the compile-time index sequence. The folded expressions are expanded into multiple independent distribution operations during compilation, and each distribution operation corresponds to one output port of the current node. Each of the output distribution channels transmits the received data carrier to the input port of the connected downstream node.

8. A visual heterogeneous data stream operator encapsulation system, characterized in that, The system includes a scheduling engine layer, an encapsulation middleware layer, and a business operator layer; the encapsulation middleware layer includes: The node base class module provides a port management interface to uniformly manage the input and output ports of operators; A template derivation module is used to derive template node classes from the node base class module using variable parameter templates, and to set up business calculation interfaces in the template node classes. The template parameters of the template node classes include input type parameter packages and output type parameter packages to determine the input type, output type, number of inputs, and number of outputs of the operator at compile time. The input type parameters of the business calculation interface correspond one-to-one with each of the input types in the input type parameter package, and the output type parameters correspond one-to-one with each of the output types in the output type parameter package. The synchronization configuration module is used to automatically configure an input synchronization mechanism during the node initialization phase based on the number of input ports determined at compile time by the input type parameter package; the input synchronization mechanism is used to trigger node execution when all input data is ready; and to automatically configure an output distribution mechanism based on the number of outputs. The transmission encapsulation module is used to respond to the node being triggered to execute, obtain a uniformly typed erased data carrier from the input synchronization mechanism, restore the data carrier to the corresponding strongly typed input data according to the input type parameter packet, call the business computing interface to process the strongly typed input data, and encapsulate the processing result into a uniformly typed erased data carrier. The distribution configuration module is used to transmit the encapsulated data carrier to the corresponding downstream port through the output distribution mechanism.

9. The system according to claim 8, characterized in that, The synchronization configuration module includes: The queue creation unit is used to create an independent message queue for each input port according to the number of inputs during the node initialization phase. The message queue is used to buffer data received from the upstream node. A combined synchronization unit is used to create a combined synchronization node, connecting each of the message queues to its respective input port; the combined synchronization node listens to the head status of each message queue, and when there are valid data frames at the head of each message queue, it merges the valid data frames at the head of each message queue into a combined data unit for output, thereby triggering node execution; The combined data unit contains multiple data items arranged in the order of the input ports, with each data item corresponding to one input port.

10. The system according to claim 8, characterized in that, The template derivation module also includes: The template parameter configuration unit is used to use the input type parameter package and the output type parameter package as variable template parameters of the template node class, so that the template node class can obtain all input types and all output types of the operator at compile time; A type statistics unit is used to count the number of types in the input type parameter package and the output type parameter package respectively using compile-time operators; The port number determination unit is used to take the number of types of the input type parameter packet as the number of input ports of the operator, and the number of types of the output type parameter packet as the number of output ports of the operator; The interface setting unit is used to set the business calculation interface in the template node class. The input parameters of the business calculation interface correspond one-to-one with each input type in the input type parameter package, and the output parameters correspond one-to-one with each output type in the output type parameter package.

Images