A distributed control system and control method for large-scale deterministic networks

Abstract

The application relates to a distributed control system for a large-scale deterministic network, comprising a Det-AS, an inter-domain, a Det-AS controller and a deterministic service manager, wherein the large-scale deterministic network is divided into multiple independent Det-ASs according to different network requirements and physical characteristics; when deterministic forwarding is performed between the domains, each domain is abstracted into a node with an intra-domain delay attribute, and routing reachability information is exchanged with adjacent domain nodes through a routing management protocol; then, an optimal path is calculated according to a routing selection strategy to perform deterministic data forwarding, so that the inter-domain data forwarding delay is minimized; each Det-AS is equipped with a controller which is used for intra-domain path calculation, resource configuration, protocol issuing and information collection and the like; the deterministic service manager is used for receiving and collecting user requirements and application program requests of user equipment on an access network side, is responsible for configuration and management of deterministic services, and provides network service configuration requirements between end-to-end application systems.

Description

Technical Field

[0001] This invention relates to a distributed control method for large-scale deterministic networks, and more particularly to a distributed control system for large-scale deterministic networks, as well as a cross-domain forwarding method for end-to-end deterministic service flows, belonging to the fields of Internet protocols and Internet technology. Background Technology

[0002] With the rise of deterministic services such as autonomous driving, telemedicine, and smart grids, and the rapid development of Industry 4.0, the demand for network communication has further increased. The traditional "best-effort" data transmission method of Ethernet can no longer meet the transmission requirements of bounded latency, zero jitter, and precise time synchronization for deterministic services. To address the timeliness and accuracy needs of these emerging real-time and time-sensitive applications, the Internet Engineering Task Force (IETF), an international standardization organization, has proposed several drafts and standards aimed at providing reliable and deterministically guaranteed network transmission for these applications. However, existing technologies are insufficient to guarantee low-latency end-to-end data forwarding in large-scale deterministic networks with time-sensitive applications. Meanwhile, to achieve forwarding of deterministic services, the working group has proposed several candidate technologies based on periodic forwarding, such as Cycle Queuing and Forwarding (CQF) and Cycle Specified Queuing and Forwarding (CSQF). However, how to achieve low-latency end-to-end data forwarding in open, large-scale deterministic networks remains a significant unresolved issue.

[0003] Currently, with the advancement of the Industrial Internet and the evolution of deterministic network standards, deterministic network technology has transitioned from Local Area Networks (LANs) to Wide Area Networks (WANs) and then to large-scale deterministic networks spanning WANs. For deterministic services with stringent requirements regarding latency, jitter, bandwidth, and packet loss, existing key deterministic network technologies mainly include Time-Sensitive Networking (TSN), Deterministic Networking (DetNet), and Deterministic IP (DIP).

[0004] TSN originated from the Audio Video Bridging (AVB) task group of the IEEE 802.1 standards committee, and further evolved into the TSN task group. The TSN task group expanded its application areas from professional audio and video fields to industrial scenarios, automotive networks, and mobile communications, and developed a series of standards based on specific application requirements. These standards cover aspects such as time synchronization, high reliability, deterministic latency, and resource management. TSN achieves deterministic transmission in LANs by establishing a general time-sensitive mechanism for the Ethernet protocol, suitable for real-time applications within small networks. However, the mechanisms in TSN have strict latency limitations and require high-precision clock synchronization, making them insufficient to support deterministic networking scenarios in WANs. With the increasing demand for deterministic networks, the application scope of deterministic networks is gradually expanding beyond the scope of LANs.

[0005] DetNet is a novel network technology developed by the IETF DetNet Working Group, which extends the related protocol standards proposed by the TSN Task Group. To meet the deterministic performance requirements of WANs, DetNet integrates Internet Technology (IT) and Operational Technology (OT), possessing both Layer 2 and Layer 3 capabilities. It aims to solve the problem of implementing deterministic transmission paths on Layer 2 bridging and Layer 3 routing segments, thereby providing extremely low packet loss rates and end-to-end transmission latency for specific real-time applications. The DetNet Working Group extends deterministic networking to WANs through technologies such as Internet Protocol (IP) and Multiprotocol Label Switching (MPLS) to achieve deterministic transmission over a wider range. However, the working group currently primarily studies IP networks under single or closed management control, which has the drawback that IP networks under single or closed management control are insufficient to support solving complex network problems. Therefore, achieving end-to-end low-latency data forwarding in large-scale deterministic networks remains a significant challenge.

[0006] As a novel Layer 3 deterministic network architecture, DIP technology focuses on the data forwarding plane and employs frequency synchronization mechanisms to provide a boundary between low latency, low packet latency variation (jitter), and high reliability. Deterministic network technologies primarily focus on providing reliability guarantees at Layers 2 and 3, without adequately considering forwarding latency control. DetNet, by combining routing protocols, achieves deterministic forwarding at the routing level, extending deterministic networking to WANs, but also increases the difficulty of large-scale network deployment. Therefore, DIP technology introduces a periodic scheduling mechanism as an innovative breakthrough in forwarding technology, achieving large-scale deterministic transmission through periodic and queue mapping. However, this method still requires static network topology and flow settings and cannot effectively address issues such as device and traffic micro-bursts in emerging networks, making it difficult to achieve deterministic transmission. Therefore, without prior dynamic environmental information, DIP technology cannot be well applied in large-scale deterministic networks. Currently, research on deterministic transmission in large-scale, long-distance networks still faces many unresolved issues.

[0007] In open network environments like large-scale deterministic networks, obtaining global network information becomes extremely difficult, and optimizing end-to-end deterministic data forwarding latency remains highly challenging. The aforementioned deterministic network technologies do not consider targeted improvements for this environment, primarily exhibiting the following problems: 1) In large-scale deterministic networks, traffic bursts and congestion are unavoidable and unpredictable. Faced with the characteristics of open networks, complex and flexible topologies, and long-distance link transmission, existing technologies cannot support stable and reliable data forwarding, potentially leading to uncontrolled end-to-end network latency and jitter. Furthermore, deterministic network technology standards are still under development, and existing technologies face numerous challenges in ensuring the timeliness and determinism of data transmission. Therefore, they struggle to meet the demands of new technologies and application scenarios for deterministic services. 2) Currently, existing deterministic network technologies primarily target single or closed-management networks. However, most deterministic services require data forwarding in cross-domain environments, and this type of deterministic data forwarding is insufficient to meet the global deterministic service requirements of open, large-scale networks. Furthermore, traditional single-controller management of open networks suffers from excessive controller load, making it difficult to meet the performance requirements of large-scale networks. Therefore, single-controller management is unsuitable for large-scale deterministic networks. Existing technologies have not yet provided a targeted solution for achieving end-to-end low-latency data forwarding in open, large-scale deterministic networks. Therefore, a new solution is urgently needed to address these technical problems. Summary of the Invention

[0008] This invention addresses the problems existing in the prior art by providing a distributed control method for large-scale deterministic networks. The problem this technical solution aims to solve is to achieve end-to-end low-latency data forwarding control in open, large-scale deterministic networks. It provides a distributed control system and method for large-scale deterministic networks to solve the problems of high latency and jitter in open network environments.

[0009] To address the aforementioned problems, this invention provides a divide-and-conquer approach to managing large-scale deterministic networks. It designs a distributed control system and method for large-scale deterministic networks, enabling distributed processing of these networks. Specifically, this invention divides the large-scale deterministic network into multiple Deterministic Autonomous Systems (Det-AS) based on different network requirements and physical characteristics, and designs inter-domain routing strategies to achieve cross-domain deterministic service flow forwarding in open networks. The goal of this invention is to achieve optimal routing within and between Det-ASes to minimize end-to-end global network data forwarding latency in open networks, thereby achieving more efficient and reliable data processing and management.

[0010] Each Det-AS contains several network devices and physical links within its domain and deploys a controller for traffic management, routing, and forwarding within the domain to minimize intra-domain latency. Each Det-AS adheres to end-to-end global deterministic service constraints, such as latency requirements; intra-domain latency must be less than the global latency limit. Within a Det-AS, network devices communicate only with other network devices within the same domain, while only ERs with special functions can perform cross-domain communication. Multiple ERs can exist within each domain, enabling cross-domain deterministic service flow forwarding through multiple adjacent Det-ASes.

[0011] Deterministic forwarding between Det-ASes abstracts each Det-AS as a special node with intra-domain latency attributes and achieves deterministic inter-domain communication across domains through routing management protocols and routing selection strategies. First, ERs within a domain establish neighbor relationships with adjacent domain nodes through routing management protocols and exchange route reachability information. Simultaneously, each domain node can learn the route reachability information of other domain nodes through the ER. Then, the ER calculates the optimal route forwarding path from this route reachability information based on a certain routing selection strategy and performs deterministic forwarding of data packets according to the optimal path to minimize inter-domain latency.

[0012] In a distributed control system, the controller is responsible for orchestrating data forwarding plane resources, maintaining network topology and status information through the southbound interface; and provides a northbound interface to the application plane to receive requests and instructions from applications; through the eastbound and westbound interfaces between controllers, the network status and network congestion levels of other domains can be understood, so as to carry out cooperation and coordination among distributed controllers, thereby realizing cross-domain data forwarding of the entire open, large-scale deterministic network.

[0013] The technical solution of this invention is as follows: a distributed control system for large-scale deterministic networks, the control system comprising Deterministic Autonomous System (Det-AS), inter-domain communication, Det-AS controller, and deterministic service manager.

[0014] In a Det-AS, multiple independent and deterministic network areas are defined within an open wide-area deterministic network. Each Det-AS is equipped with a controller and several network devices. The controller manages and controls each Det-AS to achieve traffic management and deterministic data forwarding within the domain.

[0015] When performing deterministic forwarding between domains, each domain is abstracted into a node with intra-domain latency attributes. The node exchanges route reachability information with neighboring domain nodes through a routing management protocol. Then, an optimal path is calculated based on the routing selection strategy to perform deterministic data forwarding, thereby minimizing the data forwarding latency between domains.

[0016] The Det-AS controller, where each Det-AS is equipped with one controller, is used for path calculation, resource configuration, protocol distribution, and information collection within the domain, or for collaboration and information exchange with other controllers.

[0017] The Deterministic Service Manager collects user requests and application requests from user equipment on the network side, obtains network information from the network side, and provides network capabilities to users on demand. It is primarily responsible for the configuration and management of deterministic user equipment and network resources.

[0018] The interface relationships between the various components of the control system are as follows:

[0019] In the data forwarding plane, network devices within each domain upload information such as network status, links, and nodes to the controllers in the control plane via the southbound control interface. Simultaneously, each controller receives requests for various deterministic service flows from the application plane via the northbound service interface and issues network configurations, protocols, and routing calculation results via the southbound control interface. Inter-controller interaction is facilitated through eastbound and westbound interfaces for collaboration and coordination, enabling message synchronization and information exchange between domains, thereby facilitating better decision-making and management of cross-domain data forwarding.

[0020] The control system also includes ER and FR. The ER is used to connect adjacent Det-AS and other external networks. It is mainly responsible for deterministic data forwarding and routing between domains. Through the routing management protocol issued by the controller and the routing policy adopted between domains, the deterministic service flow is transmitted from the user equipment at the sending end to the domain of the user equipment at the receiving end through the ER of several domains. The FR is a key network device in the routing forwarding process. It is used to deterministically forward the deterministic service flow entering the domain according to the routing rules and routing calculation results pre-configured in each domain. Finally, the deterministic service flow is sent to the ER of each domain until the terminal device receives the service flow.

[0021] A network architecture for a distributed control system oriented towards large-scale deterministic networks is proposed. This architecture includes an application plane, a control plane, and a data forwarding plane, with each plane being separate from the others and interacting through provided open interfaces.

[0022] The application plane, the upper application layer in this network architecture, primarily consists of a deterministic service manager and deterministic service applications with varying Quality of Service (QoS) requirements. These deterministic applications are those with strict time constraints, such as autonomous driving, smart homes, and smart factories. They configure and manage deterministic user equipment through the deterministic service manager, including acquiring and distributing deterministic service requirements and QoS specifications. The application plane mainly serves deterministic service applications, receiving requests and instructions from applications via API protocols. Simultaneously, it interacts with the controller in the control plane by sending deterministic service flow requirements and routing policies through the service northbound interface, and obtains network status and statistics from the control plane for network resource allocation.

[0023] The control plane primarily consists of one or more controllers. Unlike other network architectures, this invention proposes one controller per Det-AS. Each controller possesses computational, management, and interaction functions, and is primarily responsible for network configuration. Network configuration includes configuring and managing basic network parameters such as flow tables, router port bandwidth limits, and link state information. It mainly targets the network infrastructure within its domain, configuring and managing network devices within its respective domain through protocols such as the Network Configuration Protocol (Netconf). Simultaneously, the controller transmits network state information such as latency through east-west interfaces and interacts with other adjacent controllers to meet interoperability requirements.

[0024] The data forwarding plane connects to the control plane via the control southbound interface and undertakes the actual forwarding and processing tasks of various deterministic services in the wide-area deterministic network. This plane consists of several Det-AS, each of which is composed of network device elements such as ER and FR. At the same time, this plane performs deterministic forwarding of service flows according to the routing rules and policies pre-configured by the controller. In the open wide-area deterministic network environment, deterministic service flows are forwarded by the user equipment at the sending end, pass through several Det-AS and multiple forwarding paths, and are finally sent to the user equipment at the receiving end.

[0025] A distributed control method for large-scale deterministic networks, the method comprising the following steps:

[0026] Step 1: The user equipment at the sending end reports its deterministic service requirements to the Deterministic Service Manager via the API protocol for user configuration. The Det-AS controller transmits the deterministic services from the user side to the network side via the UNI protocol for network management.

[0027] Step 2: On the network side, a distributed control network architecture is adopted to divide the open, large-scale deterministic network into multiple independent Det-AS, and the Det-AS is abstracted into domain nodes with latency attributes. Cross-domain data forwarding is carried out through the construction of domain nodes.

[0028] Step 3: Within Det-AS, each domain is configured and scheduled by a controller. The controller performs route calculations based on the network topology information, traffic requirements, and performance indicators within the domain to obtain the optimal route result within the domain. The calculation result is then sent to the network devices within the domain for data forwarding to minimize latency within the domain.

[0029] Step 4: When communicating between Det-AS, each domain is regarded as a node with intra-domain latency attributes. The controller distributes the routing management protocol to the ERs in each domain. The ERs obtain the routing reachability information of other domain nodes through the routing management protocol and select the optimal path between domains according to the routing selection strategy. The ERs forward deterministic data according to the optimal inter-domain routing path to minimize inter-domain latency.

[0030] Step 5: After the above steps, the intra-domain delay and inter-domain delay will be calculated separately. Intra-domain delay is the delay caused when forwarding data within the same domain; inter-domain delay is the data forwarding delay caused when crossing different domains. This invention then sums the calculated intra-domain and inter-domain delays to obtain a bounded and relatively small end-to-end data forwarding delay for large-scale deterministic networks.

[0031] This solution presents a distributed control system and method for large-scale deterministic networks. It distributes the processing of the current large-scale deterministic network to enable cross-domain forwarding of deterministic service flows in open networks. The solution employs the interface relationships between the components of the distributed control system: network devices in the data forwarding plane interact with the deterministic Autonomous System Manager (Det-AS) in the control plane via the southbound control interface; controllers interact with each other via the eastbound interface; and the deterministic service manager in the application plane interacts with the control plane via the northbound service interface to exchange deterministic service requirements. The network architecture for open wide-area deterministic networks in this solution divides the large-scale deterministic network into multiple Det-ASs according to different network requirements, each deployed and scheduled by its own controller. This network architecture includes an application plane, a control plane, and a data forwarding plane, which are separated from each other and interact through provided open interfaces. In this solution, the deterministic service manager collects user requirements and application requests from user devices accessing the network, obtains network information from the network side, and provides network capabilities to users as needed. The deterministic Autonomous System Controller (Det-AS) collects network status information and distributes network parameter configuration and management within each Det-AS to perform operations such as route calculation and deterministic forwarding. Deterministic Autonomous System (DAS): An open, wide-area deterministic network is divided into multiple independent and deterministic network regions. Each DAS is controlled and managed by a controller.

[0032] Deterministic intra-autonomous system (Autonomous System) data forwarding: Deterministic intra-autonomous system data forwarding is performed based on the intra-autonomous system flow tables issued by the controller. This includes flow management, mapping, scheduling, and deterministic forwarding. Deterministic inter-autonomous system (Autonomous System) data forwarding: The controller calculates the optimal intra-autonomous route and its corresponding latency based on routing operations, and abstracts the intra-autonomous latency into multi-degree domain nodes with latency attributes. Deterministic data forwarding between domain nodes is performed through inter-autonomous system routing tables.

[0033] Compared with the prior art, the present invention has the following advantages: (1) The present invention designs a distributed control system and method for large-scale deterministic networks, which divides the current large-scale deterministic network into multiple deterministic autonomous systems according to different network requirements, physical characteristics, etc., for distributed processing, and realizes the forwarding of end-to-end deterministic service flows in open networks by designing intra-domain and inter-domain routing strategies; (2) It provides a new network architecture for open wide-area deterministic networks, including an application plane (deterministic service manager), a control plane (deterministic autonomous system controller), and a data forwarding plane (for open wide-area deterministic networks, including intra-domain forwarding and inter-domain forwarding. The controller and the deterministic autonomous system have a one-to-one correspondence, and the wide-area deterministic network is divided into multiple deterministic autonomous systems; (3) The interaction method between the planes. The application plane sends deterministic requests to the control plane through the northbound service interface, and the control plane performs deterministic configuration and manages the network devices of the data forwarding plane through the southbound control interface; (4) The topology construction scheme of the domain nodes. The present invention realizes cross-domain by designing a three-layer topology architecture. Data forwarding. The three-layer topology architecture includes the original open network topology, the deterministic autonomous system topology, and the topology of domain nodes with attributes; (5) A scheme to abstract a domain node with a delay attribute for each deterministic autonomous system. Based on different deterministic service requirements, intra-domain network topology, routing paths, etc., multiple delays of each deterministic autonomous system are obtained, and the optimal intra-domain delay is selected through routing strategies; (6) Inter-domain deterministic data forwarding scheme. In the network topology composed of domain nodes, multi-degree domain nodes perform cross-domain deterministic data forwarding according to the inter-domain routing table; (7) Functional modules and roles of the deterministic service manager. User network devices or applications first obtain deterministic service requirements through the service manager, and then forward the user-side requirements to the network side through the deterministic service manager, avoiding the details of intra-network propagation and network configuration; (8) Functional modules and roles of the deterministic autonomous system controller. The deterministic autonomous system controller includes two major functions: scheduling and management, and plays a role in the deterministic autonomous system it is responsible for. Using a distributed controller can solve the problem of excessive load in the single controller management mode in large-scale deterministic networks. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of a network architecture for open wide-area deterministic networks according to an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram illustrating the topology construction process of a domain node according to an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of data request forwarding within a deterministic autonomous region according to an embodiment of the present invention;

[0037] Figure 4This is a schematic diagram of the data forwarding process between deterministic autonomous systems according to an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram illustrating the establishment of end-to-end cross-domain communication for large-scale deterministic networks according to an embodiment of the present invention.

[0039] Figure 6 This is a schematic diagram illustrating the deterministic user configuration and network management operation process according to an embodiment of the present invention. Detailed Implementation

[0040] To enhance understanding of the present invention, the embodiments will be described in detail below with reference to the accompanying drawings.

[0041] Example 1: See Figure 1 A distributed control system for large-scale deterministic networks, the control system comprising Det-AS, inter-domain, Det-AS controller, and deterministic service manager.

[0042] In a Det-AS, multiple independent and deterministic network areas are defined within an open wide-area deterministic network. Each Det-AS is equipped with a controller and several network devices. The controller manages and controls each Det-AS to achieve traffic management and deterministic data forwarding within the domain.

[0043] When performing deterministic forwarding between domains, each domain is abstracted into a node with intra-domain latency attributes. The node exchanges route reachability information with neighboring domain nodes through a routing management protocol. Then, an optimal path is calculated based on the routing selection strategy to perform deterministic data forwarding, thereby minimizing the data forwarding latency between domains.

[0044] The Det-AS controller, where each Det-AS is equipped with one controller, is used for path calculation, resource configuration, protocol distribution, and information collection within the domain, or for collaboration and information exchange with other controllers.

[0045] The Deterministic Service Manager receives and collects user requests and application requests from user equipment on the access network side. It is responsible for configuring and managing deterministic services and provides network service configuration requirements between end-to-end application systems.

[0046] The interface relationships between the various components of the control system are as follows:

[0047] In the data forwarding plane, network devices within each domain upload information such as network status, links, and nodes to the controllers in the control plane via the southbound control interface. Simultaneously, each controller receives requests for various deterministic service flows from the application plane via the northbound service interface and issues network configurations, protocols, and routing calculation results via the southbound control interface. Inter-controller interaction is facilitated through eastbound and westbound interfaces for collaboration and coordination, enabling message synchronization and information exchange between domains, thereby facilitating better decision-making and management of cross-domain data forwarding.

[0048] The control system also includes ER and FR. The ER is used to connect adjacent Det-AS and other external networks. It is mainly responsible for deterministic data forwarding and routing between domains. Through the routing management protocol issued by the controller and the routing policy adopted between domains, the deterministic service flow is transmitted from the user equipment at the sending end to the domain of the user equipment at the receiving end through the ER of several domains. The FR is a key network device in the routing forwarding process. It is used to deterministically forward the deterministic service flow entering the domain according to the routing rules and routing calculation results pre-configured in each domain. Finally, the deterministic service flow is sent to the ER of each domain until the terminal device receives the service flow.

[0049] Example 2: See Figure 1 This paper describes a network architecture for a distributed control system oriented towards large-scale deterministic networks. The network architecture includes an application plane, a control plane, and a data forwarding plane, which are separated from each other and interact through provided open interfaces.

[0050] The application plane, the upper application layer in this network architecture, primarily consists of a deterministic service manager and deterministic service applications with varying QoS levels. These deterministic applications are those with strict time requirements, such as autonomous driving, smart homes, and smart factories. They configure and manage deterministic user equipment through the deterministic service manager, including acquiring and distributing deterministic service requirements and QoS specifications. The application plane mainly serves deterministic service applications, receiving requests and instructions from applications via API protocols. Simultaneously, it interacts with the controller in the control plane by sending deterministic service flow requirements and routing policies through the service northbound interface, and obtains network status and statistics from the control plane for network resource allocation.

[0051] The control plane primarily consists of one or more controllers. Unlike other network architectures, this invention proposes one controller per Det-AS. Each controller possesses computational, management, and interaction functions, and is mainly responsible for network configuration. Network configuration includes configuring and managing basic network parameters such as flow tables, router port bandwidth limits, and link state information. It primarily targets the network infrastructure within its domain, configuring and managing network devices within its respective domain through protocols such as Netconf. Simultaneously, the controller transmits network state information such as latency through east-west interfaces and interacts with other adjacent controllers to meet interoperability requirements.

[0052] The data forwarding plane connects to the control plane via the control southbound interface and undertakes the actual forwarding and processing tasks of various deterministic services in the wide-area deterministic network. This plane consists of several Det-AS, each of which is composed of network device elements such as ER and FR. At the same time, this plane performs deterministic forwarding of service flows according to the routing rules and policies pre-configured by the controller. In the open wide-area deterministic network environment, deterministic service flows are forwarded by the user equipment at the sending end, pass through several Det-AS and multiple forwarding paths, and are finally sent to the user equipment at the receiving end.

[0053] Example 3: See Figures 1-6 A distributed control method for large-scale deterministic networks, the method comprising the following steps:

[0054] Step 1: The user equipment at the sending end reports its deterministic service requirements to the Deterministic Service Manager via the API protocol for user configuration. The Det-AS controller transmits the deterministic services from the user side to the network side via the UNI protocol for network management.

[0055] Step 2: On the network side, a distributed control network architecture is adopted to divide the open, large-scale deterministic network into multiple independent Det-AS, and the Det-AS is abstracted into domain nodes with latency attributes. Cross-domain data forwarding is carried out through the construction of domain nodes.

[0056] Step 3: Within Det-AS, each domain is configured and scheduled by a controller. The controller performs route calculations based on the network topology information, traffic requirements, and performance indicators within the domain to obtain the optimal route result within the domain. The calculation result is then sent to the network devices within the domain for data forwarding to minimize latency within the domain.

[0057] Step 4: When communicating between Det-AS, each domain is regarded as a node with intra-domain latency attributes. The controller distributes the routing management protocol to the ERs in each domain. The ERs obtain the routing reachability information of other domain nodes through the routing management protocol and select the optimal path between domains according to the routing selection strategy. The ERs forward deterministic data according to the optimal inter-domain routing path to minimize inter-domain latency.

[0058] Step 5: After the above steps, the intra-domain delay and inter-domain delay will be calculated separately. Intra-domain delay is the delay caused when forwarding data within the same domain; inter-domain delay is the data forwarding delay caused when crossing different domains. This invention then sums the calculated intra-domain and inter-domain delays to obtain a bounded and relatively small end-to-end data forwarding delay for large-scale deterministic networks.

[0059] With reference to the accompanying diagram, the process of each step is explained in detail below:

[0060] See Figure 6 This section introduces the deterministic business manager and Det-AS controller included in step 1, as well as their specific functional modules and collaborative workflows.

[0061] The Deterministic Service Manager includes a user configuration module. This module primarily consists of uploading deterministic service requirements and distributing deterministic function configurations. The user configuration module establishes a temporary connection with the Det-AS controller when initiating service communication, and is responsible for distributing the acquired deterministic service requirements to controllers within the control plane via the UNI protocol.

[0062] The functional modules of the Det-AS controller include a domain-wide network management module. This module primarily comprises management and scheduling. Management functions include resource management, topology management, and service flow management, while scheduling mainly includes route calculation and data scheduling. The domain-wide network management module is responsible for processing service flows entering the domain. Basic information such as the network topology within each Det-AS is permanently stored, but service flows need to be processed according to the requirements of different user devices. The Det-AS controller issues different configuration commands to the ER and FR. Specifically, the Det-AS controller issues forwarding flow tables to the FR for data forwarding within the domain. Simultaneously, the Det-AS controller issues routing management protocols and routing results within each Det-AS to the ER for establishing inter-domain links and forwarding deterministic data. Furthermore, Det-AS controllers collaborate and coordinate across domains by exchanging route reachability information to ensure the real-time performance and accuracy of service flow transmission.

[0063] The specific workflow for user configuration and network management includes the following steps:

[0064] Step 11: First, the user equipment on each terminal interconnects with the Deterministic Service Manager via the API protocol, establishing a temporary connection. The Deterministic Service Manager, through the user configuration module, performs function retrieval and user requirement acquisition on the terminal's user equipment. It then configures deterministic network characteristics on the terminal user equipment, facilitating the Deterministic Service Manager's configuration of terminal information. This avoids the need for internal network propagation and network configuration details for deterministic services. It also reduces communication overhead and latency in the network, allowing deterministic service requirements to directly influence intra-domain and inter-domain data forwarding processes.

[0065] Step 12: After obtaining user requirements and configuring deterministic features from the Deterministic Service Manager, the domain network management module in the Det-AS controller processes QoS requests from end-user devices and manages the domain network. Each Det-AS controller performs resource management and device management functions through protocols such as Netconf, obtains network topology information within each domain, and collects information on the ability of each network device within each domain to support deterministic network technologies. After obtaining the domain network status information, the Det-AS controller, based on the characteristics of flow management, collects and manages information such as the user's deterministic service requirements and flow characteristics, including source address, destination address, service flow size, and QoS.

[0066] Step 13: Further, after collecting flow information, network resource information, and topology information through various management function modules, operations such as route calculation and resource management are performed. After completing route calculation and other operations, the Det-AS controller distributes the calculation results to each network device within the domain via protocols such as Netconf. Specifically, the Det-AS controller distributes the forwarding flow table to the FRs in each domain and distributes the routing management protocol and latency results within each Det-AS to the ER, thereby guiding the routing and scheduling process of deterministic service flows.

[0067] Step 14: Further, the Det-AS controller establishes a direct connection with other Det-AS controllers by configuring the routing management protocol. Each Det-AS controller exchanges information such as network topology, routing policies, network configuration, and network status through the east-west interface, enabling better deterministic forwarding of cross-domain service flows. The east-west interface is used for communication and cooperation between Det-AS controllers. Det-AS controllers can exchange network status information for their respective managed domains, such as link status, topology changes, and load conditions. This information exchange allows for the creation and understanding of a global network view.

[0068] Step 15: Through the above steps, network devices within each domain forward data according to the configuration and instructions of each Det-AS controller, while between domains, deterministic forwarding of cross-domain service flows is performed according to the optimal route in the inter-domain routing table through a configured routing management protocol. This distributed control and management approach ensures that the entire open, large-scale network can be configured and managed on demand, achieving low-latency data forwarding across global end-to-end cross-domain networks.

[0069] See Figure 2 This section details the process of dividing deterministic autonomous regions and constructing corresponding domain nodes for open large-scale networks in step 2, in order to achieve low-latency forwarding of deterministic service flows.

[0070] The construction process is illustrated using a three-layer architecture. The upper layer represents the network topology formed by the domain nodes corresponding to each Det-AS, the middle layer represents the division of each Det-AS, and the lower layer represents the actual topology of the original open network. Since the determinism of cross-domain latency remains an important issue in open network environments, the distributed control method provided by this invention can abstract a large-scale open network into multiple domain nodes. The number of domains is unlimited, and it supports access from multiple terminals.

[0071] The specific topology construction process for domain nodes is as follows:

[0072] In traditional open, large-scale networks, multiple network devices are integrated to form a complex network topology. When deterministic service flows are sent from the sending user equipment to the receiving user equipment, the forwarding of the service flow involves multiple routing paths. These routing paths are intricate and suffer from issues such as link redundancy and overlap. To meet diverse network and management needs and reduce end-to-end data forwarding latency in open networks, this complex open network can be divided into multiple independent Det-AS (Dedicated Internet Applications). Each Det-AS has its own network devices and management structure to enable internal routing optimization and data transmission control.

[0073] Within each Det-AS, devices from the original open network are mapped to their respective Det-AS. Further, through the replanning and management of Det-AS, a limited number of ERs and multiple FRs are configured for efficient management. The ER plays a crucial role between different autonomous systems, responsible for forwarding deterministic traffic flows across domains. However, in the process of forwarding end-to-end deterministic traffic flows, to calculate the optimal latency for the entire open network, firstly, the optimal latency within each Det-AS is calculated by replanning the routing paths within each Det-AS. Then, cross-domain data forwarding is performed through the ER. Communication between Det-AS is also a complex process, requiring comprehensive consideration of routing paths between different Det-ASes. This invention constructs domain nodes with latency attributes to calculate the optimal path and latency between domains.

[0074] At the domain node topology layer, the calculated optimal latency within each Det-AS is considered an attribute of the domain node, and each domain node with this latency attribute corresponds one-to-one with each domain in the Det-AS topology layer. Inter-domain communication treats each domain as a single entity, implementing deterministic traffic forwarding across domains through the execution of routing management protocols. First, neighbor relationships are established with other domain nodes, exchanging route reachability information through ERs supporting routing management protocols. Upon receiving route reachability information from neighboring domain nodes, the ER processes this routing information. The ER may learn multiple route forwarding paths. For example... Figure 2 As shown, there may be multiple routing paths for deterministic service flows between domain nodes. Then, based on the learned route reachability information, the ER uses these as candidate routes and performs optimal route selection according to the routing strategy adopted between domains to obtain the globally optimal data forwarding latency. Figure 2 As can be seen, the routing paths between domain nodes and the routing paths between Det-AS correspond to each other. Compared to the routing paths of the original open network, the routing paths between domain nodes constructed in this invention are superior.

[0075] See Figure 3 This section details the intra-domain data request forwarding process within a single domain in step 3, including the main entities establishing intra-domain communication and the data forwarding flow.

[0076] The main entities required to establish communication within Det-AS are as follows:

[0077] Within a Det-AS, there are two types of routers: ERs and FRs. They perform different functions and roles in the network. ERs are located at the edge of each domain, connecting different Det-ASs. They can also exchange route reachability information with other domain nodes through the routing management protocol configured by the controller, enabling cross-domain service flow forwarding.

[0078] The Data Forwarding Plane (FR) resides within each domain and is responsible for forwarding data packets within the domain according to certain rules and paths until the service flow reaches the receiving user equipment. The FR in the data forwarding plane can forward deterministic service flows within the domain based on the routing and forwarding paths deployed and configured by the controller.

[0079] The communication process initiated by a single Det-AS and the specific data forwarding process include the following steps:

[0080] Step 31: When forwarding data within a single domain, the user equipment at the sending end or the ER device in another domain first sends one or more deterministic service flows to the ER device in the current domain.

[0081] Step 32: After receiving the service flow, the ER uploads network status information such as network topology, link information, and node information within the domain to the Det-AS controller responsible for the domain in order to perform deterministic data forwarding within the domain.

[0082] Step 33: Each Det-AS controller, based on the collected user information and network topology, will perform tasks such as calculating routing paths within the domain, reserving resources, and managing flows. Specifically, each service flow undergoes service flow processing through the data forwarding processing engine in the routing device, such as... Figure 3 As shown, when a deterministic service flow enters the router's processing engine, it is first classified. Then, the service flow is mapped to the corresponding priority queue based on its flow identifier, flow characteristics, priority, and other attributes. Each queue is assigned a different priority to distinguish between different levels of deterministic service flows, facilitating service flow shaping and scheduling, thereby enabling route calculation, resource reservation, and flow management.

[0083] Step 34: The Det-AS controller configures and manages router devices within the domain based on the routing calculation results, and distributes the domain's forwarding flow tables and routing management protocols to each network device. Based on the priority queues defined in the flow tables, deterministic service flows are mapped to the corresponding queues for queuing. Traffic shaping and scheduling operations are performed based on the traffic conditions, network status, and buffer size of each queue for dequeueing. Simultaneously, the Det-AS controller employs various scheduling algorithms to determine the transmission order of service flows in each queue. These algorithms can schedule service flows based on strategies such as priority, gating lists, bandwidth guarantees, and minimum latency. Once the transmission queue is determined, data packets are retrieved from the corresponding queue and sent to the router's output port, awaiting forwarding to the next-hop router.

[0084] Step 35: After the Det-AS controller distributes the domain flow tables to each network device, each network device configures the flow tables to facilitate deterministic forwarding of service flows. The flow table includes header fields, counters, action tables, and clock synchronization information. The header field information includes flow ID, source address, destination address, input port, output port, and priority. The counter contains statistical information about data packets and service flows. The action table refers to the actions taken when forwarding data packets to the Det-AS controller and router ports, as well as the processing procedures for service flows. The clock synchronization record records the clock synchronization mechanism used and the forwarding cycle of the current service flow.

[0085] Step 36: After configuring flow tables for each network device within the Det-AS, service flows will be forwarded. The next-hop router is selected based on the forwarding flow table configured in the FR to ensure that service flows correctly reach the ER within the Det-AS along the selected path, until the forwarding latency of deterministic service flows within the Det-AS is obtained.

[0086] Step 37: Based on the data forwarding within each Det-AS described above, the routing results for each Det-AS can be obtained, thus acquiring the intra-domain latency and out-degree of the corresponding domain node for each Det-AS. For example, Figure 3 There are two distinct routing paths to the ER, each corresponding to a different intra-domain latency. Each latency corresponds to an outgoing edge of a domain node. Based on the different latencies and their corresponding outgoing degrees, the result is abstracted into a multi-degree domain node with latency attributes. This multi-degree domain node can have multiple links connecting it to other domain nodes.

[0087] Step 38: Further, the out-degree of each multi-degree domain node corresponds to a specific outgoing edge of that node. Through this outgoing edge, service flows can be sent from the current domain node to other domain nodes. Each outgoing edge can connect to different domain nodes. Finally, based on the designed inter-domain routing and forwarding strategy, the optimal intra-domain latency and the corresponding outgoing edge are selected for deterministic data forwarding.

[0088] Throughout this process, traffic classification and selective transmission are implemented to meet the latency and jitter requirements of deterministic service flows, ensuring they are transmitted according to predetermined priorities and paths. This provides reliable network services, adapting to the needs of different levels of deterministic services. Simultaneously, by outputting multi-domain nodes with latency attributes, deterministic service flows are forwarded across domains.

[0089] See Figure 4This section describes the data forwarding process between Det-ASes in step 4. Based on the multi-degree domain nodes with latency attributes corresponding to each Det-AS obtained in step 3, a new network topology is constructed from these domain nodes. This network topology may contain several multi-degree domain nodes. The inter-domain communication establishment and data forwarding process will now be described in detail. Specifically, it includes the following steps:

[0090] Step 41: First, based on the data forwarding within the domain, multiple routing results for each domain can be obtained, the intra-domain forwarding delay t corresponding to each routing path, and the outgoing edge k corresponding to the optimal delay. Each Det-AS is then abstracted as a multi-degree domain node with a delay attribute T. Domain nodes establish connection relationships with adjacent domain nodes through ER to establish inter-domain communication.

[0091] Step 42: When deterministic service flows perform cross-domain data forwarding, the ER supporting the routing management protocol will handle the forwarding of these flows. The ER establishes neighbor relationships with other domain nodes using the routing management protocol. After establishing neighbor relationships, the ER begins exchanging route reachability information. This information includes network state information for each Det-AS, the requirements of the deterministic service, intra-domain latency results, and the corresponding outgoing interface.

[0092] Step 43: After receiving the route reachability information from its neighbor, the ER parses the route information to extract the destination address of the service flow sent by the end user equipment and the route reachability information of the neighboring domain nodes. The ER stores the parsed domain node route information in the inter-domain routing table to record information such as the domain node's route path, destination node, origin node, intra-domain latency, and corresponding outgoing interface. Simultaneously, the ER can also receive route information updates and add them to the inter-domain routing table, updating the information accordingly.

[0093] Step 44: Further, based on different Det-AS delays t and the outgoing edges k of multi-degree domain nodes, the link states between domain nodes have multiple outcomes (such as a, b, and c), and the number of links is finite and deterministic. The number of links between domains is determined by the outgoing edges of each multi-degree domain node, and each delay outcome within a domain corresponds to an outgoing edge of a domain node, facilitating deterministic forwarding of service flows. Therefore, the ER can receive multiple path information leading to the domain of the receiving end's user equipment. The ER uses this path information as candidate routes and performs optimal route selection based on the routing strategy adopted between domains (such as the shortest path first algorithm).

[0094] Step 45: This invention aims to minimize end-to-end latency, therefore a routing path with the minimum latency is selected. Figure 4In the given information, there are three paths, a, b, and c, that can lead from the originating domain T1 (where the sending user equipment resides) to the destination domain T4 (where the receiving user equipment resides), where T1 represents the optimal network latency for domain 1. The ER (Inter-Relational Router) determines the optimal route (e.g., path a) based on a latency-minimizing routing strategy and adds it to the inter-domain routing table. Finally, the ER performs deterministic data forwarding between domains based on the learned route reachability information and routing strategy.

[0095] See Figure 5 Taking the end-to-end cross-domain data forwarding process as an example, this paper introduces the communication establishment process of the entire open, large-scale deterministic network in step 5, as well as the calculation process of end-to-end data forwarding latency. This process will be described using a four-layer architecture, from top to bottom: deterministic service configuration, deterministic controller management, intra-domain data forwarding, and inter-domain data forwarding. Specifically, it includes the following steps:

[0096] Step 51: First, the deterministic service manager collects deterministic service requirements from end-user devices via API protocols, and then manages these deterministic service requirements so that they can be sent to the control plane.

[0097] Step 52: When the deterministic service manager sends deterministic service flows to the network side via the UNI protocol, the network side manages the current large-scale network through the distributed Det-AS control method. First, the Det-AS controller registers network resource requirements such as latency, jitter, and bandwidth, as well as application requirements such as QoS, load balancing, and access control.

[0098] Step 53: Further, after the Det-AS controller obtains deterministic service requirements and network status information within the domain, it configures and manages network devices within the domain using protocols such as Netconf. The Det-AS controller performs route calculations and resource reservations based on the collected network status information and application requirements. After route calculations, each router and other network device within the Det-AS forwards deterministic data within the domain according to the optimal path obtained from the route calculation results. Different routing strategies may select different paths for communication, resulting in network latency and corresponding out-degrees across multiple domains.

[0099] Step 54: The Det-AS controller issues appropriate routing management protocol configuration information to each ER in the domain, as well as the calculation results obtained from the above steps, including the routing results, latency, and corresponding out-degrees within the domain. When performing deterministic communication between domains, each domain is abstracted as a multi-degree domain node with intra-domain latency attributes. Deterministic service flow forwarding across domains is achieved through the interconnection between multi-degree domain nodes.

[0100] Step 55: Data forwarding between multi-domain nodes is performed through inter-domain routing tables, using pre-defined routing selection policies and routing management protocols to achieve deterministic routing. From Figure 5 As can be seen, there may be multiple routing paths between domain nodes. However, in order to obtain end-to-end global low-latency data forwarding, when transmitting service flows between different Det-AS, the service flows cross domains from the initial domain node to the target domain node according to the optimal path.

[0101] Step 56: Further, distributed controllers within the control plane exchange network status information such as latency and bandwidth via east-west interfaces to adjust routing paths between Det-ASes and domains in a timely manner. Through intra-domain and inter-domain coordination and cooperation, deterministic service flows can be successfully received by user equipment at the receiving end. Simultaneously, the deterministic forwarding employed in cross-domain ERs is applicable to various data plane encapsulation methods, including but not limited to IP forwarding, MPLS, and segmented routing. Through the deterministic forwarding mechanism, this cross-domain deterministic communication method can guarantee the performance requirements of service flows, such as latency and jitter. Furthermore, cross-domain communication can support different data plane encapsulation methods, enabling the network to adapt to different application scenarios and needs.

[0102] Step 57: After the above steps, summing the intra-domain data forwarding latency and inter-domain data forwarding latency obtained from distributed computing yields the end-to-end data forwarding latency for a large-scale deterministic network. Finally, verification can be performed to determine whether this latency meets the requirements of deterministic service flows, ensuring the validity of the configuration and enabling deterministic configuration of terminal devices.

[0103] Step 58: Calculate the end-to-end latency, jitter, and other QoS network metrics for the deterministic service flow, and report the results to the controller of the domain where the receiving user equipment resides. If errors or anomalies are found in the metric values, routing needs to be re-executed. The terminal network device reports the message to each Det-AS controller via API and feeds it back to the controller of the sending user equipment's domain. Repeat the above steps until inter-domain cooperative communication can ensure that the cross-domain end-to-end deterministic latency and jitter are within acceptable limits.

[0104] The distributed control network architecture and cross-domain end-to-end low-latency data forwarding control scheme for large-scale deterministic networks provided by this invention can achieve efficient and stable deterministic data forwarding in open network environments such as large-scale networks, WANs, and hybrid networks. This invention can meet the deterministic communication needs of different industries and application fields, and provides an effective solution for achieving reliable cross-domain end-to-end low-latency data forwarding control.

[0105] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention. Equivalent transformations or substitutions made based on the above technical solutions all fall within the scope of protection of the claims of the present invention.

Claims

1. A distributed control system for large-scale deterministic networks, characterized in that, The control system includes Det-AS, inter-domain communication, Det-AS controller, and deterministic service manager, wherein the inter-domain communication is an inter-domain deterministic forwarding structure. In a Det-AS, multiple independent and deterministic network areas are defined within an open wide-area deterministic network. Each Det-AS is equipped with a controller and several network devices. The controller manages and controls each Det-AS to achieve traffic management and deterministic data forwarding within the domain. When performing deterministic forwarding between domains, each domain is abstracted into a node with intra-domain latency attributes. The node exchanges route reachability information with neighboring domain nodes through a routing management protocol. Then, an optimal path is calculated based on the routing selection strategy to perform deterministic data forwarding, thereby minimizing the data forwarding latency between domains. The Det-AS controller, where each Det-AS is equipped with one controller, is used for path calculation, resource configuration, protocol distribution, and information collection within the domain, or for collaboration and information exchange with other controllers. The Deterministic Service Manager receives and collects user requests and application requests from user equipment on the access network side. It is responsible for configuring and managing deterministic services and provides network service configuration requirements between end-to-end application systems. The interface relationships between the various components of the control system are as follows: In the data forwarding plane, network devices in each domain upload their domain's network status, link, and node information to the controller in the control plane through the control southbound interface. At the same time, each controller receives various deterministic service flow requests from the application plane through the service northbound interface, and issues network configurations, protocols, and routing calculation results through the control southbound interface. The interaction between controllers is carried out through the eastbound and westbound interfaces for corresponding cooperation and coordination, so as to facilitate message synchronization and information exchange between domains, thereby making better decisions and management of cross-domain data forwarding.

2. The distributed control system for large-scale deterministic networks according to claim 1, characterized in that, The control system also includes border routers (ERs) and forwarding routers (FRs). ERs connect adjacent Det-ASs and other external networks, and are responsible for deterministic data forwarding and routing between domains. Through the routing management protocol issued by the controller and the routing policies adopted between domains, deterministic service flows are transmitted from the sending user equipment through the ERs of several domains to the domain where the receiving user equipment is located. FRs are key network devices in the routing and forwarding process. They are used to deterministically forward deterministic service flows entering the domain according to the routing rules and routing calculation results pre-configured for each domain, and finally send the deterministic service flows to the ERs of each domain until the terminal equipment receives the service flow.

3. A distributed control method for large-scale deterministic networks, characterized in that, The method includes the following steps: Step 1: The user equipment at the sending end reports the requirements of deterministic services to the deterministic service manager through the application programming interface (API) protocol for user configuration. The Det-AS controller transmits the deterministic services from the user side to the network side through the user network information (UNI) protocol for network management. Step 2: On the network side, a distributed control network architecture is adopted to divide the open, large-scale deterministic network into multiple independent Det-AS, and the Det-AS is abstracted into domain nodes with latency attributes. Cross-domain data forwarding is carried out through the construction of domain nodes. Step 3: Within Det-AS, each domain is configured and scheduled by a controller. The controller performs route calculations based on the network topology information, traffic requirements, and performance indicators within the domain to obtain the optimal route result within the domain. The calculation result is then sent to the network devices within the domain for data forwarding to minimize latency within the domain. Step 4: During communication between Det-AS, each domain is treated as a node with intra-domain latency attributes. The controller distributes the routing management protocol to the boundary routers (ERs) in each domain. The ERs obtain the routing reachability information of other domain nodes through the routing management protocol and select the optimal path between domains according to the routing selection policy. The ERs forward deterministic data according to the optimal inter-domain routing path to minimize inter-domain latency. Step 5: After the above steps, the intra-domain delay and inter-domain delay will be calculated respectively. Intra-domain delay is the delay caused when forwarding data within the same domain; inter-domain delay is the data forwarding delay caused when crossing different domains. The calculated intra-domain and inter-domain delays are then summed to obtain the bounded and relatively small end-to-end data forwarding delay of a large-scale deterministic network.

4. The distributed control method for large-scale deterministic networks according to claim 3, characterized in that, The specific process in step 1 is as follows: The specific workflow for user configuration and network management includes the following steps: Step 11: First, the user equipment of each terminal interconnects with the deterministic service manager through the API protocol to establish a temporary connection. The deterministic service manager uses the user configuration module to perform function retrieval and user requirement acquisition on the user equipment of the terminal, and configures the deterministic network characteristics on the user equipment of the terminal. Step 12: After obtaining user requirements and configuring deterministic features from the deterministic service manager, the domain network management module in the Det-AS controller processes QoS requests from end-user devices and performs domain network management. Each Det-AS controller performs resource management and device management functions through the Netconf protocol, obtains network topology information within each domain, and collects the capabilities of each network device within each domain to support deterministic network technologies. After obtaining the domain network status information, the Det-AS controller collects and manages the user's deterministic service requirements and flow characteristic information based on the characteristics of flow management. Step 13: After collecting flow information, network resource information, and topology information through various management function modules, route calculation and resource management operations are performed. Once the route calculation is complete, the Det-AS controller distributes the calculation results to each network device within the domain via the Netconf protocol. Specifically, the Det-AS controller distributes the forwarding flow table to the forwarding routers (FRs) of each domain and distributes the route management protocol and latency results from each Det-AS to the ER, thereby guiding the routing and scheduling process of deterministic service flows. Step 14: The Det-AS controller establishes a direct connection with other Det-AS controllers by configuring the routing management protocol. Each Det-AS controller exchanges network topology, routing policies, network configuration, and network status information through the east-west interface using the routing management protocol. This facilitates better deterministic forwarding of cross-domain service flows. The east-west interface is used for communication and cooperation between Det-AS controllers. Det-AS controllers can exchange network status information for their respective managed domains, including link status, topology changes, and load conditions. This information exchange allows for the creation and understanding of a global network view. Step 15: Through the above steps, network devices within each domain forward data according to the configuration and instructions of each Det-AS controller, while between domains, deterministic forwarding of cross-domain service flows is performed according to the optimal route in the inter-domain routing table through the configured routing management protocol. This distributed control and management approach ensures that the entire open, large-scale network can be configured and managed on demand, achieving low-latency data forwarding of global end-to-end cross-domain networks.

5. The distributed control method for large-scale deterministic networks according to claim 3, characterized in that, In step 2, the partitioning of Det-AS and the construction of corresponding domain nodes for open large-scale networks are carried out to achieve low latency for deterministic service flows. The construction process is represented by a three-layer architecture: the upper layer represents the network topology composed of the domain nodes corresponding to each Det-AS, the middle layer represents the partitioning of each Det-AS, and the lower layer represents the actual topology of the original open network. The specific topology construction process of the domain nodes is as follows: In a primitive, open, large-scale network, multiple network devices are integrated to form a complex network topology. When deterministic service flows are sent from the sending user equipment to the receiving user equipment, there are multiple routing paths for the service flow forwarding. At the same time, the routing paths are intricate and have multiple problems such as link redundancy and overlap. In order to meet different network and management needs and reduce the end-to-end data forwarding latency of the open network, this open and complex network can be divided into multiple independent Det-AS. Each Det-AS has its own network devices and management structure to realize internal routing optimization and data transmission control. Within each Det-AS, the devices of the original open network are mapped to the corresponding Det-AS. By replanning and managing the Det-AS, efficient management of each domain can be achieved. At the domain node topology layer, the optimal latency calculated in each Det-AS is regarded as the attribute of the domain node. These domain nodes with latency attributes correspond one-to-one with each domain in the Det-AS topology layer.

6. The distributed control method for large-scale deterministic networks according to claim 3, characterized in that, Step 3, the communication process initiated by a single Det-AS and the specific data forwarding process, includes the following steps: Step 31: When forwarding data within a single domain, the sending user equipment or ER device from another domain first sends one or more deterministic service flows to the ER device in the current domain. Step 32: After receiving the service flow, the ER uploads the network topology, link information, node information, and network status information within the domain to the Det-AS controller responsible for that domain, in order to perform deterministic data forwarding within the domain. Step 33: Each Det-AS controller will perform intra-domain routing path calculation, resource reservation, and flow management tasks based on the collected user information and network topology information. Step 34: The Det-AS controller configures and manages router devices within the domain based on the routing calculation results, distributes the domain's forwarding flow tables and routing management protocols to each network device, and maps deterministic service flows to the corresponding queues for queuing according to the priority queues in the flow tables. It performs traffic shaping and scheduling operations based on the traffic conditions, network status, and buffer size in each queue for dequeueing. Simultaneously, the Det-AS controller employs multiple scheduling algorithms to determine the transmission order of service flows in each queue. Step 35: After the Det-AS controller distributes the intra-domain flow tables to each network device, each network device configures the flow tables to facilitate deterministic forwarding of service flows. Step 36: After configuring flow tables for each network device within the Det-AS, service flows will be forwarded. The next-hop router will be selected based on the forwarding flow table configured in the FR to ensure that the service flow correctly reaches the ER within the Det-AS along the selected path, until the forwarding latency of the deterministic service flow within the Det-AS is obtained. Step 37: Based on the data forwarding within each Det-AS, obtain the routing results for each Det-AS, thereby obtaining the intra-domain latency and the outgoing degree of the corresponding domain node for each Det-AS. Based on different latencies and corresponding outgoing degrees, abstract the results into a multi-degree domain node with latency attributes. This multi-degree domain node can have multiple paths connecting to other domain nodes. Step 38: The out-degree of each multi-degree domain node corresponds to a specific outgoing edge of that node. Through this outgoing edge, the service flow can be sent from the current domain node to other domain nodes. Each outgoing edge can connect to different domain nodes. Finally, according to the designed inter-domain routing and forwarding strategy, the optimal latency within the domain and the corresponding outgoing edge are selected for deterministic forwarding of data.

7. The distributed control method for large-scale deterministic networks according to claim 3, characterized in that, Step 4, which involves establishing inter-domain communication and forwarding data, specifically includes the following steps: Step 41: First, based on the data forwarding within the domain, obtain multiple routing results for each domain, the intra-domain forwarding delay t corresponding to each routing path, and the outgoing edge k corresponding to the optimal delay. Then, abstract each Det-AS as a multi-degree domain node with a delay attribute T. Domain nodes establish connections with adjacent domain nodes through ERs to establish inter-domain communication. Step 42: When deterministic service flows perform cross-domain data forwarding, the ER supporting the routing management protocol will handle the forwarding of the cross-domain deterministic service flows. The ER establishes neighbor relationships with other domain nodes using the routing management protocol. After establishing neighbor relationships, the ER begins to exchange routing reachability information. This routing reachability information includes the network state information of each Det-AS, the requirements of the deterministic service, the intra-domain latency results, and the corresponding outgoing interface. Step 43: After receiving the route reachability information from its neighbor, the ER parses the route information, extracts the destination address of the service flow sent by the end user equipment and the route reachability information of the neighboring domain nodes, and stores the parsed domain node route information in the inter-domain routing table to record the domain node's route path, destination node, initial node, intra-domain latency, and corresponding outgoing interface information. Simultaneously, the ER can also receive route information updates, add the updated route information to the inter-domain routing table, and update it accordingly. Step 44: Based on different Det-AS delays t and outgoing edges k of multi-degree domain nodes, the link status between domain nodes can have multiple results. The number of links is finite and deterministic. The number of links between domains is determined by the outgoing edges of each multi-degree domain node, and each delay result within a domain corresponds to an outgoing edge of a domain node, facilitating deterministic forwarding of service flows. Therefore, the ER can receive multiple path information leading to the domain of the receiving end's user equipment. The ER uses this path information as candidate routes and performs optimal route selection based on the routing strategy adopted between domains. Step 45: The ER determines the optimal route path based on the routing selection strategy that minimizes latency, and adds the optimal route path to the inter-domain routing table. Finally, the ER performs deterministic data forwarding between domains based on the route reachability information and routing selection strategy learned above.

8. The distributed control method for large-scale deterministic networks according to claim 3, characterized in that, Step 5 is implemented as follows: Step 51: First, the deterministic service manager collects deterministic service requests from end-user devices via the API protocol, and then manages these deterministic service requests to facilitate their transmission to the control plane. Step 52: When the deterministic service manager sends deterministic service flows to the network side via the UNI protocol, the network side manages the current large-scale network through the distributed Det-AS control method. First, the Det-AS controller registers latency, jitter, and bandwidth network resource requirements, as well as QoS, load balancing, and access control application requirements. Step 53: After the Det-AS controller obtains deterministic service requirements and network status information within the domain, it configures and manages the network devices within the domain through Netconf-related protocols. The Det-AS controller performs route calculations and resource reservation operations based on the collected network status information and application requirements. After the route calculation, each router network device within the Det-AS will perform deterministic data forwarding within the domain according to the optimal path obtained from the route calculation results. Different routing strategies may select different paths for communication, resulting in network latency and corresponding out-degrees across multiple domains. Step 54: The Det-AS controller issues appropriate routing management protocol configuration information to each ER within the domain, along with the calculation results obtained from the above steps, including intra-domain routing results, latency, and corresponding out-degrees. During inter-domain deterministic communication, each domain is abstracted as a multi-degree domain node with intra-domain latency attributes. Cross-domain deterministic service flow forwarding is achieved through the interconnection of multi-degree domain nodes. Step 55: Data forwarding between multi-domain nodes is performed through inter-domain routing tables. Deterministic routing is used between domains based on predefined routing selection policies and routing management protocols in the routing tables. Step 56: Distributed controllers within the control plane exchange latency, bandwidth, and network status information via east-west interfaces to adjust routing paths between Det-ASes and domains in a timely manner. Through intra-domain and inter-domain coordination and cooperation, deterministic service flows can be successfully received by user equipment at the receiving end. Furthermore, the deterministic forwarding used in cross-domain ERs is applicable to various data plane encapsulation methods, including IP forwarding, MPLS, and segmented routing. Step 57: Following the above steps, the intra-domain data forwarding latency and inter-domain data forwarding latency obtained from distributed computing are summed to obtain the end-to-end data forwarding latency of the large-scale deterministic network. Finally, the latency is verified to determine whether it meets the requirements of deterministic service flow, thus ensuring the validity of the configuration, and deterministic configuration is performed on the terminal devices. Step 58: Calculate the end-to-end latency and jitter QoS network metrics of the deterministic service flow, and report the calculation results to the controller of the domain where the receiving user equipment is located. If errors or anomalies are found in the metric values, routing needs to be re-performed. The terminal network device reports the message to the controller of the domain where the sending user equipment is located through the API via each Det-AS controller and feeds it back. Repeat the above steps until inter-domain cooperative communication can ensure that the cross-domain end-to-end deterministic latency and jitter are within an acceptable range.

Images