In-service software upgrade with active service monitoring

Abstract

Embodiments of the present disclosure relate to in-service software upgrade with active service monitoring. A system for performing in-service software upgrade on a network device is provided. In response to a software upgrade command, the system generates an upgrade database based on a state database storing a data plane state and a control plane state associated with the network device. The network device is managed by a management unit including a data plane management subunit and a control plane subunit, and the upgrade database stores at least the data plane state to allow the data plane management subunit to operate based on the upgrade database. The system upgrades the data plane management subunit and the control plane subunit separately without interrupting services provided by the network device. The system monitors the state database and the upgrade database to detect events associated with the network device. In response to determining that a trigger condition is satisfied, the system performs an action to prevent network failure or error.

Description

Technical Field

[0001] This disclosure relates in general to Software In Service Upgrade (ISSU) for network devices. More specifically, this disclosure relates to an ISSU process that, while performing an ISSU on a network device, proactively monitors the services provided by the network device to prevent network failures that may be caused by the ISSU process. Background Technology

[0002] In-Service Software Upgrade (ISSU) is a feature that allows users to perform software (or firmware images) upgrades on a device / system without affecting the services provided by the device / system or restarting the network device / system. In the case of network devices (e.g., switches or routers), enabling ISSU means providing the ability to upgrade their software without interrupting traffic through the network device (e.g., the switch or router continues to forward or route packets) or without interfering with the network (e.g., without causing loops in the network).

[0003] Conventional ISSU methods typically require that during a software upgrade, the device be placed in a state where it cannot respond to changes caused by external events, such as link failures or network reconfigurations. This can lead to upgrade failures or require a time-consuming device reboot. When the device is rebooted, it cannot provide normal service. In some cases, certain external events occurring during a system's ISSU can cause the system or even the entire network to become unstable. Furthermore, existing ISSU implementations in network devices require temporarily disabling some or all control protocols, limiting their deployment to very simple network topologies or requiring users to take numerous preparatory steps before performing the ISSU operation.

[0004] Workarounds to these limitations can be developed using on-device virtualization or containerization. However, such implementations have high hardware requirements, thus increasing the overall cost of the device or making this approach unsuitable for resource-constrained devices. Summary of the Invention

[0005] According to one aspect of this disclosure, a computer execution method is provided for performing software upgrades in a service on a network device. The method includes: generating an upgrade database based on a state database in response to a software upgrade command, the state database storing both data plane state and control plane state associated with the network device, wherein the network device is managed by a management unit including a data plane management subunit and a control plane subunit, and wherein the upgrade database stores at least the data plane state to allow the data plane management subunit to operate based on the upgrade database; upgrading the management unit by respectively upgrading the data plane management subunit and the control plane subunit without interrupting the services provided by the network device; monitoring the control plane state in the state database and the data plane state in the upgrade database to detect events associated with the network device during the upgrade by the management unit; and performing actions to prevent network failures or errors in response to determining that triggering conditions are met based on the detected events and a predefined set of criteria. Attached Figure Description

[0006] Figure 1 A diagram illustrating the separation between the data plane management function block and the control plane function block of a switch management unit according to one aspect of this application is presented.

[0007] Figure 2A The illustration shows a distributed rack switch with in-service software upgrade (ISSU) capability according to one aspect of this application.

[0008] Figure 2B The illustration shows a centralized rack switch with in-service software upgrade (ISSU) capability according to one aspect of this application.

[0009] Figure 2C The illustration shows a stacked switch system with in-service software upgrade (ISSU) capability according to one aspect of this application.

[0010] Figure 3 The diagram illustrates the data plane management path and control plane path in a rack switch according to one aspect of this application.

[0011] Figure 4 A flowchart illustrating the software upgrade process in a dynamic service according to one aspect of this application is presented.

[0012] Figure 5A The illustration shows the separation between the control plane state and the data plane state according to one aspect of this application.

[0013] Figure 5BThe illustration shows a rack switch operating without a data plane management function block according to one aspect of this application.

[0014] Figure 5C The illustration shows a rack switch with an upgraded data plane management function block according to one aspect of this application.

[0015] Figure 5D The illustration shows a rack switch with an upgraded control plane function block according to one aspect of this application.

[0016] Figure 6 The illustration shows a computer system that facilitates a dynamic ISSU process according to one aspect of this application.

[0017] In the accompanying drawings, the same reference numerals refer to the same elements. Detailed Implementation

[0018] The following description is presented to enable any person skilled in the art to make and use these examples and to provide them in the context of a particular application and its requirements. Various modifications to the disclosed examples will be apparent to those skilled in the art, and the general principles defined herein can be applied to other examples and applications without departing from the spirit and scope of this disclosure. Therefore, the scope of this disclosure is not limited to the examples shown, but should be accorded the widest scope consistent with the principles and features disclosed herein.

[0019] This disclosure provides a solution for In-Service Software Upgrade (ISSU) that does not require any service shutdown and can be implemented on resource-constrained devices. To enable ISSU for a network device, the network device's control plane can be decoupled from its data plane management, allowing the network device to continue serving the network while its software is upgraded. Decoupling the control plane can involve generating an ISSU database based on a centralized state database of the network device, allowing the pre-upgrade data plane configuration to be maintained. The ISSU process can be executed on the network device by separately upgrading the data plane management software and the control plane software. While the data plane management software is being upgraded, the control plane software remains in service. After the upgrade is complete, the data plane management software can regain control of the data plane hardware based on the pre-upgrade data plane configuration. The control plane software can be upgraded after the data plane management software upgrade. Separately upgrading the data plane management software and the control plane software may result in inconsistencies between the data plane and control plane states. Therefore, after upgrading the control plane software, the control plane state will converge with the data plane state to complete the entire ISSU process on the network device. The proposed ISSU solution can respond to dynamic changes in the network during the upgrade process to maintain network stability. More specifically, the proposed ISSU system or process can proactively monitor numerous services based on a predefined set of criteria for critical services during the upgrade process to detect external or internal events that may affect network performance or the upgrade process itself. In response to the detection of such events, the ISSU system can perform appropriate actions (e.g., abort the software upgrade or restart network devices) to prevent network failure.

[0020] In-Service Software Upgrade (ISSU) is a feature that allows users to perform software (or firmware images) upgrades on a device / system without affecting the services provided by the device / system or restarting the network device / system. In the case of network devices (e.g., switches or routers), enabling ISSU means providing the ability to upgrade their software without interrupting traffic through the network device (e.g., the switch or router continues to forward or route packets) or without interfering with the network (e.g., without causing loops in the network).

[0021] Conventional ISSU methods typically require that during a software upgrade, the device be placed in a state where it cannot respond to changes caused by external events, such as link failures or network reconfigurations. This can lead to upgrade failures or require a time-consuming device reboot. When the device is rebooted, it cannot provide normal service. In some cases, certain external events occurring during a system's ISSU can cause the system or even the entire network to become unstable. Furthermore, existing ISSU implementations in network devices require temporarily disabling some or all control protocols, limiting their deployment to very simple network topologies or requiring users to take numerous preparatory steps before performing the ISSU operation.

[0022] Workarounds to the aforementioned limitations can be developed using on-device virtualization or containerization. However, such implementations have high hardware requirements, thus increasing the overall cost of the device or making this approach unsuitable for resource-constrained devices. This disclosure provides an ISSU solution that overcomes these limitations without increasing hardware requirements. The proposed ISSU solution allows users to upgrade the software of network devices without disabling any services or network control protocols, and the solution can be implemented in resource-constrained devices such as edge switches. Furthermore, the proposed solution allows users to define a set of criteria for critical services, which can be monitored during the upgrade process to detect external or internal events that may require remedial measures (e.g., pausing or interrupting the upgrade, or restarting the network device) to avoid network instability that could lead to network failure. Because the proposed ISSU solution monitors and reacts to dynamic changes in the network during device upgrades, it can also be referred to as Dynamic ISSU.

[0023] To implement dynamic ISSU on network devices (e.g., switches), the software of the control device (e.g., the switch operating system) can be divided into multiple (e.g., two) functional blocks, which can be separated from each other. For example, the switch operating system can be divided into a data plane management functional block and a control plane functional block. The data plane management functional block controls all processes used to configure hardware (such as packet forwarding application-specific integrated circuits (ASICs)). The control plane functional block can control processes for receiving and sending packets to and from the network (e.g., control packets) or managing user-visible services. Note that both the data plane management functional block and the control plane functional block of the switch operating system can be part of the switch's control plane. The separation between the control plane functional block and the data plane management functional block allows these two functional blocks to be upgraded separately. For example, while the data plane management functional block is being upgraded (which may involve shutting it down and then restarting it with a higher version), the separated control plane functional block continues to operate to facilitate continuous monitoring of the network and internal device status to respond to dynamic changes in the network.

[0024] Furthermore, to enable dynamic ISSU, the control plane state of the device to be upgraded should be separated from its data plane state. The switch's control plane state is a state shared by the switch's management software and the network, and may include networking protocols such as Spanning Tree Protocol (STP), Open Shortest Path First (OSPF), and Border Gateway Protocol (BGP). The switch's data plane state describes the switch's configuration. In one respect, a copy of the device's data plane state can be generated before the upgrade to maintain a stable snapshot of the pre-upgrade data plane configuration. This way, when the data plane management software is upgraded and regains control of the hardware, the hardware configuration can be based on the device's pre-upgrade data plane state. The control plane function blocks can be upgraded after the data plane management function blocks are upgraded. While the data plane state is not dynamically updated during the upgrade, the control plane state is dynamically updated by the control plane software based on network events. After the ISSU process, the control plane state and data plane state can converge.

[0025] Figure 1 A diagram illustrating the separation between the data plane management function block and the control plane function block of a switch management unit according to one aspect of this application is presented. Note that complete separation between the data plane management function block and the control plane function block can occur at the start of the ISSU process. Figure 1 A switch management unit 100 is shown, which may include various software and hardware components for managing and controlling switch operations, such as implementing protocols and forwarding packets.

[0026] exist Figure 1 In the switch management unit 100, a data plane management function block 102, a control plane function block 104, and a status database 106 may be included. The data plane management function block 102 may include components for configuring switch hardware (e.g., packet forwarding ASIC). Figure 1 (Not shown in the image). Control plane function block 104 may include components for receiving and sending packets (e.g., control packets) to the network or managing user-visible services. Figure 1 (Not shown in the image). The state database 106 may include state information associated with the control plane and data plane. The switch's data plane state describes how the data plane hardware is configured. The switch's control plane state is a state generally shared by the switch's management software and the network, and may include network protocols such as Spanning Tree Protocol (STP), Open Shortest Path First (OSPF), Border Gateway Protocol (BGP), etc.

[0027] Furthermore, the switch management unit 100 may include a temporary database, referred to as the ISSU database (or simply the upgrade database) 108. According to one aspect, the ISSU database 108 is generated in response to an upgrade command (e.g., an ISSU command). For example, in response to receiving an upgrade command, the switch management unit 100 can generate the ISSU database 108 by copying the entire state database 106. In this example, the ISSU database 108 is a clone of the state database 106 and contains both the data plane state and control plane state of the switch at the time of cloning. According to one aspect, copying all contents of the state database 106 to the ISSU database 108 can be accomplished using an intelligent "copy-on-write" mechanism, which copies only the content that has changed, thereby reducing the memory requirements of cloning the state database 106. This can be implemented using functionality available in the Linux kernel. In a separate example, in response to receiving an upgrade command, the switch management unit 100 can generate the ISSU database 108 by copying only the data plane state information included in the state database 106. Thus, the ISSU database 108 only contains the data plane state of the switch at the current moment.

[0028] A temporary database (e.g., ISSU database 108) is generated to maintain a copy of the data plane state before the upgrade, allowing for separation between the data plane management function block 102 and the control plane function block 104. In fact, the control plane function block 104 and the state database 106 can form a separate control plane 110, which continues to operate normally (e.g., sending and receiving control packets) without disabling any control protocols when the data plane management function block 102 is upgraded. On the other hand, the upgrade of the data plane management function block 102 can be performed based on the data plane state stored in the ISSU database 108. Note that during the upgrade, the separated control plane does not affect the pre-upgrade data plane state stored in the ISSU database 108, ensuring that the data plane state corresponds to the state of the stable network before the upgrade.

[0029] The switch management unit 100 may include a status monitoring function block 112 for monitoring the data plane status and control plane status included in two databases (i.e., status database 106 and ISSU database 108) to monitor services provided by the switch (e.g., monitoring network events and events on the switch). The status monitoring function block 112 can be activated by an upgrade command, which can originate from a user or a process. Errors on the switch (e.g., hardware failures) or errors in the network (e.g., link failures) or network events that may affect upgrades (e.g., network reconfiguration) can also be monitored by monitoring changes in status.

[0030] To reduce the resources required for monitoring, the status monitoring function block 112 does not need to monitor all services or all aspects of the network. Depending on one aspect, the status monitoring function block 112 can be configured to monitor services selected based on a predefined set of criteria. Service selection criteria can be user-defined or automatically generated by the upgrade process. For example, the status monitoring function block 112 can be configured to monitor services considered critical (e.g., services essential to applications or affecting network stability). Services or service groups can be defined as critical based on their type and application requirements. For example, for financial applications, services associated with user authentication might be considered critical.

[0031] In addition to their type, services can also be defined as critical based on other criteria associated with the service, such as ports, interfaces, coverage (or tunnels), and AAA (Authentication, Authorization, and Accounting) events. During the upgrade process, the State Monitoring Function Block 112 can be configured to monitor events that may affect these services, allowing the upgrade process to be aborted if necessary to minimize the overall impact beyond the upgraded device. Examples of critical services may include control plane protocols, control plane packet management, data plane services, and physical ports. For example, a port or set of ports on a switch can be defined as critical (e.g., for connecting to the core switch). Therefore, the State Monitoring Function Block 112 can be configured to monitor the status of these ports during the upgrade process. If the State Monitoring Function Block 112 detects an event associated with a port requiring action from the switch to avoid network-wide failure (e.g., a link event or port failure), the switch can take action. In one example, the switch can abort or rollback the upgrade process to reset the switch to a previous state where the required action can be taken.

[0032] Furthermore, certain protocols can also be defined as critical, allowing the status monitoring function block 112 to be configured to monitor network events related to critical protocols. For example, a user can define STP as critical. Therefore, the status monitoring function block 112 can be configured to monitor network reconvergence events that may cause loops in the network. In addition to events occurring in the network or on the switch hardware, the status monitoring function block 112 can also be configured to monitor internal failures of the switch management unit 100 (e.g., process crashes) that may affect the network and cannot be recovered during upgrade processes.

[0033] The switch management unit 100 may also include an upgrade control function block 114 for controlling the upgrade process. More specifically, the upgrade control function block 114 may be configured to receive event notifications from the status monitoring function block 112 and perform actions in response to one or more events that meet trigger conditions. According to one aspect, the actions performed by the upgrade control function block 114 may include, but are not limited to: pausing the upgrade process, rolling back the upgrade process to a previous state, aborting the upgrade process, restarting the device being upgraded, etc. Examples of trigger conditions may include, but are not limited to: the possibility of upgrade failure, the possibility of network instability, the possibility of failure or interruption of critical services, etc. For example, during the upgrade process, the status monitoring function block 112 may detect a fault in a port connected to the core switch, and the upgrade control function block 114 may determine that ignoring such a fault during the upgrade could lead to network instability. In response, the upgrade control function block 114 may take action to abort the upgrade process to allow the switch to take remedial actions, such as redirecting traffic to a backup port. Once the problem is resolved, the upgrade control function block 114 may restart the upgrade process. In another example, during the upgrade process, status monitoring function block 112 detects a reconvergence event in the network that could potentially cause a traffic loop. In response, upgrade control function block 114 can abort the upgrade process to allow the switch to react to the network reconvergence event to prevent network-wide traffic loops.

[0034] The switch management unit 100 can be used to manage different types of switches, including rack switches and stacked switches. Figure 2A The illustration depicts a distributed rack switch with in-service software upgrade (ISSU) capability according to one aspect of this application. Figure 2A In the rack switch 200, there are a management unit 202, a backup management unit 204, and several line cards (e.g., line cards 206-212) connected to the backplane 214.

[0035] According to one embodiment, management unit 202 and standby management unit 204 may each include processing resources (e.g., processor and memory) and various interfaces that can facilitate the management and control of rack switch 200. Figure 2A (Not shown in the diagram). Furthermore, management unit 202 may include a data plane management subunit 216, a status database 218, and a control plane subunit 220. The status database 218 stores information associated with the data plane and control plane states of the rack switch 200. Backup management unit 204 provides redundancy and is typically placed in standby mode; its own data management subunit and control plane subunit are not activated (therefore in...). Figure 2A(Not shown in the image). The backup management unit 204 may include a backup status database 222, which may be synchronized with the status database 218.

[0036] exist Figure 2A In the example shown, rack switch 200 has a distributed architecture, meaning that each line card has its own built-in processing resources (e.g., processor and memory) to manage the data plane hardware on the line card. For example, line card 206 may include a data plane management subunit 224 that can receive management packets from data plane management subunit 216 in management unit 202 to manage / control the data plane hardware 226 on line card 206. Figure 2A It is also shown that each line card may include multiple ports (e.g., port 1 and port 2) for general interface connections with the network (e.g., sending packets to and receiving packets from the network).

[0037] Figure 2B The illustration depicts a centralized rack switch with in-service software upgrade (ISSU) capability according to one aspect of this application. Figure 2B In this configuration, the rack switch 230 may include a management unit 232, a backup management unit 234, and multiple line cards (e.g., line cards 236-242) attached to the backplane 244. The management unit 232 and the backup management unit 234 can be connected to... Figure 2A The management unit 202 and the backup management unit 204 shown are similar. However, with Figure 2A Unlike the distributed architecture shown, rack switch 230 has a centralized architecture, meaning all processing is handled by a centralized management unit 232, and line cards have no processing capabilities. For example... Figure 2B As shown, there is no data plane management subunit on each line card to control the data plane components; instead, the management and control of the data plane components are performed by the data plane management unit 232.

[0038] Figure 2C The illustration depicts a stacked switch system with in-service software upgrade (ISSU) capability according to one aspect of this application. The stacked switch system 250 may include multiple stacked switches, including a conductor switch 252 and a backup switch 270. The conductor switch 252 may include a management unit 254, and the backup switch 270 may include a backup management unit 272. The management unit 254 and the backup management unit 272 may be respectively similar to... Figure 2AThe diagram shows management unit 202 and standby management unit 204. For example, management unit 204 may include a data plane management subunit 256, a control plane subunit 258, and a status database 260; standby management unit 272 may include a standby database 274. Additionally, each switch may include its own data plane management subunit, which can receive management packets from data plane management subunit 256 of management unit 254 to manage / control the data plane hardware on the switch. For example, conductor switch 252 may include data plane management subunit 262 that manages data plane hardware 264, and standby switch 270 may include data plane management subunit 276 that manages data plane hardware 278.

[0039] Please note, Figures 2A-2C Only the active components within the switch are illustrated prior to the upgrade process performed on the switch. Therefore, the ISSU database is not shown in the management unit of each switch, as it is generated in response to the upgrade command at the start of the upgrade process.

[0040] like Figures 2A-2C As shown, the dynamic ISSU solution can be implemented in different types of switches by separating data plane management functions and control plane functions. As previously mentioned, data plane management functions are responsible for configuring switch hardware (e.g., packet forwarding ASICs). Taking a distributed rack switch as an example, the data plane management functions executed by the line card processor can receive management packets from the data plane management functions executed by the rack processor to configure the hardware on the line cards. On the other hand, the control plane functions executed by the rack processor can send or receive network packets via the data plane hardware. Note that packet forwarding from the data plane hardware to the control plane function blocks is independent of the data plane management function blocks themselves.

[0041] Figure 3 The diagram illustrates the data plane management path and control plane path in a rack switch according to one aspect of this application. Figure 3 In the middle, the rack switch 300 can be similar to Figure 2A The rack switch 200 shown may include a management unit 302, a backup management unit 304, and several line cards (e.g., line cards 306-312) attached to a backplane 314. Figure 3 A data plane management path is illustrated (e.g., path 316 between the data plane management subunit in management unit 302 and the data plane management subunit in line card 306). For example, the data plane management subunit in management unit 302 can send / receive management packets to / from the data plane management subunit in line card 306 via path 316 to facilitate the data plane management subunit in managing / configuring the data plane hardware in line card 306. Figure 3In the text, data plane management paths are marked with solid double arrows.

[0042] In addition to the data plane management path, Figure 3 A control plane path (e.g., path 318 between the control plane subunit in management unit 302 and the data plane hardware in line card 312) is also shown. For example, the control plane subunit in management unit 302 can send / receive network packets (e.g., control packets) to / from the data plane hardware in line card 312 via path 318. The data plane hardware can transmit packets to and receive packets from an external network. Figure 3 In the diagram, the control plane path is marked with a dashed double arrow.

[0043] Both the data plane management path and the control plane path of the line card can be implemented using the same hardware, such as a peripheral component interconnect (PCIe) bus that couples the rack processor and the line card. For example, data plane management path 316 could be a PCIe bus connecting management unit 302 and line card 306, and control plane path 318 could be a PCIe bus connecting management unit 302 and line card 312. However, separate hardware paths can also be used to implement the data plane management path and the control plane path. Because the communication between the data plane management and the control plane is independent of the implemented hardware, the dynamic ISSU solution is flexible enough to be implemented in different types of switches, including but not limited to rack switches with distributed processing, rack switches with centralized processing, and stacked switches.

[0044] Figure 4 A flowchart illustrating a software upgrade (ISSU) process in a dynamic service according to one aspect of this application is presented. During operation, the network device to be upgraded receives a command to start the ISSU process (operation 402). The command can be received from a user (e.g., via a command-line interface (CLI)) or from the process. The network device can be a switch, router, Wi-Fi access point, Internet of Things (IoT) device, etc. Because this dynamic ISSU solution does not require virtualization or containerization, the hardware requirements for the device to be upgraded can be low, making the dynamic ISSU solution suitable for small-scale, low-cost devices. To enable dynamic ISSU, the management unit (e.g., processor) of the device to be upgraded should have a data plane management function block, a separate control plane function block, and a centralized state database, and... Figures 2A-2C Similar to the example shown. A centralized database stores both the data plane state and the control plane state. Note that during normal operation, the data plane state can be updated by the data plane management function block to reflect changes in the data plane hardware, and by the control plane function block to reflect changes in the network. The control plane function block can also update the control plane state based on changes in the network.

[0045] In response to receiving an ISSU command, the device generates a copy of at least the state information associated with the data plane (operation 404). According to one aspect of this application, a clone of the state database can be generated. The cloned database (also referred to as the ISSU database) may include the data plane state and control plane state of the device to be upgraded. According to another aspect, the ISSU database can be generated by copying only the data plane state, thereby reducing memory usage by including only a copy of the data plane state. Various mechanisms can be used to generate the ISSU database. For example, a smart "copy-on-write" mechanism can be used to reduce the memory requirements of copying. The generation of the ISSU database allows for the separation of the control plane state and the data plane state, where the control plane state is retained in the state database, and the data plane state at the start of the upgrade is retained in the ISSU database. Note that once the ISSU database is generated, the data plane management function block can operate based on the ISSU database, while the control plane function block operates independently based on the state database. During the ISSU process, the control plane function block updates the control plane state based on dynamic changes in the network without affecting the data plane state in the ISSU database.

[0046] Figure 5A The illustration shows the separation between the control plane state and the data plane state according to one aspect of this application. Taking a rack switch as an example... Figure 5A The rack switch 500 is shown to include a management unit 502, a backup management unit 504, and multiple line cards (e.g., line cards 506-512) attached to a backplane 514.

[0047] Figure 5A It is also shown that the ISSU database 516 is generated by cloning the state database 518, such that the data plane state used by the data plane management function block is stored in the ISSU database 516, while the control plane state is stored in the state database 518. The control plane function block within the management unit 502 can now operate independently of the data plane management function block.

[0048] return Figure 4 After generating a copy of the data plane state, the device can upgrade the data plane management function block. This may include first shutting down or terminating the data plane management function block (operation 406), and then restarting the upgraded version of the data plane management function block (operation 408). Note that the upgraded version of the data plane management function block has previously been downloaded (e.g., stored in memory). While the data plane management function block is being upgraded, the control plane function block continues to operate normally, and the synchronization between the control plane state and the network state remains stable.

[0049] When the data plane management function block is disabled during the upgrade process, the device's data plane operates in autonomous mode. In other words, the device operates without its hardware management software running. Figure 5B The illustration shows a rack switch operating without a data plane management function block according to one aspect of this application.

[0050] To reduce the hardware requirements (e.g., memory and processing power) for implementing dynamic ISSU, according to one aspect, when the upgraded data plane management function block is restarted, it can migrate to a standby management unit (e.g., a standby processor), and the contents of the ISSU database are synchronized to the standby state database. This allows the upgraded data plane management function block to use the data plane state in the standby state database to control the data plane hardware. Figure 5C The illustration shows a rack switch with an upgraded data plane management function block according to one aspect of this application. Figure 5C In this process, the upgraded Data Plane Management Function Block 520 is migrated to the Standby Management Unit 504, and the data plane state stored in the Standby State Database 522 is synchronized with the data plane state stored in the ISSU database. After synchronization, the ISSU database can be deleted. When the Data Plane Management Function Block 520 is reattached to the data plane hardware on the line card, it uses the data plane state stored in the Standby State Database 522 to configure the hardware. Note that the data plane state in the ISSU database is not updated by the control plane during the ISSU process. It is assumed that the network was stable before the ISSU process. Therefore, the pre-upgrade data plane state reflects the state of the device in a stable network. Starting the upgraded Data Plane Management Function Block 520 using the pre-upgrade data plane state is unlikely to cause errors in the device or network.

[0051] return Figure 4 After the data plane management function block is upgraded, the device can upgrade the control plane function block (operation 410). According to one aspect, upgrading the control plane function block may also involve first terminating the currently running control plane function block and then restarting the upgraded version of the control plane function block. When the control plane function block is upgraded, the device temporarily loses its control plane. However, the downtime of the control plane is very brief (e.g., less than a few seconds) and can be comparable to the time required to switch the management of the device from the current management unit to a standby management unit in response to a fault. Figure 5D The illustration depicts a rack switch with an upgraded control plane function block according to one aspect of this application. The upgraded control plane function block can be started on the same management unit (e.g., the same processor) that executes the upgraded data plane management function block. Figure 5DIn the example shown, the upgraded control plane function block 524 is also migrated to the standby management unit 504. In fact, the standby management unit 504 no longer operates in standby mode, and the standby state database becomes the primary state database, whose stored data plane and control plane states are continuously updated by the upgraded data plane management function block and the upgraded control plane function block 524. On the other hand, the previously active management unit 502 is placed in standby mode, and its state database becomes the standby state database. Figure 5D It is shown that after ISSU, the upgraded data plane management function block and control plane function block were migrated to different management units, and the previous management unit was placed in standby mode.

[0052] Return again Figure 4 After upgrading the control plane function block, the data plane state and control plane state can converge (operation 412), thus completing the ISSU process. Because the data plane state in the ISSU state database is not updated by the control plane function block during the ISSU process (i.e., it is "frozen" to its state at the start of the upgrade), while the control plane state continues to be updated by the control plane, the data plane state and control plane state may diverge (e.g., the data plane state may become inconsistent with the network state due to network changes during the upgrade process). For stable networks, a small amount of divergence is manageable, and the data plane state and control plane state in the state database can converge after the upgrade process. However, large divergences between the data plane state and control plane state can affect network stability and should be avoided.

[0053] To avoid network instability caused by the upgrade process, Figure 4It is also shown that when ISSU is in progress, the device actively monitors the dynamic state of the network (or services in the control plane) and the dynamic state of the device (or hardware state in the data plane) (Operation 414). According to one aspect, such monitoring can be accomplished by monitoring the state associated with the device (e.g., the data plane state in the ISSU database and the control plane state in the state database) during the upgrade process. To conserve resources, not all services are monitored. According to one aspect, only services that meet a predefined set of criteria are monitored. One criterion may be that the service is "critical". A service may be critical based on its type or its associated port, interface, coverage, or AAA events associated with the service. When an event associated with a critical service is detected, the device can determine whether a trigger condition is met (Operation 416). Exemplary trigger conditions may include, but are not limited to: the event may cause upgrade failure, the event may cause network instability, the detected event may cause network failure or error, the event may cause critical service failure or interruption, etc. In addition to external events (e.g., events on the network or device), certain internal events may also meet the trigger conditions (e.g., the process crashes and cannot be restarted during the upgrade). If the trigger condition is met, remedial actions can be taken (Operation 418). Remedial actions may include, but are not limited to: pausing the upgrade process, rolling the upgrade process back to a previous state, terminating the upgrade process, or restarting the device being upgraded.

[0054] Figure 6 The illustration depicts a computer system facilitating a dynamic ISSU process according to one aspect of this application. The computer system 600 includes a processor 602, a memory 604, and a storage device 606. Furthermore, the computer system 600 may be coupled to a peripheral input / output (I / O) user device 610, such as a display device 612, a keyboard 614, and a pointing device 616. The storage device 606 may store an operating system 618, a dynamic ISSU system 620, and data 640. According to one aspect, the computer system 600 may be part of a network device (e.g., a switch or router).

[0055] The dynamic ISSU system 620 may include instructions that, when executed by the computer system 600, cause the computer system 600 or processor 602 to perform the methods and / or processes described in this disclosure. Specifically, the dynamic ISSU system 620 may include instructions for generating a copy of the data plane state (state copy instruction 622), instructions for upgrading data plane management function blocks (data plane management upgrade instruction 624), instructions for upgrading control plane function blocks (control plane upgrade instruction 626), instructions for monitoring the data plane and control plane states to detect events on network devices and in the network (state monitoring instruction 628), instructions for determining whether a detected event meets triggering conditions based on a predefined set of criteria (triggering condition determination instruction 630), and instructions for performing appropriate actions in response to a detected event (action execution instruction 632). Data 640 may include a state database 642 and a temporary ISSU database 644.

[0056] Overall, this disclosure provides systems and methods for providing In-Service Software Upgrade (ISSU), which can be flexibly implemented across a variety of network devices, including low-cost, resource-constrained network devices. To enable ISSU, the management / control software of the network device to be upgraded should decouple its control plane software from its data plane management software. During normal operation of the network device, both the control plane software and the data plane management software operate based on a centralized state database. When the ISSU process is initiated, a temporary ISSU database is generated to store at least one copy of the data plane state. During the ISSU process, the data plane management software operates based on the ISSU database, and the control plane software operates based on the state database. This allows the data plane management software and control plane software to be upgraded separately without interrupting the services provided by the network device or disabling any control protocols. Furthermore, during the ISSU process, the data plane state in the ISSU database and the control plane state in the state database are monitored to detect internal and external events that may affect the upgrade process or network stability. In response to detected events meeting trigger conditions, appropriate actions can be taken to prevent errors or failures in the network.

[0057] One aspect of this application provides a system and method for performing software upgrades in service on a network device. During operation, in response to a software upgrade command, the system generates an upgrade database based on a state database that stores data plane and control plane states associated with the network device. The network device is managed by a management unit including a data plane management subunit and a control plane subunit, and the upgrade database stores at least the data plane state to allow the data plane management subunit to operate based on the upgrade database. The system upgrades the management unit by upgrading the data plane management subunit and the control plane subunit separately without interrupting the services provided by the network device. The system monitors the control plane state in the state database and the data plane state in the upgrade database to detect events associated with the network device during management unit upgrades. In response to determining that triggering conditions are met based on the detected events and a predefined set of criteria, the system performs actions to prevent network failures or errors.

[0058] In a variation of this approach, generating an upgrade database involves either creating a clone of the state database or copying the data plane state into the upgrade database.

[0059] In one variation of this approach, upgrading the data plane management subunit includes: terminating the data plane management subunit without affecting the operation of the control plane subunit; implementing a newer version of the data plane management subunit; and reattaching the newer version of the data plane management subunit to the data plane hardware on the network device based on the data plane status in the upgrade database.

[0060] In a further variation, the management unit executes on the first processor, and the execution of a newer version of the data plane management subunit includes executing a newer version of the data plane management subunit on a second standby processor.

[0061] In further variations, the upgraded control plane subunit includes: terminating the control plane subunit, and executing a newer version of the control plane subunit on a second standby processor.

[0062] In variations of this approach, the predefined set of criteria includes one or more of the following: the detected event is associated with a critical service; the detected event is associated with a critical port; the detected event is associated with a critical interface; the detected event is associated with critical coverage; the detected event is a predefined AAA (Authentication, Authorization, and Accounting) event; the detected event is associated with a predefined network protocol; and the detected event is an internal failure of the management unit.

[0063] In one variation of this, the triggering conditions include one or more of the following: the detected event may cause a software upgrade in the service to fail; the detected event may cause network instability; the detected event may cause a network failure or error; and the detected event may interrupt critical services.

[0064] In variations of this approach, the actions include one or more of the following: pausing the management unit upgrade; rolling back the management unit upgrade to a previous state; aborting the management unit upgrade; and restarting the network device.

[0065] In a variation of this, the upgrade management unit further includes converging the data plane states in the upgrade database and the control plane states in the state database.

[0066] The methods and processes described in the detailed description section can be embodied in code and / or data that can be stored in a computer-readable storage medium as described above. When a computer system reads and executes the code and / or data stored on the computer-readable storage medium, the computer system executes the methods and processes embodied in data structures and code and stored within the computer-readable storage medium.

[0067] Furthermore, the methods and processes described above can be included in hardware modules or devices. Hardware modules or devices may include, but are not limited to, application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), dedicated or shared processors that execute specific software modules or code segments at specific times, and other programmable logic devices now known or developed later. When a hardware module or device is activated, it executes the methods and processes included therein.

[0068] The foregoing description is presented for illustrative and descriptive purposes only. It is not intended to be exhaustive or to limit the scope of this disclosure to the form disclosed. Therefore, many modifications and variations will be apparent to those skilled in the art.

Claims

1. A computer execution method for performing software upgrades in a service on a network device, the method comprising: In response to a software upgrade command, an upgrade database is generated based on a state database, the state database storing both the data plane state and the control plane state associated with the network device, wherein the network device is managed by a management unit, the management unit including a data plane management subunit and a control plane subunit, and wherein the upgrade database stores at least a copy of the data plane state prior to the upgrade; Without interrupting the services provided by the network device, the management unit is upgraded by separately upgrading the data plane management subunit and the control plane subunit, wherein upgrading the data plane management subunit includes: The data plane management subunit is terminated without affecting the operation of the control plane subunit; Restart the data plane management subunit to a newer version; and Based on the copy of the data plane state prior to the upgrade stored in the upgrade database, the newer version of the data plane management subunit is reattached to the data plane hardware on the network device; Monitor the control plane status in the status database and the data plane status in the upgrade database to detect events associated with the network device during the upgrade of the management unit; and In response to the detection of the event and a predefined set of criteria, determining that the triggering condition has been met, actions are taken to prevent network failures or errors.

2. The method of claim 1, wherein generating the upgrade database comprises: Create a clone of the state database or copy the data plane state to the upgrade database.

3. The method of claim 1, wherein the management unit is executed on a first processor, and wherein executing the newer version of the data plane management subunit includes executing the newer version of the data plane management subunit on a second standby processor.

4. The method of claim 3, wherein upgrading the control plane subunit comprises: The control plane subunit is terminated; as well as A newer version of the control plane subunit is restarted on the second backup processor.

5. The method according to claim 1, wherein the predefined set of standards includes one or more of the following: The detected events are associated with critical services; The detected events are associated with key ports; The detected events are associated with key interfaces; The detected events are associated with critical coverage; The detected events are pre-defined Authentication, Authorization, and Billing (AAA) events; The detected events are associated with predetermined network protocols; as well as The detected event is an internal fault of the management unit.

6. The method of claim 1, wherein the triggering condition includes one or more of the following: The detected event could cause a software upgrade to fail in the service; The detected events can lead to instability in the network; The detected events can cause malfunctions or errors in the network; as well as The detected events could disrupt critical services.

7. The method of claim 1, wherein the action comprises one or more of the following: Suspend the upgrade of the management unit; Roll back the upgrade of the management unit to its previous state; The upgrade of the management unit is suspended; as well as Restart the network device.

8. The method of claim 1, wherein upgrading the management unit further comprises: The data plane states in the upgrade database and the control plane states in the state database are converged.

9. A computer system, comprising: processor; as well as A storage device, coupled to the processor and storing instructions, which, when executed by the processor, cause the processor to perform a method for facilitating software upgrades in services on a network device, the method comprising: In response to a software upgrade command, an upgrade database is generated based on a state database, the state database storing both the data plane state and the control plane state associated with the network device, wherein the network device is managed by a management unit, the management unit including a data plane management subunit and a control plane subunit, and wherein the upgrade database stores at least a copy of the data plane state prior to the upgrade; Without interrupting the services provided by the network device, the management unit is upgraded by separately upgrading the data plane management subunit and the control plane subunit, wherein upgrading the data plane management subunit includes: The data plane management subunit is terminated without affecting the operation of the control plane subunit; Restart the data plane management subunit to a newer version; and Based on the copy of the data plane state prior to the upgrade stored in the upgrade database, the newer version of the data plane management subunit is reattached to the data plane hardware on the network device; Monitor the control plane status in the status database and the data plane status in the upgrade database to detect events associated with the network device during the upgrade of the management unit; and In response to the detection of the event and a predefined set of criteria, determining that the triggering condition has been met, actions are taken to prevent network failures or errors.

10. The computer system of claim 9, wherein generating the upgrade database comprises: Create a clone of the state database or copy the data plane state to the upgrade database.

11. The computer system of claim 9, wherein the management unit is executed on a first processor, and wherein executing the newer version of the data plane management subunit includes executing the newer version of the data plane management subunit on a second standby processor.

12. The computer system of claim 11, wherein upgrading the control plane subunit comprises: The control plane subunit is terminated; as well as A newer version of the control plane subunit is restarted on the second backup processor.

13. The computer system of claim 9, wherein the predefined set of standards includes one or more of the following: The detected events are associated with critical services; The detected events are associated with key ports; The detected events are associated with key interfaces; The detected events are associated with critical coverage; The detected events are pre-defined Authentication, Authorization, and Billing (AAA) events; The detected events are associated with predetermined network protocols; as well as The detected event is an internal fault of the management unit.

14. The computer system of claim 9, wherein the triggering condition includes one or more of the following: The detected event could cause a software upgrade to fail in the service; The detected events can lead to instability in the network; The detected events can cause malfunctions or errors in the network; as well as The detected events could disrupt critical services.

15. The computer system of claim 9, wherein the action includes one or more of the following: Suspend the upgrade of the management unit; Roll back the upgrade of the management unit to its previous state; The upgrade of the management unit is suspended; as well as Restart the network device.

16. The computer system of claim 9, wherein upgrading the management unit further comprises: The data plane states in the upgrade database and the control plane states in the state database are converged.

17. A non-transitory computer-readable storage medium storing instructions, said instructions, when executed by a computer, causing the computer to perform a method for facilitating software upgrades in services on a network device, said method comprising: In response to a software upgrade command, an upgrade database is generated based on a state database, the state database storing both the data plane state and the control plane state associated with the network device, wherein the network device is managed by a management unit, the management unit including a data plane management subunit and a control plane subunit, and wherein the upgrade database stores at least a copy of the data plane state prior to the upgrade; Without interrupting the services provided by the network device, the management unit is upgraded by separately upgrading the data plane management subunit and the control plane subunit, wherein upgrading the data plane management subunit includes: The data plane management subunit is terminated without affecting the operation of the control plane subunit; Restart the data plane management subunit to a newer version; and Based on the copy of the data plane state prior to the upgrade stored in the upgrade database, the newer version of the data plane management subunit is reattached to the data plane hardware on the network device; Monitor the control plane status in the status database and the data plane status in the upgrade database to detect events associated with the network device during the upgrade of the management unit; and In response to the detection of the event and a predefined set of criteria, determining that the triggering condition has been met, actions are taken to prevent network failures or errors.

18. The non-transitory computer-readable storage medium according to claim 17, wherein... The predefined set of standards includes: The detected events are associated with critical services; The detected events are associated with key ports; The detected events are associated with key interfaces; The detected events are associated with critical coverage; The detected events are pre-defined Authentication, Authorization, and Billing (AAA) events; The detected events are associated with predetermined network protocols; as well as The detected event is an internal fault of the management unit.

Images