Method, system, and computer-readable medium for managing multiple software versions in a software-defined wide area network (SD-WAN)

Abstract

A method for managing multiple software-defined wide area network (SD-WAN) software versions includes providing a controller that defines an overall configuration for the SD-WAN and creates configuration registries for individual SD-WAN devices that implement their respective portions of the overall configuration. The controller generates first and second configuration registries for first and second SD-WAN devices running first and second versions of the SD-WAN software, respectively, in an intermediate configuration language registry format. The controller transmits the first and second configuration registries to the first and second SD-WAN devices, respectively, and the first and second SD-WAN devices use the first and second configuration registries to configure themselves. The first and second SD-WAN devices negotiate with each other about common SD-WAN features that can be used to implement the configurations specified in the first and second configuration registries.

Description

[Technical field]

[0001] Priority claim This application claims the benefit of U.S. Patent Application Serial No. 17 / 002,762, filed August 25, 2020, and U.S. Provisional Patent Application Serial No. 63 / 009,235, filed April 13, 2020, the disclosures of each of which are incorporated herein by reference in their entirety.

[0002] Technical Field The subject matter described herein relates to software defined wide area networks (SD-WANs). More particularly, the subject matter described herein relates to managing multiple SD-WAN software versions. [Background technology]

[0003] background In SD-WAN, network nodes, referred to as SD-WAN devices, communicate with each other over a wide area network through tunnels, referred to herein as conduits. In one exemplary network architecture, a central controller defines a single configuration for the SD-WAN and ensures that the individual elements of the SD-WAN are properly configured to realize the overall configuration. The controller also manages software updates for the SD-WAN. Achieving SD-WAN functionality requires intelligent collaboration between SD-WAN network elements. Such intelligent collaboration requires that SD-WAN network elements run compatible SD-WAN software versions. SD-WAN software is software that runs on SD-WAN devices to realize SD-WAN functionality.

[0004] In some SD-WAN network architectures, all SD-WAN devices run the same version of SD-WAN software, and when a new SD-WAN software version becomes available, the controller forces all of the SD-WAN devices to update to the new release at the same time. Forcing all SD-WAN devices to run the same software version and update to new software versions at the same time ensures compatibility between the SD-WAN devices. However, forcing all SD-WAN devices to update to new versions of SD-WAN software at the same time results in network unavailability during the update process. Because SD-WANs can span all time zones, there is no convenient time for a network-wide SD-WAN software update that would not result in a daytime service outage for some users.

[0005] In some SD-WAN architectures, SD-WAN network elements in the same network are permitted to run different software versions. However, these different versions are typically patch releases with minor bug fixes and substantially no new features. Allowing only different software versions that are bug fixes with substantially no new features ensures compatibility between SD-WAN network elements implementing these different software versions, but slows the introduction of new features. In such architectures, updating the SD-WAN to a major release with new features still requires all SD-WAN network elements to be updated at the same time.

[0006] In the SD-WAN and computer networking industry, network protocol compatibility is often provided by defining a set of features to be built upon and using only that set of features. For example, Internet protocol version 4 (IPv4) has a set of features, such as header field formats, that nodes must adhere to to be able to communicate with each other. As long as all nodes agree on the set of features defined by IPv4, communication is possible. If it is desirable to change the size of a header field, such as the IPv4 address, then the network nodes must upgrade to a new protocol, such as IP version 6 (IPv6), which is not compatible with IPv4.

[0007] It is desirable to allow different versions of SD-WAN software to exist simultaneously on different SDWAN nodes, where different SD-WAN versions include more than just bug fixes; it may include at least some features that are incompatible. Another problem associated with allowing different software versions to exist on SD-WAN equipment is that the configuration of the equipment controlled by the controller is tied to the software level being run by the equipment. Requiring the controller to know how to create configurations for each software level realized by each network element is undesirable, especially when the SD-WAN equipment may be running newer software than the controller. Also, when all SD-WAN equipment runs the same software version, it is simpler to configure the SD-WAN equipment because the controller is only required to create configurations that are tied to a single software version. However, in networks where SD-WAN equipment runs different software versions, a mechanism is needed to reduce controller complexity in creating configurations for the SD-WAN equipment.

[0008] In light of these and other challenges, therefore, a need exists for improved methods, systems, and computer-readable media for managing different SD-WAN software versions. Summary of the Invention

[0009] overview The subject matter described herein provides a methodology for an SD-WAN controller to precisely control the software revisions of each network element in an SD-WAN and ensure compatibility between all elements of the SD-WAN. The SD-WAN controller can maintain a single configuration that describes the SD-WAN and knows which features of the SD-WAN are supported by which software versions. The SD-WAN controller can present a user with configuration options that are supported by all elements of the SD-WAN and can prevent enabling features that cannot be used when some network elements are running software versions that do not support a feature.

[0010] The low-level configuration options (herein referred to as the "registry") of each network element may vary depending on the software level. Also, as noted above, forcing the SD-WAN controller to know how to create a registry for each network element is undesirable, especially when the network elements may run newer software than the controller. To avoid this challenge, the subject matter described herein defines an intermediate configuration language registry, implemented in the described example using extensible markup language (XML), that defines the configuration of a particular network element without detailing the low-level details. The SD-WAN controller creates and distributes the intermediate configuration language configuration registry to the SD-WAN network elements. The SD-WAN network elements use the intermediate language configuration registry to configure the SD-WAN network elements. SD-WAN network elements implementing different software versions then negotiate with each other using a versioning protocol to implement compatible SD-WAN features, such as communication protocol message formats and shared memory parameters.

[0011] Using the subject matter described herein, an SD-WAN controller can generate a set of intermediate configuration language registries that can act on any software revision in the SD-WAN network, allowing the SD-WAN controller to provide the option to upgrade the software of any element in the network without restriction, even if the software upgrade adds significant new functionality, makes changes to the network protocol, and only a portion of the network elements are upgradeable.

[0012] When SD-WAN network elements form connections to each other, referred to herein as conduits, they use a versioning protocol to negotiate the details of the conduit communication protocol to be used. By negotiating a set of compatible features using the versioning protocol, it is always ensured that newer software can run the conduit communication protocol in a mode compatible with older software versions, and the SD-WAN Controller ensures that features that require a newer mode of the conduit communication protocol are not configured.

[0013] Unlike other industry solutions, the subject matter described herein works just as well when downgrading software, which is useful when you encounter a bug in the new software and need to revert the software.

[0014] According to one aspect of the subject matter described herein, a method for managing multiple SD-WAN software versions is provided. The method includes providing a controller that defines an overall configuration for an SD-WAN and creates configuration registries for individual SD-WAN devices to implement their respective portions of the overall configuration. The method includes generating, at the controller and in an intermediate configuration language registry format, a first configuration registry for a first SD-WAN device that runs a first version of the SD-WAN software. The method further includes generating, at the controller and in an intermediate configuration language registry format, a second configuration registry for a second SD-WAN device that runs a second version of the SD-WAN software that is different from the first version of the SD-WAN software. The method further includes transmitting the first configuration registry to the first SD-WAN device and transmitting the second configuration registry to the second SD-WAN device. The method further includes receiving the first configuration registry at the first SD-WAN device and using the first configuration registry to configure the first SD-WAN device. The method further includes receiving a second configuration registry at the second SD-WAN device and using the second configuration registry to configure the second SD-WAN device. The method further includes negotiating between the first SD-WAN device and the second SD-WAN device common SD-WAN capabilities that can be used to achieve the configurations specified in the first and second configuration registries.

[0015] According to another aspect of the subject matter described herein, generating the first and second configuration registries in the intermediate configuration language includes generating the first and second configuration registries in a format in which attribute names are assigned to SD-WAN components.

[0016] According to another aspect of the subject matter described herein, using the first and second configuration registries to configure the first and second SD-WAN devices includes identifying names of attributes in the first and second configuration registries that correspond to configuration elements used by the first and second SD-WAN devices to provide the SD-WAN service, reading values ​​of the attributes, and storing the values ​​in records in a configuration database maintained by the first and second SD-WAN devices.

[0017] According to another aspect of the subject matter described herein, generating the first and second configuration registries includes receiving as input a highest intermediate configuration language registry version supported by the first and second SD-WAN devices, and generating the first and second configuration registries in the highest intermediate configuration language registry version supported.

[0018] According to another aspect of the subject matter described herein, negotiating between the first SD-WAN device and the second SD-WAN device includes negotiating using a versioning protocol implemented by the first and second SD-WAN devices.

[0019] According to another aspect of the subject matter described herein, the step of negotiating using a versioning protocol includes the steps of transmitting a versioning protocol check message from the first SD-WAN device to the second SD-WAN device, the versioning protocol check message including a record indicating a highest version of the protocol supported by the first SD-WAN device, and the step of negotiating using a versioning protocol further includes the steps of receiving, at the second SD-WAN device, the versioning protocol check message and determining whether the second SD-WAN device supports the version of the protocol identified in the record in the versioning protocol check message; In response to a determination that the second SD-WAN device supports the version of the protocol identified in the record in the versioning protocol check message, sending a versioning protocol OK message indicating support for the version of the protocol indicated in the versioning protocol check message; and in response to a determination that the second SD-WAN device does not support the version of the protocol identified in the record in the versioning protocol check message, sending a versioning protocol NOKay message indicating a highest version of the protocol supported by the second SD-WAN device that is lower than the version of the protocol identified in the record in the versioning protocol check message.

[0020] According to another aspect of the subject matter described herein, the protocols negotiated using the versioning protocol include a protocol defining a format for conduit messages for carrying traffic between the first SD-WAN device and the second SD-WAN device through a conduit between the first SD-WAN device and the second SD-WAN device.

[0021] According to another aspect of the subject matter described herein, the protocol negotiated using the versioning protocol includes an encryption negotiation method for negotiating encryption parameters to be used to encrypt traffic transmitted through a conduit between a first SD-WAN device and a second SD-WAN device.

[0022] According to another aspect of the subject matter described herein, the protocols negotiated using the versioning protocol include a protocol defining a format for control messages sent between the first SD-WAN device and the second SD-WAN device through a conduit between the first SD-WAN device and the second SD-WAN device.

[0023] According to another aspect of the subject matter described herein, the control messages include shared memory management messages.

[0024] According to another aspect of the subject matter described herein, a system for managing multiple software-defined wide area network (SD-WAN) software versions is provided. The system includes a controller for defining an overall configuration for the SD-WAN and creating configuration registries for individual SD-WAN devices to implement their respective portions of the overall configuration, the controller generating first and second configuration registries in an intermediate configuration language and transmitting the first and second configuration registries. The system further includes a first SD-WAN device, the first SD-WAN device running a first version of the SD-WAN software and for receiving the first configuration registry and using the first configuration registry to configure the first SD-WAN device. The system further includes a second SD-WAN device, the second SD-WAN device running a second version of the SD-WAN software different from the first version of the SD-WAN software and for receiving the second configuration registry and using the second configuration registry to configure the second SD-WAN device. The first and second SD-WAN devices are configured to negotiate with each other for common SD-WAN capabilities that are usable to achieve configurations specified in the first and second configuration registries.

[0025] According to another aspect of the subject matter described herein, the first and second SD-WAN devices are configured to identify names of attributes in the first and second configuration registries that correspond to configuration elements used by the first and second SD-WAN devices to provide an SD-WAN service, read values ​​of the attributes, and store the values ​​in records in configuration databases maintained by the first and second SD-WAN devices.

[0026] According to another aspect of the subject matter described herein, the controller is configured to receive as input a highest intermediate configuration language registry version supported by the first and second SD-WAN devices and generate the first and second configuration registries in the highest intermediate configuration language registry version supported.

[0027] According to another aspect of the subject matter described herein, the first and second SD-WAN devices are configured to negotiate with each other for compatible SD-WAN capabilities using a versioning protocol implemented by the first and second SD-WAN devices.

[0028] According to another aspect of the subject matter described herein, a non-transitory computer-readable medium is provided having executable instructions stored thereon that, when executed by a processor of the computer, control the computer to perform a plurality of steps. The plurality of steps includes providing a controller that defines an overall configuration for an SD-WAN and creates configuration registries for individual SD-WAN devices to implement their respective portions of the overall configuration. The plurality of steps further includes generating, at the controller and in an intermediate configuration language registry format, a first configuration registry for a first SD-WAN device that runs a first version of the SD-WAN software. The plurality of steps further includes generating, at the controller and in an intermediate configuration language registry format, a second configuration registry for a second SD-WAN device that runs a second version of the SD-WAN software that is different from the first version of the SD-WAN software. The plurality of steps further includes transmitting the first configuration registry to the first SD-WAN device and transmitting the second configuration registry to the second SD-WAN device. The plurality of steps further includes receiving the first configuration registry at the first SD-WAN device and using the first configuration registry to configure the first SD-WAN device. The steps further include receiving a second configuration registry at the second SD-WAN device and using the second configuration registry to configure the second SD-WAN device. The steps further include negotiating between the first SD-WAN device and the second SD-WAN device common SD-WAN features that are usable to achieve the configurations specified in the first and second configuration registries.

[0029] The subject matter described herein may be implemented in hardware, software, firmware, or any combination thereof. Thus, the terms "function," "node," or "module" as used herein refer to hardware for implementing the described functions, which may also include software and / or firmware components. In one exemplary implementation, the subject matter described herein may be implemented using a computer-readable medium having executable instructions stored thereon that, when executed by a processor of a computer, control a computer to perform a number of steps. Exemplary computer-readable media suitable for implementing the subject matter described herein include non-transitory computer-readable media, such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits. In addition, computer-readable media implementing the subject matter described herein may be located on a single device or computing platform, or may be distributed among multiple devices or computing platforms. [Brief description of the drawings]

[0030] [Figure 1] FIG. 1 is a network diagram illustrating an example SD-WAN network architecture. [Diagram 2] FIG. 1 is a network diagram illustrating the distribution of a configuration registry in an intermediate configuration language format to SD-WAN network elements running different software versions. [Diagram 3] FIG. 13 is a message flow diagram illustrating capability negotiation between SD-WAN network elements where an SD-WAN that implements a lower version of the protocol under negotiation initiates a version check for the protocol. [Figure 4]FIG. 13 is a message flow diagram illustrating capability negotiation between SD-WAN network elements where an SD-WAN network element that supports a higher version of the protocol being negotiated initiates a version check for the protocol. [Diagram 5] FIG. 13 is a message flow diagram illustrating capability negotiation between SD-WAN network elements where the SD-WAN network elements each support multiple versions of the protocol being negotiated. [Figure 6] FIG. 13 is a message flow diagram illustrating capability negotiation between SD-WAN network elements when the SD-WAN network elements each support multiple versions of the protocol being negotiated and the SD-WAN element initiating the version check uses an intermediate version that is between two versions but is not supported by the other SD-WAN network element. [Figure 7] 1 is a flowchart illustrating exemplary steps for managing multiple SD-WAN software versions. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Detailed Description The subject matter described herein includes methods, systems, and computer-readable media for managing SD-WAN network elements running different versions of SD-WAN software. As described above, an SD-WAN controller generates an intermediate configuration language registry and distributes the registry to SD-WAN network elements, at least some of which run different versions of SD-WAN software. The network elements receive the intermediate configuration language registry and use the intermediate configuration language registry to configure the SD-WAN network elements. The SD-WAN network elements running different software versions then negotiate with each other to identify a compatible set of SD-WAN features. Before describing the details of the intermediate configuration language and its use in SD-WAN, an example SD-WAN architecture will be described.

[0032] In the exemplary architecture, the SD-WAN is called an adaptive private network (APN). The SD-WAN network elements are called APN appliances (APNAs), and the APNAs are connected to each other through tunnels called conduits. FIG. 1 shows an example of such an APN architecture. In FIG. 1, the architecture includes multiple APNAs 100, 102, 104, 106, 108, 110, and 112 that are connected to each other through conduits 116, 118, 120, 122, 124, and 126. A conduit is a connection between APN appliances. A conduit may be used to carry traffic between APNAs through the WAN 128. A controller 130 controls the overall configuration and operation of the APN by distributing the configuration to the APNAs 100, 102, 104, 106, 108, 110, and 112. If the APNAs 100, 102, 104, 106, 108, 110, and 112 are running different APN software versions, the controller 130 must distribute configurations that prevent incompatible features from being implemented by the APNAs 100, 102, 104, 106, 108, 110, and 112. As described above, the controller 130 handles such configurations by distributing an intermediate configuration language registry to the APNAs 100, 102, 104, 106, 108, 110, and 112 via one or more conduits 132. The APNAs 100, 102, 104, 106, 108, 110, and 112 use the intermediate configuration language registry to configure the APNAs 100, 102, 104, 106, 108, 110, and 112 to implement SD-WAN features. The APNAs 100, 102, 104, 106, 108, 110, and 112 negotiate with each other to achieve compatible APN functionality.

[0033] FIG. 2 is a diagram illustrating the controller 130 generating and sending configuration registries to APNAs 106 and 108 running different versions of APN software. In FIG. 2, the controller 130 includes at least one processor 200 and memory 202. The controller 130 also includes a configuration manager 203 that generates and distributes configuration registries for configuring the APNAs 106 and 108. The configuration registries are in an intermediate configuration language registry format, which is described in more detail below. The configuration manager 203 includes a configuration compiler 204 that generates a configuration registry based on configuration options selected by a user via a configuration editor 205. The configuration editor 205 presents the user with options to configure the APN device based on the APN software version being run by the APN device so that only compatible configurations are generated and sent to the APN device. The configuration editor 205 will know the highest common registry version supported by the devices in the network and apply it when running the compiler to present editable configuration objects to the user.

[0034] In one prior implementation of the controller 130, the compiler 204 generated a set of registry files containing numbers representing configuration options and their values ​​that were read by the APN software in each APN device and used to populate the configuration database. Below is an example of a portion of a configuration file that one prior implementation of the compiler 204 might have received as input:

[0035]

number

[0036] The example configurations listed above configure a device to function as a master controller of a particular model using a particular security key. A previous implementation of the compiler 204 converted the above example configurations into the following comma separated value (CSV) configuration registry, which would have been communicated to the device. Below is one of the CSV configuration files, called main.reg:

[0037]

number

[0038] Below is another CSV file that was generated for the example configuration above, this one called sites.reg.

[0039]

number

[0040] To read these registries, the APN software running on each APN device must know what the first number on each line means. For example, 724 comes from the following #define statement in the APN software code:

[0041]

number

[0042] The 724 ID number is specific to the version of the APN software and compiler version being run by the APN device. For example, if this ID number was generated using compiler version 7.3, it would not be readable by APN software version 8.0 because the ID number values ​​for configuration settings are not guaranteed to be the same between different APN software versions.

[0043] To allow compiler 204 to generate registries for different software versions, as described above, compiler 204 will receive as input the highest registry version supported by all APN devices in the network and generate an intermediate configuration language registry for the APN devices in the highest supported intermediate configuration language registry format. Using the example in the previous paragraph, if one APN device runs APN software version 7.3 and supports intermediate configuration language registry format 7.3, and another APN device runs APN software version 8.0 and supports intermediate configuration language registry formats 7.3 and 8.0, compiler 204 will receive intermediate configuration language registry version 7.3 as input and generate an intermediate configuration language registry according to intermediate configuration language registry format 7.3. In addition, rather than associating each configuration item in the intermediate configuration language registry format with an ID number that may change between software releases as described in more detail below, the configuration items are identified by a name, which will remain the same for configuration items used between APN software releases. If a configuration item is added by a software release, the item will receive a new name. APN devices running different software versions may have different capabilities and may negotiate with each other for a common set of supported APN features.

[0044] 2, APNAs 106 and 108 each include at least one processor 206 and memory 208. APNAs 106 and 108 also include a configuration generator 210 for using a configuration registry in an intermediate configuration language received from controller 130 to configure APNAs 106 and 108. In one example, configuration generator 210 may generate a script that populates configuration database 211 with configuration data read from fields in the configuration registry. In the exemplary implementation described herein, the language used to create the configuration registry received from controller 130 is XML, and configuration generator 210 reads the names and values ​​of XML attributes and uses these values ​​to populate corresponding records in configuration database 211. Once the configuration data is populated in configuration database 211, APN software 212 reads the configuration in database 211 and creates a corresponding system configuration in memory 208. The APN software 212 uses the system configuration in memory 208 to implement APN functions such as building and maintaining conduits, sending traffic through the conduits, and timing synchronization.

[0045] Each of the APNAs 106 and 108 also includes a capability negotiator 214 for negotiating compatible capabilities with other APNAs in the network. The capability negotiator 214 may utilize a versioning protocol to negotiate compatible capabilities. An example use of the versioning protocol will be described below.

[0046] As mentioned above, the subject matter described herein enables the APNAs to run different versions of the APN software. In the illustrated example, APNA 106 runs version 1.0 of the APN software 212 and APNA 108 runs version 1.1 of the APN software 212.

[0047] The intermediate configuration language does not require the controller 130 to know the underlying details of each APNA. The version of the intermediate configuration language used to create the registry will be the highest version supported by all APNAs in the network. In one example, the version number set for the registry is the same as the lowest APN software version supported by the network. In the illustrated example, APNA 106 uses APN software version 1.0 and APNA 108 uses APN software version 1.1. Since APN software version 1.0 is the lowest software version supported by the nodes in the network, the configuration registry will be created using version 1.0 of the intermediate configuration language.

[0048] Below is an exemplary specification of the intermediate configuration language used to create the intermediate configuration registry.

[0049]

number

[0050] Table 1 below provides definitions of the fields in the exemplary configuration registry discussed above.

[0051] [Table 1] TIFF0007610616000007.tif139162

[0052] In the intermediate configuration language registry format defined above, SD-WAN components are assigned names rather than ID numbers that change between software releases. For example, a component for a WAN link may be assigned a field name of WAN_Link and a component for a conduit may be assigned a field name of Conduit. The field names correspond to the components that the APN software uses to configure the APN equipment to achieve the corresponding functionality. The APN software running on each device is only required to know how to read the attributes and values ​​in the intermediate configuration language registry format. There is no longer a requirement that the APN software be able to read a registry file consisting of comma-separated values ​​with ID numbers corresponding to the components.

[0053] As described above, once each APN device receives its configuration registry from controller 130, an automatically generated script reads the configuration registry XML file, inserts the configuration into configuration database 211, and notifies APN software 130 that the configuration data has been written to configuration database 211. Reading the configuration data from the XML file may include reading values ​​for the XML attributes identified above and writing the values ​​of the attributes to corresponding records in configuration database 211.

[0054] For the APN software startup case, APN software 212 will wait until this notification is received and proceed to read from configuration database 211. For the configuration update case, the automatically generated script will also inform APN software 212 of the type of registry change so that APN software 212 knows how to read those new changes from configuration database 211 and the type of configuration update that needs to be made. For configuration updates, because APN software 212 and the script that populates values ​​from the configuration registry are both writing to configuration database 211, these writes must be coordinated so that the APN software 212 and the script do not corrupt each other's data.

[0055] As described above, when the APN devices receive their configuration from the controller 130, the APN devices negotiate with each other for compatible APN capabilities using a versioning protocol, which is referred to herein as TVP, which stands for Talari Versioning Protocol.

[0056] To support multiple SD-WAN or APN software versions, a new TVP record type is added to negotiate a different APN message version to use. If an unknown TVP record type is received, it will be ignored and the receiving APN will use the default APN message version. The current version of TVP uses type, length, value (TLV) for each record, making it easy to add new records and to ignore unknown records.

[0057] One type of message that APN nodes may use TVP to negotiate compatible message formats is the messages used to build and maintain conduits between APN devices located at different sites. These conduits are tunnels that carry encrypted traffic between sites and may span multiple WAN links. Conduit traffic is encapsulated in an envelope with a header that contains a sequence number and a timestamp. This envelope allows packet loss, latency, and jitter to be measured so that traffic is directed on the best path through the network and allows mitigation of network impairments. The protocol that defines the message format for conduit messages is called TRP, which stands for Talari Reliable Protocol.

[0058] The TRP version record can be used as an example of an extension to the TVP that is needed to support multiple software versions. For the first multi-version release of APN (SD-WAN) software, there will be no TRP version record required. For future releases, new records will be created if the TRP header needs to change. The following scenarios show different use cases where the TVP is used to negotiate the TRP version between different APN sites.

[0059] Scenario 1 and Figure 3 show the use of TVP to negotiate a TRP record format, where the TRP version check is initiated by site 1, which only knows a single TRP version: Scenario 1: Site 1 runs APN software version 8.3.0.0.0 and has TRP version 8.3.0.0.0. Site 2 runs APN software version 8.4.0.0.0 and supports TRP versions 8.3.0.0.0 and 8.4.0.0.0. In the first line of the message flow diagram in Figure 2, Site 1 sends a TVP check to Site 2. Site 1 knows only a single TRP version, so the TVP check in line 1 does not contain a TRP record. When Site 2 receives the TVP check, it knows that Site 1 can only use TRP 8.3.0.0.0, so Site 2 uses TRP version 8.3.0.0.0 to communicate with Site 1. The TVP OK in line 2 does not contain a TRP version record.

[0060] Scenario 2 and Figure 4 show the use of TVP to negotiate the TRP version, where Site 2, which supports multiple TRP versions, initiates the TVP version check: Scenario 2: In scenario 2, site 1 and site 2 have the same APN software version and TRP version as in scenario 1. However, in scenario 2, site 2 sends a TVP check message to site 1 first. Because site 2 supports multiple TRP versions, when site 2 sends a TVP check first, site 2 will include its highest supported TRP version record, 8.4.0.0.0, in the TVP check message. When site 1 receives the TVP check message, site 2 will ignore the unknown TVP record. Therefore, site 2 will send a TVP OK message to site 1 without a TRP version record. Upon receiving the TVP OK message without a TRP version record, site 2 will determine that it needs to use TRP version 8.3.0.0.0 to communicate with site 1.

[0061] Scenario 3 and Figure 5 show the use of TVP to negotiate the TRP version when one site supports two TRP versions and the other site supports three TRP versions: Scenario 3: Site 1 runs APN software version 8.4.0.0.0 and supports TRP versions 8.3.0.0.0 and 8.4.0.0.0. Site 2 runs APN software version 8.5.0.0.0 and supports TRP versions 8.3.0.0.0, 8.4.0.0.0, and 8.5.0.0.0. In Figure 5, Site 2 sends a first TVP check message to Site 1. The TVP check message contains a TRP version record 8.5.0.0.0. Because Site 1 does not support TRP version 8.5.0.0.0, Site 1 replies with a TVP NOKay message with TRP version 8.4.0.0.0 (the next higher TRP version supported by Site 1). Because Site 2 can support TRP version 8.4.0.0.0, Site 2 sends another TVP check message using TRP version 8.4.0.0.0 and receives a TVP OK from Site 1. After the message flow in Figure 5, Site 1 and Site 2 can exchange tunnel management messages using TRP version 8.4.0.0.0.

[0062] Scenario 4 and Figure 6 show the use of TVP to negotiate the TRP version when one site supports two TRP versions and the other site supports three TRP versions: Scenario 4: Site 1 runs APN software version 8.4.0.3.0 and supports TRP versions 8.3.0.0.0, 8.4.0.0.0, and 8.4.0.3.0. Site 2 runs APN software 8.5.0.0.0 and supports TRP versions 8.3.0.0.0, 8.4.0.0.0, and 8.5.0.0.0. In the example shown in Figure 6, Site 1 first sends a TVP check message to Site 2. The TVP check message contains a TRP version record of 8.4.0.3.0. Site 2 does not support TRP version 8.4.3.0, so Site 2 replies with a TVP no-know with TRP version 8.4.0.0.0. TRP version 8.4.0.0.0 is the highest TRP version that Site 2 supports, which is lower than the TRP version received from Site 1. Because Site 1 can support TRP version 8.4.0.0.0, Site 1 sends another TVP check message with TRP version 8.4.0.0.0 and receives a TVP OK from Site 2. After the message flow shown in Figure 6, Site 1 and Site 2 can use TRP version 8.4.0.0.0 to manage and communicate through the tunnel between Site 1 and Site 2.

[0063] In Figures 3-6, TVP is used to negotiate the version of the tunneling protocol message between APN devices located at different sites. In another example, TVP may be used to negotiate the tunnel encryption negotiation method to be used by APN devices located at different sites. To use TVP to perform such negotiation, a new TVP record for the encryption negotiation method is added to the TVP protocol. Before performing encryption negotiation, APN devices located at different sites may utilize a default encryption negotiation method obtained from a configuration file received from the controller. The negotiation of the encryption negotiation method may be performed in the same manner as the negotiation of the TRP version in Figures 3-6. That is, one site will send a TVP check message to another site with an encryption negotiation record that specifies the highest version of the encryption negotiation method supported by the sending site. If the receiving site supports the encryption negotiation method, it will respond with a TVP OK message containing an encryption negotiation record indicating support for the encryption negotiation method version supported by the sending site. If the receiving site does not support the encryption negotiation method, it will respond with the highest encryption negotiation method version supported by the receiving site, which is lower than the encryption negotiation method received in the TVP Check message from the sending site. Once the two sites have agreed on an encryption negotiation method, the encryption negotiation method can be used to negotiate tunnel encryption parameters between the two sites.

[0064] Once a conduit is established between two sites, the two sites exchange control messages through the tunnel. Examples of such control messages include heartbeat messages, nag messages, SACK / SNAK messages, time synchronization messages, network resume messages, and shared memory management messages. Heartbeat messages are messages sent through the conduit to maintain and determine the state of the conduit. Time synchronization messages are used to synchronize the clocks of APN devices with a central clock maintained by the controller. Selective acknowledgment messages (SACK) are used to acknowledge receipt of specific data. Selective negative acknowledgement messages (SNAK) are used to convey data that was not received. Nag messages may also be used to indicate data that was not received. Similar to the TRP version and encryption negotiation method, versions for these types of control messages may be negotiated using the TVP protocol using the message flows shown in Figures 3-6, except that the TVP protocol carries records indicating the control message types and control message versions supported by the sending site.

[0065] Using shared memory management messages as an example, APNs located at different sites may communicate with each other using shared memory. To communicate using shared memory, messages are exchanged using a protocol referred to herein as the APN Memory Protocol or APN MEM. The APN MEM protocol has versions that define the features and APN MEMs. New APN MEM types can be added if fields need to be changed in the APN MEM message. The TVP negotiation procedure shown in Figures 3-6 may be used to negotiate an APN MEM type to use between the two ends of the conduit. The APN MEM will use APN version specific pack / unpack functions to convert between shared memory information and APN MEM messages. The affected APN MEM messages will provide a callback function to unregister the old APN MEM type and register the new APN MEM type. This function will be called if the TVP negotiates for a new APN MEM message type.

[0066] In addition to the control protocol versions and message types mentioned above, the following message types and corresponding protocol features may be negotiated using the message flows shown in Figures 3-6, with appropriate records added to the TVP protocol to identify the type of protocol being negotiated and the current version of the protocol supported by the APNA at each end of the conduit: · RACP (redundant appliance configuration protocol) is a high availability (HA) protocol used when two appliances serve a site in a redundant configuration. The two appliances communicate their status and negotiate which appliance should be active and represent the site; · TAP (Talari Application Protocol): The TAP protocol is used to carry user data between sites through conduits; Dynamic Conduits / Cloud Connect: These messages are used to trigger the creation of conduits dynamically based on traffic flow; · Bandwidth Testing: Messages used to test conduit bandwidth; · Path MTU: Path MTU is the maximum message transmission unit size supported on the path; · Traceroute: Traceroute is a route discovery protocol used to discover hops between a traffic source and a destination; · Header Compression: defines the header compression method used in conduit messages; Dynamic Route Sharing: Dynamic Route Sharing messages allow APNAs to exchange routing information that is dynamically learned from networks, for example, via Open Shortest Path First (OSPF) or Border Gateway Protocol (BGP).

[0067] 7 is a flow chart illustrating an example process for managing multiple software versions in an SD-WAN. Referring to FIG. 7, at step 700, the process includes providing a controller that defines an overall configuration for the SD-WAN and creates configuration registries for individual SD-WAN devices to implement their respective portions of the overall configuration. For example, the controller 130 may present a user with valid APN configuration options given the software versions being run by the APNAs in the network. The controller 130 may generate a configuration registry for each APNA in an intermediate configuration language format from the configuration selected by the user.

[0068] At step 702, the process further includes generating, at the controller and in an intermediate configuration language registry format, a first configuration registry for a first SD-WAN device running a first version of the SD-WAN software. As described above, in one example, the intermediate configuration language is XML, where components are assigned attribute names and the attribute names are given corresponding values. The attribute names are common across software releases so that a compiler that generates the intermediate configuration language configuration registry is not required to generate a registry in a format specific to the low-level details of each device. The compiler is provided as input an intermediate configuration language registry version that is the highest version supported by all APN devices in the network, and the compiler generates an intermediate configuration language registry in that version for each of the SD-WAN devices being configured. Note that the configurations provided to different devices will be different due to, for example, different network connections of each device. However, each device will be able to read the intermediate configuration language registry it receives from the controller.

[0069] At step 704, the process further includes generating, at the controller and in the intermediate configuration language registry format, a second configuration registry for a second SD-WAN device running a second version of the SD-WAN software that differs from the first version of the SD-WAN software. Even though the first and second SD-WAN devices run different software versions, they will each receive a configuration registry in the highest version of the intermediate configuration language that they both understand. Such a system assumes that a higher version of the APN software can read a lower version of the intermediate configuration language, which is facilitated by the fact that the intermediate configuration language uses names for configuration objects rather than ID numbers that change between software releases.

[0070] The process further includes sending the first configuration registry to the first SD-WAN device and the second configuration registry to the second SD-WAN device at step 706. For example, the controller 130 may send the configuration registry to the SD-WAN device being configured.

[0071] At step 708, the process further includes receiving a first configuration registry at the first SD-WAN device and using the first configuration registry to configure the first SD-WAN device. At step 710, the process further includes receiving a second configuration registry at the second SD-WAN device and using the second configuration registry to configure the second SD-WAN device. Using the configuration registry to configure the first and second SD-WAN devices may include identifying names of XML attributes in the first and second configuration registries that correspond to configuration elements used by the first and second SD-WAN devices to provide SD-WAN services, reading values ​​of the XML attributes, and storing the values ​​in records in the configuration database 211. The SD-WAN software running on the first and second SD-WAN devices writes system configurations to memory 208 using the configurations stored in their respective configuration databases 211. The SD-WAN Software uses system configurations to enable SD-WAN functions, such as establishing and maintaining conduits, sending traffic through those conduits, performing timing synchronization, and negotiating SD-WAN features with other SD-WAN devices.

[0072] At step 712, the process further includes negotiating between the first SD-WAN device and the second SD-WAN device common capabilities usable to implement the configurations identified in the first and second configuration registries. For example, the SD-WAN devices may negotiate SD-WAN capabilities such as conduit message formats, encryption negotiation methods, control message protocol versions, etc. The negotiation may be performed using the example messaging and versioning protocols shown in FIGS. 3-6.

[0073] It will be understood that various details of the presently disclosed subject matter can be changed without departing from the scope of the presently disclosed subject matter, and the foregoing description is merely illustrative and not limiting.

Claims

1. 1. A method for managing multiple software-defined wide area network (SD-WAN) software versions, the method comprising: providing a controller that defines an overall configuration for the SD-WAN and creates configuration registries for individual SD-WAN devices that implement their respective parts of the overall configuration; generating, at the controller and in an intermediate configuration language registry format, a first configuration registry for a first SD-WAN device running a first version of SD-WAN software; generating, at the controller and in the intermediate configuration language registry format, a second configuration registry for a second SD-WAN device running a second version of the SD-WAN software that is different from the first version of the SD-WAN software; sending the first configuration registry to the first SD-WAN device and sending the second configuration registry to the second SD-WAN device; receiving the first configuration registry at the first SD-WAN device and using the first configuration registry to configure the first SD-WAN device; receiving the second configuration registry at the second SD-WAN device and using the second configuration registry to configure the second SD-WAN device; and negotiating between the first SD-WAN device and the second SD-WAN device common SD-WAN capabilities that can be used to implement configurations specified in the first and second configuration registries.

2. 2. The method of claim 1, wherein generating the first and second configuration registries in an intermediate configuration language includes generating the first and second configuration registries in a format in which attribute names are assigned to SD-WAN components.

3. 3. The method of claim 2, wherein using the first and second configuration registries to configure the first and second SD-WAN devices includes identifying names of attributes in the first and second configuration registries that correspond to components used by the first and second SD-WAN devices to provide SD-WAN services, reading values ​​of the attributes, and storing the values ​​in records in a configuration database maintained by the first and second SD-WAN devices.

4. The method of any one of claims 1 to 3, wherein the step of generating the first and second configuration registries includes the steps of receiving as input a highest intermediate configuration language registry version supported by the first and second SD-WAN devices, and generating the first and second configuration registries in the highest intermediate configuration language registry version supported.

5. The method of any one of claims 1 to 4, wherein the step of negotiating between the first SD-WAN device and the second SD-WAN device includes negotiating using a versioning protocol implemented by the first and second SD-WAN devices.

6. The step of negotiating using a versioning protocol includes: The step of negotiating using a versioning protocol further comprises: transmitting a versioning protocol check message from the first SD-WAN device to the second SD-WAN device, the versioning protocol check message including a record indicating a highest version of a protocol supported by the first SD-WAN device; and receiving, at the second SD-WAN device, the versioning protocol check message and determining whether the second SD-WAN device supports the version of the protocol identified in the record in the versioning protocol check message; transmitting, at the second SD-WAN device, in response to determining that the second SD-WAN device supports the version of the protocol identified in the record in the versioning protocol check message, a versioning protocol OK message indicating support for the version of the protocol indicated in the versioning protocol check message; and transmitting, at the second SD-WAN device, a versioning protocol no-query message indicating a highest version of the protocol supported by the second SD-WAN device that is lower than the version of the protocol identified in the record in the versioning protocol check message, in response to determining that the second SD-WAN device does not support the version of the protocol identified in the record in the versioning protocol check message.

7. 7. The method of claim 6, wherein the protocols negotiated using the versioning protocol include a protocol defining a format for conduit messages for carrying traffic between the first SD-WAN device and the second SD-WAN device through a conduit between the first SD-WAN device and the second SD-WAN device.

8. 7. The method of claim 6, wherein the protocols negotiated using the versioning protocol include an encryption negotiation method for negotiating encryption parameters to be used to encrypt traffic transmitted through a conduit between the first SD-WAN device and the second SD-WAN device.

9. 7. The method of claim 6, wherein the protocols negotiated using the versioning protocol include a protocol defining a format for control messages transmitted between the first SD-WAN device and the second SD-WAN device through a conduit between the first SD-WAN device and the second SD-WAN device.

10. The method of claim 9 , wherein the control messages include shared memory management messages.

11. 1. A system for managing multiple software-defined wide area network (SD-WAN) software versions, the system comprising: a controller for defining an overall configuration for the SD-WAN and creating configuration registries for individual SD-WAN devices implementing their respective portions of the overall configuration, the controller generating first and second configuration registries in an intermediate configuration language and transmitting the first and second configuration registries, the system further comprising: a first SD-WAN device, the first SD-WAN device running a first version of SD-WAN software, for receiving the first configuration registry, and for using the first configuration registry to configure the first SD-WAN device, the system further comprising: a second SD-WAN device, the second SD-WAN device running a second version of the SD-WAN software different from the first version of the SD-WAN software, for receiving the second configuration registry, and for using the second configuration registry to configure the second SD-WAN device; The first and second SD-WAN devices are configured to negotiate with each other for common SD-WAN capabilities that can be used to implement configurations specified in the first and second configuration registries.

12. 12. The system of claim 11, wherein the intermediate configuration language registry format includes attribute names assigned to SD-WAN components.

13. 13. The system of claim 12, wherein the first and second SD-WAN devices are configured to identify names of the attributes in the first and second configuration registries that correspond to components used by the first and second SD-WAN devices to provide SD-WAN services, read values ​​of the attributes, and store the values ​​in records in configuration databases maintained by the first and second SD-WAN devices.

14. The system of any one of claims 11 to 13, wherein the controller is configured to receive as input a highest intermediate configuration language registry version supported by the first and second SD-WAN devices, and to generate the first and second configuration registries in the highest intermediate configuration language registry version supported.

15. The system of any one of claims 11 to 14, wherein the first and second SD-WAN devices are configured to negotiate with each other for compatible SD-WAN capabilities using a versioning protocol implemented by the first and second SD-WAN devices.

16. negotiating using the versioning protocol includes: sending a versioning protocol check message from the first SD-WAN device to the second SD-WAN device, the versioning protocol check message including a record indicating a highest version of a protocol supported by the first SD-WAN device; and negotiating using the versioning protocol further includes: receiving, at the second SD-WAN device, the versioning protocol check message and determining whether the second SD-WAN device supports the version of the protocol identified in the record in the versioning protocol check message; sending, at the second SD-WAN device, in response to determining that the second SD-WAN device supports the version of the protocol identified in the record in the versioning protocol check message, a versioning protocol OK message indicating support for the version of the protocol indicated in the versioning protocol check message; and transmitting, at the second SD-WAN device, a versioning protocol no-query message indicating a highest version of the protocol supported by the second SD-WAN device that is lower than the version of the protocol identified in the record in the versioning protocol check message, in response to determining that the second SD-WAN device does not support the version of the protocol identified in the record in the versioning protocol check message.

17. 17. The system of claim 16, wherein the protocols negotiated using the versioning protocol include a protocol defining a format for conduit messages for carrying traffic between the first SD-WAN device and the second SD-WAN device through a conduit between the first SD-WAN device and the second SD-WAN device.

18. 17. The system of claim 16, wherein the protocols negotiated using the versioning protocol include an encryption negotiation method for negotiating encryption parameters to be used to encrypt traffic transmitted through a conduit between the first SD-WAN device and the second SD-WAN device.

19. 17. The system of claim 16, wherein the protocols negotiated using the versioning protocol include a protocol defining a format for control messages transmitted between the first SD-WAN device and the second SD-WAN device through a conduit between the first SD-WAN device and the second SD-WAN device.

20. A program for causing a computer processor to execute the method according to any one of claims 1 to 10.

Images