Create and deploy packages to devices in your fleet based on actions derived from a machine learning model

Optimizing the package deployment plan through machine learning models solves the challenge of deploying firmware packages to hardware devices in multiple data centers around the world, enabling secure, reliable and efficient deployment, monitoring the impact of deployment on the hardware.

CN113711180BActive Publication Date: 2024-11-29MICROSOFT TECHNOLOGY LICENSING LLC
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202080027710.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-03-26
Publication Date
2024-11-29
Estimated Expiration
2040-03-26

AI Technical Summary

Technical Problem

Deploying firmware or other low-level system code packages on hardware devices distributed across multiple data centers around the world is challenging, requiring secure, reliable and reliable deployment, while monitoring the impact of deployment on the hardware.

Method used

Using machine learning models to create and optimize package deployment plans, automatically create deployment plans by receiving payload parameters and cluster parameters, including specifying operations to ensure security and reliability, and monitoring impacts during deployment.

Benefits of technology

Achieving secure, reliable and reliable package deployment on globally distributed hardware devices reduces manual intervention, improves deployment efficiency, and ensures minimized deployment impact by monitoring cluster status.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113711180B_ABST
    Figure CN113711180B_ABST
Patent Text Reader

Abstract

Systems and methods are provided for creating packages based on operations derived from a machine learning model or other automatic feedback model and deploying the packages to devices in a fleet. As an example, a method for creating a package including a payload for deployment to a group of devices is provided. The method includes: receiving a payload, wherein the payload has an associated set of payload parameters related to deploying the payload to the group of devices. The method also includes: using a processor, automatically creating a package for deployment to the group of devices, wherein the package includes instructions for deploying the payload to the group of devices, and wherein the instructions specify at least one of a plurality of operations that are derived from the machine learning model based on at least a subset of the associated set of payload parameters.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] It is difficult to deploy packages including firmware or other low-level system code to components in a fleet of machines containing hardware corresponding to a cloud. Public clouds include a global network of servers that perform various functions, including storing and managing data, running applications, and delivering content or services such as streaming video, email, office productivity software, or social media. Servers and other components may be located in data centers around the world. While public clouds provide services to the public over the Internet, enterprises may use private or hybrid clouds. Private and hybrid clouds also include a network of servers housed in a data center.

[0002] Data centers include not only servers, but also other components such as network switches, routers, and other equipment. Servers and other components may be provided by different vendors and may include different types or versions of motherboards, CPUs, memory, and other devices. In addition to computer, network, and storage components, data centers also include other components such as chassis, racks, power supply units, and other such components.

[0003] Each of these devices may require low-level system code including firmware. It is challenging to deploy packages to a wide variety of devices that may be distributed in many data centers around the world. Therefore, there is a need for methods and systems for deploying packages to devices in a fleet. Summary of the invention

[0004] In one example, the present disclosure relates to a method for creating a package including a payload for deployment to a set of devices. The method may include: receiving a payload, wherein the payload has an associated set of payload parameters related to deploying the payload to the set of devices. The method may also include: using a processor, automatically creating a package for deployment to the set of devices, wherein the package includes instructions for deploying the payload to the set of devices, and wherein the instructions specify at least one of a plurality of operations that are derived from a machine learning model based on at least a subset of the associated set of payload parameters.

[0005] In another example, the present disclosure relates to a method of deploying a package to a cluster. The method may include: evaluating the cluster to determine a set of cluster parameters associated with deploying the package to the cluster. The method may include: using a processor, automatically creating a deployment plan for deploying the package to the cluster, wherein the deployment plan includes instructions for deploying the package to the cluster, and wherein the instructions specify at least one of a plurality of operations, the plurality of operations being derived from a machine learning model based on at least a subset of the set of cluster parameters.

[0006] In yet another example, the present disclosure relates to a system for deploying a package to a cluster. The system may be configured to: evaluate the cluster to determine a set of cluster parameters associated with deploying the package to the cluster. The system may be configured to: automatically create, using a processor, a deployment plan for deploying the package to the cluster, wherein the deployment plan includes instructions for deploying the package to the cluster, and wherein the instructions specify at least one of a plurality of operations, the plurality of operations being derived from a machine learning model based on at least a subset of the set of cluster parameters.

[0007] In yet another example, the present disclosure relates to a method for creating a package including a payload for deployment to a set of devices. The method may include: receiving a payload, wherein the payload has an associated set of payload parameters related to deploying the payload to the set of devices. The method may also include: using a processor, automatically creating a package for deployment to the set of devices, wherein the package includes instructions for deploying the payload to the set of devices, and wherein the instructions specify at least one of a plurality of operations, the plurality of operations being derived from an automatic feedback model based on at least a subset of the associated set of payload parameters.

[0008] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present disclosure is illustrated by way of example and not limitation in the accompanying figures, in which like reference numerals indicate similar elements. Elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale.

[0010] Figure 1 A diagram showing a system environment for deploying packages to devices in a fleet including hardware in a cloud according to one example;

[0011] Figure 2 is a block diagram of a system including deployment and monitoring of a cluster according to an example;

[0012] Figure 3 is a block diagram of a cluster in a data center according to an example;

[0013] Figure 4 A block diagram showing deployment and monitoring according to one example;

[0014] Figure 5 shows a diagram of a memory 500 including modules having instructions for performing operations associated with deployment and monitoring;

[0015] Figure 6 shows components of a package according to one example;

[0016] Figure 7 An example of a segmented deployment according to an example is shown;

[0017] Figure 8 A flowchart showing a method for deploying a package according to one example;

[0018] Fig. 9 A diagram showing a scan tree of hardware in one phase according to one example;

[0019] Fig.10 A diagram showing a scan tree of hardware in another stage according to one example;

[0020] Fig.11A and Fig. 11B A flowchart showing a method for deploying a package according to one example;

[0021] Fig.12 shows an impact table according to an example;

[0022] Fig.13 A machine learning system according to an example is shown;

[0023] Fig.14 According to an example, a method including Fig.13 memory for instructions and data used with machine learning systems;

[0024] Fig.15 A flowchart illustrating a method for creating a package including a payload for deployment to a group of devices according to one example;

[0025] Fig.16 A flowchart showing a method for deploying a package to a cluster according to one example;

[0026] Fig.17 A deployment dashboard according to one example is shown. DETAILED DESCRIPTION

[0027] The examples described in this disclosure involve creating and deploying packages including payloads to a fleet. Specific examples involve creating and deploying packages based on operations derived from machine learning models. It is difficult to deploy packages including firmware or other low-level system code to components including hardware in the cloud. Public clouds include a global network of servers that perform various functions, including storing and managing data, running applications, and delivering content or services such as streaming video, email, office productivity software, or social media. Servers and other components may be located in data centers around the world. Although public clouds provide services to the public over the Internet, enterprises may use private clouds or hybrid clouds. Private and hybrid clouds also include a network of servers housed in a data center.

[0028] Data centers include not only servers, but also other components such as network switches, routers, and other equipment. Servers and other components may be provided by different vendors and may include different types or versions of motherboards, CPUs, memory, and other devices.

[0029] Each of these devices may require low-level system code including firmware. Deploying packages to a wide variety of devices that may be distributed in many data centers around the world is challenging. This is because the deployment of packages needs to be done safely, securely, and reliably. There are several external factors that can affect security, robustness, and reliability goals. As an example, deployments are often much larger than can be managed at one time, especially when some deployments have high-impact potential. Certain types of changes or goals require the explicit consent of other parties to gate deployment (for example, potential power or performance impacts). In addition, the impact of the deployment of packages may need to be monitored to ensure a safe and reliable deployment. Finally, payloads often include firmware or other code from other companies and must be evaluated and tested to ensure safety.

[0030] To ensure safe, secure, and reliable deployment of packages, specific examples of the present disclosure involve ensuring quality payloads, proper validation and testing, and monitoring the impact on the fleet. Specific examples involve using machine learning to improve the creation and deployment of packages.

[0031] Figure 1 A diagram of a system environment 100 for deploying packages to devices in a cluster of hardware in a cloud according to one example is shown. Examples of devices include, but are not limited to, baseboard management controllers (BMCs), CPUs, GPUs, FPGAs, FPGA instances, rack managers / controllers, chassis managers / controllers, power supply unit controllers, storage devices (e.g., SSDs or HDDs), network devices (e.g., switches, routers, firewalls, and bridges), or any other device in a data center that may need to be updated. A package may include a payload that may include instructions, low-level system code, firmware, settings, configurations, or other information that may need to be updated. The system environment 100 may include payload submission 102, package creation 110, package repository 120, deployment and monitoring 140, compute / storage / network 160, control plane 170, data plane 180.

[0032] Continue to refer Figure 1, package creation 110 may include scanning 112, testing 114, and packaging 116. In this example, a payload may be received via payload submission 102. A self-service portal for a payload engineering team may be used to implement payload submission 102. Thus, in this example, payload submission 102 may provide a graphical user interface via which any payload may be submitted. As part of payload submission 102, a variety of relevant information may be obtained and stored in a database (e.g., a database 212 associated with deployment and monitoring). This may be obtained by submitting a questionnaire to the submitter. Alternatively or additionally, information may be included as metadata associated with the payload. The information may include information related to deployment, changes, testing, and impact. Payload-related information or any information derived from the submitted information is referred to as parameters associated with the payload.

[0033] Still refer to Figure 1 , scanning 112 may include scanning the payload for various parameters associated with the payload. As an example, as part of the process, metadata associated with the payload may be scanned and extracted. The extracted information may be processed to determine whether it meets the submission criteria. Meeting the submission criteria may indicate that the submission is a valid submission. After the submission is verified, the extracted information may be stored as a record in a database associated with deployment and monitoring 140. The information may also have a timestamp. Scanning 112 may also include scanning the payload for any viruses or other undesirable artifacts. Testing 114 may include testing the payload by updating a specific target device with the payload to ensure that the payload will work as expected when installed. Packaging 116 may include packaging the payload. Additional steps involved as part of the process are described in other parts of this disclosure.

[0034] Continue to refer Figure 1 As shown in dashed box 130, machine learning can be used to deploy machine learning models associated with deployment and monitoring 140. Similarly, as shown in dashed box 150, machine learning can be used to deploy machine learning associated with compute / storage / network 160, control plane 170, and data plane 180. Additional details on the use of such machine learning models are provided in other parts of this disclosure.

[0035] Figure 2 is based on an example including deployment and monitoring (e.g., Figure 1Block diagram of a system 200 of deployment and monitoring 140) and clusters 210. As used in the present disclosure, the term cluster may include, but is not limited to, some or all of the data centers owned by a cloud service provider, some or all of the data centers owned and operated by a cloud service provider, one or all of all data centers owned by a cloud service provider operated by customers of the service provider, any other combination of data centers, a single data center, or even some clusters in a particular data center. Deployment and monitoring 140 may be coupled to clusters 210 via a data plane 220. Deployment and monitoring 140 may be coupled to clusters 210 via a control plane 230. Clusters 210 may include one or more data centers, which in turn may include computing clusters / storage / network devices. Thus, in this example, clusters 210 may include cluster 1 240, cluster 2 260, and cluster Q 280, where Q may be an integer greater than 1. Cluster 1 240 may be coupled to deployment and monitoring 140 via bus 222; cluster 2 260 may be coupled to deployment and monitoring 140 via bus 224; cluster Q 280 may be coupled to deployment and monitoring 140 via bus 226. Cluster 1 240 may be coupled to deployment and monitoring 140 via bus 232; cluster 2 260 may be coupled to deployment and monitoring 140 via bus 234; cluster Q 280 may be coupled to deployment and monitoring 140 via bus 236. Figure 2 As shown in , machine learning can be coupled to deployment and monitoring 140. Additional details associated with machine learning are provided later in this disclosure. Figure 2 Not shown, deployment and monitoring 140 may be coupled to the network via any number of intermediate networks (e.g., wide area networks, optical networks, wireless networks, wired networks, or other types of networks). Figure 2 Thus, the term bus as used in this disclosure includes, but is not limited to, signal lines, signal lines coupled via routers, switches, other network devices, signal lines coupled via any type of network, wireless connections, combinations of signal lines and wireless connections, switching fabrics, etc. Although Figure 2 A specific number of clusters of cluster 210 are shown arranged in a specific manner, but cluster 210 may include more or fewer clusters. Figure 2 While a particular arrangement of deployment and monitoring 140 is shown in relation to cluster 210 , deployment and monitoring 140 may be arranged differently, including portions distributed across several locations and interconnected via different types of networks or buses.

[0036] Figure 33 is a block diagram of a cluster 300 in a data center according to an example. Cluster 300 can be one of the clusters included in a cluster managed using deployment and monitoring 140. In this example, cluster 300 can include a top-of-rack (TOR) switch that can be used to interconnect racks of hardware components. Thus, cluster 300 can include TOR switches 302, 304, and 306. Each TOR switch can couple at least some of the components in cluster 300. As an example, TOR switch 304 can interconnect rack 310, rack 330, and rack 350. Each rack can include a rack manager and several racks, which can include components such as servers, network storage, network hardware (e.g., switches, routers, and bridges), etc. Rack 310 can include several chassis, such as chassis 314, 316, 318, and 320. Similarly, rack 330 can include several chassis, such as chassis 334, 336, 338, and 340. In addition, rack 350 may include multiple chassis, such as chassis 354, 356, 358, and 360. Each rack may include a rack manager that is configured to interface with a deployment and monitoring system such as deployment and monitoring 140. Thus, rack 310 may include rack manager 312, rack 330 may include rack manager 332, and rack 350 may include rack manager 352. Each chassis may include servers organized into groups, such as blades. Some or all of the chassis may also include network and storage devices. Each chassis may also include fans to provide cooling air to the servers or other components housed within the chassis. Although Figure 3 Specific arrangements of racks, switches, chassis, and components within the chassis are shown, but the systems and methods disclosed herein are not limited to any specific arrangements. Therefore, the systems and methods are applicable to any organization of a data center for both the data plane and the control plane.

[0037] Figure 4 Deployment and monitoring 400 according to one example is shown (e.g., Figure 1400). Deployment and monitoring 400 may include a processor 402, I / O devices 404, memory 406, display 408, sensor 410, deployment database 412, and network interface 414, which may be interconnected via bus 420. Processor 402 may execute instructions stored in memory 406. I / O device 404 may include components such as a keyboard, a mouse, a voice recognition processor, or a touch screen. Memory 406 may be any combination of non-volatile memory or volatile memory (e.g., flash memory, DRAM, SRAM, or other types of memory). Display 408 may be any type of display, such as an LCD, LED, or other type of display. Sensor 410 may include a telemetry or other type of sensor configured to detect and / or receive information (e.g., a condition associated with a device). Sensor 410 may include a sensor configured to sense a condition associated with a CPU, memory or other storage component, FPGA, motherboard, baseboard management controller, etc. Sensors 410 may also include sensors configured to sense conditions associated with racks, chassis, fans, power supply units (PSUs), etc. Sensors 410 may also include sensors configured to sense conditions associated with network interfaces, controllers (NICs), top-of-rack (TOR) switches, mid-rack (MOR) switches, routers, power distribution units (PDUs), rack-level uninterruptible power supply (UPS) systems, etc.

[0038] Continue to refer Figure 4 , the sensors 410 can be implemented in hardware, software, or a combination of hardware and software. Some sensors 410 can be implemented using a sensor API, which can allow the sensors 410 to receive information via the sensor API. Software configured to detect or listen for specific conditions or events can transmit any conditions associated with the equipment being monitored by the deployment and monitoring 400 via the sensor API. Remote sensors or other telemetry devices can be incorporated into the data center to sense conditions associated with the components installed therein. Remote sensors or other telemetry can also be used to monitor other adverse signals in the data center and provide information to deployment and monitoring. As an example, if the fan cooling the rack stops working, the sensor may sense this situation and report it to the deployment and monitoring function. This type of monitoring can ensure that any second-order effects of the deployment are detected, reported, and corrected.

[0039] Still refer to Figure 4 , the deployment database 412 can be used to store records related to payload submissions and packages. In addition, the deployment database 412 can also store data used to generate reports related to deployment. Additional details about the functionality of the deployment database 412 are recorded in other parts of this disclosure.

[0040] The network interface 414 may include a communication interface such as Ethernet, cellular radio, Bluetooth radio, UWB radio, or other type of wireless or wired communication interface. The bus 420 may be coupled to both the control plane and the data plane. Figure 4 Deployment and monitoring 400 is shown as including a certain number of components arranged and coupled in a certain manner, and it may include fewer or more components arranged and coupled differently. In addition, the functions associated with deployment and monitoring 400 may be distributed as desired.

[0041] Figure 5 The memory 500 (eg, Figure 4 406) of the memory, the memory including a memory for executing and deploying and monitoring 400 (and Figure 1 Memory 500 may include a pre-scan module 502, a planning module 504, a packaging module 506, a verification module 508, a deployment module 510, and a deployment monitor 512. Pre-scan module 502 may evaluate payload parameters provided as part of a payload submission (e.g., payload submission 102). Pre-scan module 502 may evaluate the impact of deployment of any package that includes the payload (e.g., using Figure 4 The results generated by the pre-scan module 502 can be recorded in Figure 4 database 412. The planning module 504 can build on the work performed by the pre-scan module 502 and determine any additional planning information or steps that will be needed and record the results back to the database 412. The planning module 504 can determine risk factors associated with the planned deployment. The planning module 504 can also determine gates and watchdogs that may be needed to ensure a safe and secure deployment. Details related to the gates and watchdogs can be recorded in the database 412. Based on all of this information, the planning module 504 can evaluate the coverage of the current configuration of the relevant areas of a given cluster that can be provided by the various stages used for deployment and verification. Automated deployment can be carried out by deploying as part of the verification of the package into a primary stage (e.g., a stage including nodes with no workloads), then a secondary stage (e.g., a stage including nodes with non-customer workloads), then a minimum scan tree (described below), and then a cluster deployment, the scope of which can vary based on the package and payload. Taking into account the current cluster usage and composition, a minimum scan tree for the cluster can be automatically generated to achieve the target coverage. Relative to Fig. 9 and Fig.10Additional details are provided regarding the generation of the minimum scan tree for the cluster. Finally, if parameters associated with the payload or packet indicate that additional segmentation is recommended (e.g., a generation before another generation of equipment), then a suggested segmentation plan can be generated. The generated information can be stored in database 412.

[0042] Still refer to Figure 5 , the packaging module 506 can process the submitted payload and build a deployment package. As part of this process, the packaging module 506 can generate deployment instructions based on the payload parameters. The packaging module 506 can specify a set of test-in-production (TIP) devices that can be targeted for deployment before further rolling out the deployment. In one example, the TIP mechanism can be used to perform minimal coverage testing to verify packages in key configurations. Figure 6 600. Thus, example package 600 may include one or more payloads for deployment. In this example, package 600 may include payload 1 602, payload 2 604, and payload N 606. N may be an integer greater than 1. Package 600 may also include health monitor 608. Health monitor 608 may include information about what to monitor and trigger thresholds associated with the monitored information. Package 600 may also include package deployment instructions 610. In one example, package deployment instructions 610 may include operations or actions that specify a deployment plan. Package deployment instructions may also include instructions about monitoring second-order effects at a more general level.

[0043] refer to Figure 5 , the verification module 508 can receive the completed package and verify the package. This process may include starting a test in production (TIP) deployment to the target device. The deployment monitor 512 can be implemented as a logical service to monitor the progress of the active deployment. The status of each submission and subsequent deployment can be tracked, including the start time of the deployment, the end time of the deployment, the running time of the deployment, and any delays in the deployment. The deployment monitor 512 can also track performance indicators related to the deployment. Table 1 below shows some major key performance indicators (KPIs) that can be tracked by the deployment monitor 512.

[0044]

[0045] Table 1

[0046] The deployment monitor 512 may also track additional KPIs, which are referred to as secondary KPIs in Table 2 below.

[0047]

[0048]

[0049] Table 2

[0050] Figure 7 An example of a segmented deployment 700 according to an example is shown. Package deployment may affect hundreds of unique components, each of which may have a corresponding SKU or unique identifier to identify the component type that distinguishes the component from other component types. An example component may be a specific CPU version produced by Intel. In this example, deploying firmware to an Intel CPU may affect dozens of unique CPU versions produced by Intel, which may be deployed as part of a fleet receiving a firmware update. A specific CPU version may have a corresponding SKU to identify the CPU of that version. A fleet may include thousands of CPUs with that specific CPU version. A fleet may include many other versions of Intel CPUs and thousands (or less) of CPUs of each of these other versions. In this example, deployment may be completed securely by following a secure deployment process. An example secure deployment process may include first scanning the fleet to determine the diversity of SKUs in the fleet. The process may include deployment and monitoring 140, which continuously scans the fleet and keeps track of each unique triple or 3-tuple, including a generation associated with the hardware, a manufacturer associated with the hardware, and a SKU associated with the device. Figure 4 The deployment database 412 may include a table for keeping track of unique triplets.

[0051] Continue to refer Figure 7In one example, a deployment test setup (e.g., cluster portion 710) may be organized in stages that may conceptually correspond to blast radiuses. Thus, each stage may include more and more hardware such that package deployments to an increasing number of stages may cover more and more hardware in the cluster. Thus, STAGE 1 720 may include hardware (e.g., servers) that are configured to process only synthetic workloads. As an example, servers organized as part of STAGE 1 720 may host virtual machines that do not serve any customer workloads or other workloads that may adversely affect a customer or another user. In this example, since any packages deployed to STAGE 1 720 will not affect any real workloads, using the blast radius analogy, the blast radius may be zero. STAGE 2 730 may include hardware (e.g., servers) that are configured to process real-time workloads, which, if impacted, may adversely affect the workloads of at least some users, but will not adversely affect the workloads of any customers. Thus, in this example, using the blast radius analogy, STAGE 2 730 may include at least a certain number of servers that, if affected by the package deployment, may affect at least a small number of users' workloads. One way to estimate the impact on customers' workloads is to classify servers based on the number of containers or the number of virtual machines (VMs). Thus, in one example, as part of STAGE 2 730, only those servers with a container count or VM count below a threshold (e.g., two containers per server or two VMs per server) may be targeted for deployment. STAGE 3 740 may include a large number of servers that, if affected, may adversely affect at least a small number of customers' workloads. In one example, STAGE 740 may also include servers with a larger container count per server or a larger VM count per server. STAGE 4 750 and STAGE 5 760 may include an increasing and diverse number of servers that, if affected, may adversely affect an increasing number of customer workloads. Again, in one example, an increasing number of customer workloads may correspond to an increasing number of container counts per server or VM counts per server. Of course, other markers of the customer's workload can also be used to determine which servers or other hardware to include in one stage or another. In one example, the blast radius of the deployment can be managed by first deploying the package to as few stages as possible. Figure 7 A specific number of stages for a secure deployment is shown, and more or fewer stages may be used.

[0052] Figure 8Flowchart 800 of a method for deploying a package according to one example is shown. Step 802 may include receiving a submission. After receiving the submission, some of the subsequent processing steps may be performed in parallel. Thus, in this example, one path may involve creating a package and validating the package, while a parallel path may involve pre-scanning and planning deployment. As part of the packaging path, the package may be automatically created and validated. Thus, step 804 may include creating a package that may include a payload, package deployment instructions (e.g., configuration instructions or settings), and health monitors (e.g., Figure 6 In this example, you can Figure 4 The processor 402 executes the corresponding Figure 5 This step is performed when the instruction of the packaging module 506 of . The package can also include a set of test in production (TIP) goals determined for verifying the package. Step 808 can include verifying the package. In this example, the package can be verified by Figure 4 The processor 402 executes the corresponding Figure 5 This step is performed when the pre-scan module 502 is instructed. Packet verification can include automatically processing packets as part of TIP. Any results obtained through these steps can be stored in Figure 4 The deployment database 412 is in the deployment database 412.

[0053] Continue to refer Figure 8 , planning the path may involve automatically determining risk factors and any gates that may be required. Thus, in this example, step 806 may include pre-scanning the payload and any other submission parameters to determine risk factors associated with the deployment of any package including the submitted payload. Figure 4 Risk factors and gates are tracked in the deployment database 412 of Figure 4 The processor 402 executes the corresponding Figure 5 This step is performed when the pre-scan module 502 of the system is instructed. Based on these parameters, Figure 7 STAGE 1 720 evaluates the coverage of a smaller number of servers or other types of equipment in an organization, and / or Figure 7 STAGE 2 730 will provide the current configuration of a given cluster. As part of this step, deployment and monitoring (e.g., Figure 1The deployment and monitoring 140) may provide a suggested deployment plan (step 810). The deployment plan may include the degree of parallelism that can be achieved during packaging and deployment. An example level of parallelism during deployment may involve whether all servers in a rack that are part of a single cluster can receive packages in parallel. The deployment plan may also include the level of testing and validation required before the package is deployed to the cluster. The deployment plan may also take into account the importance of the customer and their workloads. Thus, a particular customer may have devices that are critical to their operations, and any deployment of packages to these devices may require additional sign-off. The details associated with the deployment plan may be stored in Figure 4 In this example, the deployment database 412 or other storage of Figure 4 The processor 402 executes the corresponding Figure 5 This step is performed when the planning module 504 is instructed. The deployment plan can be approved or disapproved in step 812. In one example, the decision can be made by an administrator associated with the cluster.

[0054] Still refer to Figure 8 If the deployment plan is approved in step 812, then in step 814, the package can be deployed. In this example, the deployment plan can be approved in step 814. Figure 4 The processor 402 executes the corresponding Figure 5 This step is performed when the deployment module 510 is instructed. Fig. 9 and Fig.10 Additional details of deployment using the minimum scanning tree method are provided. Alternatively, if the deployment is rejected in step 812, then in step 816, the submission can be rejected. As part of this step, deployment and monitoring can record the rejection of the submission. Although Figure 8 A specific number of steps are shown performed in a specific order, but more or fewer steps may be performed in the same order or in a different order.

[0055] Fig. 9 An example is shown for Figure 7 FIG. 9 is a diagram of a scan tree 900 of hardware in STAGE 1 720 of FIG. As previously described, STAGE 1 720 may include hardware (e.g., servers) that are configured to process only synthetic workloads. As an example, a server organized as part of STAGE 1 720 may host a virtual machine that does not process any customer workloads or other workloads that may adversely affect customers or other users. In this example, Fig. 9Two generations (G1 910 and G2 940) of hardware are shown to exist in STAGE 1 720. Each device with a unique SKU (or some other item identifier) ​​that may be the target of deployment of a payload (e.g., firmware) may be represented as an edge of a scan tree. In this example, each device may be identified by a triple (3-tuple) including: (1) the generation of the hardware in the data center where the device is located, (2) the manufacturer of the server or other device that has the device, and (3) the SKU associated with the device. Thus, in this example, STAGE 1 720 may include the following triples: G1 910, M1 912, SKU 1 922; G1 910, M1 912, SKU 2 924; G1 910, M2 914, SKU 3 926; G1 910MM 916, SKU 7 928; and G1 910, MM 916, SKU 9 930. STAGE 1 720 may also include the following triplets: G2 940, M1 942, SKU 1 952; G2 940, M1 942, SKU 2 954; G2 940, M3 944, SKU 5 956; and G2 940, MM 946, SKU 8 958. Information about the triplets in the scan tree 900 corresponding to STAGE 1 720 may be stored in a database (e.g., Figure 4 deployment database 412). Although Fig. 9 The scan tree is shown with only two generations of hardware; other generations of hardware may have additional scan trees. Fig. 9 A specific number of manufacturers and a specific number of SKUs are shown, but other manufacturers and SKUs may also be present.

[0056] Fig.10 According to an example Figure 7 FIG. 1000 is a diagram of a scan tree 1000 of hardware in STAGE 2 730 of FIG. As previously described, STAGE 2 730 may include at least some number of servers that, if affected by package deployment, may affect at least a small number of users' active workloads. In this example, Fig.10 There are two generations (G1 1010 and G2 1040) of hardware in STAGE 2 730. Each device with a unique SKU (or some other identifier) ​​that can be targeted for deployment of a payload (e.g., firmware) can be represented as an edge of the scan tree. In this example, similar to the example for Fig. 9In the described example, each device can be identified by a 3-tuple including: (1) the generation of the hardware in the data center where the device is located, (2) the manufacturer of the server or other device that has the device, and (3) the SKU associated with the device. 1034; G1 1010, MM 1016, SKU 8 1036; and G1 1010, MM 1016, SKU 9 1037. 1038. STAGE 2 730 may also include the following triplets: G2 1040, M1 1042, SKU 1 1052; G2 1040, M1 1042, SKU 2 1054; G2 1040, M1 1042, SKU 3 1056; G2 1040, M3 1044, SKU 4 1058; G2 1040, M3 1044, SKU 5 1060; G2 1040, MM 1046, SKU 7 1062; and G2 1040, MM 1046, SKU 8 1064. In this example, the scan tree of STAGE 2 730 includes components of higher diversity. As an example, the scan tree of STAGE 2 730 includes SKU 3 1026 as part of G1 generation hardware provided by manufacturer M1. In addition, the scan tree of STAGE 2 730 includes additional types of components for G1 generation provided by manufacturer MM. Similarly, for G2 generation hardware, the scan tree includes other types of components. Information about the triples in the scan tree 1000 corresponding to STAGE 2 730 may be stored in a database (e.g., Figure 4 deployment database 412). Although Fig.10 The scan tree is shown with only two generations of hardware, but additional scan trees may exist for other generations of hardware. Fig.10 A specific number of manufacturers and a specific number of SKUs are shown, but others may exist.

[0057] Fig.11A and Fig. 11BA flow chart of a method for deploying a package according to an example is shown. Step 1102 may include scanning hardware in a cluster to obtain information about the hardware. In one example, step 1102 may include Figure 5 The pre-scan module 502 scans the hardware in the cluster to obtain information about the hardware components deployed in the cluster. The information obtained by scanning the hardware can be stored in Figure 4 The scanned information may include information about the generation, manufacturer, and SKU associated with each hardware component in the fleet. This information may be organized in one or more tables and stored in Figure 4 The deployment database 412 is in the deployment database 412.

[0058] Step 1104 may include classifying the hardware in the cluster into deployment categories divided by volume. In one example, classifying the hardware in the cluster into deployment categories by volume may include processing the scan information about the hardware. Figure 5 The deployment category may include at least one type (or category) of components that may require a deployment package. Thus, in one example, each deployment category may include those components that may receive the same or similar payloads. The classification information may be stored in Figure 4 The planning module 504 may also determine risk factors associated with the planned deployment. As an example, the planning module 504 may determine the types of customers that may be affected by the deployment. The planning module 504 may further determine gates and watchdogs that may be required to ensure a safe and secure deployment. Details related to gates and watchdogs may also be recorded in Figure 4 The deployment database 412 of FIG. Based on all of this information, the planning module 504 can evaluate the coverage that various phases for deployment and validation can provide for the current configuration of the relevant areas of the given fleet.

[0059] Step 1106 may include mapping the package to the device selected for deployment. As part of this step, the planning module 504 may create information (e.g., a table or a set of tables) that maps the package to the device selected for deploying the package. This information may be stored in Figure 4 The deployment database 412 is in the deployment database 412.

[0060] Step 1108 may include scanning the hardware in STAGE 1 to determine whether the selected diversity goal is met. If the selected diversity goal is met, the process may proceed to process STAGE A 1110. Otherwise, the process may proceed to process STAGE B 1112. In one example, as part of this step, Fig. 9 and Fig.10, the planning module 504 can build (or process an existing) minimum scan tree. The goal can be to achieve a reasonable amount of confidence in deploying the package in a manner consistent with operational characteristics, such as with minimal disruption to the customer's workload. As an example, for Fig. 9 As described, if a package is to be deployed to FPGAs having five different SKUs in a cluster, the minimum scan tree may include a selected set of triplets. In STAGE 1, the selected diversity target is allowed to be overridden. The selected diversity target may be the percentage of different types of SKUs that may receive the package as part of the planning phase of the deployment. Thus, in this example, as long as 80% of the different types of FPGAs receive the package, it may be sufficient to meet the selected diversity target. Assuming, as part of this example, that STAGE 1 only includes 50% of the different types of FPGAs, the planning module 504 may conclude that the selected diversity target is not met and processing may continue to process STAGE B 1112. Alternatively, if STAGE 1 includes 80% of the different types of FPGAs, then the planning module 504 may proceed to process STAGE A 1110.

[0061] about Fig. 11B , if scanning the hardware in STAGE 1 does not achieve the selected diversity goal, the process can continue from processing stage B 1112. Therefore, step 1114 can include scanning the hardware in STAGE 2 to determine whether the selected diversity goal is met. In this example, as described with respect to FIG. 11, the planning module 504 can build (or process an existing) minimum scanning tree. Similar to the Fig.11A In the example of the FPGA of STAGE 1, the planning module 504 can scan the hardware in STAGE 2 to determine whether the selected diversity goal is met. Therefore, if the combination of the SKUs of the FPGAs in STAGE 1 and STAGE 2 includes 80% of the SKUs of the FPGAs, the planning module 504 can proceed to process stage A 1110. Otherwise, the process can proceed to step 1116.

[0062] Step 1116 may include scanning the hardware in STAGE 3 to determine whether the selected diversity goal is met. In this example, the planning module 504 may be configured to perform the following operations in conjunction with the hardware in STAGE 3: Fig.10 The STAGE 3 minimum scan tree can be constructed (or processed) in a similar manner as described above. Fig.11AFor example, if the FPGA in STAGE 1, STAGE 2, and STAGE 3 has a combination of SKUs, the planning module 504 can scan the hardware in STAGE 3 to determine whether the selected diversity goal is met. Therefore, if the combination of SKUs of the FPGAs in STAGE 1, STAGE 2, and STAGE 3 includes 80% of the SKUs of the FPGAs, the planning module 504 can proceed to process stage A 1110. Otherwise, the process can proceed to step 1118.

[0063] Step 1118 may include continuing to scan additional stages until the selected diversity target is met or all remaining stages have been scanned. In one example, the selected diversity target may be selected based on package type. Alternatively or additionally, the selected diversity target may be selected based on impact type. Thus, for a particular package type, the selected diversity target may be 75% of the SKUs, while for another package type, the selected diversity target may be 90%.

[0064] Once the state of the fleet has been determined and the minimum scan tree for deployment of a particular package or group of packages has been determined, processing can proceed to the next steps. These steps can include determining the speed of the deployment. In one example, the speed of the deployment can be related to the number of gates that the deployment includes. Each gate can correspond to a wait period (e.g., a particular number of hours, days, or months) that specifies the time that the deployment can be delayed after each step of the deployment process. As an example, for deploying a particular package to a CPU, the deployment can be gated within 24 hours of deployment to the minimum scan tree; after 24 hours, the package can be deployed to the CPUs in the rest of the fleet with the associated SKU. In one example, when the deployment involves monitoring the CPUs via a control plane (e.g., Figure 2 When a gate is used to process a device that is processed by the control plane 230 of the control plane 230 relative to a device that is processed via the data plane, the gate may specify a longer wait period (e.g., Figure 2 data plane 220).

[0065] In another example, the speed of a deployment may be related to the impact of the deployment. Thus, the number of gates and the waiting time periods specified by the gates may depend on the impact of the deployment on the fleet. As an example, certain deployments may be characterized as having an impact, while other deployments may be characterized as having no impact. Deployments may also be characterized on a sliding scale between having no impact and having an impact. The process may include a planning module 504 that considers both the type of package and the type of impact of the package. Information corresponding to the impact, including the impact type and the type of package, may be stored in a database (e.g., Figure 4 In a table in the deployment database 412).

[0066] Fig.121 shows an impact table 1200 according to an example. The impact table 1200 can be used to keep track of the impact of package deployment on various devices in the fleet. As an example, the impact table 1200 can be used to classify the impact on the device into multiple impact types and to classify the package into multiple package types. In one example, the impact table 1200 can be stored in Figure 4 12. In this example, the impact table 1200 may include information organized into rows and columns, including impact types 1210 in the rows and package types 1240 in the columns. The impact types 1210 may include CPU pause 1212, storage pause 1214, network pause 1216, FPGA pause 1218, restart 1220, power increase 1222, performance decrease 1224, and thermal impact 1226. The package types 1240 may include microcode 1242, universal extensible firmware interface (UEFI) / basic input / output system (BIOS) 1244, baseboard management controller (BMC) 1246, solid state drive (SSD) 1248, hard disk drive (HDD) 1250, and FPGA 1252. FPGA 1252 may also include two package subtypes: FPGA platform 1254 and FPGA image 1256. Although the impact table 1200 shows specific information organized in a specific manner, more or less information may be included and organized in different manners. In addition, the information in the impact table 1200 can be enclosed in other types of data structures including linked lists or other structures. As another example, the information in the impact table can be distributed so that the impact information of each package can be included with the package as metadata or other data structures associated with the package.

[0067] Continue to refer to Fig.12 In this example, as shown in the table, deploying a package with microcode can cause the CPU to pause for less than X seconds (Xs); deploying a package with UEFI / BIOS can cause the CPU to pause for less than X seconds (Xs); and deploying the package to an FPGA platform can cause the CPU to pause for less than X seconds (Xs), where X is a number. In addition, in this example, deploying the package to an SSD can cause a storage pause of Y seconds (Ys), while deploying the package to an HDD can cause a storage pause of Z seconds (Zs), where each of Y and Z is a number. In addition, deploying the package to an FGPA platform can cause a network pause of less than F seconds (Fs). Deploying a package to an FPGA platform can cause the FPGA to pause for less than P seconds (Ps). On the other hand, deploying a package to an FPGA image can be image-specific.

[0068] Still refer to Fig.12, in this example, the impact table 1200 may include information about the packet types that may cause a restart. Thus, in this example, deploying UEFI / BIOS packages and deploying packages to FPGA platforms may always cause a restart; however, deploying packages to SSDs or HDDs may only cause a restart at certain times. The impact table 1200 may also include information about power changes (e.g., increased power) based on deployment of specific package types. For microcode, UEFI / BIOS, SSD, and HDD package types, the power increase may be small; however, for FPGA platform package types, the power increase may be moderate, and for FPGA images, the power increase may be image-specific. The impact table 1200 may also include information about performance changes (e.g., performance degradation) based on deployment of specific package types. For microcode, UEFI / BIOS, SSD, and HDD package types, the performance degradation may be small. The impact table 1200 may also include information about changes in thermal impact (e.g., higher or lower thermal impact) based on deployment of specific package types. For microcode, UEFI / BIOS, and BMC package types, the thermal impact may be low.

[0069] Although the impact table 1200 includes information about a particular package type and an impact type, the impact table 1200 may include specific information about each of additional or fewer package types and impact types. As an example, the impact table 1200 may include information about the impact of deploying a package to a network interface controller (NIC), a top-of-rack (TOR) switch, a mid-rack (MOR) switch, a router, a power distribution unit (PDU), and a rack-level uninterruptible power supply (UPS) system.

[0070] Fig.13 A machine learning system 1300 according to an example is shown. The machine learning system 1300 may include a processor 1302, an I / O device 1304, a memory 1306, a sensor 1310, a display 1320, and a network interface 1322, which may be interconnected via a bus system 1330. The bus system 1330 may be coupled to a data plane (e.g., Figure 2 data plane 220) and a control plane (e.g., Figure 2 The processor 1302 may execute instructions stored in the memory 1306. The memory 1306 may be any combination of non-volatile memory or volatile memory (e.g., flash memory, DRAM, SRAM, or other types of memory). The sensor 1310 may include a telemetry or other type of sensor configured to detect and / or receive information (e.g., a condition associated with a device).

[0071] Continue to refer Fig.13, the sensors 1310 may include sensors configured to sense conditions associated with a CPU, memory or other storage components, FPGAs, motherboards, baseboard management controllers, and the like. The sensors 1310 may also include sensors configured to sense conditions associated with racks, chassis, fans, power supply units (PSUs), and the like. The sensors 1310 may also include sensors configured to sense conditions associated with network interface controllers (NICs), top of rack (TOR) switches, mid-rack (MOR) switches, routers, power distribution units (PDUs), rack-level uninterruptible power supply (UPS) systems, and the like. The sensors 1310 may be implemented using a sensor API, which may allow the sensors 1310 to receive information via the sensor API. Software configured to detect or listen for specific conditions or events may communicate any conditions associated with the equipment being monitored by the deployment and monitoring 400 via the sensor API. Remote sensors or other telemetry devices incorporated into the data center to sense conditions associated with components installed therein may sense conditions and provide information to the sensors 1310 or the processor 1302. In addition, the deployment and monitoring may also transmit data related to the event or condition to the sensor 1310 or the processor 1302. As an example, by Figure 4 Any events or conditions sensed by sensor 410 may be provided to processor 1302 as desired.

[0072] Display 1320 may be any type of display, such as an LCD, LED, or other type of display. Network interface 1322 may include a communication interface, such as Ethernet, cellular radio, Bluetooth radio, UWB radio, or other type of wireless or wired communication interface. Fig.13 The machine learning system 1300 is shown as including a specific number of components arranged and coupled in a specific manner, and it may include fewer or additional components arranged and coupled differently. In addition, the functions associated with the machine learning system 1300 can be distributed as desired.

[0073] Fig.14 Memory 1400 (e.g., Fig.13 1406). In this example, instructions may be organized in memory 1400 in blocks or modules that include code, data, or both. In this example, memory 1400 may include a learning-based analyzer (LBA) 1410, training data 1420, a machine learning (ML) model 1430, an impact table 1440, payload parameters 1450, package parameters 1460, deployment parameters 1470, and fleet parameters 1480. Although Fig.14 While instructions and data are shown organized in a particular manner, the instructions and data may be combined or distributed in various ways.

[0074] Continue to refer Fig.14 , a learning-based analyzer (LBA) 1410 can implement a supervised learning algorithm that can be trained based on input data, and once trained, it can make predictions or indications based on the training. In this example, the LBA 1410 can implement techniques such as linear regression, support vector machines (SVM) for regression settings, random forests for regression settings, gradient boosting trees for regression settings, and neural networks. Linear regression can include modeling the past relationship between independent variables and dependent output variables. The neural network can include artificial neurons for creating an input layer, one or more hidden layers, and an output layer. Each layer can be encoded as a weight matrix or vector represented in the form of coefficients or constants, which may have been obtained via offline training of the neural network. The neural network can be implemented as a recurrent neural network (RNN), a long short-term memory (LSTM) neural network, or a gated recurrent unit (GRU). All information required for a supervised learning-based model can be converted into a vector representation corresponding to any of these techniques. Taking LSTM as an example, an LSTM network can contain a series of repeated RNN layers or other types of layers. Each layer of the LSTM network can consume an input at a given time step, for example, the state of the layer from the previous time step, and can produce a new set of outputs or states. In the case of using LSTM, a single block of content can be encoded as a single vector or multiple vectors. As an example, a word or a combination of words (for example, a phrase, sentence, or paragraph) can be encoded as a single vector. Each block can be encoded into a single layer of the LSTM network (for example, a specific time step). The LSTM layer can be described using a set of equations, as shown below:

[0075] i t =σ(W xi xt+W hi h t-1 +W ci c t-1 +b i

[0076] f t =σ(W xf x t +W hf h t-1 +W cf c t-1 +b f )

[0077] c t =f t c t-1 i t tanh(W xc x t +Whc h t-1 +b c )

[0078] o t =σ(W xo x t +W ho h t-1 +W co c t +b o )

[0079] h t =o t tanh(c t )

[0080] In this example, within each LSTM layer, the input and hidden states can be processed using a combination of vector operations (e.g., dot products, inner products, or vector additions) or nonlinear operations, if necessary.

[0081] Although Fig.14 LBA 1410 is described as including instructions that may be encoded as hardware corresponding to an A / I processor. In this case, some or all of the functionality associated with the learning-based analyzer may be hard-coded or otherwise provided as part of the A / I processor. As an example, the A / I processor may be implemented using an FPGA having the necessary functionality.

[0082] The training data 1420 may be data that may be used to train a neural network model or similar machine learning model. In one example, the training data 1420 may be used to train a machine learning model to minimize an error function associated with the deployment of a package. In one example, the minimization of the error function may be obtained by obtaining user feedback on various payload and package parameters, and determining appropriate weights for convolution operations or other types of operations to perform as part of machine-based learning. As an example, users in a test environment may be provided with a set of pre-selected mapping functions with known payload and package parameters, and asked to select the mapping function they prefer.

[0083] ML model 1430 may include a machine language model that may be used as part of machine learning system 1300. ML model 1430 may include a model created by a training process. In this example, training data 1420 may include target attributes, such as a selected diversity target for deploying a package. An appropriate machine learning algorithm included as part of LBA 1410 may find patterns in training data 1420 that map a given set of input parameters (e.g., payload parameters and package parameters) to the selected diversity target for deploying the package. In another example, the machine learning algorithm may find patterns in training data 1420 that map input parameters to deployment classifications. Example deployment classifications may include at least two categories: with impact or without impact. Other machine language models may also be used. As an example, training data 1420 may be used to train a machine language model that maps input package types to any impacts associated with the deployment of the package. The impacts may be represented in a form similar to that described with respect to impact table 1440. Thus, impact table 1440 may be used with Fig.12 The impact table 1200 is similar or identical.

[0084] The payload parameters 1450 may include parameters associated with the payload. In one example, the payload parameters may include the type of payload, the target SKU of the payload, the degree of change caused by the deployment of the payload, any prerequisites, any known impacts, and the required deployment time. The payload parameters 1450 may be extracted from metadata associated with the payload or otherwise obtained through the submission process as previously described.

[0085] The package parameters 1460 may include parameters associated with a package including a payload. In one example, the package parameters 1460 may include information about the type of health monitoring included in the package. The package parameters 1460 may also include the package type and the gates and watchdogs required to deploy the package.

[0086] Deployment parameters 1470 may include information about the rollout plan. As an example, deployment parameters 1470 may include an assessment of target conditions required for deployment. These conditions may include information about whether any device resets, node restarts, node re-arrangements, power cycles, or disk reformatting are required. These parameters may be included as part of the instructions and / or metadata associated with the package.

[0087] The fleet parameters 1480 may include information about an entire fleet or a subset of the fleet that may be a deployment target. The fleet parameters may include information related to the types of items (e.g., SKUs) associated with the data centers in the fleet or a subset of the fleet. The information may include the quantity of each of the SKUs. In addition, the fleet parameters 1480 may include additional details about the data centers included in the fleet or a subset of the fleet. As an example, the information about the data centers may include location information, the AC voltage supply of the data center (e.g., 120 volts or 240 volts), operator information (e.g., whether the data center is operated by a service provider or by a customer of the service provider). The fleet parameters 1480 may be used to determine the number of data centers included in the fleet or a subset of the fleet. Figure 5 The deployment module 510 evaluates the cluster parameters 1480. Some of the cluster parameters 1480 may be stored in Figure 4 The deployment database 412 is in the deployment database 412.

[0088] ML model 1430 may include a model that is trained to prioritize targets with the least impact. Thus, in one example, the ML model may learn that when a node needs to be restarted, empty nodes should be deployed first—because they are not running any workloads. ML model 1430 may also include a model that can be trained to receive parameters associated with payloads, packages, deployments, and clusters as inputs, and determine whether some of the deployment steps can be performed in parallel. In addition, ML model 1430 may include a model that can be trained to receive parameters associated with payloads, packages, deployments, and clusters as inputs, and determine specific gates and watchdogs that may be required during deployment to a cluster. In addition, ML model 1430 may include a model that can be trained to receive parameters associated with payloads, packages, deployments, and clusters as inputs, and determine the type of health monitoring that should be included as part of the deployment of the package. Finally, other automatic feedback models may also be used. As an example, such automatic feedback models may not rely on machine learning; instead, they may rely on other feedback mechanisms to allow automatic creation of packages for deployment or to allow automatic creation of deployment plans for deploying packages to clusters. Regardless, in some cases, the automatic feedback model can be learned using a machine language model such as a reinforcement learning model.

[0089] Fig.15 Flowchart 1500 illustrates a method for creating a package including a payload for deployment to a group of devices according to an example. Step 1502 may include receiving a payload, wherein the payload has an associated set of payload parameters related to deploying the payload to the group of devices. As previously described, the payload may be received via a submission portal or otherwise. The payload parameters may be about Fig.14 Payload parameters 1450 explained.

[0090] Step 1504 may include automatically creating, using a processor, a package for deployment to the group of devices, wherein the package includes instructions for deploying a payload to the group of devices, and wherein the instructions specify at least one of a plurality of operations derived from a machine learning model based on at least a subset of payload parameters for the associated group. In this example, processor 1302 may execute instructions stored in memory 1306 (e.g., instructions corresponding to learning-based analyzer 1410) to perform this step. The instructions to deploy the payload may specify operations, such as the number of gates and / or watchdogs required for deployment. The operations may involve any deployment parameters related to deploying the package to the group of devices (e.g., Fig.14 The deployment parameters 1470 of the package. As an example, the operation can specify a deployment schedule and scope including a rollout plan. The operation can also include health monitoring information for package deployment. The health monitoring information can include what to monitor and trigger thresholds associated with the monitoring information.

[0091] In one example, automatically creating a package for deployment to the group of devices may include processing metadata or other submission parameters associated with the payload. The machine learning model may be trained based on training data, the training data including mappings between at least a subset of the associated group of payload parameters and a set of labels that classify the impact of deploying the payload to the group of devices. In one example, the set of labels may include a first label that classifies the impact as having an impact and a second label that classifies the impact as having no impact. Fig.14 Any of the ML models 1430 described can be trained and used as explained above.

[0092] Fig.16 Flowchart 1600 of a method for deploying a package to a cluster is shown according to an example. Step 1602 may include evaluating the cluster to determine a set of cluster parameters associated with deploying the package to the cluster. In this example, processor 1302 may execute instructions stored in memory 1306 (e.g., instructions corresponding to learning-based analyzer 1410) to perform this step. Evaluating the cluster may include processing metadata associated with the cluster. The metadata may include information about the use and composition of the cluster. The metadata may be stored in Figure 4 In one example, as previously described with respect to Fig.11A and Fig. 11B As explained, assessing the cluster may also include scanning hardware associated with the cluster.

[0093] Step 1604 may include, using a processor, automatically creating a deployment plan for deploying the package to the cluster, wherein the deployment plan includes instructions for deploying the package to the cluster, and wherein the instructions specify at least one of a plurality of operations derived from a machine learning model based on at least a subset of the set of cluster parameters. In this example, processor 1302 may execute instructions stored in memory 1306 (e.g., instructions corresponding to learning-based analyzer 1410) to perform this step. The machine learning model may be trained based on training data, wherein the training data includes a mapping between at least a subset of the cluster parameters and at least one label associated with the deployment plan. In addition, the machine learning model may be trained based on feedback regarding the deployment of the package to the cluster. The plurality of operations may include actions corresponding to monitoring the deployment of the package to the cluster. Thus, as previously described, the deployment monitor may monitor the deployment to the cluster. Regarding Figure 5 Deployment monitor 512 provides additional details about the deployment monitor. The operation may include information about what to monitor and trigger thresholds associated with the monitored information. In addition, the operation may include actions corresponding to generating information about a minimum scanning tree including a set of devices in the cluster. Fig.11A and Fig. 11B Provides additional details concerning the generation of information about the minimum scan tree.

[0094] In one example, the package can be rolled out across the fleet in stages in a manner that has minimal impact on customer workloads. Thus, the package can first be deployed to empty nodes (e.g., nodes that do not host any workloads). Next, the package can be deployed to those nodes that have a minimum number (e.g., two) of workloads (determined based on, for example, container counts or counts of virtual machines supported by the node). Next, the package can be deployed to those nodes that have slightly more workloads, and so on. This can limit the blast radius and help contain any damage to customer workloads if the deployment disrupts hardware functionality.

[0095] A logical reporting service can also be implemented to keep track of deployments in real time. This service can access Figure 4 The data in the deployment database 412 and data stored in other sources can be used to automatically generate reports. BI business analysis tools to implement reporting services. You can also implement and deploy dashboards.

[0096] Fig.17 A deployment dashboard 1700 is shown according to one example. The deployment dashboard 1700 includes information related to the current state of the fleet regarding deployment of various package types. The information can be displayed and tracked in real time. The information related to the current state of the fleet can be stored in Figure 4 The deployment database 412 and can be Figure 5The deployment monitor 512 of the deployment dashboard 1700 is retrieved and displayed via the deployment dashboard 1700. In this example, the deployment dashboard 1700 may include freshness information indicating the current state of the fleet regarding the deployment of a particular package type. In this example, the deployment dashboard 1700 may include a meter indicating the current fleet state by package type. The deployment dashboard 1700 may also indicate whether the meter is associated with a control plane device or a data plane device. In addition, the deployment dashboard 1700 may indicate that the deployment being tracked is low impact or has an impact. Thus, the meter 1710 may indicate the current state of the deployment of a package associated with a chassis manager. The meter 1720 may indicate the current state of the deployment of a package associated with a power supply unit (PSU). The meter 1730 may indicate the current state of the deployment of a package associated with a CPU microcode. The meter 1730 may indicate the current state of the deployment of a package associated with a CPU microcode. The meter 1740 may indicate the current state of the deployment of a package associated with a baseband management controller (BMC). Meter 1750 may indicate the current status of deployment of packages related to UEFI / BIOS. Meter 1760 may indicate the current status of deployment of packages related to HDD / SDD. Fig.17 While deployment dashboard 1700 is shown with a particular number of meters organized in a particular manner, deployment dashboard 1700 may include additional or fewer meters that may be organized differently. As an example, deployment dashboard 1700 may include meters for tracking other package types including, for example, FPGA platforms, FPGA images, network interface controllers (NICs), top-of-rack (TOR) switches, mid-rack (MOR) switches, routers, power distribution units (PDUs), and rack-level uninterruptible power supply (UPS) systems.

[0097] In addition, other dashboards can be provided, including a dashboard for tracking each active deployment. Each such dashboard can show the progress of deployments, including the current deployment rate and the estimated time to completion. In addition to active deployments, pending deployments can also be displayed. For pending deployments, the dashboard can include the status of the deployment, such as submitted, packaged, tested, pending, aborted, or completed. Deployment monitors (e.g., Figure 5 The deployment monitor 512 of ) can make additional details about each deployment (active or pending) available. In addition to the dashboard, the deployment monitor 512 can also provide reports on key results, such as deployment security, deployment time, detection efficiency, deployment impact, and deployment parallelism.

[0098] In summary, the present disclosure relates to a method for creating a package including a payload for deployment to a group of devices. The method may include receiving a payload, wherein the payload has an associated set of payload parameters related to deploying the payload to the group of devices. The method may also include automatically creating, using a processor, a package for deployment to the group of devices, wherein the package includes instructions for deploying the payload to the group of devices, and wherein the instructions specify at least one of a plurality of operations, the plurality of operations being derived from a machine learning model based on at least a subset of the associated set of payload parameters.

[0099] Automatically creating a package for deployment to the set of devices may also include processing metadata or other submission parameters associated with the payload. The machine learning model may be trained based on training data, the training data including a mapping between at least a subset of an associated set of payload parameters and a set of labels that classify the impact of deploying the payload to the set of devices. The set of labels may include a first label that classifies the impact as having an impact and a second label that classifies the impact as having no impact.

[0100] The plurality of operations may include actions related to: a schedule associated with deploying a package to a set of devices. The plurality of operations may include actions related to: a gate associated with deploying a package to a set of devices. The plurality of operations may include actions related to: a watchdog associated with deploying a package to a set of devices.

[0101] In another example, the present disclosure relates to a method for deploying a package to a cluster. The method may include evaluating the cluster to determine a set of cluster parameters associated with deploying the package to the cluster. The method may also include automatically creating, using a processor, a deployment plan for deploying the package to the cluster, wherein the deployment plan includes instructions for deploying the package to the cluster, and wherein the instructions specify at least one of a plurality of operations, the plurality of operations being derived from a machine learning model based on at least a subset of the set of cluster parameters.

[0102] Evaluating the fleet may include processing metadata associated with the fleet. The machine learning model may be trained based on training data, the training data including mappings between at least a subset of the fleet parameters and at least one label associated with the deployment plan. The machine learning model may be trained based on feedback regarding deployment of the package to the fleet.

[0103] The plurality of operations may include actions corresponding to monitoring deployment of a package to a cluster. The plurality of operations may include actions corresponding to generating information about a minimum spanning tree including a set of devices in the cluster.

[0104] In yet another example, the present disclosure relates to a system for deploying a package to a cluster. The system may be configured to evaluate the cluster to determine a set of cluster parameters associated with deploying the package to the cluster. The system may also be configured to automatically create, using a processor, a deployment plan for deploying the package to the cluster, wherein the deployment plan includes instructions for deploying the package to the cluster, and wherein the instructions specify at least one of a plurality of operations derived from a machine learning model based on at least a subset of the cluster parameters.

[0105] As part of evaluating the fleet, the system can also be configured to process metadata associated with the fleet. The machine learning model can be trained based on training data, the training data including mappings between at least a subset of the fleet parameters and at least one label associated with the deployment plan. The machine learning model can be trained based on feedback regarding deployment of the package to the fleet.

[0106] The plurality of operations may include actions corresponding to monitoring deployment of a package to a cluster. The plurality of operations may include actions corresponding to generating information about a minimum spanning tree including a set of devices in the cluster.

[0107] In yet another example, the present disclosure relates to a method for creating a package including a payload for deployment to a group of devices. The method may include receiving a payload, wherein the payload has an associated set of payload parameters related to deploying the payload to the group of devices. The method may also include automatically creating, using a processor, a package for deployment to the group of devices, wherein the package includes instructions for deploying the payload to the group of devices, and wherein the instructions specify at least one of a plurality of operations derived from an automatic feedback model based on at least a subset of the associated set of payload parameters.

[0108] Automatically creating a package for deployment to the set of devices may include processing metadata or other submission parameters associated with the payload. The automatic feedback model may include a reinforcement learning model trained based on training data, the training data including a mapping between at least a subset of the payload parameters for the associated set and a set of labels that classify the impact of deploying the payload to the set of devices. The set of labels may include a first label that classifies the impact as impactful and a second label that classifies the impact as less impactful.

[0109] The plurality of operations may include actions related to scheduling associated with deploying the package to the set of devices. The plurality of operations may include actions related to gates associated with deploying the package to the set of devices. The plurality of operations may include actions related to watchdogs associated with deploying the package to the set of devices.

[0110] It should be understood that the methods, modules and components described herein are merely exemplary. Alternatively or additionally, the functions described herein may be performed at least in part by one or more hardware logic components. For example, but not limited to, illustrative types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc. In an abstract but still clear sense, any arrangement of components that implement the same function is effectively "associated" to achieve the desired function. Therefore, any two components combined to achieve a specific function herein can be regarded as "associated" to each other to achieve the desired function, regardless of the architecture or intermediate components. Similarly, any two components so associated can also be regarded as "operably connected" or "coupled" to each other to achieve the desired function.

[0111] Functionality associated with some of the examples described in this disclosure may also include instructions stored in non-transitory media. The term "non-transitory media" as used herein refers to any medium that stores data and / or instructions that cause a machine to operate in a particular manner. Exemplary non-transitory media include non-volatile media and / or volatile media. Non-volatile media include, for example, hard disks, solid-state drives, magnetic disks or tapes, optical disks or tapes, flash memory, EPROM, NVRAM, PRAM, or other such media, or network versions of such media. Volatile media include, for example, dynamic memory such as DRAM, SRAM, cache, or other such media. Non-transitory media is different from transmission media, but can be used in conjunction with transmission media. Transmission media is used to transmit data and / or instructions to or from a machine. Exemplary transmission media include coaxial cables, fiber optic cables, copper wires, and wireless media, such as radio waves.

[0112] In addition, those skilled in the art will recognize that the boundaries between the functions of the above-mentioned operations are merely illustrative. The functions of multiple operations can be combined into a single operation, and / or the functions of a single operation can be distributed in additional operations. In addition, alternative embodiments can include multiple instances of a particular operation, and the order of the operations can be changed in various other embodiments.

[0113] Although the present disclosure provides specific examples, various modifications and changes may be made without departing from the scope of the present disclosure as set forth in the following claims. Therefore, the specification and drawings are to be regarded as illustrative rather than restrictive, and all such modifications are intended to be included within the scope of the present disclosure. Any benefits, advantages, or solutions to problems described herein with respect to specific examples should not be construed as key, required, or essential features or elements of any or all the claims.

[0114] Furthermore, as used herein, the terms "a" or "an" are defined as one or more than one. Furthermore, the use of introductory phrases such as "at least one" and "one or more" in the claims should not be interpreted as implying that the introduction of another claim element by the indefinite article "a" or "an" limits any particular claim containing such introduced claim element to an invention containing only one such element, even if the same claim includes the introductory phrases "one or more" or "at least one" and an indefinite article such as "a" or "an". The same applies to the use of definite articles.

[0115] Unless otherwise stated, terms such as "first" and "second" are used to arbitrarily distinguish between the elements such terms describe. Therefore, these terms are not necessarily intended to indicate temporal or other prioritization of such elements.

Claims

1. A method for creating a package, the package including a payload and for deployment to a set of devices, the method comprising: receiving the payload, wherein the payload has an associated set of payload parameters related to deploying the payload to the set of devices; as well as Using a processor, automatically create the package for deployment to the set of devices, wherein the package includes instructions for deploying the payload to the set of devices, and wherein the instructions specify a plurality of operations related to actions corresponding to gates and watchdogs associated with deployment of the payload to the set of devices, and wherein the plurality of operations are derived from a machine learning model based on at least a subset of the associated set of payload parameters, wherein the machine learning model is trained based on training data including a mapping between at least the subset of the associated set of payload parameters and a set of labels that classify an effect of deploying the payload to the set of devices. 2 . The method of claim 1 , wherein the automatically creating the package for deployment to the set of devices comprises processing metadata or other submission parameters associated with the payload.

3. The method of claim 1 , wherein the machine learning model is trained to minimize an error function associated with the deployment of the package. 4 . The method of claim 1 , wherein the set of labels includes a first label that classifies the influence as influential and a second label that classifies the influence as not influential. 5 . The method of claim 1 , wherein the plurality of operations further comprises actions related to which information to monitor and any trigger thresholds associated with the monitored information.

6. The method of claim 1, wherein the plurality of operations further comprises actions related to: a schedule associated with deploying the payload to the set of devices.

7. A method for deploying a package to a group of devices in a cluster, the package including a payload, the method comprising: evaluating the fleet to determine a set of fleet parameters associated with deploying the package to the fleet; Using a processor, automatically create a deployment plan for deploying the package to the fleet, wherein the deployment plan includes instructions for deploying the package to the fleet, and wherein the instructions specify a plurality of operations related to actions corresponding to gates and watchdogs associated with deployment of the payload to the set of devices, and wherein the plurality of operations are derived from a machine learning model based on at least a subset of the set of fleet parameters, wherein the machine learning model is trained based on training data including a mapping between at least the subset of the fleet parameters and at least one label associated with the deployment plan. The method of claim 7 , wherein evaluating the fleet comprises processing metadata associated with the fleet.

9. The method of claim 7, wherein the machine learning model is trained to minimize an error function associated with deployment of the package.

10. The method of claim 7, wherein the machine learning model is trained based on feedback regarding deployment of the package to the cluster.

11. The method of claim 7, wherein the plurality of operations further comprises actions corresponding to monitoring information including actions related to which information to monitor and any trigger thresholds associated with the monitored information.

12. The method of claim 7, wherein the plurality of operations further comprises an action corresponding to generating information regarding a minimum scanning tree including the set of devices in the cluster.

13. A system for deploying a package to a set of devices in a cluster, the package comprising a payload, the system being configured to: evaluating a fleet to determine a set of fleet parameters associated with deploying the package to the fleet; and Using a processor, automatically create a deployment plan for deploying the package to the fleet, wherein the deployment plan includes instructions for deploying the package to the fleet, and wherein the instructions specify a plurality of operations, the plurality of operations are related to actions corresponding to gates and watchdogs associated with deployment of the payload to the set of devices, and the plurality of operations are derived from a machine learning model based on at least a subset of the set of fleet parameters, wherein the machine learning model is trained based on training data including a mapping between at least the subset of the fleet parameters and at least one label associated with the deployment plan.

14. The system of claim 13, wherein the system is further configured to process metadata associated with the fleet as part of evaluating the fleet.

15. The system of claim 13, wherein the machine learning model is trained to minimize an error function associated with deployment of the package.

16. The system of claim 13, wherein the machine learning model is trained based on feedback regarding deployment of the package to the fleet.

17. The system of claim 13, wherein the plurality of operations further comprises an action corresponding to monitoring deployment of the package to the cluster.

18. The system of claim 13, wherein the plurality of operations further comprises an action corresponding to generating information regarding a minimum scan tree including the set of devices in the cluster.

19. A method for creating a package, the package including a payload and for deployment to a set of devices, the method comprising: receiving the payload, wherein the payload has an associated set of payload parameters related to deploying the payload to the set of devices; as well as Using a processor, automatically creating the package for deployment to the set of devices, wherein the package includes instructions for deploying the payload to the set of devices, and wherein the instructions specify a plurality of operations, the plurality of operations relating to actions corresponding to gates and watchdogs associated with deployment of the payload to the set of devices, and the plurality of operations are derived from an automatic feedback model based on at least a subset of the associated set of payload parameters, wherein the automatic feedback model includes a reinforcement learning model trained based on training data including mappings between at least the subset of the associated set of payload parameters and a set of labels that classify an impact of deploying the payload to the set of devices, wherein the set of labels includes: a first label that classifies an impact of deploying the payload to the set of devices as having an impact; and a second label that classifies an impact of deploying the payload to the set of devices as no impact.

20. The method of claim 19, wherein the automatically creating the payload for deployment to the set of devices comprises processing metadata or other submission parameters associated with the payload.

21. The method of claim 19, wherein the automatic feedback model comprises a reinforcement learning model trained to minimize an error function associated with deployment of the package.

22. The method of claim 19, wherein the plurality of operations further comprises actions related to which information to monitor and any trigger thresholds associated with the monitored information.

23. The method of claim 19, wherein the plurality of operations further comprises actions related to: a schedule associated with deploying the payload to the set of devices.

Citation Information

Patent Citations

  • Fast deployment across cloud platforms

    CN106796508A

  • A method and device for automatically deploying a model training environment

    CN109408062A