Improved Testing of Personalized Servers in Edge Computing

Abstract

This disclosure describes systems, methods, and devices related to testing provisioned servers at an edge computing device. The edge computing device may detect that a server has been provisioned to access a public network cloud using a backbone router of the edge computing device; provide a neural network trained based on training data including labeled configuration data and feature weights for assessing a probability that the performance of the server will meet a performance criterion; input settings and configurations associated with the provisioning of the server as inputs to the neural network; and use the neural network to generate a confidence score indicative of the probability based on the input and the training data.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application is related to U.S. patent application Ser. No. 63 / 380,135, filed October 19, 2022, entitled "Improved Testing of Personalized Servers in Edge Computing," and claims priority under 35 U.S.C. § 119(e) to that U.S. patent application, the entire contents of which are incorporated herein by reference for all purposes.

[0002] Embodiments of the present invention generally relate to systems and methods for network edge computing systems. [Background technology]

[0003] When a customer premises device is provisioned at a customer location, the provisioning may involve downloading software and configuration for the device from a centralized server, often using a file transfer protocol. The download and configuration may require a significant amount of time, and there may be some risk of data corruption during the download. Summary of the Invention

[0004] Users may provision and turn up personal servers in edge computing environments to provide direct access to public resources, such as the Internet or other cloud resources. Server provisioning in edge computing environments may allow users to customize the server settings and configuration, and may allow user selection of settings, configurations, and operating systems.

[0005] Once a user provisions a server in the edge computing environment (e.g., bare metal server as a service), the edge computing environment may use artificial intelligence trained to determine the type of server settings and configuration to monitor and the criteria by which the server's performance will be evaluated. The artificial intelligence may identify a subset of all setting and configuration data for the server without requiring user selection of server data for analysis, analyze, set, and adjust weights for the setting and configuration data being analyzed, and set and adjust criteria against which the setting and configuration data is compared for performance analysis. The artificial intelligence may be trained to generate a reliability score based on the setting and configuration data of the server provisioned using the edge computing environment. The reliability score may indicate the probability that the server will meet performance criteria.

[0006] The artificial intelligence may detect drift in the server's settings, configuration, and / or performance from an expected baseline. The artificial intelligence may compare data from the server with data from another server / topology to detect correlations (e.g., indicating expected or unexpected drift and indicating the root cause of the drift). If the drift is unexpected or the cause is not identified, the edge computing environment may notify a user of the server. If the confidence score is below a score threshold, indicating that the server is performing or not performing adequately based on its settings and configuration, the edge computing environment may notify a user of the server. [Brief explanation of the drawings]

[0007] [Figure 1] 1 illustrates an exemplary network environment for edge computing, according to one embodiment.

[0008] [Figure 2] FIG. 2 is a schematic diagram of the artificial intelligence of FIG. 1 used to test a server used in edge computing, according to one embodiment.

[0009] [Figure 3] 1 is a flowchart illustrating a process for testing a server used in edge computing, according to one embodiment.

[0010] [Figure 4] FIG. 1 illustrates an example of a computing system that may be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] Aspects of the present disclosure involve systems and methods for automating network edge computing collection and analysis of system data.

[0012] As the amount of data moving between client devices and network clouds increases, edge computing can enable improved scalability and efficiency of data delivery by directing client devices to smaller cloud environments. In a distributed environment, applications may reside at the customer premises edge, providing a shorter, more direct path from client devices to the edge cloud than to a public cloud, potentially improving latency and overall efficiency. To facilitate such edge computing, customer premises equipment may use edge gateway devices that can provide network routing and security services, data filtering, and application hosting, as well as connectivity between on-premise applications and the edge cloud. Edge clouds may provide compute and storage services, such as bare metal, network storage, and virtualization services (e.g., private clouds and virtual machines). Deploying bare metal servers in an edge cloud may be referred to as bare metal as a service.

[0013] An edge cloud may enable bare metal servers with customized configurations for direct connection to the network backbone (e.g., direct connection to the Internet without the need for a firewall). When a user adds a bare metal server, the user may be able to choose the settings and configurations to implement, which may be undesirable for the hardware and software.

[0014] Once a server has been operating in a network environment for some time, some technologies enable the capture and analysis of performance data to monitor server performance. Existing technologies select specific performance data to monitor and test. However, existing technologies preclude analysis of specific performance data and require selection of which data to monitor and which data not to monitor. Improved performance monitoring by edge computing devices may not limit the server performance data that is monitored. However, when there may be thousands of files and settings to test, the edge computing device may not identify which data to monitor unless instructed to analyze specific data and ignore other data.

[0015] In one or more embodiments, an edge computing device may monitor data for bare-metal servers configured via an edge cloud as the servers are being built (e.g., immediately after being built). The edge computing device may use trained artificial intelligence to know which performance data to analyze and which performance criteria to use for a given device. For example, the artificial intelligence may include a neural network that receives settings, firmware versions, configurations, performance tests, etc. as input. Training data for the neural network may include settings, configurations, performance, etc. labeled as good or bad, and weights for performance features. The neural network may generate as output a confidence level that the server is a good server based on whether the selected settings, configurations, versions, etc. indicate good or bad performance. Although a user may directly add a server to a network backbone (e.g., a backbone router directly connected to the Internet) with selected settings and configurations that may be available for selective implementation, this may not be desirable in actual operation. Therefore, the neural network may need to determine which data to monitor and which criteria to use to evaluate good performance, and may determine a confidence level for the newly created server that the newly created server will perform satisfactorily. If the confidence level is low (e.g., below a threshold) for a server, the edge computing device may follow up with the user and notify them that the newly created server may not perform satisfactorily due to the specific settings or configurations selected for the server. The edge computing device may also determine whether and when the user returns the server as an indication of poor performance.

[0016] In one or more embodiments, the edge computing device does not need to ask the user what data to monitor for the newly built server, but the edge computing device may allow manual testing to be performed.

[0017] When an operating system is deployed (e.g., on a newly built server), many settings may change. For example, multiple configurations of the operating system may be available for implementation in an edge computing environment. In one or more embodiments, the edge computing device may determine to what extent and why the configuration changes relative to a baseline (e.g., an existing topology and template). The edge computing device may perform tests on existing equipment and / or a new topology to detect the configuration changes and their root causes.

[0018] In one or more embodiments, the collection and posting of baseline node data may be automated. To ingest data from newly built servers directly connected to backbone routers, the servers may run scripts to send data to a central collection point. The neural network may compare the ingested data with old / gold baseline data to look for drift. If the neural network detects a change between systems, it may look for correlations across other systems (e.g., to determine if the change was expected and / or to identify the cause of the change). If the neural network detects drift, it may generate an alarm about the unexpected change or when the cause of the change cannot be identified. The neural network may implement the data change as a new gold baseline for evaluation purposes in some situations.

[0019] In one or more embodiments, when a user wants to provision a bare metal server through an edge cloud, the user may select one of multiple available operating systems, settings, and configurations tailored to the user's needs. However, the selected operating system, settings, and configuration may not perform well with certain hardware, software, or firmware. The neural network may predict, using a reliability score, whether the newly provisioned server will perform well based on the selected operating system, settings, and configuration.

[0020] The above description is for purposes of illustration and not intended to be limiting. Numerous other examples, configurations, processes, etc. may exist, some of which are described in more detail below. Exemplary embodiments will now be described with reference to the accompanying drawings.

[0021] FIG. 1 illustrates an exemplary network environment 100 for edge computing, according to one embodiment.

[0022] Referring to FIG. 1 , a network environment 100 may include client devices 102 at a customer premises edge 104 connecting to an edge cloud 106. The edge cloud 106 may connect the client devices 102 to a public cloud 110 (e.g., the Internet, a cloud provider, etc.) using a core network 108. The edge cloud may include artificial intelligence (AI) 112 for evaluating the settings, configuration, and performance of bare metal servers 114 provisioned as bare metal-as-a-service servers using the edge cloud 106. Generally, the edge cloud 106 provides an example of an edge site of a network or a collection of multiple networks where computing services may be provided to customers (e.g., client devices 102) connected to or otherwise in communication with the edge cloud 106. By providing the edge cloud 106, computing services may be provided to customers with lower latency than if the computing environment for the computing services were contained deeper in the network or further away from the requesting customer.

[0023] To provision one of the bare metal servers 114, a user may provide the server's name, select the server's operating system, select the operating system version, select the server's physical location, select the required server size (e.g., configuration), select the CPU, number of cores, and memory, add the server's Internet Protocol address, and select the server's network. As a result, the server is automatically provisioned for the user (as opposed to the server being physically sent to the user to set up connections and configuration). Exemplary server configurations may include: 4 cores: E3 / 16GB RAM / 2x1TB 7200 RAID1 (0.91TB usable); 12 cores: E5 / 64GB RAM / 4x2TB 7200 RAID5 (5.46TB usable); 20 cores: E5 / 128GB RAM / 6x2TB 7200 RAID5 (9.09TB usable); and others, depending on the location / datacenter.

[0024] In one or more embodiments, the AI ​​112 may receive all settings, configurations, and performance data for the bare metal server 114 and may be trained to predict whether the bare metal server 114 will meet performance criteria. For example, not all of the network settings selected for the bare metal server 114 may function satisfactorily with the selected hardware, software, and / or firmware of the bare metal server 114.

[0025] FIG. 2 is a schematic diagram of the artificial intelligence 112 of FIG. 1 used to test servers used in edge computing, according to one embodiment.

[0026] 2, the artificial intelligence 112 may receive as inputs settings 202, firmware versions 204, configurations 206, and (optionally) performance data 208 from the bare metal server 114 of FIG. 1 for analysis of the bare metal server 114. The artificial intelligence 112 may be trained using training data 210, which may include settings and performance data labeled as good or bad, so that the artificial intelligence 112 (e.g., a neural network) may recognize whether the inputs indicate strong or weak performance of the bare metal server 114.

[0027] In one or more embodiments, training data 210 may be generated by testing other devices and topologies to determine which combinations of settings and configurations for hardware and software perform well and which do not. The input received from bare metal server 114 may include all settings, configurations, and performance data (e.g., rather than a subset of the data that artificial intelligence 112 may require for analysis against pre-established criteria). Artificial intelligence 112 may learn which criteria (e.g., a subset of inputs) to analyze and which weights to apply to the inputs (e.g., indicating which inputs are more or less likely to indicate strong or poor performance).

[0028] In one or more embodiments, based on the input and the training data 210, the artificial intelligence 112 may generate a reliability score 212 for the bare metal server for which the input is analyzed. The reliability score 212 may indicate the probability that the bare metal server will function satisfactorily. If the reliability score 212 is below a score threshold, the edge cloud 106 may notify a user of the poorly performing bare metal server and / or disable or modify settings or configurations identified as causing the poor performance. The artificial intelligence 112 may compare the input to expected criteria (e.g., thresholds) and may detect drift. The drift may be expected (e.g., based on similar performance of other devices / topologies using the same settings / configurations) or unexpected. The edge cloud 106 may notify a user of the drifting bare metal server and whether the drift is unexpected or expected.

[0029] FIG. 3 is a flow chart illustrating a process 300 for testing a server used in edge computing, according to one embodiment.

[0030] In block 302, a device (or system, e.g., edge cloud 106 of FIG. 1 ) may detect that a server (e.g., bare metal server 114 of FIG. 1 ) has been provisioned to access the internet and / or other resources (e.g., cloud-based resources) using the device's backbone router (e.g., of core network 108 of FIG. 1 ). Server provisioning may include selecting a network, operating system, operating system version, hardware, and other settings and configurations for deploying the server using them.

[0031] In block 304, the device may provide a neural network (e.g., artificial intelligence 112 of FIG. 1) to analyze the provisioned server data and detect whether the server is performing satisfactorily (e.g., based on learned criteria and training data).

[0032] In block 306, the device may input the server settings and configuration data into the neural network.

[0033] In block 308, the device may use a neural network to generate a reliability score for the server based on the training data and input. The neural network may learn which criteria to analyze, how much weight to give settings and configurations in the analysis, and whether the settings and configuration data are likely to result in strong or poor performance (e.g., based on comparison with learned criteria thresholds and training data that represent settings, configurations, hardware, and software combinations that have been tested for performance). The reliability score may indicate the probability that the server will perform well.

[0034] At block 310, optionally, if the reliability score is below a threshold score and / or performance drift (eg, from an expected performance standard), the device may indicate an alarm to a user of the server.

[0035] At block 312, optionally, the device may continue to use the neural network to learn and update the reliability score of the server implementation and its performance and / or its criteria used to generate the reliability score based on human verification.

[0036] It is understood that the above description is intended to be illustrative and not restrictive.

[0037] FIG. 4 is a block diagram illustrating an example of a computing device or computer system 400 that may be used to implement embodiments of the network components disclosed above. For example, the computing system 400 of FIG. 4 may represent at least a portion of the above-described network environment 100 shown in FIG. 1. The computer system (system) includes one or more processors 402-406, one or more edge computing devices 409 (e.g., of the edge cloud 106 of FIG. 1), and a hypervisor 411 (e.g., for instantiating and running virtual machines and bare-metal servers, such as virtual network functions). The processors 402-406 may include one or more internal-level caches (not shown) and a bus controller 422 or bus interface unit (BUI) for directing interaction with a processor bus 412. The processor bus 412, also known as a host bus or front-side bus, may be used to couple the processors 402-406 to a system interface 424. The system interface 424 may be connected to the processor bus 412 to interface other components of the system 400 with the processor bus 412. For example, system interface 424 may include a memory controller 418 for interfacing main memory 416 with processor bus 412. Main memory 416 typically includes one or more memory cards and control circuitry (not shown). System interface 424 may also include an input / output (I / O) interface 420 for interfacing one or more I / O bridges 425 or I / O devices with processor bus 412. One or more I / O controllers and / or I / O devices, such as I / O controller 428 and I / O device 430, may be connected to I / O bus 426, as shown.

[0038] I / O devices 430 may also include input devices (not shown), such as an alphanumeric input device, which includes alphanumeric and other keys for communicating information and / or command selections to processors 402-406. Another type of user input device includes a cursor control, such as a mouse, trackball, or cursor direction keys for communicating directional information and command selections to processors 402-406 and for controlling cursor movement on a display device.

[0039] The system 400 may include a dynamic storage device referred to as main memory 416, or random access memory (RAM), or other computer-readable device coupled to the processor bus 412 for storing information and instructions to be executed by the processors 402-406. The main memory 416 may also be used to store temporary variables or other intermediate information during execution of instructions by the processors 402-406. The system 400 may also include read-only memory (ROM) and / or other static storage devices coupled to the processor bus 412 for storing static information and instructions for the processors 402-406. However, the system outlined in FIG. 4 is only one possible example of a computer system that may employ or be configured in accordance with aspects of the present disclosure.

[0040] According to one embodiment, the above techniques may be performed by computer system 400 in response to processor 404 executing one or more sequences of one or more instructions contained in main memory 416. These instructions may be read into main memory 416 from another machine-readable medium, such as a storage device. Execution of the sequences of instructions contained in main memory 416 may cause processors 402-406 to perform the process steps described herein. In alternative embodiments, circuitry may be used in place of or in combination with software instructions. Thus, embodiments of the present disclosure may include both hardware and software components.

[0041] Machine-readable media include any mechanism for storing or transmitting information in a form (e.g., software, processing application) readable by a machine (e.g., a computer). Such media may take the form of non-volatile and volatile media, but may include, without limitation, removable data storage media, non-removable data storage media, and / or external storage devices made available via wired or wireless network architectures with such computer program products, including one or more database management products, web server products, application server products, and / or other additional software components. Examples of removable data storage media include compact disc read-only memories (CD-ROMs), digital versatile disc read-only memories (DVD-ROMs), magneto-optical disks, flash drives, etc. Examples of non-removable data storage media include internal magnetic hard disks and solid-state drives (SSDs), etc. The one or more memory devices 406 may include volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM), etc.) and / or non-volatile memory (e.g., read-only memory (ROM), flash memory, etc.).

[0042] A computer program product including mechanisms for implementing systems and methods according to the presently described technology may reside in main memory 416, which may be referred to as a machine-readable medium. It will be understood that a machine-readable medium may include any tangible, non-transitory medium that can store or encode instructions for performing any one or more of the operations of the present disclosure for execution by a machine, or that can store or encode data structures and / or modules utilized by or associated with such instructions. A machine-readable medium may include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more executable instructions or data structures.

[0043] Embodiments of the present disclosure include various steps described herein that may be performed by hardware components or embodied in machine-executable instructions that can be used to cause a general-purpose or special-purpose processor programmed with the instructions to perform the steps. Alternatively, the steps may be performed by a combination of hardware, software, and / or firmware.

[0044] Various modifications and additions may be made to the exemplary embodiments described without departing from the scope of the present invention. For example, while the above-described embodiments refer to particular features, the scope of the present invention also includes embodiments having different combinations of features and embodiments that do not include all of the features described. Accordingly, the scope of the present invention is intended to encompass all such alternatives, modifications, and variations, together with all equivalents thereof.

Claims

1. 1. A method for testing a server provisioned in an edge computing device, comprising: detecting, by at least one processor of the edge computing device, that a server has been provisioned to access a public network cloud using a backbone router of the edge computing device; providing, by the at least one processor, a neural network for assessing the probability that the server's performance meets a performance criterion, the neural network being trained based on training data including labeled configuration data and feature weights to learn how server settings and configurations are indicative of server performance; inputting, by the at least one processor, first settings and configurations associated with the provisioning of the server as inputs to the neural network; and generating, by the at least one processor, a confidence score indicative of the probability based on the input and the training data using the neural network. A method comprising:

2. The method of claim 1 , wherein the first settings and configurations include all settings and configurations selected for the server for the provisioning of the server.

3. determining, using the neural network, the first setting and the subset of configurations to monitor for the server; The method of claim 2 further comprising:

4. The method of claim 3 , wherein determining the subset occurs without user selection of the subset.

5. determining that the confidence score is less than a threshold score; and presenting an indication to a user that the confidence score is below the threshold score. The method of claim 1 further comprising:

6. detecting drift of the configuration or the edge computing device when compared to a threshold performance criterion; determining a cause of the drift based on a comparison of the settings and configuration with an existing network topology implemented using the edge computing device. The method of claim 1 , further comprising:

7. The method of claim 6 , further comprising presenting an indication of the cause of the drift to a user.

8. updating the criteria by which the neural network generates the confidence score based on the confidence score; The method of any one of claims 1 to 5, further comprising:

9. 1. A system for testing a server provisioned in an edge computing device, comprising: at least one processor of the edge computing device coupled to a memory of the edge computing device; Equipped with wherein the at least one processor: detecting that a server is provisioned to access a public network cloud using a backbone router of the edge computing device; providing a neural network for assessing the probability that the server's performance meets a performance criterion, the neural network being trained based on training data including labeled configuration data and feature weights to learn how server settings and configurations are indicative of server performance; inputting as input to the neural network first settings and configurations associated with the provisioning of the server; and using the neural network to generate a confidence score indicative of the probability based on the input and the training data; configured to: system.

10. The system of claim 9 , wherein the first settings and configurations include all settings and configurations selected for the server for the provisioning of the server.

11. 11. The system of claim 10, wherein the at least one processor is further configured to determine, using the neural network, the first setting and the subset of configurations to monitor for the server.

12. The system of claim 11 , wherein determining the subset occurs without user selection of the subset.

13. The at least one processor further comprises: determining that the confidence score is less than a threshold score; and presenting an indication to a user that the confidence score is below the threshold score. configured to: The system of claim 9.

14. The at least one processor further comprises: detecting drift of the configuration or the edge computing device when compared to a threshold performance standard; determining a cause of the drift based on a comparison of the settings and configuration with an existing network topology implemented using the edge computing device; 14. The system of claim 9, configured to:

15. The at least one processor further comprises: presenting to a user an indication of the cause of the drift. configured to: The system of claim 14.

16. The at least one processor further comprises: updating the criteria by which the neural network generates the reliability score based on the reliability score; 14. The system of claim 9, configured to:

17. 1. A device for testing a server provisioned in an edge computing device, comprising: at least one processor coupled to a memory, the at least one processor comprising: detecting that a server is provisioned to access a public network cloud using a backbone router of the edge computing device; providing a neural network for assessing the probability that the server's performance meets a performance criterion, the neural network being trained based on training data including labeled configuration data and feature weights to learn how server settings and configurations are indicative of server performance; inputting as input to the neural network first settings and configurations associated with the provisioning of the server; and using the neural network to generate a confidence score indicative of the probability based on the input and the training data; configured to: device.

18. The device of claim 17 , wherein the first settings and configurations include all settings and configurations selected for the server for the provisioning of the server.

19. 20. The device of claim 18, wherein the at least one processor is further configured to determine, with the neural network, the first setting and the subset of configurations to monitor for the server.

20. The device of claim 19 , wherein determining the subset occurs without user selection of the subset.

Images