Adaptive log data level in computing systems

Abstract

The present disclosure relates to adaptive log data levels in computing systems. Embodiments are disclosed for improving remote diagnosis of computer systems. Some embodiments obtain operational parameter values and log data from a plurality of network devices and provide the operational parameter values and log data to a machine learning model. The model is trained to identify a root cause of a degradation of the computer system based on the operational parameter values and log data and provide a recommendation for log data level settings for the network devices. If the model identifies a root cause of the degradation with sufficient confidence, a remedial action is identified and applied to the computer system. If the confidence is insufficient, the log data level settings for the network devices are modified based on the model's recommendation. The process is performed in an iterative manner, for example, such that the model receives log data based on its recommended log data levels until a root cause is identified with sufficient confidence.

Description

[0001] Cross-reference of related applications

[0002] This application claims the benefit of U.S. Application No. 17 / 200,229, filed March 12, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application generally involves using machine learning, statistical analysis, and information theory to manage information collected from computer systems. Background Technology

[0004] As the computer industry matures, solutions are sold partly based on maintenance costs. With ubiquitous digital connectivity, the ability to remotely support and manage products from a centralized location has become possible. Centralized support and management allow for the maintenance of capability centers that are impossible under a distributed product support model. The specialized skills provided by these capability centers, built upon a large number of installed products, are used to improve support efficiency and reduce costs. However, the challenges of remotely supporting and managing installed products remain. Therefore, there is a need to improve remote management and support methods. Attached Figure Description

[0005] This disclosure is illustrated by way of example and not limitation in the accompanying drawings, wherein similar reference numerals indicate similar elements, and wherein

[0006] Figure 1 This is an overview diagram of an example system that implements one or more embodiments of the disclosed embodiments.

[0007] Figure 2 This is a block diagram of an example access point implementing one or more embodiments of the disclosed embodiments.

[0008] Figure 3 An example network management device is shown.

[0009] Figure 4 An example network node is shown.

[0010] Figure 5 An example communication device is shown.

[0011] Figure 6A The diagram illustrates an example log data level management table.

[0012] Figure 6B This is an overview diagram of a network implementing one or more of the disclosed embodiments.

[0013] Figure 7 This is a flowchart of example method 700 for collecting log data rate information.

[0014] Figure 8 This is a flowchart of a method for maintaining the rate at which log data is recorded.

[0015] Figure 9 It is a flowchart of a process for modifying the log data level of one or more components or devices.

[0016] Figure 10 An example data flow of a machine learning model implemented in one or more of the disclosed embodiments is illustrated.

[0017] Figure 11 An example training data stream implemented in one or more of the disclosed embodiments is illustrated.

[0018] Figure 12 Example machine learning modules are shown according to some examples of this disclosure.

[0019] Figure 13 This is a flowchart of an example method for modifying the log data level of one or more components or devices. Detailed Implementation

[0020] The following description and accompanying drawings fully illustrate specific embodiments to enable those skilled in the art to practice them. Other embodiments may be combined with structural changes, logical changes, electrical changes, process changes, and other modifications. Parts and features of some embodiments may be included in or replace parts and features of other embodiments.

[0021] Many computer systems collect information to facilitate the diagnosis of various anomalies that may occur during operation. Recently, solutions using artificial intelligence (AI), such as machine learning models, have been developed to diagnose the operation of computer systems. These AI-based solutions ingest relatively large amounts of data to diagnose problems and, in some cases, identify the root cause.

[0022] The increasing success of Network Monitoring as a Service (MaaS) products has further fueled the trend toward larger datasets. When providing Network Monitoring as a Service, network-related information is collected from the customer's site. In some cases, this network information is collected from one or more of the following: end-user devices (such as smartphones, laptops, desktop computers, printers, and other devices), network components (such as access points, switches, routers, or other network components), or server computers at the customer's site (such as mail servers or file servers).

[0023] Various analytics tools, some of which are AI-based, are used to analyze data from these devices to identify operational degradation. Some embodiments measure Service Level Experience (SLE) and compare the measured SLE to predefined metrics to determine whether a computer system attached to the network or any network component is operating correctly. In some cases, appropriate actions can be identified to mitigate anomalous operation, and in others, the system can be returned to normal operation without human intervention.

[0024] As monitored systems become more complex, the amount of data collected and analyzed to identify root causes and provide corrective actions is growing exponentially. This increased data volume not only challenges network bandwidth and throughput but also data storage and computing power. In some cases, machine learning models examine data generated over relatively extended periods to appropriately identify anomalous behavior, and in others, to identify root causes and potentially invoke corrective actions. Even in an era of relatively abundant and inexpensive stable storage, storing data over such extended periods can still consume significant amounts of storage.

[0025] The disclosed embodiments provide autonomous control over logging activity levels for multiple computing components. The logging data levels are configured to provide sufficient data to diagnose computer system operations and, in some cases, to identify remedial actions. The logging data levels are also configured to minimize the impact of logging activity on the system.

[0026] Some embodiments identify a first portion of a first network system that is experiencing anomalous operation (e.g., experiencing SLE degradation). A second portion of a second network system that is operating nominally is identified (the first and second network systems may be equivalent or different in some embodiments). Then, some embodiments compare the logging data of the first and second portions. In some embodiments, the log data levels between the two systems are adjusted to produce similar log data streams (e.g., the log data levels of similar components are set to equivalent values). In some cases, a comparison of similar log data streams is performed to isolate the cause of the anomalous operation of the first network system.

[0027] In some embodiments, the data collection rate is controlled by a predetermined threshold. The rate and / or total amount of data collected is monitored, and if the monitored information exceeds a predetermined threshold, some embodiments of the disclosed embodiments automatically reduce the log data level of one or more components to restore the rate and / or total amount to within a predefined range and / or below a predefined threshold.

[0028] Some embodiments implement a machine learning model system to provide a computer system with identifications of causes of degradation. The machine learning model system also provides probabilities associated with the causes. These probabilities indicate the likelihood that the identified cause is actually causing the degradation. The machine learning model also provides recommendations for log data level settings for one or more devices and / or components of those devices. In some embodiments of these embodiments, if the probability associated with a cause identified by the machine learning model system is below a predefined threshold, the log data level settings of one or more components and / or devices of the computer system are modified according to the recommendations from the machine learning model system. This process is then performed iteratively, wherein the machine learning model system then analyzes the log data generated as a result of the recommended log data level settings and generates new identifications of possible causes and probabilities associated with those causes. Within the range of identified high-probability causes, corresponding remedial actions are determined, and these remedial actions are then executed to resolve the degradation.

[0029] Therefore, the disclosed embodiments improve system operation by increasing the effectiveness of remote diagnostic capabilities and reducing their cost. The disclosed embodiments ensure that information collection is targeted at network neighborhoods experiencing SLE degradation and is set to the minimum level necessary for diagnosing the problems experienced.

[0030] Figure 1This is an overview diagram of an example system 100 implementing one or more embodiments of the disclosed embodiments. Example system 100 includes multiple access points (AP1 142, ..., APX144, AP1'150, ..., APX'152), where each access point may be a wireless access point, router, switch, or any other device capable of providing network access; multiple authentication, authorization, and accounting (AAA) servers (AAA server 110 is shown by example); multiple dynamic host configuration protocol (DHCP) servers (only one DHCP server 116 is shown by example); multiple domain name system (DNS) servers (only one DNS server 122 is shown by example); multiple web servers (only one web server 128 is shown by example); and a network management system (NMS) 136 (e.g., an Automatic Log Data Level Setting (ALLS) system (server)), coupled together via a network 134 (e.g., the Internet and / or an enterprise intranet). In some embodiments, network 134 includes one or more routers 185 and / or one or more switches 180. Network communication links 143, 145, 171, and 173 couple access points (AP1 142, APX 144, AP1'150, APX'152) to network 134, respectively. Network communication link 111 couples an AAA server (AAA server 110 is shown by example) to network 134. Network communication link 117 couples a DHCP server (e.g., DHCP server 116 is shown by example) to network 134. Network communication link 133 couples a Virtual Network Assistant (VNA) server (only one VNA server 132 is shown by example) to network 134. Network communication link 123 couples a DNS server (e.g., DNS server 122 is shown by example) to network 134. Network communication link 129 couples a web server (e.g., web server 128) to network 134. System 100 also includes multiple user equipment devices (UE1 138, ..., UEZ 140, UE1'146, ..., UEZ'148). User equipment is any wired, wireless, or optical device that provides network access to communication devices or automated devices (such as IoT devices) used by users (such as people). Some of the UEs (138, 140, 146, 148) are wireless devices that move within system 100.

[0031] Within system 100, in at least some embodiments, access points are located at multiple different customer sites. Customer site 1102 (e.g., a shopping mall) includes access points (AP 1 142, ..., AP X 144). Customer site 2 104 (e.g., a stadium) includes access points (AP 1 150, ..., AP X 152). As illustrated by example... Figure 1 As shown, UEs (UE 1 138, ..., UE Z 140) are currently located at customer site 1 102; UEs (UE 1' 146, ..., UE Z' 148) are currently located at customer site 2 104. In some embodiments, each of the server, router, switch, AP, UE, NMS, and other servers includes a system log or error log module, wherein each of these devices records the status of the device, including normal operating status and error conditions.

[0032] Figure 2 This is a block diagram of an example access point configured to implement one or more embodiments of the disclosed embodiments. In some embodiments, Figure 2 The access point 200 shown implements the above-mentioned... Figure 1 The access points discussed are any one of AP 1142, ..., APX 144, AP 1'150, ..., APX'152.

[0033] Access point 200 includes a wired interface 230, wireless interfaces 236 and 242, a hardware processor 206 (e.g., a CPU), a memory 212, and components 208 (e.g., components of hardware components, such as circuitry), which are coupled together via a bus 209 through which various components exchange data. The wired interface 230 includes a receiver 232 and a transmitter 234. The wired interface 230 couples access point 200 to... Figure 1The network and / or network 134 (e.g., the Internet). A first wireless interface 236 (e.g., a Wi-Fi interface and / or an 802.11 interface) includes a receiver 238 coupled to a receiving antenna 239, through which an access point receives wireless signals from a communication device (e.g., a wireless terminal); and a transmitter 240 coupled to a transmitting antenna 241, through which the access point transmits wireless signals to the communication device (e.g., a wireless terminal). A second wireless interface 242 (e.g., a Bluetooth interface) includes a receiver 244 coupled to a receiving antenna 245, through which an access point receives wireless signals from a communication device (e.g., a wireless terminal); and a transmitter 246 coupled to a transmitting antenna 247, through which the access point transmits wireless signals to the communication device (e.g., a wireless terminal).

[0034] Memory 212 includes routines 228 and data / information 216. Routines 228 include components of component 218 (e.g., components of software components) and application programming interface (API) 220. Data / information 216 includes configuration information 222, a device status log including error events and normal events captured as messages in system logs or error logs 224, and log data level information 226 storing log data level settings information for the storage device. In some embodiments, the stored log data level settings information is based on instructions from a network management system.

[0035] Figure 3 An example network management system 300 is illustrated. In one or more of the disclosed embodiments, the network management system controls the log data level settings of one or more managed devices and / or components. In some embodiments, the network management system 300 is a network management node, such as a network management server, or a network management automated defect reporting and parsing server. In some embodiments, Figure 3 The network management system 300 is Figure 1 The network management system (NMS) 136. In some embodiments, the network management system 300 is an access point or part of an access point, such as... Figure 1 By way of any of the access points or devices shown in the examples.

[0036] The network management system 300 includes a communication interface 330, a hardware processor 306, output devices 308 (e.g., a display, printer, etc.), input devices 310 (e.g., a keyboard, keypad, touchscreen, mouse, etc.), a memory 312, and components 340 (e.g., components of hardware components, such as circuit components). These components 340 are coupled together via a bus 309, through which, in at least some embodiments, the various elements exchange data and information. In some embodiments, the communication interface 330 includes an Ethernet interface. The communication interface 330 couples the network management system 300 to a network and / or the Internet. The communication interface 330 includes a receiver 332 through which the network management system 300 can receive data and information, such as service-related information, such as messages logged in system logs or error logs from various devices such as AAA servers, DHCP servers, web servers, routers, switches, and transmitters 334. The network management system 300 can send data and information through these devices, such as configuration information and instructions, such as instructions to access points, routers, switches, or any other server or client attached to the network, to restart, change operating parameters, download and install another software version, etc.

[0037] In the example implementation, the network management server can instruct devices associated with the network or devices in a specific neighborhood of the network to increase or decrease the log data level of a specific device or a specific network neighborhood.

[0038] Memory 312 includes routines 328 and data / information 317. Routines 328 include components of part 318 (e.g., components of software parts) and application programming interface (API) 320. Data / information 317 includes configuration information 322, and captured messages in system log 324 including message and / or message fields and timing information (e.g., the time when message logs are recorded in system logs or error logs). Log data level setting 326 defines the log data level for each device managed by example NMS 300. Depending on the specific implementation, log data level setting 326 indicates the amount of data collected as a result of a specific log data level set on a specific device. Data information 317 also includes status and measurement information from various devices 319, including RSSI information from mobile clients and user equipment (UE), data rates implemented on various links, mobility time, and information from devices such as Figure 1 Other SLE-related measurements of various devices associated with networks such as network 134.

[0039] An automatic log data level setting routine, which is part of SW component 318 according to a specific implementation, uses log data level setting 326 to ensure that the total data generated by the entire system log does not exceed a predetermined threshold.

[0040] Figure 4 An example network node 400 is shown. In one or more of the disclosed embodiments, network node 400 implements a device or server (e.g., a router, switch, AAA server, DHCP server, DNS server, VNA, web server, etc.) or network device (such as router 185, switch 180, etc.) attached to network 134. Figure 1 AAA server 110, DHCP server 116, DNS server 122 or web server 128, router 185, or Figure 1 One or more of the switches 180 implement about Figure 4 The network node 400 describes the structure of the device. The network node 400 (e.g., a server) includes a communication interface 402 (e.g., an Ethernet interface), a hardware processor 406, an output device 408 (e.g., a display, printer, etc.), an input device 410 (e.g., a keyboard, keypad, touchscreen, mouse, etc.), a memory 412, and components 416 (e.g., components of hardware modules, such as circuitry), which are coupled together via a bus 409. In some embodiments, the various components exchange data and information via the bus 409. The communication interface 402 couples the network node 400 to a network and / or the Internet. Although only one interface is shown by example, those skilled in the art will recognize that some embodiments of routers and / or switches have multiple communication interfaces. The communication interface 402 includes a receiver 420 through which the network node 400 (e.g., the server) can receive data and information. In some embodiments, the data and information include operational information, such as registration requests, AAA services, DHCP requests, Simple Notification Service (SNS) lookups, or web page requests. The communication interface 402 also includes a transmitter 422, through which the network node 400 (e.g., a server) can send data and information, such as configuration information, authentication information, web page data, etc.

[0041] Memory 412 includes routines 428, data / information 430, and log data level settings 436. Routines 428 include components of component 432 (e.g., components of software components) and information 430, as well as API 434. Information 430 includes system logs and / or error logs. Routines 428 also include one or more functions that implement the device's logging capabilities. The logging capabilities are customized to correspond to the log data level information stored in the log data level settings 436. In some embodiments, the log data level settings 436 are controlled by a network management system. For example, in some embodiments, the network management system sets the log data level in network node 400 by sending a network message indicating the log data level to network node 400.

[0042] Figure 5 An example communication device 500 is illustrated. In one or more of the disclosed embodiments, the communication device 500 implements a user equipment (UE) device, such as UE 1 138, ..., UE Z 140, ..., UE Z140, UE 1'146, ..., or UE Z'148. The example communication device 500 includes a wired interface 502, a wireless interface 504, a hardware processor 506 (e.g., a CPU), an output device 508, an input device 510, a memory 512, and components 516 (e.g., components of hardware modules, such as circuitry) coupled together via a bus 509 through which various components exchange data. The wired interface 502 includes a receiver 520 and a transmitter 522. The wired interface 502 couples the communication device 500 to a network (e.g., network 134, such as the Internet).

[0043] Wireless interface 504 includes cellular interface 524, a first additional wireless interface 526 (e.g., an 802.11 WiFi interface), and a second additional wireless interface 527 (e.g., a Bluetooth interface). Cellular interface 524 includes receiver 532 coupled to receiver antenna 533, through which communication device 500 receives wireless signals from access points (e.g., such as any one or more of the following: AP 1 142, ..., APX 144, AP 1'150, ..., APX'152); and transmitter 534 coupled to transmission antenna 535, through which communication device 500 transmits wireless signals to APs (e.g., AP 1 142, ..., APX 144, AP 1'150, ..., APX'152). The first additional wireless interface 526 includes a receiver 536 coupled to a receiving antenna 537, through which the communication device 500 receives wireless signals from a communication device (e.g., an access point); and a transmitter 538 coupled to a transmitting antenna 539, through which the communication device 500 transmits wireless signals to the communication device. The second additional wireless interface 527 includes a receiver 540 coupled to a receiving antenna 541, through which the communication device 500 receives wireless signals from the communication device; and a transmitter 542 coupled to a transmitting antenna 543, through which the communication device 500 transmits wireless signals to the communication device.

[0044] Memory 512 includes routines 528, data / information 517, and log data level settings 518. Routines 528 include components of component 515 (e.g., components of software components). In at least some embodiments, data / information 517 includes configuration information and any additional information required for the normal operation of UE 500. In at least some embodiments, data information includes system logs or error logs. Log data level settings 518 define the device's log data level. In some embodiments, log data level settings 518 are set based on instructions from a network management system and, in some embodiments, instructions from a log data level setting server, which is included as part of the network management system in some embodiments.

[0045] Figure 6A The illustration depicts an example log data level management data 600 according to one or more embodiments. Although Figure 6A The log data level management data 600 is presented in tabular format, but the disclosed embodiments are not limited to storing the log data level management data 600 in tabular format.

[0046] Log data level management data 600 includes a device identifier 620 in the first column and a component identifier 625 in the second column. In this context, each component identifier 625 uniquely identifies a separable component of the computer system. For example, one component identifier 625 identifies one or more of the following in various respects: a software component, a hardware component, a device, a computer, a client device, or another component. Log data level 630 is stored in a third column. In some embodiments, the level of detail or volume of log data generated by a component increases numerically with the log data level. Other embodiments reduce the level of detail or volume of log data as the log data level increases. The rate of log data generated at each log data level for each component is represented in the rightmost column of the table.

[0047] Part 602 illustrates the data rate of log recording information generated by a first component (ID C1) of the first device (ID-1) when it is instructed to operate at, for example, log data levels 1 to 9. The resulting log recording data rates 635 are illustrated as R111 to R119.

[0048] Similarly, parts 604, 606, 608, and 610 illustrate a specific component (ID C) of the j-th first device (ID-j). k The data rate of log recording information generated by this component when operating at log data levels (e.g., 1 to 9) is specified. The resulting log recording data rate 635 is illustrated as R. jkl , where j represents the device ID, k represents a specific component within device j, and l represents the log data level.

[0049] Based on an example implementation, the log recording data rate R jkl The configuration is determined experimentally and automatically. According to another example implementation, the data rate of the log data is monitored. Then, data similar to the log data level management data 600 is updated based on the rate determined by monitoring. In some embodiments, the maximum rate is stored as a portion of the log data level management data and / or the average rate.

[0050] Figure 6B This is an overview diagram of a network implementing one or more of the disclosed embodiments. Figure 6BTwo access points, AP 654A and AP 654B, are shown. Each access point communicates with multiple wireless terminals. For example, AP 654B is associated with and / or communicates with wireless terminals 652A, 652B, and 652D. AP 654A is associated with and / or communicates with wireless terminals 652C and 652E. Each of APs 654A and AP 654B also communicates with a backhaul server 656. In some embodiments, AP 654B communicates with a switch located between the backhaul server 656 and the AP. In at least some embodiments, the backhaul server 656 provides network connectivity between APs 654A and AP 654B and other devices accessible via a wired network.

[0051] AP 654A and AP 654B also communicate with the Network Management System (NMS) 658. In some embodiments, the NMS 658 communicates with the network management system (NMS) 658 described above. Figure 1 The NMS 136 discussed is similar. Therefore, in some embodiments, Figure 6B Access points and wireless terminals located at Figure 1 Similar to any one or more devices at site 102 and / or site 104.

[0052] Some embodiments provide NMS 658 with one or more of the following operational parameter values: AP 654A, AP 654B, wireless terminal 652A, wireless terminal 652B, wireless terminal 652C, wireless terminal 652D, and wireless terminal 652E. In some embodiments, the operational parameter values ​​are provided to a machine learning model to identify the cause of degradation of system 650. In some embodiments, NMS 658 also receives from the machine learning model one or more possible causes of degradation, and in some embodiments, an output of the probability associated with each of the identified possible causes. In some embodiments, NMS 658 also establishes the level or detail of data logging by one or more of the following: AP 654A, AP 654B, backhaul server 656, wireless terminal 652A, wireless terminal 652B, wireless terminal 652C, wireless terminal 652D, or wireless terminal 652E. As discussed above, in some embodiments, NMS 658 is configured to establish logging levels through one or more of these devices and / or one or more components of each device in order to provide sufficient data to the machine learning model to identify the root cause of a degradation in system operation, while also ensuring that the logging levels do not saturate or otherwise overwhelm the capabilities of system 650.

[0053] Although Figure 6AThe illustration illustrates that different log data levels typically produce different log data rates. In some embodiments, modifications to the log data level alter the set of specific operational parameters being logged. For example, a first set of parameters is logged at a first log data level, and a second set of parameters is logged at a second log data level. The second set of parameters includes the first set of parameters plus at least one additional operational parameter not included in the first set. Therefore, some embodiments store a data structure that maps the log data levels of one or more devices and / or components to sets of operational parameters logged at the corresponding data levels. In some embodiments, the data structure is provided to a machine learning model, as discussed below, to allow the machine learning model to map between log data levels of devices and / or components and which operational parameter values ​​are logged at a particular log data level.

[0054] Figure 6B It also shows Figure 6B The illustrated devices include message exchanges 660A-G between some devices. One or more message exchanges in 660A-G may consist of only one message, but in some embodiments, they may also represent one or more messages transmitted from one device to another. Message exchanges 660A-G illustrate one or more operational parameter values ​​and / or log data being directly or indirectly communicated to NMS 658. For example, message exchange 660A shows data flowing from wireless terminal 652B to AP 654B. This data may include operational parameter values ​​and / or log data generated by wireless terminal 652B. Upon receiving operational parameter values ​​and / or log data, AP 654B provides this information to NMS 658 via message exchange 660E. As discussed above, in some embodiments, NMS 658 controls... Figure 6B The illustrated devices represent the log data level or level of detail in logging for one or more devices. To this end, in some embodiments, the NMS 658 sets the log data level of a wireless terminal, such as a wireless terminal 652B, by sending the log data level to the AP 654B via message exchange 660E. The AP 654B then forwards the command to the wireless terminal 652B via message exchange 660A to set its log data level to the level commanded by the NMS 658. The NMS 658 can... Figure 1 Any of the devices illustrated receives operating parameter values ​​and / or log data, and controls them in a similar manner. Figure 6B Any of the devices shown in the diagram. Although Figure 6B Direct message communication between AP 654A, AP 654B and NMS 658 is depicted, but in some embodiments, communication between AP and NMS 658 flows through backhaul server 656.

[0055] Figure 7 This is a flowchart of an example method 700 for collecting log recording data rate information. In some embodiments, the following refers to... Figure 7 One or more of the functions discussed in method 700 are executed by a hardware processing circuitry. For example, one or more hardware memories (e.g., memory 212, memory 312, memory 412, memory 512) store instructions (e.g., routine 228, routine 328, routine 428, routine 528) that, when executed, configure the hardware processing circuitry (e.g., hardware processor 206, hardware processor 306, hardware processor 406, hardware processor 506) to perform one or more of the functions of method 700 discussed below.

[0056] Method 700 begins at start operation 705 and then moves to operation 710, where the device identifier, component identifier, and initial log data level of the identified device and component are set.

[0057] In operation 715, the logging data rate is measured. In some embodiments, measuring the logging data rate includes monitoring network traffic visible to the device performing method 700. Some embodiments employ packet or network message filters that separate logging data from other network data transmitted over the network. For example, in some embodiments, network traffic with a specific destination port or source port is considered logging data. In some other embodiments, a specific URL in the network traffic identifies the traffic as logging data. Therefore, in these embodiments, the filter filters the traffic including the specific URL and identifies the logging data in that traffic. In some embodiments, the logging data is collected via an application programming interface (API) supported by the component that generates the logging data. The API is accessed on a regular computer or, in some cases, locally, or invoked from a separate computer via a computer network. The rate is then determined based on the collected or detected logging data. According to yet another implementation, the logging data rate is determined by, for example,... Figure 1 An NMS such as the NMS136 or the NMS 658 in Figure 6 measures the log data when it receives log data messages from a specific device.

[0058] In operation 717, log records are stored in the data store.

[0059] Decision operation 720 determines whether the additional logging settings of the currently selected device and component are available for analysis. For example, some devices support different logging data levels 3, 4, 5, 6, 7, or other levels, where each level generates a different amount and / or rate of data. Therefore, decision operation 720 determines whether all logging data levels supported by the device have been evaluated by method 700 for a particular device and component. If there are more logging data levels to be evaluated, method 700 moves from decision operation 720 to operation 725, which adds or otherwise selects additional logging data levels that have not yet been evaluated by method 700. Then, method 700 moves from operation 725 to operation 715, and measures and stores the data rate associated with the new logging settings. If there are no other logging data levels to be evaluated for the selected device and component, method 700 moves to decision operation 730, which determines whether the logging data rate of the additional components of the currently selected device still needs to be evaluated. If additional attachments are indeed retained, method 700 moves from decision operation 730 to operation 735, where the additional components of the existing selected device are identified.

[0060] Then, method 700 moves from operation 735 to operation 715, and measures and stores the data rate associated with the new device component. If no other components of the currently selected device are to be evaluated, method 700 moves from decision operation 730 to decision operation 740, which evaluates whether there is an additional device whose logging data rate is to be evaluated. If an additional device exists, method 700 moves from decision operation 740 to operation 745, where the additional device is selected. Then, method 700 moves from operation 745 to operation 715, and measures and stores the data rate associated with the new device; and method 700 continues iterating. Otherwise, if no remaining devices are to be evaluated, method 700 moves from decision operation 740 to termination operation 750.

[0061] Figure 8 This is a flowchart of an example method for maintaining the rate at which log data is recorded. In some embodiments, the following describes... Figure 8 One or more of the functions discussed in method 800 are executed by a hardware processing circuitry. For example, one or more hardware memories (e.g., memory 212, memory 312, memory 412, memory 512) store instructions (e.g., routine 228, routine 328, routine 428, routine 528) that, when executed, configure the hardware processing circuitry (e.g., hardware processor 206, hardware processor 306, hardware processor 406, hardware processor 506) to perform one or more of the functions of method 800 discussed below.

[0062] Method 800 begins at start operation 805 and then moves to operation 810. In operation 810, a command is received to change the log data level of a component. In some embodiments, the command instructs the device's log data level to be set to a specific level (e.g., some devices do not support component log data level settings). In some embodiments, operation 810 includes modifying the log data level based on the command. For example, in some embodiments, the device performing method 800 calls the device's API to change the device's log data level (or the component with the device's log data level).

[0063] In operation 815, log data is received at the new data rate.

[0064] Operation 820 measures the data rate of the log data received in Operation 815.

[0065] In operation 825, a logging data rate is created for a given log data level setting. In one example implementation, the measurement is the maximum data rate of the logging data rate received in operation 815. In some other embodiments, a weighted average of the rates measured in the past and the most recent within the logging data received in operation 815 is determined. For example, some embodiments determine the rate according to Equation 1 below:

[0066] Measurement of data rate (t) =

[0067] Equation 1: α*Measured data rate (t) + (1-α)*Stored data rate (t-1)

[0068] in

[0069] α is a pre-determined filtering parameter (0 < α < 1).

[0070] In various embodiments, other measurements of data rate are employed, such as linear and nonlinear combinations of previously determined and recently determined data rates.

[0071] In operation 830, the determined log recording data rate is stored (e.g., as denoted by...). Figure 8 R jkl The data rate (such as the data rate) and the log data rate replaces the old storage's log data rate.

[0072] The stored data rate (t) = Equation 2 for measuring the data rate (t)

[0073] After operation 830, method 800 moves to end operation 850.

[0074] Figure 9This is a flowchart illustrating an example method for modifying the log data level of one or more components or devices. In some embodiments, the following describes... Figure 9 One or more of the functions discussed in Method 900 are executed by a hardware processing circuitry. For example, one or more hardware memories (e.g., memory 212, memory 312, memory 412, memory 512) store instructions (e.g., routine 228, routine 328, routine 428, routine 528) that, when executed, configure the hardware processing circuitry (e.g., hardware processor 206, hardware processor 306, hardware processor 406, hardware processor 506) to perform one or more of the functions of Method 900 discussed below.

[0075] Method 900 begins at start operation 905 and then proceeds to operation 910. In operation 910, one or more metrics are determined based on the operation of the computer system. In some embodiments, the metrics represent the service level experience provided by the computer system. For example, in some embodiments, the metrics include one or more metrics related to throughput measurement, network jitter measurement, signal strength measurement (e.g., Received Signal Strength Indication (RSSI)), mobility measurement, response time measurement, latency measurement, and measurements related to packet error, count of dropped packets, dropped connections, number of active users, hardware processor utilization, memory, I / O bandwidth, or other computer system performance.

[0076] Decision operation 915 determines whether the service level experience of the computer system has been degraded. For example, in some embodiments of decision operation 915, the metrics determined in operation 910 are evaluated against one or more criteria. For example, in some embodiments, the metrics are evaluated against one or more thresholds or ranges of values ​​to determine whether the computer system is experiencing a degradation in service level experience.

[0077] In Operation 920, the root cause of the degradation is determined. Some embodiments use standalone or third-party tools, such as Juniper Network Virtual Network Analyst (VNA), to determine the source of the degradation. In some embodiments, log data is fed to a machine learning model, and the machine learning model identifies the root cause of the degradation.

[0078] Decision operation 922 evaluates whether the confidence level associated with the root cause identification is higher than a threshold. For example, in some embodiments, the machine learning model of operation 920 outputs a relevant confidence level or probability associated with identifying the root cause and an indication of the root cause of the degradation. If the confidence level or probability is lower than a predefined threshold, method 900 moves from decision operation 922 to operation 923. In operation 923, log data similar to that of a computer system is obtained and used to increase the confidence level of the root cause of the SLE degradation. Otherwise, if the confidence level or probability associated with the root cause identification is higher than the confidence threshold, method 900 moves from decision operation 922 to operation 925.

[0079] In operation 925, devices within the neighborhood of the root cause of the degradation are identified. For example, in some embodiments, if operation 920 identifies a first device as the root cause of the degradation, other devices communicating with the first device are included as part of the neighborhood. In some embodiments, other devices within a maximum physical distance are included as part of the neighborhood. In some embodiments, devices having a signal strength measurement above a threshold at the first device are included in the neighborhood.

[0080] In operation 930, the log data level for each device (and / or a component of each device) in the neighborhood is determined. In some embodiments, the log data level for each device is set based on the proximity of the device to the offending device identified in operation 920. Thus, devices closer to the offending device can be configured by ALLS to increase the level of detail in logging relative to less nearby devices.

[0081] In some embodiments of operation 930, the probability that each device in the neighborhood causes a performance degradation identified in decision operation 915 is determined. The log data level of the device or component is then set based on its corresponding probability.

[0082] In some embodiments, operation 930 establishes a priority list of devices and / or components in the neighborhood. In some embodiments, the priority of devices or components in the priority list is based on the probability that each device or component in the neighborhood will cause performance degradation. In some embodiments, the priority of devices or components in the priority list is based on the type of device (e.g., access point, wireless terminal, server, or other device type), the version number of the software running on the device, or other factors.

[0083] In Operation 935, the new aggregated log recording data rate is based on the log data level of Operation 930 and, for example, as mentioned above... Figure 6A The log data level management data discussed is estimated to be 600 similar data.

[0084] Decision operation 940 determines whether the new aggregated log recording data rate is greater than a predetermined threshold. If the new rate is lower than the threshold, method 900 moves from decision operation 940 to operation 945, where the devices and / or components of the computer system are instructed to set their log data levels according to the levels determined in operation 930. Then, method 900 moves from operation 945 to operation 910.

[0085] If the new rate determined in operation 935 exceeds a threshold, method 900 moves from decision operation 940 to operation 950, where components or devices are identified for a reduction in their log data levels. In some embodiments, consultation includes a priority list of devices and / or components in the neighborhood (e.g., the priority list determined in some embodiments of operation 930 above). The log data level reduction is then set in a manner consistent with the priority list (e.g., higher-priority devices / components receive less log data level reduction than lower-priority devices). In some embodiments, the log data level reduction is weighted based on device priority (where lower numerical priority has higher priority than higher numerical priority). Alternatively, the log data level reduction is weighted based on the inverse of the device priority. After the log data level has been appropriately reduced, method 900 moves to operation 930 and processing continues.

[0086] Figure 10 The illustration shows an example data flow of a machine learning model implemented in one or more of the disclosed embodiments. In some embodiments, the machine learning model 1018 is implemented by NMS 136, as described above regarding Figure 1 The discussion continues. In some other embodiments, the machine learning model 1018 is implemented by a computer system that is physically different from and communicates with the NMS 136.

[0087] Figure 10 Data stream 1000 illustrates a machine learning model trained to provide recommendations regarding log data levels set for one or more devices and / or components of those devices. Through the set log data levels, machine learning model 1018 can facilitate improvements in identifying the root causes of performance degradation in the monitored system. Machine learning model 1018 also provides indications of the root causes and the probabilities associated with identifying the root causes. Therefore, in some embodiments, the machine learning model is trained to customize the log data levels for one or more devices and / or components to increase the probability of identifying the root cause, while also providing the amount of log data necessary to provide a sufficiently high probability of identifying the root cause.

[0088] Figure 10A machine learning model 1018 is shown that receives historical information 1030 and tag / annotation information 1035. In some embodiments, the machine learning model 1018 also receives log recording rate data 1040. In some embodiments, the log recording rate data 1040 is related to the above description... Figure 6A The described data is similar. Log recording rate data 1040 allows machine learning model 1018 to map a specific log data level to the amount or rate at which log data is generated when those different levels are active. As discussed above, some embodiments maintain a mapping between log data levels of one or more devices and / or device components and a set of operational parameters for logging at a specific level of the log data level. In some embodiments, this information is provided to machine learning model 1018. This information allows machine learning model 1018 to form associations between specific log data levels of devices and / or device components.

[0089] Figure 10 The illustration shows a machine learning model 1018 that generates log data level recommendations, including a device identifier 1052, a component identifier 1054, and a log data level 1056. The log data level indicates the machine learning model's recommendation of log data levels that may contribute to increasing the probability associated with the root cause identified by the machine learning model. In some embodiments, NMS 136 and / or NMS 658 receive log data level recommendations from the machine learning model 1018 and then apply the recommended log data level settings by communicating with the indicated device and / or intermediate device (e.g., in some embodiments, log data level recommendations for a wireless terminal cause the NMS to communicate the log data level to an access point associated with the wireless terminal). The access point then transmits the log data level settings to the wireless terminal.

[0090] Figure 10The diagram also illustrates a machine learning model 1018 that generates cause information, including a cause indicator 1062 and a probability 1064 associated with the cause identified by the cause indicator 1062. In some embodiments, the machine learning model 1018 generates one or more pairs of cause information and associated probability 1064 (indicated by the cause indicator 1062). In some embodiments, the cause indicator 1062 indicates a device and / or a component of a device that causes degradation of a computer system monitored or otherwise analyzed by the machine learning model 1018. For example, the cause indicator 1062 indicates, for example, the IP or station address of a specific device that causes degradation. Some embodiments assign component identifiers to device components, such as wireless interfaces, CPUs, or other hardware or software components. Thus, in some embodiments, the cause indicator 1062 includes a component identifier that identifies such a device component as the root cause. For example, some embodiments provide the ability to reset specific software components without resetting the entire device or the device hardware. In some cases, if the cause indicator 1062 indicates that a specific software component of the device is a possible cause of degradation, then, as discussed below, mitigation actions to resolve the degradation include, in some embodiments, sending an instruction to the device instructing it to reset the identified software component.

[0091] The associated probability indicates the likelihood or probability that the system being analyzed by the machine learning model 1018 is undergoing degradation based on the cause. Therefore, some implementations are then able to consider multiple possible root causes, their associated probabilities, and take remedial actions associated with one or more of the highest probability root causes.

[0092] Figure 11 An example training data stream implemented in one or more of the disclosed embodiments is illustrated. Figure 11 First, the above text regarding... Figure 10 The historical information 1030 discussed includes, in some embodiments, a time series 1110 of operating parameter values ​​and a time series 1120 of log data. The time series 1110 of operating parameter values ​​includes multiple operating parameter time series, each representing a single operating parameter value over time. In at least some embodiments, the operating parameter values ​​are device-specific and therefore include an identifier of the device from which they originate. In some embodiments, the operating parameter values ​​include one or more of the following: CPU utilization, memory utilization, I / O utilization, throughput measurement, latency measurement, or other measurements of computer system performance or capacity. Figure 11 An example structure for each entry in the time series of operating parameters is shown. Figure 11The entries in the time series shown in some embodiments include a device identifier 1112 that identifies the device from which the operating parameter value originates; a component identifier 1114 that identifies a component of the identified device from which the operating parameter value originates; an operating parameter identifier 1116 that indicates the operating parameter value being provided (e.g., whether the operating parameter value represents CPU utilization or memory utilization); and the operating parameter value itself 1118. A timestamp 1119 identifying the time when the data was acquired is also included.

[0093] The time series of log data 1120 includes data from one or more network devices and / or components of those devices (e.g., for example...). Figure 6B (Any one or more of the devices illustrated herein). In some embodiments, each device or device type generates different types of log information. For example, in some embodiments, a first component generates latency measurements as log data, while in some embodiments, a second component generates CPU utilization measurements as log data. Figure 11 The time series 1120 of the log data is intended to represent multiple time series of log data from corresponding combinations of multiple devices / components. Example individual data entries in the time series 1120 of the log data are shown as including device identifier 1122, component identifier 1124, and log data 1126. The timestamp 1128 identifying the time when the log data was recorded is also included.

[0094] Figure 11 It also shows Figure 10 Tag / annotation information 1035. For example... Figure 11 As shown, in some embodiments, the tag / annotation information 1035 includes a root cause indication time series 1130 and / or a log data level setting recommendation time series 1140. Figure 11 An example of the content of each entry in the time series of the root cause indicator is shown. Figure 11 The root cause indicator, as shown in some embodiments, includes a device identifier 1132, a component identifier 1134, a cause identifier 1136, a probability 1137 indicating that the indicated cause is the actual cause of the degradation, and a time 1138. Time 1138 provides the sorting rules between the cause identifier 1136, the probability 1137, and data included in other time series (such as time series 1110 and / or time series 1120).

[0095] Figure 11One embodiment illustrating entries in time series 1140 for log data level setting recommendations includes a device identifier 1142, a part identifier 1144, a log data level recommendation 1146, and a time 1148. Some embodiments associate entries in each of time series 1110, 1120, 1130, and 1140 via their respective time fields 1119, 1128, 1138, and 1148. Therefore, a machine learning model, during training, forms associations between these entries based on the time fields, and can then rely on these associations or time correlations when generating appropriate root cause indications and / or log data level setting recommendations.

[0096] Figure 12 An example machine learning module 1200 according to some examples of this disclosure is shown. The example machine learning module 1200 utilizes a training module 1210 and a prediction module 1220. The training module 1210 uses historical information 1030 as input to the feature determination module 1250a. In at least some embodiments, the historical information 1030 is labeled. As described above regarding... Figure 11 As discussed, in some embodiments, example historical information 1030 includes historical operating parameter values, such as any of the operating parameter values ​​discussed above, such as, but not limited to, CPU utilization, memory utilization, latency measurement, error count, collision measurement, or throughput measurement. In some embodiments, historical information 1030 also includes one or more indications of log data generated by network devices and / or components of those devices, wherein the network devices are included in the monitored system.

[0097] In some embodiments, the tag / annotation information 1035 includes possible root causes associated with (included in the history information 1030) corresponding operation parameter values ​​and / or log record data.

[0098] Feature determination module 1250a determines one or more features based on the historical information 1030. Generally, features are a set of information inputs and are information determined to predict a specific outcome. In some examples, features include historical activity data, but in other examples, in some embodiments, features are a subset of historical activity data. In some embodiments, features are encoded as feature vectors 1260. In some embodiments, feature determination module 1250a uses one or more heuristics to determine the features in feature vector 1260 when processing historical information 1030. Machine learning algorithm 1270 generates machine learning model 1018 based on feature vector 1260 and label / annotation information 1035.

[0099] In prediction module 1220, current information 1290 is used as input to feature determination module 1250b. In the disclosed embodiments, current information 1290 includes indications similar to those described above with respect to historical information 1030. However, current information 1290 provides these indications for concurrent log activity or operational parameter values ​​of the monitored system. For example, concurrent activity of the monitored system is provided to feature determination module 1250b to determine, in some embodiments, whether the monitored system is experiencing an operational problem, and if so, what the most likely root cause is. Current information 1290 also allows machine learning model 1018 to generate recommendations for log data level settings to potentially increase the probability associated with one or more root cause determinations.

[0100] Feature determination module 1250b determines, based on current information 1290, the same or a different set of features as feature determination module 1250a determined based on historical information 1030. In some examples, feature determination modules 1250a and 1250b are the same module. Feature determination module 1250b generates feature vector 1215. In some embodiments, feature determination module 1250b uses one or more heuristics to determine features in feature vector 1215 while processing current information 1290. Feature vector 1215 is then provided as input to machine learning model 1018 to generate output 1295. An example of output 1295 is referenced above. Figure 10 Discussions have been conducted. For example, output 1295 may include log data level recommendations, such as those including one or more of device identifier 1052, part identifier 1054, and / or log data level 1056. In some embodiments, output 1295 includes one or more indications of a root cause and / or associated probability 1064, such as a cause indicator 1062. In some embodiments, training module 1210 operates offline to train machine learning model 1018. However, in some embodiments, prediction module 1220 is also designed to operate online. In at least some embodiments, machine learning model 1018 is updated periodically via additional training and / or user feedback.

[0101] Machine learning algorithm 1270 can select from many different potential supervised or unsupervised machine learning algorithms. Examples of supervised learning algorithms include artificial neural networks, Bayesian networks, instance-based learning, support vector machines, decision trees (e.g., Iterative Bisection 3, C4.5, Classification and Regression Trees (CART), Chi-Square Automatic Interaction Detector (CHAID), etc.), random forests, linear classifiers, quadratic classifiers, k-nearest neighbors, linear regression, logistic regression, hidden Markov models, artificial life-based models, simulated annealing, and / or virology. Examples of unsupervised learning algorithms include expectation-maximization algorithms, vector quantization, and information bottleneck methods. Unsupervised models may not have a training module 1210. In an example embodiment, a regression model is used, and machine learning model 1018 is a vector of coefficients corresponding to the learning importance of each feature (or combination of features) in feature vectors 1260 and 1215. In some embodiments, to compute a score, the dot product of the coefficient vectors of feature vector 1215 and machine learning model 1018 is used.

[0102] Figure 13 This is a flowchart illustrating an example method for modifying the log data level of one or more components or devices. In some embodiments, the following describes... Figure 13 One or more of the functions discussed in method 1300 are executed by a hardware processing circuitry. For example, one or more hardware memories (e.g., memory 212, memory 312, memory 412, memory 512) store instructions (e.g., routines 228, 328, 428, 528) that, when executed, configure the hardware processing circuitry (e.g., hardware processor 206, hardware processor 306, hardware processor 406, hardware processor 506) to perform one or more functions of method 1300 discussed below. In some embodiments, the following refers to… Figure 13 One or more of the functions discussed in Method 1300 are performed by the network management system.

[0103] After initiating operation 1305, method 1300 proceeds to operation 1310. In operation 1310, operating parameter values ​​for multiple network devices are acquired. For example, in some embodiments, based on... Figure 6BOne or more devices, similar to any one or more of the illustrated devices, acquire operating parameter values. As discussed above, operating parameter values ​​indicate the operation of the device and / or computer system as a whole. Example operating parameters include CPU utilization, memory utilization, I / O utilization, number of running tasks, latency measurement, throughput measurement, jitter measurement, mobility measurement, signal strength measurement (e.g., Received Signal Strength Indication (RSSI)), error count (e.g., count of dropped packets), number of packets transmitted and / or received within a time period, or other parameters. Operation 1310 also acquires log data from multiple network devices. Log data is generated by each of the network devices based on the current log data level setting of the respective device. For example, in some embodiments, the level of detail (or quantity) of log data generated by the device increases as the device's log data level setting increases. In various embodiments, log data includes, for example, execution trace data of the device and / or components of the device (e.g., data indicating invoked methods, input parameters and / or output parameters of invoked methods, data indicating inputs to the device or component, inputs indicating outputs to the device or component).

[0104] Operation 1315 provides operation parameter values ​​and log data to the machine learning model. For example, as mentioned above... Figures 10 to 11 As described, some embodiments implement a machine learning model (e.g., machine learning model 1018) that receives time series of operation parameter values ​​(e.g., time series of operation parameter values ​​1110) and / or time series of log data (e.g., time series of log recording data 1120).

[0105] In operation 1320, the indication of the downgrade cause and the probability associated with that cause are determined. For example, as mentioned above... Figure 10 Some embodiments discussed implement a machine learning model that provides one or more cause indicators (e.g., cause indicator 1062) and associated probability pairs (e.g., probability 1064). The machine learning model also provides recommended log data level indicators. For example, as... Figure 10 As shown, some embodiments of the machine learning model indicate the log data level (e.g., log data level 1056) of a device (e.g., via device identifier 1052) and / or a component of the device (e.g., component identifier 1054). Some embodiments of method 1300 implement a machine learning model that returns multiple cause / probability pairs. Then, some embodiments rank the cause / probability pairs from highest to lowest probability. These highest-ranked pairs are then evaluated by method 1300 as described below.

[0106] Decision operation 1325 determines whether the probability of the cause (received in operation 1320) is higher than a predefined threshold. In some embodiments, the probability associated with the cause indicates the confidence level of the system being analyzed by the machine learning model that the cause has caused a degradation. If the probability is high enough (e.g., higher than a predefined threshold), method 1300 moves from decision operation 1325 to operation 1335. If the probability is not high enough (e.g., the criterion for testing whether the probability is high enough is not met, e.g., comparing the probability to a predefined probability threshold), method 1300 moves from decision operation 1325 to operation 1330.

[0107] In operation 1330, the log data level settings of multiple network devices are modified to align with log data level settings recommended by machine learning models or otherwise provided. For example, some devices implement an application programming interface (API) that enables the network management system to modify the device's log data level settings, thereby changing the amount or level of detail of the log data generated by the device. In some embodiments, examples of changing the level of detail include providing traces of method calls and exits as log data at a first logging level and providing additional traces of execution within a method (e.g., branches within the method) as log data at a second logging level. Another example of log data level differences in some embodiments is that the device and / or device components generate operation parameter values ​​of a first set of variables as log data at a given first logging data level setting, and generate operation parameter values ​​of a second set of variables as log data at a given second logging data level setting, wherein the second variables include the first set of variables but also include at least one additional variable not included in the first set of variables. This pattern may replicate three, four, five, or more logging data levels (e.g., where progressive logging data levels increase logging of at least one additional variable relative to lower logging levels).

[0108] After modifying the log data level settings, method 1300 moves back from operation 1330 to operation 1310. Note that in subsequent executions of operation 1310, the log data obtained is based on the log data level modified during the previous execution of operation 1330.

[0109] If the probability of a cause generated by the machine learning model is sufficiently high (e.g., it meets a criterion), then operation 1335 identifies a remedial action based on the cause. Some embodiments maintain a mapping between causes and remedial actions. Therefore, some embodiments of operation 1335 rely on this mapping to identify remedial actions based on causes. In some embodiments, a mapping of multiple causes identified by the machine learning model is mapped to remedial actions. Thus, in these embodiments, if the machine learning model identifies a first cause pair, where each cause has a sufficiently high association probability, then the mapping identifies a first remedial action. If the machine learning model identifies a second cause pair, where each cause has a sufficiently high association probability, then the mapping identifies a second remedial action. In some instances, the first cause pair and the second cause pair share a common cause. At least in some embodiments, the mapping still identifies two distinct remedial actions for the two pairs.

[0110] In operation 1340, a remedial action is performed. Examples of remedial actions include: resetting or restarting a specific device among multiple network devices, changing parameter values ​​of one or more network devices (e.g., changing the memory allocation of network buffer space, transient swap space, or other configuration settings), or receiving an alert (e.g., via any messaging technology, such as email, text, voice, or other messaging technologies).

[0111] After operation 1340 is completed, method 1300 moves to end operation 1350.

[0112] The techniques of various embodiments can be implemented using software, hardware, and / or a combination of software and hardware. Various embodiments relate to apparatuses, such as mobile nodes, mobile wireless terminals, base stations (e.g., access points), and communication systems. Various embodiments also relate to methods, such as methods for controlling and / or operating communication devices (e.g., wireless terminals (UEs), base stations, control nodes, access points, and / or communication systems). Various embodiments also relate to non-transitory machines, such as computer-readable media, such as ROM, RAM, CDs, hard disks, etc., which include machine-readable instructions for controlling the machine to implement one or more steps of the method.

[0113] It should be understood that the specific order or hierarchy of the steps in the disclosed process is illustrative. Based on design preferences, it should be understood that the specific order or hierarchy of the steps in the process may be rearranged while remaining within the scope of this disclosure. The appended method claims present the elements of each step in an illustrative order, but this does not imply limitation to the specific order or hierarchy presented.

[0114] In various embodiments, the devices and nodes described herein are implemented using one or more modules to perform steps corresponding to one or more methods, such as signal generation, transmission, processing, and / or receiving steps. Therefore, in some embodiments, modules are used to implement various features. Such modules can be implemented using software, hardware, or a combination of software and hardware. In some embodiments, each module is implemented as a separate circuit, wherein the device or system includes separate circuitry for implementing the functionality corresponding to each described module. Multiple methods or method steps in the methods or method steps described above can be implemented using machine-executable instructions, such as software, included in a machine-readable medium (such as a memory device, e.g., RAM, floppy disk, etc.), to control a machine, e.g., a general-purpose computer with or without additional hardware, to implement, for example, all or parts of the methods described above in one or more nodes. Thus, among other things, various embodiments relate to a machine-readable medium, e.g., a non-transitory computer-readable medium, including machine-executable instructions for causing a machine (e.g., a processor and associated hardware) to perform one or more steps of one or more of the methods described above. Some embodiments relate to a device including a processor configured to implement one, more, or all steps of one or more methods of an example aspect.

[0115] In some embodiments, one or more processors (e.g., CPUs) of one or more devices (e.g., communication devices such as wireless terminals (UEs) and / or access nodes) are configured to perform steps of methods described as being performed by the devices. The processor configuration can be achieved by using one or more modules (e.g., software modules) to control the processor configuration and / or by including hardware (e.g., hardware modules) in the processor to perform the steps and / or control the processor configuration. Thus, some, but not all, embodiments relate to a communication device, such as a user equipment, having a processor, that includes modules corresponding to each step of the various described methods performed by the device including the processor. In some, but not all, embodiments, the communication device includes modules corresponding to each step of the various described methods performed by the device including the processor. Modules can be implemented purely in hardware, e.g., as circuits, or can be implemented using software and / or hardware or a combination of software and hardware.

[0116] Some embodiments relate to a computer program product comprising a computer-readable medium including code for causing a computer or multiple computers to perform various functions, steps, actions, and / or operations, such as one or more steps described above. According to embodiments, the computer program product may, and sometimes does, include different code for each step to be performed. Thus, the computer program product may, and sometimes does, include code for each individual step of a method, such as a method of operating a communication device (e.g., a wireless terminal or node). The code may take the form of machine-executable instructions (e.g., computer-executable instructions) stored on a computer-readable medium, such as RAM (random access memory), ROM (read-only memory), or other types of storage devices. In addition to relating to a computer program product, some embodiments also relate to a processor configured to implement one or more of the various functions, steps, actions, and / or operations of one or more methods described above. Therefore, some embodiments relate to a processor, such as a CPU, graphics processing unit (GPU), digital signal processing (DSP), etc., configured to implement some or all of the steps of the methods described herein. This processor can be used, for example, in a communication device or other device described in this application.

[0117] In view of the foregoing description, many additional variations of the methods and apparatuses with respect to the various embodiments described above will be apparent to those skilled in the art. Such variations are to be considered within the scope of this disclosure. The methods and apparatuses can be used, and in various embodiments, with BLE, LTE, CDMA, Orthogonal Frequency Division Multiplexing (OFDM), and / or various other types of communication technologies that can be used to provide a wireless communication link between an access node and a mobile node. In some embodiments, the access node is implemented as a base station that establishes a communication link with a user equipment device (e.g., a mobile node) using OFDM and / or CDMA. In various embodiments, the mobile node is implemented as a notebook computer, a personal digital assistant (PDA), or other portable device for implementing these methods, which includes receiver / transmitter circuitry and logic and / or routines.

[0118] In the detailed description, numerous specific details are set forth to provide a thorough understanding of some embodiments. However, those skilled in the art will understand that some embodiments can be practiced without these specific details. In other instances, well-known methods, processes, components, units, and / or circuits have not been described in detail to avoid obscuring the discussion.

[0119] Some embodiments can be used in conjunction with a variety of devices and systems, such as user equipment (UE), mobile device (MD), wireless station (STA), wireless terminal (WT), personal computer (PC), desktop computer, mobile computer, laptop computer, notebook computer, tablet computer, server computer, handheld computer, handheld device, personal digital assistant (PDA) device, handheld PDA device, on-board device, off-board device, hybrid device, in-vehicle device, non-in-vehicle device, mobile or portable device, consumer device, non-mobile or non-portable device, wireless communication station, wireless communication device, wireless access point (AP), wired or wireless router, wired or wireless modem, video device, audio device, audio / video (A / V) device, wired or wireless network, wireless local area network, wireless video local area network (WVAN), local area network (LAN), wireless local area network (WLAN), personal local area network (PAN), wireless PAN (WPAN), etc.

[0120] Some embodiments may be used in conjunction with devices and / or networks operating according to: existing Wireless Gigabit Alliance (WGA) specifications (Wireless Gigabit Alliance Inc. WiGig MAC and PHY Specifications—Release 11, April 2011, Final Specification) and / or future versions and / or derivatives thereof; devices and / or networks operating according to: existing IEEE 802.11 standards (IEEE 802.11-2012, IEEE Information Technology Standards—Telecommunications and Information Exchange between Systems Local Area Networks and Metropolitan Area Networks—Specific Requirements Part 11: Wireless LAN Media Access Control (MAC) and Physical Layer (PHY) Specifications, March 29, 2012; IEEE 802.11ac-2013 (“IEEE P…”)… IEEE 802.11ac-2013, IEEE Information Technology Standards – Telecommunication and Information Exchange Between Systems – Local Area Networks and Metropolitan Area Networks – Specific Requirements – Part 11: Wireless LAN Media Access Control (MAC) and Physical Layer (PHY) Specification – Revision 4: Enhancement of Extremely High Throughput for Operation in the Sub-6 GHz Band (December 2013); IEEE 802.11ad (“IEEE P 802.11ad-2012, IEEE Information Technology Standards – Telecommunication and Information Exchange Between Systems – Local Area Networks and Metropolitan Area Networks – Specific Requirements – Part 11: Wireless LAN Media Access Control (MAC) and Physical Layer (PHY) Specification – Revision 3: Enhancement of Extremely High Throughput in the 60 GHz Band”, December 28, 2012); IEEE-802.11REVmc (“IEEE IEEE 802.11-REVmcTM / D3.0, June 2014 Draft Information Technology Standard – Remote Communication and Information Exchange Between Systems – Local Area Networks and Metropolitan Area Networks – Specific Requirements – Part 11: Wireless LAN Media Access Control (MAC) and Physical Layer (PHY) Specifications”; IEEE 802.11-ay (P802.11ay Information Technology Standard – Remote Communication and Information Exchange Between Systems – Specific Requirements – Part 11: Wireless LAN Media Access Control (MAC) and Physical Layer (PHY) Specifications – Revision: Throughput Enhancement in Unlicensed Bands Above 45 GHz); IEEE 802.11-2016 and / or future versions and / or derivatives thereof; devices and / or networks operating according to: existing Wi-Fi Alliance (WFA) Point-to-Point (P2P) Specification (Wi-Fi P2P Technical Specification, Version 1).5. (August 2014) and / or future versions and / or derivatives thereof; devices and / or networks operating under existing cellular specifications and / or protocols, such as the 3rd Generation Partnership Project (3GPP), 3GPP Long Term Evolution (LTE) and / or future versions and / or derivatives thereof; units and / or devices that are part of the aforementioned networks or use one or more of the aforementioned protocols, etc.

[0121] Some embodiments can be used in conjunction with the following: one-way and / or two-way radio communication systems, cellular radio telephone communication systems, mobile phones, cellular phones, wireless phones, personal communication system (PCS) devices, PDA devices including wireless communication devices, mobile or portable global positioning system (GPS) devices, devices including GPS receivers or transceivers or chips, devices including RFID elements or chips, multiple-input multiple-output (MIMO) transceivers or devices, single-input multiple-output (SIMO) transceivers or devices, multiple-input single-output (MISO) transceivers or devices, devices having one or more internal antennas and / or external antennas, digital video broadcasting (DVB) devices or systems, multi-standard wireless devices or systems, wired or wireless handheld devices, such as smartphones, Wireless Application Protocol (WAP) devices, etc.

[0122] Some embodiments can be used, for example, in conjunction with one or more of the following types of wireless communication signals and / or systems: radio frequency (RF), infrared (IR), frequency division multiplexing (FDM), orthogonal FDM (OFDM), orthogonal frequency division multiple access (OFDMA), time division multiplexing (TDM) with FDM, time division multiple access (TDMA), multi-user MIMO (MU-MIMO), space division multiple access (SDMA), extended TDMA (E-TDMA), general packet radio service (GPRS), extended GPRS, code division multiple access (CDMA), wideband CDMA (WCDMA), and CDMA. 2000, Single-carrier CDMA, Multi-carrier CDMA, Multi-carrier Modulation (MDM), Discrete Multi-tone (DMT), Bluetooth, Global Positioning System (GPS), Wi-Fi, Wi-Max, ZigBee™, Ultra-Wideband (UWB), Global System for Mobile Communications (GSM), 2G mobile networks, 2.5G mobile networks, 3G mobile networks, 3.5G mobile networks, 4G mobile networks, 5G or 6G mobile networks, 3GPP, Long Term Evolution (LTE), LTE Advanced, Enhanced Data Rate Evolution of GSM (EDGE), etc. Other embodiments can be used in a variety of other devices, systems, and / or networks.

[0123] Some illustrative embodiments can be used in conjunction with a WLAN (Wireless Local Area Network) (e.g., a Wi-Fi network). Other embodiments can be used in conjunction with any other suitable wireless communication network (e.g., WLAN, "piconet", WPAN, WVAN, etc.).

[0124] Some illustrative embodiments can be used in conjunction with wireless communication networks that communicate in frequency bands of 2.4 GHz, 5 GHz, and / or 60 GHz. However, other embodiments can be implemented using one or more other suitable wireless communication frequency bands (e.g., extremely high frequency (EHF) bands (millimeter wave (mmWave) bands), such as bands between 20 GHz and 300 GHz, WLAN bands, WPAN bands, bands according to WGA specifications, etc.).

[0125] While only a few simple examples of various device configurations have been provided above, it should be understood that many variations and arrangements are possible. Furthermore, this technique is not limited to any particular channel but is generally applicable to one or more frequency ranges / channels. Moreover, as discussed, this technique may be useful in unlicensed spectrum.

[0126] While embodiments are not limited in this respect, terms such as “processing,” “operation,” “calculation,” “determine,” “establish,” “analyze,” “check,” etc., may refer to one or more operations and / or one or more processes of a computer, computing platform, computing system, communication system or subsystem, or other electronic computing device, which manipulate and / or transform physical (e.g., electronic) quantities in the registers and / or memory of a computer into other data that are also represented as physical quantities in the registers and / or memory of a computer or other information storage medium that may store instructions for performing the operations and / or processes.

[0127] While embodiments are not limited in this respect, the term "multiple" as used herein may include, for example, "a plurality" or "two or more". The term "multiple" may be used throughout the specification to describe two or more components, devices, elements, units, parameters, circuits, etc. For example, "multiple stations" may include two or more stations.

[0128] It may be advantageous to clarify the definitions of certain words and phrases used throughout this document: the terms “comprising” and “including” and their derivatives mean including but not limited to; the term “or” is inclusive, meaning and / or; the phrases “associated with” and “related to” and their derivatives may mean including, contained within, interconnected with, contained, included in, connected to or connected to, coupled to or coupled with, able to communicate with, cooperate with, interleaved, parallel, proximate, combined with or combined with, having, having the characteristics of, etc.; and the term “controller” means any device, system or part thereof that controls at least one operation, such device may be implemented in hardware, circuit system, firmware or software, or at least a combination of two of these. It should be noted that the functionality associated with any particular controller may be centralized or distributed, local or remote. Definitions of certain words and phrases are provided in this document, and it will be understood by those skilled in the art that, in many, if not most, cases, such definitions apply to both prior and future use of the words and phrases defined therein.

[0129] Example embodiments have been described with respect to communication systems and protocols, techniques, means, and methods for performing communication, such as in wireless networks or generally operating using one or more of any communication protocols. Such examples are home or access networks, wireless home networks, wireless corporate networks, etc. However, it should be understood that, in general, the systems, methods, and techniques disclosed herein are equally applicable to other types of communication environments, networks, and / or protocols.

[0130] For purposes of explanation, numerous details have been set forth in order to provide a thorough understanding of the present technology. However, it should be understood that this disclosure can be practiced in a variety of ways beyond the specific details set forth herein. Furthermore, while the exemplary embodiments described herein illustrate various co-located components of the system, it should be understood that various components of the system may be located in remote portions of a distributed network (such as a communication network, node) within a domain host and / or the Internet, or within a dedicated secure, insecure, and / or encrypted system, and / or within network operation or management equipment located inside or outside the network. As an example, a domain host may also refer to any device, system, or module that manages or configures or communicates with any one or more aspects of the network or communication environment and / or transceivers and / or stations and / or one or more access points described herein.

[0131] Therefore, it should be understood that system components can be combined into one or more devices, or split among devices (such as transceivers, access points, stations, domain hosts, network operation or management devices, nodes), or co-located on specific nodes of a distributed network (such as a communication network). Based on the following description, it should be understood, and for computational efficiency reasons, that system components can be positioned anywhere within a distributed network without affecting their operation. For example, various components can be located in domain hosts, nodes, domain management devices (such as MIBs), network operation or management devices, one or more transceivers, stations, one or more access points, or some combination thereof. Similarly, one or more functional parts of the system can be distributed between transceivers and associated computing devices / systems.

[0132] Furthermore, it should be understood that various links (including one or more communication channels / elements / lines of connecting elements) can be wired or wireless links or any combination thereof, or one or more other known or later-developed elements capable of supplying data to and / or transmitting data from connecting elements. The term "module" as used herein can refer to any known or later-developed hardware, circuit system, software, firmware, or combination thereof capable of performing the functions associated with that element. As used herein, the terms determination, calculation, and operation, and their variations, are used interchangeably and include any type of methodology, process, technique, mathematical operation, or protocol.

[0133] Furthermore, while some of the example embodiments described herein relate to the transmitter portion of a transceiver performing certain functions or the receiver portion of a transceiver performing certain functions, this disclosure is intended to include corresponding and complementary transmitter-side or receiver-side functions in the same transceiver and / or one or more other transceivers, and vice versa.

[0134] Some example embodiments of enhanced communication are described herein. However, it should be understood that, in general, the systems and methods described herein are equally applicable to any type of communication system utilizing any one or more protocols in any environment, including wired communication, wireless communication, power line communication, coaxial cable communication, fiber optic communication, etc.

[0135] Example 1 is a method executed by a hardware processing circuitry system, comprising: obtaining one or more operating parameter values ​​and first log data for each of a plurality of network devices included in a networked computer system; providing the operating parameter values ​​and the first log data to a machine learning model system; obtaining a recommended log data level for each of the plurality of network devices from the machine learning model system and based on the provided operating parameter values ​​and log data; setting the log data level for each of the plurality of network devices based on the recommended log data level of the respective network device; obtaining second log data from each of the plurality of network devices based on the set log data level of the respective network device; providing the second log data to the machine learning model system; obtaining an indication of a cause for degradation of the networked computer system from the machine learning model system and based on the second log data; determining a remedial action based on the cause; and applying the remedial action to the networked computer system.

[0136] In Example 2, the subject of Example 1 may optionally include: obtaining probabilities associated with indicative causes from a machine learning model system, and identifying remedial actions based on those probabilities.

[0137] In Example 3, the subject of any one or more of Examples 1 to 2 may optionally include one or more of the operating parameter values ​​indicating one or more of the following: processor utilization, memory utilization, throughput measurement, latency measurement, jitter measurement, mobility measurement, signal strength measurement, or error counting.

[0138] In Example 4, the subject of any one or more of Examples 1 to 3 may optionally be included, wherein the machine learning model system operates using a machine learning model trained via time-dependent operating parameter values, log data levels, root cause indicators, or probabilities.

[0139] In Example 5, the subject matter of any one or more of Examples 1 to 4 may optionally include multiple network devices comprising at least one access point and at least one wireless terminal associated with the access point.

[0140] In Example 6, the subject matter of any one or more of Examples 1 to 5 may optionally include: providing the machine learning model system with log rate data that defines the rate at which each of the multiple network devices generates data at different log data levels, wherein the recommended log data level for each of the multiple network devices obtained from the machine learning model system is also based on the log rate data.

[0141] In Example 7, the subject matter of any one or more of Examples 1 to 6 may optionally include: providing a machine learning model system with a mapping between different log data levels and the set of operational parameters included in the log data at each of the different log data levels, wherein the acquisition of the recommended log data level for each of the multiple network devices from the machine learning model system is also based on the mapping.

[0142] Example 8 is a system including hardware processing circuitry; one or more memories storing instructions that, when executed, configure the hardware processing circuitry system to perform operations including: obtaining one or more operating parameter values ​​and first log data for each of a plurality of network devices included in a networked computer system; providing the operating parameter values ​​and first log data to a machine learning model system; obtaining a recommended log data level for each of the plurality of network devices from the machine learning model system and based on the provided operating parameter values ​​and log data; setting the log data level for each of the plurality of network devices based on the recommended log data level of the respective network device; obtaining second log data from each of the plurality of network devices based on the set log data level of the respective network device; providing the second log data to the machine learning model system; obtaining an indication from the machine learning model system and based on the second log data of the networked computer system regarding a degradation cause; identifying a remedial action based on the cause; and applying the remedial action to the networked computer system.

[0143] In Example 9, the subject of Example 8 may optionally include operations that further include: obtaining the probability associated with the indicated cause from the machine learning model system, and identifying remedial actions based on the probability.

[0144] In Example 10, the subject of any one or more of Examples 8 to 9 may optionally include one or more of the operating parameter values ​​indicating one or more of the following: processor utilization, memory utilization, throughput measurement, latency measurement, jitter measurement, mobility measurement, signal strength measurement, or error counting.

[0145] In Example 11, the subject of any one or more of Examples 8 to 10 may optionally include the machine learning model system operating using a machine learning model trained via time-dependent operating parameter values, log data levels, root cause indicators, or probabilities.

[0146] In Example 12, the subject matter of any one or more of Examples 8 to 11 may optionally include multiple network devices comprising at least one access point and at least one wireless terminal associated with the access point.

[0147] In Example 13, the subject matter of any one or more of Examples 8 to 12 may optionally include: providing the machine learning model system with log rate data that defines the rate at which each of the multiple network devices generates data at different log data levels, wherein the recommended log data level for each of the multiple network devices obtained from the machine learning model system is also based on the log rate data.

[0148] In Example 14, the subject matter of any one or more of Examples 8 to 13 may optionally include operations that further include: providing a machine learning model system with a mapping between different log data levels and the set of operational parameters included in the log data of each of the different log data levels, wherein the recommended log data level for each of the multiple network devices obtained from the machine learning model system is also based on the mapping.

[0149] In Example 15, the subject matter of any one or more of Examples 8 to 14 may optionally include operations that further include: providing the machine learning model system with data defining the amount of log data generated by each of the multiple network devices at different log data levels, wherein the recommended log data level for each of the multiple network devices obtained from the machine learning model system is also based on data.

[0150] In Example 16, the subject of any one or more of Examples 8 to 15 may optionally include the setting of the log data level in response to the determination that the probability associated with a cause identified by the machine learning model system is below a predefined threshold.

[0151] In Example 17, the subject of any one or more of Examples 9 to 16 may optionally include an identifier for a remedial action that is in response to a probability higher than a predefined threshold.

[0152] Example 18 is a non-transitory computer-readable storage medium including instructions that, when executed, configure a hardware processing circuitry system to perform operations including: obtaining one or more operating parameter values ​​and first log data for each of a plurality of network devices included in a networked computer system; providing the operating parameter values ​​and the first log data to a machine learning model system; obtaining a recommended log data level for each of the plurality of network devices from the machine learning model system and based on the provided operating parameter values ​​and log data; setting the log data level for each of the plurality of network devices based on the recommended log data level of the respective network device; obtaining second log data from each of the plurality of network devices based on the set log data level of the respective network device; providing the second log data to the machine learning model system; obtaining an indication of a degradation cause of the networked computer system from the machine learning model system and based on the second log data; determining a remedial action based on the cause; and applying the remedial action to the networked computer system.

[0153] In Example 19, the subject of Example 18 may optionally include operations that further include: obtaining probabilities associated with the indicated cause from a machine learning model system, and identifying remedial actions based on the probabilities.

[0154] In Example 20, the subject matter of any one or more of Examples 18 to 19 may optionally include one or more of the operating parameter values ​​indicating one or more of the following: processor utilization, memory utilization, throughput measurement, latency measurement, jitter measurement, mobility measurement, signal strength measurement, or error counting.

[0155] In Example 21, the subject of any one or more of Examples 18 to 20 may optionally include the machine learning model system operating using a machine learning model trained via time-dependent operating parameter values, log data levels, root cause indicators, or probabilities.

[0156] In Example 22, the subject matter of any one or more of Examples 18 to 21 may optionally include multiple network devices comprising at least one access point and at least one wireless terminal associated with the access point.

[0157] In Example 23, the subject matter of any one or more of Examples 18 to 22 may optionally include operations that further include: providing the machine learning model system with log rate data that defines the rate at which each of the multiple network devices generates data at different log data levels, wherein the recommended log data level for each of the multiple network devices obtained from the machine learning model system is also based on the log rate data.

[0158] In Example 24, the subject matter of any one or more of Examples 18 to 23 may optionally include operations that further include: providing a machine learning model system with a mapping between different log data levels and a set of operational parameters included in the log data at each of the different log data levels, wherein the recommended log data level for each of the multiple network devices obtained from the machine learning model system is also based on the mapping.

[0159] In Example 25, the subject matter of any one or more of Examples 18 to 24 may optionally include operations that further include: providing the machine learning model system with data defining the amount of log data generated by each of the multiple network devices at different log data levels, wherein the recommended log data level for each of the multiple network devices obtained from the machine learning model system is also based on data.

[0160] In Example 26, one or more of the topics in Examples 18 through 25 may optionally include a setting for the log data level in response to determining that the probability associated with a cause identified by the machine learning model system is below a predefined threshold.

[0161] In Example 27, the subject of any one or more of Examples 19 to 26 may optionally include the identification probability of a remedial action in response to a predefined threshold.

[0162] In Example 28, the subject matter of any one or more of Examples 1 to 27 may optionally include: providing the machine learning model system with data defining the amount of log data generated by each of the multiple network devices at different log data levels, wherein the recommended log data level for each of the multiple network devices obtained from the machine learning model system is also based on data.

[0163] In Example 29, one or more of the topics in Examples 1 to 28 may optionally include the setting of the log data level in response to the determination that the probability associated with a cause identified by the machine learning model system is below a predefined threshold.

[0164] In Example 30, the subject of any one or more of Examples 2 through 29 may optionally include an identifier of a remedial action in response to a probability higher than a predefined threshold.

[0165] Regarding IEEE 802.11 and / or and / or Low-power transceivers, along with associated communication hardware, software, and communication channels, are described to illustrate some example systems and methods. However, to avoid unnecessarily obscuring this disclosure, well-known structures and devices that can be shown in block diagrams or otherwise generalized are omitted in the following description.

[0166] While the flowchart described above has been discussed with respect to a specific sequence of events, it should be understood that changes to that sequence may occur without materially affecting the operation of one or more embodiments. Additionally, the exemplary techniques described herein are not limited to the specifically illustrated embodiments, but can also be used with other exemplary embodiments, and each described feature can be claimed individually and separately.

[0167] The system described above can be implemented on one or more wireless telecommunications devices / systems (such as IEEE 802.11 transceivers). Examples of wireless protocols that can be used with this technology include IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, IEEE 802.11ac, IEEE 802.11ad, IEEE 802.11af, IEEE 802.1101ah, IEEE 802.181ai, IEEE 802.181aj, IEEE 802.11aq, IEEE 802.11ax, Wi-Fi, LTE, 4G, etc. WirelessHD, WiGig, WiGi, 3GPP, Wireless LAN, WiMAX, DensiFi SIG, Unifi SIG, 3GPP LAA (Licensed Auxiliary Access), etc.

[0168] Additionally, systems, methods, and protocols can be implemented to improve one or more of the following: a dedicated computer, a programmable microprocessor or microcontroller and one or more peripheral integrated circuit elements, an ASIC or other integrated circuit, a digital signal processor, hardwired electronic or logic circuits (such as discrete component circuits), programmable logic devices (such as PLDs, PLAs, FPGAs, PALs), modems, transmitters / receivers, any comparable device, etc. In general, any device capable of implementing, and subsequently, capable of implementing, the state machines described herein can benefit from the various communication methods, protocols, and techniques disclosed herein.

[0169] Examples of processors as described in this article may include, but are not limited to, at least one of the following: 800 and 801, featuring 4G LTE integration and 64-bit computing 610 and 615, with 64-bit architecture A7 processor, M7 motion coprocessor, series, Core TM Processor family, Processor family, Atom TM Processor family, Intel Processor family, i5-4670K and i7-4770K 22nm Haswell, i5-3570K 22nm Ivy Bridge, FX TM Processor family, FX-4300, FX-6300 and FX-8350 32nmVishera, Kaveri processor, Texas Jacinto C6000 TM Automotive infotainment processor, Texas OMAP TM Automotive-grade mobile processors Cortex TM -M processor, Cortex-A and ARM926EJ-S TM processor, The AirForce BCM4704 / BCM4703 wireless network processor, AR7100 wireless network processing unit, and other industry equivalent processors can perform computing functions using any known or future-developed standards, instruction sets, libraries, and / or architectures.

[0170] Furthermore, the disclosed methods can be implemented in software using easy-to-use object-oriented or object-based software development environments that provide portable source code usable on a variety of computer or workstation platforms. Alternatively, the disclosed systems can be implemented partially or entirely in hardware using standard logic circuits or VLSI designs. Whether to implement the system according to the embodiments using software or hardware depends on the system's speed and / or efficiency requirements, specific functions, and the specific software or hardware system or microprocessor or microcomputer system being used. The communication systems, methods, and protocols described herein can be readily implemented in hardware and / or software by a person skilled in the art based on the functional descriptions provided herein and with general knowledge of the fundamentals of the computer and telecommunications fields using any known or later-developed systems or structures, devices, and / or software.

[0171] Furthermore, the disclosed methods can be readily implemented in software and / or firmware that can be stored on storage media to improve performance in general-purpose computers, special-purpose computers, microprocessors, etc., where the controller and memory cooperate in programming. In these instances, the system and methods can be implemented as programs embedded in personal computers, such as applets, JAVA.RTM, or CGI scripts, as resources residing on servers or computer workstations, as routines embedded in dedicated communication systems or system components, etc. The system can also be implemented by physically integrating the system and / or methods into software and / or hardware systems such as communication transceivers.

[0172] Therefore, it is evident that systems and methods for enhancing and improving session user interfaces have been provided at least. While embodiments have been described in conjunction with several examples, it will be apparent to those skilled in the art that many alternatives, modifications, and variations may be or are obvious. Consequently, this disclosure is intended to cover all such alternatives, modifications, equivalents, and variations within the spirit and scope of this disclosure.

Claims

1. A computer networking method, comprising: From a network of network devices included in a networked computer system, obtain one or more operating parameter values ​​and first log data for each of the network devices; The operation parameter values ​​and the first log data are provided to the machine learning model system; The machine learning model system obtains the recommended log data level for each of the plurality of network devices based on the provided operating parameter values ​​and log data; The log data level of each of the plurality of network devices is set based on the recommended log data level for the respective network device; Second log data is obtained from each of the plurality of network devices based on the log data level set by the corresponding network device. The second log data is provided to the machine learning model system; The machine learning model system obtains an indication of the reason for the degradation of the networked computer system based on the second log data; Identify remedial actions based on the stated reasons; as well as The remedial action is applied to the networked computer system.

2. The computer networking method according to claim 1 further includes: The probability associated with the indicated cause is obtained from the machine learning model system, and the remedial action is identified based on the probability.

3. The computer networking method according to claim 2, wherein the identifier for the remedial action is a response to the probability being higher than a predefined threshold.

4. The computer networking method according to any one of claims 1 to 3, wherein the one or more operating parameter values ​​indicate one or more of the following: processor utilization, memory utilization, throughput measurement, latency measurement, jitter measurement, mobility measurement, signal strength measurement, or packet error counting.

5. The computer networking method according to any one of claims 1 to 3, wherein the machine learning model system operates using a machine learning model trained via time-related operating parameter values, log data levels, root cause indicators, or probabilities associated with said causes.

6. The computer networking method according to any one of claims 1 to 3, wherein the plurality of network devices includes at least one access point and at least one wireless terminal associated with the access point.

7. The computer networking method according to any one of claims 1 to 3, further comprising: The following log rate data is provided to the machine learning model system, the log rate data defining the rate at which each of the plurality of network devices generates data at different log data levels, wherein the acquisition of the recommended log data level for each of the plurality of network devices from the machine learning model system is also based on the log rate data.

8. The computer networking method according to any one of claims 1 to 3, further comprising: The machine learning model system is provided with a mapping between different log data levels and the set of operational parameters included in the log data at each of the different log data levels, wherein the acquisition of the recommended log data level for each of the plurality of network devices from the machine learning model system is also based on the mapping.

9. The computer networking method according to any one of claims 1 to 3, further comprising: The following data is provided to the machine learning model system, the data defining the amount of log data generated by each of the plurality of network devices at different log data levels, wherein the acquisition of the recommended log data level for each of the plurality of network devices from the machine learning model system is also based on the data.

10. The computer networking method according to any one of claims 1 to 3, wherein the setting of the log data level is a response to determining that the probability associated with a cause identified by the machine learning model system is below a predefined threshold.

11. A computer network system, comprising: Hardware processing circuit system; One or more memories store instructions that, when executed, configure the hardware processing circuitry system to perform operations, including: From a network of network devices included in a networked computer system, obtain one or more operating parameter values ​​and first log data for each of the network devices; The operation parameter values ​​and the first log data are provided to the machine learning model system; The machine learning model system obtains the recommended log data level for each of the plurality of network devices based on the provided operating parameter values ​​and log data; The log data level of each of the plurality of network devices shall be set based on the recommended log data level of the respective network device; Second log data is obtained from each of the plurality of network devices based on the log data level set by the corresponding network device. The second log data is provided to the machine learning model system; The machine learning model system obtains an indication of the reason for the degradation of the networked computer system based on the second log data; Identify remedial actions based on the stated reasons; and The remedial action is applied to the networked computer system.

12. The computer network system according to claim 11, further comprising: The probability associated with the indicated cause is obtained from the machine learning model system, and the remedial action is identified based on the probability.

13. The computer network system of claim 12, wherein the identifier for the remedial action is a response to the probability being higher than a predefined threshold.

14. The computer network system according to any one of claims 11 to 13, wherein the one or more operating parameter values ​​indicate one or more of the following: processor utilization, memory utilization, throughput measurement, latency measurement, jitter measurement, mobility measurement, signal strength measurement, or packet error count.

15. The computer network system according to any one of claims 11 to 13, wherein the machine learning model system operates using a machine learning model trained via time-related operating parameter values, log data levels, root cause indicators, or probabilities associated with said causes.

16. The computer network system according to any one of claims 11 to 13, wherein the plurality of network devices includes at least one access point and at least one wireless terminal associated with the access point.

17. The computer network system according to any one of claims 11 to 13, wherein the operation further comprises: The following log rate data is provided to the machine learning model system, the log rate data defining the rate at which each of the plurality of network devices generates data at different log data levels, wherein the acquisition of the recommended log data level for each of the plurality of network devices from the machine learning model system is also based on the log rate data.

18. The computer network system according to any one of claims 11 to 13, wherein the operation further comprises: The machine learning model system is provided with a mapping between different log data levels and the set of operational parameters included in the log data at each of the different log data levels, wherein the acquisition of the recommended log data level for each of the plurality of network devices from the machine learning model system is also based on the mapping.

19. The computer network system according to any one of claims 11 to 13, wherein the operation further comprises: The following data is provided to the machine learning model system, the data defining the amount of log data generated by each of the plurality of network devices at different log data levels, wherein the acquisition of the recommended log data level for each of the plurality of network devices from the machine learning model system is also based on the data.

20. The computer network system according to any one of claims 11 to 13, wherein the setting of the log data level is a response to determining that the probability associated with a cause identified by the machine learning model system is below a predefined threshold.

21. A non-transitory computer-readable storage medium, comprising instructions that, when executed, configure a hardware processing circuitry system to perform an operation, the operation comprising: From a network of network devices included in a networked computer system, obtain one or more operating parameter values ​​and first log data for each of the network devices; The operation parameter values ​​and the first log data are provided to the machine learning model system; The machine learning model system obtains the recommended log data level for each of the plurality of network devices based on the provided operating parameter values ​​and log data; The log data level of each of the plurality of network devices is set based on the recommended log data level for the respective network device; Second log data is obtained from each of the plurality of network devices based on the log data level set by the corresponding network device. The second log data is provided to the machine learning model system; The machine learning model system obtains an indication of the reason for the degradation of the networked computer system based on the second log data; Identify remedial actions based on the stated reasons; as well as The remedial action is applied to the networked computer system.

22. The non-transitory computer-readable storage medium of claim 21, further comprising: The probability associated with the indicated cause is obtained from the machine learning model system, and the remedial action is identified based on the probability.

23. The non-transitory computer-readable storage medium of claim 22, wherein the identification of the remedial action is a response to the probability being higher than a predefined threshold.

24. The nontransitory computer-readable storage medium according to any one of claims 21 to 23, wherein the one or more operating parameter values ​​indicate one or more of the following: processor utilization, memory utilization, throughput measurement, latency measurement, jitter measurement, mobility measurement, signal strength measurement, or packet error counting.

25. The nontransitory computer-readable storage medium according to any one of claims 21 to 23, wherein the machine learning model system operates using a machine learning model trained via time-dependent operating parameter values, log data levels, root cause indicators, or probabilities associated with said causes.

26. The nontransitory computer-readable storage medium according to any one of claims 21 to 23, wherein the plurality of network devices includes at least one access point and at least one wireless terminal associated with the access point.

27. The non-transitory computer-readable storage medium according to any one of claims 21 to 23, wherein the operation further comprises: The following log rate data is provided to the machine learning model system, the log rate data defining the rate at which each of the plurality of network devices generates data at different log data levels, wherein the acquisition of the recommended log data level for each of the plurality of network devices from the machine learning model system is also based on the log rate data.

28. The non-transitory computer-readable storage medium according to any one of claims 21 to 23, wherein the operation further comprises: The machine learning model system is provided with a mapping between different log data levels and the set of operational parameters included in the log data under each of the different log data levels, wherein the acquisition of the recommended log data level for each of the plurality of network devices from the machine learning model system is also based on the mapping.

29. The non-transitory computer-readable storage medium according to any one of claims 21 to 23, wherein the operation further comprises: The following data is provided to the machine learning model system, the data defining the amount of log data generated by each of the plurality of network devices at different log data levels, wherein the acquisition of the recommended log data level for each of the plurality of network devices from the machine learning model system is also based on the data.

30. The nontransitory computer-readable storage medium according to any one of claims 21 to 23, wherein the setting of the log data level is a response to determining that the probability associated with a cause identified by the machine learning model system is below a predefined threshold.

Images