Enhanced conversation interface for network management

Abstract

The embodiment of the invention relates to an enhanced conversation interface for network management. A network management system is disclosed that provides an interface to enable diagnosis and troubleshooting of a remotely managed multi-site network. Some embodiments provide a natural language interface, while other embodiments provide a chat robot type interface that communicates with a technician via traditional textual information on a display screen. A diagnostic and troubleshooting capability searches for a central data repository that receives device attribute information from each site of a multi-site network. A query of the data repository is initiated to obtain additional data on the device based on the device or user matching the portion of the entity. A response to the query is then provided based on the attributes of the device.

Description

[0001] This application is a divisional application of the invention patent application with the application date of July 29, 2021, application number 202110865722.3, and invention name “Enhanced Conversation Interface for Network Management”.

[0002] This application claims the benefit of U.S. Application No. 17 / 158,853, filed on January 26, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates to management of communication networks, and more particularly to methods and / or apparatus for providing an enhanced natural language conversation user interface for a network management system. Background Art

[0004] Users of complex networks (such as enterprise networks) may experience a reduction in system level experience (SLE), which may be caused by various problems. In order to support such networks and solve problems when they occur, many complex tools have been developed that analyze and diagnose problems of these networks based on various log data and statistical information associated with how the network operates. An example of such a tool is the Virtual Network Assistant (VNA) tool provided by Juniper Networks. When identifying the root cause of network problems, VNA uses machine learning, information theory, and Bayesian statistics. In at least some cases, VNA also provides mitigation of network problems by prompting manual procedures or automatically implementing corrective measures without human assistance.

[0005] The use of network equipment is common in enterprises and other public places. Often, when users of network equipment experience a degradation in service levels, they contact a support group to resolve the issue. However, these support groups must be funded and can often represent a significant expense to the organizations that provide the funding. Therefore, technology that reduces the burden on support groups when supporting a network would reduce expenses and increase the return on capital. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The embodiments herein may be better understood by referring to the following description in conjunction with the accompanying drawings, wherein like reference numbers indicate identical or functionally similar elements, and wherein:

[0007] Figure 1A is an overview diagram of a system that implements one or more of the disclosed embodiments.

[0008] Figure 1B An example data flow implemented in one or more of the disclosed embodiments is shown.

[0009] Figure 2An example user interface is shown that may be implemented in one or more of the disclosed embodiments.

[0010] Figure 3 Illustrated is a dialog output of an example troubleshooting session in one or more of the disclosed embodiments.

[0011] Figure 4 An example query session implemented by one or more of the disclosed embodiments is illustrated.

[0012] Figure 5 Illustrated is an example query session 500 implemented in one or more of the disclosed embodiments.

[0013] Figure 6 Illustrated is an example query session 600 implemented in one or more of the disclosed embodiments.

[0014] Figure 7 An example query session implemented in one or more of the disclosed embodiments is illustrated.

[0015] Figure 8 is a block diagram of an example access point 800 in accordance with one or more of the disclosed embodiments.

[0016] Fig. 9 is a block diagram of an example network management system 900 .

[0017] Figures 10A to 10C Illustrated are example data structures maintained by one or more of the disclosed embodiments.

[0018] Fig.11 is a flow chart of an example method for acting on a command.

[0019] Fig.12 An example of a simplified lookup table 1200 is illustrated in accordance with one or more embodiments.

[0020] Fig.13 is a flow chart of a method for providing a diagnostic interface to a managed multi-site network. DETAILED DESCRIPTION

[0021] Embodiments of a network management system are described. The network management system provides diagnostics and / or troubleshooting of a remotely managed multi-site network. In various embodiments, the multi-site network includes one or more of: a wireless network, a wired network, or an optical network. Some embodiments provide diagnostics and / or troubleshooting via voice commands. In these embodiments, responses to voice commands are conditioned on the current state of the multi-site network. The current state of each site is reflected in a centralized data repository. Such a centralized data repository acts as a data source for satisfying natural language queries.

[0022] In some embodiments, a natural language processor (NLP) receives a spoken query and separates the query into at least two components, an intent and an entity. The intent is the "verb" of the NLP query, and in various embodiments, may include a "list" verb or a "troubleshoot" verb. The entity is the object of the verb. Thus, if the spoken command is "list all APs", the entity is "all APs" and the intent is "list".

[0023] The disclosed embodiments utilize context-sensitive schemes to interpret entities provided by a natural language processor. For example, in some cases, natural language commands are ambiguous. A command to "list unsatisfactory devices" may require a first set of criteria to be evaluated for a first type of device, and a second set of criteria to be evaluated for a second type of device. For example, in other words, "satisfactory" for a mobile phone is defined differently from satisfaction for a wireless access point. The disclosed embodiments recognize context-sensitive differences such as these by providing criteria or other conditions for "satisfactory", which vary based on the type of device for which information is queried. Additional ambiguity is presented when a term has multiple meanings. As an example, "Lisa" is both a person's name and a computer's name. Therefore, commands such as "What is Lisa's identity" have multiple interpretations. For example, if the user connected to a multi-site wireless network is named "Lisa", the query can be interpreted as requesting the identity of the user itself. The identity of user Lisa needs to be evaluated based on the first set of criteria. If a computer named "Lisa" is also connected to the multi-site wireless network, the identity of the computer needs to be evaluated under the second set of criteria. The disclosed embodiments handle this ambiguity by applying different criteria to evaluate the identity of "Lisa".

[0024] Some of the disclosed embodiments also refine responses to voice commands based on the results of the query itself. In some cases, query responses indicating abnormal conditions, such as low service level experience (SLE), are prioritized over other query responses indicating normal conditions. In the above example, some embodiments collect identity information for a user named "Lisa" and a computer named "Lisa". These embodiments then evaluate the collected identity information to determine whether the identity information indicates an abnormal condition or nominal performance. Based on this classification of the identity information, the two interpretations are ranked, and a response is provided based on the ranking. Thus, for example, identity information indicating an abnormal condition is ranked higher than identity information indicating a nominal condition, thereby forming the basis for a response to the query.

[0025] Following are some examples of interactions between a user and an embodiment of a network management system.

[0026] Example 1:

[0027] Verbal input: "List Mojave devices"

[0028] Intent => "List"

[0029] Entity => "Mojave"

[0030] Fuzzy search of a data repository (e.g., data repository 106) ("Mojave")

[0031] Fuzzy search returns indication that Mojave refers to "Mobile Client OS"

[0032] From the “Mobile Client OS”, the disclosed embodiments infer the following properties:

[0033] Entity_type => Mobile Client

[0034] Property_type=>OS

[0035] property_value=>Mojave

[0036] The disclosed embodiments then construct a query to obtain a list of matching results. In the above example, a user types a query requesting a list of Mohave devices. The intent is identified as "list" and the entity is identified as "Mohave." The entity (e.g., Mohave) is then used to perform a fuzzy search on the data repository 106. The search results indicate that Mohave is the OS of the client. An inference is then made that the entity_type is mobileclient, the attribute is OS, and the attribute value is Mojave. Using this understanding of the query, some of the disclosed embodiments interpret that the requested list is a list of mobile devices that use the Mohave OS. This understanding helps to construct a list of parameters that are passed to a query tool that performs a search on one or more data repositories. Some embodiments rely on a configuration file or data repository to obtain a list of mobile clients that have Mohave as their OS.

[0037] Example 2:

[0038] User input: "Troubleshooting Mojave devices"

[0039] Intent => Troubleshooting

[0040] Entity => Mojave

[0041] Fuzzy search of a data repository (e.g., data repository 106) (Mojave)

[0042] Fuzzy search results indicate Mojave is the client operating system

[0043] Entity Inference:

[0044] entity_type=>mobile client

[0045] property_type=>os,

[0046] property_value=>Mojave

[0047] Based on this information, some of the disclosed embodiments construct a query to obtain a list of matching results. In Example 2, a user types a query requesting troubleshooting for a Mohave device. The intent is identified as "troubleshooting" and the entity is identified as "Mohave." A fuzzy search is performed against the data repository for "Mohave." The search results indicate that Mohave is the OS of the client. The following inferences are then made: the entity_type is a mobile client, the attribute is an OS, and the attribute value is Mojave. Using this understanding of the user input, the disclosed embodiments interpret that the requested list is a list of issues related to mobile devices using the Mohave OS. This understanding helps clarify the intent, and after obtaining a list of Mohave devices from the data repository, the disclosed embodiments invoke a query tool to obtain data related to the performance of devices running the Mohave OS. In various embodiments, the information presented to the user depends on the number of items found that match the search criteria for mobile clients with Mohave as their OS. If the search results indicate that there are no devices with Mohave OS, some embodiments display a message stating "I could not find any Mojave clients. Can you tell me more about the device you want to troubleshoot?" If the search results indicate that there is only a single device with Mohave OS, some embodiments invoke one or more additional query tools to obtain more information about the root cause of the problem affecting the performance of the specific device. The results from these additional invocations are then displayed.

[0048] In some embodiments, if the search results indicate that there are less than a predefined threshold number of devices with Mohave OS (e.g., three (3)), then at least in some embodiments, additional tools are again invoked to determine the root cause of any issues affecting the performance of these particular devices. Responses to the user are then provided based on the information obtained from these additional invocations. Some embodiments allow the user to select from a list of identified devices, and then additional queries are performed based on the selected devices.

[0049] In some embodiments, if the search results indicate more than a predefined number of qualifying devices, additional devices, then a summary report of these devices is prepared. For example, in some embodiments, one or more query tools are invoked to obtain information related to the root cause of the problem affecting these devices, including historical information related to these devices. The summary report then includes historical and identity information for devices with Mojave OS. An example of such a conversation is "I found ten Mojave devices, one has rebooted, two devices recently changed configuration, and three devices have AP health issues. Are there any specific devices you would like to troubleshoot further?"

[0050] Example 3:

[0051] User input: "Troubleshooting AP43."

[0052] Intent => Troubleshooting

[0053] Entity => AP43

[0054] The disclosed embodiment then performs a fuzzy search for "AP43" against the data repository. The fuzzy search returns data indicating that AP43 corresponds to the AP model name attribute in the data repository.

[0055] From the AP model name attribute, inferences are made as outlined below:

[0056] Entity Inference:

[0057] entity_type=>AP

[0058] property_type=>model

[0059] property_value=>AP3

[0060] Construct a query to get a list of matching results

[0061] In Example 3, a user types a query requesting troubleshooting an AP named AP43. The intent is identified as "troubleshooting" and the entity is identified as "AP43". A fuzzy search is performed on the entity "AP43" against the data repository. The fuzzy search identifies that AP43 is an AP model name attribute. Therefore, the search results indicate that the user wants to troubleshoot an AP of a specific model (e.g., AP43). Based on the AP model name attribute, some of the disclosed embodiments construct a query for a troubleshooting tool. The query includes parameters that cause the troubleshooting tool to obtain identity information related to a problem affecting the operation of an AP that has a model name attribute corresponding to "AP43".

[0062] If the troubleshooting tool provides output indicating that there are no devices (e.g., APs) with that particular model name attribute value, some embodiments display a message stating "I can't find any devices with model AP43. Can you tell me more about the device you want to troubleshoot?" If the troubleshooting tool output indicates that there is only a single device with an AP model name attribute value equal to "AP43," some embodiments perform a second invocation of the troubleshooting tool again or a different troubleshooting tool to obtain additional root cause information related to the problem associated with the single device. Output is then generated that is derived from the output of the second invocation.

[0063] If a query to the data repository indicates that the number of devices with an AP model name attribute equivalent to "AP43" is within a predefined range, some embodiments invoke a troubleshooting tool to obtain information related to the root cause of the problem affecting the performance of the AP with the model name. Some embodiments then display a message derived from the information related to the root cause.

[0064] In some embodiments, a UI that allows the user to select one of the devices is presented, and then additional details are provided. In some embodiments, a natural language processor provides a command that allows the user to verbally select one of the devices.

[0065] If the number of devices identified from the data repository exceeds a predefined threshold, some embodiments generate a summary report rather than providing details of troubleshooting information about each device. In some embodiments, the summary report takes the form of "I found ten APs of model AP43, one of which was rebooted, two recently changed configurations, and three were attached to problematic switches. Are there any specific APs you would like to debug further?"

[0066] In a similar example, a large number of APs are configured with numeric names, such as AP1, AP2, ... AP78. The naming of the APs in this example is ambiguous as to whether AP43 refers to an AP model name attribute value or an AP name attribute value. In view of this ambiguity, a fuzzy search for AP43 indicates whether "AP43" is an AP model name attribute value or an AP identity name (ID) attribute value. Some embodiments then generate a dialog based on the ambiguity, such as indicating "I found 10 APs of model AP43, 1 of which has been rebooted, 2 have changed configurations, and 3 are attached to problematic switches. In addition, I found an AP named AP43. Is there a specific AP you would like to debug further?" Thus, summary information for each device identified via the AP model name attribute or the AP identity name attribute is provided.

[0067] Example 4:

[0068] User input: "Dissatisfied users on iOS in the past 7 days"

[0069] Intent => unhappy_user

[0070] Entity => Last 7 Days

[0071] Entity => iOS

[0072] Based on the time zone of the user's browser, some embodiments determine that 'last 7 days' is translated into a start_epoch and an end_epoch. The disclosed embodiment then performs a fuzzy search of ("iOS") and determines that iOS matches the client device operating system attribute value. Since the intent is an dissatisfied user, the system interprets this as a troubleshooting intent, and a parameter set for the query tool is generated to request devices with an SLE that indicates poor or at least below nominal performance. The following inference is then made: the query requests information about any device that has the identified operating system name as an entity attribute value. The search tool is invoked and parameters are passed to it to collect information about devices that match the inference. Output derived from the results is then generated.

[0073] If no iOS devices with SLEs indicating substandard performance are identified, a message is displayed or voiced back to the user, such as "I have not found any unsatisfactory iOS devices in the past 7 days."

[0074] Other example intentions achieved by one or more of the disclosed embodiments include, but are not limited to:

[0075] Failure timelines (failure_timelines): Provides error events of the equipment in the time dimension.

[0076] mutual_information: Provides mutual information about a specific attribute. impact_scope: This intent identifies the scope of the issue, such as affected components, users, OS, device type, WLAN, radio band, etc.

[0077] Notice: This intent provides for a single word input that will result in multiple searches being performed in parallel. For example, if a user types "mist-office", some of the disclosed embodiments provide a response indicating a list of search results such as AP (mist-office-lobby), site (mist-office-Cupertino), etc. In some embodiments, if a user types: [webhook], some embodiments generate a response providing semantic search results for a company's FAQ document.

[0078] Troubleshoot: This command causes some of the disclosed embodiments to provide a response indicating information from a troubleshooting tool, such as a Virtual Network Assistant. The troubleshooting tool is invoked with a parameter set determined by the entity accompanying the intent in the spoken command. Restart: This intent returns the conversation context to an initial state.

[0079] rank ntwk-element: An example invocation of this intent is "rank dns server". In response to this intent, some embodiments rank Domain Name Service (DNS) servers based on their activity. A response to this intent is generated based on the ranking. For example, some embodiments output the names of a list of more active DNS servers, where the activity is based on the number of DNS requests satisfied during a predetermined elapsed time period.

[0080] Unhappy_client: In response to this intent, some embodiments generate a list of client devices in the organization, where the client devices in the list are based on error rates associated with the client devices.

[0081] Troubleshoot application: In response to this intent, some embodiments trigger the invocation of a troubleshoot application for a specific application. For example, some applications implement troubleshooting command line options, which in some embodiments are invoked in response to this intent.

[0082] Figure 1A is an overview diagram of a system that implements one or more of the disclosed embodiments. Figure 1A A plurality of network sites are shown, network site 101A, network site 101B, network site 101C, and network site 101D. Each of the network sites includes one or more access points. Figure 1A Example deployments are shown that each include one access point. Figure 1ANetwork site 101A is shown to include access point 104A, network site 101B includes access point 104B, network site 101C includes access point 104C, and network site 101D includes access point 104D. Each network site also includes one or more devices connected to its corresponding access point. For example, network site 101A includes device 102A and device 102B that communicate with access point 104A. Network site 101B includes device 102C and device 102D, both of which communicate with access point 104B. Network site 101C includes device 102E and device 102F, both of which communicate with access point 104C. Network site 101D includes device 102G and device 102H, both of which communicate with access point 104D. Device information from each of the network sites is stored in data repository 106. In some embodiments, user information and / or document information associated with one or more of the network sites 104A to 104D is also stored in data repository 106. In some embodiments, a network agent running at each of network sites 101A-101D scans the local network and uploads new information, such as device, user, or document information, to data repository 106. Thus, for example, if a new user attaches to the network site, the agent detects the new user and uploads the new user's attribute information to data repository 106. Similarly, if a user detaches from the network site, the agent detects that the user does not exist and removes information about the user from data repository 106 or marks the user as inactive on the site.

[0083] The device information provided to the data repository 106 defines attribute values ​​for a plurality of attributes. The attributes include attributes of the device communicating at the corresponding network site. For example, the attributes include one or more of the following: device host name, device, address information, device model information, device operating system type information, device bios version information, device resident application information, or other attributes, as further discussed below. The information is periodically or otherwise continuously provided to the data repository 106. This periodic or cyclic uploading of information from each of the network sites 101A to 101D to the data repository 106 provides a dynamic data set available in the data repository 106. Thus, for example, when a device having a particular attribute value is connected to (or added to) the network at one of the network sites 101A to 101D, the information is reflected in the data repository 106 after some intermediate delay. In various embodiments, the devices managed by the disclosed embodiments include one or more of: a network switch, a router, a wireless access point, a mobile device (such as a smart phone), a laptop, a desktop computer, a printer, a scanner, a copier, or other electronic device capable of being attached and / or connected to a computer network. When the device leaves the network (or is removed from the network), the information is removed from the data repository 106 after some relatively small delay. Thus, the data repository 106 reflects the attribute information of the devices operating at each of the network sites 101A to 101D.

[0084] Network sites 101A-101D are remotely managed via network management system 108. In some embodiments, network management system 108 implements a voice response system that allows user 110 to issue commands via microphone 114 and receive responses via speaker 112. In some other embodiments, user 110 interacts with network management system 108 via a display-based user interface, which may be presented on mobile device 116 or console terminal 118.

[0085] Figure 1B An example data flow implemented in one or more of the disclosed embodiments is shown. Figure 1B The above reference Figure 1A The data repository 106 discussed. The query interface 124, which can be a natural language query interface or a chatbot or a screen / keyboard interface, defines the intent of the query. The disclosed embodiment supports multiple intents, where two intents are in Figure 1B126 and 128. The disclosed embodiments select different query tools based on each of the intents. Thus, if the query interface 124 issues a query that includes the intent 126, the first query tool 130 is selected. If the query interface 124 issues a query that includes the intent 128 (e.g., "troubleshooting"), the second query tool 132 is selected.

[0086] In some embodiments, a parameter set for the selected query tool is also generated. In some embodiments, the parameter set is generated based on the intent and entities associated with the query. Figure 1B As shown, if intent 126 is generated by query interface 124 and includes first entity 142, first parameter set 134 is generated and provided to the selected query tool, i.e., first query tool 130. If query interface 124 generates intent 126 with second entity 144, second parameter set 136 is generated and provided to the selected query tool (i.e., first query tool 130).

[0087] If the query interface 124 issues the intent 128, if the query includes the third entity 146, then the third parameter set 138 is generated and provided to the second query tool 132. If the fourth entity 148 is associated with the intent 128, then the fourth parameter set 140 is generated. Both the first query tool 130 and the second query tool 132 operate on the data repository 106, such as Figure 1A As shown, data repository 106 receives attribute information from each of sites 101A through 101D.

[0088] Figure 2 An example user interface implemented in one or more of the disclosed embodiments is shown. Example user interface 200 displays a debugging session. User interface 200 shows a welcome message 201. The user then types a command 203 that includes an intent and an entity. In this case, the intent is "unsatisfied" and the entity is "clients in the past 7 days." As a result of command 203, the disclosed embodiments generate responses 205 and 207, followed by message 209 that communicates the name of the client obtained from the VNA experiencing the problem. Message 209 includes a list of devices, including zones associated with each of the devices. Zone 211 associated with device "abhis-phone" is shown in Figure 2 Shown in.

[0089] Figure 3 Illustrated is a dialog output 300 of an example troubleshooting session in one or more of the disclosed embodiments. Figure 3The user is shown typing command 311. Alternatively, in some embodiments, the command is provided by selecting an entry that identifies a specific client identified in command 311. For example, in some embodiments, command 311 is typed by the user by selecting an appropriate area of ​​message 209, such as described above with reference to Figure 2 Area 211 of discussion.

[0090] In response to the command, some embodiments of the disclosed embodiments query the Figure 1A The data repository 106 of the disclosed embodiment can be used to obtain parameter values ​​associated with the identified client. From these parameter values, some embodiments of the disclosed embodiment can identify one or more problems of the identified client. Figure 3 Presentation of a first message 313 is shown, followed by information 315 conveying the root cause of the problem experienced by the identified client.

[0091] Figure 4 An example query session implemented by one or more of the disclosed embodiments is illustrated. Figure 4 The example query session of 400 shows a query message 402 requesting a list of clients. The example query session 400 then displays a message 404 indicating that 38 clients are identified. A second message 406 conveys the names of the identified clients obtained from the data repository 106. If the second message 406 does not fit on the screen, the query session 400 provides the user with the ability to swipe the display up or down to view additional clients.

[0092] Figure 5 An example query session 500 implemented in one or more of the disclosed embodiments is illustrated. Command 531 requests a list of Mojave clients. In response to the query message, the disclosed embodiments identify two clients with Mojave OS. A first message 533 is presented, followed by a second message 535 that includes the name of the identified client with Mojave OS.

[0093] Figure 6 An example query session 600 implemented in one or more of the disclosed embodiments is illustrated. A query message 602 indicating a "webhook" requests a list of available reference documents. The example query session then displays information 645 conveying the names of the available documents. Some embodiments provide for selection of one of the available documents, and upon such selection, the document is opened.

[0094] Figure 7 An example query session implemented in one or more of the disclosed embodiments is illustrated. Figure 7A response message 704 is shown indicating the number of most active DNS servers found in response to command 702. The example query session 700 then displays a ranked list 706 of DNS servers in descending order according to their activity levels.

[0095] To facilitate the above functionality, some embodiments store the above reference Figure 1A The attribute information discussed is as a vector. In some embodiments, using readily available software tools such as Word2Vec, the attribute values ​​are encoded as vectors. Therefore, each of the attribute values ​​can be represented as a position in the vector space mapped by the encoding. In the case where an exact match between the entity typed by the user and the attribute value in the data repository 106 is not identified, the attribute value closest to the query entity in the vector space is determined. In some embodiments, the matching attribute value closest to the queried entity in the vector space is considered to be a match with the queried entity. In some embodiments, a match is only considered to be found if the distance is less than a predefined threshold.

[0096] Figure 8 is a block diagram of an example access point 800 according to one or more of the disclosed embodiments. Figure 1A One or more access points 104A to 104D discussed below implement reference Figure 8 One or more components described.

[0097] The access point 800 includes a wired interface 830, a first wireless interface 836, a second wireless interface 842, a processor 806 (e.g., a CPU), a memory 812, and components of a module 808, such as components of a hardware module, such as components of a circuit, which are coupled together via a bus 809, and various components can exchange data and information through the bus 809. The wired interface 830 includes a receiver 832 and a transmitter 834. The wired interface couples the access point 800 to a network and / or the Internet. The first wireless interface 836 can support a Wi-Fi interface, such as an IEEE 802.11 interface, which includes a receiver 838 coupled to a receiving antenna 839 and a transmitter 840 coupled to a transmitting antenna 841. The access point can receive wireless signals from a communication device (e.g., a wireless terminal) via the receiver 838, and the access point can transmit wireless signals to a communication device (e.g., a wireless terminal) via the transmitter 840. The second wireless interface 842 can support The access point 840 includes an interface, which includes a receiver 844 coupled to a receiving antenna 845 and a transmitter 846 coupled to a transmitting antenna 847. The access point can receive wireless signals from a communication device (e.g., a wireless terminal) via the receiver 844, and the access point can transmit wireless signals to a communication device (e.g., a wireless terminal) via the transmitter 846.

[0098] Memory 812 includes routines 814 and data / information 816. Routine 814 includes components of modules 818 (e.g., components of software modules) and application programming interfaces (APIs) 820. Data / information 816 includes configuration information 822, message event flow capture 824, and a set of remedial actions 826 to be taken if an abnormal message flow is found.

[0099] Fig. 9 is a block diagram of an example network management system 900. In some embodiments, the above reference Figure 1A The NMS108 discussed implements the following reference Fig. 9 One or more components of the network management system 900 are discussed.

[0100] The NMS 900 includes a communication interface 930 (e.g., an Ethernet interface), a processor 906, an output device 908 (e.g., a display, a printer, etc.), an input device 910 (e.g., a keyboard, a keypad, a microphone, a touch screen, a mouse, etc.), a memory 912, and components of a module 940, such as components of a hardware module, such as components of a circuit, which are coupled together via a bus 909, and various components can exchange data and information through the bus 909. The communication interface 930 couples the NMS 900 to a network and / or the Internet. The communication interface 930 includes a receiver 932, through which the network monitoring system can receive data and information, such as service-related information, such as message flow, connection time information, and success / failure-related information from an access point, such as for radio connection establishment, AA service, DHCP request, simple notification service (SNS) lookup, and web page request, and includes a transmitter 934, through which the NMS 900 can send data and information, such as configuration information and instructions, such as instructions for access point restart, change transmission power, add SSID, etc.

[0101] The memory 912 includes routines 914 and data / information 917. The routines 914 include components of modules 918 (e.g., components of software modules) and application programming interfaces (APIs) 920. The data / information 917 includes configuration information 922, captured message event streams 924, and a collection of natural language processing (NLP) related data 926. In an example embodiment, the memory 912 includes Figure 1A data repository 106.

[0102] Figures 10A to 10CAn example data structure maintained by one or more of the disclosed embodiments is illustrated. Although the data structure is described with reference to FIG. 10 as a relational database table, other embodiments utilize other data organization methods. For example, some embodiments utilize traditional memory structures such as arrays or linked lists, trees, queues, graphs, or other data structures. In other embodiments, unstructured data storage techniques are relied upon. In some embodiments, the following reference Figures 10A to 10C One or more of the data structures discussed above are stored in the data repository 106, as described above with reference to Figure 1A discussed.

[0103] Figures 10A to 10B A mobile device model table 10500, an AP model name table 10510, an AP ID / name table 10520, a client operating system table 10530, a site table 10540, a switch model table 10550, a switch OS table 10560, an application name table 10570, a user ID table 10580, and an entity table 10590 are shown. Other information used for enhanced querying is stored in a similar data structure. In other embodiments, some data fields are not used, and similar different data fields are incorporated.

[0104] As discussed above, in some embodiments, attribute information from multiple network sites is collected in a data repository (e.g., data repository 106). The attribute information includes attribute values ​​associated with at least one or more devices that exist at or communicate with the network site. These attributes indicate one or more of the following: user name, user identifier, device manufacturer information, software version information, installed application information, operating system name, device name (e.g., host name and / or other device name of the device), site name, or other information.

[0105] The mobile device model table 10500 includes an ID field 10501, an entity name field 10502, the number of sites where the device was detected 10503, the site IDs 10504a to 10504n where the device was detected, and a manufacturer name 10505. In some embodiments, the table is kept up to date by dynamically updating the table when a new device model is detected, when a new device model is detected at any particular site, or when a particular device model is no longer present at a particular site.

[0106] The AP model name table 10510 includes an entity id field 10511, an entity name field 10512, a site number 10513 where the AP model is detected, and site IDs 10514a to 10514n where the AP is detected. In some embodiments, the AP model name table 10510 is kept up to date by dynamically updating the AP model name table 10510 when a new AP model is detected, when a new AP model is detected at any particular site, or when a particular AP model name no longer exists.

[0107] The AP ID / name table 10520 includes an entity ID field 10521, an entity name field 10522, a number of sites 10523 where devices from the AP ID are detected, and site IDs 10524a to 10524n of sites where devices with the AP ID are detected. In some embodiments, the table is kept up to date by dynamically updating the table when a new AP ID is detected, when a new AP ID model is detected at any particular site, or when a particular AP ID no longer exists at a particular site.

[0108] The client operating system table 10530 includes an entity ID field 10531, an entity name field 10532, a site number 10533 field where a device with the OS is detected, and a field for site IDs 10534a to 10534n indicating the site where the device with the OS is detected. In some embodiments, the table is kept up to date by dynamically updating the table when a new OS is detected, when a new OS is detected at any particular site, or when a particular OS no longer exists at a particular site.

[0109] The site table 10540 includes an entity ID field 10541, an entity name field 10542, the number of devices in the site 10543, and a list of devices deployed on the site 10544a to 10544n. In some embodiments, the table is kept up to date by dynamically updating the table when a new site ID is detected, when a new device is detected at any particular site, or when a particular site ID is no longer associated with the network.

[0110] The switch model table 10550 includes an entity ID field 10551, an entity name field 10552, a site number 10553 at which the switch model is detected, and site IDs 10554a to 10554n at which such switch model is detected. In some embodiments, the table is kept up to date by dynamically updating the table when a new switch model is detected, when a new switch model is detected at any particular site, or when a particular switch model no longer exists at a particular site.

[0111] The switch OS table 10560 includes an entity ID field 10561, an entity name field 10562, the site number 10563 where the switch OS is detected, and the site IDs 10564a to 10564n where the switch model is detected. In some embodiments, the table is kept up to date by dynamically updating the table when a new switch OS is detected, when a new switch OS is detected at any particular site, or when a particular switch OS no longer exists at a particular site.

[0112] The application name table 10570 includes an entity ID field 10571, an entity name field 10572, a site number 10573 where the application is detected to be running, and site IDs 10574a to 10574n where such application is detected. In some embodiments, the table is kept up to date by dynamically updating the table when new applications are detected, when new applications are detected to be running at any particular site, or when a particular application no longer exists at a particular site.

[0113] The user ID table 10580 includes an entity ID field 10581, an entity name field 10582, the number of sites 10583 for which the user ID is detected, the site IDs 10584a to 10584n for which such user ID is detected, and the site IDs 10585a to 10585k for which the user has privileges (e.g., a special symbol such as "all" is used for an IT technician with global privileges to view information from sites). In some embodiments, the table is kept up to date by dynamically updating the table when a new user ID is detected, or when a new user ID is detected at any particular site, when a particular user no longer exists at a particular site, or when the viewing privileges of a particular user change.

[0114] The entity table 10590 includes the entity name 10591, an entity ID field 10592, the number of ways the entity can be interpreted 10593, and the addresses 10594a through 10594n where the associated tables for each interpretation are located. For example, if the entity Mojave is interpreted as: a) OS name, b) site name, and c) user ID, the 10593 field would store the number 3 followed by the addresses 10594a through 10594c of the associated tables, a) the address of the client OS table 10530, b) the address of the site table 10540, and c) the user ID table 10580. In some embodiments, the table is kept up to date by dynamically updating the table when new entities are detected, or when the number of meanings associated with a particular entity changes.

[0115] In some embodiments, user ID table 10580 is used to determine the privileges of the user or session making the query. The privileges indicate which sites the query invoked by the user can obtain information from. For example, in some embodiments, the privileges indicate attribute information that the session or user has access to a device at a first site but not a second site, and other information indicating the privileges associated with each user (such as the level of confidential information that can be shared with the user) is also contemplated, but for simplicity, is not shown in the drawings.

[0116] This privilege information then qualifies which data is returned to the user / session in response to the query. For example, when table 10590 indicates that the entity is a value of an operating system name attribute, such as ("List Mojave Devices"), in some embodiments, one of fields 10594a through 10594k identifies the table, and specifically the address of client operating system table 10530, which contains the names and IDs of all mobile device OSes. The system uses entries 10534a through 10534n to identify sites that have Mojave OS running on mobile clients and for which the user has privileges to obtain information, as indicated by entries 10584a through 10584k.

[0117] In some embodiments, heuristics are used to infer a) that because the request is related to the OS of the mobile device, then the request is about the mobile client. These embodiments then use this information and the fact that the entity is determined to be an OS named Mojave to construct a query to the site table 10540 and obtain a list of Mojave devices at the specified site where the user has privileges to view information. This information is then presented to the user.

[0118] According to another example (2), a request to "troubleshoot a Mojave device" is received. The request is decoded into an intent of troubleshooting and an entity of Mojave. A fuzzy query is performed on the "Mojave" entity value against entity table 10590. Through this search, some embodiments determine that the request references a mobile device operating system attribute value. Field 10594 identifies a table that provides information about mobile device operating system attribute values. One or more of table entries 10534a to 10534n are referenced to identify a site that the user has access to, and the site includes a client device running Mojave OS.

[0119] In some embodiments, a heuristic is invoked to determine that because the request is associated with a mobile device operating system name attribute value, the request is about a mobile device. Based on this inference, a query that invokes a troubleshooting tool is constructed. Based on the query, the troubleshooting tool obtains a list of Mojave devices that have experienced a reduction in SLE. This information is then presented to the user via a display or via audio.

[0120] It should be noted that since the intent is determined to be a request to troubleshoot the system, in some embodiments, a heuristic determination query is constructed using entities obtained via a fuzzy search identification. The method constructs a query to a data repository and / or network analysis tool that returns a list of Mojave clients with the lowest SLE, including the root cause of the SLE reduction.

[0121] According to another example (3), a command is provided as "troubleshoot AP43". The intent is identified as "troubleshoot" and the entity is identified as "AP43". A fuzzy query is run against entity table 10590, searching for the "AP43" entity. The result of the search indicates that the request references the AP model name "AP43" or a specific AP with an ID of AP43. Field 10593 indicates that the term "AP43" has two different meanings, and fields 10594a to 10594b identify two tables, specifically the address of a first table (e.g., AP model name table 10510) storing AP model names and the address of a second table (e.g., 10520) storing AP IDs / names. The system uses AP model name table 10510 and name table 10520 to identify sites where the AP43 model is installed and sites that have an AP named "AP43" and for which the user has the privilege to obtain information.

[0122] The system then uses heuristics to infer that a) because the request is related to troubleshooting, the method should query a network analysis tool, such as a VNA tool. The system uses this information and the fact that the entity is determined to be an AP model AP43 or an AP named AP43, and constructs a query to the VNA to obtain the identities of all APs of model AP43 and all APs named AP43 that exhibit a reduction in SLE from a specific site that the user has privileged access to information about. This information is then presented to the dialog engine and displayed to the user.

[0123] It should be noted that since the intent was determined to be a request to troubleshoot the system, the heuristic determination query was constructed using entities obtained using fuzzy search. The query returns a list of APs with the lowest SLE, including the root cause of the SLE degradation.

[0124] Another example command is "Unsatisfied users on iOS in the past 7 days." The intent is identified as "Unsatisfied users," and the entities are identified as "iOS" and "Last 7 days." In some embodiments, the heuristics translate "Unsatisfied users" into an intent for troubleshooting. The past 7 days are translated into specific start and stop times based on the time associated with the user's browser.

[0125] The fuzzy query is performed against entity table 10590 for the "iOS" entity. The query results indicate that "iOS" is associated with the mobile device operating system name attribute value. Field 10594 identifies the client operating system table 10530 and specifically identifies the address of the table containing the mobile device operating system attribute name value. The client operating system table 10530 and specifically fields 10534a to 10534n identify the site where the iOS OS runs and the user has privileges to obtain information.

[0126] Heuristic inference is that since the request is related to troubleshooting, the troubleshooting query application will be called, and then the request should query the VNA tool. Then, the parameters of the query tool are generated based on the reference to the iOS device and the specified time window. The generated parameter list requests identity information. When the query is executed, the result consistent with the user's privilege level is returned, and which iOS devices are identified to show SLE reduction. This information is then presented to the user visually or audibly. According to another example embodiment, the input of the troubleshooting tool (e.g., VNA) includes the site where the user is allowed to obtain information, thereby eliminating the need to filter information based on user privileges.

[0127] It should be noted that since the intent was determined to be a request to troubleshoot the system, the heuristics determined that the query was performed using entities obtained using fuzzy search. Therefore, the parameters of the query were generated accordingly, and the query returned a list of APs with the worst SLE. This information also indicates the root cause of the SLE.

[0128] Fig. 10C An alternative example data structure is shown that may be implemented in one or more of the disclosed embodiments. Figures 10A to 10B The data structure is the same as Fig. 10C The data structures of are discussed as relational database tables. However, these are just examples, and some embodiments utilize other data structure architectures, such as unstructured data storage libraries, linked lists, arrays, or other structures.

[0129] Fig. 10C A site table 10600, a device table 10610, a user table 10620, an attribute table 10630, an attribute name table 10640, a query tool table 10645, and a parameter specification table 10650 are shown. The site table 10600 includes a site identifier field 10602, a site name field 10604, and a site address field 10606. The site identifier field 10602 uniquely identifies a particular site. The site name field 10604 stores a friendly name for the site (e.g., "San Jose"). The site address field 10606 stores the address of the identified site.

[0130] The device table 10610 includes a device identifier field 10612, a device name field 10614, a site identifier field 10616, and a device type field 10618. The device identifier field 10612 uniquely identifies the device. The site identifier field 10616 identifies at which site the device is located. The site identifier field 10616 can be cross-referenced with the site identifier field 10602. The device type field 10618 stores an indicator of the device type (e.g., AP, station, router, switch).

[0131] The user table 10620 includes a user identifier field 10622, a username field 10624, a site identifier field 10626, a device identifier field 10628, and a permissions field 10629. The user identifier field uniquely identifies a particular user who has established a session at a particular site (identified by the site identifier field 10626). The username field 10624 stores the username. The device identifier field 10628 identifies the device associated with the user and can be cross-referenced with the device identifier field 10612. The permissions field 10629 defines the permissions of the user. For example, the permissions field 10629 defines which site information the identified user has access to. Some queries against the data repository 106 only provide data mapped to sites for which the user has permissions.

[0132] Document table 10660 includes document identifier field 10662, document title field 10664, site field 10666, and document data field 10668. Document identifier field uniquely identifies the document. Document title field 10664 describes the title of the document. In some embodiments, site field 10666 identifies the site where the document resides. In some embodiments, site field 10666 is not implemented or otherwise inoperative because the document can be obtained from any site. Document data field 10668 defines the data of the document itself. Version field 10669 defines the document version.

[0133] The attribute table 10630 stores attribute information for devices and users. The attribute table 10630 includes an attribute identifier field 10632 and an identifier field 10634. In some embodiments, the identifier field 10634 identifies one of a device, a user, or a document. Some embodiments identify other types of query targets. The attribute table 10630 also includes an attribute type field 10636 and an attribute value field 10638.

[0134] Attribute identifier field 10632 uniquely identifies a particular attribute value for a particular user / device. Device identifier / user identifier field 10634 identifies which device or user has the attribute described by the row of attribute table 10630. Attribute type field 10636 identifies the attribute type described by the row of attribute table 10630. For example, attribute type field 10636 identifies whether the attribute is an operating system name attribute, an operating system version attribute, an IP address attribute, or another attribute type. Attribute value field 10638 stores the value of the attribute itself. Thus, for example, if attribute type field 10636 indicates that the attribute is an operating system name attribute, the attribute value field stores the name of the operating system, such as "Windows", "Mosaic", "Unix", "DEC Rainbow", or some other operating system name.

[0135] Attribute name table 10640 includes an attribute type field 10642 and an attribute friendly name field 10644. Attribute type field 10642 can be cross-referenced with attribute type field 10636. Attribute friendly name field 10644 provides the character-based name of the attribute itself. Thus, for example, attribute friendly name field 10644 stores "Operating System Name" for an attribute that stores a value for the device operating system name.

[0136] Query tool table 10645 includes intent identifier field 10646 and query tool field 10648. Intent identifier field 10646 identifies a specific intent provided in a query or command. For example, a first value in intent identifier field 10646 identifies an intent of "list," while a second value in intent identifier field 10646 identifies an intent of "troubleshoot." Query tool field 10648 identifies a specific query tool to use when querying data repository 106. For example, a first value in query tool field 10648 identifies a Virtual Network Assistant tool, while a second value in query tool field 10648 identifies a second tool, such as a SQL query tool. Query tool table 10645 is used by some embodiments to identify an appropriate query tool to use for a specific intent included in a query / command.

[0137] Parameter specification table 10650 includes intent identifier field 10652, query target type field 10654, and parameter specification field 10656. Intent identifier field 10652 operates in a similar manner to intent identifier field 10646. Query target type field 10654 identifies the type of device being queried. Thus, for example, at least in some embodiments, query target type field 10654 identifies a user, document, AP, switch, mobile device, or laptop device depending on its value. Parameter specification field 10656 defines the parameters that will be passed to the query tool based on the intent and the type of device being queried. As a simple example of a parameter specification field, in some embodiments, the example parameter specification field is defined as "IP address = $IP address; request = disconnected connection". In this example, the IP address of the queried device is dynamically populated into the $IP action variable, and the parameter set requests the query tool to obtain multiple disconnected connections from the queried device.

[0138] Fig.11 is a flow chart of an example method for acting on a command. Fig.11 Method 1100 discussed by Fig. 9 The method 1100 begins at start operation 1105 and then moves to operation 1110 where a command is received.

[0139] In operation 1115, intent and entities are extracted from the command. In some embodiments, operation 1115 is performed by a natural language processor.

[0140] In operation 1120, a fuzzy search is performed based on the entity. As discussed above, in some embodiments, the fuzzy search searches for attribute values ​​that are similar to or equivalent to the specified entity in a data structure similar to table 10590. As discussed above, in some embodiments, if an exact match is not found, a match within an edit distance or a threshold distance in the vector space is identified as a match.

[0141] The results of the fuzzy search are used in operation 1125 to enhance the understanding of the user entity and related entities. For example, when the user types "list Mojave devices", the fuzzy search indicates that Mojave matches the operating system name attribute value and the identified OS is running on the mobile client.

[0142] Operation 1130 generates a query based on the enhanced understanding provided by operation 1125. Generating the query includes generating a parameter set for a query tool. In some cases, operation 1130 also determines which tables in the data repository 106 should be queried for the requested information. Some embodiments use the enhanced insight to obtain additional information and provide a response.

[0143] If the decision operation 1135 determines that the requested information resides in a table, then the method 1100 queries the appropriate table at operation 1140 .

[0144] If the decision operation 1145 determines that the requested information resides in a static configuration DB or file, then the method 1100 evaluates the configuration DB or file in operation 1150 and obtains the requested information.

[0145] If decision operation 1155 determines that the requested information is best obtained from a query tool, then method 1100 invokes the query tool and obtains the requested information in operation 1160. Otherwise, method 1100 gathers information from other resources such as web pages, or simply redirects the user's device to a URL with the requested input.

[0146] The method 1100 then proceeds to operation 1170, where the results of the method 1100 are presented. The presentation may occur via an electronic display (such as a display on the mobile device 116 or the console terminal 118) or via an audio output (such as the speaker 112), as described above with reference to Figure 1A The method 1100 then returns to operation 1110 .

[0147] Although the above explanation refers to obtaining information from one of the data repositories, those skilled in the art will recognize that in some embodiments, information is collected from multiple sources. The process of collecting information from multiple sources is highlighted in the accompanying drawings with dotted lines. For example, after collecting information from the table at operation 1140, the process continues to decision operation 1145 and assesses whether there is any additional information that may need to be collected from a configuration file, a query tool, or any other data repository.

[0148] Similarly, after collecting information from the configuration file in operation 1150, the process continues to decision operation 1155 and assesses whether there is any additional information that may need to be collected from a query tool (e.g., a VNA) or any other data repository, and after collecting information from the VNA in operation 1160, the process continues to operation 1165 and assesses whether there is any additional information that may need to be collected from any other data repository.

[0149] Fig.12 An example lookup table 1200 is illustrated in accordance with one or more embodiments. Those skilled in the art should recognize that other implementations (such as linked lists, rational DBs, etc.) are also covered by our invention.

[0150] Column 1220 is an example of an entity stored in the example lookup table 1200. Items 1221 to 1229 provide a list of OSs used by clients and / or servers associated with the network. For example, the OS in row 1221 is Mojave, and the OS in row 1222 is iOS, etc.

[0151] Items 1231 to 1239 provide a list of AP IDs associated with the network. For example, the ID of the AP in row 1231 is AP13, the ID of the AP in row 1232 is Mist, the ID of the AP in row 1233 is Mist182, and so on.

[0152] Items 1242 to 1249 provide a list of IDs of users associated with the network. For example, the ID of the user indicated by item 1242 is John Smith, the ID of the user in row 1243 is Sally Brown, and so on.

[0153] Column 1202 identifies whether the item is an OS (by inserting a specific character, such as "x" in the corresponding entry). Column 1203 provides the vector value of the corresponding item. For example, the OS name "Mojave" is converted to a vector [13, 4, 67, 90] by a program such as Word2Vec, and the OS name "iOS" is converted to [89, 34, 55, 2], etc.

[0154] Column 1204 identifies whether an item is an AP (by inserting a specific character, such as "x" in the corresponding entry). Column 1205 provides the vector value of the corresponding item. For example, an AP named "AP13" is converted to a vector [34,346,3,63] by a program such as Word2Vec, an AP named "Mist" is converted to [12,3,46,7], and an AP named "Mist182" is converted to [76,351,2,36], etc.

[0155] Similarly, column 1206 identifies whether the item is a user ID (by inserting a specific character, such as "x" in the corresponding entry). Column 1207 provides the vector value of the corresponding item. For example, the user ID "John Smith" is converted to the vector [764, 45, 2, 19] by a program such as Word2Vec, and the user ID "Sally Brown" is converted to [86, 91, 63, 4], etc.

[0156] Fig.13 is a flow chart of a method for providing a diagnostic interface to a managed multi-site network. Fig.13One or more functions discussed and method 1300 are performed by hardware processing circuitry. For example, in some embodiments, instructions (e.g., routine 914) stored in a memory (e.g., memory 912) configure a hardware processor (e.g., processor 906) to perform one or more functions discussed below with reference to method 1300.

[0157] After starting operation 1305, method 1300 moves to operation 1310. In operation 1310, input including the intent and entity of the query is received. For example, as discussed above, some of the disclosed embodiments provide an interface to the user via a traditional display / keyboard / mouse interface (or a touch screen interface on a smart phone, for example), or provide a natural language interface for input from, for example, a microphone. Through this interface, the intent and entity components of the command or query are obtained.

[0158] In operation 1320, attribute values ​​that match the entity are identified. In some embodiments, a data structure storing attribute values ​​for devices attached to the multi-site network is searched to identify attribute values ​​that match the entity. For example, as discussed above, some embodiments maintain a data repository (e.g., data repository 106) that stores attribute values ​​for devices attached to the multi-site network (e.g., sites 101A to 101D). An example of an organization of a data structure storing attribute values ​​is described in Fig. 10C . In some embodiments, identification is performed using a fuzzy search. Fuzzy search provides the ability to match entities to attribute values ​​that are not necessarily exact matches, but rather within the "edit distance" of the entity or within a predefined distance in a vector space of attribute values. For example, some embodiments utilize tools such as "Word2Vec" to convert entity values ​​(e.g., device attribute values) into vectors, which can then be mapped to a vector space. By mapping entities to the same vector space, some embodiments identify entities that are within a predefined distance from an attribute value and determine that the entity "matches" such an attribute value.

[0159] In some embodiments, the attributes included in the data repository indicate one or more of the following: model name of the device, access point model number, access point name, operating system of the device, name of an application executed on the device or account name associated with the device, device manufacturer, site name, user name, document title, document version number.

[0160] In operation 1330, the query target associated with the attribute value is identified. In some embodiments, one or more heuristics are called to identify the query target based on the identified attribute value. As an example, if the identified attribute value is the name of the mobile device operating system, the heuristic maps the identified attribute value to the mobile device including the identified mobile device operating system. Therefore, the query target is the mobile device including the identified mobile device operating system. As another example, if the identified attribute value is the model name of the AP, then in some embodiments, the heuristic identifies the query target of the AP having the model name equivalent to the identified attribute value. As another example, if the identified attribute value is the name of multiple users, the heuristic identifies the query target of the user with the name. Therefore, in some embodiments, by including the account name linked to or associated with the user with the identified name, the heuristic fully specifies the query target, and / or adds the last name to the query target as needed, so that the query for the user will be completed successfully. Therefore, in various embodiments, the heuristic can adopt multiple mappings, each of which maps from one or more identified attribute values ​​to one or more query targets. In some embodiments, the heuristic also includes a rule that converts the identified attribute value to a query target (e.g., in some embodiments, the heuristic that maps from the identified user name to the account name of the user with that name includes a database query to extract the account name from the data repository based on the user's name, and then constructs the query target based on the extracted account name.

[0161] In operation 1340, the tool to be invoked is identified based on the intent. For example, as described above with reference to Fig. 10C As discussed, some embodiments map intents to specific query tools. Thus, a "troubleshooting" intent invokes a first query tool, while a "listing" intent invokes a second query tool.

[0162] In operation 1350, a parameter set is generated based on the identified query target and intent. For example, in some embodiments, the parameters passed to the query tool vary based on the query target. For example, a device, user, or document may require different parameters passed to the query tool to obtain relevant information. Fig. 10C As discussed in, some embodiments map query target types such as documents, access points, users, or other devices to parameter specifications (e.g., via field 10656) and / or criteria (e.g., via field 10658).

[0163] In operation 1360, the identified query tool is invoked using the generated parameter set. It is noted that some embodiments of method 1300 identify multiple different query targets of different types and therefore invoke the query tool multiple times to collect information about multiple query targets. Based on the state of the data repository and one or more parameters provided to the query tool, the invoked query tool generates an output. The output of the query tool is then parsed or decoded by the disclosed embodiments to generate a response to the original query or command including the entity and intent of operation 1310.

[0164] A response to the original command / query is then provided based on the results of invoking the query tool(s) in operation 1365. For example, in some embodiments, providing a response includes generating an audio signal indicative of the response. For example, in some embodiments that utilize a natural language processor to receive commands, an audio signal is generated. In other embodiments, providing a response includes generating an output on an electronic display, such as to a chatbot or a more traditional window-oriented user interface.

[0165] In some embodiments, the response to the original command includes generating a signal to restart or reset a device, such as a switch or access point. For example, in some embodiments, the response indicates that the switch or access point is unresponsive, has detected an internal fault, or has performance or throughput issues that may be corrected by a reset or restart action. Thus, based on the response to the query, some embodiments automatically or without additional human input invoke mitigation actions on the network device.

[0166] In some embodiments, the invocation of the query tool obtains performance metrics from one or more devices. For example, in some embodiments, which data is obtained from the query target is based on the intent. For example, in some embodiments, the troubleshooting intent invokes the query tool to collect metrics that can be used to determine the SLE of one or more devices identified as the query target.

[0167] Some embodiments apply one or more criteria to the results of the call to determine whether one or more query targets are operating in a nominal manner or experiencing relatively poor SLE (e.g., the criteria test whether one or more operating parameters meet one or more criteria indicating poor performance or poor SLE). In some embodiments, the criteria are obtained via a table that maps intent and query target type (e.g., document, AP, smartphone, or user) to criteria (e.g., in some embodiments, the AP criteria tests the number of connected devices and / or the latency of communications, while in some embodiments, the user criteria tests the number of disconnections or failed login attempts made by the user). In some embodiments, if one or more criteria are met, an indication that the device is experiencing good or poor SLE is provided.

[0168] Some of the disclosed embodiments then rank the query targets based on the extent to which the query targets experience poor SLE. Thus, some embodiments prioritize displaying information related to query targets experiencing problems, and in some embodiments, reduce or even suppress information related to query targets that appear to experience nominal performance.

[0169] Some embodiments periodically receive attribute information about devices attached to each site in a multi-site network. Figure 1A As discussed above, multiple sites (such as sites 101A to 101D) provide device, user, and document attribute and status information to data repository 106. In some embodiments, the attribute and / or status information is stored in an attribute data structure, such as the one described above with reference to FIG. Fig. 10C Attribute table 10630 discussed. Thus, when a new device, user, or document is added to the network, this periodic monitoring detects the same information and retrieves the attribute information from the data repository. When a device, user, or document leaves the network, the periodic monitoring detects the absence of the same information and deletes the attribute information from the data repository (e.g., data repository 106).

[0170] In some embodiments, the default attribute value for each site is also periodically determined. For example, in some embodiments, the default attribute value is determined based on the attribute values ​​of most query targets located at a particular site. Thus, for example, if most devices attached to a particular site run version 3.0 of a particular operating system, then by using a heuristic, version 3.0 is determined as the default attribute value for the operating system version attribute value for devices attached to that particular site. At least in some embodiments, different sites are determined to have different default operating system version attribute values.

[0171] After operation 1365 is completed, method 1300 moves to end operation 1370 .

[0172] The techniques of various embodiments may be implemented using software, hardware, and / or a combination of software and hardware. Various embodiments relate to an apparatus, such as a mobile node, a mobile wireless terminal, a base station (e.g., an access point), a communication system. Various embodiments also relate to methods, such as methods of controlling and / or operating a communication device (e.g., a wireless terminal (UE), a base station, a control node, an access point, and / or a communication system). Various embodiments also relate to a non-transitory machine (e.g., a computer) readable medium, such as a ROM, RAM, CD, hard disk, etc., which includes machine-readable instructions for controlling a machine to implement one or more steps of the method.

[0173] It is to be understood that the specific order or hierarchy of steps in the disclosed process is an example of an example approach. Based on design preferences, it is to be understood that the specific order or hierarchy of steps in the process can be rearranged while remaining within the scope of the present disclosure. The attached method claims present elements of the various steps in a sample order and are not meant to be limited to the specific order or hierarchy presented.

[0174] In various embodiments, the devices and nodes described herein are implemented using one or more modules to perform steps corresponding to one or more methods, such as signal generation, transmission, processing and / or receiving steps. Therefore, in some embodiments, various features are implemented using modules. Such modules can be implemented using software, hardware, or a combination of software and hardware. In some embodiments, each module is implemented as an independent circuit, wherein the device or system includes a separate circuit for implementing the functions corresponding to each described module. Many of the above methods or method steps can be implemented using machine executable instructions (such as software), which are included in a machine readable medium (such as a memory device, such as RAM, a floppy disk, etc.) to control a machine, such as a general-purpose computer with or without additional hardware, to implement all or part of the above methods, for example, in one or more nodes. Therefore, among other things, various embodiments relate to a machine readable medium, such as a non-transient computer readable medium, including machine executable instructions for causing a machine (such as a processor and associated hardware) to perform (multiple) one or more steps of the above methods. Some embodiments relate to a device including a processor, which is configured to implement one, more or all steps of one or more methods of an example aspect.

[0175] In some embodiments, one or more processors (e.g., CPUs) of one or more devices (e.g., communication devices and / or access nodes such as wireless terminals (UEs)) are configured to perform the steps of the methods described as being performed by the devices. The configuration of the processor can be achieved by using one or more modules (e.g., software modules) to control the processor configuration and / or by including hardware (e.g., hardware modules) in the processor to perform the described steps and / or control the processor configuration. Therefore, some but not all embodiments relate to a communication device, such as a user equipment, having a processor, which includes a module corresponding to each of the steps of the various described methods performed by the device in which the processor is included. In some but not all embodiments, the communication device includes a module corresponding to each of the steps of the various described methods performed by the device in which the processor is included. The module can be implemented purely in hardware, such as as a circuit, or can be implemented using software and / or hardware or a combination of software and hardware.

[0176] Some embodiments relate to a computer program product, including a computer-readable medium, which includes a code for causing a computer or multiple computers to implement various functions, steps, actions and / or operations (such as one or more steps described above). Depending on the embodiment, the computer program product can and sometimes does include different codes for each step to be performed. Therefore, the computer program product can and sometimes does include codes for each independent step of the method, such as a method of operating a communication device (such as a wireless terminal or node). The code can be a machine (such as a computer) executable instruction stored on a computer-readable medium (such as a RAM (random access memory), a ROM (read-only memory) or other type of storage device). In addition to relating to a computer program product, some embodiments also relate to a processor, which is configured to implement one or more of the various functions, steps, actions and / or operations of one or more of the above-mentioned methods. Therefore, some embodiments relate to a processor, such as a CPU, a graphics processing unit (GPU), a digital signal processing (DSP) unit, etc., which is configured to implement some or all steps of the method described herein. The processor can be used in, for example, a communication device or other device described in this application.

[0177] In view of the above description, many additional variations of the methods and devices of the various embodiments described above will be apparent to those skilled in the art. Such variations are considered to be within the scope of the present disclosure. The method and device can and in various embodiments are used together with BLE, LTE, CDMA, orthogonal frequency division multiplexing (OFDM) and / or various other types of communication technologies, which can be used to provide a wireless communication link between an access node and a mobile node. In some embodiments, the access node is implemented as a base station, which uses OFDM and / or CDMA to establish a communication link with a user equipment device (e.g., a mobile node). In various embodiments, the mobile node is implemented as a notebook computer, a personal data assistant (PDA) or other portable device, including a receiver / transmitter circuit and logic and / or routines, for implementing these methods.

[0178] In the following description, many specific details are set forth in order to provide a thorough understanding of some embodiments. However, it will be appreciated by those skilled in the art that some embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, units and / or circuits are not described in detail to avoid confusing the discussion.

[0179] Some embodiments may be used in conjunction with various devices and systems, such as user equipment (UE), mobile device (MD), wireless station (STA), wireless terminal (WT), personal computer (PC), desktop computer, mobile computer, laptop computer, notebook computer, tablet computer, server computer, handheld computer, handheld device, personal digital assistant (PDA) device, handheld PDA device, vehicle-mounted device, off-vehicle device, hybrid device, vehicle device, non-vehicle device, mobile or portable device, consumer device, non-mobile or non-portable device, wireless communication station, wireless communication device, wireless access point (AP), wired or wireless router, wired or wireless modem, video device, audio device, audio and video (A / V) device, wired or wireless network, wireless area network, wireless video area network (WVAN), local area network (LAN), wireless LAN (WLAN), personal area network (PAN), wireless PAN (WPAN), etc.

[0180] Some embodiments may be used in conjunction with devices and / or networks operating in accordance with the existing Wireless Gigabit Alliance (WGA) specification (Wireless Gigabit Alliance, Inc. WiGig MAC and PHY Specification Version 1.1, April 2011, Final Specification) and / or future versions and / or derivatives thereof, in accordance with the existing IEEE 802.11 standards (IEEE 802.11-2012, IEEE Standard for Information Technology--Telecommunications and Information Exchange between Local and Metropolitan Area Networks--Specific Requirements Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specification, March 29, 2012; IEEE 802.11ac-2013 ("IEEE Standard for Information Technology--Technology for Telecommunications and Information Exchange between Local and Metropolitan Area Networks--Specific Requirements Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specification, March 29, 2012); IEEE 802.11ac-2013 ("IEEE Standard for Information Technology--Technology for Telecommunications and Information Exchange between Local and Metropolitan Area Networks", ... P802.11ac-2013, IEEE Standard for Information Technology - Telecommunications and Information Exchange between Systems - Local and Metropolitan Area Networks - Specific Requirements - Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications - Amendment 4: Enhancements for Very High Throughput for Operation in Frequency Bands Below 6 GHz, December 2013); IEEE 802.11ad (“IEEE P802.11ad-2012, IEEE Standard for Information Technology - Telecommunications and Information Exchange between Systems - Local and Metropolitan Area Networks - Specific Requirements - Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications - Amendment 3: Enhancements for Very High Throughput for Operation in Frequency Bands Below 6 GHz, December 28, 2012); IEEE-802.11REVmc (“IEEE 802.11-REVmcTM / D3.0, June 2014 Draft Standard for Information Technology - Telecommunications and Information Exchange between System Local Area Networks and Metropolitan Area Networks, Specific Requirements; Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications"); IEEE802.11-ay (P802.11ay Information Technology Standard--Telecommunications and Information Exchange between System Local Area Networks and Metropolitan Area Networks--Specific Requirements Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications--Amendment: Enhanced Throughput for Operation in Unlicensed Bands Above 45 GHz)), devices and / or networks operating in accordance with the existing Wireless Fidelity (Wi-Fi) Alliance (WFA) Peer-to-Peer (P2P) Specifications (Wi-Fi P2P Technical Specification, Version 1.5, August 2014) and / or future versions and / or derivatives thereof, devices and / or networks operating according to existing cellular specifications and / or protocols (e.g., Third Generation Partnership Project (3GPP), 3GPP Long Term Evolution (LTE)) and / or future versions and / or derivatives thereof, units and / or devices that are part of the above networks or operate using any one or more of the above protocols, etc.

[0181] Some embodiments may be used in conjunction with: one-way and / or two-way radio communication systems, cellular radiotelephone communication systems, mobile phones, cellular phones, wireless phones, personal communication system (PCS) devices, PDA devices incorporating wireless communication devices, mobile or portable global positioning system (GPS) devices, devices incorporating GPS receivers or transceivers or chips, devices incorporating RFID elements or chips, multiple-input multiple-output (MIMO) transceivers or devices, single-input multiple-output (SIMO) transceivers or devices, multiple-input single-output (MISO) transceivers or devices, devices with one or more internal antennas and / or external antennas, digital video broadcasting (DVB) devices or systems, multi-standard radio devices or systems, wired or wireless handheld devices (such as smart phones), wireless application protocol (WAP) devices, etc.

[0182] Some embodiments may be used in conjunction with one or more types of wireless communication signals and / or systems, such as radio frequency (RF), infrared (IR), frequency division multiplexing (FDM), orthogonal FDM (OFDM), orthogonal frequency division multiple access (OFDMA), FDM time division multiplexing (TDM), time division multiple access (TDMA), multi-user MIMO (MU-MIMO), space division multiple access (SDMA), extended TDMA (E-TDMA), general packet radio service (GPRS), extended GPRS, code division multiple access (CDMA), wideband CDMA (WCDMA), CDMA 2000, single carrier CDMA, multi-carrier CDMA, multi-carrier modulation (MDM), discrete multi-tone (DMT), Bluetooth, Global Positioning System (GPS), Wi-Fi, Wi-Max, ZigBeeTM, Ultra-Wideband (UWB), Global System for Mobile Communications (GSM), 2G, 2.5G, 3G, 3.5G, 4G, fifth generation (5G) or sixth generation (6G) mobile networks, 3GPP, Long Term Evolution (LTE), Advanced LTE, Enhanced Data Rates for GSM Evolution (EDGE), etc. Other embodiments may be used in various other devices, systems and / or networks.

[0183] Some exemplary embodiments may be used in conjunction with a WLAN (wireless local area network), such as a Wi-Fi network. Other embodiments may be used in conjunction with any other suitable wireless communication network, such as a wireless area network, a "piconet", a WPAN, a WVAN, etc.

[0184] Some exemplary embodiments may be used in conjunction with a wireless communication network that communicates on a frequency band of 2.4 GHz, 5 GHz, and / or 60 GHz. However, other embodiments may be implemented using any other suitable (multiple) wireless communication frequency bands, such as extremely high frequency (EHF) bands (millimeter wave (mmWave) bands), such as bands within the frequency band between 20 GHz and 300 GHz, WLAN bands, WPAN bands, bands according to WGA specifications, etc.

[0185] Although only some simple examples of various device configurations are provided above, it is to be understood that many variations and permutations are possible. Moreover, the technology is not limited to any particular channel, but is generally applicable to any frequency range / channel(s). Moreover, and as discussed, the technology may be useful in unlicensed spectrum.

[0186] Although the embodiments are not limited in this respect, discussions utilizing terms such as, for example, "processing," "computing," "calculating," "determining," "establishing," "analyzing," "examining," and the like may refer to operations (or processes) of a computer, computing platform, computing system, communication system or subsystem, or other electronic computing device that manipulate and / or transform data represented as physical (e.g., electronic) quantities within the computer's registers and / or memory into other data similarly represented as physical quantities within the computer's registers and / or memory or other information storage medium, which may store instructions for performing the operations and / or processes.

[0187] Although the embodiments are not limited in this regard, the terms "plurality" and "aplurality" used herein may include, for example, "plurality" or "two or more". Throughout the specification, the terms "plurality" or "a plurality" may be used to describe two or more components, devices, elements, units, parameters, circuits, etc. For example, "a plurality of stations" may include two or more stations.

[0188] It may be helpful to state definitions of certain words and phrases used in this document: the terms "include" and "comprise" and their derivatives mean including but not limited to; the term "or" is inclusive, meaning and / or; the phrases "associated with" and "associated with" and their derivatives may mean include, be included within, interconnect with, be interconnected with, contain, be contained within, connect to or connect with, couple to or couple with, be communicative with, cooperate with, interleave, juxtapose, be proximate to, be bound to or bound with, have, have the property of, etc.; and the term "controller" means any device, system, or portion thereof that controls at least one operation, such device may be implemented in hardware, circuitry, firmware, or software, or some combination of at least two. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. Definitions for certain words and phrases are provided throughout this document, and those of ordinary skill in the art should understand that in many, if not most instances, such definitions apply to prior, as well as future uses of such defined words and phrases.

[0189] Example embodiments have been described with respect to communication systems and protocols, techniques, components and methods for performing communications, such as in a wireless network, or generally in any communication network operating using any communication protocol(s). Such examples are home or access networks, wireless home networks, wireless corporate networks, etc. However, it should be appreciated that, in general, the systems, methods and techniques disclosed herein are equally effective for other types of communication environments, networks and / or protocols.

[0190] For the purpose of explanation, many details are stated in order to provide a thorough understanding of the present technology. However, it should be understood that the present disclosure can be practiced in a variety of ways beyond the specific details stated herein. In addition, although the example embodiments illustrated herein show various components of the parallel system, it is to be understood that the various components of the system can be located at a remote part of a distributed network (such as a communication network, a node) within a domain master and / or the Internet, or located within a dedicated secure, unsecure and / or encryption system and / or within a network operation or management device located inside or outside the network. As an example, a domain master can also be used to refer to any device, system or module that manages and / or configures any one or more aspects of the network or communication environment described herein and / or (multiple) transceivers and / or stations and / or (multiple) access points, or communicates with them.

[0191] Therefore, it should be understood that the components of the system can be combined into one or more devices, or split between devices, such as transceivers, access points, stations, domain masters, network operation or management devices, nodes, or are arranged in parallel on specific nodes of a distributed network (such as a communication network). It will be understood from the following description, and for reasons of computing efficiency, the components of the system can be arranged at any position within the distributed network without affecting its operation. For example, various components can be located in a domain master, a node, a domain management device (such as a MIB), a network operation or management device, (multiple) transceivers, stations, (multiple) access points, or some combination thereof. Similarly, one or more functional parts of the system can be distributed between a transceiver and an associated computing device / system.

[0192] Further, it should be understood that the various links (including (multiple) any communication channels / elements / lines connecting elements) can be wired or wireless links or any combination thereof, or any other known or later developed (multiple) elements that can supply and / or transmit data between connected elements. The term module used herein can refer to any known or later developed hardware, circuit system, software, firmware or a combination thereof that can perform the functionality associated with the element. As used herein, the terms determine, calculate and compute and variations thereof are used interchangeably and include any type of methodology, process, technology, mathematical operation or protocol.

[0193] Moreover, although some of the example embodiments described herein relate to a transmitter portion of a transceiver performing certain functions or a receiver portion of a transceiver performing certain functions, the present disclosure is intended to include corresponding and complementary transmitter-side or receiver-side functionality in the same transceiver and / or (multiple) other transceivers, respectively, and vice versa.

[0194] The example embodiments are described with respect to enhanced communications. However, it should be understood that, in general, the systems and methods herein are equally effective for any type of communication system utilizing any one or more protocols in any environment, including wired communications, wireless communications, power line communications, coaxial cable communications, fiber optic communications, etc.

[0195] Example systems and methods are related to IEEE 802.11 and / or and / or The present invention is described with reference to a low-energy transceiver and associated communication hardware, software, and communication channels. However, in order to avoid unnecessarily obscuring the present disclosure, the following description omits well-known structures and devices that may be shown in block diagram form or otherwise summarized.

[0196] Although the above flow charts have been discussed with respect to a particular sequence of events, it should be appreciated that changes to this sequence may occur without materially affecting the operation of the embodiment(s). Additionally, the example techniques illustrated herein are not limited to the specific illustrated embodiments, but may also be used with other example embodiments, and each of the described features may be independently and separately claimed.

[0197] The above-described system may be implemented on (multiple) wireless telecommunication devices / systems, such as IEEE 802.11 transceivers, etc. Examples of wireless protocols that may be used with the technology include IEEE 802.11a, IEEE 802.11b, IEEE802.11g, IEEE 802.11n, IEEE 802.11ac, IEEE 802.11ad, IEEE 802.11af, IEEE 802.11ah, IEEE 802.11ai, IEEE 802.11aj, IEEE 802.11aq, IEEE 802.11ax, Wi-Fi, LTE, 4G, WirelessHD, WiGig, WiGi, 3GPP, Wireless LAN, WiMAX, DensiFi SIG, Unifi SIG, 3GPP LAA (Licensed Assisted Access), etc.

[0198] Additionally, the systems, methods, and protocols may be implemented to improve one or more of: a special purpose computer, a programmable microprocessor or microcontroller and (multiple) peripheral integrated circuit components, an ASIC or other integrated circuit, a digital signal processor, a hardwired electronic or logic circuit (such as a discrete component circuit), a programmable logic device (such as a PLD, PLA, FPGA, PAL), a modem, a transmitter / receiver, any comparable components, etc. In general, any device capable of implementing a state machine (and thus capable of implementing the methodology illustrated herein) may benefit from the various communication methods, protocols, and techniques according to the present disclosure provided herein.

[0199] Examples of the processor described herein may include, but are not limited to, at least one of the following: 800 and 801, with 4G LTE integration and 64-bit computing 610 and 615, with 64-bit architecture A7 processor, M7 motion coprocessor, series, Core TM Processor family, Processor family, Atom TM Processor family, Intel Processor family, i5-4670K and i7-4770K 22nm Haswell, i5-3570K 22nmIvy Bridge, FX TM Processor family, FX-4300, FX-6300 and FX-8350 32nmVishera, Kaveri Processor, Texas Jacinto C6000 TM Automotive Infotainment Processors, Texas OMAP TM Automotive-grade mobile processors, Cortex TM -M processor, Cortex-A and ARM926EJ-S TM processor, Air Force BCM4704 / BCM4703 Wireless Networking Processors, AR7100 Wireless Networking Processing Unit, other industry equivalent processors, and may use any known or future developed standards, instruction sets, libraries and / or architectures to perform computing functions.

[0200] In addition, the disclosed method can be easily implemented in software using an object or object-oriented software development environment, which provides portable source code that can be used on various computer or workstation platforms. Alternatively, the disclosed system can be partially or completely implemented in hardware using standard logic circuits or VLSI designs. Whether software or hardware is used to implement the system according to the embodiment depends on the speed and / or efficiency requirements of the system, the specific functions, and the specific software or hardware system or microprocessor or microcomputer system used. Through the functional description provided herein and utilizing the general basic knowledge of computer and telecommunication technology, using any known or later developed system or structure, device and / or software, the communication system, method and protocol illustrated herein can be easily implemented in hardware and / or software by a person of ordinary skill in the applicable field.

[0201] Furthermore, the disclosed methods may be readily implemented in software and / or firmware that may be stored on a storage medium to enhance the performance of a programmed general purpose computer, a special purpose computer, a microprocessor, etc., with a controller and memory cooperating. In these instances, the systems and methods may be implemented as programs embedded on a personal computer, such as applet, JAVA.RTM or CGI script, as resources resident on a server or computer workstation, or as routines embedded in a special purpose communication system or system component, etc. The system may also be implemented by physically incorporating the system and / or method into a software and / or hardware system, such as the hardware and software systems of a communication transceiver.

[0202] It is apparent, therefore, that at least systems and methods for enhancing and improving conversational user interfaces have been provided. Although embodiments have been described in conjunction with a number of embodiments, it is apparent that many alternatives, modifications, and variations will be or will be apparent to those of ordinary skill in the applicable arts. Therefore, the present disclosure is intended to encompass all such alternatives, modifications, equivalents, and variations within the spirit and scope of the present disclosure.

[0203] In view of the above disclosure, various examples are set forth below. It should be noted that one or more features of the examples, taken alone or in combination, should be considered within the disclosure of the present application.

[0204] Example 1 is a system for querying a multi-site network, comprising: a hardware processing circuit system; and one or more hardware memories storing instructions, which when executed configure the hardware processing circuit system to perform operations, the operations comprising: a network management server periodically receiving information from a plurality of network agents installed at a corresponding plurality of network sites, the information defining attribute values ​​of devices communicating on a local network of the corresponding network sites; the network management server storing the received information in a data repository; receiving intent and entities of an NLP query from a natural language processor (NLP); identifying a query application based on the intent; searching information stored in a data repository based on the entity; identifying a plurality of attribute values ​​matching the entity based on the results of the search, at least two of the plurality of attribute values ​​being associated with different attributes; obtaining a corresponding query target list from the data repository and based on each of the different attributes; querying each of the query targets via the identified query application; receiving a response to each of the queries; and responding to the NLP query based on the response.

[0205] In Example 2, the subject matter of Example 1 optionally includes, wherein obtaining the corresponding query target list includes: converting multiple attribute values ​​into query targets based on one or more heuristics.

[0206] In Example 3, the subject matter of any one or more of Examples 1 to 2 optionally includes that the operation also includes: periodically determining default attribute values ​​associated with corresponding network sites based on defined attribute values ​​of devices communicating on each local network in the local network; and determining a parameter set based on the default attribute values, wherein the query is based on the determined parameter set.

[0207] In Example 4, the subject matter of any one or more of Examples 1 to 3 optionally includes that the operation further includes: in response to determining that the intent is associated with the troubleshooting request: for each query target list, determining a parameter set, the parameter set being configured to obtain service level experience (SLE) information, wherein the query for each query target is based on the corresponding parameter set; determining which of the service level experience information in the service level experience information represents a bad SLE, wherein based on the determination, a response to the NLP query indicates which query targets experience the bad SLE; and responding to the NLP query based on the determination.

[0208] In Example 5, the subject matter of Example 4 optionally includes that the operation further includes: identifying a corresponding set of criteria from a data repository and based on each of the different attributes; determining whether each set of criteria in the set of criteria is satisfied based on a response to each query in the query; and based on whether each set of criteria has been satisfied, identifying which query target in the query target list represents at least one query target that has experienced poor SLE, wherein the responding is based on the at least one query target.

[0209] In Example 6, the subject matter of any one or more of Examples 1 to 5 optionally includes that the operation also includes: monitoring a network local to the networked site; based on the monitoring, detecting a new device on the network; determining one or more attribute values ​​associated with the new device; and adding an association between the one or more attribute values, the new device, and the network site to a data repository.

[0210] In Example 7, the subject matter of Example 6 optionally includes that the operations further include: based on the monitoring, detecting that the device has left the network; and in response to the detecting, deleting the attribute value associated with the device from the data database.

[0211] In Example 8, the subject matter of any one or more of Examples 1 to 7 optionally includes, wherein the attributes of the device include one or more of the following: a model name of the device, an access point model number, an access point name, an operating system of the device, a name of an application executed on the device or an account name associated with the device, a device manufacturer, a site name, a user name.

[0212] In Example 9, the subject matter of any one or more of Examples 1 to 8 optionally includes wherein the intent indicates a troubleshooting request, or the intent indicates a property list request, and wherein when troubleshooting is requested, a first query application is identified, and when a property list is requested, a second query application is identified.

[0213] In Example 10, the subject matter of any one or more of Examples 1 to 9 optionally includes, wherein responding to the NLP query based on the response includes: generating an audio signal based on the response, and transmitting the audio signal.

[0214] In Example 11, the subject matter of any one or more of Examples 1 to 10 optionally includes that the operation further includes: determining privileges associated with an account that performs the NLP query, wherein acquisition of the query target or querying of each of the query targets is based on the privileges.

[0215] Example 12 is a method for querying a multi-site network, comprising: a network management server, periodically receiving information from multiple network agents installed at corresponding multiple network sites, the information defining attribute values ​​of devices communicating on the local network of the corresponding network sites; the network management server, storing the received information in a data repository; receiving the intent and entity of an NLP query from a natural language processor (NLP); identifying a query application based on the intent; searching information stored in the data repository based on the entity; identifying multiple attribute values ​​that match the entity based on the search results, at least two of the multiple attribute values ​​being associated with different attributes; obtaining a corresponding query target list from the data repository and based on each of the different attributes; querying each of the query targets via the identified query application; receiving a response to each of the queries; and responding to the NLP query based on the response.

[0216] In Example 13, the subject matter of Example 12 optionally includes, wherein obtaining the corresponding query target list includes: converting the plurality of attribute values ​​into query targets based on one or more heuristics.

[0217] In Example 14, the subject matter of any one or more of Examples 12 to 13 optionally includes periodically determining default attribute values ​​associated with corresponding network sites based on defined attribute values ​​of devices communicating on each local network in the local network; and determining a parameter set based on the default attribute values, wherein the query is based on the determined parameter set.

[0218] In Example 15, the subject matter of any one or more of Examples 12 to 14 optionally includes, in response to determining that the intent is associated with a troubleshooting request: for each query target list, determining a parameter set configured to obtain service level experience (SLE) information, wherein the query for each query target is based on the corresponding parameter set; determining which of the service level experience information represents a bad SLE, wherein based on the determination, a response to the NLP query indicates which query targets experience the bad SLE; and responding to the NLP query based on the determination.

[0219] In Example 16, the subject matter of Example 15 optionally includes, from a data repository and based on each of the different attributes, identifying a corresponding set of criteria; determining whether each set of criteria in the set of criteria is satisfied based on a response to each query in the query; and based on whether each set of criteria has been satisfied, identifying which query target in a list of query targets represents at least one query target that has experienced poor SLE, wherein the response is based on the at least one query target.

[0220] In Example 17, the subject matter of any one or more of Examples 12 to 16 optionally includes monitoring a network local to a networked site; based on the monitoring, detecting a new device on the network; determining one or more attribute values ​​associated with the new device; and adding an association between the one or more attribute values, the new device, and the network site to a data repository.

[0221] In Example 18, the subject matter of Example 17 may optionally include, based on the monitoring, detecting that the device has left the network; and in response to the detecting, deleting the property value associated with the device from the data database.

[0222] In Example 19, the subject matter of any one or more of Examples 12 to 18 optionally includes, wherein the attributes of the device include one or more of the following: a model name of the device, an access point model number, an access point name, an operating system of the device, a name of an application executed on the device or an account name associated with the device, a device manufacturer, a site name, a user name.

[0223] In Example 20, the subject matter of any one or more of Examples 12 to 19 optionally includes, wherein the intent indicates a troubleshooting request, or the intent indicates a property list request, and wherein when troubleshooting is requested, a first query application is identified, and when a property list is requested, a second query application is identified.

[0224] In Example 21, the subject matter of any one or more of Examples 12 to 20 optionally includes, wherein responding to the NLP query based on the response includes: generating an audio signal based on the response, and transmitting the audio signal.

[0225] In Example 22, the subject matter of any one or more of Examples 12 to 21 may optionally include determining privileges associated with an account executing the NLP query, wherein the acquisition of the query target or the querying of each of the query targets is based on the privileges.

[0226] Example 23 is a non-transitory computer-readable storage medium comprising instructions that, when executed, configure a hardware processing circuit system to perform operations to query a multi-site network, including: by a network management server, periodically receiving information from a plurality of network agents installed at corresponding plurality of network sites, the information defining attribute values ​​of devices communicating on the local network of the corresponding network sites; by the network management server, storing the received information in a data repository; receiving the intent and entity of an NLP query from a natural language processor (NLP); identifying a query application based on the intent; searching for information stored in the data repository based on the entity; identifying a plurality of attribute values ​​that match the entity based on the results of the search, at least two of the plurality of attribute values ​​being associated with different attributes; obtaining a corresponding query target list from the data repository based on each of the different attributes; querying each of the query targets via the identified query application; receiving a response to each of the queries; and responding to the NLP query based on the response.

[0227] In Example 24, the subject matter of Example 23 optionally includes, wherein obtaining the corresponding query target list includes: converting multiple attribute values ​​into query targets based on one or more heuristics.

[0228] In Example 25, the subject matter of any one or more of Examples 23 to 24 optionally includes periodically determining default attribute values ​​associated with corresponding network sites based on defined attribute values ​​of devices communicating on each local network in the local network; and determining a parameter set based on the default attribute values, wherein the query is based on the determined parameter set.

[0229] In Example 26, the subject matter of any one or more of Examples 23 to 25 optionally includes, in response to determining that the intent is associated with a troubleshooting request: for each query target list, determining a parameter set configured to obtain service level experience (SLE) information, wherein the query for each query target is based on the corresponding parameter set; determining which of the service level experience information represents a bad SLE, wherein based on the determination, a response to the NLP query indicates which query targets experience the bad SLE; and responding to the NLP query based on the determination.

[0230] In Example 27, the subject matter of Example 26 optionally includes, based on each of the different attributes, identifying a corresponding set of criteria from a data repository; based on a response to each query in the query, determining whether each set of criteria in the set of criteria is satisfied; and based on whether each set of criteria is satisfied, identifying which query target in the query target list represents at least one query target that experiences poor SLE, wherein the responding is based on the at least one query target.

[0231] In Example 28, the subject matter of any one or more of Examples 23 to 27 optionally includes monitoring a network local to a networked site; based on the monitoring, detecting a new device on the network; determining one or more attribute values ​​associated with the new device; and adding an association between the one or more attribute values, the new device, and the network site to a data repository.

[0232] In Example 29, the subject matter of Example 28 may optionally include, based on the monitoring, detecting that the device has left the network; and in response to the detecting, deleting the attribute value associated with the device from the data database.

[0233] In Example 30, the subject matter of any one or more of Examples 23 to 29 optionally includes, wherein the attributes of the device include one or more of the following: a model name of the device, an access point model number, an access point name, an operating system of the device, a name of an application executed on the device or an account name associated with the device, a device manufacturer, a site name, a user name.

[0234] In Example 31, the subject matter of any one or more of Examples 23 to 30 optionally includes, wherein the intent indicates a troubleshooting request, or the intent indicates a property list request, and wherein when troubleshooting is requested, a first query application is identified, and when a property list is requested, a second query application is identified.

[0235] In Example 32, the subject matter of any one or more of Examples 23 to 31 optionally includes, wherein responding to the NLP query based on the response includes: generating an audio signal based on the response, and transmitting the audio signal.

[0236] In Example 33, the subject matter of any one or more of Examples 23 to 32 may optionally include determining privileges associated with an account executing the NLP query, wherein the acquisition of the query target or the querying of each of the query targets is based on the privileges.

Claims

1. A system for querying a network, include: Processing circuit system; as well as one or more memories storing instructions that, when executed, configure the processing circuitry to perform operations comprising: Determine the intent and entity of web-related queries; Based on the entity of the query, obtain multiple query targets; Based on the intent of the query, query each of the plurality of query targets to obtain information indicating performance of the network corresponding to each of the plurality of query targets; and Based on the query, a response to the query is generated for display on a computing device, wherein the response to the query includes an indication that at least one of the query targets in the plurality of query targets is associated with sub-nominal performance of the network.

2. A method for querying a multisite network, include: Determine the intent and entity of web-related queries; Based on the entity of the query, obtain multiple query targets; querying each of the plurality of query targets to obtain information based on the intent of the query, the information indicating a performance of the network corresponding to each of the plurality of query targets; as well as Based on the query, a response to the query is generated for display on a computing device, wherein the response to the query includes an indication that at least one of the query targets in the plurality of query targets is associated with sub-nominal performance of the network.

3. A non-transitory computer-readable storage medium comprising instructions which, when executed, configure a hardware processing circuit system to perform the method of claim 2.

Images