Network diagnostics

Abstract

A method for network diagnostics is presented. In the method, a user query for network diagnostics of natural language expressions is obtained. Based on the user query, a target workflow is determined that includes a set of machine-executable tasks for the network diagnostic process on a target category for the network diagnosis, and diagnostic results are generated by executing the set of machine-executable tasks in the network diagnostic process.

Description

Technical Field

[0001] The present disclosure generally relates to the field of computers, and more particularly, to methods, devices, and computer program products for network diagnosis. Background Art

[0002] In large-scale data center networks, answering network diagnosis queries from users still heavily relies on manual on-duty services. However, this approach requires a great deal of experience and considerable effort from network engineers, who must repeatedly go through a large number of monitoring dashboards and logs. Summary of the Invention

[0003] In a first aspect of the present disclosure, a method for network diagnosis is provided. The method includes: obtaining a user query for network diagnosis expressed in natural language; based on the user query, determining a target workflow including a set of machine-executable tasks for a network diagnosis process on a target category for network diagnosis, where the target category is associated with one or more network components; and generating a diagnosis result by executing the set of machine-executable tasks during the network diagnosis process.

[0004] In a second aspect of the present disclosure, an electronic device is provided. The electronic device includes: a computer processor coupled to a computer-readable memory unit, the memory unit including instructions that, when executed by the computer processor, implement the method according to the first aspect of the present disclosure.

[0005] In a third aspect of the present disclosure, a computer program product is provided. The computer program product includes a computer-readable storage medium having program instructions embodied thereon, the program instructions being executable by an electronic device to cause the electronic device to execute the method according to the first aspect of the present disclosure.

[0006] The Summary of the Invention is provided to introduce a selection of concepts in a simplified form, which will be further described in the following Detailed Description. The Summary of the Invention is not intended to identify the key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Brief Description of the Drawings

[0007] The above and other objects, features, and advantages of the present disclosure will become more apparent from a more detailed description of some embodiments of the present disclosure in the accompanying drawings, where like reference numerals generally refer to like components in the embodiments of the present disclosure.

[0008] Figure 1 An example environment is shown in which example embodiments of the present disclosure can be implemented;

[0009] Figure 2A diagram showing an example of the level of abstraction in NETASSISTANT according to some embodiments of the present disclosure;

[0010] Figure 3 A diagram showing an example of the structure of a network diagnosis platform according to some embodiments of the present disclosure;

[0011] Figure 4 A flowchart showing an example of a method for network diagnosis according to some embodiments of the present disclosure; and

[0012] Figure 5 A block diagram of an electronic device in which various embodiments of the present disclosure can be implemented. Detailed Description

[0013] The principles of the present disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are for illustrative purposes only and are helpful for those skilled in the art to understand and implement the present disclosure, without implying any limitation on the scope of the present disclosure. The disclosure described herein can be implemented in various ways different from those described below.

[0014] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0015] References in the present disclosure to "one embodiment", "an embodiment", "example embodiment", etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but not necessarily every embodiment includes the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an example embodiment, it is submitted that, whether or not explicitly described, it is within the knowledge of those skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments.

[0016] It should be understood that although the terms "first" and "second" etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the example embodiments, the first element may be referred to as the second element, and similarly, the second element may be referred to as the first element. As used herein, the term "and / or" includes any and all combinations of one or more of the listed terms.

[0017] The terms used in this document are for the purpose of describing specific embodiments only and are not intended to limit the example embodiments. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "comprises", "comprising", "has", "having", "includes" and / or "including" when used herein specify the presence of the stated features, elements and / or components, etc., but do not preclude the presence or addition of one or more other features, elements, components and / or combinations thereof.

[0018] The principles of the present disclosure will now be described with reference to some embodiments. It should be understood that these embodiments are for illustrative purposes only and help those skilled in the art to understand and implement the present disclosure, without implying any limitation on the scope of the present disclosure. The disclosure described herein can be implemented in various ways different from those described below. In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0019] It can be understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and related rules.

[0020] It can be understood that before using the technical solutions disclosed in the various embodiments of the present invention, the type, scope of use and usage scenarios of the information involved in this application should be notified to the user in an appropriate manner according to the relevant laws and regulations, and the authorization of the user should be obtained.

[0021] For example, in response to receiving an activity request from a user, a prompt message is sent to the user to clearly notify the user that the requested operation will require the acquisition and use of user information. Therefore, the user can independently choose whether to provide information to software or hardware such as an electronic device, an application, a server or a storage medium that performs the operations of the technical solution of this application according to the prompt message.

[0022] As an optional but non-limiting implementation manner, the manner of sending a prompt message to the user in response to receiving an active request from the user may include, for example, a pop-up window, and the prompt message may be presented in the form of text in the pop-up window. In addition, the pop-up window may also carry a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.

[0023] It can be understood that the above process of notifying and obtaining user authorization is only illustrative and does not limit the implementation of the present disclosure. Other methods that meet the relevant laws and regulations are also applicable to the embodiments of the present disclosure.

[0024] As used herein, the term "model" is referred to as the association between inputs and outputs learned from training data, and thus can generate corresponding outputs for a given input after training. The generation of the model can be based on machine learning techniques. Generally, a machine learning model can be constructed that receives input information and makes predictions based on the input information. For example, a classification model can predict the class of input information in a predetermined set of classes. As used herein, "model" can also be referred to as "machine learning model", "learning model", "machine learning network", or "learning network", which are used interchangeably herein. Example Environment

[0025] Figure 1 FIG. shows a block diagram of an example environment 100 in which various embodiments of the present disclosure can be implemented. Figure 1 The environment 100 includes a network diagnostic platform 110, which can support the operation of network diagnosis. The user 102 can communicate with the network diagnostic platform 110 via, for example, a terminal device 120 associated with the user 102. The user 102 can be referred to as a user of the network diagnostic platform 110. In some example embodiments, the network diagnostic platform 110 can include or be implemented as a digital assistant 112.

[0026] The digital assistant 112 can be configured to have the ability of intelligent conversation. As Figure 1 shown, the digital assistant 112 can be integrated within the network diagnostic platform 110 as part of the network diagnostic platform 110 to assist in task processing within the network diagnostic platform 110.

[0027] In some other example embodiments, the digital assistant 112 can be configured as a stand-alone application, such as a web application or other types of applications. In such examples, the digital assistant 112 and the network diagnostic platform 110 can be considered the same application. The digital assistant 112 can be provided to assist users with various task processing requirements in different applications and scenarios. During the interaction with the digital assistant 112, the user inputs an interaction message, and the digital assistant 112 responds to the user input to provide a reply message. Generally, the digital assistant 112 can support users to input questions in natural language and perform tasks and provide responses based on the understanding of natural language input and logical reasoning capabilities.

[0028] In some example embodiments, the digital assistant 112 can interact with the user 102 as a contact. For example, the digital assistant 112 can be implemented in an instant messaging (IM) application. The digital assistant 112 can interact with the user 102 in a single chat session. In some embodiments, the digital assistant 112 can interact with multiple users in a group chat session including multiple users.

[0029] For each user 102, an interaction window 104 of the network diagnostic platform 110 or the digital assistant 112 can be presented via the terminal device 120, such as a session window with the digital assistant 112. The user 102 can enter a session message in the session window. The network diagnostic platform 110 can determine a reply message of the digital assistant 112 and present it to the user 102 in the interaction window 104.

[0030] In some example embodiments, depending on the configuration of the network diagnostic platform 110, the interaction messages with the network diagnostic platform 110 can include messages in a multimodal form, such as text messages (such as natural language text), voice messages, image messages, video messages, etc.

[0031] The client of the network diagnostic platform 110 can operate at the terminal device 120 of the user 102 and can support the interaction between the user 102 and the network diagnostic platform 110.

[0032] In some example embodiments, the implementation of at least some functions of the network diagnostic platform 110 and / or the implementation of at least some functions of the digital assistant 112 in the network diagnostic platform 110 can be model-based. During the creation and / or operation of the network diagnostic platform 110, one or more models 130 can be used, for example, the capabilities of one or more models 130. In the network diagnostic platform 110, the digital assistant 112 can utilize the model 130 to understand the user input and provide a response to the user based on the output of the model 130.

[0033] Although the model 130 is separated from the network diagnostic platform 110 as Figure 1 shown, one or more models 130 can be deployed at the network diagnostic platform 110 or a remote server. In some example embodiments, the model 130 can be a machine learning model, a deep learning model, a learning model, a neural network, etc. In some other embodiments, the model can be based on a language model (LM). The language model can acquire question and answer capabilities by learning from a large corpus. The model 130 can also be based on other suitable models.

[0034] It should be understood that the network diagnostic platform 110 can operate on a suitable electronic device. The electronic device here can be referred to as any type of device having computing capabilities, including a terminal device or a server device.

[0035] For example, the terminal device can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, netbooks, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / cameras, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof.

[0036] For example, the server device can include a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, etc. In some embodiments, the network diagnosis platform 110 can be implemented based on cloud services.

[0037] It should be understood that the description of the structure and functions of the environment 100 is for illustrative purposes only and does not imply any limitation on the scope of the present disclosure.

[0038] As described above, solving user problems in a data center network currently relies on a large amount of manual services and there is no good automated solution. The reasons behind this come from both the network user side and the network engineer side.

[0039] From the perspective of network users, limited by network domain knowledge and necessary permissions, they usually lack a network-wide view and cannot access various network monitoring primitives. As a result, network users often need to turn to network engineers for manual assistance.

[0040] From the perspective of network engineers, they have various network monitoring primitives to monitor, a large amount of monitoring data to process, and too many but non-negligible network incidents to investigate. Therefore, extracting useful information from the vast data and responding to network users becomes very cumbersome and time-consuming, and highly dependent on the expertise of each network engineer. Therefore, we believe there is a gap between the network diagnosis needs of network users and the network monitoring primitives in a large data center network.

[0041] Embodiments of the present disclosure propose a solution for network diagnosis. According to an embodiment of the present disclosure, a user query for network diagnosis in natural language expression is obtained. Based on the user query, a target workflow including a set of machine-executable tasks is determined, and the set of machine-executable tasks is for the network diagnosis process of a target category of network diagnosis, and a diagnosis result is generated by executing the set of machine-executable tasks in the network diagnosis process. Example process of network diagnosis

[0042] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0043] Inspired by the technology of task-oriented dialogue systems in the field of natural language processing (NLP), a virtual assistant (chatbot) can be constructed to adopt network-related queries in natural language and utilize the experience from network engineers to perform appropriate diagnostic functions.

[0044] Figure 2 A diagram showing an example of the abstraction level in NETASSISTANT according to some embodiments of the present invention. As Figure 2 shown, the main objectives of this solution are divided into three hierarchical requirements, namely, the chat layer 210, the workflow layer 220, and the data layer 230. Network users can have chat requirements from the chat layer 210. Then, the chat layer 210 may require appropriate diagnostic functions from the workflow layer 220. The workflow layer 220 may require efficient data retrieved from network monitoring primitives, such as from the data layer 230. Conversely, each layer provides corresponding abstractions to its upper layer.

[0045] Figure 3 A diagram showing an example of the structure of the network diagnosis platform 110 according to some embodiments of the present disclosure. Referring to Figure 3 this, an example process of network diagnosis according to the present disclosure will be described in detail.

[0046] As Figure 3 shown, the network diagnosis platform 110 can be composed of three main functional modules, namely, the dialogue engine module 310, which provides a dialogue environment for users and understands the intent and slots from user queries; the workflow engine module 320, which converts the knowledge and experience from network engineers into workflows and processes appropriate workflows for each user query; and the data engine module 330, which manages all underlying distributed network monitoring data memories and provides high-performance data retrieval.

[0047] The user query 301 on network diagnosis can be obtained through the dialogue engine module 310. For example, the user query 301 can be expressed in natural language. For example, the dialogue engine module 310 can provide a dialogue environment for network users (such as user 102 as Figure 1 shown), and the dialogue environment presents a dialogue experience with multi-round question and answer. The input from user 102 can be plain text in natural language. The output form can be rich text, pictures, interactive components, and hyperlinks.

[0048] For the task-oriented dialogue system proposed according to the present disclosure, the key challenge is to establish a natural language understanding (NLU) component. The key objective of the NLU component is to understand slots and intents. The intent of each question can determine which diagnostic function (workflow) is used for network diagnosis, and the slots will be the input parameters.

[0049] The dialogue engine module 310 can provide session-based dialogue management to manage user questions. All user questions belonging to the same session will be converted into a list of questions for understanding. That is to say, the dialogue engine module 310 can obtain user queries based on the session-based dialogue. The user query can include one or more questions.

[0050] The first step for understanding is to parse the slots from the question. The slots in this article can be the names of any components in the data center network. Therefore, nouns from the network can be collected and classified into different types according to their roles in the network. For example, regions, countries, states / provinces, AZs, offices, buildings, pods, subnets / clusters, servers, switches, circuits, and optical modules.

[0051] These words can be divided into two groups of words. The first set of words contains the above terms and the spoken expressions of the terms themselves (network users may also know and directly use the terms in their questions), which can be called the spoken naming set (hereinafter also referred to as the first naming set). The second set of words contains the terms strictly defined in our data center network specifications, which can be called the specification naming set (hereinafter also referred to as the second naming set).

[0052] The specification naming set can be a subset of the spoken naming set. The mapping between the two sets can be obtained and / or established by the network diagnosis platform 110. These two sets can be used for slot parsing, standardization, and encoding. The spoken naming set can be first used to parse the slots from the user question. Then, the mapping between the two sets and the specification naming set can be used to standardize each word. Thereafter, each word can be used by using its category name. For example, the word "Santa Clara" can be encoded with "available_zone_#14" from the original sentence for further intent understanding and parsing of the parameter {"az_1": ["US-WEST-1"]} for workflow processing.

[0053] For example, by using the dialogue engine module 310, the network diagnosis platform 110 can parse multiple reference slots from the user query 301 (for example, one or more questions related to the user query 301). If the network diagnosis platform 110 determines that at least one first slot extracted from the multiple reference slots belongs to the first naming set, the network diagnosis platform 110 can determine at least one second slot corresponding to the at least one first slot from the second naming set based on the mapping between the first naming set and the second naming set. Then, the network diagnosis platform 110 can encode the at least one second slot with the corresponding category name.

[0054] It can process encoded questions for intent understanding (i.e., text classification). As an option, a word-level convolutional neural network (CNN) can be used to classify the category / intent of a given question. As another option, an LLM solution can be considered.

[0055] Generally, a machine learning model is used to classify at least one encoded second word slot to determine the target category / intent of network diagnosis. In a word-level CNN-based solution, a trained classifier can be used to classify the category / intent of a given question. To achieve this, a large number of questions can be collected for each intent as the training dataset of an offline word-level CNN classifier. At runtime after the encoding process, the classifier generates a classification score for the text for each intent and selects the matching intent based on a predefined threshold.

[0056] In an LLM-based solution, the powerful understanding skills of the LLM can be used to perform multiple-choice questions. Few-shot prompting can be used to add context learning, where different intents in the prompt are presented and the language model is made to select. An example of a prompt word can be expressed as: "Intent check_switch_traffic" for checking the traffic indicator of a switch. Intent check_host_network... Please select the corresponding intent for the following question: My devbox is unreachable. VMhost_ip_#19.

[0057] Since there may already be more than 100 intents, which would make the generated prompt statements too long, a relatively small set of intents (about 10) can be selected to shorten the prompt. To achieve this, vector embeddings are applied to the user question and the description of the intent, and a similarity-based search method can be utilized to select the intent.

[0058] For example, a set of prompt words associated with multiple reference categories corresponding to at least one encoded second word slot can be provided to the LLM. If the LLM selects a reference category based on the set of prompt words, the reference category can be determined as the target category of network diagnosis.

[0059] If the determined target category and the associated word slot are clear, the network diagnosis platform 110 can continue with the diagnostic function associated with the target category. If the target category is not clear, the network diagnosis platform 110 can interact with the user to indicate the missing supplementary information. If no target category is determined, the network diagnosis platform 110 can respond with a manual page containing how to use the network diagnosis platform 110 and examples of user queries.

[0060] Then, when the identified target category and the associated slots are clear, the network diagnosis platform 110 can determine a target workflow that includes a set of machine-executable tasks for the network diagnosis process of the target category of network diagnosis. The target workflow can be selected from the workflow library 326 in the workflow engine module 320, and the workflow engine module 320 can provide a framework for network engineers to create multiple workflows based on their expertise.

[0061] For example, the workflow engine module 320 can include atomic functions 322, a workflow converter 324, and a workflow library 326 as shown in Figure 3 which work together seamlessly to enable efficient workflow creation and execution. The expressed atomic functions 322 allow network engineers to create simple, flexible, and interactive diagnostic logics. The workflow converter 324 converts the logics into executable workflows, and the workflow library 326 supports the triggering and execution of workflows at runtime.

[0062] The atomic functions 322 form the building blocks of our workflow system, each encapsulating a specific method unique to a particular network entity and its metric data. Entities can include switches, links, interfaces, availability zones, etc. The decision method checks the metric data of the entity within a time range and determines whether any anomalies exist. Additionally, the atomic functions 322 can include optional action procedures to analyze specific scenarios in the workflow, such as pausing the service on a switch to check if it causes overload or triggering a route trace to show possible routing paths.

[0063] For all common network components, a set of atomic functions can be customized based on monitoring primitives to meet various diagnostic requirements. For example, based on the sFlow data of a link, it can be checked if the total utilization is high, or if there are any kinds of traffic violations of QoS rules, or if there is any traffic flow that experiences a sudden increase or decrease. There can be multiple algorithm implementations for the same function. The threshold details can be customized based on experience and expertise.

[0064] The set of atomic functions 322 can be configured to become a troubleshooting guide (TSG). The TSG includes multiple atomic functions that allow specifying the data entities to be analyzed using decision logic within a given time range. The TSG operates like a state machine, which exists at any given point in a specific stage (atomic function). The result of each atomic function determines the handover between stages.

[0065] The configuration of the TSG can be performed through a visual interface. This configuration includes the selection of atomic functions and thresholds, arranging them into a flowchart and specifying inputs such as switch names or time periods. Different versions of the TSG with the same name can be customized according to the input values. For example, switches from different vendors can have different check_switch_hardware TSGs. The TSG can be dynamically updated and adjusted as the network architecture changes.

[0066] Then, the workflow converter 324 can convert the TSG into executable workflow logic and add basic exception handling, supplementary background knowledge, associated network events, and common help entries. For example, all workflows will be added with logic to check for any known / ongoing network problems or changes, and recommend other tools for non-network problems. The executable workflows can be stored in the workflow library 326, which can be responsible for selecting and executing appropriate workflows based on the intent and parameters from the dialogue engine. Examples of the most commonly used workflows are listed below: Table 1

[0067] That is, the workflow engine module 320 can be used to create / provide / update multiple candidate workflows for processing network diagnostics on corresponding categories. Each candidate workflow can include a set of machine-executable tasks sorted by specific processing logic for the category.

[0068] The network diagnostic platform 110 can retrieve monitoring data from the data engine module 330 to execute the set of machine-executable tasks in the target workflow during the network diagnostic process. That is, the data engine module 330 can be responsible for providing abstractions and query entries for network monitoring data.

[0069] In some embodiments, the data engine module 330 can utilize two strategies to ensure effective data retrieval. If data sources that are relatively small in volume (such as switch configurations) can be stored and analyzed in their current form. These sources do not change frequently, so it is feasible to analyze them in real time.

[0070] Although the generated data may contain a large amount of normal expected behavior, only a small portion of the data is valuable for diagnosis. Thus, only abnormal data, which is marked as an alert, can be monitored. The execution of some atomic functions using high-volume data sources can be moved to become an always-on operator that generates alerts. Then the alerts are tracked, and in the case where the high-volume data source is queried, the alerts can be retrieved instead of the full data. For example, only a small portion of the syslog data, namely BGP flapping or line card errors, can actually help diagnose underlying network problems.

[0071] When generating a diagnostic result 302 by executing a set of machine-executable tasks during network diagnosis, the network diagnosis platform 110 can provide the diagnostic result 302 to a user from whom it receives a user query. The diagnostic result 302 can also be provided to a network engineer, who can solve network problems based on the diagnostic result 302 and / or a network problem handling device, which can solve network problems by executing appropriate machine-executable tasks.

[0072] Some examples of network diagnosis using the diagnostic platform 110 will be further described below.

[0073] As an example, when a user's SQL cluster encounters a connection anomaly of this instance, the user can query the IP address. The network diagnosis platform 110 can identify it as the IP of a virtual machine and execute the check_ias_ip_network workflow. After performing a multi-dimensional diagnosis on the virtual machine, the workflow discovers that the link between its physical machine and the Top-of-Rack (ToR) switch has started to have a significant number of CRC errors from SNMP monitoring data, and the ToR switch also reports CRC errors from the system log. Therefore, the network diagnosis platform 110 can respond with an unhealthy diagnostic conclusion and notify the network engineer on current standby of the problem.

[0074] As another example, a user can report a suspected network problem in the data center at around 2 AM because her hosted service has a fault wave and the logs show many connection errors. The network diagnosis platform 110 can execute the check_idc_network workflow and discover that there are spikes in packet drops and increased latency, which last for about 2 minutes according to the connection monitoring results. The involved IP pairs can be aggregated into a / 48 IPv6 subnet. The workflow also notices that there are several BGP withdrawal events, whose peers point to the same switch and there are several ECMP imbalance anomalies from nearby switches. Therefore, the network diagnosis platform 110 can conclude that there is a short-term network problem and the root cause is due to a power outage event.

[0075] As another example, a user can query the network status of his / her service to investigate the reason for performance degradation in the core service. The network diagnosis platform 110 can discover multiple ping drops and high-latency alerts between regions and configuration changes made. The network diagnosis platform 110 can also discover that the network team has already noticed and participated in the problem in advance, and has an initial conclusion that the configuration change may be the root cause. Therefore, the network diagnosis platform 110 can respond to the conclusion from the network team and recommend a downgrade to the service algorithm.

[0076] In this way, the network diagnosis solution proposed by the present disclosure can significantly reduce the workload of network users and network engineers, and provide excellent user experience and performance. Example Processes and Apparatuses

[0077] Figure 4 FIG. 4 shows a flowchart of a method 400 for network diagnosis according to some example implementations of the present disclosure. The method 400 may be implemented at a network diagnosis platform 110 as shown in Figure 1 FIG.

[0078] At block 410, a user query for network diagnosis in natural language expression is obtained. At block 420, based on the user query, a target workflow including a set of machine-executable tasks is determined, and the set of machine-executable tasks is for a target category of network diagnosis processes. The target category is associated with one or more network components. At block 430, a diagnosis result is generated by executing the set of machine-executable tasks during the network diagnosis process.

[0079] In some embodiments, obtaining the user query includes: obtaining the user query according to a session-based conversation, where the user query includes one or more questions.

[0080] In some embodiments, determining the target workflow includes: parsing a plurality of reference word slots from the user query; and if it is determined that at least one first word slot extracted from the plurality of reference word slots belongs to a first named set, based on the mapping between the first named set and a second named set, determining at least one second word slot corresponding to the at least one first word slot from the second named set; determining the target category of network diagnosis based on the at least one second word slot; and selecting the target workflow from a plurality of candidate workflows in a workflow library based on the target category.

[0081] In some embodiments, obtaining the mapping between the first named set and the second named set, where the mapping is determined based on a first set of network terms expressed in spoken language included in the first named set and a second set of network terms expressed in a specific manner included in the second named set.

[0082] In some embodiments, determining the target category of network diagnosis based on the at least one second word slot includes: encoding the at least one second word slot by using a corresponding category name; and determining the target category based on the classification of the at least one encoded second word slot by using a machine learning model.

[0083] In some embodiments, the machine learning model includes a large language model (LLM), and at least one encoded second word slot is classified by using the LLM in the following manner: determining a set of prompt words associated with a plurality of reference categories corresponding to at least one encoded second word slot; obtaining, based on the set of prompt words, a reference category selected by the LLM; and determining the reference category as the target category for network diagnosis.

[0084] In some embodiments, method 400 further includes obtaining a plurality of candidate workflows for processing network diagnosis on corresponding categories, where each candidate workflow includes a set of machine-executable tasks sorted by specific processing logic for the category.

[0085] In some embodiments, method 400 further includes identifying one or more operational network data as an alert set based on determining an anomaly in the one or more operational network data.

[0086] In some embodiments, generating a diagnostic result includes: checking whether at least one alert retrieved from the alert set is associated with the set of machine-executable tasks in the network diagnosis process; and if it is determined that at least one alert is associated with the set of machine-executable tasks in the network diagnosis process, executing the set of machine-executable tasks based on the network monitoring data corresponding to at least one alert.

[0087] In some embodiments, method 400 further includes providing the diagnostic result to a user from whom a user query is received. In some embodiments of the present disclosure, a non-transitory computer program product is provided. The non-transitory computer program product includes a computer-readable storage medium having program instructions embodied thereon. The program instructions can be executed by an electronic device to cause the electronic device to execute a method for network diagnosis. The method includes: obtaining a user query for network diagnosis expressed in natural language. Based on the user query, determining a target workflow including a set of machine-executable tasks for the network diagnosis process of the target category of network diagnosis, where the target category is associated with one or more network components. Generating a diagnostic result by executing the set of machine-executable tasks in the network diagnosis process. In some embodiments of the present disclosure, the method further includes other steps described in the present disclosure.

[0088] Figure 5 A block diagram of an electronic device 500 in which various embodiments of the present invention can be implemented is illustrated. It should be understood that Figure 5 The illustrated electronic device 500 is for illustrative purposes only and does not imply any limitation to the functions and scope of the present disclosure in any way. The electronic device 500 can be used to implement the above method 500. As Figure 5As shown, the electronic device 500 can be a general-purpose electronic device. The electronic device 500 can include at least one or more processors or processing units 510, a memory 520, a storage unit 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560.

[0089] The processing unit 510 can be a physical or virtual processor and can implement various processes based on the program 525 stored in the memory 520. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing ability of the electronic device 500. The processing unit 510 can also be referred to as a central processing unit (CPU), a microprocessor, a controller, or a microcontroller.

[0090] The electronic device 500 generally includes various computer storage media. Such media can be any media accessible to the electronic device 500, including but not limited to volatile and non-volatile media, or removable and non-removable media. The memory 520 can be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), or flash memory), or any combination thereof. The storage unit 530 can be any removable or non-removable media and can include machine-readable media such as a memory, a flash drive, a disk, or another other media that can be used to store information and / or data and can be accessed in the electronic device 500.

[0091] The electronic device 500 can also include additional removable / non-removable volatile / non-volatile storage media. Although not shown in Figure 5 , a disk drive for reading and / or writing to / from a removable and non-volatile disk and an optical disk drive for reading and / or writing to / from a removable non-volatile optical disk can be provided. In this case, each drive can be connected to a bus (not shown) via one or more data media interfaces.

[0092] The communication unit 540 communicates with another electronic device via a communication medium. Additionally, the functions of the components in the electronic device 500 can be implemented by a single computing cluster or multiple computer machines that can communicate via a communication connection. Thus, the electronic device 500 can operate in a networked environment using a logical connection to one or more other servers, networked personal computers (PCs), or other general network nodes.

[0093] The input device 550 can be one or more of various input devices, such as a mouse, a keyboard, a trackball, a voice input device, etc. The output device 560 can be one or more of various output devices, such as a display, a speaker, a printer, etc. With the aid of the communication unit 540, the electronic device 500 can also communicate with one or more external devices (not shown) (such as a storage device and a display device), where one or more devices enable a user to interact with the electronic device 500 or any device (such as a network card, a modem, etc.), enabling the electronic device 500 to communicate with one or more other electronic devices (if needed). Such communication can be performed via an input / output (I / O) interface (not shown).

[0094] In some embodiments, some or all components of the electronic device 500 can also be arranged in a cloud computing architecture, rather than integrated in a single device. In a cloud computing architecture, components can be provided remotely and work together to implement the functions described in this disclosure. In some embodiments, cloud computing provides computing, software, data access, and storage services, which will not require an end user to know the physical location or configuration of the system or hardware providing these services. In various embodiments, cloud computing uses a suitable protocol to provide services via a wide area network (such as the Internet). For example, a cloud computing provider provides an application via a wide area network, which can be accessed through a web browser or any other computing component. The software or components and corresponding data of the cloud computing architecture can be stored on a server at a remote location. The computing resources in a cloud computing environment can be consolidated or distributed at locations in a remote data center. The cloud computing infrastructure can provide services through a shared data center, although they appear as a single access point for users. Thus, the cloud computing architecture can be used to provide the components and functions described herein at a remote location from a service provider. Alternatively, they can be provided from a conventional server or directly installed or otherwise installed on a client device.

[0095] The functions described herein can be performed at least in part by one or more hardware logic components. For example, but not limited to, illustrative types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0096] The program code for the methods of implementing the subject matter described herein can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code can be executed entirely or partially on the machine, partially on the machine and partially on a remote machine, or entirely on the remote machine or server.

[0097] In the context of this disclosure, a machine-readable medium can be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be either a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0098] Moreover, although the operations are presented in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or in sequential order, or that all of the illustrated operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these details should not be construed as limitations on the scope of the subject matter described herein, but rather as descriptions of features that may be specific to particular embodiments. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, the various features described in a single implementation can also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0099] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0100] From the foregoing, it will be appreciated that the specific embodiments of the presently disclosed technology have been described for purposes of illustration, but various modifications can be made without departing from the scope of the disclosure. Accordingly, the technology of the disclosure is not limited to the appended claims.

[0101] Embodiments of the subject matter and the functional operations described in this disclosure can be implemented in various systems, digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer-readable medium for execution by, or to control the operation of, a data processing apparatus. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter implementing a machine-readable propagated signal, or combinations of one or more of them. The term “data processing unit” or “data processing apparatus” includes all apparatus, devices, and machines for processing data, e.g., including programmable processors, computers, or multiple processors or computers. The apparatus can also include code that creates an execution environment for the computer program being discussed, e.g., code constituting processor firmware, a protocol stack, a database management system, an operating system, or combinations of one or more of them, in addition to the hardware.

[0102] A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including a compiled or interpreted language, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program being discussed, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program can be deployed to execute on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0103] Processors suitable for executing computer programs include, for example, any one or more processors of general and special microprocessors and any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include or be operably coupled to receive data from or transfer data to one or more mass storage devices, which are used to store data, such as magnetic, magneto-optical or optical disks. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM and flash memory devices. The processor and memory can be supplemented by, or incorporated in, special logic circuitry.

[0104] The specification and the drawings are to be considered only as exemplary, where the exemplary devices are examples. As used herein, unless the context clearly dictates otherwise, the use of "or" is intended to include "and / or".

[0105] Although this disclosure contains many details, these should not be construed as limitations on the scope of any disclosure or claimed subject matter, but rather should be construed as descriptions of features that may be specific to particular embodiments of a particular disclosure. Certain features described in the context of separate embodiments in this disclosure may also be implemented combinatorially in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented separately in multiple embodiments or in any suitable sub-combination. Additionally, although features may be described above as acting in certain combinations and even initially claimed as such, in some cases, one or more features from a claimed combination may be excised from the combination, and the claimed combination may be directed to a sub-combination or variation of a sub-combination.

[0106] Similarly, although operations are shown in the drawings in a particular order, this should not be understood to require that such operations be performed in the particular order shown or in sequential order, or that all shown operations be performed to achieve a desired result. Additionally, the separation of various system components in the embodiments described in this disclosure should not be understood to require such separation in all embodiments. Only some embodiments and examples are described, and enhancements and variations can be made based on what is described and shown in this disclosure.

Claims

1. A network diagnosis method, comprising: Obtaining a user query for the network diagnosis expressed in natural language; Based on the user query, determining a target workflow including a set of machine-executable tasks, the set of machine-executable tasks being a network diagnostic process for a target category of the network diagnostic, wherein the target category is associated with one or more network components; as well as A diagnosis result is generated by executing the set of machine-executable tasks in the network diagnosis process.

2. The method according to claim 1, wherein obtaining the user query comprises: The user query is obtained according to a session-based dialog, wherein the user query includes one or more questions.

3. The method according to claim 2, wherein determining the target workflow comprises: parsing a plurality of reference word slots from the user query; If it is determined that at least one first word slot extracted from the plurality of reference word slots belongs to a first named set, determining at least one second word slot corresponding to the at least one first word slot from the second named set based on a mapping between the first named set and the second named set; determining the target category of the network diagnosis based on the at least one second word slot; as well as The target workflow is selected from a plurality of candidate workflows in a workflow library based on the target category.

4. The method according to claim 3, further comprising: The mapping between the first naming set and the second naming set is obtained, wherein the mapping is determined based on a first network terminology set expressed in spoken language and included in the first naming set and a second network terminology set expressed in a specific manner and included in the second naming set.

5. The method according to claim 3, determining the target category of the network diagnosis based on the at least one second word slot, comprising: encoding the at least one second word slot using a corresponding class name; The target category is determined based on a classification of at least one encoded second word slot using a machine learning model.

6. The method of claim 3, wherein the machine learning model comprises a Large Language Model (LLM), and wherein the at least one encoded second word slot is classified by using the LLM in the following manner: determining a set of cue words associated with a plurality of reference categories corresponding to the at least one encoded second word slot; as well as Acquire a reference category selected by the LLM based on the prompt word set; as well as The reference category is determined as the target category of the network diagnosis.

7. The method according to claim 3, further comprising: The plurality of candidate workflows for processing the network diagnosis on corresponding categories are obtained, wherein each candidate workflow includes a set of machine-executable tasks ordered by a specific processing logic for a category.

8. The method according to claim 1, further comprising: If an anomaly is determined for one or more operational network data, the one or more operational network data are identified as an alarm set.

9. The method of claim 8, wherein generating the diagnosis result comprises: checking whether at least one alarm retrieved from the alarm set is associated with the set of machine-executable tasks in the network diagnostic process; as well as If it is determined that the at least one alarm is associated with the set of machine-executable tasks in the network diagnosis process, the set of machine-executable tasks is executed based on the network monitoring data corresponding to the at least one alarm.

10. The method according to claim 1, further comprising: The diagnostic result is provided to a user from which the user query was received.

11. An electronic device comprising a computer processor coupled to a computer readable memory unit, the memory unit comprising instructions that when executed by the computer processor implement a network diagnostic method, the method comprising: Obtaining a user query for the network diagnosis expressed in natural language; Based on the user query, determining a target workflow including a set of machine-executable tasks, the set of machine-executable tasks being a network diagnostic process for a target category of the network diagnostic, wherein the target category is associated with one or more network components; as well as A diagnosis result is generated by executing the set of machine-executable tasks in the network diagnosis process.

12. The apparatus according to claim 11, wherein obtaining the user query comprises: The user query is obtained according to a session-based dialog, wherein the user query includes one or more questions.

13. The apparatus of claim 12, wherein determining the target workflow comprises: parsing a plurality of reference word slots from the user query; If it is determined that at least one first word slot extracted from the plurality of reference word slots belongs to a first named set, determining at least one second word slot corresponding to the at least one first word slot from the second named set based on a mapping between the first named set and the second named set; determining the target category of the network diagnosis based on the at least one second word slot; as well as The target workflow is selected from a plurality of candidate workflows in a workflow library based on the target category.

14. The apparatus according to claim 13, further comprising: The mapping between the first naming set and the second naming set is obtained, wherein the mapping is determined based on a first network terminology set expressed in spoken language and included in the first naming set and a second network terminology set expressed in a specific manner and included in the second naming set.

15. The apparatus of claim 13, wherein determining the target category of the network diagnosis based on the at least one second word slot comprises: encoding the at least one second word slot using a corresponding class name; classifying the at least one encoded second word slot by using a machine learning model; as well as Based on the classification of the at least one encoded second word slot, the target category is determined.

16. The apparatus of claim 13, wherein the machine learning model comprises a Large Language Model (LLM), and wherein classifying the at least one encoded second word slot comprises: determining a set of cue words associated with a plurality of reference categories corresponding to the at least one encoded second word slot; as well as Acquire a reference category selected by the LLM based on the prompt word set; as well as The reference category is determined as the target category of the network diagnosis.

17. The apparatus of claim 13, further comprising: The plurality of candidate workflows for processing the network diagnosis of a corresponding category are obtained, wherein each candidate workflow includes a set of machine-executable tasks ordered by a category-specific processing logic.

18. The apparatus of claim 11, further comprising: If an anomaly is determined for one or more operational network data, the one or more operational network data are identified as an alarm set.

19. The apparatus of claim 18, wherein generating the diagnostic result comprises: checking whether at least one alarm retrieved from the alarm set is associated with the set of machine-executable tasks in the network diagnostic process; as well as If it is determined that the at least one alarm is associated with the set of machine-executable tasks in the network diagnosis process, the set of machine-executable tasks is executed based on the network monitoring data corresponding to the at least one alarm.

20. A computer program product, the computer program product comprising a non-transitory computer-readable storage medium having program instructions implemented therein, the program instructions being executed by an electronic device to cause the electronic device to perform a network diagnostic method, the method comprising: Obtaining a user query for the network diagnosis expressed in natural language; determining a target workflow including a set of machine-executable tasks, the set of machine-executable tasks being directed to a network diagnostic process for a target category of the network diagnostic, wherein the target category is associated with one or more network components; as well as The diagnostic result is generated by executing the set of machine-executable tasks during the network diagnostic process.

Images