Network operation and maintenance method and apparatus based on large language model, and device and storage medium
Through a large language model, the user's intentions are identified and network operation and maintenance tasks are automatically arranged, and the problems of complexity and low efficiency of network operation and maintenance in the existing technology are solved, and the effect of simplifying operation and improving operation and maintenance efficiency is achieved.
Patent Information
- Application Number
- PCT/CN2024/076422
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-07-31
- Filing Date
- 2024-02-06
- Publication Date
- 2024-11-14
AI Technical Summary
The complexity of the existing network operation and maintenance system and the familiarity of the administrator lead to low network operation and maintenance efficiency. Administrators need to perform a large number of operations or input instructions to complete operation and maintenance tasks. The operation process is complicated, resulting in low network operation and maintenance efficiency.
The network operation and maintenance method based on the large language model is adopted, and by identifying the user's network operation and maintenance intentions, the operation and maintenance tasks are automatically arranged, the operation and maintenance process is simplified, the operation and maintenance difficulty is reduced, and the operation and maintenance efficiency is improved. The method includes using a large language model to identify intents from the user's conversation message, obtaining tasks to be executed, and outputting response results without multiple interactions.
It simplifies the network operation and maintenance operation process, reduces the difficulty and complexity of operation and maintenance, improves network operation and maintenance efficiency, and enhances users' dependence on network operation and maintenance systems and user stickiness.
Smart Images

Figure CN2024076422_14112024_PF_FP_ABST
Abstract
Description
Network operation and maintenance method, device, equipment and storage medium based on large language model
[0001] This application claims priority to Chinese patent application No. 202310525410.7, filed on May 10, 2023, entitled “A method, apparatus, and system for network operation and maintenance,” and to Chinese patent application No. 202310957610.X, filed on July 31, 2023, entitled “Network operation and maintenance method, apparatus, device, and storage medium based on a large language model.” The entire contents of the aforementioned patent applications are incorporated into this application by reference. Technical Field
[0002] The present application relates to the field of communication technology, and in particular to a network operation and maintenance method, apparatus, device, and storage medium based on a large language model. Background Art
[0003] In related technologies, multiple network operation and maintenance systems are typically deployed within a network, and administrators complete network operation and maintenance tasks by operating on each network operation and maintenance system. The difficulty and complexity of network operation and maintenance tasks depend on the functions of the network operation and maintenance systems and the administrator's familiarity with the network and the network operation and maintenance systems. For example, if the administrator is not familiar with the network and the network operation and maintenance systems, the difficulty and complexity of network operation and maintenance tasks will increase, resulting in low network operation and maintenance efficiency. In another example, the administrator must perform a large number of operations or input a large number of commands on the network operation and maintenance system in order to execute operation and maintenance tasks on the network through the network operation and maintenance system. This complex operation process leads to low network operation and maintenance efficiency.
[0004] Summary of the Invention
[0005] The embodiments of the present application provide a network operation and maintenance method, apparatus, device, and storage medium based on a large language model, which can improve network operation and maintenance efficiency. The technical solution is as follows:
[0006] On the first aspect, a network operation and maintenance method based on a large language model is provided. In this method, for an input dialogue message, the network operation and maintenance intention of the dialogue message is first determined based on the large language model; then, based on the network operation and maintenance intention, the network operation and maintenance task to be executed is obtained, and then the operation and maintenance task is executed and a dialogue response is output.
[0007] This method uses a large language model to identify the user's network operation and maintenance intention from the user's conversation messages, and then establishes the network operation and maintenance task that the user expects to perform on the network based on the network operation and maintenance intention. After the network operation and maintenance task is completed, the execution result of the network operation and maintenance task is provided to the client. In this process, the user only needs to provide conversation messages in the form of a conversation, without the need for multiple interactions with the client, which simplifies the operational process of network operation and maintenance, thereby improving network operation and maintenance efficiency.
[0008] In one possible implementation, the network operation and maintenance intention includes the operation and maintenance task intention and the operation and maintenance scenario intention. On this basis, the above-mentioned process of obtaining the network operation and maintenance task can be: based on the large language model and the application programming interface corresponding to the operation and maintenance scenario intention, the network operation and maintenance task is orchestrated for the operation and maintenance task intention to obtain task information, wherein the application programming interface here is used to complete the network operation task under the operation and maintenance scenario intention, and the task information is the orchestrated network operation and maintenance task, and the task information indicates the calling process of the application programming interface when executing the network operation and maintenance task.
[0009] Based on the possible implementation methods described above, network operation and maintenance tasks are automatically orchestrated through a large language model, eliminating the need for manual orchestration by users. This simplifies the network operation and maintenance process, reduces the difficulty of network operation and maintenance, and thus improves network operation and maintenance efficiency.
[0010] In one possible implementation, the operation and maintenance task intent includes multiple operation and maintenance task sub-intentions. On this basis, the above-mentioned process of obtaining task information based on the large language model can be: based on the large language model and the application programming interface, network operation and maintenance sub-tasks are orchestrated for multiple operation and maintenance task sub-intentions respectively to obtain multiple sub-task information; based on the multiple sub-task information, task information of the operation and maintenance task is obtained, wherein the sub-task information is the orchestrated network operation and maintenance sub-task, and the sub-task information indicates the calling process of the application programming interface when executing the network operation and maintenance sub-task.
[0011] When a network operation and maintenance task includes multiple network operation and maintenance subtasks, manual orchestration of the network operation and maintenance task is difficult and requires users to have a high level of understanding of various application program interfaces (APIs) of the network. Based on the above possible implementation methods, network operation and maintenance tasks are automatically orchestrated through a language model without the need for manual orchestration by users, which simplifies the network operation and maintenance process, reduces the difficulty of network operation and maintenance, and thus improves network operation and maintenance efficiency.
[0012] In one possible implementation, the large language model is not local, and the network operation and maintenance tasks are orchestrated by calling a remote large language model. For example, the description information of the application programming interface is first obtained, and then based on the operation and maintenance task intent, the first prompt statement and the description information of the application programming interface, the task orchestration prompt information is generated, and the task orchestration prompt information is sent to the device running the large language model, and then the task information returned by the device is received.
[0013] Based on the above possible implementation methods, by calling the large language model in other devices to complete the orchestration of network operation and maintenance tasks, the consumption of local computing resources can be reduced.
[0014] In one possible implementation, the description information of the application programming interface is obtained from the network management device, for example, by sending an information acquisition request to the network management device and receiving the description information returned by the network management device, wherein the information acquisition request indicates obtaining the description information of the application programming interface.
[0015] Based on the above possible implementation methods, by obtaining the current description information of the application programming interface from the network side, the large language model can orchestrate appropriate network operation and maintenance tasks based on the current description information of the application programming interface, thereby improving the rationality of the orchestrated network operation and maintenance tasks.
[0016] In one possible implementation, the operation and maintenance scenario is intended to be any one of a network planning scenario, a network construction scenario, a network maintenance scenario, a network optimization scenario, a network data query scenario, and a knowledge question and answer scenario.
[0017] Based on the above possible implementation methods, the methods provided in the embodiments of the present application can provide users with final question-and-answer results for dialogue messages in various operation and maintenance scenarios, thereby not limiting the operation and maintenance scenarios and providing users with operation and maintenance services for various operation and maintenance scenarios, thereby increasing users' dependence on the network operation and maintenance system and improving user stickiness of the network operation and maintenance system.
[0018] In one possible implementation, the network operation and maintenance intention includes a first operation and maintenance task sub-intention and a second operation and maintenance task sub-intention. On this basis, after determining such network operation and maintenance intention, the method also includes the following steps: sending the first operation and maintenance task sub-intention to the first device, and sending the second operation and maintenance task sub-intention to the second device; then, obtaining the execution result of the first network operation and maintenance sub-task from the first device, and obtaining the execution result of the second network operation and maintenance sub-task from the second device; based on the execution result of the first network operation and maintenance sub-task and the execution result of the second network operation and maintenance sub-task, outputting the dialogue response.
[0019] Based on the above possible implementation methods, by distributing each operation and maintenance task sub-intention in the operation and maintenance task intent to multiple devices, multiple devices process the operation and maintenance task sub-intention in parallel, thereby improving the processing efficiency of the operation and maintenance task intent and reducing the response time of the dialogue message.
[0020] In a possible implementation, the message processing device is located in the cloud, or the message processing device is located in an operation support system, a network element management system, or a network management system of the network.
[0021] In one possible implementation, the network operation and maintenance intention is determined by calling a large language model. For example, operation and maintenance intention prompt information is first generated based on the dialogue message and the second prompt statement. The second prompt statement is used to prompt the large language model to identify the network operation and maintenance intention of the dialogue message, and prompt the large language model to output the identified network operation and maintenance intention; then, the operation and maintenance intention prompt information is sent to the device running the large language model; and the network operation and maintenance intention returned by the device is received.
[0022] Based on the above possible implementation methods, the network operation and maintenance intention of the conversation message is identified through the first language model in other devices, which can reduce the consumption of local computing resources.
[0023] In a possible implementation, the other devices are located in the cloud, or the decision-making device is located in an operation support system, a network element management system, or a network management system of the network.
[0024] In a second aspect, a network operation and maintenance device based on a large language model is provided, which includes a functional module for executing the method provided in the first aspect or any optional manner of the first aspect.
[0025] In a third aspect, a computer device is provided, comprising a processor configured to execute program code so that the computer device executes to implement the method provided in the first aspect or various optional implementations of the first aspect.
[0026] In a fourth aspect, a computer-readable storage medium is provided, in which at least one program code is stored. The program code is read by a processor so that a computer device executes the method provided in the first aspect or various optional implementations of the first aspect.
[0027] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] FIG1 is an architecture diagram of a network operation and maintenance system based on a large language model provided in an embodiment of the present application;
[0029] FIG2 is a schematic diagram of a deployment of a network operation and maintenance system provided in an embodiment of the present application;
[0030] FIG3 is a schematic diagram of another deployment of a network operation and maintenance system provided in an embodiment of the present application;
[0031] FIG4 is a schematic diagram of another deployment of a network operation and maintenance system provided in an embodiment of the present application;
[0032] FIG5 is a schematic diagram of another deployment of a network operation and maintenance system provided in an embodiment of the present application;
[0033] FIG6 is a schematic diagram of another deployment of a network operation and maintenance system provided in an embodiment of the present application;
[0034] FIG7 is a schematic diagram of another deployment of a network operation and maintenance system provided in an embodiment of the present application;
[0035] FIG8 is a schematic diagram of another deployment of a network operation and maintenance system provided in an embodiment of the present application;
[0036] FIG9 is a schematic diagram of another deployment of a network operation and maintenance system provided in an embodiment of the present application;
[0037] FIG10 is a flow chart of a network operation and maintenance method based on a large language model provided in an embodiment of the present application;
[0038] FIG11 is a schematic diagram of a dialogue interface provided in an embodiment of the present application;
[0039] FIG12 is a flow chart of another network operation and maintenance method based on a large language model provided in an embodiment of the present application;
[0040] FIG13 is a flowchart of another network operation and maintenance method based on a large language model provided in an embodiment of the present application;
[0041] FIG14 is a schematic diagram of another dialogue interface provided in an embodiment of the present application;
[0042] FIG15 is a flowchart of another network operation and maintenance method based on a large language model provided in an embodiment of the present application;
[0043] FIG16 is a schematic diagram of the structure of a network operation and maintenance device based on a large language model provided in an embodiment of the present application;
[0044] FIG17 is a schematic structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0045] In order to facilitate understanding of the technical solution proposed in this application, some of the terms involved in this application are explained as follows:
[0046] A large language model (LLM) is a computer model capable of processing and generating natural language. LLMs are an AI technology based on deep learning that predicts the next word or sentence by learning the statistical patterns and semantic information of language data. LLMs typically consist of multiple neural networks (such as feedforward neural networks and recurrent neural networks). The feedforward neural network extracts features from the input text, while the recurrent neural network models the text sequence. By continuously training on large amounts of text data, LLMs gradually improve their understanding and prediction capabilities of natural language. As LLMs improve, they can be applied to a variety of application areas, such as natural language processing, machine translation, text generation, robotics, machine learning, speech recognition, and image processing. Therefore, LLMs are also called multimodal large language models (MLLMs).
[0047] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.
[0048] Figure 1 is an architecture diagram of a network operation and maintenance system based on a large language model provided in an embodiment of the present application. Referring to Figure 1, the network operation and maintenance system 100 includes a client 10, a message processing device 20, a decision device 30, and a network management device 50 of a network 40, wherein the network 40 generally refers to a device cluster composed of multiple devices, and these multiple devices may be network devices, and the network 40 composed of network devices is a communication network for providing communication functions. These multiple devices may also be computer devices, and the network 40 composed of multiple computer devices is a computer device cluster for providing computing functions. These multiple devices may also be storage devices, and the network 40 composed of multiple storage devices is a storage device cluster for providing data storage functions. The network 40 can be deployed in the cloud (private cloud or public cloud), or it may not be deployed in the cloud. Here, the embodiment of the present application does not limit the deployment environment of the network 40.
[0049] Network management device 50 is connected to network 40 and is used to manage and / or maintain the resources of network 40, such as hardware resources, software resources, and service resources. Hardware resources refer to the hardware resources of network 40, software resources refer to the software resources of network 40, and service resources refer to the resources of services provided by network 40 to the outside world, such as computing resources, storage resources, and communication resources. Communication resources refer to resources related to communication in network 40, such as network bandwidth, network latency, network packet loss, and network infrastructure. Network infrastructure refers to all hardware and software used to support organizational network communications. Taking network 40 as an example, a communication network composed of network devices, network management device 50 has the management capabilities to plan, build, operate, maintain, and optimize the network devices in network 40.
[0050] The client 10 is used to provide a human-computer interaction interface, receive dialogue messages input by a user (such as an administrator of the network 40) regarding the network 40, and connect to the message processing device 20 to provide the user's dialogue messages to the message processing device 20. The client 10 also displays the dialogue response of the message processing device 20 to the dialogue messages on the human-computer interaction interface to achieve human-computer dialogue. The human-computer interaction interface can be a portal based on question-and-answer interaction. The user can use the human-computer interaction interface to send questions or non-question messages regarding the network 40 to the message processing device 20 in natural language. The question or non-question messages are provided to the message processing device 20 in the form of dialogue messages, and the message processing device 20 responds to the question or non-question messages.
[0051] The message processing device 20 is a network operation and maintenance assistant in the network operation and maintenance system 100. The message processing device 20 is connected to the client 10, the decision-making device 30 and the network management device 50. The message processing device 20 is equivalent to the link (or bridge) between the client 10, the decision-making device 30 and the network management device 50. The message processing device 20 is used to process the dialogue message provided by the client. For the dialogue message provided by the client 10, the large language model in the decision-making device 30 is called to analyze the network operation and maintenance intention of the dialogue message, and dynamic data on the network 40 side related to the network operation and maintenance intention is obtained from the network management device 50. The dynamic data on the network 40 side of the network operation and maintenance intention is provided to the large language model in the decision-making device 30, and the large language model is called to orchestrate network operation and maintenance tasks for the network operation and maintenance intention. The network operation and maintenance tasks are sent to the network management device 50, and the network management device 50 executes the sent network operation and maintenance tasks to manage and / or maintain the resources of the network 40, and feeds back the execution results of the network operation and maintenance tasks to the message processing device 20. Furthermore, the message processing device 20 provides the execution results of the network operation and maintenance tasks to the client 10 as a dialogue response, and the client 10 displays the execution results in an information-based or graphical manner on the human-computer interaction interface.
[0052] The decision-making device 30 is the decision-maker in the network operation and maintenance system 100. The decision-making device 30 is used to provide a large language model in the field of network operation and maintenance. Taking the network 40 as a communication network as an example, the large language model is trained based on the training data of the communication network in the field of network operation and maintenance and the basic large language model in the communication field, so that the large language model has the ability to analyze the user intention of the dialogue message in the field of network operation and maintenance (i.e., network operation and maintenance intention), arrange network operation and maintenance tasks based on the network operation and maintenance intention, and convert the execution results of the network operation and maintenance tasks into user-understandable information. Among them, the training data includes text information involved in the process of users performing operation and maintenance work on the communication network, log data of the communication network, log analysis data, product codes, work orders, etc., and the information that users can understand includes information expressed in natural language, charts, recommendation information, guidance information, navigation information, etc.
[0053] In some possible implementations, the decision device 30 also has the ability to infer and train a large language model, and the large language model provided by the decision device 30 is inferred and trained by the model service 30. In other possible implementations, the decision device 30 does not have the ability to infer and train a large language model, and the large language model provided by the decision device 30 is inferred and trained by a device other than the decision device 30, and the large language model is simply deployed on the decision device 30.
[0054] Next, using network 40 as an example, we will describe the deployment of client 10, message processing device 20, decision-making device 30, and network management device 50, as well as the process of network operation and maintenance based on a large language model. For other types of networks 40, the deployment of client 10, message processing device 20, decision-making device 30, and network management device 50, as well as the process of network operation and maintenance based on a large language model, can refer to the communication network.
[0055] The client 10 is located in the operation support system (OSS) of the terminal or network 40, the message processing device 20 is located in the cloud, the OSS of the network 40, the element management system (EMS) or the network management system (NMS), the decision-making device 30 is located in the cloud, the OSS, the EMS or the NMS, and the network management device 50 is located in the cloud, the OSS, the EMS or the NMS. Based on this, each of the message processing device 20, the decision-making device 30 and the network management device 50 can be deployed separately in the cloud, the OSS, the EMS or the NMS, or can be deployed in combination in at least one of the cloud, the OSS, the EMS or the NMS.
[0056] The following describes possible deployment methods of various modules in the network operation and maintenance system 100 in conjunction with the deployment diagrams of the network operation and maintenance system shown in Figures 2 to 9.
[0057] For each deployment method shown in Figures 2 to 9, there is at least one client 10, and each client 10 can be deployed on a terminal or OSS. In the case of multiple clients 10, some of the multiple clients 10 may be deployed on the terminal, while others may be deployed on the OSS. Alternatively, all of the multiple clients 10 may be deployed on the terminal or on the OSS. The embodiments of this application do not limit the deployment location of each client 10.
[0058] As shown in Deployment Method 1 in Figure 2, the message processing device 20, decision-making device 30, and network management device 50 are all deployed in the EMS / NMS (representing EMS or NMS). Each client 10 aggregates the conversation messages it receives to the message processing device 20. For each conversation message, the message processing device 20 uses the large language model in the decision-making device 30 to analyze the network operation and maintenance intent of the conversation message. Based on the network operation and maintenance intent, the network management device 50 performs operations such as network planning, network construction, network maintenance, network optimization, or network data query on the network 40 (i.e., performs network operation and maintenance tasks on the network 40).
[0059] In deployment mode 2, as shown in Figure 3, message processing device 20 and decision-making device 30 are both deployed on the OSS. There are multiple EMS / NMSs, each of which houses at least one network management device 50. Message processing device 20 connects to the network management devices 50 in each EMS / NMS. Each client 10 aggregates received session messages to message processing device 20. For each session message, message processing device 20 uses the large language model in decision-making device 30 to analyze the network operation and maintenance intent of the session message. Based on this intent, it selects at least one network management device 50 from among the multiple network management devices 50 to perform network operation and maintenance tasks on network 40.
[0060] As shown in Deployment Mode 3 in Figure 4 , multiple EMS / NMSs are also deployed. Each EMS / NMS is equipped with at least one message processing device 20 and at least one network management device 50. Each message processing device 20 is connected to the network management device 50 within its respective EMS / NMS. A decision-making device 30 is deployed in the cloud and connected to the speech processor 20 within each EMS / NMS. Any client 10 sends a received conversation message to any message processing device 20 within any EMS / NMS. This message processing device 20 uses the large language model within the decision-making device 30 to analyze the network operation and maintenance intent of the conversation message. Based on this intent, the management manager 50 within the corresponding EMS / NMS performs network operation and maintenance tasks on the network 40.
[0061] As shown in Deployment Mode 4 in Figure 5 , message processing device 20 is deployed on the OSS. For example, there is one OSS and at least one message processing device 20. Each of these message processing devices 20 can be deployed within the same OSS. Alternatively, there may be multiple OSSs and multiple message processing devices 20, with at least one message processing device 20 deployed within each OSS. A decision-making device 30 is deployed in the cloud and connects to each message processing device 20 within each OSS, allowing these message processing devices 20 to share the large language model stored in the decision-making device 30. There are also multiple EMS / NMSs, each of which has at least one network management device 50 deployed within it. Each network management device 50 is connected to each message processing device 20 within each OSS. Any client 10 sends a received session message to any message processing device 20 within any OSS. The message processing device 20 uses the large language model stored in the decision-making device 30 to analyze the network operation and maintenance intent of the session message. Based on this intent, it selects at least one network management device 50 from among the multiple network management devices 50 to perform network operation and maintenance tasks on the network 40.
[0062] For any of the above deployment methods 1 to 4, the network 40 may be connected to multiple network management devices 50, and the multiple network management devices 50 have the same management authority over the network 40. For example, each network management device 50 can perform network planning, network construction, network maintenance, network optimization, or network data query operations on the network 40. In other deployment methods, the multiple network management devices 50 of the network 40 have different management levels, and network management devices of different management levels have different management authority over the network 40. For example, if the multiple network management devices 50 are divided into network management device 51 and network management device 52, the management level of network management device 51 is higher than the management level of network management device 52. Network management device 51 is the upper-level manager of network management device 52, and network management device 52 can have at least one.
[0063] As shown in Deployment Mode 5 in Figure 6 , a network management device 51 is deployed on the OSS. Multiple EMSs / NMSs each have at least one network management device 52 deployed. Network management device 51 connects to multiple network management devices 52, which are all connected to network 40. Message processing device 20 and decision-making device 30 are both deployed in the cloud. Message processing device 20 connects to network management device 51 and each of these network management devices 52. Session messages from each client 10 are aggregated to message processing device 20. Message processing device 20 uses the large language model in decision-making device 30 to analyze the network operation and maintenance intent of the session messages and orchestrate network operation and maintenance tasks based on these intents. Message processing device 20 then issues these network operation and maintenance tasks to network management device 51. Network management device 51 then issues these network operation and maintenance tasks to network 40. Alternatively, message processing device 20 issues these network operation and maintenance tasks to network management device 52, which then executes them on network 40. Of course, the message processing device 20 may not be connected to each network management device 52. Accordingly, the message processing device 20 performs network operation and maintenance tasks through the network management device 52. The network management device 51 and multiple network management devices 52 may also be deployed in OSS or EMS / NMS.
[0064] The above-described deployment method 5 is illustrated by an example in which network management devices 50 of different management levels share the same message processing device 20. In other embodiments, each network management device 50 of different management levels is configured with a message processing device 20. For example, in a network operation and maintenance system 100 including a network management device 51, multiple network management devices 52, multiple message processing devices 20, and multiple decision-making devices 30, the multiple message processing devices 20 are divided into a message processing device 21 and multiple message processing devices 22. The message processing device 21 and the network management device 51 have the same management level. The message processing device 21 is configured to the network management device 51. The message processing device 22 and the network management device 52 have the same management level. The multiple message processing devices 22 are each configured to a network management device 52, making the message processing device 21 the upper-level manager of each message processing device 22.
[0065] In deployment mode 6, as shown in Figure 7, the OSS is deployed with a message processing device 21, a decision-making device 31, and a network management device 51. The message processing device 21 is connected to the decision-making device 31 and the network management device 51, respectively. There are multiple EMS / NMSs, each of which is deployed with a message processing device 22, a decision-making device 32, and a network management device 52. Each message processing device 22 is connected to the decision-making device 32 and the network management device 52 within its own EMS / NMS. Clients 10 are connected to the message processing device 21, which is also connected to each message processing device 22. The network management device 51 is connected to each network management device 52. The session messages of each client 10 are aggregated to the message processing device 21. The message processing device 21 calls the large language model in the first decision device 31 to analyze the network operation and maintenance intention of the session message, and sends the network operation and maintenance intention to a message processing device 22. This message processing device 22 calls the large language model in the second decision device 32 in the EMS / NMS where it is located, and arranges network operation and maintenance tasks for the network operation and maintenance intention. The network operation and maintenance tasks are sent to the network management device 52, and the network management device 52 performs the network operation and maintenance tasks on the network 40. Alternatively, when the network operation and maintenance intent includes multiple operation and maintenance task sub-intents, the message processing device 21 sends the multiple operation and maintenance task sub-intents to the message processing devices 22 in different EMS / NMSs. Each message processing device 22 that receives the operation and maintenance task sub-intents calls the large language model in the second decision device 32 in its EMS / NMS to orchestrate a network operation and maintenance sub-task for the received operation and maintenance task sub-intent, calls the network management device 52 to which it is connected to execute the network operation and maintenance sub-task on the network 40, and returns the execution result of the network operation and maintenance sub-task to the message processing device 21. In this way, the message processing device 21 can obtain the execution results of the multiple network operation and maintenance sub-tasks and, based on the execution results of the multiple network operation and maintenance sub-tasks, returns the execution result of the network operation and maintenance task corresponding to the network operation and maintenance intent to the client 10. Alternatively, the message processing device 21 does not send the network operation and maintenance intent to the message processing device 22, but calls the large language model in the first decision device 31 to orchestrate the network operation and maintenance task for the network operation and maintenance intent, and executes the network operation and maintenance task by calling the application programming interface in the network management device 51.
[0066] The above-mentioned deployment method 6 is illustrated by an example in which message processing devices 20 at different management levels are configured with different large language models. In other embodiments, message processing devices 20 at different management levels share the same large language model. As shown in the deployment method 7 in Figure 8, a decision device 30 is deployed in the cloud, a message processing device 21 and a network management device 51 are deployed in the OSS, and multiple network management devices 52 with the same management level are divided into network management devices 521 and network management devices 522. There are multiple EMS / NMSs, some of which are deployed with message processing devices 22 and network management devices 521, while other EMS / NMSs are deployed with network management devices 522 and no message processing devices 22. The message processing devices 21 and 22 are both connected to the decision device 30 to share the large language model in the decision device 30. The session messages of each client 10 are aggregated to the message processing device 21. The message processing device 21 calls the large language model in the decision device 30 to analyze the network operation and maintenance intention of the session message. If the network operation and maintenance intention includes multiple operation and maintenance task sub-intentions, the message processing device 21 sends the multiple operation and maintenance task sub-intentions to the message processing devices 22 in different EMS / NMSs respectively. Each message processing device 22 that receives the operation and maintenance task sub-intention calls the large language model in the decision device 30 respectively, arranges the network operation and maintenance sub-tasks for the operation and maintenance task sub-intentions it receives, calls the network management device 521 in the EMS / NMS where it is located to execute the network operation and maintenance sub-tasks on the network 40, and returns the execution results of the network operation and maintenance sub-tasks to the message processing device 21, so that the message processing device 21 returns the execution results of the network operation and maintenance tasks corresponding to the network operation and maintenance intentions to the client 10 based on the execution results of the multiple network operation and maintenance sub-tasks. If the network operation and maintenance intent does not include multiple operation and maintenance task sub-intents, the message processing device 21 invokes the large language model in the decision device 30 to orchestrate the network operation and maintenance tasks for the network operation and maintenance intent, and executes the network operation and maintenance tasks by invoking the application programming interface in the network management device 521 or 522. Regarding deployment method 7, in other embodiments, similar to deployment method 6, each EMS / NMS may also be deployed with a message processing device 22. Accordingly, regarding deployment method 6, in other embodiments, similar to deployment method 7, there may also be situations where the EMS / NMS does not have a message processing device 22.
[0067] For any of the deployment methods 5 to 7, the example of dividing multiple network management devices 50 into network management devices of two management levels is used for explanation. In other embodiments, multiple network management devices 50 can also be divided into network management devices of more than two management levels. For network management devices of two adjacent management levels, the network management device with a higher management level is connected to the network management device with a lower management level. Here, the embodiment of the present application does not limit the number of management levels of the network management devices. For multi-level network management devices, when the message processing device 20 does not distinguish between management levels (such as deployment method 5), the message processing device 20 can be connected to the network management device with the highest management level in the multi-level network management devices, or connected to network management devices at all levels, and perform network operation and maintenance tasks through the network management device 20 to which it is connected. When the message processing device 20 distinguishes between management levels (such as deployment method 6 or deployment method 7), the network management devices 20 and the message processing device 20 of the same management level are deployed in the same subsystem, and the network management devices 20 and the message processing device 20 of the same management level are connected so that the message processing device 20 calls the network management device 20 of the same management level.
[0068] For any deployment method from deployment method 1 to deployment method 7, the example is shown as follows: neither OSS nor EMS / NMS is deployed in the cloud. In other embodiments, at least one of the OSS and EMS / NMS in any deployment method can also be deployed in the cloud. Taking deployment method 5 as an example, both OSS and EMS / NMS in deployment method 5 are deployed in the cloud, resulting in deployment method 8 shown in Figure 9.
[0069] Each of the above-mentioned message processing device 20, model processor 30, and network management device 50 can be implemented in the form of hardware, software, or a combination of software and hardware. Here, the embodiment of the present application does not limit the implementation method of the message processing device 20, model processor 30, and network management device 50. The message processing device 20, model processor 30, and network management device 50 can be independently integrated into different computer devices, or at least two of the message processing device 20, model processor 30, and network management device 50 can also be integrated into the same computer device. Here, the embodiment of the present application does not limit the integration method of the message processing device 20, model processor 30, and network management device 50.
[0070] Next, for the network operation and maintenance system introduced above, the implementation process of the network operation and maintenance method is introduced as follows.
[0071] FIG10 is a flow chart of a network operation and maintenance method based on a large language model provided in an embodiment of the present application. The method is applied to any of the network operation and maintenance systems introduced above, and the method includes the following steps.
[0072] 1001. The client displays a dialogue interface in which a dialogue message for the network is input.
[0073] The client is any of the clients 10 described above, and the network is the network 40 described above. The network-specific dialogue message instructs the execution of tasks related to network operation and maintenance. The dialogue message is expressed in natural language or structured statements. The embodiments of the present application do not limit the representation method of the dialogue message. When the dialogue message is expressed in natural language, it facilitates the user to communicate with the network operation and maintenance system in natural language, reducing the difficulty of the user's network operation and maintenance work.
[0074] The conversation interface is a human-computer interaction interface provided by the client. For example, FIG11 shows a schematic diagram of a conversation interface provided by an embodiment of the present application. As shown in FIG11 , conversation interface 1100 includes a conversation window 111 for inputting and displaying conversation messages. Assuming that the network includes multiple access points (APs), as shown in FIG11 , user A enters conversation message 11a, "Which AP is the busiest today?", in conversation window 111. Upon detecting conversation message 11a, the client displays it in conversation window 1111.
[0075] Figure 11 illustrates the example of a question-type dialog message. In other embodiments, the dialog message may be a non-question message, such as "Detect whether there are any faulty APs in the network." Furthermore, Figure 11 illustrates the example of a text message. In other embodiments, the user may also input a voice message.
[0076] 1002. The client outputs a network operation and maintenance request to the first message processing device based on the dialogue message. The network operation and maintenance request indicates a network operation and maintenance intention based on the dialogue message and performs a network operation and maintenance task on the network.
[0077] Among them, the first message processing device is any message processing device connected to the client in the network operation and maintenance system. When the network operation and maintenance system includes one message processing device, the first message processing device is this message processing device. When the network operation and maintenance system includes multiple message processing devices, if these multiple message processing devices do not distinguish between management levels, then the first message processing device is any message processing device among these multiple message processing devices. If these multiple message processing devices have different management levels, then the first message processing device is the message processing device with the highest management level among the multiple message processing devices (such as the message processing device 21 in Figure 7 or Figure 8).
[0078] The network operation and maintenance request includes the conversation message. When the client detects the conversation message in the conversation interface, it generates a network operation and maintenance request based on the conversation message and outputs the network operation and maintenance request to the first message processing device. In response, the first message processing device receives the network operation and maintenance request and executes the following step 1003 after receiving the network operation and maintenance request.
[0079] 1003. The first message processing device obtains a dialogue message in the network operation and maintenance request, and determines a network operation and maintenance intention based on the first language model and the dialogue message.
[0080] The network operation intention is the user's intention regarding network operation identified from the conversation message, that is, the operation related to the network operation that the user expects to perform (i.e., the network operation task). The network operation intention includes the operation task intention and the operation scenario intention. The operation task intention indicates the network operation task that the user expects to perform on the network, and the operation scenario intention refers to the operation scenario of the network operation task that the user expects to perform. Network operation involves multiple operation scenarios, such as network planning scenarios, network construction scenarios, network maintenance scenarios, network optimization scenarios, network data query scenarios, and knowledge question and answer scenarios. The operation scenario intention refers to any of these multiple operation scenarios. The network planning scenario refers to an operation scenario related to network planning, the network construction scenario refers to an operation scenario related to network construction, the network optimization scenario refers to an operation scenario related to network optimization, and the network data query scenario refers to an operation scenario for querying network data, such as network device configuration data, load data, network link data, etc. The knowledge question and answer scenario refers to an operation scenario in which questions about network operation knowledge are asked.
[0081] The first large language model is a large language model connected to the first message processing device. The first type of language model is deployed on the first message processing device and executed by the first message processing device, or the first large language model is deployed on a decision-making device (referred to as the first decision-making device) and executed by the first decision-making device. When the first large language model is deployed on the first decision-making device, the first message processing device uses the first large language model to identify the network operation and maintenance intent of the conversation message, such as in steps A1 to A3 below.
[0082] Step A1: The first message processing device sends an operation and maintenance intention prompt information to the first decision-making device. The operation and maintenance intention prompt information is used to prompt the first large language model to recognize the network operation and maintenance intention of the dialogue message.
[0083] The operation and maintenance intention prompt information includes the dialogue message and description information of various operation and maintenance scenarios.
[0084] In one possible implementation, the first message processing device generates operation and maintenance intention prompt information based on the dialogue message and the second prompt statement, and the second prompt statement is used to prompt the large language model to identify the network operation and maintenance intention of the dialogue message, and prompt the large language model to output the identified network operation and maintenance intention. Exemplarily, the second prompt statement is provided by an intention prompt template, and the first message processing device splices the dialogue message and the description information of various operation and maintenance scenarios in the intention prompt template of the large language model to obtain the operation and maintenance intention prompt information. Among them, the intention prompt template is a template for prompting the first large language model to output the network operation and maintenance intention. The intention prompt template includes a message field, a scenario field and a second prompt statement. The message field is used to store the dialogue message, the scenario field is used to store the description information of various network operation and maintenance scenarios, and the second prompt statement is the template content in the intention prompt template.
[0085] Take the intent prompt template of {User asks: X[]. Which of the following scenarios is this about? Return as a number, Y[]. What task does the user want to perform? Return as text} as an example. Assuming the conversation message is "Which AP devices are the busiest today?", and the network has multiple operation and maintenance scenarios such as network data query, viewing network reports, and viewing fault information, the first message processing device fills the conversation message in the message field X[] of the intent prompt template and fills the description information of these multiple operation and maintenance scenarios in the scenario field Y[] to splice the following operation and maintenance intent prompt information:
[0086] {User asks: Which APs are the busiest today?
[0087] Which of the following scenarios does this apply to? Return as a number.
[0088] 0. Other
[0089] 1. Online query
[0090] 2. View network reports
[0091] 3. Check the fault information.
[0092] What task does the user want to perform? Returns as text.
[0093] After obtaining the operation and maintenance intention prompt information, the first message processing device sends the operation and maintenance intention prompt information to the first decision-making device.
[0094] Step A2: The first decision-making device receives the operation and maintenance intention prompt information, and performs network operation and maintenance intention recognition on the dialogue message based on the first language model to obtain the network operation and maintenance intention.
[0095] The first decision-making device is a decision-making device to which the first message processing device is connected.
[0096] The first decision-making device inputs the operation and maintenance intention prompt information into the first large language model, and the first large language model performs operation and maintenance scenario intention recognition and operation and maintenance task intention recognition on the dialogue message based on the prompt of the operation and maintenance intention prompt information, and outputs the operation and maintenance scenario intention and operation and maintenance task intention of the dialogue message.
[0097] Still taking the example in step A1 as an example, assuming that the operation and maintenance scenario intention identified by the first language model is a network data query scenario, the first language model outputs the number "1" and the operation and maintenance task intention "to determine which APs carry more devices during data collection based on the channel usage of each AP."
[0098] Step A3: The first decision-making device outputs the network operation and maintenance intention to the first message processing device, and the first message processing device obtains the network operation and maintenance intention.
[0099] Based on steps A1 to A3, by calling the first language model in the decision-making device to identify the network operation and maintenance intention of the dialogue message, the consumption of local computing resources can be reduced.
[0100] The above steps A1 to A3 are explained by taking the example of calling the first language model once to obtain the network operation and maintenance intention. In other embodiments, the first message processing device obtains the network operation and maintenance intention by calling the first language model multiple times, such as the following steps B1 to B4.
[0101] Step B1: The first message processing device sends scenario intention prompt information to the first decision-making device, where the scenario intention prompt information is used to prompt the first large language model to be called to identify the operation and maintenance scenario intention of the dialogue message.
[0102] Among them, the scenario intention prompt information includes the dialogue message and description information of various operation and maintenance scenarios.
[0103] After receiving the conversation message, the first message processing device combines the conversation message and descriptions of multiple operation and maintenance scenarios into a scenario prompt template of the first language model to obtain scenario intent prompt information. The scenario intent prompt template is used to prompt the first language model to output the operation and maintenance scenario intent. The scenario intent prompt template includes a message field, a scenario field, and a third prompt statement. The third prompt statement is the template content in the scenario intent prompt template and is used to prompt the first language model to output the operation and maintenance scenario intent of the conversation message.
[0104] Take the scenario intent prompt template as {User asks: X[]. Which of the following scenarios is this about? Return as a number, Y[]} as an example. Assuming the conversation message is "Which AP devices are the busiest today?", and the network has multiple operation and maintenance scenarios such as network data query, viewing network reports, and viewing fault information, the first message processing device fills the conversation message in the message field X[] of the scenario intent prompt template and fills the description information of multiple operation and maintenance scenarios in the scenario field Y[] to splice the following scenario intent prompt information:
[0105] {User asks: Which APs are the busiest today?
[0106] Which of the following scenarios does this apply to? Return as a number.
[0107] 0. Other
[0108] 1. Online query
[0109] 2. View network reports
[0110] 3. Check the fault information.
[0111] After obtaining the scenario intention prompt information, the first message processing device sends the scenario intention prompt information to the first decision-making device.
[0112] Step B2: The first decision-making device receives the scenario intention prompt information, and based on the first large language model, performs operation and maintenance scenario intention recognition on the dialogue message to obtain the operation and maintenance scenario intention, and outputs the operation and maintenance scenario intention to the first message processing device. The first message processing device receives the operation and maintenance scenario intention.
[0113] In one possible implementation, the first language model determines that an operation and maintenance scenario is to be identified for the conversation message based on the prompt statement in the scenario intent prompt information. Based on the learned association between user question samples and operation and maintenance scenarios, the model identifies the operation and maintenance scenario corresponding to the conversation message and outputs a scenario classification label for the operation and maintenance scenario corresponding to the conversation message. Continuing with the example in step B1, assuming the scenario classification label for the network data query scenario is the number "1," and the first language model identifies the operation and maintenance scenario intent as a network data query scenario, the model outputs the number "1."
[0114] The user question samples are the conversation messages used in the training process of the first language model. Through training, the first language model learns the correlation between the user question samples and the operation and maintenance scenarios. The following is an example of the process of training the first language model to learn this correlation:
[0115] For the sake of convenience, the device for training the first language model is called a training device. The training device obtains scenario classification labels for multiple operation and maintenance scenarios. The scenario classification labels refer to an operation and maintenance scenario respectively. For example, each scenario classification label is the number of an operation and maintenance scenario. Different operation and maintenance scenarios have different numbers, that is, the number is the scene identifier. For example, the knowledge question and answer scenario is numbered "1", and the troubleshooting scenario is numbered "2".
[0116] For each operation and maintenance scenario, the training device also obtains a query set for the operation and maintenance scenario. The query set includes multiple user question samples, which are dialogue messages related to the operation and maintenance scenario. For example, the query set for the knowledge question and answer scenario includes three user question samples, such as ["What is an AP?", "What are the characteristics of an AP?", and "How to configure an AP?"]. For another example, the query set for the troubleshooting scenario includes three user question samples, such as ["Which APs are the busiest today?", "How to check the number of AP connections?", and "What to do if the AP signal is unstable?"]. This example uses the query set including three user question samples as an example. In specific implementations, the query set may include more or less than three user question samples. The embodiments of the present application do not limit the number of user question samples in the query set.
[0117] For each operation and maintenance scenario, the training device adds the scenario classification label of the operation and maintenance scenario to each user question sample in the query set of the operation and maintenance scenario, and obtains multiple input sequences corresponding to the operation and maintenance scenario. Each input sequence includes a user question sample and the scenario classification label of the operation and maintenance scenario. Taking the user question sample "What is AP" as an example, the number "1" in the scenario classification label of the knowledge question and answer scenario is added to the user question sample "What is AP", and an input sequence "What is AP (1)" is obtained.
[0118] After obtaining input sequences corresponding to multiple operation and maintenance scenarios, the training device inputs the input sequences corresponding to the multiple operation and maintenance scenarios into the first language model. The first language model infers the scene classification labels of the user question samples in the multiple input sequences based on the model parameters to obtain inferred scene classification labels. Based on the error between the inferred scene classification labels and the scene classification labels in the input sequence, the model parameters are adjusted so that the error is reduced when the next inference is performed based on the adjusted model parameters. By cyclically executing the model parameter adjustment process, the model parameters are gradually refined. When the training is completed, the first language model containing the precise model parameters is used to learn the association between the user question samples and the scene classification labels, thereby achieving accurate operation and maintenance scenario inference. Based on the scene classification labels of the user question samples in the input sequence, the first language model can infer the corresponding scene classification labels. This provides appropriate processing procedures for different operation and maintenance scenarios. Since the scene classification labels are used for labeling, the problem of the first language model confusing the scene classification labels can also be effectively avoided.
[0119] When the first large language model is subsequently used, such as in step B2, after the first large language model outputs the scenario classification label of the dialogue message, the first decision-making device can return the scenario classification label to the first message processing device, and the first message processing device determines the name of the operation and maintenance scenario corresponding to the scenario classification label based on the correspondence between the scenario classification label and the name of the operation and maintenance scenario, thereby determining the operation and maintenance scenario intention of the dialogue message. Alternatively, the first decision-making device determines the name of the operation and maintenance scenario corresponding to the scenario classification label based on the correspondence between the scenario classification label and the name of the operation and maintenance scenario, and returns the name of the operation and maintenance scenario to the first message processing device. In one possible implementation, the correspondence between the scenario classification label and the name of the operation and maintenance scenario can be recorded in a scenario classification dictionary, and the scenario classification label output by the first large language model is decoded by the scenario classification dictionary to map the scenario classification label to the name of the operation and maintenance scenario, thereby determining the name of the operation and maintenance scenario corresponding to the scenario classification label. For example, if the scenario classification label output by the first language model is "1", the name of the decoded operation and maintenance scenario is "knowledge question and answer"; if the scenario classification label output by the first language model is "2", the name of the decoded operation and maintenance scenario is "troubleshooting".
[0120] Step B3: The first message processing device sends task intention prompt information to the first decision-making device based on the dialogue message. The task intention prompt information is used to prompt the first large language model to identify the operation and maintenance task intent of the dialogue message under the operation and maintenance scenario intent.
[0121] The task intention prompt information includes the dialogue message and the operation and maintenance scenario intention.
[0122] After obtaining the operation and maintenance scenario intent, the first message processing device splices the conversation message and the operation and maintenance scenario intent in the task intent prompt template of the first large language model to obtain the task intent prompt information. Among them, the task intent prompt template is a template used to prompt the first large language model to output the operation and maintenance task intent. The task intent prompt template includes a message field, a scenario intent field, and a fourth prompt statement. The message field is used to store the conversation message, the scenario intent field is used to store the operation and maintenance scenario intent, and the fourth prompt statement is the template content in the task intent prompt template. The fourth prompt statement is used to prompt the first large language model to output the operation and maintenance task intent of the conversation message under the operation and maintenance scenario intent.
[0123] The task intent prompt template is {Question X[] for the user. What kind of task does the user want to perform in scenario Y[]? Return in text form}. Assuming the conversation message is "Which AP devices are the busiest today?" and the operation and maintenance scenario intent is a network data query scenario, the first message processing device fills the conversation message in the message field X[] of the task intent prompt template and the operation and maintenance scenario intent in the scenario intent field Y[] to create the following task intent prompt information:
[0124] {In response to the user's question "Which AP devices are the busiest today?"
[0125] What kind of task does the user want to perform in the network data query scenario? Return as text.
[0126] After acquiring the task intention prompt information, the first message processing device sends the task intention prompt information to the first decision-making device.
[0127] In other embodiments, for prompt information that includes descriptive information of multiple operation and maintenance scenarios (such as operation and maintenance intention prompt information, scenario intention prompt information, and task intention prompt information), if the first language model has learned the descriptive information of various operation and maintenance scenarios, then the scene field in the prompt template corresponding to these prompt information belongs to the template content, and when splicing the prompt information, there is no need to splice the descriptive information of multiple operation and maintenance scenarios in the prompt template.
[0128] Step B4: The first decision-making device receives the task intention prompt information, and based on the first large language model, identifies the operation and maintenance task intention of the dialogue message to obtain the operation and maintenance task intention, outputs the operation and maintenance task intention to the first message processing device, and the first message processing device obtains the operation and maintenance task intention.
[0129] For example, the first decision-making device inputs the task intention prompt information into the first large language model, and the first large language model identifies the operation and maintenance task intention under the operation and maintenance scenario intention of the dialogue message based on the prompt of the task intention prompt information, outputs the operation and maintenance task intention, and the first decision-making device sends the operation and maintenance task intention to the first message processing device.
[0130] The first message processing device receives the operation and maintenance scenario intention and the operation and maintenance task intention of the dialogue message, that is, obtains the network operation and maintenance intention of the dialogue message.
[0131] 1004. The first message processing device obtains a network operation and maintenance task based on the network operation and maintenance intention.
[0132] For the first message processing device, network operation and maintenance tasks are divided into knowledge question-and-answer type network operation and maintenance tasks and non-knowledge question-and-answer type network operation and maintenance tasks. Knowledge question-and-answer type network operation and maintenance tasks do not require the first message processing device to interact with the network management device, while non-knowledge question-and-answer type network operation and maintenance tasks do require the first message processing device to interact with the network management device. Based on this, if the operation and maintenance scenario intent of the dialogue message is a knowledge question-and-answer scenario, the first message processing device will determine the operation and maintenance task intent of the dialogue message as a knowledge question-and-answer type network operation and maintenance task, query the knowledge base for information that can answer the operation and maintenance task intent, and use the queried information as the execution result of the network operation and maintenance task.
[0133] If the operation and maintenance scenario intended in the conversation message is not a knowledge question and answer scenario, the first message processing device obtains a non-knowledge question and answer network operation and maintenance task based on the network operation and maintenance intent and the first language model. The process of obtaining a non-knowledge question and answer network operation and maintenance task is described below.
[0134] For any operation and maintenance scenario other than the knowledge question and answer scenario in multiple operation and maintenance scenarios, the network management device of the network provides at least one application programming interface (API) for the operation and maintenance scenario. The API is used to complete the network operation and maintenance tasks under the operation and maintenance scenario. Different APIs complete different network operation and maintenance tasks. The process of executing network operation and maintenance tasks on the network is also the process of calling the API under the operation and maintenance scenario. Based on this, after obtaining the operation and maintenance task intention and the operation and maintenance scenario intention, the first message processing device choreographs the network operation and maintenance task for the operation and maintenance task intention based on the first language model and the API corresponding to the operation and maintenance scenario intention, obtains task information, and automatically choreographs the execution process of the network operation and maintenance task through the first language model. There is no need for manual choreographing by the user, which simplifies the network operation and maintenance process, reduces the difficulty of network operation and maintenance, and thus can improve network operation and maintenance efficiency.
[0135] Among them, the task information indicates the calling process of the corresponding API when executing the network operation and maintenance task. The API required to complete the network operation and maintenance task or complete the operation and maintenance task intention is called the first API. The first API may be all APIs corresponding to the operation and maintenance scenario intention, or it may be part of the API corresponding to the operation and maintenance scenario intention, that is, there is at least one first API. The task information includes the output parameters of the at least one first API when executing the network operation and maintenance task and the identifier of the at least one first API. In the case of calling multiple first APIs, the task information also includes the calling order of multiple first APIs when executing the network operation and maintenance task, or the task information does not include the calling order of the first API, and the position of the input parameters / identifiers of each first API in the task information is used to indicate the calling order of each first API. The task information is also a calling chain of at least one first API. When the calling chain is completed, the network operation and maintenance task is completed, and the information output by the calling chain is the execution result of the network operation and maintenance task.
[0136] In the case where the first large language model is deployed on the first decision-making device, the first message processing device obtains the task information by calling the first large language model, such as the following steps C1 to C4.
[0137] Step C1: The first message processing device obtains description information of the API corresponding to the operation and maintenance scenario intention from the first network management device.
[0138] The API description information includes functional description information and / or detection description information of the API. The functional description information is used to describe the function of the API, and the detection description information is used to describe network data detected by the API in the network, such as load data of network devices or network link data, such as bandwidth, latency, channel utilization, etc. The API corresponding to the operation and maintenance scenario intent is each API corresponding to the operation and maintenance scenario intent, or is the first API among the APIs corresponding to the operation and maintenance scenario intent.
[0139] Taking obtaining the description information of each API corresponding to the operation and maintenance scenario intention as an example, the first message processing device sends a first information acquisition request to the first network management device of the network, and the first information acquisition request indicates obtaining the description information of each API corresponding to the operation and maintenance scenario intention, wherein the first network management device is any network management device of the network to which the first message processing device is connected, and the first information acquisition request includes the identifier of the operation and maintenance scenario intention. The first network management device receives the first information acquisition request, and outputs the description information of each API corresponding to the operation and maintenance scenario intention based on the first information acquisition request. For example, the first network management device calls each API corresponding to the operation and maintenance scenario intention based on the first information acquisition request, obtains the description information of each API from each API, and sends the description information of each API corresponding to the operation and maintenance scenario intention to the first message processing device.
[0140] Taking the example of obtaining the description information of the first API corresponding to the operation and maintenance scenario intent, assuming that the operation and maintenance scenario intent corresponds to multiple APIs of different interface types, with at least one API of each interface type, the first message processing device first determines the first interface type from the multiple APIs of different interface types, and then obtains the description information of the first API from the first network management device, using each API of the first interface type as the first API. The first interface type is the interface type of the API required by the operation and maintenance task intent. For example, steps C11 to C13 are described below.
[0141] C11. The first message processing device sends interface type prompt information to the first decision-making device based on the dialogue message and the operation and maintenance scenario intention. The interface type prompt information is used to prompt the first large language model to output the interface type required to complete the operation and maintenance task intention of the dialogue message in the operation and maintenance scenario intention.
[0142] The interface type prompt information includes the dialogue message / the operation and maintenance task intention, and the interface prompt information also includes multiple interface types corresponding to the operation and maintenance scenario intention.
[0143] The first message processing device splices the dialogue message and the multiple interface types corresponding to the operation and maintenance scenario intention in the interface prompt template of the first large language model to obtain the interface prompt information. Among them, the interface prompt template is a template for prompting the first large language model to output the interface type required to complete the operation and maintenance task intention in the operation and maintenance scenario intention. The interface prompt template includes a message field, an interface type field and a fifth prompt statement. The message field is used to store the operation and maintenance task intention, the interface type field is used for the multiple interface types corresponding to the operation and maintenance scenario intention, and the fifth prompt statement is the template content in the interface prompt template. The fifth prompt statement is used to prompt the first large language model to output the interface type required to complete the operation and maintenance task intention in the operation and maintenance scenario intention.
[0144] Taking the interface prompt template as {To answer the question "{X[]}", which of the following interface types should be selected for API use? Return in digital form. Y[]} as an example, assuming that the operation and maintenance scenario intention is a network data query scenario, the network data query scenario corresponds to multiple interface types such as interface types for AP devices, switch devices, and gateway devices. After determining the network data query scenario, the first message processing device fills query (i.e., operation and maintenance task intention or dialogue message) in the message field X[] of the interface prompt template and fills multiple interface types corresponding to the network data query scenario in the interface type field Y[] to splice out the following interface type prompt information:
[0145] To answer the question "{query}", which category should you choose from for API usage? Return as a number
[0146] 1. About AP devices
[0147] 2. About switch equipment
[0148] 3. About gateway devices}.
[0149] After obtaining the interface type prompt information, the first message processing device sends the interface type prompt information to the first decision device.
[0150] C12. The first decision-making device receives the interface type prompt information, predicts the interface type required to complete the operation and maintenance task intention based on the first large language model, obtains the first interface type, and sends the first interface type to the first message processing device.
[0151] Exemplarily, the first decision-making device inputs the interface type prompt information into the first large language model. Based on the prompt information of the task intent, the first large language model determines a first interface type from multiple interface types corresponding to the operation and maintenance scenario intent, outputs the first interface type, and sends the first interface type to the first message processing device. Still using the interface type prompt information in C11 as an example, assuming that the first interface type determined by the first large language model is an interface type for an AP device, the first large language model outputs the number "1," and the first decision-making device returns the interface type "1" to the first interlocutor.
[0152] C13. The first message processing device receives the first interface type and obtains description information of the first API of the first interface type from the first network management device.
[0153] The first message processing device sends a second information acquisition request to the first network management device, where the second information acquisition request indicates obtaining description information of the API of the first interface type corresponding to the operation and maintenance scenario intention, and the second information acquisition request includes an identifier of the operation and maintenance scenario intention and the first interface type. The first network management device receives the second information acquisition request and outputs description information of each API corresponding to the first interface type based on the second information acquisition request. For example, the first network management device calls each API of the first interface type corresponding to the operation and maintenance scenario intention based on the second information acquisition request, obtains description information of each API of the first interface type from each API of the first interface type, and sends the description information of each API of the first interface type to the first message processing device. The first message processing device receives the description information of each API of the first interface type (i.e., the first API).
[0154] The first and second information acquisition requests are different information acquisition requests, both used to obtain description information of the API for the intended operation and maintenance scenario. Based on step C1 above, by obtaining the description information of the API for the intended operation and maintenance scenario from the network management device, the first language model can orchestrate appropriate network operation and maintenance tasks based on the description information of the API for the intended operation and maintenance scenario, thereby improving the rationality of the orchestrated network operation and maintenance tasks.
[0155] The above-mentioned step C1 is explained by taking the example of the first message processing device obtaining the description information of the API of the operation and maintenance scenario intent from the network management device. In other embodiments, the first message processing device stores the description information of the API of the operation and maintenance scenario intent. In this case, the first message processing device does not need to obtain the description information of the API corresponding to the operation and maintenance scenario intent from the first message processing device. For example, the first message processing device stores description information of APIs for various operation and maintenance scenarios. If the scenario intent of the dialogue message is an operation and maintenance scenario intent, the description information of the API of the scenario intent is obtained from the description information of the APIs of the multiple operation and maintenance scenarios stored locally, without obtaining the detection description information of the API corresponding to the operation and maintenance scenario intent from the first network management device. Therefore, this step C1 is an optional step and is represented by a dotted line in the accompanying drawings. In some embodiments, the first message processing device stores the scenario information of each operation and maintenance scenario. The scenario information of the operation and maintenance scenario includes the scenario description information of the operation and maintenance scenario, the description information of the API, and the prompt template related to the operation and maintenance scenario, so that the first message processing device can obtain this information locally when needed.
[0156] Step C2: The first message processing device sends task scheduling prompt information to the first decision-making device based on the description information of the API corresponding to the operation and maintenance scenario intention. The task scheduling prompt information is used to prompt the first language model to schedule a network operation and maintenance task for the operation and maintenance task based on the API corresponding to the operation and maintenance scenario intention.
[0157] The task scheduling prompt information includes the operation and maintenance task intention and the description information of the API corresponding to the operation and maintenance scenario intention.
[0158] In one possible implementation, the first message processing device generates the task orchestration prompt information based on the operation and maintenance task intent, the first prompt statement, and the description information of the API corresponding to the operation and maintenance scenario intent, wherein the first prompt statement is used to prompt the large language model to orchestrate the network for the operation and maintenance task intent based on the API corresponding to the operation and maintenance scenario intent, and prompt the large language model to output the task information of the orchestrated network operation and maintenance task. Exemplarily, the first prompt statement is provided by the task orchestration prompt template of the large language model, and the first message processing device splices the operation and maintenance task intent and the description information of the API corresponding to the operation and maintenance scenario intent in the task orchestration prompt template to obtain the task orchestration prompt information. Among them, the task orchestration prompt template is a template for prompting the first large language model to orchestrate a network operation and maintenance task, and the task orchestration prompt template includes a message field, a description field, and a second prompt statement. The message field is used to store the operation and maintenance scenario intent, the description field is used to store the description information of the API corresponding to the operation and maintenance scenario intent, and the second prompt statement is the template content in the task orchestration prompt template.
[0159] Taking the API corresponding to the operation and maintenance scenario intent as an example, assuming that the task orchestration prompt template is {{Regarding the X[] description of these APIs, please tell me which API {Y[]} should call? What parameters need to be passed? Give the answer directly, do not give the analysis process}, the first message processing device fills the query (that is, the operation and maintenance task intent or dialogue message) in the message field X[] of the task orchestration prompt template, and fills the operation and maintenance task intent (that is, the description information of the API corresponding to the operation and maintenance scenario intent) in the description field Y[] to splice out the following task orchestration prompt information:
[0160] {{About the API of AP to express the operation and maintenance task intention},
[0161] Which API should be called for {query}? What parameters should be passed? Please give the answer directly without providing the analysis process.
[0162] After obtaining the task scheduling prompt information, the first message processing device sends the task scheduling prompt information to the first decision-making device.
[0163] Step C3: The first decision-making device receives the task arrangement prompt information, and arranges the network operation and maintenance task for the operation and maintenance task intention based on the first language model and the API corresponding to the operation and maintenance scenario intention to obtain task information.
[0164] The first decision-making device inputs the task orchestration prompt information into the first large language model. The first large language model determines the first APIs used to complete the operation and maintenance intention in the operation and maintenance scenario intention, the calling order and input parameters of each first API based on the prompt of the task orchestration prompt information and the description information of the API corresponding to the operation and maintenance scenario intention and the operation and maintenance task intention. Based on the calling order and input parameters of each first API, the execution process of the network operation and maintenance task is orchestrated and task information is output. The task information includes the input parameters and calling order of each API. If the same first API is called multiple times, the task information also includes the calling order of the first API each time it is called.
[0165] In one possible implementation, the operation and maintenance task intention includes multiple operation and maintenance task sub-intentions. Accordingly, the network operation and maintenance task corresponding to the operation and maintenance task intention includes multiple network operation and maintenance sub-tasks. The multiple operation and maintenance task sub-intentions correspond to different network operation and maintenance sub-tasks. The network operation and maintenance sub-tasks are used to complete the corresponding operation and maintenance task sub-intentions.
[0166] In the case where the operation and maintenance task intent includes multiple operation and maintenance task sub-intentions, the first decision-making device arranges network operation and maintenance sub-tasks for multiple network operation and maintenance sub-tasks based on the first large language model and the API corresponding to the operation and maintenance scenario intent, obtains multiple sub-task information, and obtains the task information based on the multiple sub-task information. At this time, the task information includes multiple sub-task information, and the sub-task information indicates the calling process of at least one first API when executing the network operation and maintenance sub-task. In the case where the network operation and maintenance sub-task needs to call multiple first APIs, the sub-task information also includes the calling order of the multiple first APIs when executing the network operation and maintenance sub-task. Alternatively, the sub-task information does not include the calling order of the first APIs, and the position of the input parameters of each first API in the sub-task information is used to indicate the calling order of each first API. The sub-task information is also a calling chain of at least one first API. When the calling chain is completed, the network operation and maintenance sub-task is executed, and the information output by the calling chain is the execution result of the network operation and maintenance sub-task.
[0167] For example, the first decision-making device inputs the task orchestration prompt information into the first large language model. The first large language model determines the first API required to complete each operation and maintenance task sub-intention, the calling order and input parameters of each first API when completing each operation and maintenance task sub-intention based on the prompt of the task orchestration prompt information, based on the description information of the API corresponding to the operation and maintenance scenario intention and multiple operation and maintenance task sub-intentions. Based on the calling order and input parameters, multiple sub-task information is generated (to complete the orchestration of network operation and maintenance sub-tasks), and based on the multiple sub-task information, task information is output, and each sub-task information in the task information indicates a network operation and maintenance sub-task.
[0168] When a network operation and maintenance task includes multiple network operation and maintenance subtasks, manual orchestration of the network operation and maintenance task is difficult and requires users to have a high level of understanding of various application program interfaces of the network. Automatically orchestrating network operation and maintenance tasks through the first language model eliminates the need for manual orchestration by users, simplifies the network operation and maintenance process, reduces the difficulty of network operation and maintenance, and thus improves network operation and maintenance efficiency.
[0169] Step C4: The first decision-making device outputs the task information to the first message processing device, and the first message processing device receives the task information.
[0170] The first message processing device receives the task information and acquires the network operation and maintenance task. Based on steps C2 to C4 above, the first language model in the first call decision device is used to complete the orchestration of the network operation and maintenance task, which can reduce the consumption of local computing resources of the first message processing device.
[0171] 1005. The first message processing device executes the network operation and maintenance task and obtains an execution result.
[0172] The first message processing device executes the network operation and maintenance task according to the task information. Referring to FIG10 , in the case where a first API is required to complete the network operation and maintenance task, the task information includes input data of the first API. The first message processing device sends a first interface call request to the first network management device based on the input data of the first API. The first interface call request indicates calling the first API to execute the network operation and maintenance task, wherein the first interface call request includes the input data of the first API. After receiving the first interface call request, the first network management device inputs the input data of the first API into the first API and runs the first API. The first API sends a second interface call request to the network device in the network, wherein the second interface call request indicates calling the second API of the network device to obtain the execution result of the network operation and maintenance task, wherein the second API refers to the API between the first network management device and the network device. After receiving the second interface call request, the network device runs the second API according to the second interface call request, obtains the execution result of the network operation and maintenance task, and returns the execution result of the network operation and maintenance task to the first API. The first API outputs the execution result of the network operation and maintenance task. Based on the execution result of the network operation and maintenance task, the first network management device sends an interface call response to the first message processing device, where the interface call response includes the execution result of the network operation and maintenance task. After receiving the interface call response, the first message processing device obtains the execution result of the network operation and maintenance task from the interface call response.
[0173] If the network operation and maintenance task requires calling multiple first APIs, the first network management device sequentially calls the multiple first APIs in the first network management device according to the calling order of the first APIs in the task information, and uses the data output by the last first API called as the execution result of the network operation and maintenance task. The process of calling each first API can be referred to the process of calling a first API above and will not be repeated here.
[0174] The above description uses the example of calling a first API in a first network management device when executing a network operation and maintenance task. In other embodiments, the first message processing device executes the network operation and maintenance task by calling the first APIs in multiple first network management devices. Taking deployment method 2 shown in Figure 3 as an example, message processing device 20 in the OSS is the first message processing device, and network management devices 50 in multiple EMS / NMSs are all first network management devices. When the network operation and maintenance task includes multiple network operation and maintenance subtasks, message processing device 20 executes different task subtasks by calling the first APIs in different network management devices 50, thereby obtaining the execution results of the multiple network operation and maintenance subtasks.
[0175] Exemplarily, the message processing device 20 executes multiple network operation and maintenance subtasks in sequence according to the execution order of the multiple network operation and maintenance subtasks. Taking the execution of any network operation and maintenance subtask as an example, the message processing device 20 executes the operation and maintenance subtask according to the subtask information of the network operation and maintenance subtask. For example, in the case where a first API is required to complete the network operation and maintenance subtask, the task information includes input data of a first API. The first message processing device sends a third interface call request to a network management device 50 based on the input data of the first API. The third interface call request indicates calling the first API to execute the network operation and maintenance subtask, wherein the third interface call request includes the input data of the first API. After receiving the third interface call request, the first network management device inputs the input data of the first API into the first API and runs the first API. In the process of running the first API, the network management device 30 sends a fourth interface call request to the network device in the network. The fourth interface call request indicates calling the second API of the network device to obtain the execution result of the network operation and maintenance subtask. After receiving the fourth interface call request, the network device runs the second API according to the fourth interface call request, obtains the execution result of the network operation and maintenance subtask, and returns the execution result of the network operation and maintenance subtask to the first API of the first network management device. The first API outputs the execution result of the network operation and maintenance subtask. Based on the execution result of the network operation and maintenance subtask, the network management device 50 sends an interface call response to the first message processing device, where the interface call response includes the execution result of the network operation and maintenance subtask. After receiving the interface call response, the first message processing device obtains the execution result of the network operation and maintenance subtask from the interface call response.
[0176] After obtaining the execution results of multiple network operation and maintenance subtasks, the message processing device 20 obtains the execution result of the network operation and maintenance task based on the execution results of the multiple network operation and maintenance subtasks. For example, the multiple network operation and maintenance subtasks of a network operation and maintenance task may be parallel or serial. The input data and output data of the multiple parallel network operation and maintenance subtasks have no impact on each other. The input data of each network operation and maintenance subtask other than the first network operation and maintenance subtask in the serial multiple network operation and maintenance subtasks is the output data of the previous network operation and maintenance subtask. Based on this, when the network operation and maintenance task includes multiple parallel network operation and maintenance subtasks, the message processing device 20 uses the execution results of the multiple network operation and maintenance subtasks as the execution result of the network operation and maintenance task. In the case where the network operation and maintenance task includes multiple network operation and maintenance sub-tasks in series, the first message processing device calls an API in a network management device 50, first executes the first network operation and maintenance sub-task among the multiple network operation and maintenance sub-tasks, and uses the execution result of the first network operation and maintenance sub-task as the input data of the second network operation and maintenance sub-task, and then calls the API in the network management device 50 or other network management devices 50 to execute the second network operation and maintenance sub-task, and uses the execution result of the second network operation and maintenance sub-task as the input data of the third network operation and maintenance sub-task, and then calls the API in the network management device 50 or other network management devices 50 to execute the third network operation and maintenance sub-task, and so on, until the last network operation and maintenance sub-task is executed, and the execution result of the last network operation and maintenance sub-task is used as the execution result of the network operation and maintenance task. Of course, the network operation and maintenance task may also include multiple parallel network operation and maintenance subtasks and multiple serial network operation and maintenance subtasks. The message processing device will use the execution results of the multiple parallel network operation and maintenance subtasks and the execution results of the last network operation and maintenance subtask in the multiple serial network operation and maintenance subtasks as the execution results of the network operation and maintenance task.
[0177] For each network management device in the operation and maintenance network, the network operation and maintenance task and the network operation and maintenance subtask are both network operation and maintenance tasks to be processed by the first message processing device (referred to as the first network operation and maintenance task). The first interface call request and the third interface call request are both interface call requests received by the network management device, indicating that the API of the network management device is called to execute the first network operation and maintenance subtask. The above is explained by taking the example of the network management device actively returning the execution result of the first network operation and maintenance task to the first message processing device after executing the first network operation and maintenance task. At this time, the interface call request is also a synchronous instruction issued by the first message processing device. In other embodiments, the interface call request issued by the first message processing device to the network management device is an asynchronous instruction. At this time, after the network management device executes the first network operation and maintenance task according to the interface call request, it does not actively report the execution result of the first network operation and maintenance task to the first message processing device, but waits for the first message processing device to actively query the execution result of the first network operation and maintenance task from the network management device. Here, the embodiment of the present application does not limit the way in which the first message processing device obtains the execution result of the first network operation and maintenance task.
[0178] In this step 1005, the network operation and maintenance task is executed by the first session processor, without the need to manually send operation and maintenance instructions to the network management device, which simplifies the network operation and maintenance process, reduces the difficulty of network operation and maintenance, and thus can improve network operation and maintenance efficiency.
[0179] After obtaining the execution result of the network operation and maintenance task, the first message processing device sends a dialog response to the dialog message based on the execution result to the client (i.e., step 1006). Alternatively, if the execution result is an execution result expressed in a programming language, to facilitate the user's understanding of the execution result of the network operation and maintenance task, the first message processing device converts the execution result of the network operation and maintenance task expressed in the programming language into at least one of natural language, a chart, and a network topology diagram based on the first large language model. The first message processing device then sends a dialog response to the client based on the converted execution result.
[0180] Exemplarily, the process of converting the execution result format may be as follows: the first message processing device splices the execution result represented in the programming language into a format conversion prompt template to obtain format conversion prompt information, and sends the format conversion prompt information to the first decision-making device, wherein the format conversion prompt information is used to prompt the first large language model to convert the execution result represented in the programming language into at least one of natural language, a chart, and a network topology diagram. After receiving the format conversion prompt information, the first decision-making device inputs the format conversion prompt information into the first large language model. Based on the prompt of the format conversion prompt information, the first large language model converts the execution result represented in the programming language into at least one of natural language, a chart, and a network topology diagram, and outputs the execution result after format conversion. The first decision-making device then returns the converted execution result to the first message processing device.
[0181] 1006. The first message processing device outputs a dialogue response of the dialogue message to the client based on the execution result of the network operation and maintenance task.
[0182] The dialogue response includes the execution result of the network operation and maintenance task and the network operation and maintenance intention, or the dialogue response includes the execution result of the network operation and maintenance task but does not include the network operation and maintenance intention.
[0183] For example, if the semantic similarity between the network operation and maintenance intention and the dialogue message is greater than the similarity threshold, the dialogue message can be used as the network operation and maintenance intention. At this time, the first message processing device sends the execution result of the network operation and maintenance task as a dialogue response to the client. If the semantic similarity between the network operation and maintenance intention and the dialogue message is less than or equal to the similarity threshold, the first message processing device sends the execution result of the network operation and maintenance task and the network operation and maintenance intention as a dialogue response to the client to prompt the user of the network operation and maintenance task corresponding to the execution result of the network operation and maintenance task.
[0184] 1007. The client receives the dialogue response and displays the dialogue response in the dialogue interface.
[0185] Taking the dialogue interface 1100 shown in Figure 11 as an example, dialogue window 111 is also used to display execution results expressed in natural language. Dialogue interface 100 also includes information display window 112, which is used to display execution results expressed in charts and / or network topology diagrams. Based on this, after receiving the dialogue response, the client displays a reply message 11b in dialogue window 111 based on the dialogue response. This reply message 11b includes the text content of the dialogue response (such as the network operation and maintenance intention and / or the execution results of the network operation and maintenance task). Referring to Figure 11, the reply message 11b includes the network operation and maintenance intention expressed in natural language "We can determine which APs carry more devices in the data acquisition device based on the channel utilization of each AP" and the execution results of the network operation and maintenance task expressed in natural language "such as N5-AP-345 (A1 area, 87%), N5-AP-302 (A1 area, 83%), N5-AP-304 (A1 area, 80%)". If the dialogue response also includes the execution results expressed in a chart or network topology, the execution results expressed in a chart or network topology will be displayed in the information display window 112. Still taking 11 as an example, assuming that the dialogue response also includes an execution result represented by a network topology, the client displays the network topology (global network topology or local network topology) of the network in the information display window 112, and highlights the network devices related to the execution result in the network topology. As shown in Figure 11, the way to highlight the network devices related to the execution result is to display the related network devices in a square box. Of course, there can also be other ways of highlighting, for example, using a color that is different from other network devices to display the related network devices, or using a specific mark to mark the related network devices. Here, the embodiment of the present application does not limit the way of highlighting the related network devices. By displaying the execution results represented by natural language, charts or network topology in the dialogue interface 1100, it is convenient for users to understand the execution results of network operation and maintenance tasks.
[0186] In other embodiments, if the execution results expressed in natural language are relatively large, the client can also fold and display the execution results expressed in natural language in the reply message. For example, the client displays part of the execution results and the viewing option b1 in the reply message, and in response to the selection of the viewing option, the client displays the remaining execution results.
[0187] In other embodiments, the information display window 112 can also be linked with the dialogue window 111 to display information related to the network. For example, the device icons in the network topology displayed in the information display window 112 also support data query. When the client detects the user's selection operation on any device identifier, the client displays the network data and / or configuration data of the network device pointed to by the device icon in the dialogue window 111, thereby being able to display richer information to the user.
[0188] To clearly display a diagram or network topology in information display window 112, the size of information display window 112 is larger than that of dialog window 111. By providing dialog window 111 and information display window 112, dialog interface 100 facilitates human-computer interaction and graphically displays various network information, thereby presenting rich information to the user and facilitating user understanding of the displayed information.
[0189] The method provided by the embodiment shown in Figure 10 uses a large language model to identify the user's network operation and maintenance intention from the user's conversation message, and then establishes the network operation and maintenance task that the user expects to perform on the network based on the network operation and maintenance intention. After the network operation and maintenance task is completed, the execution result of the network operation and maintenance task is provided to the client. In this process, the user only needs to provide conversation messages in the form of a conversation, without having to interact with the client multiple times, which simplifies the operation process of network operation and maintenance, thereby improving network operation and maintenance efficiency. For conversation messages in various operation and maintenance scenarios, the method provided in the embodiment of the present application can provide users with the final question and answer results, thereby not limiting the operation and maintenance scenarios, and can provide users with operation and maintenance services for various operation and maintenance scenarios, thereby increasing users' dependence on the network operation and maintenance system and improving the user stickiness of the network operation and maintenance system.
[0190] The embodiment shown in FIG10 above is illustrated by taking the example of a conversation response including a network operation and maintenance intention and the execution result of a network operation and maintenance task. In other embodiments, the conversation response also includes a predicted intention. The predicted intention refers to the conversation intention of the next conversation message predicted for the current conversation message. The predicted intention can also be understood as the predicted next action of the user. The next action may be an operation and maintenance action, or it may be a question related to knowledge question and answer. Therefore, the predicted intention may be a network operation and maintenance intention, or it may be a knowledge question and answer intention. There is at least one predicted intention. After obtaining the execution result of the network operation and maintenance task corresponding to the conversation message, the first message processing device obtains the predicted intention of the conversation message based on the first large language model and the network operation and maintenance intention of the conversation message.
[0191] In the case where the first large language model is deployed on the first decision-making device, the first message processing device obtains the predicted intent of the dialogue message by calling the first large language model, such as the following steps D1 to D3.
[0192] D1. The first message processing device sends prediction prompt information to the first decision-making device. The prediction prompt information is used to prompt the first large language model to predict the conversation intention of the next conversation message based on the network operation and maintenance intention.
[0193] The prediction prompt information includes the network operation and maintenance intention.
[0194] The first message processing device splices the network operation and maintenance intent into the prediction prompt template of the first large language model to obtain the prediction prompt information. The prediction prompt template is used to prompt the first large language model to predict the conversation intent of the next conversation message. The prediction prompt template includes an intent field and a sixth prompt sentence. The intent field is used to store the network operation and maintenance intent. The sixth prompt sentence is the template content in the prediction prompt template and is used to prompt the first large language model to predict the conversation intent of the next conversation message.
[0195] Take the prediction prompt template as {For the question {X[]}. What task does the user want to perform next? Return in text form} as an example. Assuming the network operation and maintenance intent is "to determine which APs carry more devices in the data acquisition device based on the channel utilization of each AP", the first message processing device fills the network operation and maintenance intent in the intent field X[] of the prediction prompt template to splice the following prediction prompt information:
[0196] Regarding the issue of {determining which APs carry more devices in data acquisition devices based on the channel utilization of each AP}.
[0197] What task does the user want to perform next? Returns as text.
[0198] After obtaining the prediction prompt information, the first message processing device sends the prediction prompt information to the first decision-making device.
[0199] D2. The first decision-making device receives the prediction prompt information and predicts the conversation intention of the next conversation message of the conversation message based on the first language model to obtain the predicted intention.
[0200] For example, the first decision-making device inputs the predicted prompt information into the first large language model. Based on the predicted prompt information and the network operation and maintenance intent, the first large language model predicts the conversation intent of the next conversation message and outputs a predicted intent. Still using the example in step D1, the two predicted intents output are "View more busy APs" and "AP optimization."
[0201] D3. The first decision-making device outputs the predicted intent to the first message processing device, and the first message processing device obtains the predicted intent.
[0202] After the first message processing device obtains the predicted intent, it generates a dialogue response based on the execution result of the network operation and maintenance task, at least one of the network operation and maintenance intents, and the predicted intent, and sends the dialogue response to the client. As shown in Figure 11, the dialogue window 111 also includes a navigation bar 11c. After receiving the dialogue response, the client displays the predicted intents "View More Busy APs" and "AP Tuning" in the dialogue response as recommended navigation options c1 and c2 in the navigation bar 11c, respectively. Accordingly, if the user selects any of the recommended navigation options c1 and c2, in response to the selection operation, the client notifies the first message processing device to process the predicted intent displayed by the recommended navigation option. The user does not need to enter a new dialogue message to indicate the next action, which further simplifies the operation process of network operation and maintenance, thereby further improving network operation and maintenance efficiency.
[0203] In other embodiments, after completing the network operation and maintenance task of the dialogue message, the first message processing device also analyzes the potential network operation and maintenance intention of the dialogue message through the first large language model, and generates a network operation and maintenance task for the potential network operation and maintenance intention (i.e., the relevant network operation and maintenance task of the dialogue message). The potential network operation and maintenance intention is an intention related to the dialogue intention of the dialogue message, which may be a related network operation and maintenance intention or a related knowledge question and answer intention. The relevant network operation and maintenance task is a network operation and maintenance task generated to complete the potential network operation and maintenance intention, and the relevant network operation and maintenance task is related to the network operation and maintenance task of the dialogue message. The first message processing device executes the execution result of the relevant network operation and maintenance task, and adds the execution result of the network operation and maintenance task of the dialogue message and the execution result of the related network operation and maintenance task to the dialogue response, so that the client can display the execution result of the network operation and maintenance task and the execution result of the related network operation and maintenance task on the dialogue interface, thereby providing users with a service of intelligently executing network operation and maintenance tasks, which can enhance user experience.
[0204] In other embodiments, in the process of generating a network operation and maintenance task based on the first large language model, if critical information for generating the network operation and maintenance task (such as API input data) is missing, the first message processing device sends a supplementary prompt message to the client, and the supplementary prompt message is used to prompt the user to supplement the critical information. The client displays the supplementary prompt message, and the user can supplement the critical information in the next dialogue message. The first message processing device generates the network operation and maintenance task based on the first large language model and the critical information.
[0205] The embodiment shown in Figure 10 is illustrated by taking the example of the first message processing device obtaining a network operation and maintenance task. In other embodiments, in a network operation and maintenance system with multi-level message processing devices, if the operation and maintenance scenario intention is not a knowledge question and answer scenario and the operation and maintenance task intention includes multiple operation and maintenance task sub-intentions, the first message processing device does not create a network operation and maintenance task, and distributes the multiple operation and maintenance task sub-intentions to the message processing device at the next management level, and the message processing device at the next management level completes each operation and maintenance task sub-intention.
[0206] Exemplarily, it is assumed that there are multiple message processing devices at the next level of the first message processing device, and the message processing devices at the next level are called second message processing devices. The management levels of the multiple second message processing devices are the same, and the management level of the second message processing devices is lower than the management level of the first message processing device, that is, the first message processing device is the upper-level manager of the multiple second message processing devices. Taking Figure 7 or Figure 8 as an example, the first message processing device is the message processing device 21 in Figure 7 or Figure 8, and the multiple second message processing devices are the message processing devices 22 in Figure 7 or Figure 8. Taking the example that the operation and maintenance task intent includes the first operation and maintenance task sub-intention and the second operation and maintenance task sub-intention, and the first device and the second device in the multiple second message processing devices respectively complete the first operation and maintenance task sub-intention and the second operation and maintenance task sub-intention, refer to Figure 12 as a flowchart of another network operation and maintenance method based on a large language model provided in an embodiment of the present application, the method includes the following steps.
[0207] 1201. The client sends a network operation and maintenance request to the first message processing device. The network operation and maintenance request indicates to perform a network operation and maintenance task on the network based on the network operation and maintenance intention of the dialogue message.
[0208] Here, this step 1201 is the same as the above step 1002, and here, this step 1201 is not repeated.
[0209] 1202. The first message processing device obtains the dialogue message in the network operation and maintenance request, and determines the network operation and maintenance intention based on the first language model and the dialogue message. The network operation and maintenance intention includes a first operation and maintenance task sub-intention and a second operation and maintenance task sub-intention.
[0210] Here, this step 1202 is the same as the above step 1003, and here, this step 1202 is not repeated.
[0211] 1203. The first message processing device sends the first operation and maintenance task sub-intent to the first device, and sends the second operation and maintenance task sub-intent to the second device.
[0212] The first device is any one of the multiple second message processing devices, and the second device is any one of the multiple second message processing devices except the first device.
[0213] 1204. The first device obtains the first operation and maintenance task sub-intent, and generates a first network operation and maintenance sub-task based on the first operation and maintenance task sub-intent and the second language model connected to the first device.
[0214] The second largest language model connected to the first device is the large language model running in the first device, or the large language model running in the decision-making device. If the second largest language model is a large language model running in the decision-making device, when the first message processing device and the first device are connected to the same decision-making device, the second largest language model and the first large language model are the same model. Taking FIG8 as an example, the first large language model and the second large language model are both large language models in decision-making device 30. When the first message processing device and the first device are connected to different decision-making devices, the second large language model and the first large language model are different models. Taking FIG7 as an example, the first large language model is the large language model in decision-making device 31, and the second large language model is the large language model in decision-making device 32 connected to the first device. In this case, the second large language model and the first large language model are two large language models with related functions.
[0215] When the first message processing device sends the first operation and maintenance task sub-intention to the first device, it also sends the operation and maintenance scenario intention in the network operation and maintenance intention to the first device, so that the second message processing device can orchestrate the network operation and maintenance task (i.e., the first network operation and maintenance sub-task) for the operation and maintenance task sub-intention based on the second largest language model and the API corresponding to the operation and maintenance scenario intention, and obtain the subtask information of the first network operation and maintenance sub-task.
[0216] When the second largest language model is deployed on the second decision-making device, the first device orchestrates the first network operation subtask for the first network operation sub-intent by calling the second largest language model, such as the following steps E1 to E3.
[0217] Step E1. The first device sends subtask orchestration prompt information to the second decision-making device. The subtask orchestration prompt information is used to prompt the large language model to orchestrate the network operation and maintenance subtask for the operation and maintenance task sub-intention based on the API corresponding to the operation and maintenance scenario intention, and prompt the large language model to output the subtask information of the orchestrated network operation and maintenance subtask.
[0218] For example, the subtask arrangement prompt information includes the first operation and maintenance task sub-intent and the description information of the API corresponding to the operation and maintenance scenario intent.
[0219] The first device splices the first network intent and the description information of the API corresponding to the operation and maintenance scenario intent in the task scheduling prompt template of the large language model to obtain subtask scheduling prompt information. The process of splicing the subtask scheduling prompt information is similar to the process of splicing the task scheduling prompt information in step C2. The difference is that when splicing the subtask scheduling prompt information, at least one operation and maintenance task sub-intent is spliced in the message field of the task scheduling prompt template, while when splicing the task scheduling prompt information, the entire operation and maintenance task intent is spliced in the message field of the task scheduling prompt template. After obtaining the subtask scheduling prompt information, the second message processing device sends the subtask scheduling prompt information to the second decision-making device.
[0220] Step E2: The second decision-making device receives the subtask arrangement prompt information, and arranges the first network operation and maintenance subtask for the first operation and maintenance task sub-intent based on the second largest language model and the API corresponding to the operation and maintenance scenario intention, and obtains the first subtask information.
[0221] The second decision-making device inputs the subtask orchestration prompt information into the second largest language model. The second largest language model orchestrates the first network operation and maintenance subtask for the first operation and maintenance task sub-intention based on the prompt of the subtask orchestration prompt information and the description information of the API corresponding to the operation and maintenance scenario intention, and obtains the first subtask information. The orchestration process, for example, determines the first API required to complete the first operation and maintenance task sub-intention under the operation and maintenance scenario intention, the calling sequence and input parameters of the first API required to complete the first operation and maintenance task sub-intention, and generates the first subtask information based on the calling sequence and input parameters of the first API required to complete the first operation and maintenance task sub-intention. The second largest language model outputs the first subtask information.
[0222] Step E3: The second decision-making device outputs the first subtask information to the second message processing device, and the first device receives the first subtask information.
[0223] The first subtask information indicates the first network operation and maintenance subtask, and obtaining the first subtask information is equivalent to obtaining the first network operation and maintenance subtask.
[0224] 1205. The first device executes the first network operation and maintenance subtask and outputs the execution result of the first network operation and maintenance subtask to the first message processing device.
[0225] For example, the first device executes the first network operation and maintenance subtask according to the first subtask information. This process is the same as the process in step 1005 in which the message processing device 20 executes the operation and maintenance subtask according to the subtask information of the network operation and maintenance subtask, and will not be repeated here.
[0226] 1206. The second device obtains the second operation and maintenance task sub-intent, and generates a second network operation and maintenance sub-task based on the second operation and maintenance task sub-intent and the second largest language model connected to the second device.
[0227] Here, step 1206 is similar to step 1204 and will not be described again here.
[0228] 1207. The second device executes the second network operation and maintenance subtask and outputs the execution result of the second network operation and maintenance subtask to the first message processing device.
[0229] Among them, this step 1207 is the same as the above step 1205 and will not be repeated here.
[0230] 1208. The first message processing device receives the execution result of the first network operation and maintenance subtask and the execution result of the second network operation and maintenance subtask.
[0231] The above steps 1205, 1207 and 1208 are explained by taking the example of the first message processing device passively obtaining the execution result of the network operation and maintenance subtask. In other embodiments, the first message processing device may also actively obtain the execution result of the first network operation and maintenance subtask from the first device, and actively obtain the execution result of the second network operation and maintenance subtask from the second device. At this time, there is no need for the first device and the second device to actively send the execution results of the network operation and maintenance subtask.
[0232] 1209. The first message processing device outputs a dialogue response of the dialogue message to the client based on the execution result of the first network operation and maintenance subtask and the execution result of the second network operation and maintenance subtask.
[0233] For example, the first message processing device determines the execution result of the network operation and maintenance task based on the execution result of the first network operation and maintenance subtask and the execution result of the second network operation and maintenance subtask, and outputs a dialogue response to the client based on the execution result of the network operation and maintenance task.
[0234] The process shown in the above steps 1201 to 1209 is illustrated by taking the network operation and maintenance task intent including two operation and maintenance task sub-intentions as an example. In other embodiments, the network operation and maintenance task intent includes more than two operation and maintenance task sub-intentions. The first message processing device sends the more than two operation and maintenance task sub-intentions to multiple second message processing devices. Each second message processing device can obtain one or more operation and maintenance task sub-intentions, and orchestrate network operation and maintenance sub-tasks for one or more operation and maintenance task sub-intentions based on their respective second largest language models, obtain one or more network operation and maintenance sub-tasks, execute at least one network operation and maintenance sub-task, and send at least one network operation and maintenance sub-task to the first message processing module. In this way, the first message processing module can obtain the execution results of the network operation and maintenance sub-tasks corresponding to all operation and maintenance task sub-intentions, and output a dialogue response to the client based on the obtained execution results. When the second message processing device obtains multiple first network operation and maintenance subtasks, it executes the multiple first network operation and maintenance subtasks in sequence according to the execution order of the multiple first network operation and maintenance subtasks. This process is the same as the process in step 1005 where the message processing device 20 executes the multiple network operation and maintenance subtasks in sequence according to the execution order of the multiple network operation and maintenance subtasks.
[0235] 1210. The client receives the dialogue response and displays the dialogue response in the dialogue interface.
[0236] Here, this step 1210 is the same as the above-mentioned step 1007, and here, this step 1210 is not repeated.
[0237] The method provided by the embodiment shown in FIG12 , in addition to achieving the beneficial effects of the embodiment shown in FIG10 , distributes each operation and maintenance task sub-intent within an operation and maintenance task intent to multiple second message processing devices through the first conversation processor. These multiple second message processing devices then process the operation and maintenance task sub-intents in parallel, improving the processing efficiency of the operation and maintenance task intents and reducing the response time of dialog messages. Furthermore, the multiple second message processing devices each orchestrate network operation and maintenance sub-tasks for the issued operation and maintenance task sub-intents using the second largest language model in their respective connected second decision-making devices, avoiding task congestion in the first largest language model, further accelerating the processing efficiency of the operation and maintenance task intents, and reducing the response time of dialog messages.
[0238] The embodiments shown in Figures 10 and 12 above are all explained by taking the example of the first message processing device first obtaining the network operation and maintenance intention of the dialogue message and then orchestrating the network operation and maintenance tasks based on the network operation and maintenance intention. In other embodiments, the first message processing device completes the identification of network operation and maintenance intention and the orchestration of network operation and maintenance tasks at one time based on the first large language model, such as the flowchart of another network operation and maintenance method based on a large language model provided in an embodiment of the present application shown in Figure 13.
[0239] 1301. The user enters a dialogue message "Which AP devices are the busiest today?" on the dialogue interface of the client.
[0240] 1302. The client sends a dialog message "Which AP devices are the busiest today?" to the first message processing device.
[0241] The method of sending the dialogue message may refer to the above step 1002 .
[0242] After receiving the dialogue message, the first message processing device enters the stage 1 shown in the following steps 1303 to 1305 to perform scenario intent recognition on the dialogue message to identify the potential intent of the operation and maintenance scenario to which the dialogue message belongs.
[0243] 1303. The first message processing device splices the received conversation message into the scenario intent prompt template to obtain a modified scenario intent prompt template (i.e., scenario intent prompt information), and sends the modified scenario intent prompt template to the first large language model to request the first large language model to perform scenario intent recognition on the conversation message. The modified scenario intent prompt template is as follows:
[0244] {User asks: Which APs are the busiest today?
[0245] Which of the following scenarios does this apply to? Return as a number
[0246] 0. Other
[0247] 1. Online query
[0248] 2. View network reports
[0249] 3. Check the fault information.
[0250] 1304. The first language model receives the modified scenario intent prompt template, analyzes and infers the user's conversation message based on the modified scenario intent prompt template, determines that "Which APs are the busiest today" belongs to the network data query scenario, and returns the network data query scenario in digital form, that is, returns the number "1".
[0251] After determining the network operation and maintenance scenario, the first message processing device matches the specific scenario according to the number "1", and enters phase two shown in steps 1305 to 1312 based on the specific scenario to perform network operation and maintenance task orchestration to implement the execution scenario process.
[0252] 1305. After determining that the user's conversation message corresponds to a network data query scenario, the first message processing device splices the description information of the network data query scenario into the interface prompt template to obtain a modified interface prompt template (i.e., interface prompt information), and sends the modified interface prompt template to the first language model. The modified interface prompt template is as follows:
[0253] To answer the question "{query}", which category should you choose from for API usage? Return as a number
[0254] 1. About AP devices
[0255] 2. About switch equipment
[0256] 3. About gateway devices}.
[0257] The query here refers to the conversation message.
[0258] 1306. The first language model receives the modified interface prompt template, analyzes and infers the dialogue message, and determines that to answer the dialogue message (ie, the user's question), it is necessary to call the API of the AP device and return "1" in digital form.
[0259] 1307. After receiving the intent response about the API, the first message processing device sends a request for obtaining the API description list of the AP device (i.e., requests to obtain the description information of the API) to the first network management device, and obtains the API description list about the AP device.
[0260] 1308. The first message processing device splices the API description list of the AP device into the task scheduling prompt template to obtain a modified task scheduling prompt template, and sends the modified task scheduling prompt template to the first language model. The modified task scheduling prompt template is as follows:
[0261] {{List of API descriptions for AP}
[0262] Which API should be called for {query}? What parameters should be passed? Give the answer directly, without the analysis process.
[0263] 1309. After receiving the modified task scheduling prompt template, the first language model determines the specific API that needs to be called (i.e., the first API) and the input parameters of the API that needs to be called, and returns the API call list to the first message processing device, wherein the API call list is the task information, and the API call list includes the identifier of the API that needs to be called and the input parameters.
[0264] 1310. The first message processing device sends an API call message to the first network management device based on the returned API call list to request to call the API. The first network management device returns the API call result to the first message processing device according to the API call list in the API call message, such as returning the TopN busy AP list.
[0265] 1311. The first message processing device splices the returned TopN busy AP list into the reply prompt template to obtain a modified reply prompt template, and sends the modified reply prompt template to the first language model module to prompt the first language model module to give a reply to the dialog message. The modified reply prompt template is as follows:
[0266] {Based on {return}, please provide analysis and summary of {query}}.
[0267] The return value refers to the API call result: the list of TopN busy APs.
[0268] 1312. After receiving the modified reply prompt template, the first language model generates a response message based on the reply prompt template and returns the response message (i.e., the operation and maintenance task intent + the execution result of the network operation and maintenance task) to the first dialogue process. The response message is as follows:
[0269] We can use the channel usage of each AP to determine which APs are carrying more devices during the data collection period. The following are the APs with the most connected devices:
[0270] AP03 (SuZhou-A1 region, 87%)
[0271] h1-ap304 (SuZhou-A1 region, 83%)
[0272] szvb6oaphw36 (SuZhou-A1 region, 80%)
[0273] These APs had high channel utilization during the data collection period and can be considered relatively busy.
[0274] The process shown in steps 1305 to 1312 above is the process of orchestrating tasks based on the determined operation and maintenance scenario intent, executing one or more network operation and maintenance tasks, and obtaining response information related to the problem. In other embodiments, the first message processing device also performs stage three shown in steps 1313 to 1318 to provide recommended function navigation for the user.
[0275] 1313. The first message processing device module concatenates the dialog message query to a category prompt template of a uniform resource locator (URL) to obtain a modified category prompt template, and sends the modified category prompt template to the first language model to prompt the first language model to provide recommended navigation information for the user's question (i.e., the dialog message). The modified category prompt template is as follows:
[0276] {For the question "{query}", from which category should the URL be selected? Return as a number
[0277] 1. About AP
[0278] 2. About customer files}.
[0279] 1314. After the first large language model receives the modified category prompt template, based on the modified category prompt template, if it is determined that the user's next question is about AP, it returns 1 in digital form, and the first message processing device receives the response "1" of the large language model.
[0280] 1315. The first message processing device sends a URL acquisition request to the first network management device to request to obtain a URL list about the AP device, and receives response information returned by the first manager for obtaining the URL, wherein the response information includes a URL list about the AP device, wherein the list includes the URL and description information of the URL.
[0281] 1316. The first message processing device splices the returned list of URLs related to the AP device into the prediction prompt template to obtain a modified prediction prompt template, and sends the modified prediction prompt template to the first language model to prompt the first language model to recommend a related URL link (i.e., predicted intent) for the next conversation message. The modified prediction prompt template is as follows:
[0282] The web URL information related to the problem is as follows:
[0283] {URL list and description of each URL}
[0284] Which URL does {query} relate to? Give the URL. The output format is:
[0285] A brief description of this URL:
[0286] URL:}.
[0287] 1317. After receiving the modified prediction prompt template, the first language model recommends a related URL for the conversation intent of the next conversation message according to the prompt of the modified prediction prompt template. Assuming that the recommended content (i.e., the predicted intent) is "A brief description of this URL: View more busy AP details. URL: https: / / operation and maintenance task intent", the first language model uses the recommended content as response information and returns the response information to the first message processing device, and the first message processing device receives the response information.
[0288] 1318. The first message processing device sends the response information obtained in the above three stages to the first large language model for splicing, or sends the initial response information obtained in the above three stages to the first large language model for splicing, to obtain reply response information (i.e., dialogue response) expressed in natural language to be returned to the client. For example, the reply response information expressed in natural language returned by the first large language model is as follows:
[0289] We can use the channel utilization of each AP to determine which APs are carrying the most devices during the data collection period. The following are the APs with the most connected devices:
[0290] AP03 (SZ-A1 region, 87%)
[0291] h1-ap304 (SZ-A1 region, 83%)
[0292] szvb6oaphw36 (SZ-A1 region, 80%)
[0293] These APs have high channel utilization during the data collection period and can be considered to be relatively busy.
[0294] To view more busy AP details, please click [here](https: / / Operation and Maintenance Task Intent)}.
[0295] 1319. The first message processing device sends a reply response message to the client, and the client displays the reply response message on the dialogue interface.
[0296] The embodiment shown in Figure 13 is introduced by taking the example of recommended navigation information including the predicted intention "View more busy APs" and the URL related to the task recommendation information. In other embodiments, the predicted intention is not the URL related to the task recommendation information, that is, the URL is an optional option for the predicted intention.
[0297] The embodiments shown in Figures 10 to 13 above are all introduced by taking the example of a message processing device (such as a first message processing device and a second message processing device) calling a large language model in a decision-making device (such as a first decision-making device and a second decision-making device). In other embodiments, if the computing resources of the message processing device are sufficient, a large language model can also be deployed in the message processing device. The message processing device completes the method steps completed by the decision-making device based on the large language model based on the local large language model, thereby eliminating the need to call the medium and large language models of other devices, reducing the interaction between the message processing device and other devices, and at this time, there is no need to set up a decision-making device in the network operation and maintenance system. The embodiments shown in Figures 10 to 13 above are all explained by taking the example of a message processing device providing prompt information of a large language model (such as intention prompt information, scenario intention prompt information, task prompt information, interface type prompt information, task scheduling prompt information, format conversion prompt information, prediction prompt information, etc.) to a decision-making device. In other embodiments, the message processing device does not provide prompt information to the decision-making device, but provides the first decision-making device with relevant information spliced into the prompt information (such as conversation messages, network operation and maintenance intentions, etc.). The decision-making device splices the relevant information into the corresponding prompt template to obtain the prompt information of the large language model, and inputs the prompt information into the large language model.
[0298] The present application involves prompt templates of large language models (such as intent prompt templates, scenario intent prompt templates, task intent prompt templates, interface prompt templates, task scheduling prompt templates, conversion prompt templates, prediction prompt templates, potential intent prompt templates, etc.), which are all prompt templates of large language models, so that the large language model can output correct results according to the prompts of the prompt templates. The above is explained by taking the example of a user conducting a human-computer dialogue with the first message processing device through the dialogue window of the dialogue interface. In other embodiments, the user can also trigger a human-computer dialogue with the first message processing device through the information display window of the dialogue interface. The following is an exemplary introduction with reference to Figures 14 and 15.
[0299] As shown in the dialogue interface 1400 in Figure 14, when the user triggers the client to open the dialogue interface 1400, the client displays the dialogue interface 1400 in the initial state. In the initial state, the interface 1400 is divided into a dialogue window 1401 and an information display window 1402, wherein the dialogue window 1401 is a human-computer question and answer box, which can conduct real-time human-computer interactive dialogue, and the information display window 1402 is a real-time presented chart box, and the elements in the icon box can be selected to trigger human-computer dialogue, and the status of the elements can also be presented in the chart box based on the dialogue message and dialogue response.
[0300] When the dialogue interface 1400 is in its initial state (e.g., when the dialogue interface 1400 is just opened), the information display window 1402 displays detailed network operation and maintenance information, such as a network topology diagram, an alarm log list, and performance statistics. For example, the information display window 1402 displays a network topology diagram composed of network elements (i.e., network devices) 1-9. The user can trigger elements such as network elements and alarms in the information display window 1402 to conduct a human-computer dialogue with the message processing device targeted at the element, such as the human-computer dialogue flow chart shown in FIG15 . The human-computer dialogue flow chart includes the following steps 1 to 23.
[0301] Step 1: Assume that the client is a browser page, a dialogue interface 1400 is displayed in the browser page, and the user selects the network element 9 displayed in the network topology diagram (such as a click operation or a voice command).
[0302] Step 2: In response to the selection operation of network element 9, the client pops up a dialogue sub-interface 1403 based on network element 9 at a location associated with network element 9 (such as near network element 9) in the information display window 1402, and caches the network element information of the network element in the browser.
[0303] The network element information of the network element includes the network element identity (ID) and / or the address of the network element. The network element information is cached in the browser so that the subsequent message processing device can conduct a human-computer dialogue with the user for the selected network element based on the network element information.
[0304] Step 3: The user enters information related to network operation and maintenance in dialog box A in the dialogue sub-interface 1403. For example, the user enters the first question in dialog box A, which is a query question: query channel utilization.
[0305] Step 4: The client displays the first question input by the user in the information display window 1402 in the dialogue interface, and sends the question to the message processing device.
[0306] Step 5: The message processing device optimizes the first question, assembles the optimized question into a prompt template, and sends the assembled prompt template to the large language model. The large model determines the operation and maintenance scenario intention, operation and maintenance task intention, and orchestrates the operation and maintenance tasks.
[0307] For example, the message processing device obtains cached network element information from the browser and optimizes the user's question based on the network element information to obtain an optimized question: query the channel utilization of network element 9. In other embodiments, the message processing device does not optimize the question but instead incorporates the question into the operation and maintenance intention prompt template. Alternatively, the client optimizes the question based on the cached network element information and provides the optimized question to the message processing device, which then incorporates the optimized question into the operation and maintenance intention prompt template.
[0308] Step 6. Based on the prompt of the prompt template, the large language model identifies that the operation and maintenance scenario intention of the problem is an API call type problem, and the operation and maintenance task intention of the problem is to query the channel utilization of network element 9, and determines that the API (called the first target API) that needs to be called to query the channel utilization is returned to the client. The network operation and maintenance intention and the identifier of the first target API that needs to be called are returned to the client.
[0309] Step 7. After determining the API class call problem, obtain the parameter information of the first target AIP from the network management device, the message processing device associates the web page 9 previously selected by the user in the information display window 1402, obtains the network element information cached after the user first selects the network element from the client, and uses the network element information as the parameter information for filling in the first target API.
[0310] Step 8. The message processing device calls the first target API to query the channel utilization of network element 9, obtains the query result, assembles the query result into a format conversion prompt template, and provides the assembled format conversion prompt template to the large language model. The large language model converts the query result into a format according to the prompt of the format conversion prompt template, and returns the query result after format conversion as a dialogue response to the client. The large language model sends the dialogue response to the message processing device.
[0311] Step 9: The message processing device sends a dialogue response to the client, and the client displays the dialogue response in the dialogue window 1401 in the dialogue interface 1400: Help you query that the channel utilization rate of network element 9 is 87%.
[0312] Step 10: The user continues to input the second question in dialog box A or dialog window 1401: Why is the channel utilization rate high?
[0313] Step 11: The client displays the second question input by the user in the dialogue window 1401 and sends the second question to the message processing device.
[0314] Step 12: Similar to step 5, the message processing device optimizes the second question, assembles the optimized second question into a prompt template, and sends the assembled prompt template to the large language model. The large model determines the operation and maintenance scenario intention, operation and maintenance task intention, and orchestrates the operation and maintenance tasks.
[0315] Step 13. The large language model recognizes that the operation and maintenance scenario intention of the second question is a knowledge question and answer question based on the prompt of the prompt template, and the operation and maintenance task intention of the question is the reason for the high channel utilization of network element 9. The large language model sends the network operation and maintenance intention composed of the operation and maintenance scenario intention and the operation and maintenance task intention to the message processing device.
[0316] Step 14: The message processing device splices the network operation and maintenance intention into the knowledge question and answer prompt template, and sends the assembled knowledge question and answer prompt template to the large language model. The large language model outputs the answer to the second question according to the prompt of the knowledge question and answer prompt template, and the large language model returns the answer to the second question to the message processing device.
[0317] Step 15: The message processing device returns the message to the client, and the client displays the answer in the dialog window 1401: The reason why the channel utilization of network element 9 is high is xxxxx.
[0318] Step 16: The user continues to input the third question in dialog box A or dialog window 1401: Optimize.
[0319] Step 17: The client displays the third question input by the user in the conversation window 1401 and sends the third question to the message processing device.
[0320] Step 18: Similar to step 5, the message processing device optimizes the third question, assembles the optimized third question into a prompt template, and sends the assembled prompt template to the large language model. The large model determines the operation and maintenance scenario intention, operation and maintenance task intention, and orchestrates the operation and maintenance tasks.
[0321] Step 19. The large language model recognizes that the operation and maintenance scenario intention of the third question is a network optimization problem based on the prompt of the prompt template. The operation and maintenance task intention of this problem is to optimize the channel utilization of network element 9, and determines that the API for network optimization (second target API) needs to be called to return the network operation and maintenance intention and the identifier of the first target API that needs to be called to the client.
[0322] Step 20. After determining the network optimization problem, the message processing device obtains the parameter information of the second target AIP from the network management device, associates the web page 9 previously selected by the user in the information display window 1402, obtains the network element information cached after the user first selected the network element from the client, and uses the network element information as the parameter information for filling in the second target API.
[0323] Step 21. After determining that the network optimization problem is a network optimization problem, the message processing device also notifies the client that it is a network optimization problem. Based on the notification, the client displays a dynamic effect on the icon of network element 9 in the information display window 1402 to prompt the network element being optimized, so that it can be dynamically linked with the dialogue message in the dialogue window 1401 to achieve a dynamic effect of linkage.
[0324] Step 22. The message processing device calls the second target API to optimize the channel utilization of network element 9, obtains the optimization result, assembles it into a format conversion prompt template, and provides the assembled format conversion prompt template to the large language model. The large language model converts the optimization result into a format according to the prompt of the format conversion prompt template, and the large language model sends the optimization result after format conversion to the message processing device.
[0325] Step 23: The message processing device sends the optimized result after format conversion to the client. The client displays the optimized result “√” in the dialog window 1401 to indicate that the optimization is completed, and cancels the dynamic effect on the icon of network element 9 to restore the icon of network element 9 to its initial state.
[0326] With respect to the above-described dialogue process, the user can select the element to be operated and maintained in the information display window and then enter a simple description of the problem in the dialogue message. The user does not need to enter complex problem description information. The user can also ask questions continuously for the element selected by the user. The message processing device can obtain the user's specific question by correlating the user's operation in the information display window with the output information. Then, with the help of a large language model, the network operation and maintenance intention in the specific question is analyzed, the operation and maintenance task used to complete the network operation and maintenance intention is arranged, and the network operation and maintenance task is executed. In other embodiments, the dialogue sub-interface 1403 also includes multiple operation and maintenance options. The operation and maintenance options are used to confirm the type of operation and maintenance task to be performed on the network element (such as threat assessment, network tuning, etc.). The user can also select and confirm the operation and maintenance option without entering a question. In response to the selection and confirmation operation, the client uses the operation and maintenance task type corresponding to the selected operation and maintenance option as the user's question, without the need for user input, thereby further simplifying the user's operation process.
[0327] The above describes the method of the embodiment of the present application. The following describes the apparatus of the embodiment of the present application. It should be understood that the apparatus and equipment described below have the same inventive concept as the above-mentioned network operation and maintenance method and can achieve the beneficial effects achieved by the above-mentioned method embodiment. The network operation and maintenance apparatus and equipment will be described below in conjunction with Figures 16 and 17. It should be understood that the technical features described in the method embodiment are also applicable to the following apparatus embodiment.
[0328] Figure 16 is a structural diagram of a network operation and maintenance device based on a large language model. The device 1600 shown in Figure 16 includes: a determination module 1601, which is used to determine the network operation and maintenance intention based on the large language model and the input dialogue message; an acquisition module 1602, which is used to obtain the network operation and maintenance task based on the network operation and maintenance intention; and an execution module 1603, which is used to execute the network operation and maintenance task and output the dialogue response.
[0329] In one possible implementation, the network operation and maintenance intention includes the operation and maintenance task intention and the operation and maintenance scenario intention. The acquisition module 1602 is used to: orchestrate the network operation and maintenance task intention based on the large language model and the application programming interface corresponding to the operation and maintenance scenario intention, and obtain task information. The application programming interface is used to complete the network operation and maintenance task under the operation and maintenance scenario intention. The task information indicates the calling process of the application programming interface when executing the network operation and maintenance task.
[0330] In one possible implementation, the operation and maintenance task intent includes multiple operation and maintenance task sub-intentions, and the acquisition module 1602 is used to: based on the large language model and the application programming interface, respectively arrange network operation and maintenance sub-tasks for multiple operation and maintenance task sub-intentions, and obtain multiple sub-task information, where the sub-task information indicates the calling process of the application programming interface when executing the network operation and maintenance sub-task; based on the multiple sub-task information, obtain task information.
[0331] In one possible implementation, the acquisition module 1602 includes: an acquisition unit, used to obtain description information of the application programming interface; a generation unit, used to generate task orchestration prompt information based on the operation and maintenance task intent, the first prompt statement and the description information of the application programming interface, the first prompt statement is used to prompt the large language model to orchestrate the network operation and maintenance task for the operation and maintenance task intent based on the application programming interface, and prompt the large language model to output task information of the orchestrated network operation and maintenance task; a sending unit, used to send task orchestration prompt information to the device running the large language model; and a receiving unit, used to receive task information returned by the device.
[0332] In a possible implementation, the acquiring unit is configured to: send an information acquisition request to the network management device, where the information acquisition request indicates acquiring description information of the application programming interface; and receive the description information returned by the network management device.
[0333] In one possible implementation, the execution module 1603 is used to: send an interface call request to the network management device based on the task information, where the interface call request indicates calling an application programming interface to perform the network operation and maintenance task; and receive an interface call response returned by the network management device, where the interface call response includes the execution result of the network operation and maintenance task.
[0334] In one possible implementation, the device 1600 also includes: a sending module, used to send the first operation and maintenance task sub-intention to the first device, and send the second operation and maintenance task sub-intention to the second device; an acquisition module, also used to obtain the execution result of the first network operation and maintenance subtask from the first device, and obtain the execution result of the second network operation and maintenance subtask from the second device; an output module, used to output a dialogue response based on the execution result of the first network operation and maintenance subtask and the execution result of the second network operation and maintenance subtask.
[0335] In one possible implementation, the determination module 1601 is used to: generate operation and maintenance intention prompt information based on the dialogue message and the second prompt statement, the second prompt statement is used to prompt the large language model to identify the network operation and maintenance intention of the dialogue message, and prompt the large language model to output the identified network operation and maintenance intention; send the operation and maintenance intention prompt information to the second device running the large language model; receive the network operation and maintenance intention returned by the second device.
[0336] Through the description of the above implementation methods, technical personnel in the relevant field can understand that, for the convenience and simplicity of description, the above-mentioned device 1600 is only illustrated by the division of the above-mentioned functional modules. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0337] FIG17 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The computer device 1700 shown in FIG17 is configured as any module in the aforementioned network operation and maintenance system (such as a client, a first message processing device, a second message processing device, a first decision-making device, a second message processing device, or a first network management device). As shown in FIG17 , the computer device 1700 includes: a bus 1702, a processor 1704, a memory 1706, and a communication interface 1708. The processor 1704, the memory 1706, and the communication interface 1708 communicate with each other via the bus 1702. The computer device 1700 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computer device 1700. The bus 1702 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, FIG17 is represented by only one line, but this does not mean that there is only one bus or one type of bus. Bus 1704 may include a path for transmitting information between various components of computer device 1700 (e.g., memory 1706, processor 1704, communication interface 1708). Processor 1704 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP). Memory 1706 may include volatile memory, such as random access memory (RAM). Memory 1706 may also include non-volatile memory, such as read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid state drive (SSD). Memory 1706 stores executable program code, and processor 1704 executes the executable program code to implement the functions of any module in the aforementioned network operation and maintenance system, thereby implementing the network operation and maintenance method based on the large language model. In other words, memory 1706 stores instructions for executing the network operation and maintenance method based on the large language model.Alternatively, memory 1706 stores executable program code, and processor 1704 executes the executable program code to implement a network operation and maintenance method based on a large language model executed by any module in the network operation and maintenance system. For example, if computer device 1700 is configured as a message processing device, processor 1704 executes the executable program code to implement the network operation and maintenance method based on a large language model executed by the message processing device. Communication interface 1708 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between computer device 1700 and other devices or communication networks.
[0338] In an exemplary embodiment, a computer-readable storage medium is further provided, such as a memory including a program code, wherein the program code can be executed by a processor of a computer device to complete the network operation and maintenance method based on the large language model executed by any module in the above-mentioned network operation and maintenance system. For example, the computer-readable storage medium is a non-transitory computer-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device.
[0339] An embodiment of the present application also provides a computer program product or computer program, which includes program code, which is stored in a computer-readable storage medium. The processor of the computer device reads the program code from the computer-readable storage medium, and the processor executes the program code, so that the computer device executes the network operation and maintenance method based on the large language model executed by any module in the above-mentioned network operation and maintenance system.
[0340] Among them, the devices, equipment, computer-readable storage media or computer program products provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.
[0341] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0342] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0343] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.
[0344] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
[0345] In the description of this application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "a plurality" means two or more. Words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily mean different. In this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way. It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the dialogue messages involved in this application are all obtained with full authorization. All the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of the present disclosure, and will not be described one by one here. The above is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.
Claims
1. A network operation and maintenance method based on a large language model, characterized in that: The method comprises: Determine network operation and maintenance intentions based on the large language model and input conversation messages; Based on the network operation and maintenance intention, obtain a network operation and maintenance task; Execute the network operation and maintenance task and output a dialogue response.
2. The method according to claim 1, characterized in that The network operation and maintenance intention includes an operation and maintenance task intention and an operation and maintenance scenario intention. The obtaining of the network operation and maintenance task based on the network operation and maintenance intention includes: Based on the large language model and the application programming interface corresponding to the operation and maintenance scenario intention, the network operation and maintenance tasks are orchestrated for the operation and maintenance task intention to obtain task information. The application programming interface is used to complete the network operation and maintenance tasks under the operation and maintenance scenario intention, and the task information indicates the calling process of the application programming interface when executing the network operation and maintenance task.
3. The method according to claim 2, characterized in that The operation and maintenance task intent includes a plurality of operation and maintenance task sub-intents, and the application programming interface corresponding to the large language model and the operation and maintenance scenario intent is used to arrange the network operation and maintenance task for the operation and maintenance task intent, and the obtained task information includes: Based on the large language model and the application programming interface, respectively arrange network operation and maintenance subtasks for the multiple operation and maintenance task sub-intentions to obtain multiple subtask information, wherein the subtask information indicates a calling process of the application programming interface when executing the network operation and maintenance subtask; Based on the plurality of subtask information, the task information is acquired.
4. The method according to claim 2, characterized in that: The application programming interface corresponding to the large language model and the operation and maintenance scenario intent is used to arrange the network operation and maintenance task for the operation and maintenance task intent, and the task information obtained includes: Obtaining description information of the application programming interface; Generate task scheduling prompt information based on the operation and maintenance task intention, the first prompt statement and the description information of the application programming interface, wherein the first prompt statement is used to prompt the large language model to schedule a network operation and maintenance task for the operation and maintenance task intention based on the application programming interface, and prompt the large language model to output task information of the scheduled network operation and maintenance task; Sending the task scheduling prompt information to a device running the large language model; Receive the task information returned by the device.
5. The method according to claim 4, characterized in that The obtaining of description information of the application programming interface includes: Sending an information acquisition request to a network management device, wherein the information acquisition request indicates obtaining description information of the application programming interface; Receive the description information returned by the network management device.
6. The method according to any one of claims 2 to 5, characterized in that: The performing of the network operation and maintenance task includes: Based on the task information, sending an interface call request to a network management device, wherein the interface call request indicates calling the application programming interface to perform the network operation and maintenance task; An interface call response returned by the network management device is received, wherein the interface call response includes an execution result of the network operation and maintenance task.
7. The method according to claim 1, characterized in that The network operation and maintenance intention includes a first operation and maintenance task sub-intention and a second operation and maintenance task sub-intention. After determining the network operation and maintenance intention, the method further includes: Sending the first operation and maintenance task sub-intention to the first device, and sending the second operation and maintenance task sub-intention to the second device; Obtaining an execution result of the first network operation and maintenance subtask from the first device, and obtaining an execution result of the second network operation and maintenance subtask from the second device; Based on the execution result of the first network operation and maintenance subtask and the execution result of the second network operation and maintenance subtask, the dialogue response is output.
8. The method according to any one of claims 1 to 7, characterized in that The determining of the network operation and maintenance intention based on the large language model and the input dialogue message includes: Generate operation and maintenance intention prompt information based on the dialogue message and the second prompt sentence, wherein the second prompt sentence is used to prompt the large language model to identify the network operation and maintenance intention of the dialogue message, and prompt the large language model to output the identified network operation and maintenance intention; Sending the operation and maintenance intention prompt information to the device running the large language model; Receive the network operation and maintenance intention returned by the device.
9. A network operation and maintenance device based on a large language model, characterized in that: The device comprises: A determination module is used to determine the network operation and maintenance intention based on the large language model and the input dialogue message; An acquisition module, used to acquire network operation and maintenance tasks based on the network operation and maintenance intention; An execution module is used to execute the network operation and maintenance task and output a dialogue response.
10. The device according to claim 9, characterized in that The network operation and maintenance intention includes operation and maintenance task intention and operation and maintenance scenario intention, and the acquisition module is used to: Based on the large language model and the application programming interface corresponding to the operation and maintenance scenario intention, the network operation and maintenance tasks are orchestrated for the operation and maintenance task intention to obtain task information. The application programming interface is used to complete the network operation and maintenance tasks under the operation and maintenance scenario intention, and the task information indicates the calling process of the application programming interface when executing the network operation and maintenance task.
11. The device according to claim 10, characterized in that The operation and maintenance task intent includes multiple operation and maintenance task sub-intents, and the acquisition module is used to: Based on the large language model and the application programming interface, respectively arrange network operation and maintenance subtasks for the multiple operation and maintenance task sub-intentions to obtain multiple subtask information, wherein the subtask information indicates a calling process of the application programming interface when executing the network operation and maintenance subtask; Based on the plurality of subtask information, the task information is acquired.
12. The device according to claim 10, characterized in that The acquisition module comprises: An acquisition unit, used to acquire description information of the application programming interface; a generating unit, configured to generate task scheduling prompt information based on the operation and maintenance task intent, a first prompt statement, and description information of the application programming interface, wherein the first prompt statement is used to prompt the large language model to schedule a network operation and maintenance task for the operation and maintenance task intent based on the application programming interface, and prompt the large language model to output task information of the scheduled network operation and maintenance task; A sending unit, configured to send the task scheduling prompt information to a device running the large language model; A receiving unit is used to receive the task information returned by the device.
13. The device according to claim 12, characterized in that The acquisition unit is used for: Sending an information acquisition request to a network management device, wherein the information acquisition request indicates obtaining description information of the application programming interface; Receive the description information returned by the network management device.
14. The device according to any one of claims 10 to 13, characterized in that The execution module is used for: Based on the task information, sending an interface call request to a network management device, wherein the interface call request indicates calling the application programming interface to perform the network operation and maintenance task; An interface call response returned by the network management device is received, wherein the interface call response includes an execution result of the network operation and maintenance task.
15. The device according to claim 9, characterized in that The device also includes: A sending module, used to send the first operation and maintenance task sub-intention to the first device, and send the second operation and maintenance task sub-intention to the second device; The acquisition module is further used to acquire the execution result of the first network operation and maintenance subtask from the first device, and acquire the execution result of the second network operation and maintenance subtask from the second device; An output module is used to output the dialogue response based on the execution result of the first network operation and maintenance subtask and the execution result of the second network operation and maintenance subtask.
16. The device according to any one of claims 9 to 15, characterized in that The determination module is used for: Generate operation and maintenance intention prompt information based on the dialogue message and the second prompt sentence, wherein the second prompt sentence is used to prompt the large language model to identify the network operation and maintenance intention of the dialogue message, and prompt the large language model to output the identified network operation and maintenance intention; Sending the operation and maintenance intention prompt information to a second device running the large language model; Receive the network operation and maintenance intention returned by the second device.
17. A computer device, characterized in that: The computer device comprises a processor, and the processor is configured to execute program code so that the computer device performs the method according to any one of claims 1 to 8.
18. A computer-readable storage medium, characterized in that: At least one program code is stored in the storage medium, and the at least one program code is read by the processor to enable the computer device to execute the method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Network operation and maintenance method, device and equipment based on large language model and storage medium
CN118945047A
Fault removal method and system, electronic equipment and computer readable storage medium
CN115529256A
Intelligent operation and maintenance system and method, electronic equipment and storage medium
CN117389843A
Natural language dialog-based security help agent for network administrator
US20160219048A1
Dialog Manager for Supporting Multi-Intent Dialogs
US20200066267A1
Cited By
Method and device for outputting data processing result, equipment, medium and product
CN119148891A
Monitoring data processing method and device and computer storage medium
CN119169538A
AIOps multi-task scheduling method and system cooperatively driven by large and small models
CN120085999A
Operation and maintenance method and device based on large language model and computer program product
CN120179276A
5G private network operation and maintenance method based on large model adaptive capability and related device
CN120264329A
