Fault processing method, electronic device, storage medium, and program product

Abstract

The application discloses a fault processing method, an electronic device, a storage medium and a program product, and belongs to the technical field of communication. The method comprises the following steps: a cross-domain management and control agent perceives that an end-to-end cross-domain service fails through monitoring, wherein the management and control range of the cross-domain management and control agent comprises a plurality of management and control domains, and each management and control domain corresponds to a single-domain management and control agent; the cross-domain management and control agent interacts with a plurality of single-domain management and control agents in the management and control range of the cross-domain management and control agent, and determines a device fault causing the end-to-end cross-domain service to fail; and the cross-domain management and control agent cooperates with the plurality of single-domain management and control agents, and performs closed-loop processing on the fault of the end-to-end cross-domain service and closed-loop processing on the device fault.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a fault handling method, electronic device, storage medium, and program product. Background Technology

[0002] The global communications industry is moving from the interconnected era and the cloud era to the intelligent era. Faced with market competition, industry-wide digital transformation, and the challenges brought about by increased network complexity, such as network planning, optimization, operation and maintenance, reliability, agility, and reduction of operation expenses (OPEX), the development of optical transport networks (OTN) towards intelligence has become inevitable.

[0003] Among the many intelligent capabilities required for Autonomous Networks Optical Networks (AN OTN), automatic closed-loop control for fault monitoring and handling is a key core capability for realizing the intelligence of OTN networks. OTN optical network faults can be mainly divided into two categories: equipment faults and service faults. Service faults refer to abnormal states in OTN services (including EOO / EOS / OSU / ODUk / OCH, etc.), such as interruptions, packet loss, increased latency, or power attenuation, causing the service to be unable to operate normally or transmit service signals properly, thus affecting user experience. Equipment faults refer to abnormal operation of OTN network equipment (including boards, optical fibers, optical modules, power supplies, fans, etc.), resulting in a series of alarms and causing faults in the OTN services carried or associated with it. Equipment faults are usually the cause of service faults, affecting the normal operation of related OTN network services.

[0004] How to achieve equipment failure monitoring and handling, as well as service failure monitoring and handling, are technical problems that need to be solved in intelligent optical networks. Summary of the Invention

[0005] This application provides a fault handling method, electronic device, storage medium, and program product that can solve the problem of how to implement equipment fault monitoring and handling and service fault monitoring and handling in an intelligent optical network.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows: Firstly, a fault handling method is provided, comprising: a cross-domain management and control agent detecting a fault in an end-to-end cross-domain service through monitoring, wherein the management and control scope of the cross-domain management and control agent includes multiple management and control domains, and each management and control domain corresponds to a single-domain management and control agent; the cross-domain management and control agent interacts with the multiple single-domain management and control agents within its management and control scope to determine a device fault that caused the end-to-end cross-domain service fault; the cross-domain management and control agent collaborates with the multiple single-domain management and control agents to perform closed-loop processing of both the end-to-end cross-domain service fault and the device fault.

[0007] Secondly, a fault handling method is provided, comprising: a single-domain management agent sensing a device fault in its own management domain through monitoring; the single-domain management agent collaborating with a cross-domain management agent to perform closed-loop processing of the device fault and closed-loop processing of end-to-end cross-domain service faults caused by the device fault.

[0008] Thirdly, an electronic device is provided, comprising a processor and a memory, the memory storing a program or instructions executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the fault handling method described in the first or second aspect above.

[0009] Fourthly, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the fault handling method as described in the first or second aspect above.

[0010] Fifthly, a computer program product is provided, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform the steps of the fault handling method as described in the first or second aspect above.

[0011] In the technical solution provided in this application embodiment, a cross-domain management and control intelligent agent detects a failure in an end-to-end cross-domain service through monitoring. The management scope of the cross-domain management and control intelligent agent includes multiple management domains, each corresponding to a single-domain management and control intelligent agent. The cross-domain management and control intelligent agent interacts with the multiple single-domain management and control intelligent agents within its management scope to determine the device failure that caused the end-to-end cross-domain service failure. The cross-domain management and control intelligent agent collaborates with the multiple single-domain management and control intelligent agents to perform closed-loop processing of both the end-to-end cross-domain service failure and the device failure. Through the technical solution provided in this application embodiment, the cross-domain management and control intelligent agent can achieve closed-loop processing of both end-to-end cross-domain service failures and device failures by collaborating with single-domain management and control intelligent agents, thereby realizing closed-loop intelligent processing of both service failure monitoring and processing and device failure monitoring and processing.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0014] Figure 1 This illustrates an information and communication network operation and management process in the relevant technology; Figure 2 A schematic diagram of the system architecture in an exemplary embodiment of this application is shown; Figure 3 A flowchart of a fault handling method in an exemplary embodiment of this application is shown; Figure 4 This illustrates a closed-loop process in which a cross-domain management agent and a single-domain management agent collaboratively handle business failures in an exemplary embodiment of this application. Figure 5 This illustrates a closed-loop process in which a cross-domain management agent and a single-domain management agent collaboratively handle device faults in an exemplary embodiment of this application. Figure 6 This illustrates a closed-loop process in which a cross-domain management agent and a single-domain management agent collaboratively handle business failures in another exemplary embodiment of this application. Figure 7 This illustrates a closed-loop process in which a cross-domain management agent and a single-domain management agent collaboratively handle business failures in yet another exemplary embodiment of this application. Figure 8 This illustrates a closed-loop process in which a cross-domain management agent and a single-domain management agent collaboratively handle business failures in yet another exemplary embodiment of this application; Figure 9 A flowchart of a fault handling method in another exemplary embodiment of this application is shown; Figure 10 This illustrates a closed-loop process in which a single-domain management agent and a cross-domain management agent collaboratively handle business failures in an exemplary embodiment of this application. Figure 11 This illustrates a closed-loop process in which a single-domain control agent and a cross-domain control agent collaboratively handle device faults in an exemplary embodiment of this application. Figure 12 This illustrates a closed-loop process in which a single-domain control agent and a cross-domain control agent collaboratively handle device faults in another exemplary embodiment of this application; Figure 13 This illustrates a closed-loop process in which a single-domain management agent and a cross-domain management agent collaboratively handle business failures in another exemplary embodiment of this application. Figure 14 This illustrates a closed-loop process in which a single-domain management agent and a cross-domain management agent collaboratively handle business failures in yet another exemplary embodiment of this application. Figure 15 This application illustrates a closed-loop SOP process for OTN network fault monitoring and handling implemented collaboratively by an LM-Based OSS / OMC Agent in an exemplary embodiment. Figure 16 This application illustrates a closed-loop SOP process for OTN network fault monitoring and handling, implemented collaboratively by an LM-Based OSS / OMC Agent; Figure 17 This is a structural block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0015] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0016] Autonomous Networks (AN) technology originates from the Autonomous Networks (AN) concept proposed by the TM Forum in 2019. Currently, the AN concept has achieved industry consensus, aiming to enable the digital transformation of operator networks through the integration of network and digital technologies. This provides innovative ICT services and user experiences with zero waiting time, zero contact, and zero failures for vertical industries and consumer users, and builds self-configuration, self-healing, and self-optimizing network capabilities throughout the entire lifecycle of operator network operations. Therefore, the Autonomous Networks (AN) technology, derived from this concept, can achieve intelligent management, superior user experience, and flexible openness in optical networks through its intelligent operation and maintenance capabilities of self-configuration, self-recovery, and self-optimization, thereby meeting customer needs for improved user experience and reduced operating costs.

[0017] Among the many intelligent capabilities required for self-intelligent optical networks, automatic closed-loop control for fault monitoring and handling is a key core capability for realizing the self-intelligence of OTN networks. OTN optical network faults can be mainly divided into two categories: equipment faults and service faults.

[0018] Among them, service failure refers to abnormal states such as interruption, packet loss, increased latency, or power attenuation of OTN services (including EOO / EOS / OSU / ODUk / OCH, etc.), which cause the service to be unable to operate normally, unable to transmit service signals normally, and affect user experience.

[0019] Equipment failure refers to an abnormal operation of OTN network equipment (including boards, optical fibers, optical modules, power supplies, fans, etc.), resulting in a series of alarms and causing failures in the OTN services it carries or is associated with. Equipment failure is usually the cause of service failures, affecting the normal operation of related OTN network services.

[0020] The closed-loop, self-regulating standard operating procedures (SOPs) for both service fault monitoring and equipment fault monitoring should follow the general workflow design for information and communication network operation and management. The general workflow for information and communication network operation and management is as follows: Figure 1 As shown, it includes the following five steps: Intent management includes translating the intentions of operators or network users regarding network operation management or business services into specific management operations and strategies, as well as evaluating and providing feedback on the actual achievement of the aforementioned intentions. These tasks affect one or more of the following processes: perception, analysis, decision-making, and execution.

[0021] Sensing: This includes collecting raw data from the network (including terminals) and performing necessary preprocessing on the data (such as data cleaning, enhancement, statistics, etc.) to monitor and sense network information (including network performance, network anomalies, network events, etc.).

[0022] Analysis: This includes analyzing the acquired network information, including analyzing the current state of the network and predicting future trends based on historical data, to generate operational options or suggestions that may meet the needs of network operation and management.

[0023] Decision-making: This includes determining the most suitable executable network operations management operations (such as network configuration or adjustment) to meet network operations management needs by evaluating the operational options or recommendations provided through the evaluation and analysis process.

[0024] Execution: This includes implementing and making effective network operation and management operations within the network, based on decisions made according to the decision-making process.

[0025] Therefore, in accordance with the general process of information and communication network operation and management mentioned above, the five steps included in the closed-loop intelligent workflow SOP design for OTN network fault monitoring and handling, which includes two scenarios: service fault monitoring and handling and equipment fault monitoring and handling, are described as follows.

[0026] 1. Design of a closed-loop, self-sustaining standard operating procedure (SOP) for monitoring and handling business failures: a. Intent Management: OTN network operators or users specify the OTN network service objects that need to be monitored according to the intent requirements of OTN network operation and management. Through processes such as perception, analysis, decision-making and execution, the monitoring service objects are promptly perceived, analyzed and repaired to ensure normal network operation.

[0027] b. Perception: Monitor and collect relevant event data such as alarms, abnormal performance, and abnormal logs of OTN network services, and perceive OTN network service failures accordingly.

[0028] c. Analysis: Simultaneously analyze the impact of OTN network equipment failures that cause OTN network service failures on OTN network services (affected services will usually experience failures).

[0029] d. Decision-making: For OTN network services that have experienced failures, and based on the impact of the failures of related OTN network equipment on the OTN network services, design OTN network service recovery plans according to Service Level Agreement (SLA) levels; after the equipment causing the OTN network service failures is repaired, design OTN network service recovery plans according to SLA levels.

[0030] e. Execution: Perform OTN network service restoration in accordance with the OTN network service restoration plan; perform OTN network service restoration and verification in accordance with the OTN network recovery plan.

[0031] 2. Design of a closed-loop intelligent workflow SOP for equipment fault monitoring and handling: a. Intent Management: OTN network operators or users specify the OTN network devices that need to be monitored according to the intent requirements of OTN network operation and management. Through processes such as perception, analysis, decision-making and execution, the monitoring devices are promptly perceived, analyzed and repaired to ensure normal network operation.

[0032] b. Sensing: Monitor and collect event data such as alarms, abnormal performance, and abnormal logs that occur on OTN network devices, and use this data to sense OTN network device faults.

[0033] c. Analysis: Analyze the impact of OTN network equipment failures on OTN network services, and identify and locate the root causes of equipment failures that cause OTN network service failures.

[0034] d. Decision: Design a fault repair plan for OTN network equipment.

[0035] e. Execution: Perform OTN network equipment fault repair and verification in accordance with the OTN network equipment fault repair plan.

[0036] How can an OTN self-intelligent optical network management and control system comprehensively achieve intelligent, automatic, closed-loop control functions in complex scenarios such as equipment fault monitoring and service fault monitoring and handling across multiple vendor OTN optical network domains? The emergence of large-scale intelligent agent technology offers a possibility for solving this problem.

[0037] The rapid development of AI-generated content (AIGC) big data technology is reshaping the industry landscape, and global demand for automation and intelligent applications has reached unprecedented levels. Most telecom operators and equipment manufacturers have launched automation and digitalization strategies, introducing various innovative big data application solutions to achieve network automation and intelligence, thereby empowering employees, reducing costs, improving efficiency, and enhancing the business experience.

[0038] For example, proactive monitoring of network changes and prediction of quality issues can prevent business disruptions in advance; intelligent network operation and maintenance can be enabled through technologies such as self-decision-making in network troubleshooting and self-closed-loop network optimization. Current research and practical results show that large-scale models exhibit significant advantages over traditional AI models in intent understanding and decision-making. In particular, large-scale models built on massive amounts of data possess excellent generalization capabilities, enabling the rapid deployment and application of AI by masking local differences. Furthermore, the advantages of large-scale models in intent understanding allow them, equipped with vast amounts of telecommunications knowledge, to control, use, configure, and manage network devices through interfaces, accelerating the network's move towards self-intelligence.

[0039] Currently, the industry generally believes that intelligent agents are the basic architecture and mainstream form of commercial application for large-scale models, and also represent the development trend and application direction of large-scale models.

[0040] Large-model-based agents (LM-based agents) generally refer to intelligent computing entities that utilize large-scale, richly trained language models or multimodal models as core components. These agents possess highly intelligent understanding and generative capabilities, enabling them to more accurately perceive their environment, make decisions, and execute actions. Large models can serve as the brain or controller of these agents, and their perception and action space can be expanded through strategies such as multimodal perception and tool utilization. Through techniques like thought chains (CoT) and problem decomposition, these large-model-based agents can exhibit considerable reasoning and planning abilities. Similarly, large language models, pre-trained on massive corpora, demonstrate significant generalization capabilities, enabling seamless task transfer without updating initial task instruction parameters. Large-model-based agents have been applied in various practical scenarios, such as software development, scientific research, and especially network intelligent operation and maintenance. Due to their natural language understanding and generation capabilities, they can interact directly with humans and with each other using natural language, enabling collaboration between multiple agents and between multiple agents and humans.

[0041] In summary, leveraging the advantages of large-scale intelligent agents, especially LM-based multi-agent collaboration technology, to enable OTN optical network management and control systems to possess closed-loop high-order self-intelligent capabilities in environments with intertwined application scenarios has become a hot emerging technology in the industry for self-intelligent optical networks (AN OTN).

[0042] The main architecture of hierarchical LM-based multi-agent collaboration in the field of OTN network communication is that the cross-vendor domain management agent (OSS Agent, i.e., cross-domain management agent) used by the OTN network operator at the upper layer collaborates with the single-domain management agent (OMC Agent, i.e., single-domain management agent) used by the OTN network equipment vendor at the lower layer, thereby simultaneously completing closed-loop self-smart processing of multiple scenarios such as OTN network service fault monitoring and processing, and equipment fault monitoring and processing.

[0043] How to design such a collaborative mechanism, such as the SOP design of multi-scenario closed-loop self-intelligence to guide LM-based multi-agents to generate multi-scenario closed-loop self-intelligence thought chains CoT; and the number of dialogue rounds and content of upper and lower layer multi-agents in the LM-based hierarchy, have become the technical bottlenecks that need to be solved in the realization of high-level self-intelligence in OTN networks in related technologies.

[0044] To address this, this application provides a fault handling solution. This solution employs LM-based multi-agent collaboration to achieve closed-loop self-intelligence in multiple OTN network scenarios. In this application embodiment, a collaborative mechanism is adopted under the upper and lower layer architecture of LM-based cross-vendor OTN domain management agent (OSS Agent, also known as cross-domain management agent) and single-domain management agent (OMC Agent) to complete closed-loop self-intelligence in two scenarios: service fault monitoring and processing and equipment fault monitoring and processing.

[0045] Figure 2 The following are schematic diagrams illustrating the architecture of a network system applied in some embodiments of this application, such as... Figure 2 As shown in the embodiments of this application, an Operation Support System (OSS) is included. The OSS manages multiple control domains and can therefore also be referred to as a cross-domain transfer workbench or a multi-domain transfer workbench. This multi-domain transfer workbench is used to manage multiple control domains within its control scope. A multi-domain control agent (OSS Agent, also referred to as a cross-domain control agent) can be set up on the multi-domain transfer workbench. The control scope of the multi-domain transfer workbench includes multiple Operation and Maintenance Centers (OMCs). Each control domain's corresponding OMC can also be referred to as a single-domain control system. Each single-domain control system manages the OTN single domain under its control domain. Different single-domain control systems may correspond to different vendors. In this application, a single-domain control agent (OMC Agent) can be set up on a single-domain control system.

[0046] The intelligent agent in this application can be a large-model-based intelligent agent, which may include components such as large models, memory, planning, and execution. This intelligent agent can use a large-scale language model or multimodal model with rich training data as its core component, and has highly intelligent understanding and generation capabilities, thereby enabling it to more accurately perceive the environment, make decisions, and execute actions as an intelligent computing entity.

[0047] In this application, the hierarchical dual-agent intelligent agent technology (components such as large model, memory, planning, and execution) can effectively manage the task cycle of a specified fault monitoring and processing closed-loop instance. This includes controlling the creation, execution, suspension, listening, waiting, and wake-up of closed-loop task instances, initiating multiple rounds of conversations based on the state of each scenario, and seamlessly connecting the system's automatic processing scenarios and manual processing scenarios that occur in the closed-loop instance. This achieves the pipelined and coherent implementation of the closed-loop process SOP and RPA automation.

[0048] In addition, in this application, the dual-agent intelligent agent technology can simultaneously control and realize the dual closed loop of business fault monitoring and processing and equipment fault monitoring and processing, with the dual closed loops intertwined.

[0049] By designing and guiding large-scale model prompts, the dual-closed-loop SOP process can be transformed into a large-scale model thought chain of hierarchical dual-agent intelligent agents, thereby achieving automated and intelligent closed-loop control for fault monitoring and handling.

[0050] Figure 3 A flowchart illustrating a fault handling method shown in an exemplary embodiment of this application is provided. This method can be executed by a cross-domain management agent. Figure 3 As shown, the fault handling method mainly includes the following steps.

[0051] S310, the cross-domain management intelligent agent detects end-to-end cross-domain service failures through monitoring.

[0052] The cross-domain management agent's management scope includes multiple management domains, and each management domain corresponds to a single-domain management agent.

[0053] Among them, cross-domain management and control intelligent agents can be Figure 2 Multi-domain OSS Agent.

[0054] In some embodiments of this application, the cross-domain management agent can monitor and detect end-to-end (E2E) cross-domain OTN service failures, such as E2E cross-domain OTN service interruption or degradation.

[0055] S320, the cross-domain management agent interacts with multiple single-domain management agents within its management scope to determine the device fault that caused the end-to-end cross-domain service to fail.

[0056] In some embodiments of this application, the cross-domain management agent can monitor and perceive E2E cross-domain OTN service failures, while the single-domain management agent can simultaneously monitor and perceive devices within its own domain. The cross-domain management agent and the single-domain management agent interact to determine the device failures that cause E2E cross-domain service failures.

[0057] S330, the cross-domain management and control intelligent agent collaborates with the multiple single-domain management and control intelligent agents to perform closed-loop processing of the end-to-end cross-domain service faults and the device faults.

[0058] In some embodiments of this application, the cross-domain management agent can collaborate with multiple single-domain management agents to perform closed-loop self-learning on end-to-end cross-domain business faults and on device faults, thereby realizing the pipelined and continuous nature of closed-loop process SOPs and RPA automation.

[0059] Through the technical solutions provided in the embodiments of this application, the cross-domain management and control intelligent agent can collaborate with the single-domain management and control intelligent agent to achieve closed-loop processing of end-to-end cross-domain business faults and equipment faults, thereby realizing closed-loop self-intelligence in both business fault monitoring and processing and equipment fault monitoring and processing scenarios.

[0060] In some embodiments of this application, when service and equipment failures occur, restoring services and ensuring uninterrupted service experience are prioritized as the primary objectives, guaranteeing that the service SLA commitment remains unaffected and effectively reducing the SLA service default rate. Therefore, as Figure 4 As shown, in these embodiments, the cross-domain management agent collaborates with the multiple single-domain management agents to perform closed-loop processing of faults in the end-to-end cross-domain service, which may include the following steps: S410, the cross-domain management agent obtains at least one first fault information reported by the first single-domain management agent.

[0061] Wherein, the first device fault information is used to indicate the impact range of device faults in the first control domain corresponding to the first single-domain control agent on the end-to-end cross-domain service carried, and the at least one first single-domain control agent is one or more single-domain control agents among the plurality of single-domain control agents.

[0062] In these embodiments, the first single-domain management agent is a single-domain management agent that monitors and senses equipment failures within the scope of the cross-domain management agent. In other words, the first single-domain management agent is a single-domain management agent that monitors and senses equipment failures occurring in its own management domain within the scope of the cross-domain management agent.

[0063] In these embodiments, the first single-domain management can monitor and perceive device faults within the first management domain (i.e., the management domain managed by the first single-domain management domain), analyze and determine the impact range of the device faults in the first management domain on the cross-domain services it carries (i.e., the device fault blast radius), and report the impact range of the device faults on the cross-domain services it carries as the first fault information to the cross-domain management intelligent agent.

[0064] In some embodiments, the first single-domain management agent can initiate a session with the cross-domain management agent and report the first fault information to the cross-domain management agent through the session. The specific reporting method is not limited in this application embodiment.

[0065] S420, the cross-domain management and control agent determines the rerouting recovery path of the end-to-end cross-domain service based on the acquired fault information of at least one of the first devices.

[0066] In these embodiments, the cross-domain management agent determines the rerouting recovery path for end-to-end cross-domain services based on the first fault information reported by at least one first single-domain management agent.

[0067] In some embodiments, if multiple end-to-end cross-domain services fail, the cross-domain management agent can determine the rerouting and recovery order of each end-to-end cross-domain service based on the SLA level of each service.

[0068] S430, the cross-domain management agent sends a first instruction to at least one second single-domain management agent.

[0069] Wherein, the first instruction is used to instruct the second single-domain management agent to create a cross-segment recovery path of the rerouting recovery path within the second management domain corresponding to the second single-domain management agent, and the at least one second single-domain management agent is one or more single-domain management agents among the plurality of single-domain management agents.

[0070] In some embodiments, the cross-domain management agent can initiate a session with each of the second single-domain management agents, and send the first instruction to the second single-domain management agent through the session. The specific method is not limited in the embodiments of this application.

[0071] In these embodiments, the second single-domain management agent involved in the rerouting path of end-to-end cross-domain services may be the same as or different from the first single-domain management agent that experienced a device failure. For example, the at least one second single-domain management agent may include the at least one first single-domain management agent, or some of the at least one second single-domain management agents may be first single-domain management agents while others are other single-domain management agents. Alternatively, the at least one second single-domain management agent may be completely identical to the at least one first single-domain management agent, or none of the at least one second single-domain management agents may be first single-domain management agents. The specific situation depends on the actual device failure situation and the rerouting recovery path of end-to-end cross-domain services in the actual application, and is not specifically limited in the embodiments of this application.

[0072] In these embodiments, after receiving the first instruction, the second single-domain management agent creates a cross-segment recovery path for the rerouting recovery path within the second management domain corresponding to the second single-domain management agent.

[0073] In some embodiments, if multiple end-to-end cross-domain services fail, the cross-domain management agent can also send the rerouting recovery order of each end-to-end cross-domain service to at least one second single-domain management agent via a first instruction. After receiving the first instruction, the second single-domain management agent can create cross-segment recovery paths for each end-to-end cross-domain service within the second management domain, according to the rerouting recovery order of each end-to-end cross-domain service.

[0074] In the above embodiments, the impact analysis of cross-domain services and the design of service recovery solutions are implemented. Taking OTN services as an example, the SOP process and functional requirements are described as follows: 1. Detecting business failures: The cross-domain management and control intelligent agent (OSS Agent) monitors and detects E2E cross-domain OTN business failures (interruptions, degradation).

[0075] 2. Device Fault Sensing: Simultaneously, the Single Domain Management Agent (OMC Agent) monitors and senses device faults in its own domain, analyzes and determines the impact range of the device faults on the cross-domain services it carries, and reports the corresponding cross-domain service impact range (i.e., the device fault blast radius) to the cross-domain management agent.

[0076] 3. Design a business recovery plan: Based on the reported fault blast radius, the cross-domain management and control agent determines the order of cross-domain business rerouting recovery according to the SLA level, and issues it to the single-domain management and control agent to execute the creation of cross-domain business rerouting recovery paths across segments in each management and control domain.

[0077] In the above embodiments, when service and equipment failures occur, services are restored first, thereby ensuring that the service experience is not affected.

[0078] In some embodiments, such as Figure 5 As shown, the cross-domain management and control agent, in collaboration with the multiple single-domain management and control agents, can perform closed-loop processing of the device faults, which may include the following steps: S510, the cross-domain management agent obtains second device fault information from the at least one first single-domain management agent.

[0079] The second equipment fault information includes at least one of the root cause of the equipment fault and the equipment fault repair suggestion within the first control domain.

[0080] In these embodiments, the second device fault information may include the root cause of the device fault that occurred in the first control domain, or the second device fault information may include fault repair suggestions for the device fault that occurred in the first control domain, or it may include both the root cause of the device fault and the device fault modification suggestions, so that the cross-domain control agent can collaborate with the first single-domain control agent based on the second device fault information to achieve closed-loop processing of device faults.

[0081] For example, the second device fault information could include: the root cause of device A's fault is a port configuration error, and the suggested solution for device A is to reconfigure the port configuration information of device A. As another example, the second device fault information could also include: the root cause of device B's fault could be fiber optic aging, and the suggested solution is to replace the fiber optic cable corresponding to device B.

[0082] In some embodiments, a cross-domain management agent may proactively request a first single-domain management agent to obtain second device fault information. In these embodiments, the second device fault information obtained by the cross-domain management agent from the at least one first single-domain management agent may include the following steps: Step 11, the cross-domain management agent sends a first request to the at least one first single-domain management agent, the first request being used to request the first single-domain management agent to identify the root cause of the device failure and / or device failure repair suggestions that caused the end-to-end cross-domain service to fail. Step 12: Receive the second device fault information reported by the at least one first single-domain control agent.

[0083] For example, after monitoring and detecting an E2E cross-domain OTN service failure, the cross-domain management agent, through end-to-end cross-domain service failure delimitation analysis, sends a first request to the first single-domain management agent. This first request can specify the faulty end-to-end cross-domain service to the first single-domain management agent, requesting analysis of the root cause of the failure. The first single-domain management agent identifies and locates the device failure root cause of the end-to-end cross-domain service failure caused by the first management domain, and reports it to the cross-domain management agent.

[0084] In these embodiments, the first single-domain management agent can obtain the root cause of device failure / device failure repair suggestions through an AI model. For example, a device failure root cause identification model can be set up in the first single-domain management agent. This device failure root cause identification model can be trained through a large amount of sample data, thereby identifying the root cause of device failure. For example, the device failure root cause identification model can identify the root cause of device failure based on the alarm information output by the device. The specific method is not limited in the embodiments of this application.

[0085] In some embodiments, the cross-domain management agent can initiate a session with the first single-domain management agent, through which a first request is sent to the first single-domain management agent, and the first single-domain management agent sends the aforementioned second device fault information to the cross-domain management agent through the same session. The specific information interaction method is not limited in this application embodiment.

[0086] Through the above embodiments, when a cross-domain management agent detects an end-to-end cross-domain service failure, it can request the first single-domain management agent to analyze the root cause of the failure. Thus, the cross-domain management agent can obtain the root cause of the device failure and / or device modification suggestions when an end-to-end cross-domain service failure occurs.

[0087] In other embodiments, the first single-domain management agent may also proactively report the aforementioned second device fault information. In these embodiments, the second device fault information obtained by the cross-domain management agent from the at least one first single-domain management agent may include: the second device fault information proactively reported by the cross-domain management agent after the at least one first single-domain management agent has monitored and detected a device fault in the first management domain.

[0088] In these embodiments, the first single-domain management agent can proactively identify and locate the root cause of device failure in its own domain, and report the root cause of device failure / device failure repair suggestions to the cross-domain management agent. This enables the cross-domain management agent to obtain the root cause of device failure / device failure repair suggestions in a timely manner when a device in the first management domain fails, and thus to repair the device failure in a timely manner.

[0089] S520, based on at least one of the second device fault information, obtain a repair plan for the device fault; In this step, the cross-domain management agent can design a modification plan for the OTN network device fault based on the acquired fault information of at least one second device. For example, the cross-domain management agent can obtain a modification plan for the OTN network device fault through an AI model. For example, this AI model can design a modification plan for the OTN network device fault based on at least one fault information of a second device.

[0090] S530, Perform the repair of the equipment fault according to the repair plan and verify the repair.

[0091] In this step, cross-domain management intelligence can complete the fault repair and verification of network devices according to the equipment fault repair plan, thereby realizing the self-intelligent closed-loop processing of equipment faults.

[0092] In some embodiments, after modifying the device malfunction and verifying the repair, the failed end-to-end cross-domain service can be further restored. In these embodiments, such as... Figure 6As shown, the cross-domain management agent collaborates with the multiple single-domain management agents to perform closed-loop processing of faults in the end-to-end cross-domain service, and also includes: S440, Based on the repair verification results, obtain the response plan for the end-to-end cross-domain service according to the service level agreement spatiotemporal sorting.

[0093] In these embodiments, after the device fault is repaired, if the repair verification is successful, the end-to-end cross-domain services that underwent a failover due to the device fault (i.e., the rerouting process described above) can be restored to their normal state. In S440, the cross-domain management agent can obtain the recovery plan for each end-to-end cross-domain service based on the device fault repair verification results and according to the spatiotemporal sorting of the SLA level of each end-to-end cross-domain service.

[0094] S450, based on the response scheme, send a second instruction to at least one third single-domain management agent.

[0095] The second instruction carries the response order of multiple end-to-end cross-domain services and the response time point of each end-to-end cross-domain service, instructing the third single-domain management agent to perform cross-segment responses of the end-to-end cross-domain services within the third management domain corresponding to the third single-domain management agent. The at least one third single-domain management agent is one or more of the multiple single-domain management agents.

[0096] In these embodiments, the second instruction indicates the response order of multiple end-to-end cross-domain services and the response time corresponding to each end-to-end cross-domain service, wherein each end-to-end cross-domain service has a response time point, and the response time points of different end-to-end cross-domain services may be different.

[0097] In these embodiments, the third single-domain management agent may be the same as or different from the first or second single-domain management agent described above. Specifically, it is related to the single-domain management agent involved in the response scheme for end-to-end cross-domain services, and no specific limitation is made in the embodiments of this application.

[0098] In these embodiments, after receiving the second instruction, the third single-domain management agent can, according to the reply order of each end-to-end cross-domain service and the reply time point of each end-to-end cross-domain service, execute the corresponding end-to-end cross-domain service cross-segment reply within the third management domain at each corresponding reply time point, thereby realizing the reply of end-to-end cross-domain services.

[0099] In some embodiments, after the third single-domain management agent executes the service response according to the second instruction, it can report the service response execution result to the cross-domain management agent, so that the cross-domain management agent can be aware of the service response execution result. Therefore, in these embodiments, as... Figure 7 As shown, the cross-domain management agent collaborates with the multiple single-domain management agents to perform closed-loop processing of faults in the end-to-end cross-domain service, which may further include the following steps: S460, the cross-domain management agent receives a response message sent by at least one third single-domain management agent.

[0100] The response message carries the cross-segment response result of the third single-domain management agent executing the end-to-end cross-domain service within the third management domain.

[0101] Through these embodiments, the cross-domain management agent can obtain the cross-segment response results executed by each third single-domain management agent, and then perform corresponding processing based on the cross-segment response results. For example, if a response message sent by a third single-domain management agent indicates that the cross-segment response within the third management domain corresponding to that third single-domain management agent was unsuccessful, the cross-domain management agent can assume that there may be a device malfunction within that third management domain. Therefore, it can request that the third single-domain management agent to monitor and detect the device malfunction. After detecting the device malfunction, it can proceed according to... Figure 5 The illustrated process performs closed-loop handling of equipment failures, including equipment failure recovery and verification. Only after successful equipment failure recovery and verification is the business response re-executed. This ensures the success of the business response.

[0102] In some embodiments, to ensure the transmission quality of end-to-end cross-domain services after service response, such as... Figure 8 As shown, the cross-domain management agent collaborates with the multiple single-domain management agents to perform closed-loop processing of faults in the end-to-end cross-domain service, which may further include the following steps: S470, the cross-domain management and control intelligent agent tests and verifies the end-to-end cross-domain service after the response.

[0103] The parameters to be tested and verified include at least one of the following: latency, packet loss, optical power, and bit error rate.

[0104] In these embodiments, the cross-domain management agent can test and verify one or more end-to-end cross-domain services after the response, verifying whether at least one of the latency, packet loss, optical power, and bit error rate of the end-to-end cross-domain service after the response meets the requirements. If the requirements are not met, the agent can further reroute the end-to-end cross-domain service that does not meet the requirements or re-determine the response scheme for the end-to-end cross-domain service that does not meet the requirements, so as to ensure the transmission quality of the end-to-end cross-domain service.

[0105] Based on the same inventive concept, this application also provides another fault handling method, which is similar to the above. Figures 4 to 8 The corresponding single-domain control agent-side execution method has the same characteristics as the above. Figures 4 to 8 The methods shown are the same or similar. The following mainly describes the method executed by a single-domain control agent. For other matters not covered, please refer to the above. Figures 4 to 8 The relevant descriptions in the document will not be repeated here.

[0106] Figure 9 A flowchart illustrating a fault handling method shown in an exemplary embodiment of this application is provided. This method can be executed by a single-domain management agent. Figure 9 As shown, the fault handling method mainly includes the following steps.

[0107] S910, a single-domain management agent, detects device faults in its managed domain through monitoring.

[0108] In some embodiments of this application, a single-domain control agent can be Figure 2 The single-domain OMC Agent can be configured with components such as large models, memory, planning, and execution, and can monitor and perceive device faults within its control domain.

[0109] S920, the single-domain management agent and the cross-domain management agent work together to perform closed-loop processing of the device fault and the end-to-end cross-domain service fault caused by the device fault.

[0110] In this application, a single-domain management agent can collaborate with a cross-domain management agent to complete closed-loop processing of equipment failures and end-to-end cross-domain service failures. For example, a single-domain management agent can identify the impact range of equipment failures in its own management domain on the end-to-end cross-domain services it carries, and report this impact range to the cross-domain management agent. This allows the cross-domain management agent to recover from end-to-end cross-domain service failures based on the impact range of equipment failures in each management domain. As another example, a single-domain management agent can locate and identify the root cause and / or recovery suggestions for equipment failures in its own management domain, and report the root cause and / or recovery suggestions to the cross-domain management agent. This allows the cross-domain management agent to determine a repair plan for the equipment failure based on the root cause and / or recovery suggestions reported by the single-domain management agent, and then repair the equipment failure.

[0111] Through the technical solutions provided in the embodiments of this application, a single-domain management and control agent can collaborate with a cross-domain management and control agent to complete the intelligent closed-loop processing of equipment failures and end-to-end cross-domain service failures.

[0112] In some embodiments, such as Figure 10 As shown, the collaboration between a single-domain management agent and a cross-domain management agent to perform closed-loop processing of end-to-end cross-domain service failures caused by the device failure may include the following steps: S1010, the single-domain management and control agent obtains the impact range of the device failure on the end-to-end cross-domain services it carries; S1020, First fault information is reported to the cross-domain management and control intelligent agent, wherein the first device fault information is used to indicate the scope of the impact of the device fault on the end-to-end cross-domain service; S1030, receive a first instruction sent by the cross-domain management and control agent, wherein the first instruction is used to instruct the creation of a cross-segment recovery path for the rerouting recovery path of the end-to-end cross-domain service within the management and control domain; S1040, according to the first instruction, create a cross-segment recovery path for the rerouting recovery path of the end-to-end cross-domain service within the control domain.

[0113] In these embodiments, a single-domain management agent can collaborate with a cross-domain management agent to first restore end-to-end cross-domain services, ensuring that the service experience is not affected.

[0114] In some embodiments, such as Figure 11 As shown, the collaboration between a single-domain management agent and a cross-domain management agent to perform closed-loop processing of the device fault may include the following steps: S1110, Receive a first request sent by the cross-domain management and control agent, wherein the first request is used to request identification of the root cause of the equipment failure that caused the end-to-end cross-domain service failure and / or equipment failure repair suggestions; S1120, Identify the root cause of the equipment failure that caused the end-to-end cross-domain service failure and / or equipment failure repair recommendations; S1130, report the second device fault information to the cross-domain management and control agent, wherein the second device fault information includes at least one of the root cause of device fault and device fault repair suggestion within the first management and control domain.

[0115] In these embodiments, the cross-domain management agent receives the second device fault information, obtains a repair plan for the device fault, repairs the device fault according to the obtained repair plan, and verifies the repaired device fault.

[0116] In some embodiments, a single-domain management agent can also proactively report the aforementioned second device fault information; therefore, as Figure 12 As shown, the single-domain management agent and the cross-domain management agent collaborate to perform closed-loop processing of the device fault, which may include the following steps: S1210, after the device failure in this control domain is detected by monitoring, identify the root cause of the device failure that caused the end-to-end cross-domain service failure and / or device failure repair suggestions. S1220, Actively report second device fault information to the cross-domain management and control agent, wherein the second device fault information includes at least one of the root cause of device fault and device fault repair suggestion within the first management and control domain.

[0117] In these embodiments, the cross-domain management agent receives the second device fault information, obtains a repair plan for the device fault, repairs the device fault according to the obtained repair plan, and verifies the repaired device fault.

[0118] In some embodiments, after the equipment failure is repaired, end-to-end cross-domain services can be resumed, such as... Figure 13 As shown, the single-domain management agent and the cross-domain management agent collaborate to perform closed-loop processing of end-to-end cross-domain service failures caused by the device failure, which may further include the following steps: S1050, the single-domain management agent receives the second instruction sent by the cross-domain management agent, the second instruction carrying the reply order of multiple end-to-end cross-domain services and the reply time point of each end-to-end cross-domain service; S1060, according to the reply order of the multiple end-to-end cross-domain services and the reply time point of each end-to-end cross-domain service, execute the cross-segment reply of the end-to-end cross-domain service within the control domain.

[0119] In these embodiments, a single-domain management agent can execute a cross-segment response within its own management domain for end-to-end cross-domain services based on a second instruction sent by a cross-domain management agent, in order to respond to the failed end-to-end cross-domain service.

[0120] In the second instruction, each end-to-end cross-domain service has a response time point. The response time points for different end-to-end cross-domain services may be different, meaning that different end-to-end cross-domain services may respond at different times. After receiving the second instruction, the single-domain management agent executes the corresponding cross-segment response for each end-to-end cross-domain service within its own management domain, according to the response time point of each service.

[0121] In some embodiments, in order for the cross-domain management agent to obtain cross-segment response results, such as Figure 14 As shown, the single-domain management agent and the cross-domain management agent collaborate to perform closed-loop processing of end-to-end cross-domain service failures caused by the device failure, which may further include the following steps: S1070, the single-domain management agent sends a response message to the cross-domain management agent, wherein the response message carries the cross-segment response result of executing the end-to-end cross-domain service within the management domain.

[0122] In these embodiments, the cross-domain management agent can obtain the cross-segment response result executed by the single-domain management agent, and can then perform corresponding processing based on the cross-segment response result. For example, if the response message sent by the single-domain management agent indicates that the cross-segment response within the management domain corresponding to the single-domain management agent was unsuccessful, the cross-domain management agent can assume that there may be a device fault within that management domain. Therefore, it can request the single-domain management agent to monitor and detect the device fault. After detecting the device fault, it can proceed according to... Figure 5 The illustrated process performs closed-loop handling of equipment failures, including equipment failure recovery and verification. Only after successful equipment failure recovery and verification is the business response re-executed. This ensures the success of the business response.

[0123] Taking the scenario of cross-domain business impact analysis & business recovery solution design and cross-domain business fault root cause identification and location as an example, in this application embodiment, the SOP process and functional requirements for this scenario are as follows: 1. The OSS / OMC Agent collaboratively senses E2E cross-domain service failures and the impact of equipment failures on E2E cross-domain services, and first completes the recovery of cross-domain services, implementing the technical concept of "service self-healing first".

[0124] 2. The OMC Agent senses device faults in its domain and actively / automatically identifies and locates them, then reports the root cause / repair suggestions to the OSS Agent.

[0125] In this application, in response to the above-mentioned SOP process and functional requirements, a collaborative mechanism is adopted under the upper and lower layer two-agent architecture of LM-based cross-domain management agent (OSSAgent) and single-domain management agent (OMC Agent) to complete the closed-loop intelligent workflow of two scenarios: OTN network service fault monitoring and processing and equipment fault monitoring and processing. Figure 15 The SOP closed-loop handling process for business failures and equipment failures in this scenario is as follows: Figure 15 As shown, the SOP process for OTN network device fault monitoring and handling mainly includes the following steps: S1501, Monitoring / Sensing Device Faults. This step is implemented by the OMC Agent, which monitors and senses device faults within the domain.

[0126] S1502 and S1503: Analyze equipment failures. These two consecutive steps are implemented by the OSS / OMC Agent in collaboration. The OSS Agent initiates cross-domain service failure delimitation analysis and initiates Session C, requesting the OMC Agent to identify and locate the root cause of the equipment failure that caused the specified batch / single cross-domain service failure.

[0127] In this process, Session C: The OSS Agent requests to identify and locate the root cause of the equipment failure that caused the specified batch / single cross-domain service failure; the OMC Agent reports the root cause / repair suggestions of the equipment failure that caused the specified batch / single cross-domain service failure.

[0128] S1504: Repair Decision, i.e., designing a repair plan for OTN network equipment faults. This step is implemented by the OSS Agent, which designs the repair plan for the OTN network equipment faults.

[0129] S1505: Repair Execution, which involves executing OTN network device fault repair and verification. This step is implemented by the OSS Agent, which completes the network device fault repair and verification according to the OTN network device fault repair plan.

[0130] like Figure 15 As shown, the SOP process for OTN network service fault monitoring and handling mainly includes the following steps: S1511: Monitor / Aware of Service Faults. This step is implemented by the OSS Agent, which monitors and awareness of E2E end-to-end cross-domain OTN service faults (e.g., interruptions, degradation).

[0131] S1512: Analyze and determine the service scope affected by the device failure in this domain. This step is implemented by OSS / OMC Agent in collaboration. The OMC Agent analyzes and determines the scope of E2E end-to-end cross-domain OTN services carried by the network devices in this domain affected by the device failure, and initiates Session A to report the cross-domain service impact range (i.e., the device failure blast radius) to the OSS Agent.

[0132] S1513: Recovery Decision, i.e., designing a service recovery plan. This step is implemented by the OSS Agent. Based on the fault blast radius reported by the OMC Agent, the OSS Agent determines the batch cross-domain service rerouting recovery prioritization plan according to the SLA level, and initiates Session B to issue a rerouting recovery path creation command to the OMC Agent for cross-domain services across segments within each OMC domain. Specifically, in Session B: the OSS Agent issues rerouting recovery path prioritization and creation instructions for batch / single end-to-end E2E cross-domain OTN services; the OMC Agent receives the OSS Agent's instructions and completes the creation of cross-domain OTN service recovery paths across segments within its own domain according to the SLA prioritization.

[0133] S1514: Execute the service recovery plan. This step is implemented by the OMC Agent. The OMC Agent receives OSS instructions through Session B and completes the creation of recovery paths for cross-domain OTN services across segments within this domain, ordered according to the SLA. In Session B, the OSS Agent only informs the OMC Agent of the rerouting recovery path order for batch / single end-to-end E2E cross-domain OTN services. The recovery rerouting for each service across segments within this OTN domain can be pre-calculated by the OMC Agent.

[0134] S1515: Response Decision, i.e., designing a service response plan. This step is implemented by the OSS Agent. Based on the network equipment fault repair verification results from S1505, the OSS Agent designs batch / single end-to-end E2E cross-domain OTN service response plans according to SLA level spatiotemporal order and initiates Session D. In Session D: the OSS Agent sends batch / single service response ordering and timing instructions to the OMC Agent; the OMC Agent receives the OSS Agent's instructions, implements and completes the cross-domain OTN service response simulation / execution within its own domain according to the service response order and time points, and responds to the OSS Agent with the execution results.

[0135] S1516: Response Execution, i.e., executing the service response plan. The main body implementing this step is the OMC Agent. The OMC Agent receives OSS instructions through Session D, implements and completes the response simulation / execution of cross-domain OTN services across segments within this domain according to the service response order and the time point of each service response, and responds to the OSS Agent with the execution result.

[0136] S1517: Business Testing and Verification. This step is implemented by the OSS Agent, which performs tests and verifications on the batch / single end-to-end E2E cross-domain OTN services after receiving the response, including latency, packet loss, optical power, and bit error rate.

[0137] Taking the scenario of cross-domain business impact analysis & business recovery solution design and equipment failure analysis and reporting as an example, in this application embodiment, the SOP process and functional requirements for this scenario are as follows: 1. The OSS / OMC Agent collaboratively senses E2E cross-domain service failures and the impact of equipment failures on E2E cross-domain services, and first completes the recovery of cross-domain services, implementing the technical concept of "service self-healing first".

[0138] 2. The OMC Agent senses device faults in its domain and actively / automatically identifies and locates them, then reports the root cause / repair suggestions to the OSS Agent.

[0139] In this application, in response to the above-mentioned SOP process and functional requirements, a collaborative mechanism is adopted under the upper and lower layer two-agent architecture of LM-based cross-domain management agent (OSSAgent) and single-domain management agent (OMC Agent) to complete the closed-loop intelligent workflow of two scenarios: OTN network service fault monitoring and processing and equipment fault monitoring and processing. Figure 16 The SOP closed-loop handling process for business failures and equipment failures in this scenario is as follows: Figure 16 As shown, the SOP process for OTN network device fault monitoring and handling mainly includes the following steps: S1601: Monitoring / Sensing Device Faults. This step is implemented by the OMC Agent, which monitors and senses device faults within the domain.

[0140] S1602: Analyze equipment failures. This step is implemented by the OMC Agent, which reports the root cause / repair suggestions of the equipment failures that caused the specified batch / single cross-domain service failures and initiates a session. Session C: The OMC Agent actively reports the root cause / repair suggestions of the equipment failures that caused the specified batch / single cross-domain service failures.

[0141] S1603: Repair Decision, i.e., designing a repair plan for OTN network equipment faults. This step is implemented by the OSS Agent, which designs the repair plan for the OTN network equipment faults.

[0142] S1604: Repair Execution, which involves performing OTN network device fault repair and verification. This step is implemented by OSSAgent, which completes the network device fault repair and verification according to the OTN network device fault repair plan.

[0143] Among them, such as Figure 16 As shown, the Standard Operating Procedure (SOP) for OTN network service fault monitoring and handling includes the following steps: S1611: Monitoring / Detecting Service Faults. This step is implemented by the OSS Agent, which monitors and detects E2E end-to-end cross-domain OTN service faults (interruptions, degradation).

[0144] S1612: Analyze and determine the service scope affected by the device failure in this domain. This step is implemented by OSS / OMC Agent in collaboration. The OMC Agent analyzes and determines the scope of E2E end-to-end cross-domain OTN services carried by the network devices in this domain affected by the device failure, and initiates Session A to report the cross-domain service impact range (i.e., the device failure blast radius) to the OSS Agent.

[0145] S1613: Recovery Decision, i.e., designing a service recovery plan. This step is implemented by the OSS Agent. Based on the fault blast radius reported by the OMC Agent, the OSS Agent determines the batch cross-domain service rerouting recovery prioritization plan according to the SLA level, and initiates Session B to issue a rerouting recovery path creation command to the OMC Agent for cross-domain services across segments within each OMC domain. Specifically, in Session B: the OSS Agent issues rerouting recovery path prioritization and creation instructions for batch / single end-to-end E2E cross-domain OTN services; the OMC Agent receives the OSS Agent's instructions and completes the creation of cross-domain OTN service recovery paths across segments within its own domain according to the SLA prioritization.

[0146] S1614: Execute the service recovery plan. This step is implemented by the OMC Agent. The OMC Agent receives instructions from the OSS Agent through Session B and completes the creation of recovery paths for cross-domain OTN services across segments within this domain, ordered according to the SLA. In Session B, the OSS Agent only informs the OMC Agent of the rerouting recovery path order for batch / single end-to-end E2E cross-domain OTN services. The recovery rerouting for each service across segments within this OTN domain can be pre-calculated by the OMC Agent.

[0147] S1615: Response Decision, i.e., designing a service response plan. The OSS Agent is responsible for implementing this step. Based on the network equipment fault repair verification results in step a.5 above, the OSS Agent designs batch / single end-to-end E2E cross-domain OTN service response plans according to SLA level spatiotemporal sorting, and initiates Session D.

[0148] In Session D, the OSS Agent sends batch / single service response sorting and time point instructions to the OMC Agent; the OMC Agent receives the instructions from the OSS Agent, implements and completes the response simulation / execution of cross-domain OTN services across segments within the domain according to the service response sorting and the time point of each service response, and responds to the OSS Agent with the execution results.

[0149] S1616: Response Execution, i.e., executing the service response plan. The main body implementing this step is the OMC Agent. The OMC Agent receives OSS instructions through Session D, implements and completes the response simulation / execution of cross-domain OTN services across segments within this domain according to the service response order and the time point of each service response, and responds to the OSS Agent with the execution result.

[0150] S1617: Business Testing and Verification. This step is implemented by the OSS Agent, which performs tests and verifications on the batch / single end-to-end E2E cross-domain OTN services after receiving the response, including latency, packet loss, optical power, and bit error rate.

[0151] Among them, the above Figure 15 and Figure 16 The main difference in the processes shown is that, Figure 15 The process shown focuses on the upper-layer OSS Agent initiating cross-domain service fault delimitation analysis and initiating Session C to request the OMC Agent to identify and locate the root cause of the device fault that caused the fault of a specified batch / single cross-domain OTN service. This is the upper-layer OSS Agent actively investigating downwards.

[0152] and Figure 16 The process shown focuses on the OMC Agent reporting the root cause / repair suggestion of the device fault that caused the specified batch / single cross-domain OTN service failure, and initiating Session C: The OMC Agent actively reports the root cause / repair suggestion of the device fault that caused the specified batch / single cross-domain service failure, which is the lower-level OMC Agent actively reporting upwards.

[0153] The technical solutions provided in this application, through the collaborative mechanism of the upper and lower layer intelligent agent architecture of LM-based cross-vendor domain management agent (OSS Agent) and single-domain management agent (OMC Agent), complete the closed-loop self-intelligent processing of two scenarios: business fault monitoring and processing and equipment fault monitoring and processing, and have at least one of the following effects: 1) Through the hierarchical dual-agent intelligent agent technology (large model, memory, planning, execution and other components), the task cycle of a specified fault monitoring and processing closed loop instance can be effectively managed, including: controlling the creation, execution, suspension, listening, waiting and waking of the closed loop task instance, and initiating multiple rounds of conversations according to the status of each scenario, seamlessly connecting the system automatic processing scenario and the manual processing scenario that appears in the closed loop instance, realizing the pipelined and coherent implementation of the closed loop process SOP, and RPA automation.

[0154] 2) The dual-agent intelligent agent technology simultaneously controls and realizes a dual closed loop for business fault monitoring and processing and equipment fault monitoring and processing, with the dual closed loops intertwined.

[0155] 3) By designing and guiding the large model prompt, the dual closed-loop SOP process can be transformed into a large model thinking chain of hierarchical dual agent intelligent agents, realizing automated and intelligent closed-loop control of fault monitoring and processing.

[0156] Figure 17 A structural block diagram of an electronic device 1700 illustrating an exemplary embodiment of this application is shown. This electronic device can be implemented as a cross-domain control agent or a single-domain control agent as described above in this application.

[0157] The electronic device 1700 includes a central processing unit (CPU) 1701, a system memory 1704 including random access memory (RAM) 1702 and read-only memory (ROM) 1703, and a system bus 1705 connecting the system memory 1704 and the CPU 1701. The electronic device 1700 also includes a mass storage device 1706 for storing an operating system 1709, a client 1717, and other program modules 1711.

[0158] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 1704 and mass storage device 1706 described above can be collectively referred to as memory.

[0159] According to various embodiments of this disclosure, the electronic device 1700 can also be connected to a remote computer on a network, such as the Internet. That is, the electronic device 1700 can be connected to a network 1708 via a network interface unit 1707 connected to the system bus 1705, or it can use the network interface unit 1707 to connect to other types of networks or remote computer systems (not shown).

[0160] The memory further includes at least one instruction, at least one program, code set, or instruction set, which are stored in the memory. The central processing unit 901 executes the at least one instruction, at least one program, code set, or instruction set to implement all or part of the steps in the fault handling methods shown in the above embodiments.

[0161] In one exemplary embodiment, a readable storage medium is also provided, which stores a program or instructions that a stored program or instruction processor executes all or part of the steps in the above-described fault handling method. For example, the readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, or optical data storage device, etc.

[0162] In one exemplary embodiment, a computer program product is also provided, the computer program product including a computer program stored on a non-transitory readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform all or part of the steps in the above-described fault handling method.

[0163] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0164] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A fault handling method, characterized in that, include: The cross-domain management and control intelligent agent detects end-to-end cross-domain service failures through monitoring. The management and control scope of the cross-domain management and control intelligent agent includes multiple management and control domains, and each management and control domain corresponds to a single-domain management and control intelligent agent. The cross-domain management agent interacts with multiple single-domain management agents within its management scope to determine the equipment failure that caused the end-to-end cross-domain service failure. The cross-domain management and control agent collaborates with the multiple single-domain management and control agents to perform closed-loop processing of faults in the end-to-end cross-domain services and the equipment faults.

2. The method according to claim 1, characterized in that, The cross-domain management agent collaborates with the multiple single-domain management agents to perform closed-loop processing of faults in the end-to-end cross-domain service, including: The cross-domain management agent obtains first fault information reported by at least one first single-domain management agent, wherein the first device fault information is used to indicate the impact range of device faults in the first management domain corresponding to the first single-domain management agent on the end-to-end cross-domain service carried, and the at least one first single-domain management agent is one or more single-domain management agents among the plurality of single-domain management agents. The cross-domain management and control intelligent agent determines the rerouting recovery path of the end-to-end cross-domain service based on at least one of the first device fault information obtained. The cross-domain management agent sends a first instruction to at least one second single-domain management agent, wherein the first instruction is used to instruct the second single-domain management agent to create a cross-segment recovery path of the rerouting recovery path within the second management domain corresponding to the second single-domain management agent, and the at least one second single-domain management agent is one or more single-domain management agents among the plurality of single-domain management agents.

3. The method according to claim 2, characterized in that, The cross-domain management and control agent collaborates with the multiple single-domain management and control agents to perform closed-loop processing of the device faults, including: The cross-domain management agent obtains second device fault information from the at least one first single-domain management agent, wherein the second device fault information includes at least one of the root cause of device fault and device fault repair suggestion within the first management domain; Based on at least one of the second device fault information, obtain a repair plan for the device fault; Perform the repair of the equipment malfunction according to the repair plan and verify the repair.

4. The method according to claim 3, characterized in that, The second device fault information obtained by the cross-domain management agent from the at least one first single-domain management agent includes: The cross-domain management agent sends a first request to the at least one first single-domain management agent, the first request being used to request the first single-domain management agent to identify the root cause of the equipment failure and / or equipment failure repair suggestions that caused the end-to-end cross-domain service to fail. Receive the second device fault information reported by the at least one first single-domain management agent.

5. The method according to claim 3, characterized in that, The second device fault information obtained by the cross-domain management agent from the at least one first single-domain management agent includes: The cross-domain management agent receives the second device fault information proactively reported by the at least one first single-domain management agent after monitoring and sensing a device fault in the first management domain.

6. The method according to claim 3, characterized in that, The cross-domain management and control agent collaborates with the multiple single-domain management and control agents to perform closed-loop processing of faults in the end-to-end cross-domain service, and also includes: Based on the repair verification results, and according to the service level agreement spatiotemporal sorting, the response plan for the end-to-end cross-domain service is obtained; Based on the aforementioned response scheme, a second instruction is sent to at least one third single-domain management agent. The second instruction carries the response order of the multiple end-to-end cross-domain services and the response time point of each end-to-end cross-domain service, instructing the third single-domain management agent to perform cross-segment responses of the end-to-end cross-domain services within the third management domain corresponding to the third single-domain management agent. The at least one third single-domain management agent is one or more of the multiple single-domain management agents.

7. The method according to claim 6, characterized in that, The cross-domain management and control agent collaborates with the multiple single-domain management and control agents to perform closed-loop processing of faults in the end-to-end cross-domain service, and also includes: The cross-domain management agent receives a response message sent by the at least one third single-domain management agent, wherein the response message carries the cross-segment response result of the third single-domain management agent in executing the end-to-end cross-domain service within the third management domain.

8. The method according to claim 7, characterized in that, The cross-domain management and control agent collaborates with the multiple single-domain management and control agents to perform closed-loop processing of faults in the end-to-end cross-domain service, and also includes: The cross-domain management and control intelligent agent performs tests and verifications on the end-to-end cross-domain service after receiving the response. The test and verification parameters include at least one of the following: latency, packet loss, optical power, and bit error rate.

9. A fault handling method, characterized in that, include: A single-domain management and control agent can detect device failures within its managed domain through monitoring. The single-domain management agent and the cross-domain management agent work together to perform closed-loop processing of the device failure and the end-to-end cross-domain service failure caused by the device failure.

10. The method according to claim 9, characterized in that, The single-domain management agent and the cross-domain management agent collaborate to perform closed-loop processing of end-to-end cross-domain service failures caused by the device failure, including: The single-domain management and control agent obtains the impact range of the device failure on the end-to-end cross-domain services it carries. The first fault information reported to the cross-domain management and control agent, wherein the first device fault information is used to indicate the scope of the impact of the device fault on the end-to-end cross-domain service; The system receives a first instruction sent by the cross-domain management and control agent, wherein the first instruction is used to instruct the creation of a cross-segment recovery path for the rerouting recovery path of the end-to-end cross-domain service within the management and control domain; According to the first instruction, a rerouting recovery path for the end-to-end cross-domain service is created, which is a cross-segment recovery path within the control domain.

11. The method according to claim 10, characterized in that, The single-domain management agent and the cross-domain management agent collaborate to perform closed-loop processing of the device faults, including: The system receives a first request from the cross-domain management and control agent, wherein the first request is used to request identification of the root cause of the equipment failure that caused the end-to-end cross-domain service failure and / or equipment failure repair suggestions. Identify the root causes of equipment failures that cause the end-to-end cross-domain service to fail and / or provide equipment failure repair recommendations; The second device fault information is reported to the cross-domain management and control agent, wherein the second device fault information includes at least one of the root cause of the device fault and the device fault repair suggestion within the first management and control domain.

12. The method according to claim 10, characterized in that, The single-domain management agent and the cross-domain management agent collaborate to perform closed-loop processing of the device faults, including: After a device failure is detected in this control domain through monitoring, the root cause of the device failure and / or device failure repair recommendations that caused the end-to-end cross-domain service failure are identified. The system proactively reports second device fault information to the cross-domain management and control agent, wherein the second device fault information includes at least one of the root cause of device fault and device fault repair suggestions within the first management and control domain.

13. The method according to claim 11 or 12, characterized in that, The single-domain management agent and the cross-domain management agent collaborate to perform closed-loop processing of end-to-end cross-domain service failures caused by the device failure, and also include: The single-domain management agent receives a second instruction sent by the cross-domain management agent. The second instruction carries the response order of multiple end-to-end cross-domain services and the response time point of each end-to-end cross-domain service. According to the response order of each end-to-end cross-domain service and the response time of each end-to-end cross-domain service, the cross-segment response of the end-to-end cross-domain service within the control domain is executed.

14. The method according to claim 13, characterized in that, The single-domain management agent and the cross-domain management agent collaborate to perform closed-loop processing of end-to-end cross-domain service failures caused by the device failure, and also include: The single-domain management agent sends a response message to the cross-domain management agent, wherein the response message carries the cross-segment response result of executing the end-to-end cross-domain service within the management domain.

15. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing programs or instructions that can run on the processor, the programs or instructions being executed by the processor to implement the steps of the fault handling method as described in any one of claims 1 to 14.

16. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the fault handling method as described in any one of claims 1 to 14.

17. A computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the steps of the fault handling method as described in any one of claims 1 to 14.

Images