Alert management

Abstract

According to an example aspect of the present invention, there is provided an apparatus comprising at least one processing core and at least one memory storing instructions that, when executed by the at least one processing core, cause the apparatus at least to store a large language model, LLM, trained to handle alerts originating in a cellular communication network, process an alert from the cellular communication network using the LLM together with at least one information source distinct from the LLM, using retrieval-augmented generation to generate a response to the alert, and reconfigure the cellular communication network with the response.

Description

ALERT MANAGEMENTFIELD

[0001] The present disclosure relates to managing system alerts using large language models, for example in the context of cellular communication networks.BACKGROUND

[0002] Cellular communication networks comprise nodes in the radio access network, RAN, and in the core network, CN, which are tasked with performing various roles in the overall network. Operating conditions of these nodes change over time in dependence of communication load, component faults, software errors and interoperability problems.

[0003] Such network nodes are typically configured to issue alerts when their operating condition changes, the alerts being sent to a queue or other handling mechanism of the cellular communication network, such that network operators can take appropriate action as a response to the change in operating condition in the node. Also systems other than cellular communication networks generate alerts, for example fixed communication networks or industrial installations.SUMMARY

[0004] According to some aspects, there is provided the subject-matter of the independent claims. Some embodiments are defined in the dependent claims.

[0005] According to a first aspect of the present disclosure, there is provided an apparatus comprising at least one processing core and at least one memory storing instructions that, when executed by the at least one processing core, cause the apparatus at least to store a large language model, LLM, trained to handle alerts originating in a cellular communication network, process an alert from the cellular communication network using the LLM together with at least one information source distinct from the LLM, usingretrieval-augmented generation to generate a response to the alert, and re-configure the cellular communication network with the response.

[0006] According to a second aspect of the present disclosure, there is provided a method comprising storing a large language model, LLM, trained to handle alerts originating in a cellular communication network, processing an alert from the cellular communication network using the LLM together with at least one information source distinct from the LLM, using retrieval-augmented generation to generate a response to the alert, and re-configuring the cellular communication network with the response.

[0007] According to a third aspect of the present disclosure, there is provided a non- transitory computer readable medium having stored thereon a set of computer readable instructions that, when executed by at least one processor, cause an apparatus to at least store a large language model, LLM, trained to handle alerts originating in a cellular communication network, process an alert from the cellular communication network using the LLM together with at least one information source distinct from the LLM, using retrieval-augmented generation to generate a response to the alert, and re-configure the cellular communication network with the response.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] FIGURE 1 illustrates an example system in accordance with at least some embodiments of the present invention;

[0009] FIGURE 2 illustrates an example solution in accordance with at least some embodiments of the present invention;

[0010] FIGURE 3 illustrates an example apparatus capable of supporting at least some embodiments of the present invention;

[0011] FIGURE 4 illustrates a solution in accordance with at least some embodiments of the present invention, and

[0012] FIGURE 5 is a flow graph of a method in accordance with at least some embodiments of the present invention.EMBODIMENTS

[0013] Described herein are mechanisms to handle alerts in a system, such as a cellular communication network, using a large language model, LLM. This LLM is trained to handle alerts which originate in the system by processing them to generate a response to the alert. The response may be used to handle the alert by re-configuring one or more aspects of the system. The response may take the form of a configuration intent expressed in natural language to be parsed into technical configuration information, or the response may take the form of configuration information usable as such in re-configuring the system, such as the cellular communication network. The response may comprise a software update, for example source code to be used in a software update. The LLM may be configured to obtain contextual information on a current state of the system and to generate the response based at least in part also on this contextual information. Thus technical benefits are obtained in that the time required in generating the response is reduced compared to a situation where human operators generate the response after analysing the alert. The response may be implemented in the system automatically, or only after approval from human operators, depending on the embodiment. Actions taken in response to alerts based on the generated response may include, for example, resetting an affected node to a default state, updating software in the node, or re-configuring operating parameters, such as frequency used, in the node.

[0014] FIGURE 1 illustrates an example system in accordance with at least some embodiments. The example shown in FIGURE 1 is a cellular system, but the herein disclosed methods are not limited to being applied in a cellular context but may be applied alternatively in e.g. a fixed communication network or in an industrial process installation, such as a nuclear power station. FIGURE 1 illustrates a radio access network, RAN, 102, comprising plural base stations, which are configured to operate in accordance with a cellular communication standard, such as long term evolution, LTE, or fifth generation, 5G, also known as New Radio, NR, both as specified by the 3rdgeneration partnership project, 3GPP. Where a non-cellular system is used, access nodes, such as access points, corresponding to base stations of RAN 102 may be configured in accordance with a non-cellular communication standard such as wireless local area network, WLAN, or worldwide interoperability for microwave access, WiMAX, for example. In some embodiments, RAN 102 is absent, in these cases the network, NW, may be a wire-line network based on Ethernet or Diameter, for example.

[0015] Base stations of RAN 102 are coupled with core network nodes of core network 103 via links, which may comprise wire-line connections, for example. A few such links are illustrated in FIGURE 1. Core network nodes 110, 120, 130, 140 and 150 may comprise mobility management entities, MME, serving gateways, S-GW, access and mobility management functions, AMF, subscriber information registers, policy enforcement entities, switching nodes, alert management functions and / or network exposure functions, for example. The core network may comprise a gateway 160, enabling communication with further networks, via at least one inter-network link. Some aggregate networks may have more than one core network, which are then connected to each other using at least one communication link.

[0016] Further, in the illustrated example situation, base stations of RAN 102 are in wireless radio communication with user equipments, UEs 101. Each UE may comprise, for example, a smartphone, feature phone, tablet or laptop computer, Intemet-of-Things, loT, node, smart wearable or a connected car connectivity module, for example. Naturally, separate UEs need not be of a same type. The UEs are configured to operate using a same cellular communication standard, or standards, as the base station(s), to obtain interoperability.

[0017] Core network nodes of core network 103 may be standalone physical nodes, running on a dedicated computing substrate, or they, or at least some of them, may be virtualized network nodes, such that more than one virtualized network node may run on a same physical computing substrate. Virtualized network nodes may be migrated from one physical computing substrate to another, for example to perform load balancing between the computing substrates, or to enable software or hardware updating of the computing substrates themselves. Another reason for migrating virtualized network nodes, or their traffic, is to enable repairs in physical hardware used to run the respective node. Core network 103 may comprise both standalone physical nodes and virtualized network nodes. In addition to the physical computing substrates, also virtualized and standalone physical nodes may be updated or otherwise re-configured. Re-configuring may comprise, for example, changing at least one operating parameter or updating to a different software version. Examples of operating parameters include traffic filters, frequencies, frequency reuse patterns, prioritization rules, routing tables, routing policies and bandwidth caps affecting coverage areas, individual subscribers or subscriber classes.

[0018] Nodes of the system, such as RAN 102 nodes and core network 103 nodes, may be configured to automatically, without user intervention, generate alerts, which comprise a timestamp which indicates, in a time of the system or an external time reference, when the alert was generated in response to a change in an operating conditions of the node. The alerts further comprise indications of an event the record relates to. The event may be a benign or an error event, wherein examples of benign events include new registrations of UEs into the system, successful handovers, measured interference levels and network utilization rates. Examples of error events include a dropped call, a failed UE authorization, a timed-out protocol connection, a failed network procedure, maximum capacity reached and a timed-out random access process. An example of a failed network procedure is a reconfiguration process between core network 103 and a base station in RAN 102, which begins but which does not successfully complete, resulting in failure when a timer of the reconfiguration process expires. In some embodiments all alerts are of the error type and benign events are not reported in the system using alerts.

[0019] Alerts generated by nodes of the network, such as RAN 102 nodes or CN 103 nodes, comprise indications in the form of technical characteristics relating to the node, for example, they may comprise indications of a node model or manufacturer, identifiers of one or more nodes participating in a process to which the alert relates. Alerts may comprise error codes which assist in debugging the cause of the alert. A single failure in the network may cause a number of alerts to be generated as a direct and / or indirect consequence. For example in case a base station fails completely due to e.g. a lightning strike, all protocol connections traversing the base station will be disconnected, causing a number of alerts to be generated from nodes communicating with the failed base station. Further, attempts to re-configure the failed base station from the core network will also fail, causing yet further alerts relating to the failure. In such a case, the alerts generated as a result may all share, among other indications, an identifier of the failed base station as one of the network elements involved in the situation underlying the alert. This identifier may be used to collect these alerts together, such that the identifier acts as a defining characteristic in the set of alerts relating to the failure of the base station. These alerts may then be set in a time order in a time series based on the timestamps of these alerts, for example. The moment the base station failed will be observable in this time series as a time after which the number of alerts is greatly increased compared to the time period before this moment.

[0020] Alerts comprise system alerts and service alerts. These can be created by systems and / or software that monitor the health and the state of the network. In case a malfunction happens, and the system becomes unhealthy, alerts are generated which are sent to data pipelines, alert queues or ticketing systems, for example. Thus the alerts in a queue comprise information on the technical state of the system.

[0021] One of the core network nodes 110, 120, 130, 140, 150 may be an apparatus configured to process alerts originating in the cellular communication network. Alerts may be raised from servers, system platforms, customer-facing services, core network nodes or base stations, for example. Alerts may be conveyed to this node, which uses an LLM which has been trained to handle alerts from the cellular communication network (or other system). Modem LLM solutions can be trained to understand dynamic contextual data and provide solutions, such as code and configuration updates, debugging assistance, and analysis of text. For example, the LLM may be trained using a set of alerts recorded historically, together with a set of human-operator generated responses to the alerts. Further, the training data may include documentation of the network, tooling documentation and source code of software running in the network. Thus the LLM may be trained to generate a meaningful response to a new alert which approximates a response that human operators would create. An advantage of the LLM is that it may generate the response much faster than the humans would, as humans would need to think about the problem underlying the alert, and possible communicate among themselves. The LLM may be trained using historical alerts from the specific network in question, or from a similar network. In some cases, the historical alerts used in the training comprise both alerts from the specific network and from one or more similar networks. The LLM may be re-trained with new data to keep it up-to-date with changes in the cellular communication network. The re-training may be performed, for example, at a fixed time interval such as weekly or monthly. Generative pre-trained transformer, GPT, is an example of an LLM type. Human operators may include, for example, system operators, service operators, DevOps engineers, on-call engineers, base station operators, and / or base station maintenance personnel. An LLM as used herein may have billions, tens of billions or hundreds of billions of parameters.

[0022] A technical advantage and effect of a faster generation of the response is that the network can respond to alerts faster, which increases an effective utilization rate of the network as parts of the network remain in an outage or error state for shorter periods of time.

[0023] Once the response is obtained from the LLM, it is usable in re-configuring the network to respond to the alert, such as, for example, remedying an underlying cause of the alert, or at least reducing the effects of the underlying cause. The re-configuring may be performed without human intervention, that is, automatically, or alternatively the reconfiguring is performed only after human operators have accepted the response generated by the LLM. In case the response is a configuration intent, it may be provided to a parser which converts the configuration intent into actionable, technical configuration information, such as parameters, usable in modifying operating parameter(s) of the system. An intent as such may be information which does not comprise explicit configuration parameters, rather it may comprise what is sought to be accomplished by re-configuration, such as making a specific cell smaller. The parser may then determine which configuration information is needed to accomplish the aim expressed in the intent, and the determined configuration information may be sent to nodes to accomplish the re-configuration.

[0024] The LLM may be configured to not rely on only the training with historical alerts in the generation of the response based on the received alert. In detail, the LLM may be configured to use at least one information source distinct from the LLM in the generation of the response to the alert. Such information sources distinct from the LLM may include one or more of: a current state of the cellular communication network, an alert management guideline, source code of software running in the cellular communication network, and a node of the cellular communication network, such as, for example, the node which originated the alert which is being processed. Source code may be available in case it is openly available, such as open-source code, in case the system is controlled by a same entity as operates the LLM, or of the entity controlling the LLM has produced the software running in the system. In particular, an information source distinct from the LLM that the LLM can use when generating a response to the alert is a queue of as of yet unprocessed alerts that have been generated from the network. These alerts often comprise useful information of the state of the network. The alert management guideline is useful in that it may be updated, which will affect the way the LLM processes alerts but this does not require re-training the LLM. This is a quick way to fine-tune the operation of the LLM.

[0025] The current state of the cellular communication network may comprise the information on alerts in queue waiting to be handled, information on nodes which are in fault conditions, information on load statuses of parts of the network, information on ongoing power outages and / or information on ongoing denial-of-service attacks. Accessing sourcecode or configuration of software used in the network facilitates generation, by the LLM, of responses to alerts such that the responses include software updates. The software update may be generated by the LLM generating new source code, which may be compiled and sent to the network as a software update. This may take place without user intervention.

[0026] When querying a network node for more information, the LLM may be empowered to obtain more detail on the alert, providing the benefit that the response is more accurate in addressing the underlying cause of the alert. The node queried may be the node which originated the alert, as mentioned above, or a correspondent node of the node which originated the alert. The correspondent node has context information of protocol connections with the node which originated the alert, such context information being useful in analysing the alert as it contains more data than is in the alert itself. When the node which originated the alert is queried, it may provide one or more internal logs, which likely comprise highly relevant information pertaining to the underlying cause of the alert.

[0027] One tool the LLM may be configured to use in interfacing with the information source distinct from the LLM is retrieval augmented generation, RAG. This involves augmenting the LLM with document retrieval, for example using a vector database. Given a query, RAG involves calling a document retriever to retrieve the most relevant document, usually measured by first encoding the query and the documents into vectors, then finding the documents with vectors closest in Euclidean space to the query vector. The LLM then generates a response based on the alert, the training data used in training the LLM and the retrieved document or documents. RAG may be used by the LLM to access more than one information source distinct from the LLM in the generation of a single response to a single alert. For example, the LLM may use RAG to access the node which originated the alert (or a correspondent node thereof), a current state of the network and an alert management guideline when generating a single response.

[0028] As noted above, the response may comprise a software update to a node, or nodes, of the system. Additionally or alternatively, the response may comprise a configuration change of at least one operating parameter of the system, and / or a new software module to be installed in a node of the system. Examples of operating parameters include traffic filters, routing tables, prioritization rules, operating frequencies, and various timing parameters. The response may comprise one or more change to a service other than the one which generated the alert, in case this other service is affected by the cause of thealert, or by the response. The response may comprise a system-wide parameter change, such as a modification of a timeout value used in a specific protocol connection type, such as transmission control protocol, TCP, or real-time transport protocol, RTP.

[0029] In a cellular communication network, the node running the LLM may be configured to provide the response to a simulator to check if the response is valid in the cellular communication network, and to only perform the re-configuring of the cellular communication network with the response if the simulator indicates the response is valid. If The simulator may be run in the core network 103, or in an external system, for example. Using the simulator provides the benefit, that erroneous responses the LLM may occasionally produce are not sent to the cellular communication network for re-configuring that system, which could cause yet further problems and alerts. In other words, the quality of the alert management system is increased by the use of the simulator. The providing of the response to the simulator, the running of the simulator, and the re-configuring when the response is validated as valid in the simulator may be performed without human intervention, automating the alert management system and resulting in substantial savings in time and system availability. In case the simulator indicates the response is not valid, the apparatus, that is the node, running the LLM may be configured to reject the response and inform human operators.

[0030] Performing the re-configuring without human intervention automates, for instance, correction of common configuration mistakes, server and container systems resource allocations, and bugs in code that may interrupt live services. The LLM may be configured to identify alerts which are originated based on complex problems, and the LLM may be configured to forward such alerts to human operators for analysis rather than perform the automatic re-configuration of the system described herein.

[0031] Alternatively to informing human operators and rejecting the response, the node running the LLM may be configured to provide, responsive to the simulator indicating the response is not valid, the response to the LLM together with the alert and an error message from the simulator, to obtain a second response, and to use the second response to perform the re-configuring of the cellular communication network as a response to an indication of validity returned by the simulator as a response the second response being provided to it as input. This iterative process may be repeated a few times to seek to obtain a valid response. In case a preconfigured maximum number of iterations is reached withoutthe simulator indicating a response provided to it is valid, the iterating may be stopped and the alert be referred to human operators. While this would incur delays, the system would still overall provide dramatic savings in time if a significant proportion of incoming alerts are nonetheless successfully handled by the LLM without human intervention. Handling of alerts by human operators may require debugging multiple services, source code, and systems to find the root cause for a problem. If an issue is known, a playbook may be searched for any possible known solutions. In case of an unknown issue, manual debugging, configuration and programming for resolving the issue may be required. These investigations take more time depending on how familiar or unfamiliar the environment is for the human operator.

[0032] The simulator may implement end-to-end, E2E, testing for running a production-like environment, including but not limited to components, hardware and systems and executing the desired technical and business functionalities to validate that all desired functions of a system work as intended. The simulator may be a mix of software simulating elements of the system, and actual hardware system components interfaced with the simulator software.

[0033] In case the response fails in the cellular communication network, the apparatus running the LLM, or another node, may be configured to roll the network back to a state preceding the re-configuration with the response from the LLM. In such a case, the issue may be referred to human operators.

[0034] Overall, therefore, benefits are obtained in that time spent in manual investigation and debugging is reduced. Further, an aggregated information and working solution is enriched to the LLM model, further training the model may be conducted for the future in case similar issues arise. As the LLM is not static and does not rely exclusively on pre-trained information, service connectivity and data retrieval capabilities can provide more variety of support and enrichment of alert information with up-to-date data from live systems and services. In particular, alerts stemming from already known issues types can be solved faster when a working solution is generated and validated automatically without human intervention, reducing the time spent on investigation work. For visibility and history trail of automation activities, human operators can monitor and follow the activity of the automation to see what issues were resolved and how they were resolved.

[0035] FIGURE 2 illustrates an example solution in accordance with at least some embodiments of the present invention. This figure relates in particular to embodiments where the system is a cellular communications network. The LLM 210 is illustrated at the top. The LLM may access operating manuals 220 of the cellular communication network, source codes 230 of software running in nodes of the network, 235 runtime configurations active in the network, guidelines and playbooks 240 of the network, metadata 250 of systems and platforms comprised in the cellular communication network, service integration connectivity information 260 of the network. The service integration connectivity information 260 may further comprise logs 270 of services and connections, and back-end application programming interfaces, APIs, metadata 280.

[0036] Manuals 220, source codes 230, guidelines 240 and metadata 250 may be included in training data used to train LLM 210 to prepare the responses responsive to the alerts.

[0037] EIGURE 3 illustrates an example apparatus capable of supporting at least some embodiments of the present invention. Illustrated is device 300, which may comprise, for example, an apparatus running a core network node, the apparatus being configured to run the LLM, or another apparatus, such as a server connected with the core network, the server being configured to run the LLM. Comprised in device 300 is processor 310, which may comprise, for example, a single- or multi-core processor wherein a single-core processor comprises one processing core and a multi-core processor comprises more than one processing core. Processor 310 may comprise, in general, a control device. Processor 310 may comprise more than one processor. When processor 310 comprises more than one processor, device 300 may be a distributed device wherein processing of tasks takes place in more than one physical unit. Processor 310 may be a control device. A processing core may comprise, for example, a Cortex- A8 processing core manufactured by ARM Holdings or a Zen processing core designed by Advanced Micro Devices Corporation. A processing core or processor may be, or may comprise, at least one qubit. Processor 310 may comprise at least one AMD Opteron and / or Intel Core processor. Processor 310 may comprise at least one application-specific integrated circuit, ASIC. Processor 310 may comprise at least one field-programmable gate array, LPGA. Processor 310, optionally together with memory and computer instructions, may be means for performing method steps in device 300, such as storing, processing and re-configuring, for example. Processor 310 may be configured, at least in part by computer instructions, to perform actions.

[0038] Device 300 may comprise memory 320. Memory 320 may comprise randomaccess memory and / or permanent memory. Memory 320 may comprise at least one RAM chip. Memory 320 may be a computer readable medium. Memory 320 may comprise solid- state, magnetic, optical and / or holographic memory, for example. Memory 320 may be at least in part accessible to processor 310. Memory 320 may be at least in part comprised in processor 310. Memory 320 may be means for storing information. Memory 320 may comprise computer instructions that processor 310 is configured to execute. When computer instructions configured to cause processor 310 to perform certain actions are stored in memory 320, and device 300 overall is configured to run under the direction of processor 310 using computer instructions from memory 320, processor 310 and / or its at least one processing core may be considered to be configured to perform said certain actions. Memory 320 may be at least in part external to device 300 but accessible to device 300. Memory 320 may be transitory or non-transitory. The term “non-transitory”, as used herein, is a limitation of the medium itself (that is, tangible, not a signal) as opposed to a limitation on data storage persistency (for example, RAM vs. ROM).

[0039] Device 300 may comprise a transmitter 330. Device 300 may comprise a receiver 340. Transmitter 330 and receiver 340 may be configured to transmit and receive, respectively, information in accordance with at least one cellular or non-cellular standard. Transmitter 330 may comprise more than one transmitter. Receiver 340 may comprise more than one receiver. Transmitter 330 and / or receiver 340 may be configured to operate in accordance with a suitable communication arrangement, such as Ethernet or Diameter, for example.

[0040] Device 300 may comprise user interface, UI, 360. UI 360 may comprise at least one of a display, a keyboard, a touchscreen, a vibrator arranged to signal to a user by causing device 300 to vibrate, a speaker or a microphone. A user may be able to operate device 300 via UI 360, for example to configure LLM or alert management parameters, to manage digital files stored in memory 320 or on a cloud accessible via transmitter 330 and receiver 340.

[0041] Processor 310 may be furnished with a transmitter arranged to output information from processor 310, via electrical leads internal to device 300, to other devices comprised in device 300. Such a transmitter may comprise a serial bus transmitter arranged to, for example, output information via at least one electrical lead to memory 320 for storagetherein. Alternatively to a serial bus, the transmitter may comprise a parallel bus transmitter. Likewise processor 310 may comprise a receiver arranged to receive information in processor 310, via electrical leads internal to device 300, from other devices comprised in device 300. Such a receiver may comprise a serial bus receiver arranged to, for example, receive information via at least one electrical lead from receiver 340 for processing in processor 310. Alternatively to a serial bus, the receiver may comprise a parallel bus receiver. Device 300 may comprise further devices not illustrated in FIGURE 3.

[0042] Processor 310, memory 320, transmitter 330, receiver 340, and / or UI 360 may be interconnected by electrical leads internal to device 300 in a multitude of different ways. For example, each of the aforementioned devices may be separately connected to a master bus internal to device 300, to allow for the devices to exchange information. However, as the skilled person will appreciate, this is only one example and depending on the embodiment various ways of interconnecting at least two of the aforementioned devices may be selected without departing from the scope of the present invention.

[0043] FIGURE 4 illustrates a solution in accordance with at least some embodiments of the present invention. In the solution of FIGURE 4, alerts concerning a cellular communication network are received from nodes originating the alerts in monitoring service 420, which monitors nodes and services of the network 450. The alerts are provided from monitoring service 420 to alert queue 430, from which they are fed to FEM 410 either directly, or via ticketing system 440. The connection between queue 430 and FEM 410 is thus optional in nature, as indicated also by the dashed-line nature of the arrow between these two elements 430, 410. Human operators 4100 may observe alerts in the ticketing system 440 and, optionally, act on them, for example when FEM 410 informs human operators 4100 that their attention is needed, as described herein above.

[0044] FEM 410 receives the alert as input and, as discussed herein above, processes it to generate a response. The response may be provided to simulator 460, which returns a verification result indicating whether the response generated by the FEM as a response to the alert is valid. Simulator 460 may be an end-to-end testing simulator, for example, as described herein above. Responsive to the response being validated in simulator 460 as valid, the system may re-configure nodes and / or services of the network 450 with the response to react to the alert. It is possible, that as a result of this action some alerts in queue 430 orticketing system 440 are resolved as well, and they may be purged from queue 430 by human operators 4100, for example.

[0045] When generating the response to the alert, LLM 410 may rely, as described herein above, on information sources external to the LLM, for example by using RAG. Examples of such information sources are in FIGURE 4 logging service 470, back-end connectivity 480 and documentation 490. Documentation 490 may comprise an alert management guideline, for example. As discussed above, LLM 410 may use further information sources as well, such as source codes, nodes of the network 450 and the queue 430 of alerts that haven’t yet been handled. Plural information sources external to the LLM 410 may be used in the generation of a single response to a single alert. For example, queue 430 and alert management guideline 490 may be used.

[0046] One example of the functioning of a system as illustrated in FIGURE 4 is given in the following. The LLM monitors for alerts in the monitored target systems, such as the alert queue or the ticketing system. Once information about an alert is received, the LLM analyses the alert and begins inspecting the alert. The LLM will then connect to the services in question which have originated the alert for service healthiness and inspects their responses to generated queries. The LLM may then connect to LLM-extemal information sources in the form of auxiliary backend services that have knowledge and / or are dependent on the malfunctioning system, requesting more information on the context to fix the issue underlying the alert. The LLM may inspect known codebase using software source code to which the LLM has access, and inspect known documentation on the system. The LLM will inspect known guidelines for known solutions to fix the issue, and inspect a current state and / or configuration status of the system. The LLM may inspect logs and metrics related to the issue, and being attempting problem resolution. A response is generated comprising configuration, commands, and / or code to fix the issue.

[0047] The LLM will then submit, in this example, the response and new test cases to version control and / or the simulator for validation. The simulator verifies with end-to-end or system tests that the response will be valid in the network 450, comprising testing that: the solution is syntactically correct, the solution does not interrupt or cause unintended side effects, and existing tests work. In case a positive indication of validity is obtained, the LLM will submits the response to be deployed to thereby re-configure the network. The reconfiguration may be applied directly or via a version control system.

[0048] FIGURE 5 is a flow graph of a method in accordance with at least some embodiments of the present invention. The phases of the illustrated method may be performed in a core network node or server node configured to run the LLM, for example, or in a control device configured to control the functioning thereof, when installed therein.

[0049] Phase 510 comprises storing a large language model, LLM, trained to handle alerts originating in a cellular communication network. Phase 520 comprises processing an alert from the cellular communication network using the LLM together with at least one information source distinct from the LLM, using retrieval-augmented generation to generate a response to the alert. Finally, phase 530 comprises re-configuring the cellular communication network with the response. The re-configuring of the cellular communication network may be partial, comprising perhaps only an adjustment of one, two or three operating parameters of the cellular communication network, for example.

[0050] It is to be understood that the embodiments of the invention disclosed are not limited to the particular structures, process steps, or materials disclosed herein, but are extended to equivalents thereof as would be recognized by those ordinarily skilled in the relevant arts. It should also be understood that terminology employed herein is used for the purpose of describing particular embodiments only and is not intended to be limiting.

[0051] Reference throughout this specification to one embodiment or an embodiment means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases “in one embodiment” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Where reference is made to a numerical value using a term such as, for example, about or substantially, the exact numerical value is also disclosed.

[0052] As used herein, a plurality of items, structural elements, compositional elements, and / or materials may be presented in a common list for convenience. However, these lists should be construed as though each member of the list is individually identified as a separate and unique member. Thus, no individual member of such list should be construed as a de facto equivalent of any other member of the same list solely based on their presentation in a common group without indications to the contrary. In addition, various embodiments and example of the present invention may be referred to herein along with alternatives for the various components thereof. It is understood that such embodiments,examples, and alternatives are not to be construed as de facto equivalents of one another, but are to be considered as separate and autonomous representations of the present invention.

[0053] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the preceding description, numerous specific details are provided, such as examples of lengths, widths, shapes, etc., to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the invention.

[0054] While the forgoing examples are illustrative of the principles of the present invention in one or more particular applications, it will be apparent to those of ordinary skill in the art that numerous modifications in form, usage and details of implementation can be made without the exercise of inventive faculty, and without departing from the principles and concepts of the invention. Accordingly, it is not intended that the invention be limited, except as by the claims set forth below.

[0055] The verbs “to comprise” and “to include” are used in this document as open limitations that neither exclude nor require the existence of also un-recited features. The features recited in depending claims are mutually freely combinable unless otherwise explicitly stated. Furthermore, it is to be understood that the use of "a" or "an", that is, a singular form, throughout this document does not exclude a plurality.

[0056] As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or”, mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements.INDUSTRIAL APPLICABILITY

[0057] At least some embodiments of the present invention find industrial application in running networks, such as cellular communication networks.ACRONYMS LISTLLM large language modelRAG retrieval augmented generationRTP real-time transport protocolTCP transmission control protocol REFERENCE SIGNS LIST

Claims

CLAIMS:

1. An apparatus comprising at least one processing core and at least one memory storing instructions that, when executed by the at least one processing core, cause the apparatus at least to:- store a large language model, LLM, trained to handle alerts originating in a cellular communication network;- process an alert from the cellular communication network using the LLM together with at least one information source distinct from the LLM, using retrieval- augmented generation to generate a response to the alert, and- re-configure the cellular communication network with the response.

2. The apparatus according to claim 1, wherein the at least one information source distinct from the LLM comprises information representing a current state of the cellular communication network.

3. The apparatus according to claim 1 or 2, wherein the at least one information source distinct from the LLM comprises an alert management guideline.

4. The apparatus according to any of claims 1 - 3, wherein the at least one information source distinct from the LLM comprises source code of software running in the cellular communication network.

5. The apparatus according to any of claims 1 - 4, wherein the at least one information source distinct from the LLM comprises a node of the cellular communication network.

6. The apparatus according to claim 5, wherein the node of the cellular communication network is a node which originated the alert.

7. The apparatus according to any of claims 1 - 6, wherein the response comprises one or more of the following: a software update for at least a part of the cellular communication network, a configuration change of at least one operating parameter of the cellularcommunication network, and a new software module to be installed in the cellular communication network.

8. The apparatus according to any of claims 1 - 7, further configured to provide the response to a simulator to check if the response is valid in the cellular communication network, and to only perform the re-configuring of the cellular communication network with the response if the simulator indicates the response is valid.

9. The apparatus according to claim 8, further configured to provide, responsive to the simulator indicating the response is not valid, the response to the LLM together with an error message from the simulator, to obtain a second response, and to use the second response to perform the re-configuring of the cellular communication network.

10. The apparatus according to any of claims 1 - 9, configured to perform the re-configuring of the cellular communication network with the response without human intervention.

11. The apparatus according to claims 1 - 10, configured to perform the re-configuring of the cellular communication network with the response responsive to a human operator accepting the response.

12. A method comprising:- storing a large language model, LLM, trained to handle alerts originating in a cellular communication network;- processing an alert from the cellular communication network using the LLM together with at least one information source distinct from the LLM, using retrieval- augmented generation to generate a response to the alert, and- re-configuring the cellular communication network with the response.

13. The method according to claim 12, wherein the at least one information source distinct from the LLM comprises one or more of the following: information representing a current state of the cellular communication network, an alert management guideline, source code of software running in the cellular communication network, or a node of the cellular communication network.

14. The method according to any of claims 12 - 13, further comprising providing the response to a simulator to check if the response is valid in the cellular communication network, and only performing the re-configuring of the cellular communication network with the response if the simulator indicates the response is valid.

15. A non-transitory computer readable medium having stored thereon a set of computer readable instructions that, when executed by at least one processor, cause an apparatus to at least:- store a large language model, LLM, trained to handle alerts originating in a cellular communication network;- process an alert from the cellular communication network using the LLM together with at least one information source distinct from the LLM, using retrieval- augmented generation to generate a response to the alert, and- re-configure the cellular communication network with the response.

Images