Method, system, and computer-readable medium for autonomous network test case generation

JP2024511391A5Active Publication Date: 2024-11-06ORACLE INT CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023557165
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-03-19
Filing Date
2022-03-14
Publication Date
2024-11-06
Estimated Expiration
2042-03-14

AI Technical Summary

Technical Problem

Existing telecommunications networks face challenges in automatically generating test scenarios upon detection of faults, particularly in multi-vendor 5G environments, which are critical for maintaining network functionality and ensuring continuous integration and delivery processes.

Method used

An autonomous test case generator is employed to detect failure cases by analyzing network performance metrics, logs, and traces, using machine learning classifiers to generate and execute network functionality test cases, and integrate them into a reference library for automated testing frameworks.

Benefits of technology

This approach enables automated and efficient detection of network failures, reduces manual testing overhead, and ensures continuous network health by generating and executing test cases, thereby improving network reliability and reducing downtime.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

A method for autonomously generating network functional test cases includes detecting a fault case in a network function of a core network of a telecommunications network. The method includes autonomously generating a network functional test case based on the fault case in response to detecting the fault case. The network functional test case includes one or more network status parameters detected upon detection of the fault case. The method includes providing the network functional test case to a network testing system configured to execute the network functional test case by cycling through the one or more network status parameters and determining whether the network function repeats the fault case.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] Priority claim This application claims the benefit of priority to U.S. Patent Application Serial No. 17 / 207,393, filed March 19, 2021, the disclosure of which is incorporated herein by reference in its entirety.

[0002] The subject matter described herein relates to testing telecommunications networks. More particularly, the subject matter described herein relates to methods, systems, and computer-readable media for autonomous network test case generation. [Background technology]

[0003] background The 3rd Generation Partnership Project (3GPP) is an alliance between groups of Telecommunications Standards Associations. 3GPP has defined cellular system specifications for telecommunications networks, including 3G, 4G, and Long Term Evolution (LTE) networks.

[0004] The next generation network for 3GPP is the 5G network. The 5G specifications target higher data rates, reduced latency, energy savings, cost reduction, higher system capacity, and an increased number of connected devices.

[0005] A multi-vendor environment deploys various 5G network functions defined by 3GPP. In production deployments, faults / errors may be observed in the network functions under different traffic scenarios. Telecom vendors are adopting continuous integration / continuous delivery (CI / CD) processes, which introduce automated testing procedures before and after software delivery.

[0006] In view of these and other challenges, a need exists for a method, system, and computer-readable medium for autonomous network test case generation. Summary of the Invention [Means for solving the problem]

[0007] overview A method for autonomously generating network functional test cases includes detecting a fault case in a network function of a core network of a telecommunications network. The method includes autonomously generating a network functional test case based on the fault case in response to detecting the fault case. The network functional test case includes one or more network status parameters detected upon detection of the fault case. The method includes providing the network functional test case to a network testing system configured to execute the network functional test case by cycling through the one or more network status parameters and determining whether the network function repeats the fault case.

[0008] According to another aspect of the subject matter described herein, detecting the fault case includes analyzing one or more network performance metrics, or one or more network performance alerts, or both.

[0009] According to another aspect of the subject matter described herein, detecting the fault case includes analyzing one or more network function application logs, or one or more network function traces, or both.

[0010] According to another aspect of the subject matter described herein, executing the network functional test cases includes repeatedly executing the network functional test cases for a plurality of different network traffic conditions.

[0011] According to another aspect of the subject matter described in this specification, detecting the fault case includes providing the network function monitoring data to a machine learning classifier trained on training the monitoring data for the network function.

[0012] According to another aspect of the subject matter described in this specification, providing the network function test case to the network test system includes associating the network function test case with a network function and one or more other network function test cases for the network function in a reference library of test cases for a core network of the telecommunications network.

[0013] According to another aspect of the subject matter described in this specification, providing the network functional test case to the network test system includes determining that the network functional test case is not already present in a reference library of test cases for a core network of the telecommunications network.

[0014] According to another aspect of the subject matter described herein, a system for autonomously generating network functional test cases includes at least one processor and a memory. The system further includes an autonomous test case generator implemented by the at least one processor, the autonomous test case generator configured to detect a fault case in a network function of a core network of a telecommunications network and, in response to detecting the fault case, autonomously generate a network functional test case based on the fault case. The network functional test case includes one or more network status parameters detected upon detection of the fault case. The autonomous test case generator is configured to provide the network functional test case to a network testing system configured to execute the network functional test case by cycling through the one or more network status parameters and determining whether the network function cycles through the fault case.

[0015] According to another aspect of the subject matter described herein, detecting the fault case includes analyzing one or more network performance metrics, or one or more network performance alerts, or both.

[0016] According to another aspect of the subject matter described herein, detecting the fault case includes analyzing one or more network function application logs, or one or more network function traces, or both.

[0017] According to another aspect of the subject matter described herein, executing the network functional test cases includes repeatedly executing the network functional test cases for a plurality of different network traffic conditions.

[0018] According to another aspect of the subject matter described in this specification, detecting the fault case includes providing the network function monitoring data to a machine learning classifier trained on training the monitoring data for the network function.

[0019] According to another aspect of the subject matter described in this specification, providing the network function test case to the network test system includes associating the network function test case with a network function and one or more other network function test cases for the network function in a reference library of test cases for a core network of the telecommunications network.

[0020] According to another aspect of the subject matter described in this specification, providing the network functional test case to the network test system includes determining that the network functional test case is not already present in a reference library of test cases for a core network of the telecommunications network.

[0021] According to another aspect of the subject matter described herein, a non-transitory computer-readable medium is provided having executable instructions stored thereon that, when executed by a processor of a computer, control a computer to perform a plurality of steps, including detecting a fault case in a network function of a core network of a telecommunications network, and autonomously generating a network functional test case based on the fault case in response to detecting the fault case, the network functional test case including one or more network status parameters detected upon detection of the fault case, and providing the network functional test case to a network testing system configured to execute the network functional test case by cycling through the one or more network status parameters and determining whether the network function repeats the fault case.

[0022] The subject matter described herein may be implemented in software in combination with hardware and / or firmware. For example, the subject matter described herein may be implemented in software executed by a processor. In one exemplary implementation, the subject matter described herein may be implemented using a computer-readable medium having stored thereon computer-executable instructions that, when executed by a processor of a computer, control a computer to perform steps.

[0023] Exemplary computer-readable media suitable for implementing the subject matter described herein include non-transitory devices such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits. In addition, a computer-readable medium implementing the subject matter described herein may be located on a single device or computing platform, or may be distributed across multiple devices or computing platforms.

[0024] The subject matter described herein will now be described with reference to the accompanying drawings. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a block diagram illustrating an example 5G system network architecture. [Diagram 2] FIG. 1 is a block diagram of an exemplary network environment for an autonomous test case generator. [Diagram 3] FIG. 2 is a block diagram illustrating an exemplary structure of an autonomous test case generator. [Figure 4] FIG. 1 is a flow diagram of an exemplary method for autonomously generating network functional test cases. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] Detailed Description The subject matter described herein relates to methods, systems, and computer-readable media for preventing subscriber identifier leakage from telecommunications networks.

[0027] In a 5G telecommunications network, a network node that provides a service is called a producer network function (NF). A network node that consumes a service is called a consumer NF. A network function can be both a producer NF and a consumer NF depending on whether it consumes or provides a service. An NF instance is an instance of a producer NF that provides a service. A given producer NF may include two or more NF instances.

[0028] A multi-vendor environment deploys various 5G network functions defined by 3GPP. In production deployments, faults / errors may be observed in the network functions under different traffic scenarios. Telecom vendors are adopting continuous integration / continuous delivery (CI / CD) processes, which introduce automated testing procedures before and after software delivery.

[0029] Conventional systems have no mechanism available to automatically generate test scenarios upon detection of a fault. This specification describes a method and system for detecting fault cases by analyzing application logs, alert data, metric data, trace data, and automatically preparing test cases. The test cases can act as input to a test system, for example, a behavioral data-driven test framework such as Oracle® Communications 5G Automated Testing Suite. The method and system described herein can be used to prepare reports on fault scenarios with associated details about the core network.

[0030] Figure 1 is a block diagram illustrating an example 5G system network architecture. The architecture of Figure 1 includes an NRF 100 and an SCP 101, which may be located in the same home public land mobile network (HPLMN). The NRF 100 maintains a profile of available producer NF service instances and their supported services, and may enable consumer NFs or SCPs to subscribe to new / updated producer NF service instances and be notified of their registration.

[0031] The SCP 101 may also support service discovery and selection of producer NF instances. The SCP 101 may perform load balancing of connections between consumer NFs and producer NFs. Additionally, using the methodologies described herein, the SCP 101 may perform preferred NF location-based selection and routing.

[0032] The NRF 100 is a repository for the NF profile or service profile of a producer NF instance. To communicate with a producer NF instance, a consumer NF or SCP must obtain the NF profile or service profile of the producer NF instance from the NRF 100. An NF profile or service profile is a JavaScript object notation (JSON) data structure defined in 3GPP Technical Specification (TS) 29.510.

[0033] 1, any node (other than the NRF 100) may be a consumer NF or a producer NF depending on whether they are requesting or providing a service. In the illustrated example, the nodes include a policy control function (PCF) 102 that performs policy-related operations in the network, a user data management (UDM) function 104 that manages user data, and an application function (AF) 106 that provides application services.

[0034] 1 further includes a session management function (SMF) 108 that manages sessions between an access and mobility management function (AMF) 110 and the PCF 102. The AMF 110 performs mobility management operations similar to those performed by a mobility management entity (MME) in 4G networks. An authentication server function (AUSF) 112 performs authentication services for user equipment (UE), such as User Equipment (UE) 114, seeking access to the network.

[0035] The network slice selection function (NSSF) 116 provides network slicing services for devices that want to access specific network capabilities and characteristics associated with a network slice. The network exposure function (NEF) 118 provides application programming interfaces (APIs) for application functions that want to obtain information about Internet of Things (IoT) devices and other UEs connected to the network. The NEF 118 performs a function similar to the service capability exposure function (SCEF) in 4G networks.

[0036] The radio access network (RAN) 120 connects the user equipment (UE) 114 to the network via wireless links. The radio access network 120 may be accessed using a gNodeB (gNB) (not shown in FIG. 1 ) or other wireless access points. The user plane function (UPF) 122 may support various proxy functionalities for user plane services. One example of such a proxy functionality is a multipath transmission control protocol (MPTCP) proxy functionality.

[0037] The UPF 122 may also support performance measurement functionality, which may be used by the UE 114 to obtain network performance measurements. Also shown in Figure 1 is a data network (DN) 124, through which the UE accesses data network services, such as Internet services.

[0038] The SEPP 126 filters incoming traffic from another PLMN and provides topology hiding for traffic leaving the home PLMN. The SEPP 126 may communicate with a SEPP in a foreign PLMN that manages security for that foreign PLMN. Thus, traffic between NFs in different PLMNs may traverse two SEPP functions: a SEPP function for the home PLMN and a SEPP function for the foreign PLMN.

[0039] 1 illustrates an autonomous test case generator 150 that is part of or in communication with a core network of a telecommunications network. The autonomous test case generator 150 is implemented by at least one processor and is configured to detect fault cases in network functions of the core network of the telecommunications network and, in response to detecting the fault cases, autonomously generate network function test cases based on the fault cases.

[0040] The network functional test case includes one or more network status parameters detected upon detection of the fault case. The autonomous test case generator 150 is configured to provide the network functional test case to a network testing system configured to execute the network functional test case by cycling through the one or more network status parameters and determining whether the network functionality cycles through the fault case.

[0041] A telecommunications network core network can use the autonomous test case generator 150 to achieve one or more of the following advantages over some conventional systems: Currently, 5G deployments may involve a variety of 5G network functions deployed using different deployment models serving different PLMNs and network slices. Network operators and software vendors face the challenge of keeping networks functioning healthily with minimal or no downtime. Automated test systems may be a major priority for various network operators to adopt using certain types of network features. The autonomous test case generator 150 may enable network operators and software vendors to continue to automatically enhance test case suites in the automated test system. The autonomous test case generator 150 can be useful if integrated into the entire core network system. The autonomous test case generator 150 detects network function problems and reports them to, for example, product and operations teams, thereby improving key performance indicators (KPIs) and service level agreements (SLAs). The autonomous test case generator 150 can be used to automatically create a bug report with details related to a particular defect. The autonomous test case generator 150 may be used to reduce the overhead of the manual process of finding and creating 5G fault cases. The autonomous test case generator 150 can be useful in creating robust core network functions. The autonomous test case generator 150 may be used to periodically verify network functionality integrity using automatically generated test cases during specified times (e.g., non-peak hours) to declare the network functionality operating according to specifications.

[0042] 2 is a block diagram of an example network environment 200 for an autonomous test case generator 150. The autonomous test case generator 150 is configured to detect fault cases of a first network function 202 by receiving monitoring data. The network function 202 may be, for example, a consumer of data from a second network function 204. The autonomous test case generator 150 may perform monitoring of both the network function 202 and the network function 204.

[0043] The monitoring data for network functions 202 may include, for example, alerts and metrics 206, or logs and traces 208, or both. In some examples, detecting the fault case includes analyzing one or more network performance metrics, or one or more network performance alerts, or both. In some examples, detecting the fault case includes analyzing one or more network function application logs, or one or more network function traces, or both.

[0044] In general, the autonomous test case generator 150 may detect fault cases using any suitable technique. For example, detecting fault cases may include feeding the network function monitoring data to a machine learning classifier trained on training the monitoring data for the network function.

[0045] In response to detecting the fault case, the autonomous test case generator 150 is configured to autonomously generate a network function test case 210 for the network function 202 based on the fault case. The test case 210 identifies one or more network status parameters detected upon detection of the fault case.

[0046] The test cases 210 are provided to an automated network test system 212 that is configured to execute the test cases 210. Manual tests have traditionally required costly resources to perform and maintain, can be time consuming, lack adequate coverage, and are error prone due to their repetitive nature. This has led to the introduction and attraction of automating these tests. Automated testing is used to improve the speed of execution of verification, checking, or any other repeatable task in the software development, integration, and deployment lifecycle.

[0047] The automated network test system 212 can be a behavioral data-driven test framework, such as the Oracle® Communications 5G Automated Test Suite. The Automated Test Suite (ATS) allows network operators to execute software test cases using automated test tools and then compares the actual results with expected or predicted results. There is no user intervention in this process. The ATS can be realized as software used on the system under test to check if the system is functioning as expected and provides end-to-end regression testing of 4G and 5G scenarios including interacting test cases and network function (NF) emulation.

[0048] As network traffic evolves, test cases and reports can be updated. Using a test system where test cases are reviewed periodically and adjustments are made or new use case tests are developed can be useful to keep the network operational. In today's virtualized, cloud-native environment where 4G / 5G applications are no longer deployed on dedicated hardware, changes in the underlying environment can often occur outside the network operator's control. Using regression tests that are powerful, customized to the network operator's needs, and convey meaningful reports and data is useful. The ability to rapidly deploy these new test cases can be particularly useful for interaction and policy rule case additions that are made quickly and run daily and may include the subscriber / subscription lifecycle.

[0049] For example, the automated network testing system 212 may execute the test cases 210 by cycling through network status parameters and determining whether the network functions 202 repeat the fault case. In some examples, executing the test cases 210 includes repeatedly executing the test cases 210 for different network traffic conditions.

[0050] In some examples, the automated network testing system 212 is configured to associate the test case 210 with the network function 202 and one or more other network function test cases for the network function 202 in a reference library of test cases for a core network of a telecommunications network. Prior to adding the test case 210, the automated network testing system 212 and / or the autonomous test case generator 150 may be configured to determine that the test case 210 is not already present in the reference library of test cases for the core network of the telecommunications network.

[0051] The automated network test system 212 can use any suitable deployment model, such as, for example: · In-cluster deployment ·Out-of-cluster deployment.

[0052] According to the intra-cluster deployment model, the automated network test system 212 may coexist in the same cluster where the NFs are deployed. This deployment model is useful for intra-cluster testing.

[0053] According to the out-of-cluster deployment model, a network operator may deploy the automated test system 212 in a separate cluster other than the cluster in which the NF is deployed.

[0054] This deployment model is useful for "out-of-cluster" testing for the following reasons: -More aligned with production use cases · All NFs rarely coexist in the same cluster.

[0055] The test cases 210 may further be provided to computer systems of a network operations team 214, computer systems of a network feature product team 216, and to a bug database 218 that stores software issues associated with network features.

[0056] FIG. 3 is a block diagram illustrating an exemplary structure of the autonomous test case generator 150. As shown in FIG. The autonomous test case generator 150 is configured to receive monitoring data 302. The monitoring data 302 may include, for example, network function logs, traces, metrics, and alerts from different network functions.

[0057] The autonomous test case generator 150 is implemented on at least one processor and memory 304. The autonomous test case generator 150 may include an error detector and data analyzer 306 implemented on the at least one processor and memory 304. The autonomous test case generator 150 may include a test case creator and incident presenter 308 implemented on the at least one processor and memory 304.

[0058] In operation, the error detector and data analyzer 306 may perform one or more of the following operations: An error detector and data analyzer 306 can continuously receive and analyze monitoring data 302, such as metrics, alerts, traces, and application logs for one or more network functions. The error detector and data analyzer 306 can ensure that error logs, traces, metrics, and alert information is collected in the event of any failure. In some examples, the error detector and data analyzer 306 can apply analytics-driven rules using machine learning algorithms. For example, a classification and regression tree (CART) can be applied to the collected monitoring data and used to generate one or more error trigger points, such as input service actions, involved data, uniform resource identifiers (URIs), and failure reasons.

[0059] The error detector and data analyzer 306 may include a machine learning classifier trained on training data that is specific to the type of network function being monitored. A developer of the network function may supply appropriate training data that specifies, for example, the expected behavior of the network function under various network operating conditions.

[0060] During operation, the test case creator and incident presenter 308 may perform one or more of the following operations: The test case generator and incident presenter 308 can ensure that the detected faults are checked against a reference test library 310 that stores existing test cases. If a newly detected test case is not part of an existing test suite, the test case generator and incident presenter 308 can generate a test case, for example by storing one or more network parameters detected when detecting the fault case. The test case creator and incident presenter 308 can arrange test cases based on 5G NF type, 5G service operation, error category, or using any suitable type of category. The test case creator and incident presenter 308 can create a test case based on the detected fault case and send the test case to the automated network test system 212 if the test case does not exist in the reference test library 310. The automated network test system 212 can then store the test case in the library of test cases 312. The test case creator and incident presenter 308 can send details of detected fault cases to the computer systems of the network operations and product development teams 314.

[0061] To illustrate the operation of the autonomous test case generator 150, consider the following example of a 5G autonomous test case generator that uses processing for detected fault cases of type hypertext transfer protocol (HTTP) code 500.

[0062] Assume that error detector and data analyzer 306 detects an alert issued due to a failure by analyzing application log and trace data received in monitoring data 302. Error detector and data analyzer 306 analyzes the detected failure case and determines that the metric data includes a send failure response for HTTP code 500.

[0063] The error detector and data analyzer 306 may perform the following operations: Find the rules for HTTP status code 500 Use the action items prescribed for the rules Applying machine learning algorithms (e.g. CART) Collect 5G input data, such as, for example, following classification and judgment results from applying machine learning algorithms: o Input 5G service operation oHTTP message URL oHTTP input body Error message oError metrics and alert details.

[0064] The test case creator and incident presenter 308 can then generate the 5G test cases, for example, by storing the data as follows: Scenario: 5G NF PLMN-ID Update Expected result: 200 - OK - Success URI:http: / / <http-api-root> / NFServiceOperation / 5GNFInstanceId case: · Given a test suite to initialize Given NFID1, initialize the NF connection using ocnf-microserviceName.NFName and NFPort. Next, use NF NFID1 Next, check and set NF-Namespace Next, send a custom header for NF NFID1 Content-Type=application / Jason-patch+json Next, do a partial NF profile update PlmnIdUpdateInput.json NFID1 Next, verify the HTTP response code 200. Then set the user variable from the HTTP response nfInstaceId Next, get the NF instance with NFID1.

[0065] The test case creator and incident presenter 308 can then send the test case to the automated network testing system 212 .

[0066] The test case creator and incident presenter 308 can then send a message containing the incident details to the computer systems of the network operations and product development teams 314. For example, the message can identify: Incident Time: <> Incident details: <> · Error metric: TxErrorDetected

[0500] · Error alert: AlertCriticalRate

[0500] URI:http: / / <http-api-root> / NFServiceOperation / 5GNFInstanceId JSON body: Content.

[0067] FIG. 4 is a flow diagram of an example method 400 for autonomously generating network functional test cases.

[0068] The method 400 includes detecting 402 a fault case in a network function of a core network of a telecommunications network. In some examples, detecting the fault case includes analyzing one or more network performance metrics, or one or more network performance alerts, or both. In some examples, detecting the fault case includes analyzing one or more network function application logs, or one or more network function traces, or both.

[0069] In general, method 400 may include detecting a fault case using any suitable technique. For example, detecting a fault case may include providing the network function monitoring data to a machine learning classifier trained on training the monitoring data for the network function.

[0070] The method 400 includes, in response to detecting the fault case, autonomously generating (404) a network functionality test case based on the fault case. The network functionality test case includes one or more network status parameters detected upon detection of the fault case. The network status parameters may be any suitable type of data that characterizes the operation of the network or network functionality or both upon detection of the fault case or prior to detection of the fault case. For example, the network status parameters may identify the network load, the type of message being transmitted, network functionality log data, etc.

[0071] Generally, generating a network functional test case involves storing network status parameters so that the network test system can repeat the conditions that lead to the fault case. In some cases, the network functional test case may include operational parameters such as network load. In some cases, the network functional test case may identify certain operations that occurred prior to detecting the fault case.

[0072] For example, a network function test case may identify a sequence of messages sent to the network function prior to detecting a fault case. In these examples, the automated test system may repeat the sequence of messages after updating the network function to determine whether the network function repeats the fault case.

[0073] The method 400 includes providing 406 the network functional test cases to a network testing system configured to execute the network functional test cases by cycling through one or more network status parameters and determining whether the network functionality repeats the failure case. In some examples, executing the network functional test cases includes repeatedly executing the network functional test cases for a plurality of different network traffic conditions.

[0074] Providing the network functionality test case to the network testing system may include associating the network functionality test case with the network function and one or more other network functionality test cases for the network function in a reference library of test cases for core networks of the telecommunications network. Providing the network functionality test case to the network testing system includes determining that the network functionality test case is not already present in the reference library of test cases for core networks of the telecommunications network.

[0075] The scope of the present disclosure includes any feature or combination of features disclosed herein (explicitly or implicitly), or any generalization of the disclosed features, regardless of whether such feature or generalization alleviates any or all of the problems described herein. Accordingly, new claims may be prepared to any such combination of features during prosecution of this application (or an application claiming priority to this application).

[0076] In particular, with reference to the appended claims, features from the dependent claims may be combined with features from the independent claims, and features from each independent claim may be combined in any suitable manner and not just in the specific combinations recited in the appended claims.

Claims

1. 1. A method for autonomously generating network functional test cases, the method comprising: - detecting a fault case in a network function of a core network of a telecommunications network; and in response to detecting the fault case, autonomously generating a network functionality test case based on the fault case, the network functionality test case including one or more network status parameters detected upon detection of the fault case, the method further comprising: providing the network functional test case to a network testing system configured to execute the network functional test case by cycling through the one or more network status parameters and determining whether the network functionality repeats the failure case.

2. The method of claim 1 , wherein detecting the fault case comprises analyzing one or more network performance metrics, or one or more network performance alerts, or both.

3. The method of claim 1 or 2, wherein detecting the fault cases comprises analyzing one or more network function application logs, or one or more network function traces, or both.

4. The method of any one of claims 1 to 3, wherein executing the network functionality test cases comprises repeatedly executing the network functionality test cases for a plurality of different network traffic conditions.

5. 5. The method of claim 1, wherein detecting the fault case comprises providing network function monitoring data to a machine learning classifier trained on training the monitoring data for the network function.

6. 6. The method of claim 1, wherein providing the network functional test case to the network testing system comprises associating the network functional test case with the network function and one or more other network functional test cases for the network function in a reference library of test cases for the core network of the telecommunications network.

7. 7. The method of claim 1, wherein the step of providing the network functional test case to the network test system comprises the step of determining that the network functional test case is not already present in a reference library of test cases for the core network of the telecommunications network.

8. 1. A system for autonomously generating network functional test cases, the system comprising: at least one processor and memory; and an autonomous test case generator implemented by the at least one processor, the autonomous test case generator comprising: Detecting fault cases in network functions of a core network of a telecommunications network; and configured, in response to detecting the fault case, to autonomously generate a network functional test case based on the fault case, the network functional test case including one or more network status parameters detected upon detection of the fault case, the autonomous test case generator further comprising: and providing the network functional test case to a network testing system configured to execute the network functional test case by cycling through the one or more network status parameters and determining whether the network functionality repeats the failure case.

9. The system of claim 8 , wherein detecting the fault case comprises analyzing one or more network performance metrics, or one or more network performance alerts, or both.

10. 10. The system of claim 8 or 9, wherein detecting the fault case comprises analyzing one or more network function application logs, or one or more network function traces, or both.

11. The system of any one of claims 8 to 10, wherein executing the network functionality test cases comprises repeatedly executing the network functionality test cases for a plurality of different network traffic conditions.

12. The system of any one of claims 8 to 11, wherein detecting the fault case comprises supplying network function monitoring data to a machine learning classifier trained on training the monitoring data for the network function.

13. 13. The system of claim 8, wherein providing the network functional test case to the network testing system comprises associating the network functional test case with the network function and one or more other network functional test cases for the network function in a reference library of test cases for the core network of the telecommunications network.

14. 14. The system of claim 8, wherein providing the network functional test case to the network testing system comprises determining that the network functional test case is not already present in a reference library of test cases for the core network of the telecommunications network.

15. A computer readable program for causing a processor to carry out the method according to any one of claims 1 to 7.