ACTIVE TEST SYSTEM FOR MOBILE IoT NETWORK AND TEST METHOD USING THE SAME

Abstract

To provide an improved active test system and method for testing mobile networks.SOLUTION: An active testing system for a Mobile IoT (MIoT) network 2 providing connectivity and services to MIoT devices in a Low Power Wide Area (LPWA) technology includes at least one test probe 3 connected to the MIoT network 2 via an LTE-Uu interface 5 and / or an S1 interface, a central testing unit 5a connected to the at least one test probe via a wireless backhaul network or a fixed IP network 7, and a SIM multiplexer 12 provided for forwarding SIM data to the at least one test probe in a test field.SELECTED DRAWING: Figure 1

Description

[Background technology]

[0001] This application claims priority to German patent application DE 10 2019 207 051.5 and to U.S. patent application US 16 / 412 459, the contents of which are incorporated herein by reference.

[0002] The present invention relates to an active test system for a mobile IoT network, and also to a test method using the test system.

[0003] Test systems for mobile networks are known, for example, from US Pat. No. 5,399,413, US Pat. No. 5,499,623, US Pat. No. 5,523,637 and US Pat. No. 5,643,367.

[0004] Patent document 4 discloses a method and an apparatus for isolating radio segments in a mobile communication network. Patent document 5 discloses a method and a test system for authenticating a mobile test device in a mobile communication network. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] US 10,097,981 B1 [Patent Document 2] US 7,831,249 B2 [Patent Document 3] WO 2004 / 049746 A1 [Patent Document 4] US 9,768,893 B1 [Patent Document 5] DE 10 2005 027 027 B4 Summary of the Invention [Problem to be solved by the invention]

[0006] An object of the present invention is to improve the performance of such test systems for testing mobile networks. [Means for solving the problem]

[0007] That object is met by an active test system comprising the features of claim 1.

[0008] The test system according to the invention can perform testing of a Mobile Internet of Things (IoT) network that connects and provides services to Mobile IoT (MIoT) devices. Such testing is an active testing, i.e. it requires at least one component to actively initiate the respective test method. For example, a central test unit or a part thereof can be the component for actively initiating the test method.

[0009] The mobile IoT network under test is considered a subtype of installed 4G networks enhanced with Low Power Wide Area (LPWA) technology for device power saving, enhanced coverage, transmission of small amounts of data and tolerable latency.

[0010] The LPWA technology implemented can be LTE-M and / or NB-IoT. The LPWA mobile device connected to the MIoT network can be a smart meter, a home automation device, a building automation device, a part of a smart grid, a part of an industrial production line or pipeline management, a part of an automobile, a part of a transportation or logistics device, a drone, a part of a home security device, a part of a patient monitoring device, a part of an agricultural device for irrigation or shading, a part of a street lighting device, a part of a tracking device, a part of an industrial asset management device, a part of a retail / POS device, or a part of a wearable device (e.g., a part of a watch or a part of a smartphone). Voice services over LTE-M can also be tested.

[0011] The mobile IoT network may be connected to the MIoT application platform and / or the IoT application platform via an application server.

[0012] Such a test system can be used to perform connectivity testing of MIoT networks and / or testing of MIoT application platforms. The test system is adapted according to the IoT network architecture and scalability and can deploy one or more test probes. The test probes can be installed at different locations (test fields) within a single IoT network or among several interconnected networks. In particular, the data communication embodied by the SIM of a mobile IoT device can be simulated and / or emulated either by the LTE-Uu radio interface or by the S1 core network interface.

[0013] The SIM multiplexer can route the SIM data virtually and / or securely to at least one test probe.

[0014] The SIM multiplexer may be embodied as a support for carrying multiple SIMs, for example up to three or more SIMs.

[0015] The test system can be configured to execute a mobile IoT test procedure deploying an end-to-end active test methodology between at least one test probe and the MIoT network under test. The test system can be configured to control the test probe(s) via a specific active test platform including a central test unit. Furthermore, via the central test unit, the test system can automatically execute the IoT test procedures, collect test results and generate test reports and dashboards.

[0016] Within the test system, test methods and test sequences are deployed to test MIoT applications and / or services running over the IoT connectivity provided by the MIoT network under test.

[0017] "End-to-end" testing means that the connectivity between an MIoT device and an MIoT application server and the services provided to the MIoT device by the MIoT application are tested using data transfer to and from the MIoT device, and in particular to and from at least one test probe that represents and simulates the MIoT device.

[0018] In particular, the service data flow of the application is tested to see if it works as expected from start to finish. In particular, every step of the application and / or service is tested.

[0019] The download and upload data speeds and / or download or upload bandwidth can be tested.

[0020] Data transfer tests can be performed with different sizes of data sent and received, in particular different numbers of data packets, and / or different amounts of data.

[0021] The data transfer quality and the integrity of the data transfer can be tested.

[0022] Furthermore, the test system can be designed to test the performance of MIoT networks in the deployment of power saving mode (PSM) and / or enhanced discontinuous reception (eDRX) for MIoT applications.

[0023] The test system may be designed to configure and initiate at least one test probe to trigger and initiate at least one of a group of power save mode (PSM) or enhanced discontinuous reception (eDRX) modes in the serving MIoT network under test. The test system may be specifically designed to negotiate the power save mode and the eDRX mode in combination with the configuration and activation of the test probe to an evolved packet system (EPS) attach in the serving MIoT network under test.

[0024] The test system can be designed to configure and initiate at least one test probe to access and query the IoT application server using a variety of protocols, including but not limited to oneM2M, Hypercat, constrained application protocol (CoAP), MQTT / MQTT-SN (message queuing telemetry transport), Real-time Streaming Protocol (RTSP), or through device-specific interfaces, such as JavaScript Object Notation (JSON) and / or Extensible Markup Language (XML) over HTTP.

[0025] Literature on the oneM2M protocol can be accessed at www.onem2m.org. Information on the protocol "Hypercat" can be obtained from John Davies, Hypercat: resource discovery on the Internet of Things (January 12, 2016): IEEE Internet of Things, March 2, 2017. http: / / iot.ieee.org. Information on the protocol "CoAP" can be found in RFC7252, which can be found at https: / / tools.ietf.org / html / rfc7252. Information on JSON can be found in RFC8259 and ECMA-404. Information on RTSP can be found in RFC2326.

[0026] The signal and data exchange according to claim 2 allows testing of the most common signal messages and data types with the test system.

[0027] The arrangement as claimed in claim 3 proves to be essential for the most common testing requirements.

[0028] This applies in particular to the test system according to claim 4.

[0029] The message structure according to claim 5 is suitable for testing IoT application platforms. Alternatively or additionally applicable protocols and / or interfaces for communicating with such test systems are oneM2M, Hypercat, CoAP, RTSP, JSON, XML.

[0030] A testing method as claimed in claim 6 has the advantages as stated above with respect to a testing system according to the invention, in particular an end-to-end testing method, which in particular comprises testing an IoT application platform, in particular a server of said platform.

[0031] The method of claim 7 allows for testing the availability of IoT services of the network by simulating / emulating respective mobile IoT devices with test probes in the IoT network. The test steps may be repeated periodically during the test method. Recorded test results may be aggregated and further statistically evaluated.

[0032] The connectivity of the mobile IoT may be performed in a testing method as claimed in claim 8. Again, the repeating step may be repeated periodically and the test results may be collected for further statistical evaluation.

[0033] Such ping tests can assess the IoT network accessibility of a ping test probe and / or the ping / echo round trip time.

[0034] A testing method as claimed in claim 9 allows testing the power saving functionality of each mobile IoT device. Again, the repeating steps may be repeated periodically and the test results may be collected for further statistical evaluation.

[0035] As part of such a power saving test methodology, mobile terminated data forwarding in combination with power saving features managed by the IoT serving network under test may be tested by transmitting downlink data towards the test probe while a T3324 active timer is running, verifying that the test probe receives complete downlink data packets, monitoring and recording all test events, and repeating the above test according to a predefined schedule. Again, the repeating steps may be repeated periodically and the test results may be compiled for further statistical evaluation.

[0036] Further, in such power saving tests, mobile terminated SMS in combination with power saving features managed by the IoT serving network under test may be tested by sending an SMS to a test probe while a T3324 active timer is running, verifying that the test probe receives the SMS, monitoring and recording all test events, and repeating the above test according to a predefined test schedule. Again, the repeating step may be repeated periodically and the test results may be compiled for further statistical evaluation.

[0037] The testing method according to claim 10 allows testing the functionality of eDRX and, as a result, evaluating the capabilities of further power saving features. Again, the repeating step may be repeated periodically and the test results may be collected for further statistical evaluation.

[0038] In such an eDRX testing method, mobile terminated data forwarding in combination with eDRx functionality managed by the IoT serving network under test may be tested by transmitting downlink data towards a test probe within a Paging Time Window (PTW), verifying that the test probe receives a complete downlink data packet, monitoring and recording all test events, and repeating the above test according to a predefined test schedule. Again, the repeating steps may be repeated periodically and the test results may be collected for further statistical evaluation.

[0039] Furthermore, such eDRX testing method may test a mobile terminated SMS in combination with eDRX functionality managed by the IoT serving network under test by sending an SMS to a test probe within a Paging Time Window (PTW), verifying that the test probe receives the SMS, monitoring and recording all test events, and repeating the above test according to a predefined test schedule. Again, the repeating step may be repeated periodically and the test results may be compiled for further statistical evaluation.

[0040] The testing method according to claim 11 allows testing connectivity and unsolicited network-initiated detach requests.

[0041] The testing method as claimed in claim 12 allows testing Mobile Originated (MO) data transfers.

[0042] The test method as claimed in claim 13 allows testing Mobile Terminated (MT) data transfer.

[0043] The testing method as claimed in claim 14 allows testing mobile originated SMS transmissions.

[0044] The testing method as claimed in claim 15 allows testing mobile terminated SMS delivery.

[0045] Data and SMS data delivery can be tested after and during power save mode.

[0046] Exemplary embodiments of the present invention will be further described with reference to the accompanying drawings. [Brief description of the drawings]

[0047] [Figure 1] A key component of an active test system for a mobile Internet of Things (IoT) network includes at least one test probe connected to the IoT network via a wireless interface. [Diagram 2] 1 and 2. A diagram similar to FIG. 1 and illustrating a further embodiment of a test system for a mobile IoT network including a test probe connected to the IoT network via an S1 interface. [Diagram 3] 2 is a diagram similar to FIG. 1 and illustrating an embodiment of a test system configured to test an IoT serving network with a test connection path over a roaming interface; [Figure 4]1 is a block diagram of a test system for IoT devices that includes a test probe configured to communicate with an IoT application platform of an IoT service via MQTT / MQTT-SN messages; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0048] Figure 1 shows the main components of an active testing system 1 for a Mobile Internet of Things (IoT) network 2, represented by the various communication links shown in Figure 1. The communication links are either pure signaling paths, signaling paths embedded with IoT data, or IoT data transport paths. The Mobile IoT (MIoT) network 2 provides connectivity and services to Mobile IoT devices in the Low Power Wide Area (LPWA) technology. The LPWA frequency bands used are in the regularly licensed frequency bands. The deployed LPWA technology may be LTE-M and / or NB-IoT.

[0049] Throughout this application, the following documents are referenced, particularly with respect to standardized specifications for IoT networks: - GSM Association; Official document CLP.28-NB-IoT Deployment Guide to Basic Feature set Requirements, version 1.0, August 2, 2017 (GSMA White Paper). - Technical Specification 3GPP TS 23.682, V.15.8.0, Release 15, March 2019.

[0050] The mobile IoT network tested via the test system 1 may also be equipped with EC-GSM-IoT (Extended Coverage GSM IoT). Other communication technologies may also be used for additional network access for machine-to-machine communication. For example, examples of short-range wireless communication include Bluetooth Mesh Networking, Light-Fidelity (Li-Fi), Near-Field Communication (NFC), Wi-Fi, ZigBee, or Z-Wave. Examples of medium-range wireless communication include LTE-Advanced or LTE-Standard, examples of long-range wireless communication include Lo-RaWan, Sigfox, or Weightless, or Very Small Aperture Terminal (VSAT), and examples of wired communication include Ethernet or power line communication.

[0051] The test system 1 of Figure 1 includes one or more test probes 3 that are components of a local unit 4 of the test system 1. Several examples of such local units 4 are shown in Figure 1. The local unit 4 may include one to four or more test probes 3, for example up to fifteen or more. The test probes 3 are connected to the IoT network 2 via a wireless interface 5. Each interface 5 is shown diagrammatically in Figure 1 together with an LTE RAN (Radio Access Network) and may be embodied by multiple antenna sites.

[0052] The central test unit 5a is connected to the test probes 3 via internet networks 7, 8. Such a connection may be, for example, a permanent secure IP connection via a VPN server and a LTE / GPRS / EDGE / HSPA modem, or a semi-permanent IP connection established via a VPN server when required during testing.

[0053] 1, network 7, which may be a wireless backhaul network or a fixed IP network, includes the Internet 8 and further includes components of the Evolved Packet Core (EPC) of the 3GPP LTE communications standard. Typically, components of network 7 are also components of the IoT network 2, although this is not required and some of the components of network 7 may be separate from the IoT network 2.

[0054] A further part of the test system 1 shown on the left hand side of FIG. 1 is a test client 5 b which is also connected to the networks 7 , 8 of the test system 1 .

[0055] 1 shows two main test communication paths between the test probe 3 and an application server (AS) 6 functioning as an IoT platform. First, the NB-IoT test communication line 9 extends from the local radio interface 5 to which it belongs via a Mobile Management Entity (MME), and then has two branching methods: via a Serving Gateway (S-GW), a Packet Data Network Gateway (P-GW), and a network 7 to the AS / IoT platform 6 (so-called direct mode), and via a test line 11, a Service Capability Exposure Function (SCEF), a Services Capability Server (SCS), and the network 7 to the AS / IoT platform 6 (so-called indirect mode).

[0056] A further LTE-M test communication path 10 extends from the local radio interface 5 directly to the S-GW (i.e. without passing through the MME) and further via a Serving Gateway (S-GW), a Packet Data Network Gateway (P-GW) and the network 7 to the AS / IoT platform 6.

[0057] Every test probe is equipped with a SIM. A SIM multiplexer 12 is connected to the Internet 8 and transmits the SIM data virtually to the individual test probes in a completely secure and reliable manner. A SIM multiplexer which is part of the test system 1 is known from US Pat. No. 5,399,633.

[0058] Local unit 4 and local unit 16 include support for SIM multiplexers.

[0059] Instead of supporting a SIM multiplexer, the local unit 4 may include support for carrying multiple SIMs, for example up to three or more SIMs.

[0060] The local unit 4 is placed in a given test field and connected to the test system. In practice, multiple test probes 3 are deployed at different locations, in particular in a wide range of locations covering a national or inter-national area. As a result, at least one test probe 3 is configured to be deployed either in the home IoT network under test for testing a national IoT service, or in the visited IoT network under test for testing an international IoT roaming service.

[0061] The test system 1 is configured to execute a mobile IoT test procedure deploying an end-to-end active test method between at least one test probe 3 and an IoT network 2 under test.

[0062] Further, the test system 1 is configured to control at least one test probe 3, configured to automatically execute IoT test procedures, configured to collect test results, and further configured to generate test reports and / or dashboards.

[0063] The test system 1 is configured to exchange signalling messages and is adapted to transmit IP and / or non-IP data and / or SMS to and from the IoT network 2 under test.

[0064] The test communication path between the test probe 3 and the IoT platform 6 over the Internet network 8 includes an MQTT / MQTT-SN client / server structure, where the test probe 3 is the MQTT client and the IoT platform 6 is the server / MQTT broker. IoT application data stored in the IoT platform can be retrieved, evaluated and verified via the messaging protocol MQTT (Message Queuing Telemetry Transport).

[0065] Further components in the network and / or communication path may be a Service Capability Server (SCS) and / or an Application Server (AS). For such an SCS / AS arrangement, please refer to technical specification 3GPP TS 23.682, in particular Figure 4.2-1a.

[0066] Further possible interfaces may operate in accordance with standardized S6a, S8, SGd, or T7 roaming interfaces.

[0067] Via a test method or procedure implemented by the test system 1, the availability and quality of a serving IoT network 2 under test can be tested. During the service availability test, in particular the timing of the respective test events can be monitored and recorded in the central test unit 5a.

[0068] The test methods described are controlled by a central test unit 5a unless otherwise indicated.

[0069] An example of such a test method includes the following steps.

[0070] Each test probe 3 is configured via a test client 5b and initiated to an Evolved Packet System (EPS) attach in the serving IoT network 2 under test. After initiation of the EPS attach, completion of the attach procedure and received messages from the IoT network under test by the test probe 3 are verified by the central test unit 5a.

[0071] All test events during the setup, initiation and verification steps are monitored and recorded. These test steps are repeated according to a predefined test schedule. In particular, such repetition may be periodic re-execution of the test steps. Furthermore, the results of such multiple tests are aggregated and forwarded to a statistical evaluation and the test results are presented to a tester (human tester) via the test client 5b.

[0072] In a particular test method, a test probe is initiated to ping a server or IoT network component (eg, a P-GW) deployed in the serving IoT network 2 under test to verify network connectivity.

[0073] This "ping" is done using the respective IP software utility.

[0074] After such a ping procedure, its completion is verified, and again all test events are monitored and recorded, and the test steps are repeated according to a predefined test schedule.

[0075] In a further test method, the power saving features managed by the serving IoT network 2 under test can be tested. Testing such power saving features includes enabling a power saving mode (PSM) in the respective test probe 3, thereby setting the values ​​of the T3324 active timer and the extended T3412 timer in the test probe 3 in the local unit 4. Afterwards, starting an EPS attach of the test probe 3 in the serving IoT network 2 and verifying the completion of the attach procedure. Furthermore, it is verified whether the timer values ​​are accepted by the serving IoT network 2 by comparing these values ​​with the values ​​requested by the respective test probe 3. Furthermore, it is verified whether the extended periodic tracking area update (TAU) procedure is completed. Again, all test events are monitored and recorded during this test method and the above tests are repeated according to a predefined schedule.

[0076] A further test method tests the mobile terminated data forwarding in combination with the power saving functionality (PSM) managed by the serving IoT network 2 under test. For this purpose, during the period when the T3324 active timer is running, downlink data is transmitted to each test probe 3. It is verified that each test probe 3 receives the complete downlink data packet.

[0077] The PSM verifies whether each test probe 3 is able to wake up from a power-save state to a communication state. Such a wake-up occurs, for example, once a week for two minutes. The duty cycle of such a wake-up is negotiated in the EPS attachment procedure. The test method tests the wake-up functionality and in particular the state changes between power-save and communication states.

[0078] Again, all test events are monitored and recorded during the test method, and the above tests are repeated according to a predefined test schedule.

[0079] In a further test method for testing mobile terminated SMS in combination with power saving features managed in the serving IoT network 2 under test, an SMS is sent to the test probe 3 while the T3324 active timer is running.

[0080] Then it is verified whether the SMS is correctly delivered to the test probe. All test events during this test method are again monitored and recorded, and the above test method is repeated according to a predefined test schedule.

[0081] In a further test method, the Extended Discontinuous Reception (eDRX) functionality managed in the serving IoT network 2 under test is tested. In this method, eDRX is enabled and eDRX cycle length and Paging Time Window (PTW) values ​​are set in the respective test probe 3 of the local unit 4. Furthermore, an EPS attachment of the test probe 3 in the serving IoT network 2 under test is initiated. Completion of such attachment procedure is verified. Furthermore, it is verified whether the eDRX cycle length and PTW values ​​are accepted by the serving IoT network 2 by comparing these values ​​with the values ​​requested by the respective test probe 3. All test events of this method are monitored and recorded and the above test method is repeated according to a predefined test schedule.

[0082] In a further test method, the mobile terminated data forwarding in combination with the eDRX functionality managed by the serving IoT network 2 under test is tested. Here, downlink data is transmitted towards the respective test probe 3 within a PTW (Paging Time Window). It is verified whether the test probe 3 receives the complete downlink data packet. All test events of this test method are monitored and recorded and the test method is repeated according to a predefined test schedule.

[0083] In a further test method, the mobile terminated SMS in combination with the eDRX functionality managed by the serving IoT network 2 under test is tested. Here, an SMS is sent to each test probe 3 within a PTW. It is verified whether the SMS is correctly delivered to the test probe. All test events during this test method are monitored and recorded. The above test method is repeated according to a predefined test schedule. In a further test method, the connectivity of the IoT network 2 is tested. Here, it is verified whether each test probe 3 is requested to detach from the serving IoT network 2 after EPS attach or after a mobile originated (MO) or mobile terminated (MT) data transfer. This verification step is repeated multiple times. The results of multiple tests of this test method are tallied. From this tallied, a default EPS bearer context cutoff ratio is calculated.

[0084] In a further method, the IoT MO data forwarding provided by the serving IoT network 2 under test is tested. Here, a TCP (Transmission Control Protocol) transport protocol is deployed. Furthermore, a mobile-originated IoT data forwarding is initiated from each test probe 3 to an application server 6 located in the home network (HPMN). It is verified whether the IoT data is correctly received at the application server 6. This verification step is repeated multiple times and multiple test results are aggregated to indicate a default EPS bearer context cutoff ratio. Furthermore, a UDP (User Datagram Protocol) is introduced and the above IoT MO data forwarding test is repeated. Furthermore, a non-IP data delivery mechanism over NAS (Non-Access Stratum) signaling is introduced. Again, the IoT MO data forwarding test is repeated.

[0085] In a further test method the IoT MT (Mobile Termination) data forwarding provided by the serving IoT network 2 under test is tested. Here, after deployment of the TCP transport protocol, an application server, i.e. a server of the IoT platform 6 located in the home network, is initiated to send IoT data to the respective test probe 3. It is verified whether the IoT data is fully received by the respective test probe 3. Further steps of this test method, including the deployment of the UDP transport protocol and the deployment of a non-IP data delivery mechanism over NAS signaling, correspond to those explained above with respect to the MO data forwarding test method.

[0086] In a further test method, MO SMS transmissions through the serving IoT network 2 under test are tested, where each test probe 3 is initiated to send an SMS to a partner test probe 3 of a local unit 4 in the home network (HPMN), and it is then verified whether the SMS is correctly received by the partner test probe 3. The test is repeated multiple times and the multiple test results are aggregated for further statistical evaluation.

[0087] In a further testing method, MT SMS delivery via the tested serving IoT network 2 is tested. Here, a partner test probe 3 in the home network (HPMN) is initiated to send an SMS to a test probe 3 in the serving IoT network 2. It is verified whether the SMS sent by the partner test probe 3' is correctly delivered to the respective test probe 3 in the serving IoT network 2. Again, this test is repeated multiple times and multiple test results are compiled.

[0088] A further embodiment of a test system 15 for a mobile IoT network is described with respect to Fig. 2. Components and functions corresponding to those described above with respect to Fig. 1 are indicated with the same reference numerals and will not be described in detail again. In addition to the local units, a central test unit, a test client and a SIM multiplexer that may be present in the test system 15 are not shown in Fig. 2.

[0089] In the test system 15, a local unit 16 including a test probe 3 is embodied as an S1 core unit connected to the IoT network 2 via an S1 interface 17. A communication line 18 via this S1 interface 17 is realized as an emulated eNodeB (Evolved NodeB). Details regarding the embodiment of the S1 interface and its protocol are described in 3GPP TS 36.413 V.15.5.0, Release 15, March 2019: “Evolved Universal Terrestrial Access Network (E-UTRAN); S1 Application Protocol (S1AP)”.

[0090] Figure 3 shows another embodiment of a test system 20. Components and features corresponding to those already described with respect to Figures 1 and 2 have the same reference numerals and will not be described in detail again.

[0091] The test system 20 provides connectivity and service testing for roaming mobile IoT devices, where communication over lines 9, 10 and 11 takes place between a Home Public Mobile Network (HPMN) and a Visiting Public Mobile Network (VPMN) across a border 21. For this purpose, in addition to the SCEF module of the HPMN, the communication line 11 further includes an Interworking SCEF (IWK-SCEF) module located in the VPMN.

[0092] This roaming scheme is possible with the radio interface 5, again shown in FIG. 3, and with the S1 interface 17 according to FIG. 2 (not shown in FIG. 3).

[0093] Figure 4 shows components of a further embodiment of a test system for mobile IoT networks, including details regarding test data transfer between different test probes 3, 3'. Such different test probes 3, 3' may be from different, e.g. home / visited public mobile networks. Components and functions corresponding to those described above with respect to Figures 1 to 3 have the same reference numerals and will not be described in detail again.

[0094] A first of the test probes, the test probe 3, is connected to the IoT platform 6 via a test communication link 22, which may include a wireless interface 5 or an S1 interface 17. The test probe 3 includes an MQTT / MQTT-SN client 23 that communicates over the communication link 22 with an MQTT / MQTT-SN server / broker 24 of the IoT platform 6 under test.

[0095] Another test probe, test probe 3', in Fig. 4 communicates with the IoT platform under test 6 via another test communication line 25, which may also include the wireless interface 5 or the S1 interface 17. To this end, the further test probe 3' also includes an MQTT / MQTT-SN client 23'.

[0096] Alternatively or in addition to MQTT / MQTT-SN, at least one of the following further protocols and / or one of the following device specific interfaces may be used: oneM2M, Hypercat, CoAP, RTSP, JSON, XML.

[0097] The use of a particular protocol / interface depends on the respective IoT device and / or the respective application. As an example, CoAP is suitable for constrained networks with low bandwidth and low power.

[0098] The test probes 3, 3′ may be part of a mobile device, i.e. part of a vehicle (e.g. a bicycle or a car). In this case, the test device may initiate a tracking area update each time the respective test probe 3 enters a new tracking area in the MIoT network 2. After such a tracking area update, the data assigned to the PSM and / or the data assigned to the eDRX function may be renegotiated and / or overwritten by initiating the test system.

[0099] By using particularly one of the test methods described above, the relevant IoT data transmitted by the test probe 3 and stored in the platform under test 6 can be read by the test probe 3'. The read data is compared with the original transmitted data. A corresponding test verdict can be assigned.

[0100] The test probes 3, 3' may be located in the same home network. Alternatively, the test probes 3, 3' may be located in separate networks, as shown in Fig. 4. For example, a test probe 3 may be located in a visited public network VPMN and a further test probe 3' may be located in a home public mobile network HPMN, as shown in Fig. 4. Such a configuration allows the above-described IoT application platform test under IoT device roaming to be performed globally. [Explanation of symbols]

[0101] 1, 15, 20 Active Test System 2. IoT Network 3, 3' Test probe 4 Local Unit 5. Wireless Interface 5a Central Testing Unit 5b Test Client 6 AS / IoT Platform 7, 8 Internet network 9 NB-IoT test communication line 10 LTE-M test communication line 11, 18 Communication lines 12 SIM multiplexer 16 Local Unit 17 S1 Interface 21 Boundary 22, 25 Test communication lines 23. 23' MQTT / MQTT-SN client 24 MQTT / MQTT-SN Server / Broker

Claims

1. An active test system (1; 15; 20) for a mobile IoT network (2) that provides connectivity and services to mobile IoT (MIoT) devices in a low power wide area (LPWA) technology, the test system is designed to test the quality of the MIoT service of the serving MIoT network under test and to test the availability of the MIoT service; at least one test probe (3; 3, 3') connected to said MIOT network (2) via an LTE-Uu interface (5); and / or at least one test probe (3; 3, 3') connected to said MIOT network (2) via an S1 interface (17); a central test unit (5a) connected to said at least one test probe (3; 3, 3') via a wireless backhaul network or a fixed IP network (7); a SIM multiplexer (12) for transferring SIM data to said at least one test probe (3; 3, 3') of a test field, An active test system, characterized in that the test system configures and initiates test probes on EPS attaches in the serving MIOT network under test, verifies completion of the test procedure, monitors and records all test events, and repeats the above test procedure according to a test schedule.

2. 2. The test system of claim 1, configured to exchange signaling messages and transmit IP data, non-IP data, or SMS to and from the MIOT network (2) under test.

3. 3. The test system according to claim 1 or 2, characterized in that the at least one test probe (3; 3, 3') is configured to be located in a serving network, i.e. in a Home IoT Network (2, HPMN) to be tested for testing a national MIoT service or in a Visited IoT Network (2, VPMN) to be tested for testing an international MIoT roaming service.

4. 4. The test system of claim 3, configured to test the MIoT serving network under test over different test connections and communication paths across different MIoT network (2) components via MME, S-GW, P-GW, SCEF, IWK-SCEF, SCS, AS, and across roaming interfaces S6a, S8, SGd, T7.

5. Communicating with the MIOT platform (6) under test via MQTT / MQTT-SN messages; 5. The test system of claim 1, configured to verify availability and connectivity of the MIoT platform (6) through the underlying MIoT network, end-to-end data transfer, and data integrity between the MIoT platform (6) and the mobile IoT device.

6. A test method for testing the MIOT service quality of the serving MIOT network under test using the test system according to any one of claims 1 to 5.

7. Test the availability of MIOT services; - configuring and initiating a test probe on an EPS attach in the serving MIoT network under test; - verifying completion of said attachment procedure; - monitoring and recording all test events; repeating the above test steps according to a test schedule; The test method of claim 6, comprising:

8. Testing the connectivity of IoT networks; - initiating the test probe to ping a server in the serving MIoT network under test; - verifying the completion of said ping procedure; - monitoring and recording all test events; repeating the above test steps according to a test schedule; 8. The test method according to claim 6, comprising:

9. Testing power saving features managed by the MIoT serving network under test; - enabling PSM (power saving mode) thereby setting the values ​​of the T3324 active timer and the extended T3412 timer in said test probe; - initiating an EPS attach of the test probe in the serving MIoT network; - verifying completion of said attachment procedure; - verifying whether the timer values ​​are acceptable to the serving MIOT network by comparing them with the values ​​requested by the test probe; - verifying whether the extended periodic TAU (Tracking Area Update) procedure is acceptable; - monitoring and recording all test events; repeating the above test steps according to a test schedule; 9. The test method according to claim 6, comprising:

10. Testing eDRX functionality managed by the serving IoT network under test; - enabling Extended Discontinuous Reception (eDRX) thereby setting eDRX cycle length and paging time window (PTW) values ​​in said test probe; - initiating an EPS attachment of the test probe in the serving IoT network under test; - verifying completion of said attachment procedure; - verifying whether the eDRX cycle length and the PTW values ​​are acceptable to the serving IoT network by comparing them with the values ​​requested by the test probe; - monitoring and recording all test events; repeating the above test steps according to a test schedule; 10. The test method according to claim 6, comprising:

11. Testing the connectivity of an IoT network; - verifying whether the test probe is requested by the serving IoT network for detachment after an EPS attach or after a Mobile Originated (MO) or Mobile Terminated (MT) data transfer; repeating this verification step a number of times; aggregating results of said multiple tests to indicate a default EPS bearer context cut-off ratio; 11. The test method according to any one of claims 6 to 10, comprising:

12. Testing IoT Mobile Originated (MO) data transfer provided by the serving IoT network under test; - deploying a TCP transport protocol; - initiating a mobile-originated IoT data transfer from said test probe to an application server located in said home network (HPMN); - verifying whether said IoT data is correctly received by said application server; repeating this verification step a number of times; aggregating said multiple test results to indicate said default EPS bearer context cutoff ratio; - deploying a UDP transport protocol; - repeating the IoT MO data transfer test; - deploying a non-IP data delivery mechanism over NAS signalling; - repeating the IoT MO data transfer test; 12. The test method according to claim 6, comprising:

13. Testing IoT Mobile Terminated (MT) data transfer provided by the serving IoT network under test; - deploying a TCP transport protocol; - initiating an application server located in said home network (HPMN) to transmit IoT data to said test probe; - verifying whether said IoT data is completely received by said test probe; repeating this verification step a number of times; aggregating said multiple test results to indicate said default EPS bearer context cutoff ratio; - deploying a UDP transport protocol; - repeating the IoT MT data transfer test; - deploying a non-IP data delivery mechanism over NAS signalling; - repeating the IoT MT data transfer test; 13. The test method according to any one of claims 6 to 12, comprising:

14. Testing MO SMS transmissions over the serving IoT network under test; - initiating said test probe to send an SMS to a partner test probe in said home network (HPMN); - verifying whether said SMS is correctly delivered to said partner test probe; - repeating said test multiple times; - aggregating the results of said multiple tests; 14. The test method according to any one of claims 6 to 13, comprising:

15. Testing MT SMS delivery over the serving IoT network under test; - initiating a partner test probe in a home network to send an SMS to said test probe in said serving IoT network; - verifying whether the SMS sent by the partner test probe is correctly delivered to the test probe in the serving IoT network; - repeating said test multiple times; - aggregating the results of said multiple tests; 15. The test method according to any one of claims 6 to 14, comprising:

Images