Experimental trace in exploratory handover data collection

Abstract

The present disclosure relates to experimental tracking in exploratory handover data collection. Embodiments of the present disclosure relate to data collection for exploratory handover (HO). In one aspect, a terminal device receives, from a first network device, configuration information for performing an exploratory HO, the configuration information including an experimental ID identifying a tested HO configuration and a tested scenario for the exploratory HO. The terminal device performs the exploratory HO based on the configuration information to record at least one data sample for the exploratory HO, the at least one data sample including the experimental ID. The terminal device reports the at least one data sample.

Description

Technical Field

[0001] Various example embodiments relate to the field of communications, and particularly to apparatus, methods, devices, and computer-readable storage media for data collection during exploratory handover (HO). Background Technology

[0002] A communication network can be viewed as a facility that enables communication between two or more communication devices or provides communication devices with access to a data network. Mobile or wireless communication networks are an example of communication networks.

[0003] Such communication networks operate according to standards such as those issued by 3GPP (3rd Generation Partnership Project) or ETSI (European Telecommunications Standards Institute). Examples of such standards include the so-called 5G (fifth generation) standard, the 6G (sixth generation) standard, or other standards issued by 3GPP. Summary of the Invention

[0004] In general, the exemplary embodiments of this disclosure provide a solution for data collection in exploratory HOs, for example, using an experiment ID associated with the exploratory HO.

[0005] In a first aspect, a terminal device is provided. The terminal device includes at least one processor and at least one memory storing instructions. When executed by the at least one processor, the instructions cause the terminal device to at least: receive configuration information from a first network device for performing an exploratory hoisting (HO), the configuration information including an experiment identifier (ID) identifying a test HO configuration and a test scenario of the exploratory HO; perform the exploratory HO based on the configuration information to record at least one data sample of the exploratory HO, the at least one data sample including the experiment ID; and report at least one data sample.

[0006] In some embodiments, the experiment ID includes at least one of the following: the ID of a network function (NF) responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the ID of the execution of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0007] In some embodiments, the ID of an NF includes a fully qualified domain name (FQDN) or a numeric ID.

[0008] In some embodiments, an NF includes one of the following: a Radio Access Network (RAN) NF; a Core Domain NF; or a Network Management System (NMS).

[0009] In some embodiments, at least one data sample is recorded using over-the-top (OTT) data collection.

[0010] In some embodiments, at least one data sample includes recorded Driven Test Minimum (MDT) data.

[0011] In some embodiments, the terminal device may report at least one data sample by reporting at least one data sample to the core network device.

[0012] In some embodiments, the terminal device may report at least one data sample by reporting at least one data sample to a first network device.

[0013] In some embodiments, the first network device is a first access network device. The terminal device may report at least one data sample by reporting at least one data sample to a second access network device instead of the first access network device.

[0014] In a second aspect, a first network device is provided. The first network device includes: at least one processor; and at least one memory storing instructions, which, when executed by the at least one processor, cause the first network device to at least: receive first configuration information from a second network device for performing an exploratory hoisting (HO), the first configuration information including an experiment ID identifying a test HO configuration and a test scenario of the exploratory HO; and send second configuration information to a terminal device for performing the exploratory HO, the second configuration information including the experiment ID.

[0015] In some embodiments, the first network device may receive at least one data sample of an exploratory HO from a terminal device, the at least one data sample including an experiment ID.

[0016] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0017] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0018] In some embodiments, the first network device may also determine the target device for sending at least one data sample based on the experiment ID.

[0019] In some embodiments, the first network device may send at least one data sample to the second network device.

[0020] In some embodiments, the second network device includes a network management system (NMS) or functions within the NMS.

[0021] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0022] In some embodiments, at least one data sample is recorded using OTT data collection.

[0023] In some embodiments, at least one data sample includes recorded MDT data.

[0024] In a third aspect, a second network device is provided. The second network device includes at least one processor and at least one memory storing instructions. When executed by the at least one processor, the instructions cause the second network device to at least: send first configuration information to a first network device for performing an exploratory homepage (HO), the first configuration information including an experiment ID identifying a test HO configuration and a test scenario of the exploratory HO; receive at least one data sample recorded for the exploratory HO, the at least one data sample including the experiment ID; and associate the at least one data sample with the exploratory HO based on the experiment ID.

[0025] In some embodiments, the second network device may calculate at least one metric of the exploratory HO.

[0026] In some embodiments, the second network device may compute at least one metric of another HO, rather than an exploratory HO, which does not carry an experimental ID.

[0027] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0028] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0029] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0030] In some embodiments, at least one data sample is recorded using OTT data collection.

[0031] In some embodiments, at least one data sample includes recorded MDT data.

[0032] In some embodiments, the second network device may receive at least one data sample by receiving at least one data sample from the core network device.

[0033] In some embodiments, the second network device may receive at least one data sample by receiving at least one data sample from the first network device.

[0034] In some embodiments, the second network device includes NMS or functions within NMS.

[0035] In a fourth aspect, a method is provided. The method includes: receiving configuration information from a first network device for performing an exploratory hoisting (HO), wherein the configuration information includes an experiment ID identifying a test HO configuration and a test scenario of the exploratory HO; performing the exploratory HO based on the configuration information to record at least one data sample of the exploratory HO, wherein the at least one data sample includes the experiment ID; and reporting the at least one data sample.

[0036] In a fifth aspect, a method is provided. The method includes: receiving first configuration information from a second network device for performing an exploratory home visit (HO), the first configuration information including an experiment ID identifying a test HO configuration and a test scenario of the exploratory HO; and sending second configuration information for performing the exploratory HO to a terminal device, wherein the second configuration information includes the experiment ID.

[0037] In a sixth aspect, a method is provided. The method includes: sending first configuration information for performing an exploratory home visit (HO) to a first network device, the first configuration information including an experiment ID identifying a test HO configuration and a test scenario of the exploratory HO; receiving at least one data sample recorded for the exploratory HO, wherein the at least one data sample includes the experiment ID; and associating the at least one data sample with the exploratory HO based on the experiment ID.

[0038] In a seventh aspect, an apparatus is provided. The apparatus includes components for receiving configuration information from a first network device for performing an exploratory hoisting (HO), the configuration information including an experiment ID identifying a test HO configuration and a test scenario of the exploratory HO; components for performing the exploratory HO based on the configuration information to record at least one data sample of the exploratory HO, the at least one data sample including the experiment ID; and components for reporting the at least one data sample.

[0039] In an eighth aspect, an apparatus is provided. The apparatus includes components for receiving first configuration information from a second network device for performing an exploratory home visit (HO), the first configuration information including an experiment ID identifying a test HO configuration and a test scenario of the exploratory HO; and components for sending second configuration information for performing the exploratory HO to a terminal device, the second configuration information including the experiment ID.

[0040] In a ninth aspect, an apparatus is provided. The apparatus includes components for sending first configuration information for performing an exploratory home visit (HO) to a first network device, the first configuration information including an experiment ID identifying a test HO configuration and a test scenario of the exploratory HO; components for receiving at least one data sample recorded for the exploratory HO, wherein the at least one data sample includes the experiment ID; and components for associating the at least one data sample with the exploratory HO based on the experiment ID.

[0041] In a tenth aspect, a non-transitory computer-readable medium is provided, comprising program instructions for causing an apparatus to perform at least the methods according to the fourth to sixth aspects.

[0042] In the eleventh aspect, a computer program including instructions is provided that, when executed by a device, causes the device to perform at least the methods according to the fourth to sixth aspects.

[0043] In a twelfth aspect, a terminal device is provided. The terminal device includes a receiving circuitry configured to receive configuration information from a first network device for performing an exploratory hoisting (HO), the configuration information including an experiment ID identifying a test HO configuration and a test scenario of the exploratory HO; an execution circuitry configured to perform the exploratory HO based on the configuration information to record at least one data sample of the exploratory HO, the at least one data sample including the experiment ID; and a reporting circuitry configured to report at least one data sample.

[0044] In a thirteenth aspect, a first network device is provided. The first network device includes a receiving circuitry configured to receive first configuration information for performing an exploratory hoisting (HO) from a second network device, the first configuration information including an experiment ID identifying a test HO configuration and a test scenario of the exploratory HO; and a transmitting circuitry configured to transmit second configuration information for performing the exploratory HO to a terminal device, wherein the second configuration information includes the experiment ID.

[0045] In a fourteenth aspect, a second network device is provided. The second network device includes a transmitting circuitry configured to transmit first configuration information to a first network device for performing an exploratory hoisting (HO), the first configuration information including an experiment ID identifying a test HO configuration and a test scenario of the exploratory HO; a receiving circuitry configured to receive at least one data sample recorded for the exploratory HO, wherein the at least one data sample includes the experiment ID; and an association circuitry configured to associate the at least one data sample with the exploratory HO based on the experiment ID.

[0046] It should be understood that the summary portion is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0047] Some exemplary embodiments will now be described with reference to the accompanying drawings, in which:

[0048] Figure 1 An example communication network in which embodiments of the present disclosure may be implemented is illustrated;

[0049] Figure 2 The diagram illustrates a signaling diagram that shows an example of a data collection process for exploratory HO according to some embodiments of the present disclosure;

[0050] Figure 3 The diagram illustrates a signaling diagram that shows an example of a data collection process for exploratory HO according to some embodiments of the present disclosure;

[0051] Figure 4 The diagram illustrates a signaling diagram that shows an example of a data collection process for exploratory HO according to some embodiments of the present disclosure;

[0052] Figure 5 The diagram illustrates a signaling diagram that shows an example of a data collection process for exploratory HO according to some embodiments of the present disclosure;

[0053] Figure 6 The illustration shows a flowchart of a method implemented at a terminal device according to some embodiments of the present disclosure;

[0054] Figure 7 The illustration shows a flowchart of a method implemented at a first network device according to some embodiments of the present disclosure;

[0055] Figure 8 The illustration shows a flowchart of a method implemented at a second network device according to some embodiments of the present disclosure;

[0056] Figure 9 The illustration shows a flowchart of a method implemented at a terminal device according to some embodiments of the present disclosure;

[0057] Figure 10 The illustration shows a flowchart of a method implemented at a first network device according to some embodiments of the present disclosure;

[0058] Figure 11 The illustration shows a flowchart of a method implemented at a second network device according to some embodiments of the present disclosure;

[0059] Figure 12A simplified block diagram of an apparatus suitable for implementing embodiments of the present disclosure is illustrated; and

[0060] Figure 13 A block diagram of an example computer-readable medium according to some embodiments of the present disclosure is illustrated.

[0061] Throughout the accompanying drawings, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0062] The principles of this disclosure will now be described with reference to some exemplary embodiments. It should be understood that these embodiments are described for illustrative purposes only and to assist those skilled in the art in understanding and implementing this disclosure, and do not constitute any limitation on the scope of this disclosure. The disclosure described herein can be implemented in various other ways besides those described below.

[0063] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0064] In this disclosure, references to "an embodiment," "embodiment," and "example embodiment," etc., indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment must include that particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Moreover, when a particular feature, structure, or characteristic is described in connection with an embodiment, those skilled in the art will understand that, whether explicitly described or not, combining it with other embodiments to affect such a feature, structure, or characteristic is within the knowledge of those skilled in the art.

[0065] It should be understood that although the terms “first” and “second”, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of the exemplary embodiments, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0066] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments. The singular forms “a,” “an,” and “the” used herein also include the plural forms unless the context clearly indicates otherwise. Further understanding, the terms “comprises,” “comprising,” “has,” “having,” “includes,” and / or “including”, when used herein, specify the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof. As used herein, “at least one of the following: ” and “<at least one of a list of two or more elements>” and similar wording (where a list of two or more elements is connected by “and” or “or”) means at least any one of these elements, or at least any two or more of these elements, or at least all of these elements.

[0067] As used in this application, the term "circuit system" may refer to one or more or all of the following: (a) Pure hardware circuit implementation (such as implementation using only analog and / or digital circuit systems), and (b) A combination of hardware circuitry and software, such as (if applicable): (i) A combination of (multiple) analog and / or digital hardware circuits and software / firmware, and (ii) Any part of a hardware processor (including multiple digital signal processors), software, and memory (multiple processors) having software, which work together to enable a device (such as a mobile phone or server) to perform various functions, and (c) (Multiple) hardware circuits and / or (multiple) processors, such as (multiple) microprocessors or a portion thereof, which require software (e.g., firmware) to operate, but may be absent when operation is not required.

[0068] The definition of "circuit system" applies to all uses of the term in this application, including in any claim. As another example, as used in this application, the term "circuit system" also covers only hardware circuitry or a processor (or processors) or a portion of hardware circuitry or a processor and its accompanying software and / or firmware. For example, if applicable to a particular claim element, the term "circuit system" also covers baseband integrated circuits or processor integrated circuits for mobile devices, or similar integrated circuits in servers, cellular network devices, or other computing or network devices.

[0069] As used herein, the term "communication network" refers to a network that conforms to any suitable communication standard, such as Long Term Evolution (LTE), LTE-A Advanced (LTE-A), Wideband Code Division Multiplexing (WCDMA), High-Speed ​​Packet Access (HSPA), Narrowband Internet of Things (NB-IoT), etc. Furthermore, communication between terminal devices and network devices in a communication network can be performed according to any suitable generation of communication protocol, including but not limited to first-generation (1G), second-generation (2G), 2.5G, 2.75G, third-generation (3G), fourth-generation (4G), 4.5G, future fifth-generation (5G) and sixth-generation (6G) communication protocols and / or any other protocols currently known or to be developed in the future. Embodiments of this disclosure can be applied to various communication systems. Given the rapid development of communications, there will naturally be future types of communication technologies and systems that can be used to embody the present disclosure. This should not be construed as limiting the scope of this disclosure to the systems described above.

[0070] As used herein, the terms "first network device" and "access network device" refer to nodes in a communication network through which terminal devices access the network and receive services. Network devices can refer to base stations (BS) or access points (APs), such as Node B (NodeB or NB), evolved Node B (eNodeB or eNB), NR NB (also known as gNB), Transmitter Receiver Point (TRP), Remote Radio Unit (RRU), Radio Header (RH), Remote Radio Header (RRH), relay, low-power nodes (such as femtoseconds, picoseconds), etc., depending on the terminology and technology used.

[0071] The term "terminal device" refers to any terminal device capable of wireless communication. As an example and not a limitation, a terminal device may also be referred to as a communication device, user equipment (UE), subscriber station (SS), portable subscriber station, mobile station (MS), or access terminal (AT). Terminal devices can include, but are not limited to, mobile phones, cellular phones, smartphones, Voice over IP (VoIP) phones, wireless local loop phones, tablets, wearable terminal devices, personal digital assistants (PDAs), portable computers, desktop computers, image capture terminal devices (such as digital cameras), gaming terminal devices, music storage and playback devices, in-vehicle wireless terminal devices, wireless endpoints, mobile stations, laptop embedded devices (LEE), laptop mounted devices (LME), USB dongles, smart devices, wireless customer premises equipment (CPE), Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain environments), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. In the following description, the terms "terminal equipment", "communication equipment", "terminal", "user equipment" and "UE" are used interchangeably.

[0072] Figure 1 An example communication network in which embodiments of the present disclosure can be implemented is illustrated.

[0073] like Figure 1 As shown, the communication network 100 may include a terminal device 110, a first network device 120, a core network device 130, and a second network device 140. The first network device 120 may include access network equipment. The terminal device 110 is located within the coverage area 150 served by the first network device 120. The terminal device 110 can perform uplink transmissions with the first network device 120 via distributed units (DUs) and centralized units (CUs). The first network device 120 can be connected to the core network device 130. The terminal device 110 can be directly connected to the core network device 130.

[0074] The second network device 140 can be an NMS or a data collection database. The NMS can be a management domain (NF). The second network device 140 can communicate with the core network device 130. Furthermore, the second network device 140 can also communicate with the first network device 120 via the core network device 130, or directly with the first network device 120. The terminal device 110 can also send messages to the second network device 140.

[0075] It should be understood that the number of terminal devices and network devices is for illustrative purposes only and does not imply any limitation. The communication network 100 may include any suitable number of terminal devices and first network devices suitable for implementing embodiments of this disclosure, including access network devices, core network devices, and second network devices. Furthermore, the terminal devices, first network devices, and second network devices may use different radio access technologies (RATs), including but not limited to the previously mentioned 5G and 6G.

[0076] Communication in communication network 100 can be implemented according to any suitable communication protocol(s), including but not limited to cellular communication protocols such as first-generation (1G), second-generation (2G), third-generation (3G), fourth-generation (4G), fifth-generation (5G), and sixth-generation (6G), wireless local area network communication protocols such as IEEE 802.11, and / or any other protocols currently known or to be developed in the future. Furthermore, such communication can utilize any suitable wireless communication technology, including but not limited to: Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Frequency Division Duplex (FDD), Time Division Duplex (TDD), Multiple-Input Multiple-Output (MIMO), Orthogonal Frequency Division Multiplexing (OFDM), Discrete Fourier Transform Spread Spectrum OFDM (DFT-s-OFDM), and / or any other technologies currently known or to be developed in the future.

[0077] Before introducing this invention, we can first introduce the technical background of exploratory HO.

[0078] In communication networks, exploratory handover (HO) is a dynamic handover mechanism based on real-time network state assessment, designed to optimize the connection quality of user equipment (UE). Unlike traditional fixed-threshold handover, exploratory HO dynamically determines whether to perform a handover by actively detecting and evaluating parameters such as the signal quality and load of potential target base stations, thereby improving network performance (such as throughput, latency, and reliability) and user experience.

[0079] Exploratory HOs can be performed to test different mobility procedures or configurations for data collection and exploration. For example, it can be used with a UE in RRC_IDLE state that has no active data radio bearer (DRB) or data service.

[0080] Experiments on exploratory HO-related data and results can be collected using a 6G air interface data collection framework.

[0081] Therefore, to address the challenge of accurately exploring the impact of different mobility configurations and other optimizations without disruption, the concept of exploratory or test switching has been proposed. In most cases, the terms exploratory or test switching are used interchangeably.

[0082] Now back Figure 1 In exploratory handovers, a first network device 120, including access network equipment, can configure a test configuration for terminal device 110 to test whether the handover is successful or fails. In the latter case of a failed handover, the first network device 120 may also be interested in the cause of the failure (e.g., too early or too late). For example, in the case of Mobility Robustness Optimization (MRO) style optimization, the first network device 120 can configure terminal device 110 to test different Cell Individual Offset (CIO) values ​​and observe the results. Typically, in MROs, conclusions cannot be drawn from a single handover, but rather from statistical data collected from several handovers.

[0083] Furthermore, terminal device 110 can also be a solution for indicating the configuration it wants to test, which may need to be configured by first network device 120. For example, first network device 120 may need to configure the necessary measurements.

[0084] Given the above, several issues may arise, and one or more of these issues need to be addressed in some embodiments of this disclosure. A first issue may be how the first network device 120 and the second network device 140 are connected or associated with the test configuration or scenario and aggregated into a metric. Depending on the data collection solution, the terminal device 110 may initially record data and only report it to the different first network device 120 configured for exploratory switching much later. This may occur, for example, when collecting data using offline MDT.

[0085] The second issue could be that the results of exploratory handovers cannot be included in the mobility metrics of the operating networks (e.g., the first network device 120 and the second network device 140). The third issue could be how the terminal device 110 indicates the necessary configuration provided by the first network device 120 for the tested scenario.

[0086] In light of the foregoing discussion and analysis, some exemplary embodiments of this disclosure provide methods and apparatus related to exploratory hoisting (HO). In these embodiments, a terminal device (or UE) may receive configuration information for performing exploratory HO from a first network device. The configuration information includes an experiment ID identifying the HO configuration under test and the test scenario. The terminal device may perform exploratory HO to record at least one data sample of the exploratory HO. Furthermore, the terminal device may also report at least one data sample. A second network device may receive at least one sample recorded for the exploratory HO and then associate the at least one data sample with the exploratory HO based on the experiment ID. The experiment ID may be used to link or associate the collected data with the HO configuration under test.

[0087] Some embodiments may also provide methods and apparatus for implementing exploratory HO using terminal devices, a first network device, and a second network device.

[0088] The following will refer to Figures 1-11 The principles and implementation of exemplary embodiments of this disclosure are described in detail.

[0089] Figure 2 The illustration shows an example signaling diagram of a process 200 for exploratory HO data collection according to some embodiments of the present disclosure. Reference will be made to this diagram for discussion purposes. Figure 1 Describe process 200. Process 200 may involve, for example, Figure 1 The terminal device 110, the first network device 120, and the second network device 140 are shown.

[0090] In process 200, at point 210, the second network device 140 may send first configuration information to the first network device 120 for performing an exploratory home visit (HO). The first configuration information includes the configuration of the HO under test that identifies the exploratory HO and the experiment ID of the test scenario.

[0091] In some embodiments, the experiment ID includes the ID of the NF. The NF is responsible for exploratory HO. In some embodiments, the ID of the NF includes an FQDN or a numeric ID.

[0092] Alternatively or additionally, the Experiment ID includes the ID of the exploratory HO assigned by NF. Alternatively or additionally, the Experiment ID includes the ID of the execution of the exploratory HO, used to identify each instance of the execution for a given HO configuration. Alternatively or additionally, the Experiment ID includes a test set ID, used to group exploratory HOs and their executions.

[0093] For example, an experiment ID can consist of one or more of the following components. The first component can be the ID of the NF responsible for the experiment. This can include an FQDN, or alternatively, its numeric ID. The second component can be the experiment ID assigned by the NF. The third component can be the experiment execution ID to identify individual exploratory switches tested using the same configuration. The fourth component can be an additional test set ID to group experiments and experiment executions and collect results for data collection.

[0094] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS. In some embodiments, at least one data sample may be recorded using OTT data collection. In some embodiments, at least one data sample may include recorded MDT data.

[0095] At 220, the first network device 120 can send second configuration information for performing an exploratory hoisting (HO). The second configuration information may also include an experiment ID. At 230, the terminal device 110 can perform the exploratory HO based on the second configuration information to record at least one data sample of the exploratory HO. At 240, the terminal device 110 can report at least one data sample. At 250, the second network device 140 can receive at least one data sample recorded for the exploratory HO.

[0096] In some embodiments, terminal device 110 may report at least one data sample to first network device 120. Then, first network device 120 may determine the target device for sending the at least one data sample based on the experiment ID.

[0097] The first network device 120 can determine that the target device is the second network device 140. The first network device 120 can send at least one data sample to the second network device 140.

[0098] In some embodiments, terminal device 110 may report at least one data sample to core network device 130. Core network device 130 may then send at least one data sample to second network device 140.

[0099] In some embodiments, the first network device 120 is a first access network device. The terminal device 110 can communicate with a second access network device instead of... Figure 1 The first access network device (not shown) reports at least one data sample. The second network device may send at least one data sample to the second network device 140.

[0100] At 260, the second network device 140 can associate at least one data sample with an exploratory HO based on the experiment ID.

[0101] In some embodiments, the second network device 140 may be an NMS. The second network device 140 may calculate at least one metric of the exploratory HO.

[0102] In some embodiments, the second network device 140 may calculate at least one metric for another HO, rather than an exploratory HO, which does not carry an experimental ID.

[0103] When calculating network mobility performance metrics and KPIs, the second network device 140 can use an experiment ID to distinguish between exploratory handovers and "normal" handovers. This can be implemented as a special experiment ID for a regular homepage (HO).

[0104] For example, the second network device 140 can aggregate results based on at least one experiment ID and link them to the configuration under test. The second network device 140 can calculate the required experiment metrics.

[0105] In addition, it will be combined Figure 4 Further details will be discussed. Exploratory home visits (HOs) can be initiated by terminal device 110. More details will be provided in conjunction with... Figure 3 and Figure 5 Let's have a discussion.

[0106] Figure 3 The illustration shows an example signaling diagram of a data collection process 300 for exploratory holisting according to some embodiments of the present disclosure. Reference will be made to this diagram for discussion purposes. Figure 1 Describe process 300. Process 300 may involve, for example, Figure 1 The terminal device 110, the first network device 120, and the second network device 140 are shown.

[0107] In process 300, at point 310, terminal device 110 may send a request message to first network device 120 to initiate an exploratory home visit (HO). The request message includes the test HO configuration identifying the exploratory HO and the experiment ID of the test scenario.

[0108] In some embodiments, the experiment ID includes the ID of the NF. The NF is responsible for exploratory HO. In some embodiments, the ID of the NF includes an FQDN or a numeric ID.

[0109] Alternatively or additionally, the Experiment ID includes the ID of the exploratory HO assigned by NF. Alternatively or additionally, the Experiment ID includes the ID of the execution of the exploratory HO, used to identify each instance of the execution for a given HO configuration. Alternatively or additionally, the Experiment ID includes a test set ID, used to group exploratory HOs and their executions.

[0110] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0111] In some embodiments, at least one data sample may be recorded using OTT data collection.

[0112] In some embodiments, at least one data sample may include recorded MDT data.

[0113] In some embodiments, the first network device 120 may send a retrieval request message to the experimental database. The first network device 120 can obtain configuration information from the experimental database. The configuration information is retrieved from the experimental database based on the experimental ID. The experimental database is a repository of different exploratory hoisting experiments that may need to be performed and their related configurations. The experimental database may be hosted in a RAN domain, core domain, or management domain.

[0114] At 320, the first network device 120 can send configuration information for performing an exploratory HO.

[0115] In some embodiments, the configuration information may include an experiment ID.

[0116] At 330, terminal device 110 can perform exploratory HO based on the second configuration information to record at least one data sample of the exploratory HO.

[0117] At point 340, terminal device 110 may report at least one data sample to second network device 140. Second network device 140 may be a data collection database.

[0118] At 350, the second network device 140 can associate at least one data sample with an exploratory HO based on the experiment ID.

[0119] In some embodiments, the second network device 140 may be an NMS. The second network device 140 may calculate at least one metric of the exploratory HO.

[0120] For example, the second network device 140 can aggregate results based on at least one experiment ID and link them to the configuration under test. The second network device 140 can calculate the required experiment metrics.

[0121] In some embodiments, the second network device 140 may calculate at least one metric for another HO, rather than an exploratory HO, which does not carry an experimental ID.

[0122] When calculating network mobility performance metrics and KPIs, the second network device 140 can use an experiment ID to distinguish between exploratory handovers and "normal" handovers. This can be implemented as a special experiment ID for a regular homepage (HO).

[0123] In some embodiments, the request message includes a Radio Resource Control (RRC) establishment message. Additionally, configuration information is included in the RRC reconfiguration message.

[0124] In this way, the experiment ID is configured to the terminal device by a first network device (e.g., an access network device) and a second network device (e.g., a data collection database or NMS). The terminal device can include the experiment ID in all collected data samples to record the exploratory HO "test cases" and their corresponding results regarding the results of the exploratory HO.

[0125] This disclosure enhances the data collection process because it allows test switch recording data and results to be mapped to the configuration under test. Enabling offline data collection for exploratory HO recordings can significantly reduce collection overhead.

[0126] Figure 4 The illustration shows an example signaling diagram of a data collection process 400 for exploratory holisting according to some embodiments of the present disclosure. Reference will be made to this diagram for discussion purposes. Figure 1 Describe procedure 400. Procedure 400 may involve UE 401, gNB-1402, gNB, or core network equipment 403 and NMS 404.

[0127] like Figure 1 As shown, UE 401 can be terminal device 110, gNB-1 402 can be first network device 120, core network device 403 can be core network device 130, and NMS 404 can be second network device 140.

[0128] exist Figure 4 The image shows an example of configuring NMS for an exploratory HO experiment and collecting results.

[0129] At 410, NMS 404 can send messages related to the configuration of exploratory HO experiments to gNB-1 402.

[0130] NMS 404 or its features (such as MRO) can configure mobility settings for testing on the gNB-1 402. This can include testing different CIO values. Furthermore, NMS or MRO can associate each configuration under test and the scenarios in which it can be tested with an experiment ID.

[0131] At 420, gNB-1 402 can send information related to the configuration of the exploratory HO to UE 401 via an RRC reconfiguration message.

[0132] gNB-1 402 can first select a suitable UE. gNB-1 402 can determine that UE 401 is a suitable UE for performing exploratory HO testing. gNB-1 402 can configure the exploratory HO for UE 401 in the RRC reconfiguration message. The RRC reconfiguration message can include an experiment ID. The experiment ID can identify the handover configuration and scenario under test, and can be used to link the collected data to the configuration under test.

[0133] An experiment ID may consist of one or more of the following components: (1) the ID of the NF responsible for the experiment. This ID may include the NF's FQDN or alternative numeric IDs to reduce overhead. (2) the experiment ID assigned to it by the NF. (3) the experiment execution ID. This ID may be used to identify individual exploratory switches tested using the same configuration. (4) An additional test set ID may be provided to group experiments and experiment executions. For example, this test set ID may be used to group results during data collection.

[0134] In some embodiments, the Experiment ID may be different from the Tracking ID (MDT Tracking ID). The Experiment ID can specify the configuration being tested and the responsible NF. Furthermore, the Tracking ID cannot be used for UE-initiated scenarios, which will... Figure 5 The discussion is ongoing.

[0135] At 430, UE 401 can perform exploratory HO. Furthermore, UE 401 can record the results and any other data configured to be recorded.

[0136] UE 401 can include an exploratory ID in each record sample. This can be done, for example, using a record MDT or other data collection methods including OTT.

[0137] At 440, UE 401 can report data collection (exploratory HO recorded datasets or data samples or recorded measurements) to core network device 403.

[0138] Alternatively, UE 401 may report data collection to gNB-1 402 or other access network equipment (e.g., another gNB 403).

[0139] UE 401 may not report data collection to gNB-1 402 configured with exploratory HO. UE 401 may report data collection to another gNB 403 or core network 403 (as in the case of recording MDT).

[0140] In some embodiments, the receiving network device (e.g., gNB-1 402, another gNB 403, or core network device 403) can determine where to report the collected data. This determination can be based on the experiment ID.

[0141] In some embodiments, the determination may also use the ID of the network function responsible for the experiment and its data collection.

[0142] Back Figure 4 At 450, another gNB 403 or core network device 403 can report data to NMS 404.

[0143] At 460, NMS 404 can link or associate the experiment IDs reported in the collected data samples with the experiments stored in its experiment database to calculate the required metrics. NMS 404 can also link or associate metrics and test configurations with the corresponding experiments or tests.

[0144] This disclosure proposes an experiment ID, which is configured by the network device to the UE in the RRC reconfiguration message configuring exploratory handover. It is anticipated that the UE will include the experiment ID in all data samples recording exploratory handover and its results. Using the experiment ID, the gNB or NMS can be able to link or associate recorded data samples and results with the configuration or scenario of each exploratory handover experiment. The NMS can aggregate and compute the metrics required for exploratory handover experiments.

[0145] The absence of an experiment ID or special value can be used to indicate that the collected data is related to "actual operational handover" rather than exploratory handover. Experiment IDs can be included in normal network performance metrics and distinguish between exploratory and normal handover when calculating network mobility performance metrics and key performance metrics (KPIs).

[0146] In some other embodiments, similarly, when initiating an exploratory handover RRC connection, the UE can provide an experiment ID in the RRC establishment request message to indicate and mention potential cache exploratory handover test cases.

[0147] Figure 5 The illustration shows an example signaling diagram of a data collection process 500 for exploratory holisting according to some embodiments of the present disclosure. Reference will be made to this diagram for discussion purposes. Figure 1 Describe process 500. Process 500 may involve UE 501, gNB 502, experimental database 503, and data collection database 504.

[0148] UE 501 can be as follows Figure 1 The terminal device 110 shown, gNB 502 can be as follows: Figure 1 The first network device 120 shown, and the data collection database 504 can be as follows: Figure 1 The second network device 140 shown.

[0149] exist Figure 5 The example shown is an exploratory handover initiated by a UE. More specifically, it illustrates how to utilize the experiment ID in a UE-initiated exploratory handover.

[0150] At point 510, UE 501 can send an RRC establishment request message to gNB 502 to initiate an exploratory handover. This message may include the experiment ID.

[0151] At 520, gNB 502 can use the experiment ID to retrieve the required exploratory hoisting configuration from the experiment database 503. The experiment database is a repository of different exploratory hoisting experiments that may need to be performed and their associated configurations. The experiment database can be hosted in the RAN domain, core domain, or management domain.

[0152] At 530, gNB 502 can receive exploratory HO configurations from experimental database 503.

[0153] At 540, gNB 502 can send an RRC reconfiguration message to UE 501. gNB 502 can configure an exploratory HO for the UE via the RRC reconfiguration message using the retrieved experimental configuration. The RRC reconfiguration message may optionally include the experimental ID, although this can be known to the UE, as it is provided in 510.

[0154] At 550, UE 501 can perform exploratory HO. Furthermore, UE 501 can record results and any other data configured to be recorded.

[0155] UE 501 can include an exploratory ID in each record sample. This can be done, for example, using a record MDT or other data collection methods including OTT.

[0156] At 560, UE 501 can report data collection (recorded datasets or data samples) to data collection database 504.

[0157] At point 570, data collection database 504 can aggregate results and compute metrics for exploratory holisties (HOs). Data collection database 504 can also link results and metrics to the tested configuration based on the experiment ID.

[0158] This approach allows for the separation of exploratory switchover and exploratory headhunting (HO) data collection, while still enabling the mapping of test switchover log data and results to the configuration under test. Data collection for exploratory HOs can utilize recorded offline data collection, significantly reducing collection overhead. Otherwise, the required data would need to be immediately signaled to the target node after the exploratory switchover and further forwarded to the source node via the Xn interface.

[0159] Figure 6 A flowchart illustrating a method 600 implemented at a terminal device according to some embodiments of the present disclosure is shown. For discussion purposes, reference will be made to... Figure 1 Method 600 is described from the perspective of terminal device 110.

[0160] At point 610, terminal device 110 can first receive configuration information for performing exploratory HO from the first network device. The configuration information includes the test HO configuration that identifies the exploratory HO and the experiment ID of the test scenario.

[0161] At 620, terminal device 110 can perform exploratory HO based on configuration information to record at least one data sample of the exploratory HO, the at least one data sample including an experiment ID.

[0162] At 630, terminal device 110 can report at least one data sample.

[0163] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0164] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0165] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0166] In some embodiments, at least one data sample is recorded using OTT data collection.

[0167] In some embodiments, at least one data sample includes recorded MDT data.

[0168] In some embodiments, the terminal device may report at least one data sample by reporting at least one data sample to the core network device.

[0169] In some embodiments, the terminal device may report at least one data sample by reporting at least one data sample to a first network device.

[0170] In some embodiments, the first network device is a first access network device. The terminal device may report at least one data sample by reporting at least one data sample to a second access network device instead of the first access network device.

[0171] In some embodiments, an apparatus capable of performing any of the methods 600 (e.g., terminal device 110) may include components for performing the corresponding steps of method 600. These components may be implemented in any suitable form. For example, the components may be implemented in a circuit system or a software module.

[0172] In some embodiments, the apparatus includes components for receiving configuration information from a first network device for performing an exploratory hoc, the configuration information including an experiment ID identifying the test hoc configuration and the test scenario of the exploratory hoc; components for performing the exploratory hoc based on the configuration information to record at least one data sample of the exploratory hoc, the at least one data sample including the experiment ID; and components for reporting at least one data sample.

[0173] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0174] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0175] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0176] In some embodiments, at least one data sample is recorded using OTT data collection.

[0177] In some embodiments, at least one data sample includes recorded MDT data.

[0178] In some embodiments, the component is used to report at least one data sample by reporting at least one data sample to the core network equipment.

[0179] In some embodiments, the component is used to report at least one data sample by reporting the data sample to a first network device.

[0180] In some embodiments, the first network device is a first access network device. This component is used to report at least one data sample by reporting at least one data sample to a second access network device instead of the first access network device.

[0181] In some embodiments, the apparatus further includes components for performing other steps in some embodiments of method 600. In some embodiments, the components include at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured together with the at least one processor to cause the apparatus to execute.

[0182] Figure 7 A flowchart illustrating a method 700 implemented at a first network device according to some embodiments of the present disclosure is shown. For discussion purposes, reference will be made to... Figure 1 Method 700 is described from the perspective of the first network device 120.

[0183] At box 710, the first network device 120 can receive first configuration information for performing an exploratory HO from the second network device. The first configuration information includes the test HO configuration that identifies the exploratory HO and the experiment ID of the test scenario.

[0184] At box 720, the first network device 120 can send second configuration information for performing an exploratory HO to the terminal device, the second configuration information including the experiment ID.

[0185] In some embodiments, the first network device may receive at least one data sample of an exploratory HO from a terminal device, the at least one data sample including an experiment ID.

[0186] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0187] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0188] In some embodiments, the first network device may also determine the target device for sending at least one data sample based on the experiment ID.

[0189] In some embodiments, the first network device may send at least one data sample to the second network device.

[0190] In some embodiments, the second network device includes NMS or functions within NMS.

[0191] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0192] In some embodiments, at least one data sample is recorded using OTT data collection.

[0193] In some embodiments, at least one data sample includes recorded MDT data.

[0194] In some embodiments, an apparatus capable of performing any of the methods 700 (e.g., the first network device 120) may include components for performing the corresponding steps of method 700. These components may be implemented in any suitable form. For example, the components may be implemented in a circuit system or a software module.

[0195] In some embodiments, the apparatus includes components for receiving first configuration information for performing an exploratory home visit (HO) from a second network device, the first configuration information including an experiment ID identifying the HO configuration under test and the test scenario; and components for sending second configuration information for performing the exploratory HO to a terminal device, wherein the second configuration information includes the experiment ID.

[0196] In some embodiments, the apparatus may further include a component for receiving at least one data sample from an exploratory HO from a terminal device, wherein the at least one data sample includes an experiment ID.

[0197] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0198] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0199] In some embodiments, the apparatus may further include a component for determining a target device for sending at least one data sample based on the experiment ID.

[0200] In some embodiments, the apparatus may further include a component for transmitting at least one data sample to a second network device.

[0201] In some embodiments, the second network device includes NMS or functions within NMS.

[0202] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0203] In some embodiments, at least one data sample is recorded using OTT data collection.

[0204] In some embodiments, at least one data sample includes recorded MDT data.

[0205] In some embodiments, the apparatus further includes components for performing other steps in some embodiments of method 700. In some embodiments, the components include at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured together with the at least one processor to cause the apparatus to execute.

[0206] Figure 8 A flowchart illustrating a method 800 implemented at a second network device according to some embodiments of the present disclosure is shown. For discussion purposes, reference will be made to... Figure 1 Method 800 is described from the perspective of the second network device 140.

[0207] At box 810, the second network device 140 can send first configuration information to the first network device for performing an exploratory HO. The first configuration information includes the test HO configuration that identifies the exploratory HO and the experiment ID of the test scenario.

[0208] At box 820, the second network device 140 can receive at least one data sample recorded for an exploratory HO, the at least one data sample including an experiment ID.

[0209] At box 830, the second network device 140 can associate at least one data sample with an exploratory HO based on the experiment ID.

[0210] In some embodiments, the second network device 140 may calculate at least one metric of the exploratory HO.

[0211] In some embodiments, the second network device may compute at least one metric of another HO, rather than an exploratory HO, which does not carry an experimental ID.

[0212] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0213] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0214] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0215] In some embodiments, at least one data sample is recorded using OTT data collection.

[0216] In some embodiments, at least one data sample includes recorded MDT data.

[0217] In some embodiments, the second network device may receive at least one data sample by receiving at least one data sample from the core network device.

[0218] In some embodiments, the second network device may receive at least one data sample by receiving at least one data sample from the first network device.

[0219] In some embodiments, the second network device includes NMS or functions within NMS.

[0220] In some embodiments, an apparatus capable of performing any of the methods 800 (e.g., the second network device 140) may include components for performing the corresponding steps of method 800. These components may be implemented in any suitable form. For example, the components may be implemented in a circuit system or a software module.

[0221] In some embodiments, the apparatus includes components for sending first configuration information for performing an exploratory hoisting to a first network device, the first configuration information including an experiment ID identifying a test hoisting configuration and a test scenario for the exploratory hoisting; components for receiving at least one data sample recorded for the exploratory hoisting, the at least one data sample including the experiment ID; and components for associating the at least one data sample with the exploratory hoisting based on the experiment ID.

[0222] In some embodiments, the apparatus includes a component for calculating at least one metric of exploratory HO.

[0223] In some embodiments, the device includes components for calculating at least one metric for another HO, rather than an exploratory HO, which does not carry an experimental ID.

[0224] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0225] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0226] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0227] In some embodiments, at least one data sample is recorded using OTT data collection.

[0228] In some embodiments, at least one data sample includes recorded MDT data.

[0229] In some embodiments, the component for receiving at least one data sample may include a component for receiving at least one data sample from a core network device.

[0230] In some embodiments, the component for receiving at least one data sample may include a component for receiving at least one data sample from a first network device.

[0231] In some embodiments, the second network device includes NMS or functions within NMS.

[0232] In some embodiments, the apparatus further includes components for performing additional steps in some embodiments of method 800. In some embodiments, the components include at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured, together with the at least one processor, to cause the apparatus to execute.

[0233] Figure 9 A flowchart illustrating a method 900 implemented at a terminal device according to some embodiments of the present disclosure is shown. For discussion purposes, reference will be made to... Figure 1 Method 900 is described from the perspective of terminal device 110.

[0234] At point 910, terminal device 110 can send a request message to the first network device for initiating an exploratory HO. The request message includes the test HO configuration that identifies the exploratory HO and the experiment ID of the test scenario.

[0235] At 920, terminal device 110 can receive configuration information from the first network device for performing exploratory HO.

[0236] At 930, terminal device 110 can perform exploratory HO based on configuration information to record at least one data sample of the exploratory HO, the at least one data sample including an experiment ID.

[0237] At 940, terminal device 110 can report at least one data sample.

[0238] In some embodiments, the configuration information includes the experiment ID.

[0239] In some embodiments, the terminal device may report at least one data sample by reporting at least one data sample to a second network device.

[0240] In some embodiments, the second network device includes a data collection database.

[0241] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0242] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0243] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0244] In some embodiments, at least one data sample is recorded using OTT data collection.

[0245] In some embodiments, at least one data sample includes recorded MDT data.

[0246] In some embodiments, the request message includes a Radio Resource Control (RRC) establishment message.

[0247] In some embodiments, configuration information is included in the RRC reconfiguration message.

[0248] In some embodiments, an apparatus capable of performing any of the methods 900 (e.g., terminal device 110) may include components for performing the corresponding steps of method 900. These components may be implemented in any suitable form. For example, the components may be implemented in a circuit system or a software module.

[0249] In some embodiments, the apparatus includes components for sending a request message to a first network device for initiating an exploratory hoc, the request message including an experiment ID identifying the test hoc configuration and the test scenario of the exploratory hoc; components for receiving configuration information from the first network device for performing the exploratory hoc; components for performing the exploratory hoc based on the configuration information to record at least one data sample of the exploratory hoc, the at least one data sample including the experiment ID; and components for reporting at least one data sample.

[0250] In some embodiments, the configuration information includes the experiment ID.

[0251] In some embodiments, the component is used to report at least one data sample by reporting at least one data sample to a second network device.

[0252] In some embodiments, the second network device includes a data collection database.

[0253] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0254] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0255] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0256] In some embodiments, at least one data sample is recorded using OTT data collection.

[0257] In some embodiments, at least one data sample includes recorded MDT data.

[0258] In some embodiments, the request message includes a Radio Resource Control (RRC) establishment message.

[0259] In some embodiments, configuration information is included in the RRC reconfiguration message.

[0260] In some embodiments, the apparatus further includes components for performing other steps in some embodiments of method 900. In some embodiments, the components include at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured together with the at least one processor to cause the apparatus to execute.

[0261] Figure 10 A flowchart illustrating a method 1000 implemented at a first network device according to some embodiments of the present disclosure is shown. For discussion purposes, reference will be made to... Figure 1 Method 1000 is described from the perspective of the first network device 120.

[0262] At box 1010, the first network device 120 can receive a request message from the terminal device for initiating an exploratory HO. The request message includes the test HO configuration that identifies the exploratory HO and the experiment ID of the test scenario.

[0263] At box 1020, the first network device 120 can send configuration information to the terminal device for performing an exploratory HO.

[0264] In some embodiments, the first network device may send a retrieval request message to the experimental database; and obtain configuration information from the experimental database, retrieving the configuration information from the experimental database based on the experimental ID.

[0265] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0266] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0267] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0268] In some embodiments, the request message includes a Radio Resource Control (RRC) establishment message.

[0269] In some embodiments, configuration information is included in the RRC reconfiguration message.

[0270] In some embodiments, an apparatus capable of performing any of the methods 1000 (e.g., the first network device 120) may include components for performing the corresponding steps of method 1000. These components may be implemented in any suitable form. For example, the components may be implemented in a circuit system or a software module.

[0271] In some embodiments, the apparatus includes components for receiving a request message from a terminal device for initiating an exploratory home visit (HO), the request message including an experiment ID identifying the HO configuration to be tested and the test scenario; and components for sending configuration information to the terminal device for performing the exploratory HO.

[0272] In some embodiments, the apparatus may further include a component for sending a retrieval request message to an experimental database; and a component for retrieving configuration information from the experimental database based on the experimental ID.

[0273] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0274] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0275] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0276] In some embodiments, the request message includes a Radio Resource Control (RRC) establishment message.

[0277] In some embodiments, configuration information is included in the RRC reconfiguration message.

[0278] In some embodiments, the apparatus further includes components for performing other steps in some embodiments of method 1000. In some embodiments, the components include at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured together with the at least one processor to cause the apparatus to execute.

[0279] Figure 11 A flowchart illustrating a method 1100 implemented at a second network device according to some embodiments of the present disclosure is shown. For discussion purposes, reference will be made to... Figure 1 Method 800 is described from the perspective of the second network device 140.

[0280] At box 1110, the second network device 140 can receive at least one data sample recorded for an exploratory HO from the terminal device, the at least one data sample including the test HO configuration identifying the exploratory HO and the experiment ID of the test scenario.

[0281] At box 1120, the second network device 140 can associate at least one data sample with an exploratory HO based on the experiment ID.

[0282] In some embodiments, the second network device may calculate at least one metric of the exploratory HO.

[0283] In some embodiments, the second network device may compute at least one metric of another HO, rather than an exploratory HO, which does not carry an experimental ID.

[0284] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0285] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0286] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0287] In some embodiments, at least one data sample is recorded using OTT data collection.

[0288] In some embodiments, at least one data sample includes recorded MDT data.

[0289] In some embodiments, the second network device includes a data collection database.

[0290] In some embodiments, an apparatus capable of performing any of methods 1100 (e.g., the second network device 140) may include components for performing the corresponding steps of method 800. These components may be implemented in any suitable form. For example, the components may be implemented in a circuit system or a software module.

[0291] In some embodiments, the apparatus includes components for receiving from a terminal device at least one data sample recorded for an exploratory HO, the at least one data sample including an experiment ID identifying the tested HO configuration and the tested scenario of the exploratory HO; and components for associating the at least one data sample with the exploratory HO based on the experiment ID.

[0292] In some embodiments, the apparatus includes a component for calculating at least one metric of exploratory HO.

[0293] In some embodiments, the device includes components for calculating at least one metric for another HO, rather than an exploratory HO, which does not carry an experimental ID.

[0294] In some embodiments, the experiment ID includes at least one of the following: the ID of the NF, which is responsible for the exploratory HO; the ID of the exploratory HO assigned by the NF; the execution ID of the exploratory HO, used to identify each instance of the execution of a given HO configuration; and the test set ID, used to group the exploratory HO and the execution of the exploratory HO.

[0295] In some embodiments, the ID of an NF includes an FQDN or a numeric ID.

[0296] In some embodiments, the NF includes one of the following: RAN NF; core domain NF; or NMS.

[0297] In some embodiments, at least one data sample is recorded using OTT data collection.

[0298] In some embodiments, at least one data sample includes recorded MDT data.

[0299] In some embodiments, the second network device includes a data collection database.

[0300] In some embodiments, the apparatus further includes components for performing other steps in some embodiments of method 1100. In some embodiments, the components include at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured together with the at least one processor to cause the apparatus to execute.

[0301] Figure 12 This is a simplified block diagram of a device 1200 suitable for implementing embodiments of the present disclosure. The device 1200 can be provided to implement a communication device, such as... Figure 1 The terminal device 110, the first network device 120, or the second network device 140 are shown. As shown, device 1200 includes one or more processors 1210, one or more memories 1220 coupled to processor 1210, and one or more communication modules 1240 coupled to processor 1210.

[0302] The communication module 1240 is used for bidirectional communication. The communication module 1240 has at least one antenna to facilitate communication. The communication interface can represent any interface required for communication with other network elements.

[0303] Processor 1210 can be of any type suitable for a local technology network, and by way of non-limiting example, can include one or more of the following: general-purpose computer, special-purpose computer, microprocessor, digital signal processor (DSP), and processor based on a multi-core processor architecture. Device 1200 can have multiple processors, such as application-specific integrated circuit chips that are time-dependent on a clock synchronized with the main processor.

[0304] Memory 1220 may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memories include, but are not limited to, read-only memory (ROM) 1224, electrically programmable read-only memory (EPROM), flash memory, hard disk, compact disc (CD), digital video disc (DVD), and other magnetic and / or optical storage devices. Examples of volatile memories include, but are not limited to, random access memory (RAM) 1222 and other volatile memories that do not persist during power outages.

[0305] Computer program 1230 includes computer-executable instructions that are executed by the associated processor 1210. Program 1230 may be stored in ROM 1224. Processor 1210 may perform any suitable actions and processes by loading program 1230 into RAM 1222.

[0306] The embodiments of this disclosure can be implemented by program 1230, enabling device 1200 to execute reference... Figures 2 to 11Any process discussed in this disclosure. Embodiments of this disclosure may also be implemented by hardware or by a combination of software and hardware.

[0307] In some embodiments, program 1230 may be tangibly contained in a computer-readable medium, which may be included in device 1200 (such as memory 1220) or other storage device accessible to device 1200. Device 1200 may load program 1230 from the computer-readable medium into RAM 1222 for execution. The computer-readable medium may include any type of tangible non-volatile memory, such as ROM, EPROM, flash memory, hard disk, CD, DVD, etc.

[0308] Figure 13 An example of a computer-readable medium 1300 in the form of a CD or DVD is shown. A program 1230 is stored on the computer-readable medium.

[0309] Generally, the various embodiments of this disclosure can be implemented using hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects can be implemented using hardware, while others can be implemented using firmware or software that can be executed by a controller, microprocessor, or other computing device. Although various aspects of the embodiments of this disclosure are illustrated and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, apparatuses, systems, techniques, or methods described herein can be implemented using hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0310] This disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in a program module, which execute in a device on a target real or virtual processor to perform the above-referenced... Figures 2-11 The method described is 600 to 1100. Typically, a program module includes routines, programs, libraries, objects, classes, components, data structures, etc., that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of a program module can be combined or split among program modules as needed. The machine-executable instructions of a program module can be executed on a local or distributed device. In a distributed device, a program module can reside on both local and remote storage media.

[0311] Program code used to perform the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a stand-alone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0312] In the context of this disclosure, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, etc.

[0313] Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable media can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the foregoing. More specific examples of computer-readable storage media will include electrical connections having one or more wires, portable computer floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. The term "non-transient" as used herein is a limitation on the medium itself (i.e., tangible, not signaling), not a limitation on the persistence of data storage (e.g., RAM and ROM).

[0314] Furthermore, although operations are described in a specific order, this should not be construed as requiring the operations to be performed in the specific order shown or sequentially, or to perform all of the shown operations to obtain the desired result. In some cases, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure, but rather as descriptions of features that may be specific to particular embodiments. Certain features described in the context of a single embodiment may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0315] Although this disclosure has been described in language specific to structural features and / or methodological actions, it should be understood that the disclosure as defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features or actions described above are disclosed as exemplary forms of implementing the claims.

Claims

1. A terminal device, comprising: At least one processor; as well as At least one memory storing instructions, which, when executed by the at least one processor, cause the terminal device to at least: Receive configuration information from the first network device for performing an exploratory switching HO, wherein the configuration information includes the test HO configuration that identifies the exploratory HO and the experiment identifier ID of the test scenario; The exploratory HO is performed based on the configuration information to record at least one data sample of the exploratory HO, wherein the at least one data sample includes the experiment ID; as well as The report includes at least one data sample.

2. The terminal device according to claim 1, wherein the experiment ID includes at least one of the following: The ID of a network function NF, wherein the NF is responsible for the exploratory HO; The ID of the exploratory HO assigned by the NF; an ID of the execution of the exploratory HO, for identifying each instance of the execution of a given HO configuration; as well as Test set ID, used to group the exploratory HO and the execution of the exploratory HO; The ID of the NF includes a fully qualified domain name (FQDN) or a numeric ID; or The NF includes one of the following: Radio Access Network (RAN) NF; Core domain NF; or Network Management System (NMS).

3. The terminal device according to claim 1 or 2, wherein the at least one data sample is recorded using over-the-top OTT data collection; or The at least one data sample includes recorded driver test minimization MDT data; or The terminal device is configured to report the at least one data sample in the following manner: Report at least one data sample to the core network equipment; or The terminal device is configured to report the at least one data sample in the following manner: Report the at least one data sample to the first network device; or The first network device is a first access network device, and the terminal device is configured to report the at least one data sample in the following manner: The at least one data sample is reported to the second access network device instead of the first access network device.

4. A first network device, comprising: At least one processor; as well as At least one memory storing instructions, which, when executed by the at least one processor, cause the first network device to at least: Receive first configuration information from the second network device for performing an exploratory switching HO, wherein the first configuration information includes the test HO configuration identifying the exploratory HO and the experiment identifier ID of the test scenario; as well as Send second configuration information to the terminal device for performing the exploratory HO, wherein the second configuration information includes the experiment ID.

5. The first network device according to claim 4, wherein the first network device is further configured to: Receive at least one data sample from the exploratory HO from the terminal device, wherein the at least one sample includes the experiment ID.

6. The first network device according to claim 4 or 5, wherein the first network device is further configured to: The target device for sending the at least one data sample is determined based on the experiment ID; or The first network device is further configured to: Send the at least one data sample to the second network device.

7. The first network device according to claim 6, wherein the second network device includes an NMS or a function of the NMS.

8. A second network device, comprising: At least one processor; as well as At least one memory storing instructions, which, when executed by the at least one processor, cause the second network device to at least: Send first configuration information to the first network device for performing an exploratory HO switch, wherein the first configuration information includes the test HO configuration that identifies the exploratory HO and the experiment identifier ID of the test scenario; Receive at least one data sample recorded for the exploratory HO, wherein the at least one data sample includes the experiment ID; as well as The at least one data sample is associated with the exploratory HO based on the experiment ID.

9. The second network device according to claim 8, wherein the second network device is further configured to: Calculate at least one metric of the exploratory HO; or The second network device is also configured to: Calculate at least one metric of another HO, which is not the exploratory HO, and the other HO does not carry the experiment ID.

10. A method for communication, comprising: Receive configuration information from the first network device for performing an exploratory switching HO, wherein the configuration information includes the test HO configuration that identifies the exploratory HO and the experiment identifier ID of the test scenario; The exploratory HO is performed based on the configuration information to record at least one data sample of the exploratory HO, wherein the at least one data sample includes the experiment ID; as well as The report includes at least one data sample.

Images