LCM procedure in ai / ML based positioning

Abstract

Example embodiments of the present disclosure relate to LCM procedure in artificial intelligence / machine learning (AI / ML)-based positioning A first apparatus receives capability information of a second apparatus indicating respective response times for a plurality of life-cycle management (LCM) actions for AI / ML-based positioning; transmits, to the second apparatus, an indication of an LCM action of the plurality of LCM actions to be applied; starts a timer for the LCM action based on the capability information; in accordance with a determination that a notification of a successful completion of the LCM action is received from the second apparatus while the timer is running, determine that a test of the LCM action is successful; and in accordance with a determination that a notification of a successful completion of the LCM action is not received from the second apparatus while the timer is running, determines that a test of the LCM action is failed.

Description

LCM PROCEDURE IN AI / ML BASED POSITIONINGCROSS-REFERENCE TO RELATED APPLICATION

[0001] The present application claims priority to, and the benefit of, India Provisional Application No. 202441010944, filed Feb. 16, 2024, the contents of which are hereby incorporated by reference in their entirety.FIELD

[0002] Various example embodiments of the present disclosure generally relate to the field of telecommunication and in particular, to methods, devices, apparatuses and computer readable storage medium for life-cycle management (LCM) procedure in artificial intelligence / machine learning (AIZML)-based positioning.BACKGROUND

[0003] In the telecommunication industry, artificial intelligence / machine learning (AI / ML) have been employed in telecommunication systems to improve the performance. The 3rd Generation Partnership Project (3GPP) Release-18 started the study on AI / ML for New Radio (NR) air interface. The goal is to explore the benefits of augmenting the air interface with features enabling improved support of AI / ML-based algorithms for enhanced performance and / or reduced complexity / overhead. Several use cases are considered to enable the identification of a common AI / ML framework, including functional requirements of AI / ML architecture, which could be used in subsequent projects. It also aims to cover the interoperability and testability aspects of the AI / ML enabled features in the communication systems.SUMMARY

[0004] In a first aspect of the present disclosure, there is provided a first apparatus. The first apparatus comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the first apparatus at least to: receive, from a second apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; transmit, to the second apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; start a timer for the LCM action based on the capability information of the isecond apparatus; in accordance with a determination that a notification of a successful completion of the LCM action is received from the second apparatus while the timer is running, determine that a test of the LCM action is successful; and in accordance with a determination that a notification of a successful completion of the LCM action is not received from the second apparatus while the timer is running, determine that a test of the LCM action is failed.

[0005] In a second aspect of the present disclosure, there is provided a second apparatus. The second apparatus comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the second apparatus at least to: transmit, to a first apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; receive, from the first apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; upon a completion of the LCM action, transmit, to the first apparatus, a notification of a successful completion of the LCM action.

[0006] In a third aspect of the present disclosure, there is provided a method. The method comprises: receiving, by a first apparatus and from a second apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; transmitting, to the second apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; starting a timer for the LCM action based on the capability information of the second apparatus; in accordance with a determination that a notification of a successful completion of the LCM action is received from the second apparatus while the timer is running, determining that a test of the LCM action is successful; and in accordance with a determination that a notification of a successful completion of the LCM action is not received from the second apparatus while the timer is running, determining that a test of the LCM action is failed.

[0007] In a fourth aspect of the present disclosure, there is provided a method. The method comprises: transmitting, by a second apparatus and to a first apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; receiving, from the first apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; upon a completion of the LCM action, transmitting, to the first apparatus, a notification of a successful completion of the LCM action.

[0008] In a fifth aspect of the present disclosure, there is provided a first apparatus. The first apparatus comprises means for receiving, from a second apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial i ntel ligence / machi ne learning (AI / ML)- based positioning; means for transmitting, to the second apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; means for starting a timer for the LCM action based on the capability information of the second apparatus; means for, in accordance with a determination that a notification of a successful completion of the LCM action is received from the second apparatus while the timer is running, determining that a test of the LCM action is successful; and means for, in accordance with a determination that a notification of a successful completion of the LCM action is not received from the second apparatus while the timer is running, determining that a test of the LCM action is failed.

[0009] In a sixth aspect of the present disclosure, there is provided a second apparatus. The second apparatus comprises means for transmitting, to a first apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; means for receiving, from the first apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; and means for upon a completion of the LCM action, transmitting, to the first apparatus, a notification of a successful completion of the LCM action.

[0010] In a seventh aspect of the present disclosure, there is provided a computer readable medium. The computer readable medium comprises instructions stored thereon for causing an apparatus to perform at least the method according to the third aspect.

[0011] In an eighth aspect of the present disclosure, there is provided a computer readable medium. The computer readable medium comprises instructions stored thereon for causing an apparatus to perform at least the method according to the fourth aspect.

[0012] It is to be understood that the Summary section is not intended to identify key or essential features of embodiments of the present disclosure, nor is it intended to be used to limit the scope of the present disclosure. Other features of the present disclosure will become easily comprehensible through the following description.BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Some example embodiments will now be described with reference to the accompanying drawings, where:

[0014] FIG. 1 illustrates an example communication environment in which example embodiments of the present disclosure can be implemented;

[0015] FIG. 2 illustrates an example architecture of a functional framework for an artificial intelligence / machine learning (AI / ML) functionality;

[0016] FIG. 3 illustrates a signaling flow for a LCM procedure in AI / ML positioning in accordance with some example embodiments of the present disclosure;

[0017] FIG. 4 illustrates a detailed signaling flow for a LCM procedure in AI / ML positioning in accordance with some further example embodiments of the present disclosure;

[0018] FIG. 5 illustrates a flowchart of a method implemented at a first apparatus in accordance with some example embodiments of the present disclosure;

[0019] FIG. 6 illustrates a flowchart of a method implemented at a second apparatus in accordance with some example embodiments of the present disclosure;

[0020] FIG. 7 illustrates a simplified block diagram of a device that is suitable for implementing example embodiments of the present disclosure; and

[0021] FIG. 8 illustrates a block diagram of an example computer readable medium in accordance with some example embodiments of the present disclosure.

[0022] Throughout the drawings, the same or similar reference numerals represent the same or similar element.DETAILED DESCRIPTION

[0023] Principle of the present disclosure will now be described with reference to some example embodiments. It is to be understood that these embodiments are described only for the purpose of illustration and help those skilled in the art to understand and implement the present disclosure, without suggesting any limitation as to the scope of the disclosure. Embodiments described herein can be implemented in various manners other than the ones described below.

[0024] In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skills in the art to which this disclosure belongs.

[0025] References in the present disclosure to “one embodiment,” “an embodiment,” “an example embodiment,” and the like indicate that the embodiment described may include a particular feature, structure, or characteristic, but it is not necessary that every embodiment includes the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within theknowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0026] It shall be understood that although the terms “first,” “second,”..., etc. in front of noun(s) and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another and they do not limit the order of the noun(s). For example, a first element could be termed a second element, and similarly, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0027] As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or”, mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements.

[0028] As used herein, unless stated explicitly, performing a step “in response to A” does not indicate that the step is performed immediately after “A” occurs and one or more intervening steps may be included.

[0029] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises”, “comprising”, “has”, “having”, “includes” and / or “including”, when used herein, specify the presence of stated features, elements, and / or components etc., but do not preclude the presence or addition of one or more other features, elements, components and / or combinations thereof.

[0030] As used in this application, the term “circuitry” may refer to one or more or all of the following:(a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry) and(b) combinations of hardware circuits and software, such as (as applicable):(i) a combination of analog and / or digital hardware circuit(s) with software / firmware and(ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and(c) hardware ci rcuit(s) and or processor(s), such as a microprocessor(s) or a portionof a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.

[0031] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

[0032] As used herein, the term “communication network” refers to a network following any suitable communication standards, such as New Radio (NR), Long Term Evolution (LTE), LTE- Advanced (LTE-A), Wideband Code Division Multiple Access (WCDMA), High-Speed Packet Access (HSPA), Narrow Band Internet of Things (NB-loT) and so on. Furthermore, the communications between a terminal device and a network device in the communication network may be performed according to any suitable generation communication protocols, including, but not limited to, the first generation (1 G), the second generation (2G), 2.5G, 2.75G, the third generation (3G), the fourth generation (4G), 4.5G, the fifth generation (5G), the sixth generation (6G) communication protocols, and / or any other protocols either currently known or to be developed in the future. Embodiments of the present disclosure may be applied in various communication systems. Given the rapid development in communications, there will of course also be future type communication technologies and systems with which the present disclosure may be embodied. It should not be seen as limiting the scope of the present disclosure to only the aforementioned system. Moreover, the communication may utilize any proper wireless communication technology, comprising but not limited to: Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Frequency Division Duplex (FDD), Time Division Duplex (TDD), Multiple-Input Multiple-Output (MIMO), Orthogonal Frequency Division Multiple (OFDM), Discrete Fourier Transform spread OFDM (DFT- s-OFDM) and / or any other technologies currently known or to be developed in the future.

[0033] As used herein, the term “network device” refers to a node in a communication network via which a terminal device accesses the network and receives services therefrom. The network device may refer to a base station (BS) or an access point (AP), for example, a node B (NodeB or NB), an evolved NodeB (eNodeB or eNB), an NR NB (also referred to as a gNB), a Remote Radio Unit (RRU), a radio header (RH), a remote radio head (RRH), a relay, an Integrated Access and Backhaul (IAB) node, a low power node such as a femto, a pico, a non-terrestrial network (NTN)or non-ground network device such as a satellite network device, a low earth orbit (LEO) satellite and a geosynchronous earth orbit (GEO) satellite, an aircraft network device, and so forth, depending on the applied terminology and technology. In some example embodiments, radio access network (RAN) split architecture comprises a Centralized Unit (CU) and a Distributed Unit (DU) at an IAB donor node. An IAB node comprises a Mobile Terminal (IAB-MT) part that behaves like a UE toward the parent node, and a DU part of an IAB node behaves like a base station toward the next-hop IAB node.

[0034] The term “terminal device” refers to any end device that may be capable of wireless communication. By way of example rather than limitation, a terminal device may also be referred to as a communication device, user equipment (UE), a Subscriber Station (SS), a Portable Subscriber Station, a Mobile Station (MS), or an Access Terminal (AT). The terminal device may include, but not limited to, a mobile phone, a cellular phone, a smart phone, voice over IP (VoIP) phones, wireless local loop phones, a tablet, a wearable terminal device, a personal digital assistant (PDA), portable computers, desktop computer, image capture terminal devices such as digital cameras, gaming terminal devices, music storage and playback appliances, vehiclemounted wireless terminal devices, wireless endpoints, mobile stations, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), USB dongles, smart devices, wireless customer-premises equipment (CPE), an I nternet of Things (loT) device, a watch or other wearable, a head-mounted display (HMD), a vehicle, a drone, a medical device and applications (e.g., remote surgery), an industrial device and applications (e.g., a robot and / or other wireless devices operating in an industrial and / or an automated processing chain contexts), a consumer electronics device, a device operating on commercial and / or industrial wireless networks, and the like. The terminal device may also correspond to a Mobile Termination (MT) part of an IAB node (e.g., a relay node). In the following description, the terms “terminal device”, “communication device”, “terminal”, “user equipment” and “UE” may be used interchangeably.

[0035] As used herein, the term “resource,” “transmission resource,” “resource block,” “physical resource block” (PRB), “uplink resource,” or “downlink resource” may refer to any resource for performing a communication, for example, a communication between a terminal device and a network device, such as a resource in time domain, a resource in frequency domain, a resource in space domain, a resource in code domain, or any other combination of the time, frequency, space and / or code domain resource enabling a communication, and the like. In the following, unless explicitly stated, a resource in both frequency domain and time domain will be used as an example of a transmission resource for describing some example embodiments of the present disclosure. It is noted that example embodiments of the present disclosure are equally applicableto other resources in other domains.

[0036] As used herein, the term “model” is referred to as an association between an input and an output learned from training data, and thus a corresponding output may be generated for a given input after the training. The generation of the model may be based on machine learning (ML) techniques. The machine learning techniques may also be referred to as artificial intelligence (Al) techniques. In general, a machine learning model can be built, which receives input information and makes predictions based on the input information. For example, a classification model may predict a class of the input information among a predetermined set of classes. As used herein, “model” may also be referred to as “machine learning model”, “learning model”, “machine learning network”, or “learning network,” which are used interchangeably herein. Supervised learning refers to a process of training a model from input and its corresponding labels. The trained model is then used to infer the output.

[0037] Generally, model lifecycle management may usually include three stages, i.e. , a training stage, a validation stage, and an application stage (also referred to as an inference stage). At the training stage, a given AI / ML model may be trained (or optimized) iteratively using a great amount of training data until the model can make inference close to desired outputs in the training or labelled dataset. During the training, a set of parameter values of the model is iteratively updated until a training objective is reached. Through the training process, the AI / ML model may be regarded as being capable of learning the association between the input and the output (also referred to an input-output mapping) from the training data. At the validation stage, a validation input is applied to the trained AI / ML model to test whether the model can provide a correct output, so as to determine the performance of the model. Generally, the validation stage may be considered as a step in a training process, or may be omitted in some cases. At the inference stage, the resulting AI / ML model may be used to process a real-world model input based on the trained model obtained from the training process and to determine the corresponding model output. In some cases, a retraining or updating stage may be included in the model lifecycle management, to enable the model evolved to have better performance.

[0038] To facilitate understanding of the terminologies, some definitions of the list of terminologies used for AI / ML are provided below.

[0039] AI / ML Model: A data driven algorithm that applies AI / ML techniques to generate a set of outputs based on a set of inputs.

[0040] AI / ML model delivery: A generic term referring to delivery of an AI / ML model from one entity to another entity in any manner. Note: An entity could mean a network node / function (e.g., gNB, location management function (LMF), etc.), UE, proprietary server, etc.

[0041] AI / ML model Inference: A process of using a trained AI / ML model to produce a set of outputs based on a set of inputs.

[0042] AI / ML model testing: A subprocess of training, to evaluate the performance of a final AI / ML model using a dataset different from one used for model training and validation. Differently from AI / ML model validation, testing does not assume subsequent tuning of the model.

[0043] AI / ML model training: A process to train an AI / ML Model [by learning the input / output relationship] in a data driven manner and obtain the trained AI / ML Model for inference.

[0044] AI / ML model transfer: Delivery of an AI / ML model over the air interface in a manner that is not transparent to 3GPP signalling, either parameters of a model structure known at the receiving end or a new model with parameters. Delivery may contain a full model or a partial model.

[0045] AI / ML model validation: A subprocess of training, to evaluate the quality of an AI / ML model using a dataset different from one used for model training, that helps selecting model parameters that generalize beyond the dataset used for model training.

[0046] Data collection: A process of collecting data by the network nodes, management entity, or UE for the purpose of AI / ML model training, data analytics and inference.

[0047] Federated learning / federated training: A machine learning technique that trains an AI / ML model across multiple decentralized edge nodes (e.g., UEs, gNBs) each performing local model training using local data samples. The technique requires multiple interactions of the model, but no exchange of local data samples.

[0048] Functionality identification: A process / method of identifying an AI / ML functionality for the common understanding between the network and the UE. Note: Information regarding the AI / ML functionality may be shared during functionality identification. Where AI / ML functionality resides depends on the specific use cases and sub use cases.

[0049] Model activation: enable an AI / ML model for a specific function.

[0050] Model deactivation: disable an AI / ML model for a specific function.

[0051] Model download: Model transfer from the network to UE.

[0052] Model identification: A process / method of identifying an AI / ML model for the common understanding between the network (NW) and the UE. Note: The process / method of model identification may or may not be applicable. Note: Information regarding the AI / ML model may be shared during model identification.

[0053] Model monitoring: A procedure that monitors the inference performance of the AI / ML model.

[0054] Model parameter update: Process of updating the model parameters of a model.

[0055] Model selection: The process of selecting an AI / ML model for activation among multiplemodels for the same AI / ML enabled feature. Note: Model selection may or may not be carried out simultaneously with model activation.

[0056] Model switching: Deactivating a currently active AI / ML model and activating a different AI / ML model for a specific function.

[0057] Model update: Process of updating the model parameters and / or model structure of a model.

[0058] Model upload: Model transfer from UE to the network.

[0059] Network-side (AI / ML) model: An AI / ML Model whose inference is performed entirely at the network.

[0060] Offline field data: The data collected from field and used for offline training of the AI / ML model.

[0061] Offline training: An AI / ML training process where the model is trained based on collected dataset, and where the trained model is later used or delivered for inference. Note: This definition only serves as a guidance. There may be cases that may not exactly conform to this definition but could still be categorized as offline training by commonly accepted conventions.

[0062] Online field data: The data collected from field and used for online training of the AI / ML model.

[0063] Online training: An AI / ML training process where the model being used for inference) is (typically continuously) trained in (near) real-time with the arrival of new training samples. Note: the notion of (near) real-time vs. non real-time is context-dependent and is relative to the inference time-scale. Note: This definition only serves as a guidance. There may be cases that may not exactly conform to this definition but could still be categorized as online training by commonly accepted conventions. Note: Fine-tuning / re-training may be done via online or offline training. (This note could be removed when we define the term fine-tuning.)

[0064] Reinforcement Learning (RL): A process of training an AI / ML model from input (a.k.a. state) and a feedback signal (a.k.a. reward) resulting from the model’s output (a.k.a. action) in an environment the model is interacting with.

[0065] Semi-supervised learning: A process of training a model with a mix of labelled data and unlabelled data.

[0066] Two-sided (AI / ML) model: A paired AI / ML Model(s) over which joint inference is performed, where joint inference comprises AI / ML Inference whose inference is performed jointly across the UE and the network, i.e, the first part of inference is firstly performed by UE and then the remaining part is performed by the gNB, or vice versa.

[0067] UE-side (AI / ML) model: An AI / ML Model whose inference is performed entirely at theUE.

[0068] Unsupervised learning: A process of training a model without labelled data.

[0069] Proprietary-format models: ML models of vendor-Zdevice-specific proprietary format, from 3GPP perspective. They are not mutually recognizable across vendors and hide model design information from other vendors when shared. Note: An example is a device-specific binary executable format.

[0070] Open-format models: ML models of specified format that are mutually recognizable across vendors and allow interoperability, from the 3GPP perspective. They are mutually recognizable between vendors and do not hide model design information from other vendors when shared.

[0071] Model identification: A process / method of identifying an AI / ML model for the common understanding between the network device and the UE. Note: The process / method of model identification may or may not be applicable. Note: Information regarding the AI / ML model may be shared during model identification.

[0072] Functionality identification: A process / method of identifying an AI / ML functionality for the common understanding between the network device and the UE. Note: information regarding the AI / ML functionality may be shared during functionality identification.

[0073] FIG. 1 illustrates an example communication environment 100 in which example embodiments of the present disclosure can be implemented. It is to be understood that the elements shown in the communication environment 100 are intended to represent main functions provided within the system. As such, the blocks shown in FIG. 1 refer to specific elements in communication networks that provide these main functions. However, other network elements may be used to implement some or all of the main functions represented. Also, it is to be understood that not all functions of a communication network are depicted in FIG. 1. Rather, functions that facilitate an explanation of illustrative embodiments are represented. Further, the number of the elements shown in FIG. 1 is also for the purpose of illustration only and there may be any number of elements.

[0074] As shown, the communication environment 100 comprises a plurality of communication devices, including testing equipment (TE) 110, one or more devices under test (DUTs) 120-1 , 120- 2, ..., 120-N (collectively or individually referred to as DUTs 120) and one or more TRPs 130-1 , 130-2 etc. (collectively or individually referred to as TRPs 130). A serving area of the TRP 130 may be called a cell. The DUTs 120 may perform signal transmission and reception with the TRPs 130.

[0075] In some example embodiments, one or more AI / ML models 125-1 , 125-2, ..., 125-N(collectively or individually referred to as AI / ML models 125) may be used by the one or more DUTs 120. An AI / ML model 125 may sometimes be referred to as an Al model or an ML model for short. Different AI / ML models 125 may be configured to implement the same different algorithms in the communication environment 100. The AI / ML model 125 used by a DUT 120 may sometimes include to either a model or an AI / ML functionality.

[0076] In some example embodiments, the AI / ML models 125 are configured for AI / ML based positioning or AI / ML enabled positioning. AI / ML enabled positioning is one of the selected usecases for the study item in the development of communication networks. In some example embodiments, there have been proposed two positioning approaches. A first approach is direct AI / ML positioning where the output of the AI / ML model inference is the UE location. There are multiple options for the input of the model which includes, channel observations such as Channel Impulse Response (CIR), Power Delay Profile (PDP), Reference Signal Received Power (RSRP), Reference Signal Received Path Power (RSRPP), etc. A second approach is AIML assisted positioning where the output of the AI / ML model inference is a new measurement and / or enhancement to the existing measurements, and such new measurement and / or enhancement may be called as intermediate feature as it will be input to a second function to finally estimate the UE position. These measurements include, e.g., line-of-sight (LOS) / non-line-of-sight (NLOS) identifications, Time of Arrival (ToA), path phase, Reference Signal Time Difference (RSTD), etc.

[0077] In some example embodiments of the present disclosure, the TE 110 is configured to test the AI / ML model(s) 125 used by the DUTs 120. In some example embodiments, TE 110 may be a network entity in the core network, a base station (e.g., gNB, eNB) in RAN, or may be a terminal device (e.g., UE) or any other device that is configured for AI / ML testing.

[0078] In some example embodiments, a DUT 120 may be a terminal device (e.g., UE) that uses the AI / ML model 125 for positioning purpose. In some example embodiments, a DUT 120 may also be a network entity in the core network or a base station (e.g., gNB, eNB) in RAN which is configured to uses the AI / ML model 125 for positioning purpose.

[0079] A study of Artificial Intelligence (Al) / Machine Learning (ML) for New Radio (NR) air interface is now ongoing in 3GPP Rel-18. One of the objectives of the study was to cover the interoperability and testability aspect of the newly defined AI / ML enabled features.

[0080] Regarding interoperability and testability aspects, e.g., (RAN4), RAN4 only starts the work after there is sufficient progress on use case study in RAN1 and RAN2.

[0081] There are requirements and testing frameworks to validate AI / ML based performance enhancements and ensuring that UE and gNB with AI / ML meet or exceed the existing minimum requirements if applicable. It also needs to consider the need and implications for AI / ML processingcapabilities definition.

[0082] In Release 18, UE based direct AI / ML positioning is one of the selected sub-use cases for positioning. The work on this use case will continue in terms of the Wl in Release 19 and also in the upcoming next generations (e.g., in 6G).

[0083] For positioning accuracy enhancements, it is expected to encompass direct AI / ML positioning and AI / ML assisted positioning. For direct AI / ML positioning, the first priority use case (1stpriority) is Case 1 , i.e., UE-based positioning with UE-side model, direct AI / ML positioning; the second priority use case (2ndpriority) is Case 2b, i.e., UE-assisted / LMF-based positioning with LMF-side model, direct AI / ML positioning; another first priority use case (1stpriority) is Case 3b, i.e., NG-RAN node assisted positioning with LMF-side model, direct AI / ML positioning. For AI / ML assisted positioning, the second priority use case (2ndpriority) is Case 2a, i.e., UE-assisted / LMF- based positioning with UE-side model, AI / ML assisted positioning; and the first priority use case (1stpriority) is Case 3a, i.e., NG-RAN node assisted positioning with gNB-side model, AI / ML assisted positioning.

[0084] It is specified necessary measurements, signalling / mechanismfs) to facilitate LCM operations specific to the Positioning accuracy enhancements use cases, if any. It also needs to investigate and specify the necessary signalling of necessary measurement enhancements (if any). It also needs to enable method(s) to ensure consistency between training and inference regarding NW-side additional conditions (if identified) for inference at UE for relevant positioning sub use cases.

[0085] Some core requirements for the above two use cases for AI / ML LCM procedures and UE features are to specify necessary RAN4 core requirements for the above two use cases, and to specify necessary RAN4 core requirements for LCM procedures including performance monitoring.

[0086] FIG. 2 illustrates an example architecture 200 of a functional framework for an AI / ML functionality-based functionality. At data collection 210, one or more communication devices may collect training data, monitoring data, and inference data for the AI / ML functionality.

[0087] The training data and the monitoring data may generally include inputs of the AI / ML functionality and ground-truth labels for the corresponding inputs. The inference data generally include inputs of the AI / ML functionality for inference.

[0088] As used herein, the term “data collection” may refer to a process of collecting data by the network nodes, management entity, or UE for the purpose of AI / ML functionality training, data analytics, and inference. The data collection can further be referred to a function that provides input data to the Model Training, Management, and Inference functions. For example, the datacollection comprises training data, monitoring data, and inference data. The training data refers to the data input for the AI / ML functionality Training function. The monitoring data refers to the data input for the Management of AI / ML models or AI / ML functionalities. The inference data refers to the data input for the AI / ML Inference function.

[0089] At model training 220, an AI / ML functionality may be trained or updated using the collected training data, to provide a trained or updated AI / ML functionality. At model management 230, performance of an AI / ML mode may be evaluated using the collected monitoring data. The model management 230 may provide performance feedback or retraining request to the model training 220 for fine-tune or update the AI / ML functionality. The model management 230 may further provide a management instruction for model inference 240, e.g., for deploy or activate the AI / ML functionality with satisfied performance. At model inference 240, the AI / ML functionality is applied to output direct positioning of a terminal device or assisted positioning data of a terminal device based on inference data collected. Inference output of the AI / ML functionality may be provided for the model management, e.g., to evaluate whether the AI / ML functionality still runs with satisfied performance. In some cases, the AI / ML functionality with satisfied performance may be stored in model storage 250 in response to a model transfer or delivery request. The AI / ML functionality may then be transferred or delivered for use in model inference.

[0090] As an illustrative example, consider a scenario where the network performs functionality-based LCM and where models are not identified in the network, while the UE concurrently performs model-level management (e.g., model selection / switching / (de)activation, etc.).

[0091] Management is a function that oversees the operation (e.g., selection / (de)activation / switching / fallback) and monitoring (e.g., performance) of AI / ML models or AI / ML functionalities. This function is also responsible for making decisions to ensure the proper inference operation based on data received from the Data Collection function and the Inference function. For management instruction, information needed as input to manage the inference function. Concerning information may include selection / (de)activation / switching of AI / ML models or AI / ML-based functionalities, fallback to non-AI / ML operation (i.e., not relying on AI / ML inference process), etc. A model transfer / delivery request is used to request model(s) to the model storage function. For performance feedback / retraining request, information is needed as input for the model training function, e.g., for model (re)training or updating purposes.

[0092] Based on the Rel-18 LCM discussion in 3GPP, LCM actions may be triggered by the NW. In functionality-based LCM, the network indicates activation / deactivation / fallback / switching of AI / ML functionality via 3GPP signalling, e.g., radio resource control (RRC), medium accesscontrol (MAC) control element (MAC-CE), downlink control information (DCI), etc. In model-ID- based LCM, models are identified at the network, and network / UE may activate / deactivate / select / switch individual AI / ML models via model ID.

[0093] For evaluation of performance monitoring approaches, the following model monitoring key performance indicators (KPIs) are considered as general guidance in the communication specifications (e.g., in 3GPP TR 38.843):- Accuracy and relevance (i.e., how well does the given monitoring metric / methods reflect the model and system performance);- Overhead (e.g., signalling overhead associated with model monitoring); - Complexity (e.g., computation and memory cost for model monitoring);- Latency (i.e., timeliness of monitoring result, from model failure to action, given the purpose of model monitoring).

[0094] It is noted that other KPIs are not precluded. Relevant KPIs may vary across different model monitoring approaches.

[0095] In the latest RAN4#109 meeting, TR 38.843 with RAN4 parts was approved, Here are some extracts from the approved document in Table 1 and Table 2:Table 1Table 2

[0096] As it can be seen from above extracts LCM aspects are an important topic currently being discussed in the SI for AI / ML air interface and there are potential impacts in RAN4 requirements as well.

[0097] Furthermore, AI / ML enabled Positioning is one of the selected use-case for the study item in release 18. The RAN4#107 meeting discussions have already touched on the aspects of performance evaluation in RAN4 for different selected use cases for AI / ML enabled features, see Table 3.Table 3

[0098] Furthermore, in RAN1 discussions, as mentioned in the TR38.843, performance monitoring is one of the core components of AI / ML based functionalities mainly due to indeterministic nature of AI / ML based solutions. For performance monitoring in case of AI / ML based positioning enhancements use case following text in Table 4 is captured in the TR. Table 4

[0099] Quality of Service (QoS) Class Identifier (Cl) (QCI or 5QI (Quality Identifier) in 5G) are the terms used interchangeably in this disclosure. QCI is a scalar used to define a reference for a specific packet forwarding behaviour (e.g. packet loss rate, packet delay budget) to be provided to a Service Data Flow (SDF). This mechanism may be implemented in the access network by the QCI referencing node specific parameters that control packet forwarding treatment (e.g. scheduling weights, admission thresholds, queue management thresholds, link layer protocol configuration, etc.), that have been pre-configured by the operator at a specific node(s) (e.g. eNodeB). The following standardization characteristics are defined in 3GPP TS 23.203 for QCI. The similar mapping for 5G NR, 5QI, can be found in 3GPP TS 23.501.

[0100] As can be seen from above references that LCM related impacts for AI / ML based Positioning enhancements use case is a significantly important topic currently under discussion in 3GPP. Therefore, core requirements and testing mechanism for these core requirements to validate the correct functionality of the device is required to be standardized in RAN4.

[0101] Example embodiments of the present disclosure focuses on core requirements are identified for LCM related actions for AI / ML-based positioning use case and a test mechanism is proposed to verify these requirements.

[0102] There is a lot of interest in the requirements and testability of LCM aspects for AI / ML enabled functionalities. One of the factors that influence the performance of the AIML enabled functionality is the latency of the LCM actions between the Network and the UE.

[0103] If performance monitoring detects a performance degradation to a point where a decision to either switch this model / functionality with another model / functionality is taken or a fallback to a legacy / default algorithm. It means that the AI / ML functionality is degrading the system performance and if this functionality, with detected performance degradation, keeps running then the impact on system performance may result in drastic consequences.

[0104] Therefore, it is important to stop this model / functionality, either by falling back to legacy method or by switching to another model / functionality, within a specified time. This specified time may depend on a particular use case since different use cases would need different level ofurgency to stop / switch to different model / functionality. In the case of AI / ML based Positioning, it would also depend on the application using the positioning coordinates. In case of some applications, such as fast-moving objects in an industrial setup, a more precise and rapid location information is required therefore, any delay in the switchi ng / disabling / fall back to legacy of a bad functionality (which has been detected as the one degrading performance) would quickly lead to very drastically bad performance. Therefore, the time allowed in such a case should be less and should reflect the need of an urgent action from the DUT.

[0105] The specified time allowed to switch / disable the model / functionality should guarantee that system performance would not be allowed to be degraded to unacceptable levels. Therefore, the values of these specified times may be calculated based on simulation results and / or on field data.

[0106] Consider a sample LCM workflow with an example from the positioning use case that is being discussed.

[0107] It is assumed that UE or network detects a performance degradation, and that AI / ML based Positioning feature / functionality / model is enabled. Then the NW needs to take an appropriate LCM action to not allow further degradation in the performance (KPIs). In case the UE detects the performance degradation, UE shall indicate it to the network. These LCM actions can be: switch to another model / functionality, switch back to legacy mechanism if available, and disable the functionality.

[0108] As soon as these LCM commands are sent to the UE, UE needs to execute them accordingly. Any delay in the execution of these commands would further deteriorate the performance (KPIs).

[0109] In case of AI / ML-based positioning , this may depend on the application for which positioning coordinates are used. It can be urgent in case of fast-moving objects in an industrial setup and may be more relaxed for slow moving / static objects in both indoor and outdoor environments.

[0110] Therefore, it is important to ensure that the UE executes the command and either switches back to legacy or switches to another model / functionality within a defined time period. There is currently no requirement that can guarantee this expected behavior from the UE.

[0111] It is expected to guarantee latency of LCM actions towards the DUT for AI / ML enabled positioning. Also, there are no test mechanisms that can help validate the LCM performance latency for AI / ML enabled positioning.

[0112] Example embodiments of the present disclosure provide a solution to the above identified problems using which the latency of LCM actions can be guaranteed within the testframework for AI / ML enabled positioning.

[0113] In some example embodiments, the latency of LCM procedures is explicitly defined (e.g., in RAN4 requirements or RAN2 specifications). A device which is under test can support certain types of LCM actions or operations (e.g., switching from positioning method / functionality to another). The device can provide its capability to support certain level of delays for the supported LCM actions or operations.

[0114] The latency of the LCM actions may depend at least on the following aspects: switching the AI / ML model for the same method / functionality, switching of the AI / ML model together with the switch of the method / functionality, and / or switching to the fallback positioning method

[0115] The support of latency requirements can be ensured with the corresponding RAN4 / RAN5 tests for the AI / ML-based positioning. In some example embodiments, the tests are performed by triggering LCM operations / actions based on the commands from the TE. Optionally, the tests are performed by triggering / emulating performance degradation of the prediction by the changing the test environment and / or its parameters.

[0116] In some example embodiments, it is to trigger the change of the model / functionality or feedback (either by the TE) or monitoring of the corresponding signaling at the DUT (e.g., UE or network entity). In some other example embodiments, it is to monitor the change / improvement of the model / functionality performance at the DUT.

[0117] In some example embodiments, the latency of the LCM operation is monitored at the TE, e.g., based on the DUT signaling. In some example embodiments, the test success / failure is determined based on the latency and / or performance enhancement at the DUT.

[0118] In some example embodiments, the latency requirements are defined ensure that an LCM action for the positioning use case is executed within an allowed time budget. These requirements may be based either on simulation results or on field data or on the application using positioning coordinates.

[0119] A new test mechanism is needed to test these new requirements. In some example embodiment, either the DUT informs the TE about the switching to the legacy mechanism or to a new model / functionality using RAN1 procedures, if any. Or the TE configures a test mode to the DUT in order to get this information from the UE whenever switching to legacy or to new model / functionality occurs.

[0120] Depending upon the urgency of the application using the positioning coordinates, these LCM actions may be on different interfaces. For instance, for fast-moving objects in an industrial environment where a quick response is needed, it may be on MAC CE interface and a more stringent core / latency requirement would be in place. In case of static or slow moving objects, itmay be on RRC interface and a more relaxed core / latency requirement would be applied.

[0121] Example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0122] FIG. 3 illustrates a flowchart of a signaling flow 300 for a LCM procedure in AI / ML-based positioning in accordance with some example embodiments of the present disclosure. As shown in FIG. 3, the signaling flow 300 involves a first apparatus 301 and a second apparatus 302.

[0123] The first apparatus 301 may be or may be comprised in a TE, e.g., the TE 110 in the communication environment 100 of FIG. 1. The first apparatus 301 is configured to emulate / simulate a real wireless network. In some example embodiment, the first apparatus 301 may be a real network device (e.g., gNB) or may be a system simulator, operating as signal generators, probes, or transmission points that are used to transmit radio signal of certain type (e.g., used for SSB transmission). In some example embodiments, the first apparatus 301 may also include channel emulators and / or attenuators to emulate the propagation of the radio signal. The different propagations conditions allow to set up different LOS / NLOS conditions. As will be discussed below, reference LOS / NLOS probabilities can be selected for the test setup.

[0124] The second apparatus 302 may be or may be comprised in a DUT 120 in the communication environment 100 of FIG. 1 , which may be one or more of terminal devices, network entities, or base stations. In embodiments of the present disclosure, a second apparatus 302 may be able to implement AI / ML-based positioning (also referred to as AI / ML assisted positioning or AI / ML enabled positioning). An AI / ML model or AI / ML functionality is utilized by the second apparatus 302 to implement the AI / ML-based positioning. The output of the AI / ML model is a channel indicator for a communication channel between the second apparatus 302 and a network device, e.g., a TRP or base station. The channel indicator indicates a probability of a communication channel between the second apparatus 302 and the TRP being a LOS channel or a NOLS channel. Such a channel indicator may also be referred to as a LOS / NLOS indicator. The input to the AI / ML model may include measurement results of one or more reference signals transmitted from the TRP.

[0125] In the signaling flow 300, the second apparatus 302 transmits (305), to the first apparatus 301 , capability information of the second apparatus 302. The capability information at least indicates respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning.

[0126] In some example embodiments, the plurality of LCM actions may involve AI / ML model / functionality switching. In some example embodiments, the plurality of LCM actions may comprise at least one of the following: switching from a first AI / ML-based positioning functionality to a secondAI / ML-based positioning functionality, switching from a first AI / ML positioning model to a second AI / ML positioning model, switching from a first positioning mechanism to a second positioning mechanism, and / or switching from an AI / ML-based positioning functionality to a non-AI / ML-based positioning functionality (also referred to as a legacy positioning functionality). Within an AI / ML- based positioning functionality, the used AI / ML model can be changed, and / or the positioning mechanism (or positioning method) can be changed in the LCM procedure.

[0127] A response time for an LCM action may be determined based on a latency requirement for the LCM action. The time period for switching (i.e. the latency) from one AI / ML model / functionality to another for AI / ML-based positioning or to switching from the AI / ML model / functionality to the legacy positioning method may have impact on the end user experience. Hence the time required for switching as part of LCM procedure may adhere to the time period requirements that are negotiated between UE and the network during the configuration.

[0128] Table 5 provides an example of LCM action latency requirements depending on LCM action and different / same positioning methods.Table 5

[0129] The above table (as an example) defines the AI / ML model / functionality switching delay requirements for Positioning use case.

[0130] In some example embodiments, the respective response times for the plurality of LCM actions may be determined based on a capability indicating a model switching duration, e.g., UE capability indicating the model switching period / duration. Alternatively, or in addition, therespective response times for the plurality of LCM actions may be determined based on a category of a terminal devices to be positioned, e.g., the UE category. Alternatively, or in addition, the respective response times for the plurality of LCM actions may be determined based on a tolerance level of degradation. The tolerance level of degradation may be determined based on the simulations or field values, which shows what level of degradation may be tolerated for this use case.

[0131] Alternatively, or in addition, the respective response times for the plurality of LCM actions may be determined based on a latency requirement for an application that requires positioning information, a quality identifier in 5G (5QI) of an application that requires positioning information, and / or an application type of an application that requires positioning information. For example, for location critical applications, the response time may need to be smaller compared to other applications. Different application may have different latency requirements for the positioning coordinates availability. For instance, an indoor industrial environment which need precise and real time position of moving robots may require very strict latency requirements and thus the response time may be set to a small value. For an outdoor emergency use case would require a very precise location but may be somehow relaxed in latency requirements, and thus the corresponding response time may be set to a larger value.

[0132] It would be appreciated that some example factors impacting the response times for the plurality of LCM actions are provided above, but other factors are also applicable. For example, the response times may be based on different profiles, such as different traffic profiles or the radio environment or the speed of the moving device.

[0133] With the capability information of the second apparatus 302 obtained, the first apparatus 301 may initiate a test mechanism for validation of the AI / ML model / functionality switching latency requirements.

[0134] In some example embodiments, the first apparatus 301 is testing the AI / ML-based positioning functionality used at the second apparatus 302. A LCM procedure is initiated, along with network-side performance monitoring.

[0135] In some example embodiments, the second apparatus 302 is set to a test mode for the AI / ML-based positioning. The first apparatus 301 may send a massage (e.g., NW message or test mode message) to set the second apparatus 302 to the AI / ML-based positioning test mode. Then the test mechanism is performed with the test mode. In some example embodiments, the test mechanism may be performed without the test mode.

[0136] In some example embodiments, the capability information may indicate LCM parameters that include the response time(s) for the model / functionality switching(s). In some exampleembodiments, the capability information may indicate a response time for switching to legacy Positioning method (Alternate method. The LCM response time may be derived by the network based on the existing UE capabilities and / or based on the UE category and indicate the allowed switch response maximum time duration to DUT.

[0137] In some example embodiments, in addition to the response times for the LCM actions, the capability information may further indicate or include a flag indicating enablement or disablement of AI / ML-based positioning, referring to as “AI / ML based positioning enable / disable flag”. In some example embodiments, alternatively or in addition, the capability information may further indicate a plurality of positioning mechanisms supported for switching within an AI / ML- based positioning functionality. In some example embodiments, alternatively or in addition, the capability information may further indicate a plurality of AI / ML positioning models supported for switching. In some example embodiments, alternatively or in addition, the capability information may further indicate at least one type of signalling supported for an indication of the LCM action. For example, the capability information may positioning report signalling methods supported, e.g., RRC signalling or fast signalling using MAC CE / DCI.

[0138] The first apparatus 301 receives (310) the capability information of the second apparatus 302. The first apparatus 301 transmits (315), to the second apparatus 302, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus 302.

[0139] In some example embodiments, the first apparatus 301 may trigger the AI / ML based positioning performance degradation by any of the feasible method (like tweaking the PRS signal power transmitted by TRPs, change the PRS transmission time to induce Positioning errors etc.).

[0140] In some example embodiments, the first apparatus 301 may detect performance degradation on an AI / ML-based positioning functionality at the second apparatus 302, and / or may monitor an indication of the performance degradation from the second apparatus 302. Upon a detection of performance degradation on an AI / ML-based positioning functionality or a reception of an indication of the performance degradation from the second apparatus 302, the first apparatus 301 may decide to transmit, to the second apparatus 302, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus 302.

[0141] In some example embodiments, the first apparatus 301 may provide the periodicity when the degradation indication should be reported by the second apparatus 302 to the first apparatus 301 (i.e. no need of triggering degradation). The second apparatus 302 may determine when to report the performance degradation based on the periodicity of reporting the indication of the performance degradation provided by the first apparatus 301. In some example embodiments, the indication of the performance degradation may be provided in an aperiodic manner, or may be anone-time indication from the second apparatus 302 to the first apparatus 301. In some example embodiments, the first apparatus 301 may trigger model switch by assuming the degradation after a certain time period (i.e. no need of triggering degradation or indication from the second apparatus 302).

[0142] Upon the indication of the LCM action, the first apparatus 301 starts (325) a timer for the LCM action based on the capability information of the second apparatus 302. The timer may be set based on the corresponding response time for the LCM action. This timer may also referred to as an LCM response timer. The timer is set to restrict the allocated time for the second apparatus 302 to complete the LCM action. The first apparatus 301 may store the LCM response timer values supported and sent by the second apparatus 302 in the capability information.

[0143] In some example embodiments, depending upon the urgency of the application using the positioning coordinates, these LCM actions may be on different interfaces. The first apparatus 301 may determine a type of signaling for the indication of the LCM action based on a latency requirement for an application that requires positioning information, and then transmit, to the second apparatus 302, the indication of the LCM action using the type of signaling.

[0144] In some example embodiments, in accordance with a determination that the latency requirement for the application is corresponding to a low latency requirement, the first apparatus 301 may determine that the type of signaling to be medium access control (MAC) signaling. In some cases, if the latency requirement for the application is corresponding to a high latency requirement, the first apparatus 301 may determine that the type of signaling to be radio resource control (RRC) signaling. For instance, for fast-moving objects in an industrial environment where a quick response is needed, the indication of the LCM action may be communicated through the MAC CE interface and a more stringent core / latency requirement would be in place. In case of static or slow moving objects, the indication of the LCM action may be communicated through the RRC interface and a more relaxed core / latency requirement would be applied.

[0145] In some example embodiments, through the indication of the LCM action, the first apparatus 301 may consider at least the following aspects while deciding the model to be used for switching: retaining the same positioning method or a different positioning method, selection of the model based on the latency requirement of the QoS flow (i.e. based on 5QI latency requirements), and / or selection of the signalling method (RRC signalling or fast signalling using MAC CE / DCI) based on UE capability, 5QI latency requirements and test configuration). For example, if the application is time critical like V2X / URLLC, the fast signalling should be selected.

[0146] At the side of the second apparatus 302, the second apparatus 302 receives (320), from the first apparatus 301 , the indication of an LCM action of the plurality of LCM actions to be appliedby the second apparatus 302. Then the second apparatus 302 may perform the LCM action. Upon a completion of the LCM action, the second apparatus 302 transmits (330), to the first apparatus 301 , a notification of a successful completion of the LCM action.

[0147] At the side of the first apparatus 301 , the first apparatus 301 determines whether a notification of a successful completion of the LCM action is received from the second apparatus 302 while the timer is running.

[0148] In some cases, in accordance with a determination that a notification of a successful completion of the LCM action is received from the second apparatus 302 while the timer is running, the first apparatus 301 determines (340) that a test of the LCM action is successful. In some cases, in accordance with a determination that a notification of a successful completion of the LCM action is not received from the second apparatus 302 while the timer is running, the first apparatus 301 determines (340) that a test of the LCM action is failed.

[0149] In some example embodiments, the first apparatus 301 may monitor the positioning performance for a specified or configured time duration after at least one LCM action has been successfully applied by the second apparatus 302. If the first apparatus 301 detects no performance improvement on AI / ML-based positioning after at least one LCM action has been successfully applied by the second apparatus 302, the first apparatus 301 may decide to switch the second apparatus 302 to a non-AI / ML-based positioning functionality. In this case, the first apparatus 301 may transmit, to the second apparatus 302, an LCM action of switching from an AI / ML-based positioning functionality to a non-AI / ML-based positioning functionality.

[0150] FIG. 4 illustrates a detailed signaling flow 400 for a LCM procedure in AI / ML positioning in accordance with some further example embodiments of the present disclosure. For the purpose of illustration, the signaling flow 400 is described with reference to FIG. 1 and involves the TE 110 and DUT 120. The TE 110 may be considered as an example of the first apparatus 301 in FIG.3, and the DUT 120 may be considered as an example of the second apparatus 302.

[0151] In the signaling flow 400, the test mechanism for switching of the AI / ML model or functionality when the performance degradation is detected at the network is described. TE 110 may be configured for AI / ML-based positioning model / functionality performance monitoring as part of a LCM procedure.

[0152] At 401 a, the TE 110 is testing AI / ML enabled positioning functionality, and at 402a, a LCM procedure is turned on, along with network-side performance monitoring. At 401 b, as an optional, the TE 110 may move the DUT 120 to an AI / ML test mode. TE 110 may send a message (NW message or test mode message) to move the DUT 120 to the AI / ML positioning test mode.

[0153] At 402b, the DUT 120 transmits UE capability information to the TE 110. The UEcapability information may include one or more of the following: an AI / ML based Positioning enable / disable flag; positioning methods and / or models supported for switching; positioning report signalling methods supported (RRC signalli ng / fast signalling using MAC CE / DCI).

[0154] The UE capability information may further indicate LCM parameters that includes response time for model / fu nctionality switching. The UE capability information may further indicate response time for switching to a legacy positioning method. As an alternate, the LCM response time for switching to a legacy positioning method may be derived by a network device based on the existing UE capabilities and / or based on the UE category and indicate the allowed switch response maximum time duration to the DUT 120.

[0155] At 403, the TE 110 stores the LCM response timer values supported and sent by DUT 120 based on the received UE capability information. The LCM response timer values may be set based on the example of Table 5 as shown above.

[0156] At 404, AI / ML-based positioning measurement and reporting is started between the TE 110 and the DUT 120. During the on-going AI / ML enabled positioning test, at 405, the DUT may trigger AI / ML-based positioning performance degradation and may indicate the performance degradation to the TE 110. In some example embodiments, at 406, the TE 110 may detect AI / ML- based positioning performance degradation.

[0157] In some example embodiments, the TE 110 may trigger the AI / ML-based positioning performance degradation by any of the feasible method (like tweaking the PRS signal power transmitted by TRPs, change the PRS transmission time to induce Positioning errors etc.). In an example, the TE 110 may provide the periodicity when the degradation indication should be reported by DUT to TE (i.e. no need of triggering degradation). In another example, the TE 110 may trigger model switch assuming the degradation after certain time period (i.e. no need of triggering degradation or indication from DUT 120).

[0158] At 407, the TE 110 indicates the DUT 120 to switch to a different AI / ML model / functionality. The LCM module in TE 110 may consider at least the following aspects while deciding the model to be used for switching: o Retain same Positioning method or a different positioning method; o Selection of the model based on the latency requirement of the QoS flow (i.e. based on 5QI latency requirements); o Selection of the signalling method (RRC signalling or fast signalling using MAC CE / DCI) based on UE capability, 5QI latency requirements and test configuration). For example, if the application is time critical like V2X / URLLC, the fast signalling should be selected.

[0159] At 408, the TE 110 TE starts a LCM response timer (e.g., by referring to Table 5 shownabove) and waits for the model / functionality switch based on the UE capability information and TE configuration.

[0160] At 409, upon reception of the indication of the LCM action from the TE 110, the DUT 120 performs switching to a different AI / ML model / functionality. Then at 410, the DUT 120 may send a notification of successful AI / ML model / functionality switch to the TE 110.

[0161] The LCM response timer for the LCM action is still running at the TE 110 at 411. At a first alternative (Alt. 1) at 412, if the response (i.e., the notification) is received from the DUT 120 within the allocated time (i.e., the LCM response timer is still running), the TE 110 determines at 413 that the model / functionality response time is received within the allocated time and thus may determine at 414 that a first phase of the test is considered as PASSED. At a second alternative (Alt. 2), if the LCM response timer for the LCM action expires at 415, if the response (i.e., the notification) is not received from the DUT 120 within the allocated time (i.e., the LCM response timer expires), the TE 110 determines at 416 that the model / functionality response time is not received within the allocated time and thus may determine at 417 that a first phase of the test is considered as FAILED.

[0162] In some cases, at 418, 419, if there is no performance improvement after the AI / ML model / functionality switch after monitoring for specified / configured time duration, the TE 110 may send, at 421 , an indication to the DUT 120 to switch to Legacy positioning method. It is noted that switching to Legacy method may need RRC re-configuration or other configuration changes which shall be performed by the DUT 120 and TE 110 accordingly. At 422, the TE 110 starts a corresponding response timer and waits for the switch to legacy positioning procedure based on the DUT capability information and TE configuration. At 423, the DUT 120 performs switching to the legacy positioning procedure. At 424, the DUT 120 sends a notification to TE indicating successful switch to the legacy positioning procedure. At 425 to 427, if the notification is received from the DUT 120 within the response timer expiry, the TE 110 determines that the second phase of test may be considered as PASSED. Otherwise, the TE 110 determines that the test may be considered as FAILED. A further procedure may follow after the test mechanism.

[0163] FIG. 5 shows a flowchart of an example method 500 implemented at a first device in accordance with some example embodiments of the present disclosure. For the purpose of discussion, the method 500 will be described from the perspective of the first apparatus 301 in FIG. 3, which may be or may be comprised in the TE 110 in FIG. 1 .

[0164] At block 510, the first apparatus 301 receives, from a second apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificialintelligence / machine learning (AIZML)-based positioning.

[0165] At block 520, the first apparatus 301 transmits, to the second apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus.

[0166] At block 530, the first apparatus 301 starts a timer for the LCM action based on the capability information of the second apparatus.

[0167] At block 540, in accordance with a determination that a notification of a successful completion of the LCM action is received from the second apparatus while the timer is running, the first apparatus 301 determines that a test of the LCM action is successful.

[0168] At block 550, in accordance with a determination that a notification of a successful completion of the LCM action is not received from the second apparatus while the timer is running, the first apparatus 301 determines that a test of the LCM action is failed.

[0169] In some example embodiments, the plurality of LCM actions comprise at least one of the following: switching from a first AI / ML-based positioning functionality to a second AI / ML-based positioning functionality, switching from a first AI / ML positioning model to a second AI / ML positioning model, switching from a first positioning mechanism to a second positioning mechanism, switching from an AI / ML-based positioning functionality to a non-AI / ML-based positioning functionality.

[0170] In some example embodiments, the capability information further indicates at least one of the following: a flag indicating enablement or disablement of AI / ML-based positioning, a plurality of positioning mechanisms supported for switching within an AI / ML-based positioning functionality, a plurality of AI / ML positioning models supported for switching, or at least one type of signalling supported for an indication of the LCM action.

[0171] In some example embodiments, the respective response times for the plurality of LCM actions are determined based on at least one of the following: a latency requirement for an application that requires positioning information, a quality identifier in 5G (5QI) of an application that requires positioning information, an application type of an application that requires positioning information, a capability indicating a model switching duration, a category of a terminal devices to be positioned, or a tolerance level of degradation.

[0172] In some example embodiments, transmitting the indication of the LCM action comprises: determining a type of signaling for the indication of the LCM action based on a latency requirement for an application that requires positioning information; and transmitting, to the second apparatus, the indication of the LCM action using the type of signaling.

[0173] In some example embodiments, determining a type of signaling for the indication of the LCM action comprises: in accordance with a determination that the latency requirement for theapplication is corresponding to a low latency requirement, determining that the type of signaling to be medium access control (MAC) signaling; and in accordance with a determination that the latency requirement for the application is corresponding to a high latency requirement, determining that the type of signaling to be radio resource control (RRC) signaling.

[0174] In some example embodiments, t transmitting the indication of the LCM action comprises: in accordance with a detection of performance degradation on an AI / ML-based positioning functionality or in accordance with a reception of an indication of the performance degradation from the second apparatus, transmitting, to the second apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus.

[0175] In some example embodiments, the method 500 further comprises: transmitting, to the second apparatus, a periodicity of reporting the indication of the performance degradation.

[0176] In some example embodiments, transmitting the indication of the LCM action comprises: in accordance with a detection of no performance improvement on AI / ML-based positioning after at least one LCM action has been successfully applied by the second apparatus, transmitting, to the second apparatus, an LCM action of switching from an AI / ML-based positioning functionality to a non-AI / ML-based positioning functionality.

[0177] In some example embodiments, the first apparatus comprises test equipment, and the second apparatus comprises a device under test, and / or wherein the second apparatus is set to a test mode for the AI / ML-based positioning.

[0178] In some example embodiments, the test equipment comprises a network entity, a base station, or a terminal device, and wherein the device under test comprises a network entity, a base station, or a terminal device.

[0179] FIG. 6 shows a flowchart of an example method 600 implemented at a second device in accordance with some example embodiments of the present disclosure. For the purpose of discussion, the method 600 will be described from the perspective of the second apparatus 302 in FIG. 3, which may be or may be comprised in the DUT 120 in FIG. 1 .

[0180] At block 610, the second apparatus 302 transmits, to a first apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning.

[0181] At block 620, the second apparatus 302 receives, from the first apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus.

[0182] At block 630, upon a completion of the LCM action, the second apparatus 302 transmits, to the first apparatus, a notification of a successful completion of the LCM action.

[0183] In some example embodiments, the plurality of LCM actions comprise at least one of the following: switching from a first AI / ML-based positioning functionality to a second AI / ML-based positioning functionality, switching from a first AI / ML positioning model to a second AI / ML positioning model, switching from a first positioning mechanism to a second positioning mechanism, switching from an AI / ML-based positioning functionality to a non-AI / ML-based positioning functionality.

[0184] In some example embodiments, the capability information further indicates at least one of the following: a flag indicating enablement or disablement of AI / ML-based positioning, a plurality of positioning mechanisms supported for switching within an AI / ML-based positioning functionality, a plurality of AI / ML positioning models supported for switching, or at least one type of signalling supported for an indication of the LCM action.

[0185] In some example embodiments, the respective response times for the plurality of LCM actions are determined based on at least one of the following: a latency requirement for an application that requires positioning information, an application type of an application that requires positioning information, a capability indicating a model switching duration, a category of a terminal devices to be positioned, or a tolerance level of degradation.

[0186] In some example embodiments, receiving the indication of the LCM action comprises: receiving, from the first apparatus, the indication of the LCM action using a type of signaling, the type of signaling being determined based on a latency requirement for an application that requires positioning information.

[0187] In some example embodiments, the type of signaling is medium access control (MAC) signaling in accordance with a determination that the latency requirement for the application is corresponding to a low latency requirement, or the type of signaling is radio resource control (RRC) signaling in accordance with a determination that the latency requirement for the application is corresponding to a high latency requirement.

[0188] In some example embodiments, the method 600 further comprises: transmitting, to the first apparatus, an indication of the performance degradation on an AI / ML-based positioning functionality.

[0189] In some example embodiments, the first apparatus comprises test equipment, and the second apparatus comprises a device under test, and / or wherein the second apparatus is set to a test mode for the AI / ML-based positioning.

[0190] In some example embodiments, the test equipment comprises a network entity, a base station, or a terminal device, and wherein the device under test comprises a network entity, a base station, or a terminal device.

[0191] In some example embodiments, a first apparatus capable of performing any of the method 500 (for example, the first apparatus 301 in FIG. 3) may comprise means for performing the respective operations of the method 500. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module. The first apparatus may be implemented as or included in the first apparatus 301 in FIG. 3.

[0192] In some example embodiments, the first apparatus comprises means for receiving, from a second apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; means for transmitting, to the second apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; means for starting a timer for the LCM action based on the capability information of the second apparatus; means for, in accordance with a determination that a notification of a successful completion of the LCM action is received from the second apparatus while the timer is running, determining that a test of the LCM action is successful; and means for, in accordance with a determination that a notification of a successful completion of the LCM action is not received from the second apparatus while the timer is running, determining that a test of the LCM action is failed.

[0193] In some example embodiments, the plurality of LCM actions comprise at least one of the following: switching from a first AI / ML-based positioning functionality to a second AI / ML-based positioning functionality, switching from a first AI / ML positioning model to a second AI / ML positioning model, switching from a first positioning mechanism to a second positioning mechanism, switching from an AI / ML-based positioning functionality to a non-AI / ML-based positioning functionality.

[0194] In some example embodiments, the capability information further indicates at least one of the following: a flag indicating enablement or disablement of AI / ML-based positioning, a plurality of positioning mechanisms supported for switching within an AI / ML-based positioning functionality, a plurality of AI / ML positioning models supported for switching, or at least one type of signalling supported for an indication of the LCM action.

[0195] In some example embodiments, the respective response times for the plurality of LCM actions are determined based on at least one of the following: a latency requirement for an application that requires positioning information, a quality identifier in 5G (5QI) of an application that requires positioning information, an application type of an application that requires positioning information, a capability indicating a model switching duration, a category of a terminal devices to be positioned, or a tolerance level of degradation.

[0196] In some example embodiments, the means for transmitting the indication of the LCM action comprises: means for determining a type of signaling for the indication of the LCM action based on a latency requirement for an application that requires positioning information; and means for transmitting, to the second apparatus, the indication of the LCM action using the type of signaling.

[0197] In some example embodiments, the means for determining a type of signaling for the indication of the LCM action comprises: means for, in accordance with a determination that the latency requirement for the application is corresponding to a low latency requirement, determining that the type of signaling to be medium access control (MAC) signaling; and means for, in accordance with a determination that the latency requirement for the application is corresponding to a high latency requirement, determining that the type of signaling to be radio resource control (RRC) signaling.

[0198] In some example embodiments, the means for transmitting the indication of the LCM action comprises: means for, in accordance with a detection of performance degradation on an AI / ML-based positioning functionality or in accordance with a reception of an indication of the performance degradation from the second apparatus, transmitting, to the second apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus.

[0199] In some example embodiments, the first apparatus further comprises: means for transmitting, to the second apparatus, a periodicity of reporting the indication of the performance degradation.

[0200] In some example embodiments, the means for transmitting an indication of an LCM action comprises: means for, in accordance with a detection of no performance improvement on AI / ML-based positioning after at least one LCM action has been successfully applied by the second apparatus, transmitting, to the second apparatus, an LCM action of switching from an AI / ML-based positioning functionality to a non-AI / ML-based positioning functionality.

[0201] In some example embodiments, the first apparatus comprises test equipment, and the second apparatus comprises a device under test, and / or wherein the second apparatus is set to a test mode for the AI / ML-based positioning.

[0202] In some example embodiments, the test equipment comprises a network entity, a base station, or a terminal device, and wherein the device under test comprises a network entity, a base station, or a terminal device.

[0203] In some example embodiments, the first apparatus further comprises means for performing other operations in some example embodiments of the method 500 or the first apparatus 301 . In some example embodiments, the means comprises at least one processor; andat least one memory storing instructions that, when executed by the at least one processor, cause the performance of the first apparatus.

[0204] In some example embodiments, a second apparatus capable of performing any of the method 600 (for example, the second apparatus 302 in FIG. 3) may comprise means for performing the respective operations of the method 600. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module. The second apparatus may be implemented as or included in the second apparatus 302 in FIG. 3.

[0205] In some example embodiments, the second apparatus comprises means for transmitting, to a first apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; means for receiving, from the first apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; and means for upon a completion of the LCM action, transmitting, to the first apparatus, a notification of a successful completion of the LCM action.

[0206] In some example embodiments, the plurality of LCM actions comprise at least one of the following: switching from a first AI / ML-based positioning functionality to a second AI / ML-based positioning functionality, switching from a first AI / ML positioning model to a second AI / ML positioning model, switching from a first positioning mechanism to a second positioning mechanism, switching from an AI / ML-based positioning functionality to a non-AI / ML-based positioning functionality.

[0207] In some example embodiments, the capability information further indicates at least one of the following: a flag indicating enablement or disablement of AI / ML-based positioning, a plurality of positioning mechanisms supported for switching within an AI / ML-based positioning functionality, a plurality of AI / ML positioning models supported for switching, or at least one type of signalling supported for an indication of the LCM action.

[0208] In some example embodiments, the respective response times for the plurality of LCM actions are determined based on at least one of the following: a latency requirement for an application that requires positioning information, an application type of an application that requires positioning information, a capability indicating a model switching duration, a category of a terminal devices to be positioned, or a tolerance level of degradation.

[0209] In some example embodiments, the means for receiving the indication of the LCM action comprises: means for receiving, from the first apparatus, the indication of the LCM action using a type of signaling, the type of signaling being determined based on a latency requirement for an application that requires positioning information.

[0210] In some example embodiments, the type of signaling is medium access control (MAC) signaling in accordance with a determination that the latency requirement for the application is corresponding to a low latency requirement, or the type of signaling is radio resource control (RRC) signaling in accordance with a determination that the latency requirement for the application is corresponding to a high latency requirement.

[0211] In some example embodiments, the second apparatus further comprises: means for transmitting, to the first apparatus, an indication of the performance degradation on an AI / ML- based positioning functionality.

[0212] In some example embodiments, the first apparatus comprises test equipment, and the second apparatus comprises a device under test, and / or wherein the second apparatus is set to a test mode for the AI / ML-based positioning.

[0213] In some example embodiments, the test equipment comprises a network entity, a base station, or a terminal device, and wherein the device under test comprises a network entity, a base station, or a terminal device.

[0214] In some example embodiments, the second apparatus further comprises means for performing other operations in some example embodiments of the method 600 or the second apparatus 302. In some example embodiments, the means comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the performance of the second apparatus.

[0215] FIG. 7 is a simplified block diagram of a device 700 that is suitable for implementing example embodiments of the present disclosure. The device 700 may be provided to implement a communication device, for example, the first apparatus 301 or the second apparatus 302 as shown in FIG. 3 or the TE 110 or DUT 120 in FIG. 1. As shown, the device 700 includes one or more processors 710, one or more memories 720 coupled to the processor 710, and one or more communication modules 740 coupled to the processor 710.

[0216] The communication module 740 is for bidirectional communications. The communication module 740 has one or more communication interfaces to facilitate communication with one or more other modules or devices. The communication interfaces may represent any interface that is necessary for communication with other network elements. In some example embodiments, the communication module 740 may include at least one antenna.

[0217] The processor 710 may be of any type suitable to the local technical network and may include one or more of the following: general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on multicore processor architecture, as non-limiting examples. The device 700 may have multiple processors, such as anapplication specific integrated circuit chip that is slaved in time to a clock which synchronizes the main processor.

[0218] The memory 720 may include one or more non-volatile memories and one or more volatile memories. Examples of the non-volatile memories include, but are not limited to, a Read Only Memory (ROM) 724, an electrically programmable read only memory (EPROM), a flash memory, a hard disk, a compact disc (CD), a digital video disk (DVD), an optical disk, a laser disk, and other magnetic storage and / or optical storage. Examples of the volatile memories include, but are not limited to, a random access memory (RAM) 722 and other volatile memories that will not last in the power-down duration.

[0219] A computer program 730 includes computer executable instructions that are executed by the associated processor 710. The instructions of the program 730 may include instructions for performing operations / acts of some example embodiments of the present disclosure. The program 730 may be stored in the memory, e.g., the ROM 724. The processor 710 may perform any suitable actions and processing by loading the program 730 into the RAM 722.

[0220] The example embodiments of the present disclosure may be implemented by means of the program 730 so that the device 700 may perform any process of the disclosure as discussed with reference to FIG. 3 to FIG. 6. The example embodiments of the present disclosure may also be implemented by hardware or by a combination of software and hardware.

[0221] In some example embodiments, the program 730 may be tangibly contained in a computer readable medium which may be included in the device 700 (such as in the memory 720) or other storage devices that are accessible by the device 700. The device 700 may load the program 730 from the computer readable medium to the RAM 722 for execution. In some example embodiments, the computer readable medium may include any types of non-transitory storage medium, such as ROM, EPROM, a flash memory, a hard disk, CD, DVD, and the like. The term “non-transitory,” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM).

[0222] FIG. 8 shows an example of the computer readable medium 800 which may be in form of CD, DVD or other optical storage disk. The computer readable medium 800 has the program 730 stored thereon.

[0223] Generally, various embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects may be implemented in hardware, and other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device. Although various aspects of embodiments of the present disclosure are illustrated and described as blockdiagrams, flowcharts, or using some other pictorial representations, it is to be understood that the block, apparatus, system, technique or method described herein may be implemented in, as nonlimiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0224] Some example embodiments of the present disclosure also provide at least one computer program product tangibly stored on a computer readable medium, such as a non- transitory computer readable medium. The computer program product includes computerexecutable instructions, such as those included in program modules, being executed in a device on a target physical or virtual processor, to carry out any of the methods as described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, or the like that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. Machine-executable instructions for program modules may be executed within a local or distributed device. In a distributed device, program modules may be located in both local and remote storage media.

[0225] Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0226] In the context of the present disclosure, the computer program code or related data may be carried by any suitable carrier to enable the device, apparatus or processor to perform various processes and operations as described above. Examples of the carrier include a signal, computer readable medium, and the like.

[0227] The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable medium may include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0228] Further, although operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, although several specific implementation details are contained in the above discussions, these should not be construed as limitations on the scope of the present disclosure, but rather as descriptions of features that may be specific to particular embodiments. Unless explicitly stated, certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, unless explicitly stated, various features that are described in the context of a single embodiment may also be implemented in a plurality of embodiments separately or in any suitable sub-combination.

[0229] Although the present disclosure has been described in languages specific to structural features and / or methodological acts, it is to be understood that the present disclosure defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

WHAT IS CLAIMED IS:1 . A first apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the first apparatus at least to: receive, from a second apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; transmit, to the second apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; start a timer for the LCM action based on the capability information of the second apparatus; in accordance with a determination that a notification of a successful completion of the LCM action is received from the second apparatus while the timer is running, determine that a test of the LCM action is successful; and in accordance with a determination that a notification of a successful completion of the LCM action is not received from the second apparatus while the timer is running, determine that a test of the LCM action is failed.

2. The first apparatus of claim 1 , wherein the plurality of LCM actions comprise at least one of the following: switching from a first AI / ML-based positioning functionality to a second AI / ML-based positioning functionality, switching from a first AI / ML positioning model to a second AI / ML positioning model, switching from a first positioning mechanism to a second positioning mechanism, switching from an AI / ML-based positioning functionality to a non-AI / ML-based positioning functionality.

3. The first apparatus of claim 1 or 2, wherein the capability information further indicates at least one of the following: a flag indicating enablement or disablement of AI / ML-based positioning, a plurality of positioning mechanisms supported for switching within an AI / ML-based positioning functionality,a plurality of AI / ML positioning models supported for switching, or at least one type of signalling supported for an indication of the LCM action.

4. The first apparatus of any of claims 1 to 3, wherein the respective response times for the plurality of LCM actions are determined based on at least one of the following: a latency requirement for an application that requires positioning information, a quality identifier in 5G (5QI) of an application that requires positioning information, an application type of an application that requires positioning information, a capability indicating a model switching duration, a category of a terminal devices to be positioned, or a tolerance level of degradation.

5. The first apparatus of any of claims 1 to 4, wherein the first apparatus is caused to: determine a type of signaling for the indication of the LCM action based on a latency requirement for an application that requires positioning information; and transmit, to the second apparatus, the indication of the LCM action using the type of signaling.

6. The first apparatus of claim 5, wherein the first apparatus is caused to: in accordance with a determination that the latency requirement for the application is corresponding to a low latency requirement, determine that the type of signaling to be medium access control (MAC) signaling; and in accordance with a determination that the latency requirement for the application is corresponding to a high latency requirement, determine that the type of signaling to be radio resource control (RRC) signaling.

7. The first apparatus of any of claims 1 to 6, wherein the first apparatus is caused to: in accordance with a detection of performance degradation on an AI / ML-based positioning functionality or in accordance with a reception of an indication of the performance degradation from the second apparatus, transmit, to the second apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus.

8. The first apparatus of claim 7, wherein the first apparatus is further caused to: transmit, to the second apparatus, a periodicity of reporting the indication of the performance degradation.

9. The first apparatus of any of claims 1 to 8, wherein the first apparatus is further caused to: in accordance with a detection of no performance improvement on AI / ML-based positioning after at least one LCM action has been successfully applied by the second apparatus, transmit, to the second apparatus, an LCM action of switching from an AI / ML-based positioning functionality to a non-AI / ML- based positioning functionality.

10. The first apparatus of any of claims 1 to 9, wherein the first apparatus comprises test equipment, and the second apparatus comprises a device under test, and / or wherein the second apparatus is set to a test mode for the AI / ML-based positioning.

11. The first apparatus of claim 10, wherein the test equipment comprises a network entity, a base station, or a terminal device, and wherein the device under test comprises a network entity, a base station, or a terminal device.

12. A second apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the second apparatus at least to: transmit, to a first apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; receive, from the first apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; and upon a completion of the LCM action, transmit, to the first apparatus, a notification of a successful completion of the LCM action.

13. The second apparatus of claim 12, wherein the plurality of LCM actions comprise at least one of the following: switching from a first AI / ML-based positioning functionality to a second AI / ML-based positioning functionality,switching from a first AI / ML positioning model to a second AI / ML positioning model, switching from a first positioning mechanism to a second positioning mechanism, switching from an AI / ML-based positioning functionality to a non-AI / ML-based positioning functionality.

14. The second apparatus of claim 12 or 13, wherein the capability information further indicates at least one of the following: a flag indicating enablement or disablement of AI / ML-based positioning, a plurality of positioning mechanisms supported for switching within an AI / ML-based positioning functionality, a plurality of AI / ML positioning models supported for switching, or at least one type of signalling supported for an indication of the LCM action.

15. The second apparatus of any of claims 12 to 14, wherein the respective response times for the plurality of LCM actions are determined based on at least one of the following: a latency requirement for an application that requires positioning information, an application type of an application that requires positioning information, a capability indicating a model switching duration, a category of a terminal devices to be positioned, or a tolerance level of degradation.

16. The second apparatus of any of claims 12 to 15, wherein the second apparatus is caused to: receive, from the first apparatus, the indication of the LCM action using a type of signaling, the type of signaling being determined based on a latency requirement for an application that requires positioning information.

17. The second apparatus of claim 16, wherein the type of signaling is medium access control (MAC) signaling in accordance with a determination that the latency requirement for the application is corresponding to a low latency requirement, or the type of signaling is radio resource control (RRC) signaling in accordance with a determination that the latency requirement for the application is corresponding to a high latency requirement.

18. The second apparatus of any of claims 12 to 17, wherein the second apparatus is further caused to:transmit, to the first apparatus, an indication of the performance degradation on an AI / ML-based positioning functionality.

19. The second apparatus of any of claims 12 to 18, wherein the first apparatus comprises test equipment, and the second apparatus comprises a device under test, and / or wherein the second apparatus is set to a test mode for the AI / ML-based positioning.

20. The second apparatus of claim 19, wherein the test equipment comprises a network entity, a base station, or a terminal device, and wherein the device under test comprises a network entity, a base station, or a terminal device.

21. A method comprising: receiving, by a first apparatus and from a second apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of lifecycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; transmitting, to the second apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; starting a timer for the LCM action based on the capability information of the second apparatus; in accordance with a determination that a notification of a successful completion of the LCM action is received from the second apparatus while the timer is running, determining that a test of the LCM action is successful; and in accordance with a determination that a notification of a successful completion of the LCM action is not received from the second apparatus while the timer is running, determining that a test of the LCM action is failed.

22. A method comprising: transmitting, to a first apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; receiving, from the first apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; and upon a completion of the LCM action, transmitting, to the first apparatus, a notification of a successful completion of the LCM action.

23. A first apparatus comprising: means for receiving, from a second apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; means for transmitting, to the second apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; means for starting a timer for the LCM action based on the capability information of the second apparatus; means for, in accordance with a determination that a notification of a successful completion of the LCM action is received from the second apparatus while the timer is running, determining that a test of the LCM action is successful; and means for, in accordance with a determination that a notification of a successful completion of the LCM action is not received from the second apparatus while the timer is running, determining that a test of the LCM action is failed.

24. A second apparatus comprising: means for transmitting, to a first apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of life-cycle management (LCM) actions for artificial intelligence / machine learning (AIZML)-based positioning; means for receiving, from the first apparatus, an indication of an LCM action of the plurality of LCM actions to be applied by the second apparatus; and means for upon a completion of the LCM action, transmitting, to the first apparatus, a notification of a successful completion of the LCM action.

25. A computer readable medium comprising instructions stored thereon for causing an apparatus at least to perform the method according to claim 21 , or the method according to claim 22.

Images