Latency requirement and testing for artificial intelligence / machine learning based beam management

Abstract

Example embodiments of the present disclosure are related to methods, devices, apparatuses and computer readable storage medium for latency requirement and testing for 1cm procedures in AI / ML based beam management. In a method, a first apparatus transmits, to a second apparatus, an indication of an action for switching from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model. The first apparatus monitors a report about completion of the action within a time duration associated with a latency requirement for the action. The first apparatus determines a result of a test for the action based on a result of the monitoring.

Description

LATENCY REQUIREMENT AND TESTING FOR ARTIFICIAL INTELLIGENCE / MACHINE LEARNING BASED BEAM MANAGEMENTCROSS-REFERENCE TO RELATED APPLICATION

[0001] The present application claims priority to, and the benefit of, GB Application No. 2402294.9, filed February 19, 2024, the contents of which are herewith incorporated by reference in their entirety.FIELDS

[0002] Various example embodiments of the present disclosure generally relate to the field of telecommunication and in particular, to methods, devices, apparatuses and computer readable storage medium for latency requirement and testing for 1cm procedures in AI / ML based beam management.BACKGROUND

[0003] In the telecommunication industry, artificial intelligence / machine learning (AI / ML) have been employed in telecommunication systems to improve the performance. The 3rd Generation Partnership Project (3GPP) Release-18 started the study on AI / ML for New Radio (NR) air interface. For example, to solve problems such as how to improve the performance of air-interface functions with AI / ML techniques, what would be required to enable AI / ML techniques for the air interface, and so on.

[0004] AI / ML-based beam management targets spatial and / or time beam prediction for overhead and latency reduction. The beam management includes, for example, beam prediction in time and / or spatial domain for overhead and latency reduction, beam selection accuracy improvement, and the like. Use cases for the beam management are further studied for spatial and / or time beam prediction. The scope of spatial beam prediction (BM-Casel) is to predict the best Tx / Rx beams in different spatial locations. Conversely, time-domain beam predictions (BM-Case2) aim to predict the most likely beam to use for next time instants, e.g., beam prediction in the spatial domain (BM-Casel).SUMMARY

[0005] In a first aspect of the present disclosure, there is provided a first apparatus. The first apparatus comprises at least one processor; and at least one memory storinginstructions that, when executed by the at least one processor, cause the first apparatus at least to: transmit, to a second apparatus, an indication of an action for switching from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; monitor a report about completion of the action within a time duration, the time duration being associated with a latency requirement for the action; and determine a result of a test for the action based on a result of the monitoring.

[0006] In a second aspect of the present disclosure, there is provided a second apparatus. The second apparatus comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the second apparatus at least to: receive, from a first apparatus, an indication of an action for switching from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; transmit, to the first apparatus, a report about completion of the action for determining a result of a test for the action based on a time duration.

[0007] In a third aspect of the present disclosure, there is provided a method. The method comprises: transmitting, to a second apparatus, an indication of an action for switching from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; monitoring a report about completion of the action within a time duration, the time duration being associated with a latency requirement for the action; and determining a result of a test for the action based on a result of the monitoring.

[0008] In a fourth aspect of the present disclosure, there is provided a method. The method comprises: receiving, from a first apparatus, an indication of an action for switching from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; transmitting, to the first apparatus, a report about completion of the action for determining a result of a test for the action based on a time duration.

[0009] In a fifth aspect of the present disclosure, there is provided a first apparatus. The first apparatus comprises means for transmitting, to a second apparatus, an indication of an action for switching from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; means for monitoring a report about completion of the action within a time duration, the time duration being associated with a latency requirement for the action; andmeans for determining a result of a test for the action based on a result of the monitoring.

[0010] In a sixth aspect of the present disclosure, there is provided a second apparatus. The second apparatus comprises means for receiving, from a first apparatus, an indication of an action for switching from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; means for transmitting, to the first apparatus, a report about completion of the action for determining a result of a test for the action based on a time duration.

[0011] In a seventh aspect of the present disclosure, there is provided a computer readable medium. The computer readable medium comprises instructions stored thereon for causing an apparatus to perform at least the method according to the third aspect.

[0012] In an eighth aspect of the present disclosure, there is provided a computer readable medium. The computer readable medium comprises instructions stored thereon for causing an apparatus to perform at least the method according to the fourth aspect.

[0013] It is to be understood that the Summary section is not intended to identify key or essential features of embodiments of the present disclosure, nor is it intended to be used to limit the scope of the present disclosure. Other features of the present disclosure will become easily comprehensible through the following description.BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Some example embodiments will now be described with reference to the accompanying drawings, where:

[0015] FIG. 1 illustrates an example communication environment in which example embodiments of the present disclosure can be implemented;

[0016] FIG. 2 illustrates an example architecture of a functional framework for an AI / ML functionality;

[0017] FIG. 3 illustrates a signaling flow for a life-cycle management (LCM) procedure in AI / ML beam management in accordance with some example embodiments of the present disclosure;

[0018] FIG. 4 illustrates a detailed signaling flow for a LCM procedure in AI / ML beam management in accordance with some further example embodiments of the present disclosure;

[0019] FIG. 5 illustrates a detailed signaling flow for a LCM procedure in AI / ML beam management in accordance with some further example embodiments of the present disclosure;

[0020] FIG. 6 illustrates a flowchart of a method implemented at a first apparatus according to some example embodiments of the present disclosure;

[0021] FIG. 7 illustrates a flowchart of a method implemented at a second apparatus according to some example embodiments of the present disclosure;

[0022] FIG. 8 illustrates a simplified block diagram of a device that is suitable for implementing example embodiments of the present disclosure; and

[0023] Throughout the drawings, the same or similar reference numerals represent the same or similar element.DETAILED DESCRIPTION

[0024] Principle of the present disclosure will now be described with reference to some example embodiments. It is to be understood that these embodiments are described only for the purpose of illustration and help those skilled in the art to understand and implement the present disclosure, without suggesting any limitation as to the scope of the disclosure. Embodiments described herein can be implemented in various manners other than the ones described below.

[0025] In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skills in the art to which this disclosure belongs.

[0026] References in the present disclosure to “one embodiment,” “an embodiment,” “an example embodiment,” and the like indicate that the embodiment described may include a particular feature, structure, or characteristic, but it is not necessary that every embodiment includes the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0027] It shall be understood that although the terms “first,” “second,”..., etc. in front of noun(s) and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another and they do not limit the order of the noun(s). For example, a first element could be termed a second element, and similarly, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, theterm “and / or” includes any and all combinations of one or more of the listed terms.

[0028] As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or”, mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements.

[0029] As used herein, unless stated explicitly, performing a step “in response to A” does not indicate that the step is performed immediately after “A” occurs and one or more intervening steps may be included.

[0030] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises”, “comprising”, “has”, “having”, “includes” and / or “including”, when used herein, specify the presence of stated features, elements, and / or components etc., but do not preclude the presence or addition of one or more other features, elements, components and / or combinations thereof.

[0031] As used in this application, the term “circuitry” may refer to one or more or all of the following:(a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry) and(b) combinations of hardware circuits and software, such as (as applicable):(i) a combination of analog and / or digital hardware circuit(s) with software / firmware and(ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and(c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.

[0032] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (ormultiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

[0033] As used herein, the term “communication network” refers to a network following any suitable communication standards, such as New Radio (NR), Long Term Evolution (LTE), LTE-Advanced (LTE-A), Wideband Code Division Multiple Access (WCDMA), High-Speed Packet Access (HSPA), Narrow Band Internet of Things (NB-IoT) and so on. Furthermore, the communications between a terminal device and a network device in the communication network may be performed according to any suitable generation communication protocols, including, but not limited to, the first generation (1G), the second generation (2G), 2.5G, 2.75G, the third generation (3G), the fourth generation (4G), 4.5G, the fifth generation (5G), the sixth generation (6G) communication protocols, and / or any other protocols either currently known or to be developed in the future. Embodiments of the present disclosure may be applied in various communication systems. Given the rapid development in communications, there will of course also be future type communication technologies and systems with which the present disclosure may be embodied. It should not be seen as limiting the scope of the present disclosure to only the aforementioned system.

[0034] As used herein, the term “network device” refers to a node in a communication network via which a terminal device accesses the network and receives services therefrom. The network device may refer to a base station (BS) or an access point (AP), for example, a node B (NodeB or NB), an evolved NodeB (eNodeB or eNB), an NR NB (also referred to as a gNB), a Remote Radio Unit (RRU), a radio header (RH), a remote radio head (RRH), a relay, an Integrated Access and Backhaul (IAB) node, a low power node such as a femto, a pico, a non-terrestrial network (NTN) or non-ground network device such as a satellite network device, a low earth orbit (LEO) satellite and a geosynchronous earth orbit (GEO) satellite, an aircraft network device, and so forth, depending on the applied terminology and technology. In some example embodiments, radio access network (RAN) split architecture comprises a Centralized Unit (CU) and a Distributed Unit (DU) at an IAB donor node. An IAB node comprises a Mobile Terminal (IAB-MT) part that behaves like a UE toward the parent node, and a DU part of an IAB node behaves like a base station toward the next-hop IAB node.

[0035] The term “terminal device” refers to any end device that may be capable of wireless communication. By way of example rather than limitation, a terminal device may also be referred to as a communication device, user equipment (UE), a Subscriber Station (SS), a Portable Subscriber Station, a Mobile Station (MS), or an Access Terminal (AT). The terminal device may include, but not limited to, a mobile phone, a cellular phone, a smart phone, voice over IP (VoIP) phones, wireless local loop phones, a tablet, a wearable terminal device, a personal digital assistant (PDA), portable computers, desktop computer, image capture terminal devices such as digital cameras, gaming terminal devices, music storage and playback appliances, vehicle-mounted wireless terminal devices, wireless endpoints, mobile stations, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), USB dongles, smart devices, wireless customer-premises equipment (CPE), an Internet of Things (loT) device, a watch or other wearable, a head-mounted display (HMD), a vehicle, a drone, a medical device and applications (e.g., remote surgery), an industrial device and applications (e.g., a robot and / or other wireless devices operating in an industrial and / or an automated processing chain contexts), a consumer electronics device, a device operating on commercial and / or industrial wireless networks, and the like. The terminal device may also correspond to a Mobile Termination (MT) part of an IAB node (e.g., a relay node). In the following description, the terms “terminal device”, “communication device”, “terminal”, “user equipment” and “UE” may be used interchangeably.

[0036] As used herein, the term “resource,” “transmission resource,” “resource block,” “physical resource block” (PRB), “uplink resource,” or “downlink resource” may refer to any resource for performing a communication, for example, a communication between a terminal device and a network device, such as a resource in time domain, a resource in frequency domain, a resource in space domain, a resource in code domain, or any other combination of the time, frequency, space and / or code domain resource enabling a communication, and the like. In the following, unless explicitly stated, a resource in both frequency domain and time domain will be used as an example of a transmission resource for describing some example embodiments of the present disclosure. It is noted that example embodiments of the present disclosure are equally applicable to other resources in other domains.

[0037] As used herein, the term “model” is referred to as an association between an input and an output learned from training data, and thus a corresponding output may be generated for a given input after the training. The generation of the model may be basedon machine learning (ML) techniques. The machine learning techniques may also be referred to as artificial intelligence (Al) techniques. In general, a machine learning model can be built, which receives input information and makes predictions based on the input information. For example, a classification model may predict a class of the input information among a predetermined set of classes. As used herein, “model” may also be referred to as “machine learning model”, “learning model”, “machine learning network”, or “learning network,” which are used interchangeably herein. Supervised learning refers to a process of training a model from input and its corresponding labels. The trained model is then used to infer the output.

[0038] Generally, model lifecycle management may usually include three stages, i.e., a training stage, a validation stage, and an application stage (also referred to as an inference stage). At the training stage, a given AI / ML model may be trained (or optimized) iteratively using a great amount of training data until the model can make inference close to desired outputs in the training or labelled dataset. During the training, a set of parameter values of the model is iteratively updated until a training objective is reached. Through the training process, the AI / ML model may be regarded as being capable of learning the association between the input and the output (also referred to an input-output mapping) from the training data. At the validation stage, a validation input is applied to the trained AI / ML model to test whether the model can provide a correct output, so as to determine the performance of the model. Generally, the validation stage may be considered as a step in a training process, or may be omitted in some cases. At the inference stage, the resulting AI / ML model may be used to process a real-world model input based on the trained model obtained from the training process and to determine the corresponding model output. In some cases, a retraining or updating stage may be included in the model lifecycle management, to enable the model evolved to have better performance.

[0039] To facilitate understanding of the terminologies, some definitions of the list of terminologies used for AI / ML are provided below.

[0040] AI / ML Model: A data driven algorithm that applies AI / ML techniques to generate a set of outputs based on a set of inputs.

[0041] AI / ML model delivery: A generic term referring to delivery of an AI / ML model from one entity to another entity in any manner. Note: An entity could mean a network node / function (e.g., gNB, location management function (LMF), etc.), UE, proprietary server, etc.

[0042] AI / ML model Inference: A process of using a trained AI / ML model to producea set of outputs based on a set of inputs.

[0043] AI / ML model testing: A subprocess of training, to evaluate the performance of a final AI / ML model using a dataset different from one used for model training and validation. Differently from AI / ML model validation, testing does not assume subsequent tuning of the model.

[0044] AI / ML model training: A process to train an AI / ML Model [by learning the input / output relationship] in a data driven manner and obtain the trained AI / ML Model for inference.

[0045] AI / ML model transfer: Delivery of an AI / ML model over the air interface in a manner that is not transparent to 3GPP signalling, either parameters of a model structure known at the receiving end or a new model with parameters. Delivery may contain a full model or a partial model.

[0046] AI / ML model validation: A subprocess of training, to evaluate the quality of an AI / ML model using a dataset different from one used for model training, that helps selecting model parameters that generalize beyond the dataset used for model training.

[0047] Data collection: A process of collecting data by the network nodes, management entity, or UE for the purpose of AI / ML model training, data analytics and inference.

[0048] Federated learning / federated training: A machine learning technique that trains an AI / ML model across multiple decentralized edge nodes (e.g., UEs, gNBs) each performing local model training using local data samples. The technique requires multiple interactions of the model, but no exchange of local data samples.

[0049] Functionality identification: A process / method of identifying an AI / ML functionality for the common understanding between the network and the UE. Note: Information regarding the AI / ML functionality may be shared during functionality identification. Where AI / ML functionality resides depends on the specific use cases and sub use cases.

[0050] Model activation: enable an AI / ML model for a specific function.

[0051] Model deactivation: disable an AI / ML model for a specific function.

[0052] Model download: Model transfer from the network to UE.

[0053] Model identification: A process / method of identifying an AI / ML model for the common understanding between the network (NW) and the UE. Note: The process / method of model identification may or may not be applicable. Note: Information regarding the AI / ML model may be shared during model identification.

[0054] Model monitoring: A procedure that monitors the inference performance of theAI / ML model.

[0055] Model parameter update: Process of updating the model parameters of a model.

[0056] Model selection: The process of selecting an AI / ML model for activation among multiple models for the same AI / ML enabled feature. Note: Model selection may or may not be carried out simultaneously with model activation.

[0057] Model switching: Deactivating a currently active AI / ML model and activating a different AI / ML model for a specific function.

[0058] Model update: Process of updating the model parameters and / or model structure of a model.

[0059] Model upload: Model transfer from UE to the network.

[0060] Network-side (AI / ML) model: An AI / ML Model whose inference is performed entirely at the network.

[0061] Offline field data: The data collected from field and used for offline training of the AI / ML model.

[0062] Offline training: An AI / ML training process where the model is trained based on collected dataset, and where the trained model is later used or delivered for inference. Note: This definition only serves as a guidance. There may be cases that may not exactly conform to this definition but could still be categorized as offline training by commonly accepted conventions.

[0063] Online field data: The data collected from field and used for online training of the AI / ML model.

[0064] Online training: An AI / ML training process where the model being used for inference) is (typically continuously) trained in (near) real-time with the arrival of new training samples. Note: the notion of (near) real-time vs. non real-time is context- dependent and is relative to the inference time-scale. Note: This definition only serves as a guidance. There may be cases that may not exactly conform to this definition but could still be categorized as online training by commonly accepted conventions. Note: Fine- tuning / re-training may be done via online or offline training. (This note could be removed when we define the term fine-tuning.)

[0065] Reinforcement Learning (RL): A process of training an AI / ML model from input (a.k.a. state) and a feedback signal (a.k.a. reward) resulting from the model’s output (a.k.a. action) in an environment the model is interacting with.

[0066] Semi-supervised learning: A process of training a model with a mix of labelled data and unlabelled data.

[0067] Two-sided (AI / ML) model: A paired AI / ML Model(s) over which joint inference is performed, where joint inference comprises AI / ML Inference whose inference is performed jointly across the UE and the network, i.e, the first part of inference is firstly performed by UE and then the remaining part is performed by the gNB, or vice versa.

[0068] UE-side (AI / ML) model: An AI / ML Model whose inference is performed entirely at the UE.

[0069] Unsupervised learning: A process of training a model without labelled data.

[0070] Proprietary-format models: ML models of vendor- / device-specific proprietary format, from 3GPP perspective. They are not mutually recognizable across vendors and hide model design information from other vendors when shared. Note: An example is a device-specific binary executable format.

[0071] Open-format models: ML models of specified format that are mutually recognizable across vendors and allow interoperability, from the 3 GPP perspective. They are mutually recognizable between vendors and do not hide model design information from other vendors when shared.

[0072] Model identification: A process / method of identifying an AI / ML model for the common understanding between the network device and the UE. Note: The process / method of model identification may or may not be applicable. Note: Information regarding the AI / ML model may be shared during model identification.

[0073] Functionality identification: A process / method of identifying an AI / ML functionality for the common understanding between the network device and the UE. Note: information regarding the AI / ML functionality may be shared during functionality identification.

[0074] FIG. 1 illustrates an example communication environment 100 in which example embodiments of the present disclosure can be implemented. It is to be understood that the elements shown in the communication environment 100 are intended to represent main functions provided within the system. As such, the blocks shown in FIG. 1 refer to specific elements in communication networks that provide these main functions. However, other network elements may be used to implement some or all of the main functions represented. Also, it is to be understood that not all functions of a communication network are depicted in FIG. 1. Rather, functions that facilitate an explanation of illustrative embodiments are represented. Further, the number of the elements shown in FIG. 1 is also for the purpose of illustration only and there may be any number of elements.

[0075] As shown, the communication environment 100 comprises a plurality ofcommunication devices, including testing equipment (TE) 110, one or more devices under test (DUTs) 120-1, 120-2, ..., 120-N (collectively or individually referred to as DUTs 120) and one or more TRPs 130-1, 130-2 etc. (collectively or individually referred to as TRPs 130). A serving area of the TRP 130 may be called a cell. The DUTs 120 may perform signal transmission and reception with the TRPs 130.

[0076] In some example embodiments, one or more AI / ML models 125-1, 125-2, ... , 125-N (collectively or individually referred to as AI / ML models 125) may be used by the one or more DUTs 120. An AI / ML model 125 may sometimes be referred to as an Al model or an ML model for short. Different AI / ML models 125 may be configured to implement the same different algorithms in the communication environment 100. The AI / ML model 125 used by a DUT 120 may sometimes include to either a model or an AI / ML functionality.

[0077] In some example embodiments, the AI / ML models 125 are configured for AI / ML based beam management or AI / ML enabled beam management. AI / ML enabled beam management is one of the selected use-cases for the study item in the development of communication networks.

[0078] In some example embodiments of the present disclosure, TE 110 is configured to test the AI / ML model(s) 125 used by the DUTs 120. In some example embodiments, TE 110 may be a network entity in the core network, a base station (e.g., gNB, eNB) in RAN, or may be a terminal device (e.g., UE) or any other device that is configured for AI / ML testing.

[0079] In some example embodiments, a DUT 120 may be a terminal device (e.g., UE) that uses the AI / ML model 125 for beam management purpose. In some example embodiments, a DUT 120 may also be a network entity in the core network or a base station (e.g., gNB, eNB) in RAN which is configured to uses the AI / ML model 125 for beam management purpose.

[0080] Study and research are made to support a new artificial intelligence (AI) / machine learning (ML)-enabled radio interface for the next cellular systems. AI / ML- based beam management targets spatial and / or time beam prediction for overhead and latency reduction.

[0081] In AI / ML beam management, the beam measurements are used as input to ML model are called as Set B beams and the output of ML model is represented by Set A beams. The measured beams, e.g., Set B may be provided with the CSI reporting configurations associated with AI / ML enabled beam management.

[0082] When the UE performs beam prediction, it is understood that the network (NW) shall do the performance monitoring such that it can configure the UE to switch to other functionality / model or configure the UE to switch back to legacy. In such cases, the NW may configure the UE to derive performance monitoring metrics, KPIs or event based performance that represent ML model performance and ask UE to report monitoring metrics or monitoring outcome time to time (aperiodic or long periodic) and then NW makes decision. When the NW performs the beam prediction, the NW needs measurement reporting, for both model inference and performance monitoring purposes.

[0083] The study should also identify areas where AI / ML could improve the performance of air-interface functions. Specification impact will be assessed to improve the overall understanding of what would be required to enable AI / ML techniques for the air interface. The beam management use case is further studied for spatial and / or time beam prediction. The scope of spatial beam prediction (BM-Casel) is to predict the best Tx / Rx beams in different spatial locations. Conversely, time-domain beam predictions (BM-Case2) aim to predict the most likely beam to use for next time instants, e.g., beam prediction in the spatial domain (BM-Casel).

[0084] FIG. 2 illustrates a schematic diagram 200 of a functional framework for AI / ML for new radio (NR) air interface. As an illustrative example, consider a scenario where the network performs functionality-based Life Cycle Management (LCM) and where models are not identified in the network, while the UE concurrently performs model-level management (e.g., model selection / switching / (de)activation, etc.).

[0085] Management is a function that oversees the operation (e.g., selection, activation, deactivation, switching, or fallback) and monitoring (e.g., performance) of AI / ML models or AI / ML functionalities. This function is also responsible for making decisions to ensure the proper inference operation based on data received from the Data Collection function and the Inference function.

[0086] Management Instruction comprises information needed as input to manage the Inference function. Concerning information may include selection, activation, deactivation, or switching of AI / ML models or AI / ML-based functionalities, fallback to non-AI / ML operation (i.e., not relying on inference process), etc.

[0087] Model Transfer or Delivery Request is used to request model(s) to the Model Storage function.

[0088] Performance Feedback or Retraining Request comprises information needed as input for the Model Training function, e.g., for model (re)training or updating purposes.

[0089] Based on an existing design, LCM actions may be triggered by the NW. In functionality-based LCM, the network indicates the activation, deactivation, fallback or switching of AI / ML functionality via 3GPP signaling (e.g., Radio Resource Control (RRC), MAC-CE, Downlink Control Information (DCI)). In model-ID-based LCM, models are identified at the network, and the network or the UE may activate, deactivate, select or switch individual AI / ML models via model identity (ID).

[0090] For evaluation of performance monitoring approaches, the following model monitoring Key Performance Indicators (KPIs) are considered as general guidance: 1) Accuracy and relevance (i.e., how well does the given monitoring metric / methods reflect the model and system performance); 2) Overhead (e.g., signaling overhead associated with model monitoring); 3) Complexity (e.g., computation and memory cost for model monitoring); 4) Latency (i.e., timeliness of monitoring result, from model failure to action, given the purpose of model monitoring). It should be noted that other KPIs are not precluded. Relevant KPIs may vary across different model monitoring approaches.

[0091] It is desired for beam management that downlink (DL) transmission (Tx) beam prediction for both UE-sided model and NW-sided model encompasses: 1) spatial-domain DL Tx beam prediction for Set A of beams based on measurement results of Set B of beams (“BM-Casel”); 2) temporal DL Tx beam prediction for Set A of beams based on the historic measurement results of Set B of beams (“BM-Case2”); 3) specify necessary signaling or mechanism(s) to facilitate LCM operations specific to the Beam Management use cases, if any; 4) enabling method(s) to ensure consistency between training and inference regarding NW-side additional conditions (if identified) for inference at UE. It should be noted that common framework design is desired to support both BM-Casel and BM-Case2.

[0092] Moreover, it is desired to determine core requirements for the above two use cases for AI / ML LCM procedures and UE features have been determined, e.g., specify necessary Radio Access Network 4 (RAN4) core requirements for the above two use cases (i.e., beam management and positioning use cases), specify necessary RAN4 core requirements for LCM procedures including performance monitoring.

[0093] For UE-sided model and NW-sided monitoring, the NW will perform functionality performance monitoring, where the NW will check the performance degradation based functionality. When the NW detects performance degradation of the predicted Reference Signal Receiving Power (RSRP) or Top-K beam IDs or Top-1 beam IDs, the NW will then configure UE to switch to other functionality prediction orconfigure UE to switch to other (logical) model.

[0094] Regarding latency requirements, there are following proposed options:

[0095] Option 1 : Latency requirements of data collection for model inference and monitoring should be considered and discussed per use case, subject to the output from RAN 1 / 2.

[0096] Option 2: RAN4 should study latency requirements for data collection of model monitoring, at least for positioning and CSI compression use cases.

[0097] Option 3: RAN4 shall define the latency requirements based on RAN2’ s agreements and the MAX total latency requirements can be as follows.Ttotal= N * (fl + 12 + t3 + t4)

[0098] Option 4: Do not study latency requirements for training data collection, discuss latency requirements for any particular use case as needed.

[0099] Option 5: Consider data collection latency requirements only for inference and monitoring.

[0100] The latency requirements for LCM procedures are a part of RRM core requirements. The following procedure should be considered for the definition of core requirements : 1) Performance monitoring procedure, including performance evaluation and decision-making procedure for AI / ML functional ities / models ; 2) Functionality or Model management procedure, including selection, activation, or deactivation of functionality or model, and switching, fallback, transfer, delivery, update of functionality or model; 3) Latency or interruption requirement for above procedures .

[0101] It is important to distinguish more clearly between different types of latency requirements: for functionality performance monitoring, for training data collection, and for inference data collection. Therefore, core requirements and testing mechanism for these core requirements to validate the correct functionality of the device is required to be standardized.

[0102] A first existing design focuses on the test mechanism issue for performance requirements for AI / ML based beam management (BM) use case. It proposes the test mechanism for Top-1(%) and Top-K(%) beam ID prediction, with overhead reduction in reporting mechanism. A second existing design introduces the test mechanism for Top- 1(%) and Top-K(%) beam ID and RSRP prediction, with overhead reduction in reporting mechanism. A third existing design introduces the test mechanism for RSRP prediction, with overhead reduction in reporting mechanism.

[0103] In the present disclosure, several solutions are proposed herein to at least address a problem where core requirements are identified for LCM related actions for AI / ML based BM use case. Moreover, a test mechanism is proposed to verify these requirements.

[0104] As discussed above, there is a lot of interest in the requirements and testability of LCM aspects for AI / ML enabled functionalities. One of the factors that influence the performance of the AIML enabled functionality is the latency of the LCM actions between the Network and the UE.

[0105] If performance monitoring detects a performance degradation to a point where a decision to either switch this model / functionality with another model / functionality is taken or a fallback decision is taken, it means that the AI / ML functionality is degrading the system performance and if this functionality, with detected performance degradation, keeps running then the impact on system performance may result in drastic consequences.

[0106] The mechanism for Test Equipment (TE) / NW to switch the device under test (DUT) to stop this model / functionality would consist of messages / commands sent towards the DUT containing LCM actions, such as fall back to legacy method or switch to another model / functionality, within a specified time.

[0107] However, this specified time would depend upon the use case since different use cases would need different level of urgency to stop / switch to different model / functionality. In case of AI / ML based Beam Management use case, a bad AI / ML functionality would predict wrong top beams and it would mean that the UE won’t be transmitting / receiving on its best beams. This will consequently impact the spectral efficiency of the system and will impact overall system’s key performance indicators (KPIs). This needs to be fixed quickly but the specified time allowed in the Beam Management use case would be different in comparison of other use cases. It may be more relaxed in comparison to CSI use case (since CSI use case needs to be very reactive due to the dependence of CSI information on link adaptation and scheduling decisions), but it may be more stringent in comparison to Positioning use case, particularly when positioning coordinates are being used for static or slow-moving objects.

[0108] Some example embodiments of the present disclosure provide the mechanism to switch model / functionality, either by falling back to legacy method or by switching to another model / functionality, within a specified time. The specified time allowed to switch / di sable the model / functionality should guarantee that system performance would not be allowed to be degraded to unacceptable levels. The values of these specified times may be calculated based on simulation results and / or on field data.

[0109] A sample LCM workflow will be described below with an example from the Beam Management use case. At first, when the UE or the network detects a performance degradation, and if AI / ML based BM feature / functionality / model is enabled, the NW needs to take an appropriate LCM action to not allow further degradation in the performance (KPIs). In a case where the UE detects the performance degradation, the UE shall indicate it to network. These LCM actions may comprise at least one of the following: switch to another model / functionality; switch back to legacy mechanism if available; or disable the functionality. As soon as these LCM commands are sent to the UE, the UE needs to execute them accordingly. Any delay in the execution of these commands would further deteriorate the performance (KPIs).

[0110] In case of AI / ML BM, this may also depend on sub use case running at the moment of LCM command. For instance, Spatial Domain BM (Case-1) may require less resources to run / enable / stop / switch as compared to the Time Domain BM (Case-2) since Case-2 would require to store and process historical data as well higher number of input parameters. Therefore, it is important to ensure that the UE executes the command and either switches back to legacy or switches to another model / functionality within a defined time period. There is currently no existing requirement that can guarantee this expected behavior from the UE.[OHl] In the above-descried solution, a new core requirements is needed to ensure that LCM action for BM use case is executed within allowed time budget. These requirements may be based either on simulation results or on field data. Furthermore, a new test mechanism would be needed to test these new requirements. Either UE informs the TE about the switching to legacy or to new model / functionality using RANI procedures, if any. Alternatively, the TE configures a test mode to the UE in order to get this information from the UE whenever switching to legacy or to new model / functionality occurs.

[0112] In some example embodiments, these LCM actions may be on different interfaces. For instance, it may be on MAC CE interface as well as on RRC interface. Depending upon the sub use case, the timer requirement can be different as well. For instance, it can be more stringent for BM-Case-1 and be more relaxed for BM-Case-2.

[0113] It should be noted that there are no existing requirements that can guarantee latency of LCM actions towards the DUT for AI / ML based BM. Moreover, there are no test mechanisms that can help validate the LCM performance latency for AI / ML based BM.

[0114] The present disclosure provides several solutions to at least address the above-mentioned problem. In aid of the proposed solutions, the latency of LCM actions can be advantageously guaranteed within the test framework for AI / ML based BM.

[0115] FIG. 3 illustrates a signaling flow 300 for a LCM procedure in AI / ML beam management in accordance with some example embodiments of the present disclosure. As shown in FIG. 3, the signaling flow 300 involves a first apparatus 301 and a second apparatus 302.

[0116] The first apparatus 301 may be referred to as a TE, e.g., the TE 110 in the communication environment 100 of FIG. 1. The first apparatus 301 is configured to emulate / simulate a real wireless network. In some example embodiment, the first apparatus 301 may be a real network device (e.g., gNB) or may be a system simulator, operating as signal generators, probes, or transmission points that are used to transmit radio signal of certain type (e.g., used for SSB transmission). In some example embodiments, the first apparatus 301 may also include channel emulators and / or attenuators to emulate the propagation of the radio signal.

[0117] The second apparatus 302 may comprise a DUT 130 in the communication environment 100 of FIG. 1, which may be one or more of terminal devices, network entities, or base stations. In embodiments of the present disclosure, a second apparatus 302 may be able to implement AI / ML-based beam management (also referred to as AI / ML assisted beam management or AI / ML enabled beam management). An AI / ML model or AI / ML functionality is utilized by the second apparatus 302 to implement the AI / ML- based beam management. The output of the AI / ML model is a channel indicator for a communication channel between the second apparatus 302 and a network device, e.g., a TRP or base station. The input to the AI / ML model may include measurement results of one or more reference signals transmitted from the TRP.

[0118] In the signaling flow 300, the first apparatus 301 transmits (315), to a second apparatus 302, an indication of an action for switching from a first functionality or model for AI / ML-based beam management to a second functionality or model. The action may be used in a LCM procedure and may be referred to as “LCM action” in some cases.

[0119] The second apparatus 302 receives (320), from the first apparatus 301, the indication of the action for switching from the first functionality or model to the second functionality or model, and may switch to the second functionality or model and perform new beam prediction(s) for beam management. Upon completion of the beam prediction(s), the action may be determined as completed. The second apparatus 302 may transmit, to the first apparatus 301, a report about completion of the action. The reportmay be used for determining a result of a test for the action based on a time duration.

[0120] Specifically, after transmitting (315) the indication of the action for switching, the first apparatus 301 monitors (325) a report about completion of the action within a time duration. The time duration is associated with a latency requirement for the action. Based on a result of the monitoring, the first apparatus 301 determines (330) a result of a test for the action. For example, in the case where the second apparatus 302 transmits the report about completion of the action to the first apparatus 301, if the report is received within the time duration, the first apparatus 301 may determine that the test for the action is successful. On the other hand, if the report is not received within the time duration, the first apparatus 301 may determine that the test for the action is failed.

[0121] In addition, tthe beam prediction result may be considered. In some example embodiments, the first apparatus 301 may determine whether a beam prediction result that is obtained based on the second functionality or model leads to a system degradation. The beam prediction result may be received in the report or via a further message from the second apparatus 302. If the beam prediction result does not lead to the system degradation and the report is received within the time duration, the first apparatus 301 may determine that the test for the action is successful. Otherwise, if the beam prediction result leads to the system degradation or the report is not received within the time duration, the first apparatus 301 may determine that the test for the action is failed.

[0122] The beam prediction result may be obtained by the second apparatus 302 based on the second functionality or model. Then, the second apparatus 302 may transmit the beam prediction result in the report or via a further message to the first apparatus 301.

[0123] The time duration may be preconfigured or predefined or specified. There may be various ways for determining whether the report is received within the time duration. For example, a timer may be set for purpose of this. This timer may start at a start time point of the monitoring of the report and a length of the timer is equal to the time duration. If the report is received while the timer is running, it may be determined that the report is received within the time duration.

[0124] In some example embodiments, the action for switching functionalities or models may be selected from a group of actions. These actions may be predefined or preconfigured. The group of actions may include, for example, but not limited to the following examples.

[0125] A first action for switching (also referred to as “first switching”) from a functionality or model for AI / ML-based beam management to another functionality ormodel for AI / ML-based beam management. The first switching is associated with beam management in a spatial domain.

[0126] A second action for switching (also referred to as “second switching”) from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management. The second switching is associated with beam management in a temporal domain.

[0127] A third action for switching (also referred to as “third switching”) from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management. The third switching is associated with beam management in a spatial domain.

[0128] A fourth action for switching (also referred to as “fourth switching”) from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management. The fourth switching is associated with beam management in a temporal domain.

[0129] In some example embodiments, each of the above actions may have a corresponding latency requirement. The latency requirements may be defined to ensure that an LCM action for the beam management use case is executed within an allowed time budget. These requirements may be based either on simulation results or on field data or other suitable factors.

[0130] It should be noted that different sub use cases of AI / ML based beam management such as BM Case 1 (or Spatial Domain BM) and BM Case 2 (or Temporal Domain BM) may have different level of complexity as well different input and output sized. In addition to different structures, the speed of performance degradation for different cases may also be different. Therefore, the switching time required for different cases may also be different. An example of timer requirements for different cases of BM is descried below.

[0131] An example core requirement for LCM action latency for AI / ML BM use case is provided in the Table 1. In particular, Table 1 lists example core requirement(s) of LCM actions, including the above first action, second action, third action and fourth action.Table 1-Core requirement for LCM action latency for AI / ML BM use case

[0132] It should be noted that the timer values may be derived from the field tests and simulation results. The timer values “a. a”, “b.b”, “c.c” and “d.d” are just illustration for purpose of example, rather than limiting the actual values. In example embodiments, the timer values may be determined in various suitable ways or by considering related requirement(s).

[0133] In the present disclosure, the core requirements for performance monitoring are proposed. The proposed core requirements may be applied for both Top-K beam ID(s) prediction and RSRP prediction in spatial or time domain beam prediction.

[0134] Optionally, in some example embodiments, the second apparatus 302 may transmit (305) capability information of the second apparatus 302 to the first apparatus 301. The capability information may at least indicates respective response times for a plurality of actions for AI / ML-based beam management. The capability information may include, for example, but not limited to, a response time for switching to a functionality or model for AI / ML-based beam management, and / or a response time for switching to a functionality or model for non-AI / ML-based beam management. The first apparatus 301 may receive (310) the capability information from the second apparatus 302, and thus may have the knowledge of the capability of the second apparatus 302.

[0135] In some example embodiments, the time duration may be determined based on at least one of the capability information of the second apparatus or the latency requirement for the action.

[0136] In some example embodiments, before the first apparatus 301 transmits (315) the indication of the action to a second apparatus 302, the first apparatus 301 may receive, from the second apparatus 302, a further report indicating a further beam prediction result obtained based on the first functionality or model. The first apparatus 301 may determine whether this beam prediction result leads to a system degradation. If so, the first apparatus 301 performs the transmit (315) of the indication of the action for switching to the second functionality or model. In some embodiments, the further report may be via a channel state information (CSI) report associated with at least one enabled CSI configuration, for example, via a CSI report associated to some CSI configurations that are enabled, e.g., BM-Casel.

[0137] The report which may be transmitted from the second apparatus 302 may be transmitted via a channel state information (CSI) report associated with at least one enabled CSI configuration, for example, via a CSI report associate to some CSI configurations that are enabled, e.g., BM-Case2.

[0138] In some example embodiments, the second functionality or model may be for AI / ML-based beam management. In this case, the indication of the action may be included in a configuration of a CSI report associated with an AI / ML-based beam management functionality or model different from the second functionality or model. For example, the indication may be transmitted through CSI-reportconfig associated to another functionality of AI / ML BM mode, e.g., BM-Case2.

[0139] In some example embodiments, the second functionality or model may be for non-AI / ML-based beam management. In this case, the indication of the action is comprised in a configuration of a CSI report or a configuration of CSI measurement associated with non-AI / ML-based beam management.

[0140] In some implementations, the first apparatus 301 may be a test terminal device or a network device, and the second apparatus 302 may be a terminal device.

[0141] The solutions presented in FIG. 3 will be described in more details below with reference to FIGS. 4 and 5. FIG. 4 illustrates a detailed signaling flow 400 for a LCM procedure in AI / ML beam management in accordance with some further example embodiments of the present disclosure. The signaling flow 400 show a flow for functionality / (logical) model switching with core requirements. By way of example rather than limitation, the DUT 401 may comprise a terminal device, such as a UE or the like. In addition, the TE 402 may comprise a network device, such as a gNB or the like. In FIG. 4, the DUT 401 may be an example implementation of the second apparatus 302 inFIG. 3, and the TE 402 may be an example implementation of the first apparatus 301 in FIG. 3.

[0142] At 410, the TE 402 tests AI / ML to BM functionality. Moreover, functionality based LCM procedures are switched on along with the TE 402 side performance monitoring. At 415, the TE 402 moves or causes the DUT 401 to AI / ML BM test mode. At 420, the DUT 401 sends UE capability, for example, UE capability about AI / ML BM for RSRP or beam ID(s) prediction. The capability may include, for example, LCM response time (time budget) for functionality or (logical) model switching = G_l, G_2,...., and LCM response time for switching to legacy = L_l, L_2,....

[0143] At 425, the DUT 401 is in an AI / ML BM mode based functionality (also referred to the first functionality or model for AI / ML-based BM), and it performs RSRP prediction or beam ID(s) prediction in spatial domain or time domain. At 430, the DUT 401 reports predicted output and functionality / (logical) model notification to the TE 402. At 435, the TE 402 triggers controlled performance degradation. At 440, the TE 402 validates whether there is performance degradation. If it is determined that there is no performance degradation, the TE 402 verifies or confirms the prediction output. If it is determined that there is performance degradation, the TE 402 configures, at 445, the DUT 401 to switch to another functionality or (logical) model (also referred to the second functionality or model), and meanwhile, the TE 402 starts a timer, e.g., denoted as T_l, based on UE capability and BM-Casel or BM-Case2. The timer is also referred to as a response timer or a LCM response timer in some example embodiments.

[0144] The TE 402 triggers LCM response timer for G_K, which corresponds to the functionality / logical model G_K. K may be an integer. The LCM response timer G_K will be running until step 455 descried below. At 450, the DUT 401 switches to another functionality / (logical) model and perform new prediction. At 455, the DUT 401 reports prediction output and functionality / (logical) model switching notification.

[0145] At 460, if the functionality / (logical) model notification is not received within response timer G_K, then the test for functionality / (logical) model pass. If the functionality / (logical) model notification is not received within response timer G_K, the TE 402 starts new LCM response timer G_N for functionality / (logical) model switch. Furthermore, the TE 402 configures the DUT 401 to switch to other functionality / (logical) model and the TE 402 starts response timer based functionality / (logical) model switch. The TE 402 triggers LCM response timer for G N, which corresponds to the functionality / logical model G_N. N may be an integer. The DUT 401 switches to otherfunctionality / (logical) model, then DUT 401 performs new prediction (inference). The DUT 401 reports prediction output and functionality / (logical) notification. The LCM response timer G N will be running until this step. If the functionality / (logical) model notification is received within response timer, the test for functionality / (logical) model pass.

[0146] In an example scenario without the test mode, at 430, the DUT 401 may report predicted output and functionality / (logical) model notification to the TE 402 through channel state information (CSI) reporting. In addition, at 445, the TE 402 may configure the DUT 401 to switch to another functionality / (logical) model through CSI-report configuration associated with AI / ML BM mode. Similarly, at 455, the DUT 401 may report predicted output and functionality / (logical) model notification to the TE 402 through CSI reporting.

[0147] FIG. 5 illustrates a detailed signaling flow 500 for a LCM procedure in AI / ML beam management in accordance with some further example embodiments of the present disclosure. In contract with the signaling flow 400, the signaling flow 500 shows a flow of functionality / (logical) model switching and switch to legacy with core requirements. By way of example rather than limitation, the DUT 501 may comprise a terminal device, such as a UE or the like. In addition, the TE 502 may comprise a network device, such as a gNB or the like. In FIG. 5, the DUT 501 may be an example implementation of the second apparatus 302 in FIG. 3, and the TE 502 may be an example implementation of the first apparatus 301 in FIG. 3.

[0148] At 510, the TE 502 tests AI / ML to BM functionality. Moreover, functionality based LCM procedures are switched on along with TE 502 side performance monitoring. At 515, the DUT 501 sends UE capability, e.g., about AI / ML BM for RSRP or beam ID(s) prediction. . The capability may include, for example, LCM response time (time budget) for functionality / (logical) model switching = G_l, G_2,...., ; and LCM response time for switching to legacy = L_l, L_2,....

[0149] At 520, the DUT 501 is in AI / ML BM mode based functionality. For example, the DUT 501 performs RSRP prediction or beam ID(s) prediction in spatial domain or time domain. At 525, the DUT 501 reports a predicted output to the TE 502 and functionality / (logical) model notification. TE 502 receives the predicted output from the DUT 501 and may determine that the predicted output leads to the performance degradation. Then the TE 502 may determine to trigger a switch to a legacy mode, for example, in which a model or functionality for BM without the aid of AI / ML is used.

[0150] At 535, the TE 502 starts new LCM response timer (e.g., denoted as L_l) for legacy mode switch. The LCM response timer L_1 for legacy mode will be running until step 555 described below. At 540, the TE 502 configures the DUT 501 to switch to legacy. At 545, the TE 502 configures the DUT 501 to perform measurements based on full Set A.

[0151] At 550, the DUT 501 switches to legacy mode and is reconfigured to perform measurements with full Set A. AT 555, the DUT 501 sends acknowledgement to the TE 502. The acknowledgement indicates that the DUT 501 switches back to legacy mode. The DUT 501 may also report measurement results, e.g., beam ID(s) or RSRP, obtained in the legacy mode. At 560, if the beam ID(s) or RSRP is reported in the response timer and performance monitoring is improved, then the DUT 501 passes the test.

[0152] In an example scenario without the test mode, at 525, the DUT 501 may report predicted output and functionality / (logical) model notification to the TE 502 through CSI reporting. Furthermore, at 540, the TE 502 may configure the DUT to switch to legacy through CSI-report configuration associated with the legacy mode. Similarly, at 545, the TE 502 may configure the DUT to switch to legacy through CSI-measure configuration associated with legacy mode.

[0153] FIG. 6 shows a flowchart of an example method 600 implemented at a first apparatus in accordance with some example embodiments of the present disclosure. For the purpose of discussion, the method 600 will be described from the perspective of the first apparatus 110 in FIG. 1.

[0154] At block 610, the first apparatus 110 transmits, to a second apparatus, an indication of an action for switching from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model;

[0155] At block 620, the first apparatus 110 monitors a report about completion of the action within a time duration, the time duration being associated with a latency requirement for the action; and

[0156] At block 630, the first apparatus 110 determines a result of a test for the action based on a result of the monitoring.

[0157] In some example embodiments, the method 600 further comprises: in accordance with a determination that the report is received within the time duration, determining that the test for the action is successful; and in accordance with a determination that the report is not received within the time duration, determining that the test for the action is failed.

[0158] In some example embodiments, the method 600 further comprises: determining whether a beam prediction result that is obtained based on the second functionality or model leads to a system degradation, the beam prediction result being received in the report or via a further message from the second apparatus; in accordance with a determination that the beam prediction result does not lead to the system degradation and the report is received within the time duration, determining that the test for the action is successful; and in accordance with a determination that the beam prediction result leads to the system degradation or the report is not received within the time duration, determining that the test for the action is failed.

[0159] In some example embodiments, the method 600 further comprises: in accordance with a determination that the report is received while the timer is running, determining the report is received within the time duration.

[0160] In some example embodiments, the action is one of the following actions: first switching from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management, the first switching being associated with beam management in a spatial domain; second switching from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management, the second switching being associated with beam management in a temporal domain; third switching from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management, the third switching being associated with beam management in a spatial domain; fourth switching from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management, the fourth switching being associated with beam management in a temporal domain.

[0161] In some example embodiments, each the actions corresponds to a latency requirement.

[0162] In some example embodiments, the method 600 further comprises: receiving, from the second apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of actions for AI / ML-based beam management.

[0163] In some example embodiments, the capability information comprises at least one of: a response time for switching to a functionality or model for AI / ML-based beam management, or a response time for switching to a functionality or model for non-AI / ML- based beam management.

[0164] In some example embodiments, the time duration is determined based on at least one of the capability information of the second apparatus or the latency requirement for the action.

[0165] In some example embodiments, the method 600 further comprises: receiving, from the second apparatus, a further report indicating a further beam prediction result obtained based on the first functionality or model; and in accordance with a determination that the further beam prediction result leads to a system degradation, transmitting, to the second apparatus, the indication of the action for switching to the second functionality or model.

[0166] In some example embodiments, the further report is via a channel state information (CSI) report associated with at least one enabled CSI configuration.

[0167] In some example embodiments, the report is via a channel state information (CSI) report associated with at least one enabled CSI configuration.

[0168] In some example embodiments, the second functionality or model is for AI / ML- based beam management, and wherein the indication of the action is comprised in a configuration of a channel state information (CSI) report associated with an AI / ML-based beam management functionality or model different from the second functionality or model.

[0169] In some example embodiments, the second functionality or model is for non- AI / ML-based beam management, and wherein the indication of the action is comprised in a configuration of a channel state information (CSI) report or a configuration of CSI measurement associated with non-AI / ML-based beam management.

[0170] In some example embodiments, the first apparatus comprises a test terminal device or a network device, and the second apparatus comprises a terminal device.

[0171] FIG. 7 shows a flowchart of an example method 700 implemented at a second apparatus in accordance with some example embodiments of the present disclosure. For the purpose of discussion, the method 700 will be described from the perspective of the second apparatus 120 in FIG. 1.

[0172] At block 710, the second apparatus 120 receives, from a first apparatus 110, an indication of an action for switching from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model;

[0173] At block 720, the second apparatus 120 transmits, to the first apparatus 110, a report about completion of the action for determining a result of a test for the action based on a time duration.

[0174] In some example embodiments, the method 700 further comprises: obtaining a beam prediction result obtained based on the second functionality or model; and transmitting the beam prediction result in the report or via a further message to the first apparatus.

[0175] In some example embodiments, the action is one of the following actions: first switching from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management, the first switching being associated with beam management in a spatial domain; second switching from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management, the second switching being associated with beam management in a temporal domain; third switching from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management, the third switching being associated with beam management in a spatial domain; fourth switching from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management, the fourth switching being associated with beam management in a temporal domain.

[0176] In some example embodiments, each the actions corresponds to a latency requirement.

[0177] In some example embodiments, the method 700 further comprises: transmitting, to the first apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of actions for AI / ML-based beam management.

[0178] In some example embodiments, the capability information comprises at least one of: a response time for switching to a functionality or model for AI / ML-based beam management, or a response time for switching to a functionality or model for non-AI / ML- based beam management.

[0179] In some example embodiments, the time duration is determined based on at least one of the capability information of the second apparatus or the latency requirement for the action.

[0180] In some example embodiments, the method 700 further comprises: transmitting, to the first apparatus, a further report indicating a further beam prediction result obtained based on the first functionality or model.

[0181] In some example embodiments, the further report is via a channel state information (CSI) report associated with at least one enabled CSI configuration.

[0182] In some example embodiments, the report is via a channel state information (CSI) report associated with at least one enabled CSI configuration.

[0183] In some example embodiments, the second functionality or model is for AI / ML- based beam management, and wherein the indication of the action is comprised in a configuration of a channel state information (CSI) report associated with an AI / ML-based beam management functionality or model different from the second functionality or model.

[0184] In some example embodiments, the second functionality or model is for non- AI / ML-based beam management, and wherein the indication of the action is comprised in a configuration of a channel state information (CSI) report or a configuration of CSI measurement associated with non-AI / ML-based beam management.

[0185] In some example embodiments, the first apparatus comprises a test terminal device or a network device, and the second apparatus comprises a terminal device.

[0186] In some example embodiments, a first apparatus capable of performing any of the method 600 (for example, the first apparatus 110 in FIG. 1 may comprise means for performing the respective operations of the method 600. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module. The first apparatus may be implemented as or included in the first apparatus 110 in FIG. 1.

[0187] In some example embodiments, the first apparatus comprises means for transmitting, to a second apparatus, an indication of an action for switching from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; means for monitoring a report about completion of the action within a time duration, the time duration being associated with a latency requirement for the action; and means for determining a result of a test for the action based on a result of the monitoring.

[0188] In some example embodiments, the first apparatus further comprises: means for in accordance with a determination that the report is received within the time duration, determining that the test for the action is successful; and means for in accordance with a determination that the report is not received within the time duration, determining that the test for the action is failed.

[0189] In some example embodiments, the first apparatus further comprises: means for determining whether a beam prediction result that is obtained based on the second functionality or model leads to a system degradation, the beam prediction result being received in the report or via a further message from the second apparatus; means for inaccordance with a determination that the beam prediction result does not lead to the system degradation and the report is received within the time duration, determining that the test for the action is successful; and means for in accordance with a determination that the beam prediction result leads to the system degradation or the report is not received within the time duration, determining that the test for the action is failed.

[0190] In some example embodiments, a timer starts at a start time point of the monitoring of the report and a length of the timer is equal to the time duration, and wherein the first apparatus further comprises: means for in accordance with a determination that the report is received while the timer is running, determining the report is received within the time duration.

[0191] In some example embodiments, the action is one of the following actions: first switching from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management, the first switching being associated with beam management in a spatial domain; second switching from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management, the second switching being associated with beam management in a temporal domain; third switching from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management, the third switching being associated with beam management in a spatial domain; fourth switching from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management, the fourth switching being associated with beam management in a temporal domain.

[0192] In some example embodiments, each the actions corresponds to a latency requirement.

[0193] In some example embodiments, the first apparatus further comprises: means for receiving, from the second apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of actions for AI / ML-based beam management.

[0194] In some example embodiments, the capability information comprises at least one of: a response time for switching to a functionality or model for AI / ML-based beam management, or a response time for switching to a functionality or model for non-AI / ML- based beam management.

[0195] In some example embodiments, the time duration is determined based on at least one of the capability information of the second apparatus or the latency requirement forthe action.

[0196] In some example embodiments, the first apparatus further comprises: means for receiving, from the second apparatus, a further report indicating a further beam prediction result obtained based on the first functionality or model; and means for in accordance with a determination that the further beam prediction result leads to a system degradation, transmitting, to the second apparatus, the indication of the action for switching to the second functionality or model.

[0197] In some example embodiments, the further report is via a channel state information (CSI) report associated with at least one enabled CSI configuration.

[0198] In some example embodiments, the report is via a channel state information (CSI) report associated with at least one enabled CSI configuration.

[0199] In some example embodiments, the second functionality or model is for AI / ML- based beam management, and wherein the indication of the action is comprised in a configuration of a channel state information (CSI) report associated with an AI / ML-based beam management functionality or model different from the second functionality or model.

[0200] In some example embodiments, the second functionality or model is for non- AI / ML-based beam management, and wherein the indication of the action is comprised in a configuration of a channel state information (CSI) report or a configuration of CSI measurement associated with non-AI / ML-based beam management.

[0201] In some example embodiments, the first apparatus comprises a test terminal device or a network device, and the second apparatus comprises a terminal device.

[0202] In some example embodiments, the first apparatus further comprises means for performing other operations in some example embodiments of the method 600 or the first apparatus 110. In some example embodiments, the means comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the performance of the first apparatus.

[0203] In some example embodiments, a second apparatus capable of performing any of the method 700 (for example, the second apparatus 120 in FIG. 1) may comprise means for performing the respective operations of the method 700. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module. The second apparatus may be implemented as or included in the second apparatus 120 in FIG. 1.

[0204] In some example embodiments, the second apparatus comprises means for receiving, from a first apparatus, an indication of an action for switching from a firstfunctionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; means for transmitting, to the first apparatus, a report about completion of the action for determining a result of a test for the action based on a time duration.

[0205] In some example embodiments, the second apparatus further comprises: means for obtaining a beam prediction result obtained based on the second functionality or model; and means for transmitting the beam prediction result in the report or via a further message to the first apparatus.

[0206] In some example embodiments, the action is one of the following actions: first switching from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management, the first switching being associated with beam management in a spatial domain; second switching from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management, the second switching being associated with beam management in a temporal domain; third switching from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management, the third switching being associated with beam management in a spatial domain; fourth switching from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management, the fourth switching being associated with beam management in a temporal domain.

[0207] In some example embodiments, each the actions corresponds to a latency requirement.

[0208] In some example embodiments, the second apparatus further comprises: means for transmitting, to the first apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of actions for AI / ML-based beam management.

[0209] In some example embodiments, the capability information comprises at least one of: a response time for switching to a functionality or model for AI / ML-based beam management, or a response time for switching to a functionality or model for non-AI / ML- based beam management.

[0210] In some example embodiments, the time duration is determined based on at least one of the capability information of the second apparatus or the latency requirement for the action.

[0211] In some example embodiments, the second apparatus further comprises: meansfor transmitting, to the first apparatus, a further report indicating a further beam prediction result obtained based on the first functionality or model.

[0212] In some example embodiments, the further report is via a channel state information (CSI) report associated with at least one enabled CSI configuration.

[0213] In some example embodiments, the report is via a channel state information (CSI) report associated with at least one enabled CSI configuration.

[0214] In some example embodiments, the second functionality or model is for AI / ML- based beam management, and wherein the indication of the action is comprised in a configuration of a channel state information (CSI) report associated with an AI / ML-based beam management functionality or model different from the second functionality or model.

[0215] In some example embodiments, the second functionality or model is for non- AI / ML-based beam management, and wherein the indication of the action is comprised in a configuration of a channel state information (CSI) report or a configuration of CSI measurement associated with non-AI / ML-based beam management.

[0216] In some example embodiments, the first apparatus comprises a test terminal device or a network device, and the second apparatus comprises a terminal device.

[0217] In some example embodiments, the second apparatus further comprises means for performing other operations in some example embodiments of the method 700 or the second apparatus 120. In some example embodiments, the means comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the performance of the second apparatus.

[0218] FIG. 8 is a simplified block diagram of a device 800 that is suitable for implementing example embodiments of the present disclosure. The device 800 may be provided to implement a communication device, for example, the first apparatus 110 or the second apparatus 120 as shown in FIG. 1. As shown, the device 800 includes one or more processors 810, one or more memories 820 coupled to the processor 810, and one or more communication modules 840 coupled to the processor 810.

[0219] The communication module 840 is for bidirectional communications. The communication module 840 has one or more communication interfaces to facilitate communication with one or more other modules or devices. The communication interfaces may represent any interface that is necessary for communication with other network elements. In some example embodiments, the communication module 840 may include at least one antenna.

[0220] The processor 810 may be of any type suitable to the local technical networkand may include one or more of the following: general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on multicore processor architecture, as non-limiting examples. The device 800 may have multiple processors, such as an application specific integrated circuit chip that is slaved in time to a clock which synchronizes the main processor.

[0221] The memory 820 may include one or more non-volatile memories and one or more volatile memories. Examples of the non-volatile memories include, but are not limited to, a Read Only Memory (ROM) 824, an electrically programmable read only memory (EPROM), a flash memory, a hard disk, a compact disc (CD), a digital video disk (DVD), an optical disk, a laser disk, and other magnetic storage and / or optical storage. Examples of the volatile memories include, but are not limited to, a random access memory (RAM) 822 and other volatile memories that will not last in the power-down duration.

[0222] A computer program 830 includes computer executable instructions that are executed by the associated processor 810. The instructions of the program 830 may include instructions for performing operations / acts of some example embodiments of the present disclosure. The program 830 may be stored in the memory, e.g., the ROM 824. The processor 810 may perform any suitable actions and processing by loading the program 830 into the RAM 822.

[0223] The example embodiments of the present disclosure may be implemented by means of the program 830 so that the device 800 may perform any process of the disclosure as discussed with reference to FIG. 2 to FIG. 7. The example embodiments of the present disclosure may also be implemented by hardware or by a combination of software and hardware.

[0224] In some example embodiments, the program 830 may be tangibly contained in a computer readable medium which may be included in the device 800 (such as in the memory 820) or other storage devices that are accessible by the device 800. The device 800 may load the program 830 from the computer readable medium to the RAM 822 for execution. In some example embodiments, the computer readable medium may include any types of non-transitory storage medium, such as ROM, EPROM, a flash memory, a hard disk, CD, DVD, and the like. The term “non-transitory,” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM).

[0225] Generally, various embodiments of the present disclosure may be implementedin hardware or special purpose circuits, software, logic or any combination thereof. Some aspects may be implemented in hardware, and other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device. Although various aspects of embodiments of the present disclosure are illustrated and described as block diagrams, flowcharts, or using some other pictorial representations, it is to be understood that the block, apparatus, system, technique or method described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0226] Some example embodiments of the present disclosure also provide at least one computer program product tangibly stored on a computer readable medium, such as a non- transitory computer readable medium. The computer program product includes computerexecutable instructions, such as those included in program modules, being executed in a device on a target physical or virtual processor, to carry out any of the methods as described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, or the like that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. Machine-executable instructions for program modules may be executed within a local or distributed device. In a distributed device, program modules may be located in both local and remote storage media.

[0227] Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0228] In the context of the present disclosure, the computer program code or related data may be carried by any suitable carrier to enable the device, apparatus or processor to perform various processes and operations as described above. Examples of the carrier include a signal, computer readable medium, and the like.

[0229] The computer readable medium may be a computer readable signal medium ora computer readable storage medium. A computer readable medium may include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0230] Further, although operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, although several specific implementation details are contained in the above discussions, these should not be construed as limitations on the scope of the present disclosure, but rather as descriptions of features that may be specific to particular embodiments. Unless explicitly stated, certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, unless explicitly stated, various features that are described in the context of a single embodiment may also be implemented in a plurality of embodiments separately or in any suitable sub-combination.

[0231] Although the present disclosure has been described in languages specific to structural features and / or methodological acts, it is to be understood that the present disclosure defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

WHAT IS CLAIMED IS:

1. A first apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the first apparatus at least to: transmit, to a second apparatus, an indication of an action for switching, by the second apparatus, from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; monitor a report about completion of the action within a time duration, the time duration being associated with a latency requirement for the action; and determine a result of a test for the action based on a result of the monitoring.

2. The first apparatus of claim 1, wherein the first apparatus is caused to: in accordance with a determination that the report is received within the time duration, determine that the test for the action is successful; and in accordance with a determination that the report is not received within the time duration, determine that the test for the action is failed.

3. The first apparatus of claim 1, wherein the first apparatus is caused to: determine whether a beam prediction result that is obtained based on the second functionality or model leads to a system degradation, the beam prediction result being received in the report or via a further message from the second apparatus; in accordance with a determination that the beam prediction result does not lead to the system degradation and the report is received within the time duration, determine that the test for the action is successful; and in accordance with a determination that the beam prediction result leads to the system degradation or the report is not received within the time duration, determine that the test for the action is failed.

4. The first apparatus of any of claims 1 to 3, wherein a timer starts at a start time point of the monitoring of the report and a length of the timer is equal to the time duration, and wherein the first apparatus is caused to:in accordance with a determination that the report is received while the timer is running, determine the report is received within the time duration.

5. The first apparatus of any of claims 1 to 4, wherein the action is one of the following actions: first switching from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management, the first switching being associated with beam management in a spatial domain; second switching from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management, the second switching being associated with beam management in a temporal domain; third switching from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management, the third switching being associated with beam management in a spatial domain; fourth switching from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management, the fourth switching being associated with beam management in a temporal domain.

6. The first apparatus of claim 5, wherein each the actions corresponds to a latency requirement.

7. The first apparatus of any of claims 1 to 6, wherein the first apparatus is caused to: receive, from the second apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of actions for AI / ML-based beam management.

8. The first apparatus of claim 7, wherein the capability information comprises at least one of: a response time for switching to a functionality or model for AI / ML-based beam management, or a response time for switching to a functionality or model for non-AI / ML-based beam management.

9. The first apparatus of claim 7, wherein the time duration is determined based on at least one of the capability information of the second apparatus or the latency requirement for the action.

10. The first apparatus of any of claims 1 to 9, wherein the first apparatus is caused to: receive, from the second apparatus, a further report indicating a further beam prediction result obtained based on the first functionality or model; and in accordance with a determination that the further beam prediction result leads to a system degradation, transmit, to the second apparatus, the indication of the action for switching to the second functionality or model.

11. The first apparatus of claim 10, wherein the further report is via a channel state information (CSI) report associated with at least one enabled CSI configuration.

12. The first apparatus of any of claims 1 to 11, wherein the report is via a channel state information (CSI) report associated with at least one enabled CSI configuration.

13. The first apparatus of any of claims 1 to 12, wherein the second functionality or model is for AI / ML-based beam management, and wherein the indication of the action is comprised in a configuration of a channel state information (CSI) report associated with an AI / ML-based beam management functionality or model different from the second functionality or model.

14. The first apparatus of any of claims 1 to 12, wherein the second functionality or model is for non-AI / ML-based beam management, and wherein the indication of the action is comprised in a configuration of a channel state information (CSI) report or a configuration of CSI measurement associated with non-AI / ML-based beam management.

15. A second apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the second apparatus at least to: receive, from a first apparatus, an indication of an action for switching from a firstfunctionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; transmit, to the first apparatus, a report about completion of the action for determining a result of a test for the action based on a time duration.

16. The second apparatus of claim 15, wherein the second apparatus is caused to: obtain a beam prediction result obtained based on the second functionality or model; and transmit the beam prediction result in the report or via a further message to the first apparatus.

17. The second apparatus of claim 15 or 16, wherein the action is one of the following actions: first switching from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management, the first switching being associated with beam management in a spatial domain; second switching from a functionality or model for AI / ML-based beam management to another functionality or model for AI / ML-based beam management, the second switching being associated with beam management in a temporal domain; third switching from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management, the third switching being associated with beam management in a spatial domain; fourth switching from a functionality or model for AI / ML-based beam management to a functionality or model for non-AI / ML-based beam management, the fourth switching being associated with beam management in a temporal domain.

18. The second apparatus of claim 17, wherein each the actions corresponds to a latency requirement.

19. The second apparatus of any of claims 15 to 18, wherein the second apparatus is caused to: transmit, to the first apparatus, capability information of the second apparatus, the capability information at least indicating respective response times for a plurality of actions for AI / ML-based beam management.

20. The second apparatus of claim 19, wherein the capability information comprises at least one of: a response time for switching to a functionality or model for AI / ML-based beam management, or a response time for switching to a functionality or model for non-AI / ML-based beam management.

21. The second apparatus of claim 19, wherein the time duration is determined based on at least one of the capability information of the second apparatus or the latency requirement for the action.

22. The second apparatus of any of claims 15 to 21, wherein the second apparatus is caused to: transmit, to the first apparatus, a further report indicating a further beam prediction result obtained based on the first functionality or model, wherein the further report is via a channel state information (CSI) report associated with at least one enabled CSI configuration.

23. The second apparatus of any of claims 15 to 22, wherein the report is via a channel state information (CSI) report associated with at least one enabled CSI configuration.

24. The second apparatus of any of claims 15 to 23, wherein the second functionality or model is for AI / ML-based beam management, and wherein the indication of the action is comprised in a configuration of a channel state information (CSI) report associated with an AI / ML-based beam management functionality or model different from the second functionality or model; or wherein the second functionality or model is for non-AI / ML-based beam management, and wherein the indication of the action is comprised in a configuration of a channel state information (CSI) report or a configuration of CSI measurement associated with non-AI / ML-based beam management.

25. The second apparatus of any of claims 15 to 24, wherein the first apparatus comprises a test terminal device or a network device, and the second apparatus comprises a terminal device.

26. A method comprising: transmitting, to a second apparatus, an indication of an action for switching, by the second apparatus, from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; monitoring a report about completion of the action within a time duration, the time duration being associated with a latency requirement for the action; and determining a result of a test for the action based on a result of the monitoring.

27. A method comprising: receiving, from a first apparatus, an indication of an action for switching from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; transmitting, to the first apparatus, a report about completion of the action for determining a result of a test for the action based on a time duration.

28. A first apparatus comprising: means for transmitting, to a second apparatus, an indication of an action for switching, by the second apparatus, from a first functionality or model for artificial intelligence / machine learning (AI / ML)-based beam management to a second functionality or model; means for monitoring a report about completion of the action within a time duration, the time duration being associated with a latency requirement for the action; and means for determining a result of a test for the action based on a result of the monitoring.

29. A second apparatus comprising: means for receiving, from a first apparatus, an indication of an action for switching from a first functionality or model for artificial intelligence / machine learning (AI / ML)- based beam management to a second functionality or model; means for transmitting, to the first apparatus, a report about completion of the actionfor determining a result of a test for the action based on a time duration.

30. A computer readable medium comprising instructions stored thereon for causing an apparatus at least to perform the method of claim 26 or the method of claim 27.

Images