Prediction and test mechanism enhancement

Abstract

Embodiments of the present disclosure relate to apparatuses, methods, devices and computer readable storage medium for signal strength prediction and test mechanism enhancement In a method, a first apparatus transmits, to a second apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams. The first apparatus receives, from the second apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, and determines a result of a test of the prediction model based on the measurement result and the prediction result. In this way, the prediction accuracy for the signal strength can be tested with low overhead.

Description

PREDICTION AND TEST MECHANISM ENHANCEMENTCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to, and the benefit of, US Provisional Application No. 63 / 595947, filed Nov. 3, 2023, the contents of which are hereby incorporated by reference in their entirety.FIELD

[0002] Various example embodiments of the present disclosure generally relate to the field of telecommunication and in particular, to methods, devices, apparatuses and computer readable storage medium for signal strength prediction and test mechanism enhancement.BACKGROUND

[0003] Study and research are made to support a new Artificial Intelligence (AI) / Machine Learning (ML)-enabled radio interface for the next cellular systems, for example, to solve problems such as how to improve the performance of air-interface functions with AI / ML techniques, what would be required to enable AI / ML techniques for the air interface, and so on.

[0004] AI / ML-based beam management targets spatial and / or time beam prediction for overhead and latency reduction. The beam management includes, for example, beam prediction in time and / or spatial domain for overhead and latency reduction, beam selection accuracy improvement, and the like. Use cases for the beam management are further studied for spatial and / or time beam prediction. The scope of spatial beam prediction (BM-Casel) is to predict the best Tx / Rx beams in different spatial locations. Conversely, time-domain beam predictions (BM-Case2) aim to predict the most likely beam to use for next time instants, e.g., beam prediction in the spatial domain (BM-Casel).SUMMARY

[0005] In a first aspect of the present disclosure, there is provided a first apparatus. The first apparatus comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the first apparatus at least to: transmit, to a second apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model iassociated with a second set of beams; receive, from the second apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold; and determine a result of a test of the prediction model based on the measurement result and the prediction result.

[0006] In a second aspect of the present disclosure, there is provided a second apparatus. The second apparatus comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the second apparatus at least to: receive, from a first apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams; and transmit, to the first apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold.

[0007] In a third aspect of the present disclosure, there is provided a method. The method comprises: transmitting, at a first apparatus to a second apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams; receiving, from the second apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strengthexceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold; and determining a result of a test of the prediction model based on the measurement result and the prediction result.

[0008] In a fourth aspect of the present disclosure, there is provided a method. The method comprises: receiving, at a second apparatus from a first apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams; and transmitting, to the first apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold.

[0009] In a fifth aspect of the present disclosure, there is provided a first apparatus. The first apparatus comprises means for transmitting, to a second apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams; means for receiving, from the second apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold; and means for determining a result of a test of the prediction model based on the measurement result and the prediction result.

[0010] In a sixth aspect of the present disclosure, there is provided a second apparatus. The second apparatus comprises means for receiving, from a first apparatus, a first configuration for measurements associated with a first set of beams and a secondconfiguration for prediction with a prediction model associated with a second set of beams; and means for transmitting, to the first apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold.

[0011] In a seventh aspect of the present disclosure, there is provided a computer readable medium. The computer readable medium comprises instructions stored thereon for causing an apparatus to perform at least the method according to the third aspect.

[0012] In an eighth aspect of the present disclosure, there is provided a computer readable medium. The computer readable medium comprises instructions stored thereon for causing an apparatus to perform at least the method according to the fourth aspect.

[0013] It is to be understood that the Summary section is not intended to identify key or essential features of embodiments of the present disclosure, nor is it intended to be used to limit the scope of the present disclosure. Other features of the present disclosure will become easily comprehensible through the following description.BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Some example embodiments will now be described with reference to the accompanying drawings, where:

[0015] FIG. 1 illustrates an example communication environment in which example embodiments of the present disclosure can be implemented;

[0016] FIG. 2 illustrates an example signaling chart for prediction and test mechanism enhancement according to some example embodiments of the present disclosure;

[0017] FIG. 3 illustrates another example signaling chart for prediction and test mechanism enhancement according to some example embodiments of the present disclosure;

[0018] FIG. 4 illustrates a further example signaling chart for prediction and test mechanism enhancement according to some example embodiments of the present disclosure;

[0019] FIG. 5 illustrates a flowchart of a method implemented at a first apparatus according to some example embodiments of the present disclosure;

[0020] FIG. 6 illustrates a flowchart of a method implemented at a second apparatus according to some example embodiments of the present disclosure;

[0021] FIG. 7 illustrates a simplified block diagram of a device that is suitable for implementing example embodiments of the present disclosure; and

[0022] FIG. 8 illustrates a block diagram of an example computer readable medium in accordance with some example embodiments of the present disclosure.

[0023] Throughout the drawings, the same or similar reference numerals represent the same or similar element.DETAILED DESCRIPTION

[0024] Principle of the present disclosure will now be described with reference to some example embodiments. It is to be understood that these embodiments are described only for the purpose of illustration and help those skilled in the art to understand and implement the present disclosure, without suggesting any limitation as to the scope of the disclosure. Embodiments described herein can be implemented in various manners other than the ones described below.

[0025] In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skills in the art to which this disclosure belongs.

[0026] References in the present disclosure to “one embodiment,” “an embodiment,” “an example embodiment,” and the like indicate that the embodiment described may include a particular feature, structure, or characteristic, but it is not necessary that every embodiment includes the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0027] It shall be understood that although the terms “first,” “second,”..., etc. in front of noun(s) and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another and they do not limit the order of the noun(s). For example, a first elementcould be termed a second element, and similarly, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0028] As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or”, mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements.

[0029] As used herein, unless stated explicitly, performing a step “in response to A” does not indicate that the step is performed immediately after “A” occurs and one or more intervening steps may be included.

[0030] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises”, “comprising”, “has”, “having”, “includes” and / or “including”, when used herein, specify the presence of stated features, elements, and / or components etc., but do not preclude the presence or addition of one or more other features, elements, components and / or combinations thereof.

[0031] As used in this application, the term “circuitry” may refer to one or more or all of the following:(a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry) and(b) combinations of hardware circuits and software, such as (as applicable):(i) a combination of analog and / or digital hardware circuit(s) with software / firmware and(ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and(c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.

[0032] This definition of circuitry applies to all uses of this term in this application,including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

[0033] As used herein, the term “communication network” refers to a network following any suitable communication standards, such as New Radio (NR), Long Term Evolution (LTE), LTE-Advanced (LTE-A), Wideband Code Division Multiple Access (WCDMA), High-Speed Packet Access (HSPA), Narrow Band Internet of Things (NB-IoT) and so on. Furthermore, the communications between a terminal device and a network device in the communication network may be performed according to any suitable generation communication protocols, including, but not limited to, the first generation (1G), the second generation (2G), 2.5G, 2.75G, the third generation (3G), the fourth generation (4G), 4.5G, the fifth generation (5G), the sixth generation (6G) communication protocols, and / or any other protocols either currently known or to be developed in the future. Embodiments of the present disclosure may be applied in various communication systems. Given the rapid development in communications, there will of course also be future type communication technologies and systems with which the present disclosure may be embodied. It should not be seen as limiting the scope of the present disclosure to only the aforementioned system.

[0034] As used herein, the term “network device” refers to a node in a communication network via which a terminal device accesses the network and receives services therefrom. The network device may refer to a base station (BS) or an access point (AP), for example, a node B (NodeB or NB), an evolved NodeB (eNodeB or eNB), an NR NB (also referred to as a gNB), a Remote Radio Unit (RRU), a radio header (RH), a remote radio head (RRH), a relay, an Integrated Access and Backhaul (IAB) node, a low power node such as a femto, a pico, a non-terrestrial network (NTN) or non-ground network device such as a satellite network device, a low earth orbit (LEO) satellite and a geosynchronous earth orbit (GEO) satellite, an aircraft network device, and so forth, depending on the applied terminology and technology. In some example embodiments, radio access network (RAN) split architecture comprises a Centralized Unit (CU) and a Distributed Unit (DU) at an IAB donor node. An IAB node comprises a Mobile Terminal (IAB-MT) part that behaveslike a UE toward the parent node, and a DU part of an IAB node behaves like a base station toward the next-hop IAB node.

[0035] The term “terminal device” refers to any end device that may be capable of wireless communication. By way of example rather than limitation, a terminal device may also be referred to as a communication device, user equipment (UE), a Subscriber Station (SS), a Portable Subscriber Station, a Mobile Station (MS), or an Access Terminal (AT). The terminal device may include, but not limited to, a mobile phone, a cellular phone, a smart phone, voice over IP (VoIP) phones, wireless local loop phones, a tablet, a wearable terminal device, a personal digital assistant (PDA), portable computers, desktop computer, image capture terminal devices such as digital cameras, gaming terminal devices, music storage and playback appliances, vehicle-mounted wireless terminal devices, wireless endpoints, mobile stations, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), USB dongles, smart devices, wireless customer-premises equipment (CPE), an Internet of Things (loT) device, a watch or other wearable, a head-mounted display (HMD), a vehicle, a drone, a medical device and applications (e.g., remote surgery), an industrial device and applications (e.g., a robot and / or other wireless devices operating in an industrial and / or an automated processing chain contexts), a consumer electronics device, a device operating on commercial and / or industrial wireless networks, and the like. The terminal device may also correspond to a Mobile Termination (MT) part of an IAB node (e.g., a relay node). In the following description, the terms “terminal device”, “communication device”, “terminal”, “user equipment” and “UE” may be used interchangeably.

[0036] As used herein, the term “resource,” “transmission resource,” “resource block,” “physical resource block” (PRB), “uplink resource,” or “downlink resource” may refer to any resource for performing a communication, for example, a communication between a terminal device and a network device, such as a resource in time domain, a resource in frequency domain, a resource in space domain, a resource in code domain, or any other combination of the time, frequency, space and / or code domain resource enabling a communication, and the like. In the following, unless explicitly stated, a resource in both frequency domain and time domain will be used as an example of a transmission resource for describing some example embodiments of the present disclosure. It is noted that example embodiments of the present disclosure are equally applicable to other resources in other domains.

[0037] As discussed above, use cases for beam management have been studied forspatial and / or time beam prediction. Study is being carried out for AI / ML for air-interface corresponding to each target use case regarding aspects such as performance, complexity, and potential specification impact.

[0038] The use cases are focusing on several aspects, for example, Initial set of use cases, finalize representative sub use cases for each use case for characterization and baseline performance evaluations sub use cases. The initial set of use cases may include CSI feedback enhancement, e.g., overhead reduction, improved accuracy, prediction; the beam management, e.g., beam prediction in time, and / or spatial domain for overhead and latency reduction, beam selection accuracy improvement; positioning accuracy enhancements for different scenarios including, e.g., those with heavy NLOS conditions; finalize representative sub use cases for each use case for characterization and baseline performance evaluations; and the AI / ML approaches for the selected sub use cases need to be diverse enough to support various requirements on the gNB-UE collaboration levels.

[0039] For the use cases under consideration, performance benefits of AI / ML based algorithms for the agreed use cases in the final representative set needs to be evaluated. Meanwhile, potential specification impact, specifically for the agreed use cases in the final representative set and for a common framework needs to be assessed.

[0040] Regarding evaluation of performance benefits of AI / ML based algorithms for the agreed use cases in the final representative set, there is methodology based on statistical models, for link and system level simulations. The methodology may include, for example, the following aspects.• Extensions of 3 GPP evaluation methodology for better suitability to AI / ML based techniques should be considered as needed.• Whether field data are optionally needed to further assess the performance and robustness in real-world environments should be discussed as part of the study.• Need for common assumptions in dataset construction for training, validation and test for the selected use cases.• Consider adequate model training strategy, collaboration levels and associated implications.• Consider agreed-upon base Al model(s) for calibration.• Al model description and training methodology used for evaluation should be reported for information and cross-checking purposes.

[0041] Regarding Key Performance Indicator (KPI) for the evaluation, common KPIs and corresponding requirements for the AI / ML operations may be determined, and theuse-case specific KPIs and benchmarks of the selected use-cases may be also determined. In such determinations, performance, inference latency and computational complexity of AI / ML based algorithms may be compared to that of a state-of-the-art baseline. In addition, overhead, power consumption (including computational), memory storage, and hardware requirements (including for given processing delays) associated with enabling respective AI / ML scheme, as well as generalization capability may be considered.

[0042] As for assess of potential specification impact, specifically for the agreed use cases in the final representative set and for a common framework, both PHY layer aspects and protocol aspects may be considered.

[0043] As for the PHY layer aspects, it may consider aspects related to, e.g., the potential specification of the Al Model lifecycle management, and dataset construction for training, validation and test for the selected use cases. Use case and collaboration level specific specification impact may be also considered, which includes, for example, new signalling, means for training and validation data assistance, assistance information, measurement, and feedback.

[0044] As for the Protocol aspects, it may consider aspects related to, e.g., capability indication, configuration and control procedures (training / inference), and management of data and AI / ML model.

[0045] There are some core requirements for AI / ML, for example, performance monitoring procedure, including performance evaluation and decision making procedure for AI / ML functionalities / models, functionality / Model management procedure, including functionality / model selection / activation / deactivation, and functionality / model switching / fallback / transfer / delivery / update, latency / interruption requirement for above procedures, and so on.

[0046] When designing a high level testing framework , performance needs to be guaranteed and to avoid that a UE can easily pass the test but perform poorly in the field. This framework is not directly enforceable but should be considered for all the tests to be introduced. This also applies to LCM tests, if they are defined. Some LCM related requirements may be considered, for example, model / Functionality select / switch / activate / deactivate / fallback, model / Functionality monitoring, and so on.

[0047] Metrics for evaluation of beam management inference performance are studied. Metrics / KPIs for Beam prediction requirements / tests may include, for example, reference signal received power (RSRP) accuracy, beam prediction accuracy (Top-1(%), Top-K(%)), the successful rate for the correct prediction which is considered as maximum RSRPamong top-K predicted beams is larger than the RSRP of the strongest beam - x dB, related measurement accuracy can be considered to determine x, overhead / latency reduction, combinations of above options, and so on. The overhead / latency reduction should be considered for the requirements as the side condition.

[0048] Currently, study is made toward the network (NW) side and / or UE side model training as well as the model inference for AI / ML beam management. One of the aspects is the choice and definition of the KPIs of the testing of the output of AI / ML model / functionality training / inference.

[0049] When the UE runs inference or training for AI / ML RSRP and beam ID prediction in spatial domain and / or time domain, the NW or test equipment (TE) should be able to test the prediction accuracy of signal strength, such as RSRP, and with overhead reduction as side conditions well.

[0050] According to some example embodiments of the present disclosure, there is provided a solution for the test mechanism for RSRP prediction, with overhead reduction in reporting mechanism.

[0051] FIG. 1 illustrates an example communication environment 100 in which example embodiments of the present disclosure may be implemented. As shown in FIG. 1, the communication network 100 may include a first apparatus 110 and a second apparatus 120. The first apparatus 110 may communicate with the second apparatus 120. A prediction model may be deployed at the second apparatus 120. The prediction model may be used for any suitable use cases or to implement any suitable functionalities, for example but not limited to, channel state information (CSI) feedback enhancement, beam management (BM) (also referred to as beam prediction), positioning, etc. In some embodiments, the prediction model may be implemented with an AL / ML model or other suitable model. Monitoring or testing the performance of the prediction model may be needed.

[0052] It is to be understood that the number of second apparatus and first apparatus shown in FIG. 1 is given for the purpose of illustration without suggesting any limitations. The communication network 100 may include any suitable number of second apparatus and first apparatus.

[0053] In some example embodiments, the first apparatus 110 may include a test equipment (TE), which may be a network device for example, and the second apparatus 120 may include a device under test (DUT), which may be a terminal device. For example, an AI / ML model may be implemented at the second apparatus 120. Alternatively, a partof the AI / ML model may be implemented at the second apparatus 120. The AI / ML model (also referred to as a “model”) may provide a functionality, for example, beam prediction, signal strength prediction, CSI prediction, and so on.

[0054] As used herein, the term “communication functionality” may refer to a functionality or service provided by the prediction mode, for example, an AI / ML model. The term “communication functionality” may also be referred to as a “AI / ML based functionality” or “AI / ML enabled functionality” or “AI / ML based use case”. In some example embodiments, the prediction model at the second apparatus 120 may need to be tested or validated. In other words, the communication functionality may need to be tested or validated. For example, generalization of the communication functionality may need to be validated.

[0055] In some example embodiments, the first apparatus 110 may be a test equipment for example, a network device (e.g., a gNB), and the second apparatus 120 may include a terminal device (for example, a UE).

[0056] In the following, for the purpose of illustration, some example embodiments are described with the first apparatus 110 operating as a network device and the second apparatus 120 operating as a terminal device. However, in some example embodiments, operations described in connection with a terminal device may be implemented at a network device or other device, and operations described in connection with a network device may be implemented at a terminal device or other device.

[0057] In some example embodiments, if the first apparatus 110 is a network device and the second apparatus 120 is a terminal device, a link from the second apparatus 120 to the first apparatus 110 is referred to as an uplink (UL), and a link from the first apparatus 110 to the second apparatus 120 is referred to as a downlink (DL). In UL, the second apparatus 120 is a transmitting (TX) device (or a transmitter) and the first apparatus 110 is a receiving (RX) device (or a receiver). In DL, the first apparatus 110 is a TX device (or a transmitter) and the second apparatus 120 is a RX device (or a receiver).

[0058] Communications in the communication environment 100 may be implemented according to any proper communication protocol(s), comprising, but not limited to, cellular communication protocols of the first generation (1G), the second generation (2G), the third generation (3G), the fourth generation (4G), the fifth generation (5G), the sixth generation (6G), and the like, wireless local network communication protocols such as Institute for Electrical and Electronics Engineers (IEEE) 802.11 and the like, and / or any other protocols currently known or to be developed in the future. Moreover, thecommunication may utilize any proper wireless communication technology, comprising but not limited to: Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Frequency Division Duplex (FDD), Time Division Duplex (TDD), Multiple-Input Multiple-Output (MIMO), Orthogonal Frequency Division Multiple (OFDM), Discrete Fourier Transform spread OFDM (DFT-s-OFDM) and / or any other technologies currently known or to be developed in the future.

[0059] Example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0060] FIG. 2 illustrates an example signaling chart 200 for prediction and test mechanism enhancement according to some example embodiments of the present disclosure. For the purposes of discussion, the signaling chart 200 will be discussed with reference to FIG. 1, for example, by using the first apparatus 110 and the second apparatus 120.

[0061] As shown in FIG. 2, the first apparatus 110 transmits (205) a first configuration and a second configuration to the second apparatus 120. The first configuration is for measurements associated with a first set of beams and the second configuration is for prediction with a prediction model associated with a second set of beams.

[0062] The second apparatus 120, upon receiving (210) the first configuration and the second configuration, performs the measurements and / or prediction according to the first configuration and the second configuration and obtains a measurement result and a prediction result, respectively.

[0063] The measurement result is determined from measurements on the first set of beams based on the first configuration and indicates measured signal strengths of a third set of beams. The third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold. In some example embodiments, the measurement result may comprise strongest N measured RSRP values of the third set of beams, where N is a predetermined positive number.

[0064] The prediction result is obtained based on the prediction model and the second set of beams and indicates predicted signal strengths of a fourth set of beams. The fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold. In some example embodiments, the prediction result comprises strongest N predicted RSRP values of the fourth set of beams.The number of predicted RSRP values in the prediction result may be the same as the number of measured RSRP values in the prediction result.

[0065] In some embodiments, the first apparatus 110 may transmit, to the second apparatus 120, a message requesting the second apparatus to enter a beam prediction mode which enables the prediction with the model. The beam prediction mode may be, for example, an AI / ML BM mode which enables the prediction with the AI / ML model.

[0066] After entering the beam prediction mode, the second apparatus 120 may transmit an indication that the second apparatus 120 is operating in the beam prediction mode. The first apparatus 110, upon receiving such an indication, will be aware that the second apparatus 120 has accepted the request and now is operating in the beam prediction mode.

[0067] In some embodiments, the first apparatus 110 may transmit a configuration and CSI-RS resource of the second set of beams to the second apparatus 120. For example, the second apparatus 120 may be configured to use 16 beams in the second set of beams to predict Top-1 or Top-4 or Top-8 of the first set of beams (e.g., 64 beams in total).

[0068] In some example embodiments, the first apparatus 110 transmits, to the second apparatus 120, information about one or more resources of at least one of a synchronization signal block (SSB) or channel state information reference signal (CSI- RS). Such information may be related to the second set of beams. The second apparatus 120 receives the information about the resources from the first apparatus 110 and may use it in the subsequent prediction associated with the second set of beams. For example, the second apparatus 120 may be configured to use 16 beams in the second set of beams to predict Top-1 or Top-4 or Top-8 of the first set of beams (e.g., 64 beams in total). In this case, the second set of beams may be wide beams and the first set of beams may be narrow beams.

[0069] In addition, in some example embodiments, the first apparatus 110 may transmit, to the second apparatus 120, an indication indicating whether the SSB or the CSI-RS are to be used. Upon receiving the indication, the second apparatus 120 may be aware whether to use the SSB or the CSI-RS to perform the measurements / prediction.

[0070] In some example embodiments, the first apparatus 110 may transmit, to the second apparatus 120, a request to prepare signal strengths of the third set of beams. The second apparatus 120 may, in response to receiving this request, perform measurements on the first set of beams and may find out the third set of beams accordingly. The third set of beams may be those beams with strongest N measured RSRP values among RSRP values of the first set of beams.

[0071] The first apparatus 110 may configure the second apparatus 120 about how to report the prediction result and the measurement result. In some example embodiments, the first apparatus 110 may transmit, to the second apparatus 120, a third configuration for reporting the prediction result and the measurement result of the third set of the beams. The second apparatus 120 may report the prediction result and the measurement result according to the configured way.

[0072] Still referring to FIG. 2, the second apparatus 120 transmits (215) the measurement result and the prediction result to the first apparatus 110. The first apparatus 110 receives (220) the measurement result and the prediction result from the second apparatus 120 and determines (225) a result of a test of the prediction model based on the received measurement result and the prediction result.

[0073] The first apparatus 110 may determine (225) the result of the test in a variety of ways. In some example embodiments, the first apparatus 110 may determine whether a first range of values in the measurement result of the third set of beams matches with a second range of values in the prediction result of the fourth set of beams. If the first range mismatches with the second range, the first apparatus 110 may determine that the test of the prediction model is failed.

[0074] On the other side, if the first range matches with the second range, the first apparatus 110 may further validate the test. For instance, the first apparatus 110 may determine whether a difference between the maximum value in the first range and the maximum value in the second range is less than a tolerance margin. If the difference is less than the tolerance margin, the first apparatus 110 may determine that the test of the prediction model is passed. Otherwise, the first apparatus 110 may determine that the test of the prediction model is failed.

[0075] In an example, the first apparatus 110 may verify if the range of strongest RSRP matches with the range of strongest RSRP measurement. If they are not match, the test is failed. If both ranges match, the first apparatus 110 may compare the strongest beam RSRP with the predicted strongest beam RSRP. The test may be considered as passed if a difference (which is, for example, an absolute value) between a range of predicted RSRP of strongest beam and a range of measured RSRP values of strongest beam is less than a tolerance margin (for example, x dB), which may be illustrated as below:|range of predicted RSRP of strongest beam - range of measured RSRP values of strongest beam| < tolerance marginOtherwise, the test is Failed.

[0076] In addition to validate only the predicted signal strengths of strongest beams as discussed above, Top-K signal strength range may be further or alternatively considered. Top-K signal strength range may include a range of strength range values of strongest beams, a range of strength range values of second strongest beams, ..., a range of strength range values of Kth strongest beams, where K is a positive integer number.

[0077] In some example implementations, if the first range matches with the second range, the first apparatus 110 may determine whether a difference between the maximum value in the first range and the maximum value in the second range is less than a tolerance margin. If the difference is less than the tolerance margin, the first apparatus 110 may consider another vale range of the beams.

[0078] For example, the first apparatus 110 may determine whether a further difference between at least one value in the first range and at least one corresponding value in the second range is less than a further tolerance margin. If the further difference is less than the further tolerance margin, the first apparatus 110 may determine that the test of the prediction model is passed. Otherwise, the first apparatus 110 may determine that the test of the prediction model is failed.

[0079] In an implementation, the first apparatus 110 may first verify if the range of strongest RSRP are validated. Then, the first apparatus 110 may verify the predicted second, third until K-th strongest RSRP compare with the second, third until K-th strongest measured RSRP. If |range of predicted RSRP of second strongest beam - range of measured RSRP values of second strongest beam| < tolerance margin (for example, y dB), the test is passed. Otherwise, the test is failed. Next, TE the first apparatus 110 may keep verifying the range of third strongest RSRP until the K-th strongest RSRP.

[0080] It is to be understood that the tolerance margin requirements of strongest, second strongest,..., K-th strongest beam may be different requirements.

[0081] In some example embodiments, the second set of beams is a subset of the first set of beams, or the second set of beams is different from the first set of beams.

[0082] In some example embodiments, the measurement result comprises strongest N measured reference signal received power (RSRP) values of the third set of beams, the prediction result comprises strongest N predicted RSRP values of the fourth set of beams, and N is a predetermined positive number.

[0083] In view of the above, example embodiments of the present disclosure provide a testing framework including some impacts on the interface between the UE and the Test Equipment (TE) / NW in order to verify and validate beam prediction accuracy KPI.

[0084] In the proposed testing framework, first, a mode of functioning at the second apparatus 120 (that is, the UE side) is well known at the first apparatus 110 (that is, at the TE side). This may be achieved by a command from the TE to the UE to work in a specific mode. For instance, TE will send a command to the UE to enable AI / ML based beam management use case. The UE may then confirm the activation of the requested mode.

[0085] Optionally, the TE then configures the UE to report RSRP of Top-K strongest measured beams and predicted RSRP of Top-K beams simultaneously. As a further option, the theoretical value of the strongest beam may already be known at the TE level. It may be derived from the configuration of the test setup.

[0086] Then, UE may start RSRP prediction using a subset of beams configured by the TE. Then, UE may report measured RSRP of strongest beams and predicted RSRP of Top- K beams to the TE. As such, TE may verify if the range of strongest RSRP matches with the range of strongest RSRP measurement. Example embodiments of the verification performed by TE have been discussed above and thus are not discussed here in detail.

[0087] In the following, two example embodiments of the present disclosure are discussed with reference to FIGS. 3 and 4. In these example embodiments, the first apparatus 110 is discussed as a test equipment, such as a network device, and the second apparatus 120 is discussed as a terminal device, such as a UE. The testing framework for RSRP of Top-K beam prediction for AI / ML based functionality for beam management use-case is discussed in these example embodiments. For example, the AI / ML based functionality may be e.g., AI / ML beam management for Top-K or Top-1 RSRP prediction of Top-K beam(s) in spatial domain and time domain.

[0088] In these example embodiments, it is assumed that the UE performs training or inference to obtain the prediction. For AI / ML beam management use-case, the UE could use layer one (Ll)-RSRP (LI -RSRP) measurements as input for neural network to perform prediction. The measurement data needs to be accurate enough for high prediction accuracy.

[0089] In AI / ML beam management, BM-Casel (Spatial domain beam prediction) and BM-Case2 (Temporal domain beam prediction) are considered. For purpose of discussion, the first set of beams is also referred to as Set A and the second set of beams is also referred to as Set B in the following description.

[0090] Regarding the BM-Casel, there are several Baseline-options. In BM-Casel Baseline-option 1, the best beam within Set A of beams is selected based on the measurement of all RS resources or all possible beams of beam Set A (exhaustive beamsweeping). In BM-Casel Baseline-option 2-1 (Set B is a subset of Set A / Set B is different from Set A), the best beam within Set B is selected. In BM-Casel Baseline-option 2-2 (Set B is different from Set A), a hierarchical search for the best narrow beam from the best wide beam is performed.

[0091] For BM-Casel, Set B is subset of Set A (Top-1 or Top-K beam IDs prediction). The ML model input may be Set B beam Ll-RSRP, and the ML model output may be Top-1 or Top-K beam IDs of Set A. Model training and testing may be performed with the same Set B.

[0092] Regarding the BM-Case2, there are several Baseline-options as well. In BM- Case2 Baseline-option 1, the best beam within Set A of beams is selected based on the measurement of all RS resources or all possible beams of beam Set A (exhaustive beam sweeping). In BM-Case2 Baseline-option 2-1 (Set B is a subset of Set A / Set B is different from Set A), the best beam within Set B is selected. In BM-Case2 Baseline-option 2-2 (Set B is different from Set A), a hierarchical search for the best narrow beam from the best wide beam is performed.

[0093] Example embodiments focus on testing the RSRP of Top-K beam prediction in both BM-Casel and BM-Case2 when Set B is subset of Set A and when Set B is different from Set A, where the prediction output may be predicted RSRP of Top-K of Top-K beams of Set A. The details of the reporting of Set A measurements and testing mechanism when TE-UE are in command mode are shown in FIG. 3 and FIG. 4.

[0094] FIG. 3 illustrates an example signaling chart 300 for prediction and test mechanism enhancement according to some example embodiments of the present disclosure. It is to be understood that the signaling chart 300 is an example implementation of the signaling chart 200. For the purposes of discussion, similar to the signaling chart 200, the signaling chart 300 will be discussed with reference to FIG. 1, for example, by using the first apparatus 110 (also referred to as TE / NW hereafter) and the second apparatus 120 (also referred to as UE hereafter). In the signaling chart 300, a testing mechanism is described with reference to a case where the second apparatus 120 (referred to as UE) is configured to predict RSRP of Top-K beams of a first set of beams (e.g., denoted as Set A). In this case, the second set of beams (e.g., denoted as Set B) is a subset of Set A.

[0095] At 305, the TE / NW sends the command to UE to switch to a prediction model, e.g., an AI / ML mode, based functionality.

[0096] At 310, the UE sends confirmation to the NW / TE that it operates in AI / ML BMmode.

[0097] At 315, the UE sends a functionality indication, e.g., an indication whether UE runs inference or training in Top-K DL beam management in spatial or Top-K DL beam management in time domain.

[0098] At 320, the TE / NW checks the indication whether UE runs inference for RSRP and beam IDs prediction of Top-K beams in Set A.

[0099] At 325, the NW / TE sends configurations to UE to measure the whole Set A.

[0100] At 330, the NW sends configuration and CSI-RS resource of fixed set B beams to UE (e.g., UE might be configured to use 16 Set B beams to predict Top-1 or Top-4 or Top-8 of 64 Set A beams).

[0101] At 335, the TE / NW sends the request to UE to prepare RSRP values of Top-K strongest beams of Set A beams.

[0102] At 340, the TE / NW configures UE to report RSRP prediction of Top-K beams results and configures UE to report RSRP values of Top-K strongest beams in Set A measurements.

[0103] Alternatively, in some example embodiments, the theoretical value of the strongest RSRP, second strongest RSRP,..., K-th strongest RSRP may already be known at the TE level. It may be derived from the configuration of the test setup.

[0104] At 345, the UE performs prediction of RSRP of Top-K beam in Set A using Ll- RSRP of Set B beams as input to neural network.

[0105] At 350, the UE reports predicted RSRP and beam IDs of Top-K beams of Set A to the TE / NW.

[0106] At 355, the UE reports the RSRP measurements of Top-K beams of Set A beams to the TE / NW.

[0107] At 360, the TE / NW determines a result of a test of the prediction model based on the measurement result and the prediction result in various ways.

[0108] In one option, the TE / NW validates only the predicted RSRP of strongest beam. Specifically, the TE / NW validates whether the predicted RSRP of Top-K beams include the RSRP range of strongest beam. If the RSRP range of strongest beam matches with the predicted RSRP of strongest beam, it may determine that the test is passed. Alternatively or in addition, if the predicted RSRP of the strongest beam matches with the RSRP of the measured strongest beam then, it may determine that there is no error. If the range of predicted RSRP of the strongest beam does not match with the range of RSRP of measured strongest beam, the TE / NW may further check the tolerance margin. For example, it maydetermine whether the following condition is met,(predicted RSRP of strongest beam - RSRP values of strongest beam| < tolerance margin (x dB).If the above condition is met, the TE / NW may determine that the test validates.

[0109] Otherwise, if | predicted RSRP of strongest beam - RSRP values of strongest beam| > tolerance margin (xx dB), the TE / NW may determine that the test fails.

[0110] In another option, after validating the predicted RSRP of strongest beam, then TE validates the second, third and K-th strongest beam as well. If the RSRP range of second strongest beam matches with the predicted RSRP of second strongest beam, it may determine that the test is passed. Alternatively or in addition, if the predicted RSRP of the second strongest beam matches with the RSRP of the second strongest beam then, it may determine that there is no error.[OHl] If the range of predicted RSRP of the second strongest beam does not match with the range of RSRP of second strongest beam, the TE / NW may further check the tolerance margin. For example, if (predicted RSRP of second strongest beam - RSRP values of second strongest beam(s)| < tolerance margin (yy dB), it may determine that the test validates. If | predicted RSRP of second strongest beam - RSRP values of second strongest beam(s)| > tolerance margin (y dB), the TE / NW may determine the test fails.

[0112] The tolerance margin (y dB) is a new requirement for second strongest beam. However, it could be the same requirements (tolerance margin) with the RSRP of the strongest beam as well (x dB). In example embodiments of the present disclosure, x and y may be any suitable values for tolerance margins, which does not suggest any limitation to the present disclosure.

[0113] Also the TE / NW may keep validating the RSRP of the third,... until K-th strongest beam.

[0114] Now reference is made to FIG. 4, which illustrates a further example signaling chart 400 for prediction and test mechanism enhancement according to some example embodiments of the present disclosure. It is to be understood that the signaling chart 400 is another example implementation of the signaling chart 200. For the purposes of discussion, similar to the signaling chart 200, the signaling chart 400 will be discussed with reference to FIG. 1, for example, by using the first apparatus 110 (e.g., a test equipment, such as a network device) and the second apparatus 120 (e.g., a terminal device or UE). Similar to the signaling chart 300, in the signaling chart 400, a testing mechanism is described with reference to a case where the second apparatus 120 (referred to as UE)is configured to predict RSRP of Top-K beams of a first set of beams (e.g., denoted as Set A). In contrast to example embodiments discussed with reference to FIG. 3, in this case, the second set of beams (e.g., denoted as Set B) is different from Set A.

[0115] At 405, the TE / NW sends the command to UE to switch to a prediction model, e.g., an AI / ML mode, based functionality.

[0116] At 410, the UE sends confirmation to the NW / TE that it operates in AI / ML BM mode.

[0117] At 415, the UE sends a functionality indication, e.g., an indication whether UE runs inference or training in Top-K DL beam management in spatial or Top-K DL beam management in time domain.

[0118] At 420, the TE / NW checks the indication whether UE runs inference for Top-K beam ID prediction.

[0119] At 425, the NW / TE sends configurations and SSB or CSLRS resources for measurements of fixed Set B to UE (e.g., UE might be configured to use 16 Set B beams to predict Top-1 or Top-4 or Top-8 of 64 Set A beams, where Set B beams is wide beam and Set A beams is narrow beams).

[0120] At 430, the NW / TE sends an indication (e.g., a pointer) that the configured Set B is CSI-RS or SSB.

[0121] At 435, the TE / NW sends the configuration of SSB or CSI-RS resources of Top- K of Set A beams.

[0122] At 440, the TE / NW configures UE to report RSRP prediction of Top-K beams results and configures UE to report RSRP values of Top-K strongest beams in Set A measurements.

[0123] Alternatively, in some example embodiments, the theoretical value of the strongest beam may already be known at the TE level. It may be derived from the configuration of the test setup.

[0124] At 445, the UE performs prediction of RSRP of Top-K beam in Set A using Ll- RSRP of Set B beams as input to neural network.

[0125] At 450, the UE reports predicted RSRP and beam IDs of Top-K beams of Set A to the TE / NW.

[0126] At 455, the UE reports the RSRP measurements of Top-K beams of Set A beams to the TE / NW.

[0127] At 460, the TE / NW determines a result of a test of the prediction model based on the measurement result and the prediction result in various ways. The details of thedetermination of the result of the test have been discussed with reference to step 360, and thus is not repeated here.

[0128] In view of the above, the testing mechanism proposed in example embodiments can achieve overhead reduction. The overhead is reduced significantly since the UE is configured to report the Top-K beams in Set A (not the whole Set A and exclude Set B). Meanwhile, the KPIs (RSRP prediction with tolerance margin) can be tested for real-time monitoring mechanism.

[0129] FIG. 5 shows a flowchart of an example method 500 implemented at a first apparatus in accordance with some example embodiments of the present disclosure. For the purpose of discussion, the method 500 will be described from the perspective of the first apparatus 110 in FIG. 1.

[0130] At block 510, the first apparatus 110 transmits, to a second apparatus 120, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams;

[0131] At block 520, the first apparatus 110 receives, from the second apparatus 120, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold.

[0132] At block 530, the first apparatus 110 determines a result of a test of the prediction model based on the measurement result and the prediction result.

[0133] In some example embodiments, the method 500 further comprises: transmitting, to the second apparatus, a message requesting the second apparatus to enter a beam prediction mode which enables the prediction with the model; and receiving, from the second apparatus, an indication that the second apparatus is operating in the beam prediction mode.

[0134] In some example embodiments, the method 500 further comprises: transmitting, to the second apparatus, information about one or more resources of at least one of a synchronization signal block (SSB) or channel state information reference signal (CSI- RS).

[0135] In some example embodiments, the method 500 further comprises: transmitting, to the second apparatus, an indication indicating whether the measurements are to be performed on the SSB or the CSI-RS.

[0136] In some example embodiments, the method 500 further comprises: transmitting, to the second apparatus, a request to prepare signal strengths of the third set of beams.

[0137] In some example embodiments, the method 500 further comprises: transmitting, to the second apparatus, a third configuration for reporting the prediction result and the measurement result of the third set of the beams.

[0138] In some example embodiments, the method 500 further comprises: determining whether a first range of values in the measurement result of the third set of beams matches with a second range of values in the prediction result of the fourth set of beams; and in accordance with a determination that the first range mismatches with the second range, determining that the test of the prediction model is failed.

[0139] In some example embodiments, the method 500 further comprises: in accordance with a determination that the first range matches with the second range, determining whether a difference between the maximum value in the first range and the maximum value in the second range is less than a tolerance margin; in accordance with a determination that the difference is less than the tolerance margin, determining that the test of the prediction model is passed; and in accordance with a determination that the difference exceeds the tolerance margin, determining that the test of the prediction model is failed.

[0140] In some example embodiments, the method 500 further comprises: in accordance with a determination that the first range matches with the second range, determining whether a difference between the maximum value in the first range and the maximum value in the second range is less than a tolerance margin; in accordance with a determination that the difference is less than the tolerance margin, determining whether a further difference between at least one value in the first range and at least one corresponding value in the second range is less than a further tolerance margin; and in accordance with a determination that the further difference is less than the further tolerance margin, determining that the test of the prediction model is passed; and in accordance with a determination that the further difference exceeds the further tolerance margin, determining that the test of the prediction model is failed.

[0141] In some example embodiments, the second set of beams is a subset of the first set of beams, or wherein the second set of beams is different from the first set of beams.

[0142] In some example embodiments, the measurement result comprises strongest N measured reference signal received power (RSRP) values of the third set of beams, the prediction result comprises strongest N predicted RSRP values of the fourth set of beams, and N is a predetermined positive number.

[0143] In some example embodiments, the first apparatus comprises a test equipment or a network device, and the second apparatus comprises a terminal device.

[0144] FIG. 6 shows a flowchart of an example method 600 implemented at a second apparatus in accordance with some example embodiments of the present disclosure. For the purpose of discussion, the method 600 will be described from the perspective of the second apparatus 120 in FIG. 1.

[0145] At block 610, the second apparatus 120 receives, from a first apparatus 110, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams.

[0146] At block 620, the second apparatus 120 transmits, to the first apparatus 110, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold.

[0147] In some example embodiments, the method 600 further comprises: receiving, from the first apparatus, a message requesting the second apparatus to enter a beam prediction mode which enables the prediction with the model; and transmitting, to the first apparatus, an indication that the second apparatus is operating in the beam prediction mode.

[0148] In some example embodiments, the method 600 further comprises: receiving, from the first apparatus, information about one or more resources of at least one of a synchronization signal block (SSB) or channel state information reference signal (CSI- RS).

[0149] In some example embodiments, the method 600 further comprises: receiving, from the first apparatus, an indication indicating whether the measurements are to be performed on the SSB or the CSI-RS.

[0150] In some example embodiments, the method 600 further comprises: receiving, from the first apparatus, a request to prepare signal strengths of the third set of beams.

[0151] In some example embodiments, the method 600 further comprises: receiving, from the first apparatus, a third configuration for reporting the prediction result and the measurement result of the third set of the beams.

[0152] In some example embodiments, the second set of beams is a subset of the first set of beams, or wherein the second set of beams is different from the first set of beams.

[0153] In some example embodiments, the measurement result comprises strongest N measured reference signal received power (RSRP) values of the third set of beams, the prediction result comprises strongest N predicted RSRP values of the fourth set of beams, and N is a predetermined positive number.

[0154] In some example embodiments, the first apparatus comprises a test equipment or a network device, and the second apparatus comprises a terminal device.

[0155] In some example embodiments, a first apparatus capable of performing any of the method 500 (for example, the first apparatus 110 in FIG. 1) may comprise means for performing the respective operations of the method 500. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module. The first apparatus may be implemented as or included in the first apparatus 110 in FIG. 1.

[0156] In some example embodiments, the first apparatus comprises means for transmitting, to a second apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams; means for receiving, from the second apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold; and means for determining a result of a test of the prediction model based on the measurement result and the prediction result.

[0157] In some example embodiments, the first apparatus further comprises: means for transmitting, to the second apparatus, a message requesting the second apparatus to entera beam prediction mode which enables the prediction with the model; and means for receiving, from the second apparatus, an indication that the second apparatus is operating in the beam prediction mode.

[0158] In some example embodiments, the first apparatus further comprises: means for transmitting, to the second apparatus, information about one or more resources of at least one of a synchronization signal block (SSB) or channel state information reference signal (CSI-RS).

[0159] In some example embodiments, the first apparatus further comprises: means for transmitting, to the second apparatus, an indication indicating whether the measurements are to be performed on the SSB or the CSI-RS.

[0160] In some example embodiments, the first apparatus further comprises: means for transmitting, to the second apparatus, a request to prepare signal strengths of the third set of beams.

[0161] In some example embodiments, the first apparatus further comprises: means for transmitting, to the second apparatus, a third configuration for reporting the prediction result and the measurement result of the third set of the beams.

[0162] In some example embodiments, the first apparatus further comprises: means for determining whether a first range of values in the measurement result of the third set of beams matches with a second range of values in the prediction result of the fourth set of beams; and means for in accordance with a determination that the first range mismatches with the second range, determining that the test of the prediction model is failed.

[0163] In some example embodiments, the first apparatus further comprises: means for in accordance with a determination that the first range matches with the second range, determining whether a difference between the maximum value in the first range and the maximum value in the second range is less than a tolerance margin; means for in accordance with a determination that the difference is less than the tolerance margin, determining that the test of the prediction model is passed; and means for in accordance with a determination that the difference exceeds the tolerance margin, determining that the test of the prediction model is failed.

[0164] In some example embodiments, the first apparatus further comprises: means for in accordance with a determination that the first range matches with the second range, determining whether a difference between the maximum value in the first range and the maximum value in the second range is less than a tolerance margin; means for in accordance with a determination that the difference is less than the tolerance margin,determining whether a further difference between at least one value in the first range and at least one corresponding value in the second range is less than a further tolerance margin; and means for in accordance with a determination that the further difference is less than the further tolerance margin, determining that the test of the prediction model is passed; and means for in accordance with a determination that the further difference exceeds the further tolerance margin, determining that the test of the prediction model is failed.

[0165] In some example embodiments, the second set of beams is a subset of the first set of beams, or wherein the second set of beams is different from the first set of beams.

[0166] In some example embodiments, the measurement result comprises strongest N measured reference signal received power (RSRP) values of the third set of beams, the prediction result comprises strongest N predicted RSRP values of the fourth set of beams, and N is a predetermined positive number.

[0167] In some example embodiments, the first apparatus comprises a test equipment or a network device, and the second apparatus comprises a terminal device.

[0168] In some example embodiments, the first apparatus further comprises means for performing other operations in some example embodiments of the method 500 or the first apparatus 110. In some example embodiments, the means comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the performance of the first apparatus.

[0169] In some example embodiments, a second apparatus capable of performing any of the method 600 (for example, the second apparatus 120 in FIG. 1) may comprise means for performing the respective operations of the method 600. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module. The second apparatus may be implemented as or included in the second apparatus 120 in FIG. 1.

[0170] In some example embodiments, the second apparatus comprises means for receiving, from a first apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams; and means for transmitting, to the first apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams inwhich each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold.

[0171] In some example embodiments, the second apparatus further comprises: means for receiving, from the first apparatus, a message requesting the second apparatus to enter a beam prediction mode which enables the prediction with the model; and means for transmitting, to the first apparatus, an indication that the second apparatus is operating in the beam prediction mode.

[0172] In some example embodiments, the second apparatus further comprises: means for receiving, from the first apparatus, information about one or more resources of at least one of a synchronization signal block (SSB) or channel state information reference signal (CSI-RS).

[0173] In some example embodiments, the second apparatus further comprises: means for receiving, from the first apparatus, an indication indicating whether the measurements are to be performed on the SSB or the CSI-RS.

[0174] In some example embodiments, the second apparatus further comprises: means for receiving, from the first apparatus, a request to prepare signal strengths of the third set of beams.

[0175] In some example embodiments, the second apparatus further comprises: means for receiving, from the first apparatus, a third configuration for reporting the prediction result and the measurement result of the third set of the beams.

[0176] In some example embodiments, the second set of beams is a subset of the first set of beams, or wherein the second set of beams is different from the first set of beams.

[0177] In some example embodiments, the measurement result comprises strongest N measured reference signal received power (RSRP) values of the third set of beams, the prediction result comprises strongest N predicted RSRP values of the fourth set of beams, and N is a predetermined positive number.

[0178] In some example embodiments, the first apparatus comprises a test equipment or a network device, and the second apparatus comprises a terminal device.

[0179] In some example embodiments, the second apparatus further comprises means for performing other operations in some example embodiments of the method 600 or the second apparatus 120. In some example embodiments, the means comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the performance of the second apparatus.

[0180] FIG. 7 is a simplified block diagram of a device 700 that is suitable for implementing example embodiments of the present disclosure. The device 700 may be provided to implement a communication device, for example, the first apparatus 110 or the second apparatus 120 as shown in FIG. 1. As shown, the device 700 includes one or more processors 710, one or more memories 720 coupled to the processor 710, and one or more communication modules 740 coupled to the processor 710.

[0181] The communication module 740 is for bidirectional communications. The communication module 740 has one or more communication interfaces to facilitate communication with one or more other modules or devices. The communication interfaces may represent any interface that is necessary for communication with other network elements. In some example embodiments, the communication module 740 may include at least one antenna.

[0182] The processor 710 may be of any type suitable to the local technical network and may include one or more of the following: general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on multicore processor architecture, as non-limiting examples. The device 700 may have multiple processors, such as an application specific integrated circuit chip that is slaved in time to a clock which synchronizes the main processor.

[0183] The memory 720 may include one or more non-volatile memories and one or more volatile memories. Examples of the non-volatile memories include, but are not limited to, a Read Only Memory (ROM) 724, an electrically programmable read only memory (EPROM), a flash memory, a hard disk, a compact disc (CD), a digital video disk (DVD), an optical disk, a laser disk, and other magnetic storage and / or optical storage. Examples of the volatile memories include, but are not limited to, a random access memory (RAM) 722 and other volatile memories that will not last in the power-down duration.

[0184] A computer program 730 includes computer executable instructions that are executed by the associated processor 710. The instructions of the program 730 may include instructions for performing operations / acts of some example embodiments of the present disclosure. The program 730 may be stored in the memory, e.g., the ROM 724. The processor 710 may perform any suitable actions and processing by loading the program 730 into the RAM 722.

[0185] The example embodiments of the present disclosure may be implemented by means of the program 730 so that the device 700 may perform any process of thedisclosure as discussed with reference to FIG. 2 to FIG. 6. The example embodiments of the present disclosure may also be implemented by hardware or by a combination of software and hardware.

[0186] In some example embodiments, the program 730 may be tangibly contained in a computer readable medium which may be included in the device 700 (such as in the memory 720) or other storage devices that are accessible by the device 700. The device 700 may load the program 730 from the computer readable medium to the RAM 722 for execution. In some example embodiments, the computer readable medium may include any types of non-transitory storage medium, such as ROM, EPROM, a flash memory, a hard disk, CD, DVD, and the like. The term “non-transitory,” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM).

[0187] FIG. 8 shows an example of the computer readable medium 800 which may be in form of CD, DVD or other optical storage disk. The computer readable medium 800 has the program 730 stored thereon.

[0188] Generally, various embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects may be implemented in hardware, and other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device. Although various aspects of embodiments of the present disclosure are illustrated and described as block diagrams, flowcharts, or using some other pictorial representations, it is to be understood that the block, apparatus, system, technique or method described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0189] Some example embodiments of the present disclosure also provide at least one computer program product tangibly stored on a computer readable medium, such as a non- transitory computer readable medium. The computer program product includes computerexecutable instructions, such as those included in program modules, being executed in a device on a target physical or virtual processor, to carry out any of the methods as described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, or the like that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments.Machine-executable instructions for program modules may be executed within a local or distributed device. In a distributed device, program modules may be located in both local and remote storage media.

[0190] Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0191] In the context of the present disclosure, the computer program code or related data may be carried by any suitable carrier to enable the device, apparatus or processor to perform various processes and operations as described above. Examples of the carrier include a signal, computer readable medium, and the like.

[0192] The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable medium may include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0193] Further, although operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, although several specific implementation details are contained in the above discussions, these should not be construed as limitations on the scope of the present disclosure, but rather as descriptions of features that may be specific to particular embodiments. Unless explicitly stated, certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment.Conversely, unless explicitly stated, various features that are described in the context of a single embodiment may also be implemented in a plurality of embodiments separately or in any suitable sub-combination.

[0194] Although the present disclosure has been described in languages specific to structural features and / or methodological acts, it is to be understood that the present disclosure defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

WHAT IS CLAIMED IS:

1. A first apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the first apparatus at least to: transmit, to a second apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams; receive, from the second apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold; and determine a result of a test of the prediction model based on the measurement result and the prediction result.

2. The first apparatus of claim 1, wherein the first apparatus is caused to: transmit, to the second apparatus, a message requesting the second apparatus to enter a beam prediction mode which enables the prediction with the model; and receive, from the second apparatus, an indication that the second apparatus is operating in the beam prediction mode.

3. The first apparatus of claim 1, wherein the first apparatus is caused to: transmit, to the second apparatus, information about one or more resources of at least one of a synchronization signal block (SSB) or channel state information reference signal (CSI-RS).

4. The first apparatus of claim 3, wherein the first apparatus is caused to: transmit, to the second apparatus, an indication indicating whether the measurements are to be performed on the SSB or the CSI-RS.

5. The first apparatus of claim 1, wherein the first apparatus is caused to: transmit, to the second apparatus, a request to prepare signal strengths of the third set of beams.

6. The first apparatus of claim 1, wherein the first apparatus is caused to: transmit, to the second apparatus, a third configuration for reporting the prediction result and the measurement result of the third set of the beams.

7. The first apparatus of claim 1, wherein the first apparatus is caused to: determine whether a first range of values in the measurement result of the third set of beams matches with a second range of values in the prediction result of the fourth set of beams; and in accordance with a determination that the first range mismatches with the second range, determine that the test of the prediction model is failed.

8. The first apparatus of claim 7, wherein the first apparatus is caused to: in accordance with a determination that the first range matches with the second range, determine whether a difference between the maximum value in the first range and the maximum value in the second range is less than a tolerance margin; in accordance with a determination that the difference is less than the tolerance margin, determine that the test of the prediction model is passed; and in accordance with a determination that the difference exceeds the tolerance margin, determine that the test of the prediction model is failed.

9. The first apparatus of claim 8, wherein the first apparatus is caused to: in accordance with a determination that the first range matches with the second range, determine whether a difference between the maximum value in the first range and the maximum value in the second range is less than a tolerance margin; in accordance with a determination that the difference is less than the tolerance margin, determine whether a further difference between at least one value in the first rangeand at least one corresponding value in the second range is less than a further tolerance margin; and in accordance with a determination that the further difference is less than the further tolerance margin, determine that the test of the prediction model is passed; and in accordance with a determination that the further difference exceeds the further tolerance margin, determine that the test of the prediction model is failed.

10. The first apparatus of claim 1, wherein the second set of beams is a subset of the first set of beams, or wherein the second set of beams is different from the first set of beams.

11. The first apparatus of claim 1, wherein the measurement result comprises strongest N measured reference signal received power (RSRP) values of the third set of beams, the prediction result comprises strongest N predicted RSRP values of the fourth set of beams, andN is a predetermined positive number.

12. The first apparatus of any of claims 1 to 11, wherein the first apparatus comprises a test equipment or a network device, and the second apparatus comprises a terminal device.

13. A second apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the second apparatus at least to: receive, from a first apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams; and transmit, to the first apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on theprediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold.

14. The second apparatus of claim 13, wherein the second apparatus is caused to: receive, from the first apparatus, a message requesting the second apparatus to enter a beam prediction mode which enables the prediction with the model; and transmit, to the first apparatus, an indication that the second apparatus is operating in the beam prediction mode.

15. The second apparatus of claim 13, wherein the second apparatus is caused to: receive, from the first apparatus, information about one or more resources of at least one of a synchronization signal block (SSB) or channel state information reference signal (CSI-RS).

16. The second apparatus of claim 15, wherein the second apparatus is caused to: receive, from the first apparatus, an indication indicating whether the measurements are to be performed on the SSB or the CSI-RS.

17. The second apparatus of claim 13, wherein the second apparatus is caused to: receive, from the first apparatus, a request to prepare signal strengths of the third set of beams.

18. The second apparatus of claim 13, wherein the second apparatus is caused to: receive, from the first apparatus, a third configuration for reporting the prediction result and the measurement result of the third set of the beams.

19. The second apparatus of claim 13, wherein the second set of beams is a subset of the first set of beams, or wherein the second set of beams is different from the first set of beams.

20. The second apparatus of claim 13, wherein the measurement result comprises strongest N measured reference signal received power (RSRP) values of the third set ofbeams, the prediction result comprises strongest N predicted RSRP values of the fourth set of beams, and N is a predetermined positive number.

21. The second apparatus of any of claims 13 to 20, wherein the first apparatus comprises a test equipment or a network device, and the second apparatus comprises a terminal device.

22. A method comprising: transmitting, at a first apparatus and to a second apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams; receiving, from the second apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold; and determining a result of a test of the prediction model based on the measurement result and the prediction result.

23. A method comprising: receiving, at a second apparatus and from a first apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams; and transmitting, to the first apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold,and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold.

24. A first apparatus comprising: means for transmitting, to a second apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams; means for receiving, from the second apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold; and means for determining a result of a test of the prediction model based on the measurement result and the prediction result.

25. A second apparatus comprising: means for receiving, from a first apparatus, a first configuration for measurements associated with a first set of beams and a second configuration for prediction with a prediction model associated with a second set of beams; and means for transmitting, to the first apparatus, a measurement result indicating measured signal strengths of a third set of beams and a prediction result indicating predicted signal strengths of a fourth set of beams, wherein the measurement result is determined from measurements on the first set of beams based on the first configuration, the prediction result is obtained based on the prediction model and the second set of beams, the third set of beams is a subset of the first set of beams in which each beam has a signal strength exceeding a first strength threshold, and the fourth set of beams is a subset of the second set of beams in which each beam has a signal strength exceeding a second strength threshold.

26. A computer readable medium comprising instructions stored thereon for causing an apparatus at least to perform a method of claim 22 or a method of claim 23.

Images