Ai / ML beam prediction in wireless networks based on a composite beam ranking score
The composite beam ranking score addresses the limitations of single-beam evaluation in AI/ML-based beam prediction by assessing the accuracy of the top-K beam set, ensuring accurate beam selection and improved network performance in wireless communication networks.
Patent Information
- Application Number
- PCT/US2026/027389
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-09
- Filing Date
- 2026-05-08
- Publication Date
- 2026-11-12
AI Technical Summary
Existing AI/ML-based beam prediction methods in wireless communication networks often fail to accurately assess the performance of beam selection, leading to potential mischaracterization and degraded performance due to reliance on single-beam evaluation, neglecting the accuracy of the entire beam set.
A composite beam ranking score is introduced to evaluate the performance of AI/ML models by considering the accuracy of the top-K beam set, using a weighted score that compares predicted and measured beam rankings, ensuring the model's performance is accurately assessed by considering multiple beams.
The composite beam ranking score provides a more reliable indicator of AI/ML model performance, ensuring accurate beam selection and improved network performance by validating the prediction accuracy of the top-K beam set, thereby enhancing data transmission and reception efficiency.
Smart Images

Figure US2026027389_12112026_PF_FP_ABST
Abstract
Description
Attorney Docket No. 56990-0102WO1 / P71934WO1AI / ML BEAM PREDICTION IN WIRELESS NETWORKS BASED ON A COMPOSITE BEAM RANKING SCORECLAIM OF PRIORITY
[0001] This application claims priority to Greek Patent Application Serial No. 20250100349, filed on May 9, 2025, the entire contents of which are hereby incorporated by reference.TECHNICAL FIELD
[0002] This application relates to wireless communication networks.BACKGROUND
[0003] Wireless communication networks provide integrated communication platforms and telecommunication services to wireless user devices. Example telecommunication services include telephony, data (e.g., voice, audio, and / or video data), messaging, and / or other services. The wireless communication networks have wireless access nodes that exchange wireless signals with the wireless user devices using one or more wireless network protocols, such as protocols described in various telecommunication standards promulgated by the ETSI Third Generation Partnership Project (3GPP). The wireless communication networks facilitate mobile broadband service using technologies such as orthogonal frequency-division multiple access (OFDMA), multiple input multiple output (MIMO), advanced channel coding, massive MIMO, beamforming, and / or other features.Attorney Docket No. 56990-0102WO1 / P71934WO1SUMMARY
[0004] In an aspect, a method includes obtaining, for beam management of available beams for a wireless communication, selection information indicating a set of predicted beam values for selected beams by a machine learning model, the set of predicted beam values associated with the selected beams; obtaining a set of beam measurements corresponding to the selected beams; determining, for each of the selected beams, a difference between the predicted beam value of a selected beam and a beam measurement of the selected beam; determining, for each of the selected beams, whether the difference satisfies a threshold value; calculating, based on the determining, a value of a performance metric representing a performance of the beam prediction by the machine learning model, the value of the performance metric including a contribution from each selected beam that satisfies the threshold; and causing transmission of a report indicating a success or failure of the beam prediction by the machine learning model based on the value of the performance metric.
[0005] In some implementations, including one or more of the other implementations described herein, the method includes comparing the value of the performance metric to a threshold value, wherein the report indicates the success or the failure of the beam prediction by the machine learning model based on the comparing.
[0006] In some implementations, including one or more of the other implementations described herein, the success or the failure of the beam prediction by the machine learning model represents a rate in which the value of the performance metric satisfied the threshold value.
[0007] In some implementations, including one or more of the other implementations described herein, the threshold value is based on a configuration of a user equipment or a channel of the wireless communication.
[0008] In some implementations, including one or more of the other implementations described herein, the threshold value is based on a ratio of a size of set A beams to a size of set B beams, an order of the set A beams and the set B beams, an identifier of a user equipment, a classifier or regressor model type of the machine learning model, or a type of a case associated with the wireless communication.
[0009] In some implementations, including one or more of the other implementations described herein, calculating the value of a performance metric comprises weighting aAttorney Docket No. 56990-0102WO1 / P71934WO1contribution to the performance metric of each selected beam that satisfies the threshold based on a channel realization value.
[0010] In some implementations, including one or more of the other implementations described herein, wherein the channel realization value comprises: C( ) = K - wkl(RSRP' (n) > RSRPbk( ) — X), where wkis a normalization weighting function, k=1and l(x) returns 1 if condition is satisfied, X is a specified margin in dB, RSRPbkis a genie predicted beam index, and RSRPbgkis a machine learning model predicted beam index.
[0011] In some implementations, including one or more of the other implementations N Z C(n)described herein, the performance metric value comprises: £)(n)=where N is anumber of channel realization values for the selected beams.
[0012] In some implementations, including one or more of the other implementations described herein, the set of predicted beam values associated with the selected beams, the set of beam measurements corresponding to the selected beams, or both the set of predicted beam values associated with the selected beams and the set of beam measurements corresponding to the selected beam are ordered.
[0013] In some implementations, including one or more of the other implementations described herein, the predicted beam value is a reference signal receive power (RSRP) of a channel state information reference signal (CSI-RS).
[0014] In an aspect, obtaining, for beam management of available beams for a wireless communication, selection information indicating a set of predicted beam values for selected beams by a machine learning model, the set of predicted beam values associated with the selected beams; obtaining a set of beam measurements corresponding to the selected beams; determining a first score for the beam measurements, the first score based on a ranking of the beam measurements; determining a second score for the predicted beam values, the second score based on the ranking of the predicted beam values; calculating, based on the first score and the second score, a value of a performance metric representing a performance of the beam prediction by the machine learning model; and causing transmission of a report indicating a success or failure of the beam prediction by the machine learning model based on the value of the performance metric.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0015] In some implementations, including one or more of the other implementations described herein, the first score comprises a relevance score for each of the selected beams based on a signal strength of each of the selected beams in a genie ranked list representing a beam ranking from highest to lowest signal strength.
[0016] In some implementations, including one or more of the other implementations described herein, the second score comprises a relevance score for each of the selected beams based on a signal strength of each of the selected beams in a predicted beam ranked list representing an order of beams predicted by the machine learning model.
[0017] In some implementations, including one or more of the other implementations described herein, the first score is based on a discounting factor that discounts errors of ranking higher-ranked beams of the ranking of the beam measurements more severely than errors of ranking lower-ranked beams of the ranking of the beam measurements.
[0018] In some implementations, including one or more of the other implementations described herein, the second score is based on a discounting factor that discounts errors of ranking higher-ranked beams of the ranking of the predicted beam values more severely than errors of ranking lower-ranked beams of the ranking of the predicted beam values.
[0019] In some implementations, including one or more of the other implementations described herein, the value of a performance metric comprises a ratio of the second score to the first score.
[0020] In some implementations, including one or more of the other implementations described herein, the value of the first score comprises BRS9enie=, whereinwherein RSRP^nieincludes an RSRPkof a given genie (measured) beam bkin the set of K beams, MinkRSRP^nieincludes the lowest RSRP of a genie (measured) beam bkin the set of K beams, and MaxkRSRP^nieincludes a highest RSRP of a genie (measured) beam bkin the set of K beams.
[0021] In some implementations, including one or more of the other implementations described herein, the value of the second score comprises BRSpredlcted=Attorney Docket No. 56990-0102WO1 / P71934WO1fif, wherein and wherepvedictedjnc|u(jes anRSRPofagiven predicted beam Bkin the set of K beams,includes the lowest RSRP of a predicted beam Bkin the set of K beams, and MaxkMaxkRSRPpredictedincludes a highest RSRP of a predicted beam Bkin the set of K beams.
[0022] In some implementations, including one or more of the other implementations described herein, the method includes comparing the value of the performance metric to a threshold value, wherein the report indicates the success or the failure of the beam prediction by the machine learning model based on the comparing.
[0023] In some implementations, including one or more of the other implementations described herein, the success or the failure of the beam prediction by the machine learning model represents a rate in which the value of the performance metric satisfied the threshold value.
[0024] In some implementations, including one or more of the other implementations described herein, the threshold value is based on a configuration of a user equipment or a channel of the wireless communication.
[0025] In some implementations, including one or more of the other implementations described herein, the threshold value is based on a ratio of a size of set A beams to a size of set B beams, an order of the set A beams and the set B beams, an identifier of a user equipment, a classifier or regressor model type of the machine learning model, or a type of a case associated with the wireless communication.
[0026] In an aspect, one or more processors are configured to, when executing instructions stored in memory, perform any of the foregoing methods.
[0027] In an aspect, a non-transitory computer storage medium is encoded with instructions that, when executed by one or more processors, cause the one or more processors to perform any of the foregoing methods.
[0028] In an aspect, a system comprises one or more processors and one or more storage devices on which are stored instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform any of the foregoing methods.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0029] In an aspect, an apparatus comprising one or more baseband processors is configured to perform any of the foregoing methods.
[0030] In an aspect, a base station comprises one or more processors configured to perform any of the foregoing methods.
[0031] The details of one or more embodiments of these systems and processes are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of these systems and processes will be apparent from the description and drawings, and from the claims.Attorney Docket No. 56990-0102WO1 / P71934WO1 BRIEF DESCRIPTION OF THE FIGURES
[0032] FIG. 1 illustrates an example wireless network.
[0033] FIG. 2 is a diagram that illustrates an example of spatial-domain downlink beam prediction.
[0034] FIG. 3 A illustrates an example process of artificial intelligence and machine learning (AI / ML)-based downlink beam management based on beam prediction at a base station.
[0035] FIG. 3B illustrates an example process of AI / ML-based downlink beam management based on beam prediction at a user equipment (UE).
[0036] FIG. 4A illustrates an example system for performance monitoring using a composite beam score.
[0037] FIG. 4B illustrates an example system for performance monitoring using a composite beam score.
[0038] FIG. 5 A illustrates an example process for performance monitoring using a composite beam score.
[0039] FIG. 5B illustrates an example process for performance monitoring using a composite beam score.
[0040] FIG. 6 illustrates an example user equipment (UE).
[0041] FIG. 7 illustrates an example access node.Attorney Docket No. 56990-0102WO1 / P71934WO1DETAILED DESCRIPTION
[0042] Wireless communication networks can support multi-beam operation and beamforming techniques to achieve a higher data rate. Beam management can facilitate alignment of a pair of transmit and receive beams. For example, during a downlink beam management procedure, a user equipment (UE) measures a reference signal (e.g., SSB, CSI-RS) associated with transmit beams from a base station and selects a suitable receive beam that is best aligned with the transmit beam. To enhance this beam management functionality, artificial intelligence and machine learning (AI / ML) models and techniques can be used for beam prediction, including predictions in the spatial-domain or the time-domain. Spatial-domain downlink beam prediction (e.g., referred to as BM-Case 1) uses measurements of a configured set of downlink beams (e.g., denoted as Set B) to predict the best beam within another set of downlink beams (e.g., denoted as Set A) at present. Time-domain downlink beam prediction (e.g., referred to as BM-Case 2) uses measurements of Set B to predict the best beam within Set A on future prediction instances.
[0043] The inference performance of the AI / ML model affects accuracy in of the beam selection during beam management. Performance monitoring can ensure that the AI / ML model output results in optimal beam management. For example, the AI / ML model can assist a UE in selection / scheduling of beams to allow an increased total data rate or other optimized performance. Performance monitoring may be performed at a network (e.g., base station) and / or assisted by the UE. For example, for UE-assisted performance monitoring, the UE calculates and reports to the network a performance metric (e.g., beam prediction accuracy). Based on the reported performance metric, the network can determine how to use the AI / ML model output. The network or UE, for example, may determine that the UE should continue using the AI / ML model or to switch back to the legacy beam management operation.
[0044] In an aspect, for the spatial domain downlink beam prediction, UE-assisted performance monitoring may use a top 1 beam approach. In a top beam approach the UE compares a top measured beam, which can include a strongest beam by one or more metrics such as reference signal received power (RSRP), to a top beam predicted by the AI / ML. When the predicted and measured beams match, the network considers the AI / ML model to satisfy performance monitoring requirements. When the predicted and measured beams do not match, the network considers the AI / ML model to fail the performance monitoring requirements, potentially requiring dropping the predictions of the model, updating the model, or reverting toAttorney Docket No. 56990-0102WO1 / P71934WO1legacy operation for beam management. A metric that represents an evaluation of the AI / ML mode is also called a key performance indicator (KPI). In an example, a KPI indicates a successful rate of the AI / ML model generating a correct prediction for the set of beams in the spatial dimension. A correct prediction occur when a maximum RSRP among the top K predicted beams is larger than the RSRP of a strongest beam overall within a threshold (e.g., -xdB). In other words, the AI / ML model predicts a set of top K beams, and as long as one of the predicted beams has an RSRP that is greater than a threshold value (e.g., -xdB) from the strongest beam in the set, the prediction is considered to be successful by performance monitoring operations. In some implementations, the threshold value can be beam dependent, beam independent, or based on UE capability. In some implementations, the threshold is static (e.g., -6dB). This threshold value is also called a margin of the beam. For example, a weaker beam may have a wider margin because the weaker beam is more contaminated with measurements.
[0045] Performance monitoring using the top beam only may result in a mischaracterization of the performance of the AI / ML model because other beams of the beam set are not considered. For example, when the performance monitoring considers the beam with a maximum LI -RSRP among the top-K predicted beams or alternatively the top-1 predicted beam in the evaluation of the AI / ML model, the performance monitoring ignores evaluation of other beams in the set which may be used for transmission or reception of data if the top-1 predicted beam is not available. To evaluate the AI / ML model performance, performance monitoring may consider a correct prediction as the event of predicting top-K beam set rather than just the set containing top-1 predicted beam. For example, the network may indicate to the UE to use any of the top-K predicted beams in the field. In another example, even though it is logical for the network to use the highest RSRP beam among the top-K predicted beams for transmission to the UE, loadbalancing and scheduling requirements may constrain the network to choose one of the other top-K predicted beams. If a prediction accuracies of other beams (beside the top-1 beam) are not verified, performance of the beam prediction and selection in beam management may be degraded for transmission or reception of data.
[0046] The beam prediction model accuracy can be verified by identifying the best K-cardinality beam set rather than any K-cardinality set containing the strongest beam. In this example, if the prediction accuracy of other top-K beams is not verified, there can be a degraded in-field performance, even though a UE passes a success rate requirement test for a top-1 beam.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0047] To address these potential drawbacks, a more useful and accurate indicator of AI / ML model prediction performance in real field deployment can evaluate the successful rate of correct prediction by evaluation of a top-K beam set. For the spatial domain downlink beam prediction, UE-assisted performance monitoring may use more than just a single beam measurement for beam prediction. This disclosure describes a composite beam ranking score to consider the top-K beam set in performance monitoring, as described herein.
[0048] The UE calculates performance metrics for the AI / ML model by performing beam measurements for a set of beams and determining the prediction accuracy. The beam prediction accuracy can include a top K beam prediction accuracy, with or without a signal strength margin, by comparing prediction results of the AI / ML model to the top K beams based on the measurements of the beams. When the predicted beams together represent the strongest measured beams, the performance monitoring indicates that the AI / ML model is successfully performing beam prediction. When the predicted beams together represent weaker measured beams, the performance monitoring indicates that the AI / ML model is may not successfully be performing beam prediction, and the model should be updated or not used for beam prediction. For the beam sets of K beams, the comparison of the performance monitoring can include weighting beams of the set and comparing the predicted ranking by the AI / ML model to the measured ranking of the beams. A weighted score is generated for each ranked beam. When the score exceeds a threshold, the network considers the prediction by the AL / ML model to be sufficiently accurate to continue using the AI / ML model for beam predictions. When the score considers multiple beams of the set, the score is a called a composite beam ranking score.
[0049] This disclosure describes systems and processes for generating the composite beam ranking score for performance monitoring of AI / ML models that assist beam management. In some implementations, a UE can execute a process to generate the composite beam ranking score for BM-Case 1. The composite beam ranking score can include a weighted value for each beam (e.g., of a top-K beam set) that scores how the AI / ML model predicted the ranking of that beam. The values associated with each ranked beam can be combined into a final, normalized result that is compared to a threshold value. When the score satisfies (e.g., exceeds) the threshold, the AI / ML model is considered to be successfully predicting the top-K beams of the set.
[0050] FIG. 1 illustrates a wireless network 100. The wireless network 100 includes a UE 102 and a base station 104 connected via one or more channels 106 A, 106B across an air interfaceAttorney Docket No. 56990-0102WO1 / P71934WO1108. The UE 102 and base station 104 communicate using a system that supports controls for managing the access of the UE 102 to a network via the base station 104.
[0051] In some implementations, the wireless network 100 is a Standalone (SA) network, e.g., that incorporates Fifth Generation (5G) New Radio (NR). In some other implementations, the wireless network 100 is a Non- Standalone (NSA) network that incorporates Long Term Evolution (LTE) and 5G NR. In these implementations, the wireless network 100 may be a E-UTRA (Evolved Universal Terrestrial Radio Access)-NR Dual Connectivity (EN-DC) network, or an NR-EUTRA Dual Connectivity (NE-DC) network. Furthermore, wireless networks implementing one or more other types of communication standards are possible, including future 3GPP systems (e.g., Sixth Generation (6G)), Institute of Electrical and Electronics Engineers (IEEE) 802.11 technology, or the like. While aspects may be described herein using terminology commonly associated with 5GNR, aspects of the present disclosure can be applied to other systems, such as systems subsequent to 5G (e.g., 6G).
[0052] In the wireless network 100, the UE 102 and any other UE in the system may be, for example, any of a laptop computer, smartphone, tablet computer, machine-type device (such as smart meters or specialized devices for healthcare), intelligent transportation system, or any other wireless device. In network 100, the base station 104 provides the UE 102 network connectivity to a broader network (not shown). This UE 102 connectivity is provided via the air interface 108 in a base station service area provided by the base station 104. In some implementations, such a broader network may be a wide area network operated by a cellular network provider or may be the Internet. Each base station service area associated with the base station 104 is supported by one or more antennas integrated with the base station 104. The service areas can be divided into a number of sectors associated with one or more particular antennas. Such sectors may be physically associated with one or more fixed antennas or may be assigned to a physical area with one or more tunable antennas or antenna settings adjustable in a beamforming process used to direct a signal to a particular sector.
[0053] The UE 102 includes control circuitry 110 coupled with transmit circuitry 112 and receive circuitry 114. The transmit circuitry 112 and receive circuitry 114 may each be coupled with one or more antennas. The control circuitry 110 may include application-specific circuitry, baseband circuitry, or any of various combinations thereof. The transmit circuitry 112 and receive circuitry 114 may be adapted to transmit and receive data, respectively, and may include radio frequency (RF) circuitry and / or front-end module (FEM) circuitry.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0054] In various implementations, aspects of the transmit circuitry 112, receive circuitry 114, and / or control circuitry 110 may be integrated in various ways to implement the operations described herein. The control circuitry 110 may be adapted or configured to perform various operations, such as those described elsewhere in this disclosure related to a UE.
[0055] The transmit circuitry 112 may transmit using a plurality of multiplexed uplink physical channels. The plurality of uplink physical channels may be multiplexed, e.g., according to time division multiplexing (TDM) or frequency division multiplexing (FDM), and in some implementations, along with carrier aggregation. The transmit circuitry 112 may be configured to receive block data from the control circuitry 110 for transmission on the air interface 108.
[0056] The receive circuitry 114 may receive a plurality of multiplexed downlink physical channels from the air interface 108 and relay the physical channels to the control circuitry 110. The plurality of downlink physical channels may be multiplexed, e.g., according to TDM or FDM, e.g., along with carrier aggregation. The transmit circuitry 112 and the receive circuitry 114 may transmit and receive, respectively, both control data and content data (e.g., messages, images, video, etc.) structured within data blocks that are carried by the physical channels.
[0057] FIG. 1 also illustrates the base station 104. In some implementations, the base station 104 may be a 5G radio access network (RAN), a next generation RAN, a E-UTRAN, a nonterrestrial cell, or a legacy RAN, such as a UTRAN. As used herein, the term “5G RAN” or the like may refer to the base station 104 that operates in an NR wireless network 100, and the term “E-UTRAN” or the like may refer to a base station 104 that operates in an LTE wireless network 100. The UE 102 utilizes connections (or channels) 106A, 106B, each of which includes a physical communications interface or layer.
[0058] The base station 104 circuitry may include control circuitry 116 coupled (directly or indirectly) with transmit circuitry 118 and / or receive circuitry 120. The transmit circuitry 118 and receive circuitry 120 may each be coupled (directly or indirectly) with one or more antennas that may be used to enable communications via the air interface 108. The transmit circuitry 118 and receive circuitry 120 may be adapted to transmit and receive data, respectively, addressed to any UE connected to the base station 104. The receive circuitry 120 may receive a plurality of uplink physical channels from one or more UEs, including the UE 102.
[0059] In FIG. 1, the one or more channels 106A, 106B are illustrated as an air interface to enable communicative coupling, and can be consistent with cellular communications protocols,Attorney Docket No. 56990-0102WO1 / P71934WO1such as an LTE protocol, Advanced LTE (LTE-A) protocol, LTE-based access to unlicensed spectrum (LTE-U), NR protocol, NR-based access to unlicensed spectrum (NR-U) protocol, and / or any other communications protocol(s). In some implementations, the UE 102 may directly exchange communication data via a ProSe interface. The ProSe interface may alternatively be referred to as a sidelink (SL) interface and may include one or more logical channels, including but not limited to a Physical Sidelink Control Channel (PSCCH), a Physical Sidelink Discovery Channel (PSDCH), and a Physical Sidelink Broadcast Channel (PSBCH).
[0060] The UE 102 and the base station 104 are configured to perform operations assisted based on AI / ML models for wireless communication. For example, AI / ML models and AI / ML techniques may be used for channel state information (CSI) feedback, beam management, and / or positioning. The AI / ML techniques may also enable network automation or data collection for various network functions. Furthermore, the AI / ML techniques may support AI / ML-based network energy savings, load balancing, and mobility optimization.
[0061] The AI / ML model may be defined as a data-driven algorithm that applies AI / ML techniques to generate a set of outputs based on a set of inputs. The AI / ML model may be implemented at the UE side, the network side, or both. Generally, stages of AI / ML-related algorithms and associated complexity may be characterized by model generation and inference operation (e.g., input / output, pre- / post-process). The model generation may include model training (e.g., input / output, pre- / post-process, online / offline), model validation, and model testing. The life cycle management (LCM) of AI / Model and AI / ML functionality may be implemented for various aspects, which include, for example, data collection, model training, functionality / model identification, model delivery / transfer, model inference operation, functionality / model selection, activation, deactivation, switching and fallback operation, functionality / model monitoring, model update, and UE capability.
[0062] For AI / ML-based beam management, various AI / ML techniques, such as at least model training, model inference, and model monitoring, may be used. For AI / ML model training, the UE or the base station may generate training data. The AI / ML model training described herein is a process to train an AI / ML model in a data-driven manner and obtain the trained AI / ML model for inference of network conditions in the future. The AI / ML model inference described herein includes a process of using the trained AI / ML model to produce a set of outputs based on a set of inputs. For network-side AI / ML model inference, the UE may generate input data and transmit the generated input data to the base station. For UE-side AI / ML model inference,Attorney Docket No. 56990-0102WO1 / P71934WO1input data may be internally available at the UE. In order to evaluate the inference performance of the AI / ML model, AI / ML model monitoring (e.g., performance monitoring) may be performed.
[0063] Performance monitoring for an AI / ML model ensures the proper operation of such a model. Among various types of performance monitoring, Type 1 can be supported for both BM-Case 1 and BM-Case 2. For Type 1 performance monitoring, the UE may be configured or signaled from a network, such as the base station, for measurement and / or reporting. The UE may support two options of Type 1 performance monitoring: Option 1 is network-side performance monitoring, and Option 2 is UE-assisted performance monitoring. During Typel / Optionl network-side performance monitoring, the UE may transmit a monitoring result report to the network (e.g., for the calculation of a performance metric at the network). The monitoring result report may include measurement results from resource sets for performance monitoring (e.g., Ll-RSRP and / or RS index) and any other contents. During Typel / Option2 performance monitoring, the UE may calculate one or more performance metrics and report them or associated information to the network. Performance metrics described herein can be a beam prediction accuracy (e.g., Top-K / 1 beam prediction accuracy).
[0064] FIG. 2 illustrates an example diagram 200 of spatial-domain downlink beam prediction, according to some implementations. Spatial-domain downlink beam prediction may be performed at a UE (e.g., UE 102) or a base station (e.g., base station 104) based on measurement results of Set B of beams to predict the best beams within Set A of beams. The UE performs measurements of downlink beams 202 (e.g., Set B) on measurement resource set 204 (e.g., SSB, CSI-RS resource set) configured by the base station. Using the AI / ML model 206, the top beams 208, Beam-1 through Beam-8, may be inferred at present at the inference instance 210 based on the measurement result of downlink beams 202. The inference report 212 may include information, such as a beam index and an Ll-RSRP, of the predicted top beams 208 for the inference instance 210.
[0065] FIG. 3A illustrates an example process 300 of AI / ML-based downlink beam management based on beam prediction at a base station, according to some implementations. A UE (e.g., UE 102) and the base station (e.g., base station 104) perform a beam management process based on a beam prediction at the base station. The base station sweeps (305) through transmit beams in Set B to collect data. The UE performs (310) Ll-RSRP measurements of the transmit beams in Set B. The UE reports (315) the Ll-RSRP measurement of the transmitAttorney Docket No. 56990-0102WO1 / P71934WO1beams in Set B to the base station. The base station performs (320) beam prediction using an AI / ML model to predict top-N transmit beams in Set A. The base station sweeps (325) through the top-N transmit beams in Set A. The UE performs (330) Ll-RSRP measurements of the top-N transmit beams in Set A to identify the best beam. The UE reports (335) the identity (ID) of the best transmit beam in Set A to the base station. The base station transmits (340) with the best beam in Set A. The UE sweeps (345) through different receive beams to find the best receive beam that pairs with the selected transmit beam.
[0066] FIG. 3B illustrates an example process 350 of AI / ML-based downlink beam management based on beam prediction at a UE, according to some implementations. A UE (e.g., UE 102) and the base station (e.g., base station 104) perform a beam management process based on a beam prediction at the UE. The base station sweeps (355) through transmit beams in Set B to collect data. The UE performs (360) Ll-RSRP measurements of the transmit beams in Set B. The UE performs (365) beam prediction using an AI / ML model to predict top-N transmit beams in Set A. The UE reports (370) the top-N transmit beams in Set A. The base station sweeps (375) through the top-N transmit beams in Set A. The UE performs (380) Ll-RSRP measurements of the top-N transmit beams in Set A to identify the best beam. The UE reports (385) the ID of the best transmit beam in Set A to the base station. The base station transmits (390) with the best beam in Set A. The UE sweeps (395) through different receive beams to find the best receive beam that pairs with the selected transmit beam.
[0067] FIG. 4A illustrates an example system 400 for performance monitoring using a composite beam score. In FIG. 4, an AI / ML model 402 generates a prediction of beam indices for selection by the UE during beam management operation. A performance monitoring engine 404 (e.g., logic executed by one or more processors) compares the predicted beam indices to measured beam results (e.g., ground truth CSI-RS values) to generate a top-k beam ranking score KPIi(x) that is a modified top-1 beam ranking score as described herein. A thresholding engine 406 (e.g., logic executed by one or more processors) compares the KPI to a threshold range. When the KPI satisfies the threshold, the AI / ML model passes performance monitoring, and the UE can continue to use the AI / ML model. When the AI / ML fails performance monitoring (or a rate of failure exceeds a threshold rate), the UE changes to a different AI / ML model or falls back to legacy operation. The generation of the beam score KPIi(x) by the performance monitoring engine 404 is subsequently described in detail.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0068] FIG. 4B illustrates an example system 450 for performance monitoring using a composite beam score. In FIG. 4, an AI / ML model 452 generates a prediction of beam indices for selection by the UE during beam management operation. A performance monitoring engine 454 (e.g., logic executed by one or more processors) compares the predicted beam indices to measured beam results (e.g., ground truth CSI-RS values) to generate a composite beam ranking score as described herein. A thresholding engine 456 (e.g., logic executed by one or more processors) compares the composite beam score (KPI) to a threshold range. When the composite beam score (KPI) satisfies the threshold, the AI / ML model passes performance monitoring, and the UE can continue to use the AI / ML model. When the AI / ML fails performance monitoring (or a rate of failure exceeds a threshold rate), the UE changes to a different AI / ML model or falls back to legacy operation. The generation of the composite beam score (KPI) by the performance monitoring engine 454 is subsequently described in detail.
[0069] In an aspect, the beam ranking score includes a function of a top-K beam KPI. The performance metric is generated many times, and a statistical representation of the beam characteristics is obtained, and the statistical representation can be used to set the threshold. The composite beam ranking score (also called a composite KPI) integrates beam-specific relevance (e.g., by RSRP) and beam rank prediction accuracy, and normalizes the output relative to the RSRP level for comparison across different beam set instances over time.
[0070] The UE or network can use the KPI metric for several purposes. In a first example, the KPI is used for performance monitoring, as previously described, for activating or deactivating the AI / ML model. In a second example, the KPI is used for setting performance requirements during testing of the AI / ML model. The UE can be tested in a particular configuration for inference for AI / ML-based beam management. The KPI, in the testing example, is an established criterion for determining whether the model passes or fails under the tested UE configuration. The pass or fail threshold for the composite score is set based on the testing of the UE with the configuration or other known capability. Additionally, the threshold can be set depending on the particular AI / ML model being used.
[0071] An example of determining thresholds for the composite KPI is now described. For generating the composite beam ranking score, the performance monitoring process can consider the beams of the set by determining how the AI / ML model ranked the beams and how this relates to each of the measured beam strengths. Because different AI / ML models may have different sets of inference parameters, the success threshold can be changed based on theAttorney Docket No. 56990-0102WO1 / P71934WO1number of beams in set A and set B. In an example, a first AI / ML model may have a configuration in which the ratio of set B / set A sizes is = 8:16. A second AI / ML model may have a configuration in which the ratio of set B / set A sizes is = 8:32. Both the first and second AI / ML models generate a prediction in which the top-K beam set = 5. In this example, the second AI / ML model has a worse performance since the ratio of set B / set A is smaller for the second AI / ML model. The success threshold can be modified to accommodate this discrepancy. In other words, a smaller beam A set would result in a higher threshold, in this example. In another example, the threshold is based on a beam pattern, such as how set B beams are sampled in the spatial domain from available multiple entries of beam pairs in, for example, an azimuth direction. In this way, the threshold can be UE-capability dependent.
[0072] Three examples of AI / ML models with different inference parameters and their associated thresholds are now described. In a first example, the AI / ML model parameters include an associated identifier (ID), a set B pattern ID, a size of set A / set B, an order of set A / set B, a type of classifier / regressor model being used for the AI / ML model, and a monitoring metric (e.g., RSRP) for a top-K beams report. In a second example, the AI / ML parameters include an associated identifier (ID), a set B pattern ID, a size of set A / set B, an order of set A / set B, a type of classifier / regressor model being used for the AI / ML model, and a ratio of the values of case 1 measurement / a case 2 measurement. In a third example, the AI / ML parameters include an associated identifier (ID), a set B pattern ID, a size of set A / set B, an order of set A / set B, a ratio of the values of case 1 measurement / a case 2 measurement, including a periodicity, set B / set A size ratio, a number of set B measurements, and a number of set predictions. In addition, as another example, the third model has been trained with a channel that has larger delay / angular spread than the first two models. The first model can have a set B / set A size ratio of [8, 16] with a low delay / angular spread, the second model can have a set B / set A size ratio of [8, 32] with a low delay / angular spread and the third model can have a set B / set A size ratio of [8, 32] with a large delay / angular spread. Corresponding UE capability may indicate that a first UE uses the first model and that a second UE can use either the second or third models. The threshold ranges are model ID dependent, and therefore UE capability dependent. For example for UE1 (model A), a threshold range could be set as 90-95%. For example, for UE2 (model B), the threshold range is set at 85-90%. For example, for UE2 (model C), the threshold range is set at 80-85 %.
[0073] The generation of the beam score KPIi(x) by the performance monitoring engine 404 is now described in greater detail. The beam score KPIi(x) includes a composite beam rankingAttorney Docket No. 56990-0102WO1 / P71934WO1score for performance monitoring of AI / ML model, as previously described. The beam score KPIi(x) includes a modified top-1 beam score to extend to top-K beams. The best top-K genie beams are denoted as bk. A “genie beam” represents a beam that is measured under ideal conditions for an ideal RSRP measurement value, otherwise called an ideal beam. The top-K predicted beam is denoted as bk, k = 1, •••, K. A genie-aided sorted beam index is defined as bgk= sort(RSRPg). The measured genie RSRPs for genie predicted and AI / ML predicted indices are respectively denoted as RSRPbkand RSRPĝk. A channel realization value is denoted as n (e.g., the CDL channel realization). The corresponding RSRP value for each beam is: RSRPbk(n) and RSRPb̂k(n).
[0074] As previously described, each of the beams is weighted and compared to a threshold margin (e.g., -xdB) to determine if a beam should contribute a normalized output to the composite score. The function for the weighting and normalizing for a beam 1... K in the set of K -beams is C(n) = Σ wkퟏ(RSRPb̂gk(n) > RSRPbk(n) − X), where wkis a normalization k=1weighting function, and l(x) returns 1 if condition is satisfied, X is a specified margin in dB.NThe KPI metric is calculated as D(n) = Σ C(n) / N, where N is the number of CDL channel n=lrealizations.
[0075] An example implementation of the KPI using the modified top-1 beam score is now described. Example values for a set of K=10 beams is shown in table 1.Table 1: Example Values for Generation of the Modified Option 3 Beam Score KPIi(X)Attorney Docket No. 56990-0102WO1 / P71934WO1
[0076] As shown in Table 1, the performance monitoring engine 404 receives data representing the predicted beam indices and the ground truth “genie” beam indices and determines a performance metric score that represents how well the AI / ML model ranked the top-K predicted beams. In this example, the top K (5) genie beams bk include beams 1, 2, 3, 4, and 5, with RSRP values of -80 dB, -83 dB, -85 dB, -89 dB, and -95 dB, respectively. The AI / ML top K (5) predicted beams b̂kwere beams 2, 4, 5, 9, and 10. Given the values of the predicted indices, their RSRP values, and the top-5 beams, the performance monitoring engine 404 generates a score representing the AI / ML performance. In this example, 0 is a worst prediction, and l is a best prediction, but any scoring convention and corresponding threshold convention can be applied. In this example, the score is normalized to 0 - 1 to enable comparison across different UE configurations.
[0077] The performance monitoring logic first determines a selection order of the AI / ML model for each of the beams that have been selected. In the example in Table 1, column 4 shows an example AI / ML selection order. For example, beam 4 (e.g., the beam having an index of 4) is ranked 1 by the AI / ML model, beam 2 is ranked 2, beam 9 is ranked 3, beam 5 is ranked 4, and beam 10 is ranked 5. Column 5 shows the RSRP values of the selected beams. The selected beams are then ranked by their RSRP values, as shown in column 6.
[0078] Once the performance monitoring engine 404 ranks the beams selected by the AI / ML model, the performance monitoring engine 404 determines a difference between an RSRP of the bk ground truth top beam and its corresponding predicted beam bkof the same rank. The RSRP of the top bkbeam (rank 1, index 1) is compared to the RSRP of top ranked predicted beam bk(beam 2). In the first comparison, the first selected beam (beam 4) RSRP is -83 dB. The RSRP of the top beam (e.g., beam 1) in the set of beams is -80 dB. The difference in RSRP for beam 1 and beam 4 is -3 dB.
[0079] The RSRP difference is determined for each selected beam bkin the set of predicted beams. The RSRP value of -89 dB for predicted beam 4 (ranked 2 in the selected beams) is compared to genie beam bk index 2, which has an RSRP of -83 dB, for a difference value of -6 dB. The RSRP value of -95 dB for predicted beam 5 (ranked 3 in the selected beams) is compared to genie beam bk index 3, which has an RSRP of -85 dB, for a difference value of -10 dB. The RSRP value of -112 dB for predicted beam 9 (ranked 4 in the selected beams) is compared to genie beam bk index 4, which has an RSRP of -89 dB, for a difference value of -Attorney Docket No. 56990-0102WO1 / P71934WO123 dB. The RSRP value of -119 dB for predicted beam 10 (ranked 5 in the selected beams) is compared to genie beam bk index 5, which has an RSRP of -95 dB, for a difference value of - 24 dB.
[0080] In this example, the performance monitoring engine 404 does not apply a penalty when the AI / ML model selects beams in the wrong order. For example, the performance monitoring engine 404 re-ranks the AI / ML predicted / selected beams bkby their RSRP values. The AI / ML model selected relatively weaker beam 4 before it selected relatively stronger beam 2. However, in this example, the performance monitoring engine 404 treats this circumstance the same way as if beam 2 were selected first and beam 4 were selected second. As subsequently described, in some implementations, the order of the beams selected by the AI / ML model can affect beam weighting and the composite beam score representing AI / ML performance. As subsequently described, the beams can be logarithmically weighted such that selecting a strong beam late in the order of beams devalues that beam’s contribution in accordance with a logarithmic decay weighting.
[0081] Once the performance monitoring engine 404 determines the differences between the genie beam RSRP values and the predicted beam RSRP values, the differences can be compared to a margin threshold to determine whether the selected beam contributes to the composite score. In an example, if a difference between the RSRP of the ideal, genie beam and the RSRP of the predicted beam is greater than -x dB, the beam is given a weighting of zero. In the example of Table 1, the margin threshold is -6 dB, but other threshold values are possible. The first two predicted beams, including beam 2 and beam 4, satisfy the threshold in this example and contribute to the composite score. Beams 5, 9, and 10 do not satisfy the threshold and do not contribute to the composite score.
[0082] The performance monitoring engine 404 applies a weighting value to the predicted beams that satisfy the threshold. These weighting values normalize the composite score for different UE conditions. In this example, the composite score is normalized to be within a value of 0 to 1, with 1 being perfect prediction performance by the AI / ML model and 0 being a worst prediction performance by the AI / ML model. The performance monitoring engine weights each of the AI / ML selected beams based on the beam RSRP value and the maximum RSRP value for the top K beams. In this example, the weighting value is defined as Z- min(z) / (max(z)-min(z)), Z is the RSRP value. The Z value can be the predicted beam RSRP or the genie beam RSRP. This formula normalizes between [0,1], Other normalization techniques are alsoAttorney Docket No. 56990-0102WO1 / P71934WO1possible. In this example, beam 2 is weighted at 0.5334, and beam 4 is weighted at 0.2586. These scores are combined to form a final composite score of 0.7920. The final composite score can be compared against a threshold by the thresholding engine 406. In some implementations, the weight can be uniform for each predicted top - K beam (e.g., 1 / K). In some implementations, higher value RSRP beams are weighted more than lower value RSRP beams.
[0083] The thresholding engine 406 can include logic that compares the composite score to a threshold that is defined based on the conditions of the UE. The exact threshold can be adjusted to different values depending on the sizes of beam set A and beam set B, UE, network or channel conditions, and so forth as previously described. The threshold value specifies what is considered to be a successful prediction by the AI / ML model and what is considered to be a failed prediction by the AI / ML model. The thresholding engine 406 compares the score to the threshold and reports a success or failure. As previously described, the AI / ML model can be dropped or changed based on a success rate or a rate of failure of the model. In some implementations, the UE can drop or change the AI / ML model responsive to a single failure.
[0084] The composite score of 0.8 is different than the previous score of 1 that would have been reported for a top-1 beam by the performance monitoring. The value of 0.8 reflects that there is not perfect beam alignment between the precited beams by the AI / ML model and the ideal genie beams.
[0085] There is no requirement that any particular number of beams satisfy the threshold for the AI / ML model prediction to pass the performance monitoring. The number of beams of the set of K beams that satisfies the threshold can contribute to the composite score, while the other beams do not contribute to the composite score. In an aspect, a single beam that as a relatively high RSRP value and that is correctly predicted by the AI / ML model can be enough for the prediction to pass performance monitoring. In another aspect, the AI / ML model can correct predict several beams, but if the top beams are not predicted or if the predicted beams do not have high enough RSRP values, the prediction is considered a failure.
[0086] The generation of the beam score KPI by the performance monitoring engine 454 is now described in greater detail. The beam score KPI includes a composite beam ranking score for performance monitoring of AI / ML model, as previously described. The CBRS includes relevance scores calculated for each beam in the set of K beams for both the genie beams andAttorney Docket No. 56990-0102WO1 / P71934WO1the predicted beams. The scores are normalized by taking a ratio of the predicted beam relevance scores by the genie beam relevance scores, as described herein.
[0087] In a second example, a composite beam ranking score is described for sorted top-K K Relevancegeniebeams. A BRS value for a genie beams of the set K is denoted as BRSgenie= V — - -j£- —.k=llog2(k+l) A relevance value represents an importance or utility of a given beam, as subsequently described. A BRS value for a predicted beam of the set K is BRSpredlcted=where the relevance value represents the importance or utility of a givenbeam, as subsequently described. The composite beam ranking score (CBRS) is CBRS = BRS§enieBRSpredicted- The CBRS account for each of signal strength (RSRP), beam ranking order, and normalization of RSRP across different measurement / prediction instances. Relevance scores can a function of RSRP -based signal strength, as subsequently described. The CBRS includes a logarithmic discounting factor in which the score discounts errors in predictions of higher beam ranks more severely than lower beam ranks. Other weighting functions can be included in addition or in the alternative. The normalization compares predictions to an ideal ranking (IDCG), ensuring scores range between 0 and 1 for use over different beam measurement / prediction instances. For example, when the CBRS value is 1, there is a perfect alignment with an ideal beam ranking. When the CBRS value is 0.9, it represents a relatively high quality model prediction but one that is degraded by at least one suboptimal prediction.
[0088] The relevance of each predicted beam can be calculated as follows. Relevance scores quantify the utility or importance of each beam index (e.g. RSRP) in a ranked list. Relevance scores determine a “gain” in the beam ranking score calculation, which aggregates a usefulness of beams based on beam positions. This can also be called logarithmic discounting, as previously described, because errors in predictions of higher to lower ranked beams are logarithmically discounted. In other words, the if the AI / ML model makes an erroneous prediction for a stronger beam, such as missing the highest ranked beam or predicting the highest ranked beam as being a lower ranked beam, the error is discounted more than errors in predictions for weaker beams. Higher relevance scores at higher positions (e.g., a smaller value of lower i) contribute more to a cumulative gain due to the logarithmic discounting factor. A maximum possible KPI score is achieved when beams are ordered by descending relevance. Relevance scores define the ideal beam ranking for a set K of beams. As previously described, normalization ensures scores are comparable across different UE configurations or operationalAttorney Docket No. 56990-0102WO1 / P71934WO1scenarios, with relevance determining both the numerator (actual ranking) and denominator (ideal ranking).
[0089] The relevance scores are calculated as follows. The genie beam relevance score for each beam is calculated as shown in equation (1).whereincludes an RSRP of a given genie (measured) beam bkin the set of K beams, MinkRSRP^nieincludes the lowest RSRP of a genie (measured) beam bkin the set of K beams, and MaxkRSRP^nieincludes a highest RSRP of a genie (measured) beam bkin the set of K beams. Equation 1 indicates that relevance scores quantify the utility or importance of each beam index (e.g. RSRP) in a ranked list.
[0090] The AI / ML model predicted beam relevance score for each predicted beam is calculated as shown in equation (2).where sRP^edlctedincludes an RSRP of a given predicted beam bkin the set of K beams, udes the lowest RSRP of a predicted beam bkin the set of K beams,includes a highest RSRP of a predicted beam bkin the set of K beams. Equation 2 indicates that relevance scores quantify the utility or importance of each beam index (e.g. RSRP) in a ranked list.
[0091] The relevance scores determine the contribution of the beam in the composite beam ranking score calculation, which aggregates the usefulness of beams based on their positions (according to a logarithmic function, as previously described and shown in equation 3). For example, higher relevance scores at higher positions (lower i) contribute more to the cumulative gain due to the logarithmic discounting factor. The maximum possible score is achieved when beams are ordered by descending relevance. Relevance scores (RSRP) define an ideal ranking of beams by the AI / ML model.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0092] Equation 3 shows two aspects of the generation of the CBRS score. The first aspect is the logarithmic penalization of errors for the beams in which errors in higher ranked beam predictions are weighted more heavily than errors for lower ranked beams. In other works, the logarithmic discounting factor log2(k + 1) for a kth ranked beam causes correctly predicted beams at higher ranks to contribute more to the score than beams at lower ranks. The second aspect of Equation 3 is the normalizing of the relevance scores as a ratio of the predicted beam relevance scores to the genie beam relevance scores for each beam in the set. The sum of the relevance scores for the genie beams, once the logarithmic discounting is applied, is divided by the sum of the predicted beam relevance scores. The ratio is the composite beam ranking score.
[0093] A simplified example metric generation for unsorted beams is shown in Tables 2 and 3, with K= top 5 beams in the beam set, and i = a predicted rank position (1 to 5) for each beam. Table 2 below gives an example of scoring ranked beams to generate the beam score KPIi(x). In this example, there are K=5 beams. In this example, the beams have an optimal ranking and optimal genie beam indices. The result is an BRSgenie value of 1.831.Table 2: CBRS calculation for K=5 Sorted Beams (Genie Ranked Beams)Attorney Docket No. 56990-0102WO1 / P71934WO1|um | Becausethe value of BRSgenie= 1 0 + 0 473 +0.25 + 0.108 + 0.0 = 1.831.
[0095] The BRSpredicted is a KPI representing a performance metric obtained when the beams are unranked, and the genie indices are unknown, or the ranking is suboptimal. The normalized KPI metric value is obtained by taking a ratio of the BRSgenie representing scoring the ranked beams and the BRSpredicted scoring the unranked beams. The ratio BRSpredicted / BRSgenie represents a normalized CBRS metric, as shown in the subsequent equations.
[0096] The BRSgenie value can be normalized by the KPI value BRSpredicted for unsorted predicted beams. Table 3 below gives an example of CBRS calculation for K=5 unsorted beams.Table 3: CBRS calculation for K=5 unsorted beamsthe value of + 0 + 0.193 =1.6469.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0098] The CBRS value is the value of BRSpredlcted / BRS9enie, and this value is 1.6469 / 1.831 = 0.8994. This value is compared to a selected threshold (e.g.,.95 (fail), 0.90 (fail), 0.85 (pass), 0.80 (pass)), depending on the UE configuration.
[0099] Table 4 represents an example of generating a CBRS for a K=10 beam set with unranked beams. The beam data in Table 4 is the same as the beam data described previously in relation to Table 3, but instead of a modified option 3 top-K beam score, the performance monitoring engine 452 generates the CBRS as previously described. The CBRS value is 0.6, as subsequently described, which accounts for errors in the prediction order by the AI / ML model. This is in comparison to the modified option 3 KPI value of 0.8 previously described, which does not account for (e.g., forgives) the prediction order by the AI / ML model.Table 4: Example Values for Generation of Generation of a CBRS for a K=10 beam set with unranked beams.
[0100] Table 4 includes the same example beams described previously in relation to Table 3. The AI / ML model has selected the same beams (2, 4, 5, 9, and 10) described in relation to Table 3. The CBRS accounts for the order in which the AI / ML model ranks beams. Because, in this example, AI / ML model did not select the top beam first with the first selection (instead ranking the selected beam 4 above selected beam 2), the selection order is penalized.
[0101] The relevance scores are shown in column 5, calculated as previously described. The relevance scores are discounted by the logarithmic discounting function, as described previously. These discounted scores are shown in column 7. A normalization is performed and shown in columns 8, 9, and 10. The order of the beams is taken into account by the value inAttorney Docket No. 56990-0102WO1 / P71934WO1column 8, in which the relevance score is adjusted based on the order of the prediction. Here, beam 4 is weighted down by 0.2510 because it was improperly ordered as the first beam by the AI / ML model, though it should have been ranked as the second beam. If beam 4 were ranked first, the relevance score would not be adjusted / reduced. The final CBRS score is the value of the discounted predicted score 0.9095 divided by the discounted genie score 1.4918 for a final score of 0.61. The score takes into account all the ranked / selected beams of the k-beam set, and the score penalizes the AI / ML prediction when the first beam choice is not the strongest beam (or the AI / ML model otherwise ranks the predicted beams in the wrong order).
[0102] The first score takes all the beams of the set of K beams into account for performance management but forgives the ranking of the beams by the AI / ML model when the prediction order is incorrect. The second score (CBRS) takes all the beams of the set of K beams into account for performance management but penalizes the ranking of the beams by the AI / ML model when the prediction order is incorrect.
[0103] FIG. 5A illustrates a flowchart of an example method 500 for UE-assisted performance monitoring for time-domain downlink beam prediction, according to some implementations. For clarity of presentation, the example method 500 is described in the context of the preceding figures. For clarity of presentation, the example method 500 is described in the context of the preceding figures. For example, the method 500 can be performed by a UE (e.g., the UE 102), or any suitable system, environment, software, hardware, or combination thereof. In some implementations, operations of the method 500 can be run in parallel, in combination, in loops, or in any order. The example method 500 shown in FIG. 5 A can be modified or reconfigured to include additional, fewer, or different actions (not shown in FIG. 5 A), which can be performed in the order shown or in a different order.
[0104] The method 500 includes obtaining (502), for beam management of available beams for a wireless communication, selection information indicating a set of predicted beam values for selected beams by a machine learning model, the set of predicted beam values associated with the selected beams. The method 500 includes obtaining (504) a set of beam measurements corresponding to the selected beams. The method 500 includes (506) determining, for each of the selected beams, a difference between the predicted beam value of a selected beam and a beam measurement of the selected beam. The method 500 includes determining (508), for each of the selected beams, whether the difference satisfies a thresholdAttorney Docket No. 56990-0102WO1 / P71934WO1value. The method 500 includes calculating (510), based on the determining, a value of a performance metric representing a performance of the beam prediction by the machine learning model, the value of the performance metric including a contribution from each selected beam that satisfies the threshold. The method 500 includes causing (512) transmission of a report indicating a success or failure of the beam prediction by the machine learning model based on the value of the performance metric.
[0105] In some implementations, the method 500 includes comparing the value of the performance metric to a threshold value, wherein the report indicates the success or the failure of the beam prediction by the machine learning model based on the comparing.
[0106] In some implementations, the success or the failure of the beam prediction by the machine learning model represents a rate in which the value of the performance metric satisfied the threshold value. In some implementations, the threshold value is based on a configuration of a user equipment or a channel of the wireless communication. In some implementations, the threshold value is based on a ratio of a size of set A beams to a size of set B beams, an order of the set A beams and the set B beams, an identifier of a user equipment, a classifier or regressor model type of the machine learning model, or a type of a case associated with the wireless communication.
[0107] In some implementations, calculating the value of a performance metric comprises weighting a contribution to the performance metric of each selected beam that satisfies the threshold based on a channel realization value.
[0108] In some implementations, the channel realization value comprises: C(ri) =K - wkl(RSRP~ (n) > RSRPbk( ) — X), where wkis a normalization weighting function, fc=lbgkand l(x) returns 1 if condition is satisfied, X is a specified margin in dB, RSRPbkis a genie predicted beam index, and RSRPbgkis a machine learning model predicted beam index.N Z C(n)
[0109] In some implementations, the performance metric value comprises: D(n) = —where N is a number of channel realization values for the selected beams.
[0110] In some implementations, the set of predicted beam values associated with the selected beams, the set of beam measurements corresponding to the selected beams, or bothAttorney Docket No. 56990-0102WO1 / P71934WO1the set of predicted beam values associated with the selected beams and the set of beam measurements corresponding to the selected beam are ordered.[OHl] In some implementations, the predicted beam value is a reference signal receive power (RSRP) of a channel state information reference signal (CSI-RS).
[0112] FIG. 5B illustrates a flowchart of an example method 520 for UE-assisted performance monitoring for time-domain downlink beam prediction, according to some implementations. For clarity of presentation, the example method 520 is described in the context of the preceding figures. For clarity of presentation, the example method 520 is described in the context of the preceding figures. For example, the method 520 can be performed by a UE (e.g., the UE 102), or any suitable system, environment, software, hardware, or combination thereof. In some implementations, operations of the method 520 can be run in parallel, in combination, in loops, or in any order. The example method 520 shown in FIG. 5B can be modified or reconfigured to include additional, fewer, or different actions (not shown in FIG. 5B), which can be performed in the order shown or in a different order.
[0113] The method 520 includes obtaining (522), for beam management of available beams for a wireless communication, selection information indicating a set of predicted beam values for selected beams by a machine learning model, the set of predicted beam values associated with the selected beams. The method 520 includes obtaining (524) a set of beam measurements corresponding to the selected beams. The method 520 includes determining (526) a first score for the beam measurements, the first score based on a ranking of the beam measurements. The method 520 includes determining (528) a second score for the predicted beam values, the second score based on the ranking of the predicted beam values. The method 520 includes calculating (530), based on the first score and the second score, a value of a performance metric representing a performance of the beam prediction by the machine learning model. The method 520 includes causing (532) transmission of a report indicating a success or failure of the beam prediction by the machine learning model based on the value of the performance metric.
[0114] In some implementations, the first score comprises a relevance score for each of the selected beams based on a signal strength of each of the selected beams in a genie ranked list representing a beam ranking from highest to lowest signal strength.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0115] In some implementations, the second score comprises a relevance score for each of the selected beams based on a signal strength of each of the selected beams in a predicted beam ranked list representing an order of beams predicted by the machine learning model.
[0116] In some implementations, the first score is based on a discounting factor that discounts errors of ranking higher-ranked beams of the ranking of the beam measurements more severely than errors of ranking lower-ranked beams of the ranking of the beam measurements.
[0117] In some implementations, the second score is based on a discounting factor that discounts errors of ranking higher-ranked beams of the ranking of the predicted beam values more severely than errors of ranking lower-ranked beams of the ranking of the predicted beam values.
[0118] In some implementations, the value of a performance metric comprises a ratio of the second score to the first score.
[0119] In some implementations, wherein the value of the first score comprises BRS9enie= Relevance9eniewherein wherein fc=l log2(fc+l)7includes an RSRP of a given genie (measured) beam bkin the set of K beams, MinkRSRP^nieincludes the lowest RSRP of a genie (measured) beam bkin the set of K beams, and MaxkRSRP^nieincludes a highest RSRP of a genie (measured) beam bkin the set of K beams.
[0120] In some implementations, the value of the second score comprises BRSpredlcted=RSRPpredicted-MinkRSRPpredicted11kbk predictedwherein Re evance = - predicted - predicted,KMaxkRSRPp-MinkRSRPpbkbk and where BSRP^edlctedincludes an RSRP of a given predicted beam bkin the set of K beams, MinkRSRP^edlctedincludes the lowest RSRP of a predicted beam bkin the set of K beams, and MaxkRSRP^edlctedincludes a highest RSRP of a predicted beam bkin the set of K beams.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0121] In some implementations, the method 520 includes comparing the value of the performance metric to a threshold value, wherein the report indicates the success or the failure of the beam prediction by the machine learning model based on the comparing.
[0122] In some implementations, the success or the failure of the beam prediction by the machine learning model represents a rate in which the value of the performance metric satisfied the threshold value.
[0123] In some implementations, the threshold value is based on a configuration of a user equipment or a channel of the wireless communication.
[0124] In some implementations, the threshold value is based on a ratio of a size of set A beams to a size of set B beams, an order of the set A beams and the set B beams, an identifier of a user equipment, a classifier or regressor model type of the machine learning model, or a type of a case associated with the wireless communication.
[0125] FIG. 6 illustrates an example UE 600. The UE 600 may be similar to and substantially interchangeable with UE 102 of FIG. 1. The UE 600 may be any mobile or non -mobile computing device, such as, for example, a mobile phone, computer, tablet, industrial wireless sensors, video device (for example, cameras, video cameras, etc.), wearable devices (for example, a smart watch), relaxed-IoT devices, etc.
[0126] The UE 600 may include any / all of processor 602, RF interface circuitry 604, memory / storage 606, user interface 608, sensors 610, driver circuitry 612, power management integrated circuit (PMIC) 614, one or more antenna(s) 616, and battery 618. The components of the UE 600 may be implemented as integrated circuits (ICs), portions thereof, discrete electronic devices, or other modules, logic, hardware, software, firmware, or a combination thereof. The block diagram of FIG. 6 is intended to show a high-level view of some of the components of the UE 600. However, some of the components shown may be omitted, additional components may be present, and a different arrangement of the components shown may occur in other implementations.
[0127] The components of the UE 600 may be coupled with various other components over one or more interconnects 620, which may represent any type of interface, input / output, bus (local, system, or expansion), transmission line, trace, optical connection, etc., that allows various circuit components (on common or different chips or chipsets) to interact with one another.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0128] The processor 602 may include one or more processors. For example, the processor 602 may include processor circuitry such as, for example, baseband processor circuitry (BB) 622A, central processor unit circuitry (CPU) 622B, and graphics processor unit circuitry (GPU) 622C. The processor 602 may include any type of circuitry or processor circuitry that executes or otherwise operates computer-executable instructions, such as program code, software modules, or functional processes from memory / storage 606 to cause the UE 600 to perform operations as described herein.
[0129] The processor 602 is configured for performance monitoring by determining performance metric value that includes contributions from the top K beams of the set, rather than just the top 1 beam, as described herein. For example, the processor 602 obtains beam selection information that indicate a set of predicted beam values for selected beams by a machine learning model, such as the AI / ML models described previously. The set of predicted beam values are associated with the selected beams, such as described in relation to Table 1 and Table 4. The processor 602 may obtain a set of beam measurements corresponding to the selected beams.
[0130] The processor 602 may determine, for each of the selected beams, a difference between the predicted beam value of a selected beam and a beam measurement of the selected beam. The processor 602 determines the difference for each of the beams selected by the AI / ML model so that the performance metric value (e.g., a composite score) can account for each of the selected beams, as described herein.
[0131] The processor 602 may determine, for each of the selected beams, whether the difference satisfies a threshold value. For beams in which the difference that satisfies (e.g., is less than) the threshold value, the processor 602 may determine a weighted value associated with the beam. The processor 602 generates a value of a performance metric from the weighted values for the selected beams. The performance metric value represents a performance of the beam prediction by the machine learning model, the value of the performance metric including a contribution from each selected beam that satisfies the threshold; and causing transmission of a report indicating a success or failure of the beam prediction by the machine learning model based on the value of the performance metric.
[0132] In another aspect, the processor 602 may obtain selection information indicating a set of predicted beam values for selected beams by a machine learning model. The set of the predicted beam values are associated with the A / ML-selected beams. The processor 602 mayAttorney Docket No. 56990-0102WO1 / P71934WO1obtain a set of beam measurements corresponding to the selected beams. The processor 602 may determine a first score for the beam measurements that is based on the ranking of the genie beam measurements. For example, the first score can include a composite relevance score for each of the selected beams in which each beam is weighted based on its RSRP strength and the order in which the beam is selected or predicted. Lower ranked beams contribute less to the first score. The processor 602 may determine a second score for the beam measurements that is based on the ranking of the predicted beam measurements. For example, the second score can include a composite relevance score for each of the selected beams for predictions in which each predicted beam is weighted based on its RSRP strength and the order in which the beam is predicted. Lower ranked beams contribute less to the second score.
[0133] The processor 602 may determine, based on the first score and the second score, such as a ratio of the second score to the first score, a value of a performance metric representing a performance of the beam prediction by the machine learning model. The processor can cause transmission of a report indicating a success or failure of the beam prediction by the machine learning model based on the value of the performance metric.
[0134] In some implementations, the baseband processor circuitry 622A may access a communication protocol stack 624 in the memory / storage 606 to communicate over a 3 GPP compatible network. In general, the baseband processor circuitry 622A may access the communication protocol stack to perform user plane functions at a physical (PHY) layer, medium access control (MAC) layer, radio link control (RLC) layer, packet data convergence protocol (PDCP) layer, service data adaptation protocol (SDAP) layer, and PDU layer; and perform control plane functions at a PHY layer, MAC layer, RLC layer, PDCP layer, RRC layer, and a non-access stratum layer. In some implementations, the PHY layer operations may additionally / altematively be performed by the components of the RF interface circuitry 604. The baseband processor circuitry 622A may generate or process baseband signals or waveforms that carry information in 3 GPP-compatible networks. In some implementations, the waveforms for NR may be based cyclic prefix orthogonal frequency division multiplexing (OFDM) “CP-OFDM” in the uplink or downlink, and discrete Fourier transform spread OFDM “DFT-S-OFDM” in the uplink.
[0135] The memory / storage 606 may include one or more non -transitory, computer-readable media that includes instructions (for example, communication protocol stack 624) that may be executed by the processor 602 to cause the UE 600 to perform various operations describedAttorney Docket No. 56990-0102WO1 / P71934WO1herein. The memory / storage 606 include any type of volatile or non-volatile memory that may be distributed throughout the UE 600. In some implementations, some of the memory / storage 606 may be located on the processor 602 itself (for example, LI and L2 cache), while other memory / storage 606 is external to the processor 602 but accessible thereto via a memory interface. The memory / storage 606 may include any suitable volatile or non-volatile memory such as, but not limited to, dynamic random access memory (DRAM), static random access memory (SRAM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), Flash memory, solid-state memory, or any other type of memory device technology.
[0136] The RF interface circuitry 604 may include transceiver circuitry and radio frequency front module (RFEM) that allows the UE 600 to communicate with other devices over a radio access network. The RF interface circuitry 604 may include various elements arranged in transmit or receive paths. These elements may include, for example, switches, mixers, amplifiers, filters, synthesizer circuitry, control circuitry, etc.
[0137] In the receive path, the RFEM may receive a radiated signal from an air interface via antenna(s) 616 and proceed to filter and amplify (with a low-noise amplifier) the signal. The signal may be provided to a receiver of the transceiver that downconverts the RF signal into a baseband signal that is provided to the baseband processor.
[0138] In the transmit path, the transmitter of the transceiver up-converts the baseband signal received from the baseband processor and provides the RF signal to the RFEM. The RFEM may amplify the RF signal through a power amplifier prior to the signal being radiated across the air interface via the antenna(s) 616. In various implementations, the RF interface circuitry 604 may be configured to transmit / receive signals in a manner compatible with NR access technologies.
[0139] The antenna(s) 616 may include one or more antenna elements to convert electrical signals into radio waves to travel through the air and to convert received radio waves over the air into electrical signals. In some implementations, the antenna elements may be arranged into one or more antenna panels. The antenna(s) 616 may have antenna panels that are omnidirectional, directional, or a combination thereof, to enable beamforming and multiple input, multiple output communications. The antenna(s) 616 may include any / all of microstrip antennas, printed antennas fabricated on the surface of one or more printed circuit boards, patchAttorney Docket No. 56990-0102WO1 / P71934WO1antennas, phased array antennas, etc. The antenna(s) 616 may have one or more panels designed for one or more specific frequency bands, such as bands in FR1 or FR2.
[0140] The user interface 608 includes various input / output (VO) devices designed to enable user interaction with the UE 600. The user interface 608 includes input device circuitry and output device circuitry. Input device circuitry includes any physical or virtual means for accepting an input including, inter alia, one or more physical or virtual buttons (for example, a reset button), a physical keyboard, keypad, mouse, touchpad, touchscreen, microphones, scanner, headset, or the like. The output device circuitry includes any physical or virtual means for showing information or otherwise conveying information, such as sensor readings, actuator position(s), or other like information. Output device circuitry may include any number or combinations of audio or visual display, including, inter alia, one or more simple visual outputs / indicators (for example, binary status indicators such as light emitting diodes “LEDs” and multi -character visual outputs), or more complex outputs such as display devices or touchscreens (for example, liquid crystal displays “LCDs,” LED displays, quantum dot displays, projectors, etc.), with the output of characters, graphics, multimedia objects, and the like being generated or produced from the operation of the UE 600.
[0141] The sensors 610 may include devices, modules, or subsystems whose purpose is to detect events or changes in its environment and send the information (sensor data) about the detected events to some other device, module, subsystem, etc. Examples of such sensors include, inter alia, inertia measurement units including accelerometers, gyroscopes, or magnetometers; microelectromechanical systems or nanoelectromechanical systems including 3-axis accelerometers, 3-axis gyroscopes, or magnetometers; level sensors; temperature sensors (for example, thermistors); pressure sensors; image capture devices (for example, cameras or lensless apertures); light detection and ranging sensors; proximity sensors (for example, infrared radiation detector and the like); depth sensors; ambient light sensors; ultrasonic transceivers; microphones or other like audio capture devices; etc.
[0142] The driver circuitry 612 may include software and hardware elements that operate to control particular devices that are embedded in the UE 600, attached to the UE 600, or otherwise communicatively coupled with the UE 600. The driver circuitry 612 may include individual drivers allowing other components to interact with or control various input / output (I / O) devices that may be present within, or connected to, the UE 600. For example, driver circuitry 612 may include a display driver to control and allow access to a display device, aAttorney Docket No. 56990-0102WO1 / P71934WO1touchscreen driver to control and allow access to a touchscreen interface, sensor drivers to obtain sensor readings of sensors 610 and control and allow access to sensors 610, drivers to obtain actuator positions of electro-mechanic components or control and allow access to the electro-mechanic components, a camera driver to control and allow access to an embedded image capture device, audio drivers to control and allow access to one or more audio devices.
[0143] The PMIC 614 may manage power provided to various components of the UE 600. In particular, with respect to the processor 602, the PMIC 614 may control power-source selection, voltage scaling, battery charging, or DC-to-DC conversion.
[0144] In some implementations, the PMIC 614 may control, or otherwise be part of, various power saving mechanisms of the UE 600. A battery 618 may power the UE 600, although in some examples the UE 600 may be mounted deployed in a fixed location and may have a power supply coupled to an electrical grid. The battery 618 may be a lithium ion battery, a metal-air battery, such as a zinc-air battery, an aluminum-air battery, a lithium-air battery, and the like. In some implementations, such as in vehicle-based applications, the battery 618 may be a typical lead-acid automotive battery.
[0145] FIG. 7 illustrates an example access node 700 (e.g., a base station or gNB), according to some implementations. The access node 700 may be similar to and substantially interchangeable with base station 104. The access node 700 may include one or more of processor 702, RF interface circuitry 704, core network (CN) interface circuitry 706, memory / storage circuitry 708, and one or more antenna(s) 710. The processor 702 may include any type of circuitry or processor circuitry that executes or otherwise operates computerexecutable instructions, such as program code, software modules, or functional processes from memory / storage circuitry 708 to cause the access node 700 to perform operations as described herein.
[0146] The components of the access node 700 may be coupled with various other components over one or more interconnects 712. The processor 702, RF interface circuitry 704, memory / storage circuitry 708 (including communication protocol stack 714), antenna(s) 710, and interconnects 712 may be similar to like-named elements shown and described with respect to FIG. 6. For example, the processor 702 may include processor circuitry such as, for example, baseband processor circuitry (BB) 716A, central processor unit circuitry (CPU) 716B, and graphics processor unit circuitry (GPU) 716C.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0147] The processor 702 is configured for performance monitoring by determining performance metric value that includes contributions from the top K beams of the set, rather than just the top 1 beam, as described herein. For example, the processor 702 obtains beam selection information that indicate a set of predicted beam values for selected beams by a machine learning model, such as the AI / ML models described previously. The set of predicted beam values are associated with the selected beams, such as described in relation to Table 1 and Table 4. The processor 702 may obtain a set of beam measurements corresponding to the selected beams.
[0148] The processor 702 may determine, for each of the selected beams, a difference between the predicted beam value of a selected beam and a beam measurement of the selected beam. The processor 702 determines the difference for each of the beams selected by the AI / ML model so that the performance metric value (e.g., a composite score) can account for each of the selected beams, as described herein.
[0149] The processor 702 may determine, for each of the selected beams, whether the difference satisfies a threshold value. For beams in which the difference that satisfies (e.g., is less than) the threshold value, the processor 702 may determine a weighted value associated with the beam. The processor 702 generates a value of a performance metric from the weighted values for the selected beams. The performance metric value represents a performance of the beam prediction by the machine learning model, the value of the performance metric including a contribution from each selected beam that satisfies the threshold; and causing transmission of a report indicating a success or failure of the beam prediction by the machine learning model based on the value of the performance metric.
[0150] In another aspect, the processor 702 may obtain selection information indicating a set of predicted beam values for selected beams by a machine learning model. The set of the predicted beam values are associated with the A / ML-selected beams. The processor 702 may obtain a set of beam measurements corresponding to the selected beams. The processor 702 may determine a first score for the beam measurements that is based on the ranking of the genie beam measurements. For example, the first score can include a composite relevance score for each of the selected beams in which each beam is weighted based on its RSRP strength and the order in which the beam is selected or predicted. Lower ranked beams contribute less to the first score. The processor 702 may determine a second score for the beam measurements that is based on the ranking of the predicted beam measurements. For example, the second scoreAttorney Docket No. 56990-0102WO1 / P71934WO1can include a composite relevance score for each of the selected beams for predictions in which each predicted beam is weighted based on its RSRP strength and the order in which the beam is predicted. Lower ranked beams contribute less to the second score.
[0151] The processor 702 may determine, based on the first score and the second score, such as a ratio of the second score to the first score, a value of a performance metric representing a performance of the beam prediction by the machine learning model. The processor can cause transmission of a report indicating a success or failure of the beam prediction by the machine learning model based on the value of the performance metric.
[0152] The CN interface circuitry 706 may provide connectivity to a core network, for example, a 5th Generation Core network (5GC) using a 5GC -compatible network interface protocol such as carrier Ethernet protocols, or some other suitable protocol. Network connectivity may be provided to / from the access node 700 via a fiber optic or wireless backhaul. The CN interface circuitry 706 may include one or more dedicated processors or FPGAs to communicate using one or more of the aforementioned protocols. In some implementations, the CN interface circuitry 706 may include multiple controllers to provide connectivity to other networks using the same or different protocols.
[0153] As used herein, the terms “access node,” “access point,” or the like may describe equipment that provides the radio baseband functions for data and / or voice connectivity between a network and one or more users. These access nodes can be referred to as BS, gNBs, RAN nodes, eNBs, NodeBs, RSUs, TRxPs or TRPs, and so forth, and can include ground stations (e.g., terrestrial access points) or satellite stations providing coverage within a geographic area (e.g., a cell). As used herein, the term “NG RAN node” or the like may refer to an access node 700 that operates in an NR or 5G system (for example, a gNB), and the term “E-UTRAN node” or the like may refer to an access node 700 that operates in an LTE or 4G system (e.g., an eNB). According to various implementations, the access node 700 may be implemented as one or more of a dedicated physical device such as a macrocell base station, and / or a low power (LP) base station for providing femtocells, picocells or other like cells having smaller coverage areas, smaller user capacity, or higher bandwidth compared to macrocells.
[0154] In some implementations, all or parts of the access node 700 may be implemented as one or more software entities running on server computers as part of a virtual network, which may be referred to as a CRAN and / or a virtual baseband unit pool (vBBUP). In V2X scenarios,Attorney Docket No. 56990-0102WO1 / P71934WO1the access node 700 may be or act as a “Roadside Unit.” The term “Roadside Unit” or “RSU” may refer to any transportation infrastructure entity used for V2X communications. An RSU may be implemented in or by a suitable RAN node or a stationary (or relatively stationary) UE, where an RSU implemented in or by a UE may be referred to as a “UE-type RSU,” an RSU implemented in or by an eNB may be referred to as an “eNB-type RSU,” an RSU implemented in or by a gNB may be referred to as a “gNB-type RSU,” and the like.
[0155] Various components may be described as performing a task or tasks, for convenience in the description. Such descriptions should be interpreted as including the phrase “configured to.” Reciting a component that is configured to perform one or more tasks is expressly intended not to invoke 35 U. S. C. § 112(f) interpretation for that component.
[0156] For one or more embodiments, at least one of the components set forth in one or more of the preceding figures may be configured to perform one or more operations, techniques, processes, or methods as set forth in the example section below. For example, the baseband circuitry as described above in connection with one or more of the preceding figures may be configured to operate in accordance with one or more of the examples set forth. For another example, circuitry associated with a UE, base station, network element, etc., as described above in connection with one or more of the preceding figures may be configured to operate in accordance with one or more of the examples set forth in the following examples section.
[0157] Some embodiments described herein can include use of learning and / or non-leaming-based process(es). The use can include collecting, pre-processing, encoding, labeling, organizing, analyzing, recommending and / or generating data. Entities that collect, share, and / or otherwise utilize user data should provide transparency and / or obtain user consent when collecting such data. The present disclosure recognizes that the use of the data in the AI / ML beam prediction processes can be used to benefit users.
[0158] For example, the data can be used to train models that can be deployed to improve performance, accuracy, and / or functionality of applications and / or services. Accordingly, the use of the data enables the AI / ML beam prediction processes to adapt and / or optimize operations to provide more personalized, efficient, and / or enhanced user experiences. Such adaptation and / or optimization can include tailoring content, recommendations, and / or interactions to individual users, as well as streamlining processes, and / or enabling more intuitive interfaces. Further beneficial uses of the data in AI / ML beam prediction processes are also contemplated by the present disclosure.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0159] The present disclosure contemplates that, in some embodiments, data used by AI / ML beam prediction processes includes publicly available data. To protect user privacy, data may be anonymized, aggregated, and / or otherwise processed to remove or to the degree possible limit any individual identification. As discussed herein, entities that collect, share, and / or otherwise utilize such data should obtain user consent prior to and / or provide transparency when collecting such data. Furthermore, the present disclosure contemplates that the entities responsible for the use of data, including, but not limited to data used in association with AI / ML beam prediction processes, should attempt to comply with well-established privacy policies and / or privacy practices.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0160] Examples
[0161] Example 1 includes a method for wireless communication, the method including obtaining, for beam management of available beams for a wireless communication, selection information indicating a set of predicted beam values for selected beams by a machine learning model, the set of predicted beam values associated with the selected beams; obtaining a set of beam measurements corresponding to the selected beams; determining, for each of the selected beams, a difference between the predicted beam value of a selected beam and a beam measurement of the selected beam; determining, for each of the selected beams, whether the difference satisfies a threshold value; calculating, based on the determining, a value of a performance metric representing a performance of the beam prediction by the machine learning model, the value of the performance metric including a contribution from each selected beam that satisfies the threshold; and causing transmission of a report indicating a success or failure of the beam prediction by the machine learning model based on the value of the performance metric.
[0162] Example 2 includes the method of example 1, further including comparing the value of the performance metric to a threshold value, wherein the report indicates the success or the failure of the beam prediction by the machine learning model based on the comparing.
[0163] Example 3 includes the method of example 2, wherein the success or the failure of the beam prediction by the machine learning model represents a rate in which the value of the performance metric satisfied the threshold value.
[0164] Example 4 includes the method of example 2, wherein the threshold value is based on a configuration of a user equipment or a channel of the wireless communication.
[0165] Example 5 includes the method of example 2, wherein the threshold value is based on a ratio of a size of set A beams to a size of set B beams, an order of the set A beams and the set B beams, an identifier of a user equipment, a classifier or regressor model type of the machine learning model, or a type of a case associated with the wireless communication.
[0166] Example 6 includes the method of any of examples 1 to 5, wherein calculating the value of a performance metric comprises weighting a contribution to the performance metric of each selected beam that satisfies the threshold based on a channel realization value.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0167] Example 7 includes the method of example 6, wherein the channel realization value K -comprises: C(n) = £ wkl(RSRP (n) > RSRPbk(n) — X), where wkis a normalization fc=lbgkweighting function, and l(x) returns 1 if condition is satisfied, X is a specified margin in dB, RSRPbkis a genie predicted beam index, and RSRPbgkis a machine learning model predictedbeam index.
[0168] Example 8 includes the method of example 7, wherein the performance metric value N Z C(n)comprises: D(n) = — where N is a number of channel realization values for the selectedbeams.
[0169] Example 9 includes the method of any of examples 1 to 8, wherein the set of predicted beam values associated with the selected beams, the set of beam measurements corresponding to the selected beams, or both the set of predicted beam values associated with the selected beams and the set of beam measurements corresponding to the selected beam are ordered.
[0170] Example 10 includes the method of any of examples 1 to 9, wherein the predicted beam value is a reference signal receive power (RSRP) of a channel state information reference signal (CSI-RS).
[0171] Example 11 includes obtaining, for beam management of available beams for a wireless communication, selection information indicating a set of predicted beam values for selected beams by a machine learning model, the set of predicted beam values associated with the selected beams; obtaining a set of beam measurements corresponding to the selected beams; determining a first score for the beam measurements, the first score based on a ranking of the beam measurements; determining a second score for the predicted beam values, the second score based on the ranking of the predicted beam values; calculating, based on the first score and the second score, a value of a performance metric representing a performance of the beam prediction by the machine learning model; and causing transmission of a report indicating a success or failure of the beam prediction by the machine learning model based on the value of the performance metric.
[0172] Example 12 includes the method of example 11, wherein the first score comprises a relevance score for each of the selected beams based on a signal strength of each of the selected beams in a genie ranked list representing a beam ranking from highest to lowest signal strength.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0173] Example 13 includes the method of any of examples 11 to 12, wherein the second score comprises a relevance score for each of the selected beams based on a signal strength of each of the selected beams in a predicted beam ranked list representing an order of beams predicted by the machine learning model.
[0174] Example 14 includes the method of any of examples 11 to 13, wherein the first score is based on a discounting factor that discounts errors of ranking higher-ranked beams of the ranking of the beam measurements more severely than errors of ranking lower-ranked beams of the ranking of the beam measurements.
[0175] Example 15 includes the method of any of examples 11 to 14, wherein the second score is based on a discounting factor that discounts errors of ranking higher-ranked beams of the ranking of the predicted beam values more severely than errors of ranking lower-ranked beams of the ranking of the predicted beam values.
[0176] Example 16 includes the method of any of examples 11 to 15, wherein the value of a performance metric comprises a ratio of the second score to the first score.
[0177] Example 17 includes the method of any of examples 11 to 16, wherein the value of the first score comprises BRSgenie= £’ wherein Relevance^67116=.D C D Dgenie., c • -- ^77777, and wherein R Rtincludes an RSRP ot a given genie MaxkRSRP“* -MinkRSRP^k(measured) beam bkin the set of K beams, MinkRSRP^nieincludes the lowest RSRP of a genie (measured) beam bkin the set of K beams, andincludes a highest RSRP of a genie (measured) beam bkin the set of K beams.
[0178] Example 18 includes the method of any of examples 11 to 17, wherein the value of the Kpj predictedsecond score comprises BRSpredlcted= Ve evancek — - wherein Relevanceredlcted=l° (fc + l) ’kpredicted beam bkin the set of K beams,includes the lowest RSRP of a predicted beam bkin the set of K beams, andincludes a highest RSRP of a predicted beam bkin the set of K beams.Attorney Docket No. 56990-0102WO1 / P71934WO1
[0179] Example 19 includes the method of any of examples 11 to 18, further including comparing the value of the performance metric to a threshold value, wherein the report indicates the success or the failure of the beam prediction by the machine learning model based on the comparing.
[0180] Example 20 includes the method of example 19, wherein the success or the failure of the beam prediction by the machine learning model represents a rate in which the value of the performance metric satisfied the threshold value.
[0181] Example 21 includes the method of example 19, wherein the threshold value is based on a configuration of a user equipment or a channel of the wireless communication.
[0182] Example 22 includes the method of example 19, wherein the threshold value is based on a ratio of a size of set A beams to a size of set B beams, an order of the set A beams and the set B beams, an identifier of a user equipment, a classifier or regressor model type of the machine learning model, or a type of a case associated with the wireless communication.
[0183] Example 23 includes one or more processors configured to, when executing instructions stored in memory, perform the method of any of examples 1 to 22.
[0184] Example 24 includes a non-transitory computer storage medium encoded with instructions that, when executed by one or more processors, cause the one or more processors to perform the method perform the method of any of examples 1 to 22.
[0185] Example 25 includes a system comprising one or more processors and one or more storage devices on which are stored instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform the method of any of examples 1 to 22.
[0186] Example 26 includes an apparatus comprising one or more baseband processors configured to perform the method of any of examples 1 to 22.
[0187] Example 27 includes a base station comprising one or more processors configured to perform the method of any of examples 1 to 22.
Claims
Attorney Docket No. 56990-0102WO1 / P71934WO1WHAT IS CLAIMED IS:
1. A method comprising:obtaining a set of predicted beam values for a set of beams;obtaining a set of beam measurements corresponding to the set of beams; determining a difference between the predicted beam value of a beam and a beam measurement of the beam;determining, based on the difference, a value of a performance metric representing a performance of beam prediction by a machine learning model; andcausing transmission of a report indicating a success or a failure of the beam prediction by the machine learning model based on the value of the performance metric.
2. The method of claim 1, further comprising:comparing the value of the performance metric to a threshold value, wherein the report indicates the success or the failure of the beam prediction by the machine learning model based on the comparing.
3. The method of claim 2, wherein the success or the failure of the beam prediction by the machine learning model represents a percentage of the beams of the set for which the value of the performance metric satisfied the threshold value.
4. The method of claim 2, wherein the threshold value is based on a configuration of a user equipment or a channel of a wireless communication.
5. The method of claim 2, wherein the threshold value is based on a ratio of a size of set A beams to a size of set B beams, an order of the set A beams and the set B beams, an identifier of a user equipment, a classifier or regressor model type of the machine learning model, or a type of a case associated with a wireless communication.
6. The method of claim 1, further comprising calculating the value of a performance metric comprises weighting a contribution to the performance metric of each beam of the set that satisfies a threshold based on a channel realization value.
7. The method of claim 6, wherein the channel realization value comprises:Attorney Docket No. 56990-0102WO1 / P71934WO1kwhere w is a normalization weighting function, and l(x) returns 1 if condition is satisfied, X is a specified margin in dB, RSRPbkis a genie predicted beam index, and RSRPbgkis a machine learning model predicted beam index.
8. The method of claim 7, wherein the performance metric value comprises:N Z C(n)D(n)=— where N is a number of channel realization values for the beams ofthe set.
9. The method of claim 1, wherein the set of predicted beam values, the set of beam measurements, or both the set of the predicted beam values and the set of beam measurements are ordered.
10. The method of claim 1, wherein the predicted beam value is a reference signal receive power (RSRP) of a channel state information reference signal (CSI-RS).
11. A method comprising:obtaining a set of predicted beam values for a set of beams;obtaining a set of beam measurements corresponding to the set of beams; determining a first score for the beam measurements, the first score based on a ranking of the beam measurements;determining a second score for the predicted beam values, the second score based on the ranking of the predicted beam values;calculating, based on the first score and the second score, a value of a performance metric representing a performance of beam prediction by a machine learning model; and causing transmission of a report indicating a success or failure of the beam prediction by the machine learning model based on the value of the performance metric.Attorney Docket No. 56990-0102WO1 / P71934WO112. The method of claim 11, wherein the first score comprises a relevance score for each of the set of beams based on a signal strength of each of the set of beams in a genie ranked list representing a beam ranking from highest to lowest signal strength.
13. The method of claim 11, wherein the second score comprises a relevance score for each of the set of beams based on a signal strength of each of the set of beams in a predicted beam ranked list representing an order of beams predicted by the machine learning model.
14. The method of claim 11, wherein the first score is based on a discounting factor that discounts errors of ranking higher-ranked beams of the ranking of the beam measurements more severely than errors of ranking lower-ranked beams of the ranking of the beam measurements.
15. The method of claim 11, wherein the second score is based on a discounting factor that discounts errors of ranking higher-ranked beams of the ranking of the predicted beam values more severely than errors of ranking lower-ranked beams of the ranking of the predicted beam values.
16. The method of claim 11, wherein the value of a performance metric comprises a ratio of the second score to the first score.
17. The method of claim 11, wherein the value of the first score compriseswherein RSRPbknieincludes an RSRP of a given genie (measured) beam bkin the set7 of K beams, MinkRSRPbkincludes the lowest RSRP of a genie (measured) beam bkin the set of K beams,includes a highest RSRP of a genie (measured) beam bkin the set of K beams.Attorney Docket No. 56990-0102WO1 / P71934WO118. The method of claim 11, wherein the value of the second score comprises B ^^prewhere sRP^edlctedincludes an RSRP of a given predicted beam Bkin the set of K beams,includes the lowest RSRP of a predicted beam Bkin the set of K beams, andincludes a highest RSRP of a predicted beam Bkin the set of K beams.
19. The method of claim 11, further comprising:comparing the value of the performance metric to a threshold value, wherein the report indicates the success or the failure of the beam prediction by the machine learning model based on the comparing.
20. The method of claim 19, wherein the success or the failure of the beam prediction by the machine learning model represents a rate in which the value of the performance metric satisfied the threshold value.
21. The method of claim 19, wherein the threshold value is based on a configuration of a user equipment or a channel of a wireless communication.
22. The method of claim 19, wherein the threshold value is based on a ratio of a size of set A beams to a size of set B beams, an order of the set A beams and the set B beams, an identifier of a user equipment, a classifier or regressor model type of the machine learning model, or a type of a case associated with a wireless communication.
23. One or more processors configured to, when executing instructions stored in memory, perform the method of any preceding claim.Attorney Docket No. 56990-0102WO1 / P71934WO124. A non-transitory computer storage medium encoded with instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any of claims 1 to 22.
25. A system comprising one or more processors and one or more storage devices on which are stored instructions that are operable, when executed by the one or more processors, to cause the one or more processors to perform the method of any of claims 1 to 22.
26. An apparatus comprising one or more baseband processors configured to perform the method of any of claims 1 to 22.
27. A base station comprising one or more processors configured to perform the method of any of claims 1 to 22.
