User equipment, network node and method for evaluating model performance

Abstract

The present disclosure relates to a method (400) performed by a user equipment, UE, (220) in a radio network (100), the method comprising calculating a model performance metric based on a received configuration, where the model performance metric reflects a performance of a beam prediction model, transmitting a report indicating the calculated model performance metric.

Description

User equipment, network nodes, and methods used to evaluate model performance Technical Field

[0001] The embodiments described herein relate to user equipment (UE), network nodes, and methods for evaluating model performance. Background Technology

[0002] In today's radio networks, large antenna arrays are used to improve the performance of both the receiver and transmitter sides, especially the performance of the radio channel between the radio access node and the UE.

[0003] Large antenna arrays utilize beamforming by controlling the use of antennas. To reduce hardware costs, large antenna arrays used for high frequencies employ time-domain analog beamforming. The core idea of ​​analog beamforming is to share a single radio frequency chain among many (or potentially all) antenna elements. The limitation of analog beamforming is that, at a given time, only one beam (in one direction) can be used to transmit radio energy.

[0004] The above restrictions require the network (NW) and UE to perform beam management procedures to establish and maintain appropriate transmitter (Tx) / receiver (Rx) beam pairs.

[0005] Traditionally, this is typically performed by transmitting reference symbols in different antenna beams in the downlink, measuring the reference signal at the UE, reporting the reference signal characteristics to the serving network node, and the network node deciding which beam pair to use. This has the disadvantage of introducing delay when selecting the beam pair.

[0006] Therefore, the beam management process needs to be improved. Summary of the Invention

[0007] The subject matter described herein overcomes the aforementioned drawbacks. Further advantageous implementations of the embodiments described herein are described herein and set forth in the appended set of claims.

[0008] The purpose of the embodiments described herein is to improve the performance of radio networks.

[0009] According to one aspect of the embodiments herein, this objective is achieved by a method performed by a UE in a radio network. The UE calculates a model performance metric based on a received configuration. The model performance metric reflects the performance of the beam prediction model. The UE transmits a report indicating the calculated model performance metric.

[0010] According to one aspect of the embodiments herein, this objective is achieved by a method performed by a network node in a radio network. The network node generates a configuration for monitoring antenna beam prediction performance. The network node transmits a signal including this configuration to the UE.

[0011] According to one aspect, according to embodiments herein, this objective is achieved by providing a UE and a network node respectively configured to perform the methods herein.

[0012] Please refer to the accompanying figures, which are described briefly at the beginning. Attached Figure Description

[0013] It should be understood that the same reference numerals are used to identify the same elements shown in one or more of the figures.

[0014] Figure 1A illustrates an example of a radio network. A radio network includes multiple nodes configured to transmit radio signals within the network.

[0015] Figure 1B illustrates the transmitter and receiver antenna beams according to one or more embodiments of the present disclosure.

[0016] Figure 2 illustrates the accuracy requirements for model performance metrics.

[0017] Figure 3A illustrates signaling for antenna beam management according to one or more embodiments of the present disclosure.

[0018] Figures 3B-C illustrate the options for AI / ML models.

[0019] Figure 4 is a flowchart of a method according to one or more embodiments of the present disclosure.

[0020] Figure 5 shows a flowchart of a method according to one or more embodiments of the present disclosure.

[0021] Figure 6 illustrates real-time monitoring based on inference accuracy.

[0022] Figure 7 illustrates details of a network node according to one or more embodiments of the present disclosure.

[0023] Figure 8 shows the SSB beam selection as part of the initial access procedure according to scenario P1.

[0024] Figure 9 illustrates the CSI-RS Tx beam selection in the downlink according to the P2 scenario.

[0025] Figure 10 shows the UE Rx beam selection for the corresponding CSI-RS Tx beam in DL according to the P3 scenario.

[0026] Figure 11 shows an example where set B is a subset of set A.

[0027] Figure 12 shows an example where set A is a set of narrow beams and set B is a set of wide beams.

[0028] Figure 13 shows an alternative table of one or more performance metrics monitored by AI / ML models with BM-Case 1 and BM-Case 2.

[0029] By considering the following detailed description of one or more embodiments, those skilled in the art will gain a more complete understanding of the embodiments herein, as well as their additional advantages. It should be understood that the same reference numerals are used to identify the same elements shown in one or more of the figures. Detailed Implementation

[0030] This disclosure relates to beam management processes for large array antenna systems. In particular, it relates to systems that use transmitter or receiver beams or beam transmitter / receiver beam pairs that utilize artificial intelligence (AI) and / or machine learning (ML) for characteristic prediction.

[0031] The problem with this prediction method is the significant overhead incurred in calculating beam prediction performance metrics, including signaling, reference symbols (RS), transport resources, and user equipment (UE) measurements. A large sample size is required to obtain sufficient statistics to determine whether the model is activated / deactivated on the network NW side. The term "sample" in this paper refers to a comparison between values ​​predicted by the model and actual values ​​of those values, such as predicted signal strength and signal strength measured using reference symbols. This comparison may include calculating model performance metrics.

[0032] Based on NW reception performance metrics, the NW may, for example, have a threshold for the required accuracy needed to activate the NW / UE-side beam prediction model; for instance, for a model to be activated (or not deactivated), the required accuracy needs to be higher than 50%. Then, performance metrics such as beam prediction accuracy higher than 60% may influence the selection of a set of radio parameters in one way (e.g., how many beams to measure in Top-K), and in another way higher than 70%. For example, in the case of very high accuracy, the NW may not need to perform additional measurements to find the top-1 beam in the BM use case in the 3GPP 3rd Generation Partnership Project. For example, BM-Case 1: Spatial domain DL beam prediction of beam set A based on measurements of beam set B.

[0033] A problem arises with low-precision (incorrect) predictive models. NW might expend unnecessary resources on calculating performance metrics for models with very poor performance (i.e., those with less than, for example, 50% of the metrics). In such cases, NW is not interested in activating and using such models.

[0034] Furthermore, NW is not interested in knowing whether a poorly performing model reaches 10% or 20% accuracy; it will not activate such a model in any way. Note that 50% is an arbitrarily chosen number; for other models used for URLLC traffic, a model may require >90% accuracy to be activated.

[0035] Generally speaking, NW should spend more resources understanding whether the model's accuracy is 90% or 95% than whether the model's accuracy is 5% or 10%.

[0036] Therefore, minimizing the resource overhead, signaling overhead, and UE battery consumption used to perform beam prediction performance monitoring is a problem.

[0037] This disclosure is based in part on the understanding that when a beam prediction model performs poorly and gives very large prediction errors, it may be sufficient with just one or two samples to determine that the model is performing poorly and should be discontinued.

[0038] Therefore, the embodiments in this paper utilize early detection and deactivation of poorly performing AI / ML models.

[0039] Figure 1A illustrates an example of a radio network 100. The radio network includes multiple nodes N1-N configured to transmit radio signals within the radio network 100. 21 Each node includes one or more antennas and circuitry capable of performing radio transmissions. In a non-limiting example, a node is a radio access node and / or a user-operated node, such as a UE.

[0040] Then, certain transmission characteristics are used to perform radio signal transmission between nodes and / or UEs.

[0041] Figure 1B illustrates the transmitter and receiver antenna beams according to one or more embodiments of the present disclosure.

[0042] In the example of Figure 1B, network node 210 is shown with eight TX antenna beams TX1-TX8. In the example of Figure 1B, UE 220 is shown with eight RX antenna beams RX1-RX8. An antenna beam management procedure can then be used to select a suitable antenna pair, such as TX5 and RX1.

[0043] Large antenna array systems typically consist of multiple receive antennas RX1-RX8 and multiple transmit antennas TX1-TX8. The respective antennas utilize phase network electrical coupling and communication coupling.

[0044] In this disclosure, the antenna beam for measuring characteristics is shown as a dashed line, and the remaining antenna beams are shown as solid lines.

[0045] In one example, the UE measures the beam characteristics of the RS signals received from TX beams TX3, TX5, and TX7. The method of this disclosure then predicts the characteristics of antenna beams TX1-TX8 and then optionally sorts the antenna beams TX1-TX8 according to the suitability for node-UE communication, for example, providing antenna beams with optimal signal strength and / or signal quality.

[0046] In one example, the model performance metric can then be calculated by comparing the characteristics of the measurement signals received from the TX beams TX3, TX5, and TX7 with the prediction characteristics of the TX beams TX3, TX5, and TX7.

[0047] In one aspect of this disclosure, a method is provided to enable the NW to trigger the UE to transmit reports including early performance indicators / model performance metrics (e.g., indicators of model performance, or performance metric calculations). Early transmission of these reports benefits poorly performing AI / ML beam prediction models by reducing signaling, reference symbol resources, UE measurement overhead, and UE battery consumption in performance metric calculations. In other words, if the beam prediction results are too poor to be considered, there is no need to transmit RS, measure RS, predict beams, or report performance.

[0048] Figure 2 illustrates the accuracy requirements for model performance metrics.

[0049] The examples in this paper utilize the fact that NW is only interested in precise performance metrics of models whose performance range will activate the model, and can identify poorly performing models fairly quickly. This is highlighted in Figure 2.

[0050] The examples of embodiments described herein also provide the NW with a means to react quickly to degradation of the UE-side beam prediction model by allowing early indication of predictive degradation in observation performance monitoring performed transparently by the UE.

[0051] In summary, this disclosure includes at least a method for calculating beam prediction performance metrics in the UE, and the UE can send an early report to the NW if the performance is below expectations (e.g., below a threshold).

[0052] When the NW receives such an early report of a poorly performing AI / ML model, it disables performance metric evaluation in the UE, that is, it disables the transmission of the reference symbol RS by the serving network node that intends to perform the performance estimation. This also reduces future RS overhead, signaling overhead, and UE battery consumption (because the UE disables performance metric calculation as a response to this disablement from the NW or as a response to reporting the early report to the NW).

[0053] This disclosure has at least the following advantages: reducing the overhead of disabling RS transmission.

[0054] The early cessation of performance monitoring reduced UE battery consumption.

[0055] By disabling RS transmission, NW power consumption is reduced.

[0056] RS transmission reduces interference to neighboring cells.

[0057] The impact of poorly performing models on the NW can be reduced by providing means to react more quickly to model performance degradation (e.g., the NW requests the UE to disable functionality / models when it receives an early performance indication).

[0058] Figure 3A illustrates signaling for antenna beam management according to one or more embodiments of the present disclosure.

[0059] Action 310. Network node 210 generates a configuration for monitoring antenna beam prediction performance.

[0060] Action 315. Network node 210 then sends a first signal S1, including the configuration, to UE 220.

[0061] In one embodiment, an NW such as network node 210 configures UE 220 to monitor functionality / model based on N time instances and report performance indications at the functionality level. In one example, UE 220 is configured to perform N measurements, perform N predictions, and compute N model performance metrics or performance samples. The N model performance metrics are then aggregated into a total model performance metric in an appropriate manner. UE 220 is requested to report indications of model performance failures, while the performance metric computation remains transparent to the NW.

[0062] In another embodiment, NW-configured UE 220, such as network node 210, monitors the model based on N time instances and reports the performance metrics corresponding to the model.

[0063] Rules such as the NW indication / configuration of network node 210 regarding when UE 220 can trigger / report early performance indications.

[0064] In the context of the example above, this means using fewer than N samples to generate aggregated model performance metrics.

[0065] Signaling used for UE reporting / configuration may include one or more of the following: - Indication of whether early performance indication is allowed - Definition of events that UE 220 can trigger early performance indication - Indication of time / frequency resources that can be used for reporting - Range list, including an indication of the range of exact values ​​that UE 220 should report based on all N time instances - Threshold for tolerance error (see the real-time monitoring example below) - Minimum number of samples / time instances that UE 220 should use before assessing whether the metric is below or above the threshold - Whether UE 220 should abort ongoing performance monitoring (to conserve battery) action 320 if the tolerance error is below the threshold. UE 220 then calculates a model performance metric based on the received configuration, such as a model performance metric where the model is optionally configured to sort a set of candidate beamforming beams. The model performance metric may, for example, reflect the performance of the model.

[0066] Before calculating model performance metrics, UE 220 has performed measurements on a set of beams (e.g., TX3, TX5, and TX7 shown in Figure 1B).

[0067] The predicted characteristics of beams TX1-TX8, such as the measured and ordered signal-to-interference-plus-noise ratio (SINR) and / or the measured and ordered reference signal strength indicator (RSRP), are then compared with the measured characteristics of beams TX3, TX5, and TX7. It should be understood that the measured characteristics of the beams may include any suitable characteristics, such as the reference signal reception quality (RSRQ) and / or the received signal strength indicator (RSSI).

[0068] In one example, the configuration generated by the NW, such as network node 210, can be represented using a classifier. For example, as a list indicating the corresponding range, such as [0%, 50%, 75%, 85%, 90%, 99%, 100%]. This list can instruct UE 220 in which range it should report the exact value based on all measurement time instances (N). The number N can be included in the configuration sent to UE 220.

[0069] In another example, the configuration generated by the NW (e.g., network node 210) can be represented as a uniform list of ranges, where a single value indicates the resolution. For example, a value of 10 divides the range of 10 uniformly distributed bins [0%, 10%, 20%, ... 90%, 100%].

[0070] In another example, the configuration generated by the NW (e.g., network node 210) could be a regression value, for example, a list indicating average / median / maximum / minimum error values ​​[0-3 dB error, 3-5 dB error, 5 dB -> infinite error]. This error could, for example, represent the difference between the measured and predicted characteristics. In an embodiment, this list could be based on all measurement time instances (N) to indicate in which range the UE should report the exact value.

[0071] The range selection of an NW, such as network node 210, can be based, for example, on the accuracy required by the beam prediction model. For instance, if the model requires more than 50% accuracy. Furthermore, it can select an upper limit range based on the model's configuration; if the model is 99% accurate instead of 90% accurate, the NW, such as network node 210, can rely more on prediction rather than configuring additional measurements. For example, using option 1) instead of option 2) in the scheme shown in Figures 3B-C reduces the latency from characteristic measurement to data transmission.

[0072] Set A represents all predicted beams TX1-TX8, and set B represents the measurement beams TX3, TX5, and TX7, on which beam characteristic measurements such as RSRP and SINR are performed.

[0073] Figure 3B shows the first option, where the AI / ML model predicts the Top-1 beam.

[0074] Figure 3C shows the second option, where the AI / ML model predicts the Top-K beam.

[0075] For regression scenarios, the NW of network node 210 can, for example, select an error range based on whether such a metric can be used in link adaptation, such as the modulation and coding scheme MCS, and choose an RSRP metric estimate that should require less than x dB error.

[0076] Optionally, if model performance reporting / early delivery of reports is enabled, UE 220 also performs the following steps: Action 330. UE 220 evaluates whether the calculated model performance metric meets the criteria for early delivery of reports. This is also referred to herein as a trigger event.

[0077] Action 340. If the calculated model performance metric meets the criteria in signal S3, then UE 220 subsequently transmits the report. In other words, it reports the early performance metric.

[0078] Action 350. Network node 210 may optionally transmit signal S4, including updated configuration, to UE 220.

[0079] Action 360. If no early transmission of the report is performed / no event is triggered, UE 220 sends the model performance report / report in signal S5 after a predetermined time and / or period and / or time instance. The predetermined time and / or period and / or time instance are included in and defined by the configuration received in signal S1.

[0080] Figure 4 illustrates a flowchart of a method according to one or more embodiments of the present disclosure. Method 400 is performed by UE 220 in radio network 100. The method includes the following steps, which may be performed in any suitable order. Optional steps are indicated by dashed boxes in Figure 4.

[0081] Step 410: UE 220 calculates a model performance metric based on the received configuration, where the model is configured to rank a set of candidate beamforming beams TX1-TX8, RX1-RX8. In other words, UE 220 calculates the model performance metric based on the received configuration. The model performance metric reflects the performance of the beam prediction model. As mentioned above, the received configuration may, for example, include the configuration contained in the signal S1 received from network node 210. This step is related to action 315 above.

[0082] In some embodiments, the received configuration instruction to the UE 220 is for calculating a model performance metric based on a first number of "X" time instances. This could, for example, mean that the configuration sets the UE 220 to calculate the performance metric based on the first number of "X" time instances.

[0083] In some embodiments, such as through received configuration, the UE 220 is configured with a set of ranges for one or more model performance metrics. The ranges in the configured set of ranges may be uniformly distributed between a minimum and a maximum value, or non-uniformly distributed between a minimum and a maximum value.

[0084] Step 420: UE 220 transmits a report including calculated model performance metrics. This report may, for example, be sent to network node 210. This may include model performance metrics aggregated for each time instance. This relates to action 360 above.

[0085] Alternatively or concurrently, reports are transmitted periodically. This can be performed using predetermined times and / or periods and / or time instances, as further described with reference to Figure 3A.

[0086] In some embodiments, the report may indicate a specific range from a configured set of ranges. The specific range may include a computed performance metric.

[0087] Additionally or alternatively, the method further includes: Step 430: In some embodiments, UE 220 evaluates whether the calculated model performance metric meets the criteria for an early delivery of the report. In other words, if an event is triggered. This relates to action 330 above.

[0088] In one embodiment, the standard indicates one or more samples of performance that are below a certain performance level.

[0089] The UE can trigger an event / criteria if certain conditions are met, such as when a model performance metric falls within a reporting range (e.g., within 0-50% accuracy). For example, an event can be triggered using RRC signaling, or a criterion can be met. The NW can allocate pre-configured uplink resources, which the UE can use for the early delivery of reports including model performance metrics.

[0090] In another embodiment, the UE may request dedicated resources to report performance indications. Information can be sent using UCI or PUSCH (RRC / MAC messages).

[0091] In one embodiment, the model performance report is merely an indication of a failure in the model performance process.

[0092] In one related embodiment, a failure indication is signaled in a pre-configured single-bit PUCCH resource.

[0093] In another related embodiment, the failure indication is signaled by a dedicated scheduling request (SR).

[0094] In one embodiment, the model performance failure report also includes information on new preferred gNB beams. For example, if the model performance is poor, the predicted gNB beams reported in other gNB beam prediction reports associated with that model may be invalid. Therefore, in this case, the model performance failure report may also include the top K gNB beams based on actual measurements, enabling the network to acquire / maintain appropriate beam pair links between the gNB and the UE.

[0095] Here are some examples of different performance level configurations that may trigger the early model performance metric report delivery (if this occurs on N performance samples, where N can be 1 or greater than 1): The RSRP of the predicted best beam differs from the RSRP of the measured best beam by more than X dB (where the predicted best beam and the measured best beam can be two different beams); The predicted RSRP of the predicted best beam differs from the measured RSRP by more than X dB (i.e., comparing the predicted RSRP and the measured RSRP for the same beam); Assuming the UE predicts the RSRP of K top beams, and the UE measures the RSRP of the same K top beams. In this scenario, for example, an early performance metric report can be triggered / transmitted if the following conditions are met: - For any of the K top beams, the RSRP difference between the predicted and measured RSRP is higher than a certain threshold (e.g., assuming K=2, and the difference between the predicted and measured RSRP of the first beam is 3 dB, the difference between the predicted and measured RSRP of the second beam is 6 dB, and the threshold is 4 dB, then the early performance metric report event is triggered / transmitted because at least one of the K top beams has an RSRP difference higher than the threshold). - The RSRP difference between the predicted and measured RSRP of all K top beams is higher than a certain threshold (e.g., assuming K=2, and the difference between the predicted and measured RSRP of the first beam is 3 dB, the difference between the predicted and measured RSRP of the second beam is 6 dB, and the threshold is 4 dB). If the difference is less than 3 dB, no early performance metric report event is triggered / transmitted because at least one of the K top beams has an RSRP difference less than a threshold. - If the average RSRP difference between the predicted and measured RSRPs of all K top beams is greater than a certain threshold (e.g., assuming K=2, and the difference between the predicted and measured RSRPs of the first beam is 3 dB, the difference between the predicted and measured RSRPs of the second beam is 6 dB, and the threshold is 4 dB, then an early performance metric report event is triggered / transmitted because the average RSRP error across all K top beams is greater than the threshold (3+6 / 2 = 4, 5>4).

[0096] In some embodiments, the evaluation is based on a model performance metric calculated based on a second number of time instances less than a first number "X".

[0097] Step 440: In some embodiments, if the calculated model performance metric meets the criteria, UE 220 transmits a report. As shown in Figure 3A, early transmission of the report is performed by sending signal S3. This is related to action 340 above.

[0098] In some embodiments, the report may indicate a specific range from a configured set of ranges. The specific range may include a computed performance metric.

[0099] Additionally or alternatively, the method further includes: step 450: in some embodiments, UE 220 receives an updated configuration. This is related to action 350 above.

[0100] The optional steps to provide new performance metrics to the UE can be performed, for example, via RRC configuration or via DCI signaling.

[0101] The advantage of the latter approach (DCI signaling) is that it facilitates faster changes. For example, this could be useful if, for some reason, the NW decides at some point in time to use a different beam than the one predicted by the UE; in such cases, predicted performance measurements in the UE based on BLER / data link quality (e.g., 3GPP Alt-2) could be misleading, and the NW might want to change the performance metric calculation very temporarily.

[0102] Alternatively, updated configurations may be received via downlink control information (DCI) signaling and / or via radio resource control (RRC) signaling.

[0103] Additional or alternative locations are reported via Radio Resource Control (RRC) signaling.

[0104] Alternatively or concurrently, the report may be transmitted via uplink control information (UCI) signaling and / or physical uplink shared channel (PUSCH) signaling and / or dedicated scheduling request signaling.

[0105] Additionally or alternatively, the received configuration indication is an instruction to the UE to calculate model performance based on X time instances, wherein the report indicates the functionality level.

[0106] Additional or alternative location, where the number X is equal to or greater than 1.

[0107] Alternatively or additionally, the received configuration instruction gives the UE instructions to calculate model performance based on X time instances, and the report indicates model-specific performance measurements.

[0108] Additionally or alternatively, the received configuration indication selects from any of the following: - an indication of whether early transmission of reports is allowed, - a definition of the criteria by which the UE can trigger early transmission of reports, - an indication of the time / frequency resources available for transmitting reports, - a list of model performance ranges, - a threshold indicating the permissible prediction error of the model, - a minimum number of measurement samples and / or time instances that the UE should use before evaluating the calculated model performance, - an indication of whether the UE 220 should suspend ongoing performance monitoring if the calculated model performance metric falls below the threshold.

[0109] Additionally or alternatively, the calculated model performance metrics are based on the time distance between predictions that do not meet the evaluation criteria and / or the frequency of predictions that do not meet the evaluation criteria.

[0110] In this embodiment, the distance between two consecutive error samples (also referred to as "error events") is used to calculate the model performance metric. In this context, distance refers to the time difference between two error samples / events. An error event refers to an unsatisfactory AI / ML model output. For example, in beam prediction, an error event is an incorrect prediction by the AI / ML model, such as a difference of more than X dB between the predicted RSRP and the measured RSRP of the best-predicted beam.

[0111] Generally, if the AI / ML model performs well, error events occur rarely, and the distance d between two consecutive error events is small. err The distance d is large. Conversely, if the AI / ML model deteriorates, error events become more frequent, and the distance d between two consecutive error events increases. err Smaller. Therefore, the distance d err It can be an early indicator of model performance degradation.

[0112] In one embodiment, distance d err The statistics are calculated and used to derive model performance metrics. The statistics can be the distance d within a time window. err average and standard deviation Furthermore, the time window can be a sliding time window to track the performance of an ongoing AI / ML model. Model monitoring metrics can be... and function .if Less than (i.e., worse than) the threshold T d，err If the model deteriorates, an early performance metric report is triggered to signal a warning that the model is deteriorating; otherwise, the model is considered normal and no warning is triggered.

[0113] function It could be: , where A is a constant (e.g., A=1 or A=2).

[0114] Threshold T d，err This can be a fixed or predefined value, set based on the model's normal performance, and obtained using a reference dataset (e.g., a test dataset or training dataset). Alternatively, the threshold T... d，err It can be a dynamically changing value, for example, ,in This corresponds to the best execution instance observed during the historical reference time period (i.e., the maximum distance). The parameter p (p<1) is a scale parameter used to set the threshold, for example, p=0.1 or 0.2.

[0115] Additionally or alternatively, the calculated model performance metrics are also based on the severity classification of predictions that do not meet the evaluation criteria.

[0116] In this embodiment, model performance metrics utilize both (a) the distance between two consecutive error events and (b) the severity of the error events. This is based on the fundamental principle that as an AI / ML model deteriorates, error events tend to occur more frequently, and the model output values ​​tend to deviate further from the true values. Therefore, model performance metrics can be formulated to reflect both of these factors, which is used to determine whether an early performance metric report should be triggered / transmitted.

[0117] In one example, the model monitoring metric F could be the distance d between two consecutive errors. err The statistics of beamforming and the distance between the predicted RSRP and the true RSRP are statistical functions. A concrete example is that F is... The function.

[0118] In one example in here The mean and standard deviation corresponding to the best performing instances (maximum distance) observed during the historical reference period; and This corresponds to the mean and standard deviation of the worst RSRP error (the largest difference between the predicted RSRP and the actual RSRP) observed during the historical reference period. Parameters w1 and w2 are used to provide the relative weights between the distance of the error event and the severity of the error.

[0119] if Less than (i.e., worse than) the threshold T d，RSRP，err If the model deteriorates, an early performance metric report is sent to signal a warning flag indicating model degradation; otherwise, the model is considered normal and no warning flag is triggered.

[0120] Additionally or alternatively, the transmitted report may include the result of a function that transforms the computed model performance metric into a tag (e.g., an index). This index may, for example, indicate a specific range within a configured set of ranges. The specific range may include the computed model performance metric. Thus, only the index of the range needs to be transmitted in the report, instead of the computed performance metric. This can, for example, result in reduced signaling overhead.

[0121] In some embodiments, there exists a function f that, for any given number of samples and associated model performance / quality measurements, converts them into an index (or a binary decision regarding whether to send an early performance metric report). Function f may be predefined in the specification, but may also have one or more parameters configured (or signaled) by the network.

[0122] According to any of the foregoing embodiments, the per-sample model performance metric / quality measurement can be a binary (success / failure) or multi-step scale. In the case of a binary measurement, under measurement uncertainty, the measurement can be viewed as a binomial distribution with parameters n (the sample situation so far) and p (the probability of failure or success per sample), and the function f can be based on this distribution.

[0123] Additionally or alternatively, when calculating model performance metrics, this function uses the timestamps of the measured samples to weight the samples.

[0124] In addition to quality measurements, function f can optionally take the (relative) timing (timestamp) of the samples as input. For example, function f can give lower weight to earlier sampling times and more weight to more recent sampling times, because more recent samples better reflect the current performance under changing channel conditions.

[0125] In a non-limiting example, the NW configures the UE to estimate and / or rate beam prediction accuracy. The NW may, for example, configure different resolutions for the UE for performance metric calculations (e.g., beam prediction accuracy). An example of different ranges of model performance for accuracy metrics is shown below: Index 0, Index 1, Index 2, Index 30->50, 50->75, 75-90. Report the exact numbers.

[0126] In this example, the UE is configured to perform measurements over 20 time instances. After time instance 10, if all 10 calculated model performance metrics indicate that the prediction is incorrect, the UE will understand that the final result / model performance metric will be part of the first range (0-50% / index 0). Therefore, the UE can send early feedback (index 0) to the NW. The NW can then avoid spending more RS resources on the monitoring process (saving 10 out of 20 time instances in this simple example). The UE reports its accuracy within the first range (0->50%).

[0127] In another example, if one or more samples fall below a certain performance level, the network can be configured to allow the UE to trigger an Early Model Performance Metric Reporting (EMTM) event. For instance, if the predicted RSRP of the optimal beam and the measured actual RSRP of the optimal beams TX3, TX5, and TX7 differ by more than a certain threshold, the network can be configured to allow the UE to report an ETM event. The network can also configure the number of samples that must occur before the ETM event is triggered.

[0128] Alternatively or additionally, the configuration may include constraints for calculating performance metrics of the model.

[0129] In one example, when a sample can be part of a performance metric calculation, the NW can set constraints, such as allowing the UE to include a new sample only if it is different from a previous sample with a specific threshold (thus excluding duplicate samples).

[0130] In some embodiments, the UE can be configured to consider past measurements in addition to the current set of samples when calculating performance metrics.

[0131] In some embodiments, UE 220 receives an indication that model performance metrics are not calculated in the remaining time instances of a first number "X" time instances. This indication may, for example, indicate that model performance metrics are not calculated in the remaining time instances of the first number "X" time instances if an early report has been or will be transmitted. This indication is received in either the received configuration, the updated configuration, or separately from either the received configuration or the updated configuration.

[0132] Alternatively or concurrently, in response to a report that was transmitted earlier, UE 220 receives a request to disable the model from network node 210.

[0133] Figure 5 illustrates a flowchart of a method according to one or more embodiments of the present disclosure. Method 500 is performed by a network node 210 in a radio network 100. The method includes the following steps, which may be performed in any suitable order. Optional steps are indicated by dashed boxes in Figure 5.

[0134] Step 510: Network node 210 generates a configuration for monitoring antenna beam prediction performance. This is related to action 310 above.

[0135] Step 520: Network node 210 transmits signal S1, which includes this configuration, to UE 220. This is related to action 315 above.

[0136] Additionally or alternatively, the method also includes receiving a report indicating beam prediction performance, such as a model performance metric indicating the beam prediction model. This relates to actions 340 and / or 360 above. This is based on whether the report is an early report (action 340) or a non-early report (action 360).

[0137] Alternatively or additionally, the method may also include transmitting updated configurations. This relates to action 350 above.

[0138] Alternatively, the updated configuration may be transmitted via downlink control information (DCI) signaling and / or via radio resource control (RRC) signaling.

[0139] Additional or alternative locations receive reports via Radio Resource Control (RRC) signaling.

[0140] Alternatively, reports may be received via uplink control information (UCI) signaling and / or physical uplink shared channel (PUSCH) signaling and / or dedicated scheduling request signaling.

[0141] Additionally or alternatively, the transmitted configuration instructions are given to the UE to calculate model performance based on X time instances, and the report indicates the functionality level.

[0142] Additional or alternative location, where the number X is equal to or greater than 1.

[0143] Additionally or alternatively, the transmitted configuration instructions to the UE 220 are used to calculate model performance based on X time instances, and the report indicates model-specific performance measurements.

[0144] Additionally or alternatively, the transmission configuration indication selects from the following: an indication of whether early transmission of reports is allowed, a definition of the criteria by which the UE can trigger early transmission of reports, an indication of the time / frequency resources available for transmitting reports, a list of model performance ranges, a threshold indicating the permissible prediction error of the model, a minimum number of measurement samples and / or time instances that the UE should use before evaluating the calculated model performance, and an indication of whether the UE (220) should suspend ongoing performance monitoring if the calculated model performance metric falls below the threshold.

[0145] Additionally or alternatively, the transmitted configuration instructions to the UE 220 are used to calculate model performance metrics based on a first number of “X” time instances.

[0146] Additionally or alternatively, the received report is an earlier transmitted report, where the model performance metric for the beam prediction model is based on a second number of time instances less than the first number "X" time instances. This relates to action 340 above.

[0147] Additionally or alternatively, the network node sends an indication that performance metrics are not calculated in the remaining time instances of the first number "X" time instances. This indication may, for example, indicate that model performance metrics are not calculated in the remaining time instances of the first number "X" time instances if the UE 220 has already transmitted or will transmit an early report. This indication is received in either the received configuration, the updated configuration, or separately from either the received configuration or the updated configuration.

[0148] Additionally or alternatively, this configuration indicates a range of one or more performance metrics.

[0149] Additionally or alternatively, the report indicates a specific range within a set of configured ranges. This specific range may include computed performance metrics.

[0150] Additionally or alternatively, the range may be either uniformly distributed between the minimum and maximum values ​​or non-uniformly distributed between the minimum and maximum values.

[0151] Additionally or alternatively, the report includes an index indicating a specific range within a set of configured ranges. A specific range may include model performance metrics.

[0152] Alternatively or concurrently, in response to a received report being an earlier transmitted report, network node 220 requests UE 220 to disable the model.

[0153] Figure 6 illustrates real-time monitoring based on inference accuracy.

[0154] As described in 3GPP RAN1#114 “Ericsson-R1-2306928 Discussion on general aspects of AIML framework”, one aspect of the UE / NW side model is the real-time monitoring of the AI / ML model, where the NW can activate the model in a certain time window based on monitoring results from previous time windows. This is a resource-intensive approach, and the additional resources used for monitoring should be minimized if possible.

[0155] According to an example of an embodiment described herein, the UE can trigger the transmission of an early report, where the NW has been pre-configured with UL resources for transmitting control information in any time instance from time instance 1 to 7 in Figure 6. In this embodiment, the UE can be configured with a value of desired precision to be activated in subsequent time windows (e.g., if 6 / 7 needs to be activated, then if the UE observes 2 errors, the UE can trigger an early report). In the case where the UE observes 2 errors in time instance 1-2, it will therefore save 5 measurements.

[0156] Figure 7 illustrates details of a network node 210 according to one or more embodiments of the present disclosure.

[0157] Network node 210 can be any of the following forms: gNB, network node, desktop computer, server, laptop, mobile device, smartphone, tablet computer, smartwatch, etc. Network node 210 may include processing circuitry 912. Network node 210 may optionally include a communication interface 904 for wired and / or wireless communication. Furthermore, network node 210 may also include at least one optional antenna (not shown). The antenna may be coupled to a transceiver of the communication interface 904 and configured, for example, to transmit and / or receive wireless signals in a wireless communication system.

[0158] In one example, the processing circuitry 912 may be any selection of processors and / or central processing units and / or processor modules and / or multiple processors configured to cooperate with each other. Furthermore, network node 210 may further include memory 915. Memory 915 may contain instructions executable by the processing circuitry 912, which, when executed, cause the processing circuitry 912 to perform any of the methods and / or method steps described herein.

[0159] Communication interface 904, such as a wireless transceiver and / or a wired / wireless communication network adapter, is configured to transmit and / or receive data values ​​or parameters as signals. In embodiments, communication interface 904 enables communication directly between nodes or via a communication network.

[0160] In one or more embodiments, network node 900 may further include an input device or interface 917 configured to receive input or indication from a user and output / transmit user input signals indicating user input or indication to processing circuitry 912. In other words, input device / interface 917 receives input or indication from the user and translates it into data that processing circuitry 912 can interpret.

[0161] In one or more embodiments, network node 210 may further include display 918, which is configured to receive a display signal from processing circuitry 912 indicating an object to be presented (e.g., a text or graphical user input object), and to display the received signal as an object, such as a text or graphical user input object.

[0162] In one embodiment, the display 918 is integrated with a user input device / interface 917 and is configured to receive display signals from the processing circuitry 912 indicating an object to be presented, such as a text or graphic user input object, and to display the received signals as objects, such as text or graphic user input objects, and / or is configured to receive input or indications from a user and send user input signals indicating user input or indications to the processing circuitry 912.

[0163] In one or more embodiments, network node 210 may further include one or more sensors (not shown).

[0164] In an embodiment, the processing circuitry 912 is communicatively coupled to the memory 915 and / or the communication interface 904 and / or the input device 917 and / or the display 918.

[0165] In this embodiment, the communication interface and / or transceiver 904 communicates using wired and / or wireless communication technologies.

[0166] In an embodiment, one or more memories 915 may include a choice of hard disk RAM, a disk drive, a floppy disk drive, a magnetic tape drive, an optical disc drive, a CD or DVD drive (R or RW), or other removable or fixed media drives.

[0167] In another embodiment, network node 210 may further include and / or be coupled to one or more additional sensors (not shown) configured to receive and / or acquire and / or measure physical properties relating to the network node or its environment, and to send one or more sensor signals indicative of the physical properties to processing circuitry 912.

[0168] It should be understood that a network node includes any suitable combination of hardware and / or software required to perform the tasks, features, functions, and methods disclosed herein. Furthermore, while components of a network node are depicted as single boxes within larger boxes, or nested within multiple boxes, in practice, a network node may include multiple distinct physical components constituting a single illustrated component (e.g., memory 915 may include multiple separate hard disk drives and multiple RAM modules).

[0169] Similarly, network node 210 can consist of multiple physically separate components, each of which can have its own corresponding components.

[0170] The communication interface 904 may also include multiple sets of various components for different wireless technologies, such as, for example, GSM, WCDMA, LTE, NR, Wi-Fi, or Bluetooth wireless technologies. These wireless technologies can be integrated into the same or different chips or chipsets and other components within the network node 210.

[0171] Processing circuitry 912 is configured to perform any determination, calculation, or similar operation (e.g., certain acquisition operations) described herein as being provided by network 900. These operations performed by processing circuitry 912 may include processing information acquired by processing circuitry 912, for example, by converting the acquired information into other information, comparing the acquired or converted information with information stored in network nodes, and / or performing one or more operations based on the acquired or converted information, and making a decision as a result of said processing.

[0172] Processing circuitry 912 may include a combination of one or more of a microprocessor, controller, central processing unit, digital signal processor, application-specific integrated circuit, field-programmable gate array, or any other suitable computing device, resource, or hardware, software, and / or coding logic operable to provide computer 210 functionality, individually or in combination with other network node 210 components (e.g., device-readable media). For example, processing circuitry 912 may execute instructions stored in device-readable media 915 or in memory within processing circuitry 912. Such functionality may include providing any of the various wireless features, functions, or benefits discussed herein. In some embodiments, processing circuitry 912 may include a system-on-a-chip.

[0173] In some embodiments, the processing circuitry 912 may include one or more of an RF transceiver circuit and a baseband processing circuit. In some embodiments, the RF transceiver circuit and the baseband processing circuit may be on separate chips or chip sets, circuit boards, or units, such as radio units and digital units. In alternative embodiments, some or all of the RF transceiver circuit and the baseband processing circuit may be on the same chip or chip set, circuit board, or unit.

[0174] In some embodiments, some or all of the functionality described herein as being provided by network node 210 may be performed by processing circuitry 912, which executes instructions stored on device-readable medium 915 or memory within processing circuitry 912. In alternative embodiments, some or all of the functionality may be provided by processing circuitry 912 without executing instructions stored on separate or discrete device-readable media, such as in a hard-wired manner. In any of those embodiments, processing circuitry 912 may be configured to perform the described functionality regardless of whether instructions stored on device-readable storage media are executed. The benefits provided by such functionality are not limited to processing circuitry 912 alone or other components of the network node, but are enjoyed by network node 210 and / or end users.

[0175] Device-readable medium or memory 915 may include any form of volatile or non-volatile computer-readable memory, including but not limited to permanent storage devices, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (e.g., hard disk), removable storage media (e.g., flash drives, optical discs (CDs), or digital video discs (DVDs)) and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory device that stores information, data, and / or instructions usable by processing circuitry 912. Device-readable medium 915 may store any suitable instructions, data, or information, including computer programs, software, applications including one or more logic, rules, codes, tables, etc., and / or other instructions executable by processing circuitry 912 and usable by network node 210. Device-readable medium may be used to store any calculations performed by processing circuitry 912 and / or any data received via interface 904. In some embodiments, processing circuitry 912 and device-readable medium 915 may be considered integrated.

[0176] Communication interface 904 is used for wired or wireless communication of signaling and / or data between network node 210 and other nodes. Interface 904 may include one or more ports / terminals for transmitting and receiving data to and from network node 210, for example, via a wired connection. Interface 904 also includes radio front-end circuitry, which may be coupled to an antenna or, in some embodiments, is part of the antenna. The radio front-end circuitry may include filters and amplifiers. The radio front-end circuitry may be connected to the antenna and / or processing circuitry 912.

[0177] Examples of network nodes 210 include, but are not limited to, gNBs, gateways, smartphones, mobile phones, cellular phones, Voice over IP (VoIP) phones, wireless local loop phones, tablet computers, desktop computers, personal digital assistants (PDAs), wireless cameras, game consoles or devices, music storage devices, playback devices, wearable terminal devices, wireless endpoints, mobile stations, tablet computers, laptops, laptop embedded devices (LEEs), laptop mounted devices (LMEs), smart devices, wireless client devices (CPEs), vehicle-mounted wireless terminal devices, drones, etc.

[0178] The communication interface 904 may include wired and / or wireless networks, such as a local area network (LAN), a wide area network (WAN), a computer network, a wireless network, a telecommunications network, another similar network, or any combination thereof. The communication interface can be configured to include receiver and transmitter interfaces for communicating with one or more other devices over a communication network according to one or more communication protocols (e.g., Ethernet, TCP / IP, SONET, ATM, optical, electrical, etc.). The transmitter and receiver interfaces may share circuit components, software, or firmware, or alternatively, may be implemented separately.

[0179] In an embodiment, UE 220 may include all the features described with respect to FIG. 7, or a subset of the features described with respect to FIG. 7.

[0180] The following provides a more detailed background on beam prediction.

[0181] One of the key features of New Radio (NR) compared to previous generation wireless networks is its ability to operate at higher frequencies (e.g., above 10 GHz). The large transmission bandwidth available in these frequency ranges can potentially provide high data rates. However, as the carrier frequency increases, both path loss and penetration loss also increase.

[0182] To maintain a consistent level of coverage, highly directional antenna beams are needed to focus the radio transmitter's energy in a specific direction towards the receiver. However, both the receiver and transmitter sides require large radio antenna arrays to generate such highly directional beams.

[0183] To reduce hardware costs, large antenna arrays used for high frequencies employ time-domain analog beamforming. The core idea of ​​analog beamforming is to share a single radio frequency (RF) chain among many (or potentially all) antenna elements in a large antenna array. The limitation of analog beamforming is that at any given time, only one beam (in one direction) can be used to transmit radio energy.

[0184] The aforementioned constraints require the network (NW) and user equipment (UE) to perform beam management procedures to establish and maintain appropriate transmitter (Tx) / receiver (Rx) beam pairs. For example, a transmitter can use beam management procedures to scan a geographic area by transmitting reference signals on different candidate beams during non-overlapping time intervals and using a predetermined scanning pattern. The optimal transmit and receive beams can be identified by measuring the quality of these reference signals at the receiver side.

[0185] The beam management process in NR is defined by a set of L1 / L2 procedures that establish and maintain appropriate beam pairs for both transmitting and receiving data. The beam management process may include the following sub-procedures: beam determination, beam measurement, beam reporting, and beam scanning.

[0186] In the case of downlink transmission from NW to UE, for example, according to the NR SI technical report, the P1 / P2 / P3 beam management procedure can be performed to overcome the challenge of establishing and maintaining TX / RX beam pairs when, for example, the UE moves or there is some congestion in the environment that requires beam changes.

[0187] Although these scenarios are not directly mentioned in the 3GPP (3rd Generation Partnership Project) specifications, related procedures are defined to enable their implementation. Procedures P1-P3 are further described below.

[0188] P1: The P1 procedure enables the UE to perform measurements on different Transmit / Receive Point (TRP) Tx beams to support the selection of TRP Tx beams / (one or more) UE Rx beams. For example, during initial access, the gNB transmits SS / PBCH block synchronization signal block SSB beams in different directions to cover the entire cell. The UE measures the signal quality on the corresponding SSB signal to detect and select the appropriate synchronization signal block SSB beam, as shown in Figure 8.

[0189] Figure 8 shows the SSB beam selection as part of the initial access procedure according to scenario P1.

[0190] Random access is then transmitted on the RACH resource indicated by the selected SSB. Both the UE and the network will communicate using the corresponding beam until connection-mode beam management is activated. The network infers which SSB beam the UE has selected without any explicit signaling.

[0191] For beamforming at the transmit / receive point (TRP), it typically includes intra-TRP / inter-TRP Tx beam scanning from a set of different beams. For beamforming at the UE, it typically includes UE Rx beam scanning from a set of different beams.

[0192] P2: The P2 procedure enables the UE to perform measurements on different TRP Tx beams, potentially changing the intra-TRP / inter-TRP Tx beam(s). The network can use the SSB beam as an indication of which (narrow) CSI-RS beam to try; that is, the selected SSB beam can be used to define a candidate set of narrow CSI-RS beams for beam management. Once the Channel State Information Reference Symbol (CSI-RS) is transmitted, the UE measures the Reference Signal Strength Indicator (RS).

[0193] The network updates the UE's serving beam accordingly and may also modify a candidate set of CSI-RS beams. The network can also instruct the UE to perform measurements on the SSB. If the network receives a CSI-RS RP report from the UE, indicating that the new CSI-RS beam is superior to the old CSI-RS beam used for transmitting the Physical Downlink Control Channel / Physical Downlink Shared Channel (PDCCH) / PDSCH, the network will update the UE's serving beam accordingly and may also modify a candidate set of CSI-RS beams. The network can also instruct the UE to perform measurements on the SSB. If the network receives a report from the UE indicating that the new SSB beam is superior to the previously best SSB beam, it may prompt a corresponding update of the UE's candidate set of CSI-RS beams.

[0194] Compared to P1, the P2 process is performed on a potentially smaller set of beams for beam refinement. Note that P2 may be a special case of P1. For example, in connected mode, the network node (e.g., gNB) configures the UE with different CSI-RS and transmits each CSI-RS on the corresponding beam. The UE then measures the quality of each CSI-RS beam on its current RX beam and sends feedback on the measured beam quality. Based on this feedback, the gNB will then determine, and may indicate to the UE, which beam will be used for future transmissions. This is illustrated in Figure 9.

[0195] Figure 9 illustrates the CSI-RS Tx beam selection in the downlink according to the P2 scenario.

[0196] P3: Used to enable the UE to perform measurements on the same TRP Tx beam to change the UE Rx beam when the UE uses beamforming. Once in connected mode, the UE is configured with a set of reference signals. Based on the measurements, the UE determines which Rx beam is suitable for receiving each reference signal in the set. The network then indicates which reference signals are associated with the beam that will be used to transmit PDCCH / PDSCH, and the UE uses this information to adjust its Rx beam when receiving PDCCH / PDSCH.

[0197] In connected mode, the UE can use P3 to find the optimal Rx beam for the corresponding Tx beam. In this case, the network node / gNB maintains one CSI-RS Tx beam at a time, and the UE performs a scan and measurement of its own Rx beam for that specific Tx beam. The UE then finds the optimal corresponding Rx beam based on the measurement and uses it for reception when the gNB instructs it to use the Tx beam in the future.

[0198] Figure 10 shows the UE Rx beam selection for the corresponding CSI-RS Tx beam in DL according to the P3 scenario.

[0199] For beam management, beam measurement and reporting in NR allows the UE to be configured to report the RSRP and / or SINR of up to four beams on CSI-RS or SSB. UE measurement reports can be sent to network nodes, such as gNB, via PUCCH or PUSCH.

[0200] Reference Signal Configuration (CSI-RS) in NR: CSI-RS is transmitted on each transmit (Tx) antenna port at the network node and for each different antenna port. CSI-RS is multiplexed in the time, frequency, and code domains, allowing the UE to measure the channel between each Tx antenna port at the network node and each receive antenna port at the UE. The time-frequency resources used to transmit CSI-RS are called CSI-RS resources.

[0201] In NR, CSI-RS used for beam management is defined as a 1- or 2-port CSI-RS resource in a CSI-RS resource set, where field duplication exists. Three types of CSI-RS transmissions are supported: Periodic CSI-RS: CSI-RS is transmitted periodically in certain time slots. This CSI-RS transmission is semi-statically configured using RRC signaling with parameters such as CSI-RS resources, periodicity, and time slot offset.

[0202] Semi-persistent CSI-RS: Similar to periodic CSI-RS, it uses RRC signaling with parameters such as periodicity and slot offset to semi-statically configure resources for semi-persistent CSI-RS transmission. However, unlike periodic CSI-RS, dynamic signaling is required to activate and deactivate CSI-RS transmission.

[0203] Aperiodic CSI-RS: This is a one-time CSI-RS transmission that can occur in any time slot. In this document, "one-time" means that the CSI-RS transmission occurs only once per trigger. The CSI-RS resources used for aperiodic CSI-RS (i.e., RE positions consisting of subcarrier positions and OFDM symbol positions) are semi-statically configured. The transmission of aperiodic CSI-RS is triggered by dynamic signaling via the PDCCH using the CSI request field in the UL DCI, in the same DCI where UL resources for measurement reporting are scheduled. Multiple aperiodic CSI-RS resources can be included in a CSI-RS resource set, and the triggering of aperiodic CSI-RS is based on the resource set.

[0204] SSB: In NR, SSB consists of a pair of synchronization signals (SS), a physical broadcast channel (PBCH), and DMRS of the PBCH. SSB is mapped to 4 consecutive OFDM symbols in the time domain and to 240 consecutive subcarriers (20 RBs) in the frequency domain.

[0205] NR supports beamforming and beam scanning for SSB transmission, enabling a cell to transmit multiple SSBs in different narrow beams that are time-multiplexed. The transmission of these SSBs is limited to a half-frame time interval (5 ms). It is also possible to configure a cell to repeatedly transmit multiple SSBs in a single wide beam. The beamforming parameters for each SSB within a half-frame are designed based on the network implementation. SSBs within a half-frame are periodically broadcast from each cell. The period of a half-frame with SS / PBCH blocks is called the SSB period, denoted by SIB1.

[0206] The maximum number of SSBs within a half-frame, denoted by L, depends on the frequency band, and the temporal positions of these L candidate SSBs within the half-frame depend on the SSB's SCS. The L candidate SSBs within a half-frame are indexed temporally in ascending order from 0 to L-1. The UE knows the SSB index by successfully detecting the Physical Broadcast Channel (PBCH) and its associated Downlink Demodulation Reference Signal (DMRS). The cell does not need to transmit SS / PBCH blocks at all L candidate positions within a half-frame, and the resources of unused candidate positions can be used for data or control signaling transmission. The network implementation determines which candidate temporal position to select for SSB transmission within the half-frame, and which beam to use for each SSB transmission.

[0207] In NR, the UE can configure the following measurement resources: N≥1 CSI report settings (CSI-ReportConfig) and M≥1 resource settings (CSI-ResourceConfig).

[0208] Each CSI reporting setting is linked to one or more resource settings used for channel and / or interference measurements. The CSI framework is modular in the sense that several CSI reporting settings can be associated with the same resource settings.

[0209] Measurement resource configurations used for beam management are provided to the UE by RRC Information Elements (IEs) (CSI-ResourceConfigs). A CSI-ResourceConfig contains several NZP-CSI-RS-ResourceSets and / or CSI-SSB-ResourceSets.

[0210] The UE can be configured to use the RRC IE NZP-CSI-RS-ResourceSet to measure CSI-RS. The NZP CSI-RS resource set contains the configuration of Ks ≥ 1 CSI-RS resource. Each CSI-RS resource configuration resource includes at least the following: mapping to RE, number of antenna ports, and time-domain behavior.

[0211] Up to 64 CSI-RS resources can be grouped together in an NZP-CSI-RS-ResourceSet.

[0212] The UE can be configured to use the RRC IE CSI-SSB-ResourceSet to measure SSBs. The resource set that includes SSB resources is defined in a manner similar to the CSI-RS resources defined above.

[0213] In the case of non-periodic CSI-RS and / or non-periodic CSI reporting, network nodes use S c CSI trigger state configuration for the UE. Each trigger state contains the non-periodic CSI reporting settings to be triggered and the associated non-periodic CSI-RS resource set.

[0214] Periodic and semi-persistent resource settings can only include a single resource set (i.e., S=1). Aperiodic resource settings can have many resource sets (S>=1) because one of the S resource sets defined in the resource setting is an aperiodic trigger status indication reported by the triggering CSI.

[0215] The Measurement Reporting (NR) supports three types of CSI reports: Periodic CSI reports on the Physical Uplink Control Channel (PUCCH): CSI is reported periodically by the UE. Parameters such as periodicity and slot offset are semi-statically configured by higher-layer RRC signaling from the network node to the UE.

[0216] Semi-persistent CSI reporting on the physical uplink control channel or PUCCH: Similar to periodic CSI reporting, semi-persistent CSI reporting has a periodicity and slot offset that can be configured semi-statically. However, dynamic triggering from the network node to the UE may be required to allow the UE to start semi-persistent CSI reporting. Dynamic triggering from the network node to the UE is also required to request the UE to stop semi-persistent CSI reporting.

[0217] Non-periodic CSI reporting on the Physical Uplink Control Channel (PUSCH). This type of CSI reporting includes a single (i.e., one-time) CSI report from the UE, dynamically triggered by the network node using DCI. Some parameters related to the configuration of non-periodic CSI reporting are configured semi-statically by RRC, but the triggering is dynamic.

[0218] Each CSI report setting defines the report's content and time-domain behavior, as well as links to related resource settings.

[0219] The CSI-ReportConfig IE includes the following configurations: reportConfigType defines the time-domain behavior (periodic CSI reporting, semi-persistent CSI reporting, or non-periodic CSI reporting) and the reporting period and time slot offset for periodic CSI reports.

[0220] The `reportQuantity` defines the CSI parameters of the report—the CSI content; for example, PMI, CQI, RI, LI (Layer Indicator), CRI (CSI-RS Resource Index), and L1-RSRP, etc. Only certain combinations are possible; for example, "cri-RI-PMI-CQI" is one possible value, while "cri-RSRP" is another, and it can be said that each value of `reportQuantity` corresponds to a certain CSI pattern.

[0221] The codebookConfig defines the codebook used for PMI reporting, as well as possible codebook subset restrictions (CBSR). NR supports two types of PMI codebooks: Type I CSI and Type II CSI. Furthermore, Type I and Type II codebooks each have two different variants: Regular and Port Selection.

[0222] The reportFrequencyConfiguration defines the frequency granularity of PMI and CQI (wideband or subband) (if reported) and the CSI reporting band, which is a subset of the subband of the corresponding bandwidth portion (BWP) of CSI.

[0223] For beam management, separate time-domain measurement constraints (on / off) are applied for channel and interference. The UE can be configured to report L1-RSRP for up to four different CSI-RS / SSB resource indicators. The RSRP value reported corresponding to the first (best) CRI / SSBRI requires 7 bits of the absolute value, while the others require 4 bits of the encoded value relative to the first. L1-SINR reporting for beam management is supported in NR Release 16.

[0224] During the 3GPP meeting RAN1#109-e, the 3GPP protocol agreed to study spatial beam prediction based on AI / ML, the core idea of ​​which is to use measurements from another set of beams B to predict one or more “optimal” beams from a set of beams A.

[0225] Beam sets A and B are not yet defined (to be studied later); however, the following two examples illustrate some scenarios that may be studied in Release 18: set B is a subset of set A. For example, set A is a set of 8 SSB / CSI-RS beams (both bright and black circles) shown in Figure 11. The UE measures set B (the 4 beams indicated by the black circles). The AI / ML model should use only measurements from set B to predict the optimal beam(s) in set A(s).

[0226] Figure 11 shows an example where set B is a subset of set A. The figure illustrates a beam grid radiation pattern: each row (corresponding column) depicts a zenith angle (corresponding azimuth angle) from the antenna array. Set A has 8 beams, and set B has 4 beams (indicated by black circles).

[0227] Set A and set B correspond to two different sets of beams. For example, set A is a set of 30 narrow CSI-RS beams, and set B is a set of 8 wide SSB beams. The UE measures the beams in set B, and the AI / ML model should predict the optimal beam(s) from set A.

[0228] Figure 12 shows an example where set A is a set of narrow beams and set B is a set of wide beams.

[0229] Spatial beam prediction can be performed in either the gNB or the UE—the research project will cover both scenarios.

[0230] During 3GPP meeting RAN1#110, it was agreed to study AI / ML model training on both the NW and UE sides. Which side performs the training is expected to influence how data collection is performed, with another agreement being to study the data collection aspects of beam management. Furthermore, it was agreed to study aspects of model monitoring and their standard impact on AI / ML model inference (e.g., reporting of predictions).

[0231] The following is the content regarding performance monitoring in the 3GPP Research Project Technical Report TR 38.843 of the 3rd Generation Partnership Project.

[0232] Performance Monitoring: For performance monitoring of BM-Case 1 and BM-Case 2: - Performance metrics with one or more of the following alternatives: - Alternative 1: KPIs related to beam prediction accuracy, such as Top-K / 1 beam prediction accuracy - Alternative 2: KPIs related to link quality, such as throughput, L1-RSRP, L1-SINR, assumed BLER - Alternative 3: Performance metrics based on AI / ML input / output data distribution - Alternative 4: Evaluating L1-RSRP differences by comparing measured RSRP and predicted RSRP - Benchmarks / references for performance comparison, including: - Alternative 1: By measuring The best beam(s) obtained from a set of beams (e.g., beams from set A) indicated by the gNB - Alternative 4: Measurement of the predicted best beam(s) corresponding to the model output (e.g., comparison between the actual L1-RSRP and the predicted RSRP of the predicted Top-1 / K beams) - Signaling / configuration / measurement / reporting for model monitoring, e.g., signaling aspects related to auxiliary information (if supported), reference signals for BM-Case 1 and BM-Case 2 with UE-side AI / ML models: - Type 1 performance monitoring: - Configuration / signaling from gNB to UE for measurement and / or reporting - The UE can perform different operations: - Option 1: The UE sends a report to the NW (e.g., for calculating performance metrics in the NW) - Option 2: The UE calculates (one or more) performance metrics, or reports them to the NW, or reports events to the NW based on (one or more) performance metrics - Instructions from the NW for the UE to perform LCM operations - Note: At least the performance and reporting overhead of the model monitoring mechanism should be considered - Type 2 Performance Monitoring (UE-side Performance Monitoring): - Performance monitoring instructions / requests / reports from the UE to the gNB - Note: In some cases, instructions / requests / reports may not be required - Configuration / signaling from the gNB to the UE for performance monitoring measurements and / or reporting - The UE calculates (one or more) performance metrics, or reports them to the NW, or reports events to the NW based on (one or more) performance metrics - If used for UE-side model monitoring, the UE makes (one or more) decisions regarding model selection / activation / deactivation / switching / fallback operations - Instructions from the NW to the UE to perform LCM operations - The UE reports a set of beam measurements (one or more) of beams indicated by the gNB - signaling, such as RRC-based, L1-based - Note: performance and UE complexity, power consumption should be considered - a mechanism to facilitate the UE to detect whether the functionality / model is appropriate or no longer appropriate. Table 7.2.3-1 summarizes the applicability of various alternatives for the AI / ML model monitoring (one or more) of performance metrics for BM-Case 1 and BM-Case 2.

[0233] Figure 13 shows an alternative table of one or more performance metrics monitored by AI / ML models with BM-Case 1 and BM-Case 2.

[0234] The abbreviations used in this article.

[0235] 3GPP Third Generation Partner Program 5G Fifth Generation ACK Acknowledgment AI Artificial Intelligence AoA Angle of Arrival CORESET Control Resource Set CSI Channel State Information CSI-RS CSI Reference Signal DCI Downlink Control Information DoA Direction of Arrival DL Downlink DMRS Downlink Demodulation Reference Signal FDD Frequency Division Duplex FR2 Frequency Range 2HARQ Hybrid Automatic Repeat Request ID Identity gNB gNodeB MAC Media Access Control MAC-CE MAC Control Element ML Machine Learning NR New Radio NW Network OFDM Orthogonal Frequency Division Multiplexing PBCH Physical Broadcast Channel PCI Physical Element Identifier PDCCH Physical Downlink Control Channel PDSCH Physical Downlink Shared Channel PRB Physical Resource Block QCL Quasi-Co-location RB Resource Block RRC Radio Resource Control RSRP Reference Signal Strength Indicator RSRQ Reference Signal Receiver Quality RSSI Received Signal Strength Indicator SCS Subcarrier Spacing SINR Signal-to-Noise Ratio SSB Synchronization Block RL Reinforcement Learning RS Reference Signal Rx Receiver TB Transport Block (TDD), Time Division Duplex (TCI), Transport Configuration Indicator (TRP), Transmit / Receive Point (Tx), Transmitter (UE), User Equipment (UL), Uplink. Finally, it should be understood that the embodiments herein are not limited to the above embodiments, but also relate to and incorporate all embodiments within the scope of the appended independent claims.

Claims

1. A method (400) performed by a user equipment (UE) (220) in a radio network (100), the method comprising: The received configuration calculates (320, 410) a model performance metric, wherein the model performance metric reflects the performance of the beam prediction model, and transmits (340, 360, 420) a report indicating the calculated model performance metric.

2. The method according to claim 1, wherein, The reports are transmitted periodically.

3. The method according to any one of the preceding claims, wherein, The method further includes: evaluating whether the calculated model performance metric (320, 430) meets one or more criteria for the early transmission of the report, and if the calculated model performance metric meets the one or more criteria, transmitting the report (340, 440).

4. The method according to claim 3, wherein, The received configuration instruction to the UE (220) is used to calculate the model performance metric based on a first number "X" time instances, wherein the evaluation is based on the model performance metric, and the model performance metric is calculated based on a second number of time instances that is less than the first number "X" time instances.

5. The method according to any one of claims 3-4, wherein, The method also includes receiving (350, 450) updated configurations.

6. The method according to any one of claims 3-5, wherein, The method includes receiving an indication that model performance metrics are not computed in the remaining time instances of a first number "X" time instances, wherein the indication is received in one of the following: - the received configuration, - the updated configuration, or - separate from the received configuration and the updated configuration.

7. The method according to any one of claims 5-6, wherein, The received configuration and / or the updated configuration are received via downlink control information (DCI) signaling and / or via radio resource control (RRC) signaling.

8. The method according to any one of the preceding claims, wherein, The UE (220) is configured with a set of ranges of one or more model performance metrics, wherein the report indicates a specific range within the configured set of ranges, and the specific range includes the calculated performance metrics.

9. The method according to claim 8, wherein, The range in the configured set of ranges is any one of the following: - uniformly distributed between the minimum and maximum values, or - non-uniformly distributed between the minimum and maximum values.

10. The method according to any one of the preceding claims, wherein, The report is transmitted via Radio Resource Control (RRC) signaling and / or Uplink Control Information (UCI) signaling and / or Physical Uplink Shared Channel (PUSCH) signaling and / or Dedicated Scheduling Request (DSC) signaling.

11. The method according to any one of the preceding claims, wherein, The received configuration indication selects from any of the following: an indication of whether to allow early transmission of the report, a definition of a standard by which the UE can trigger early transmission of the report, an indication of time / frequency resources available for transmitting the report, a list of uniform and / or non-uniform model performance ranges, a threshold indicating the permissible prediction error of the model, a minimum number of measurement samples and / or time instances that the UE should use before evaluating the calculated model performance, and an indication of whether the UE (220) should suspend ongoing performance monitoring if the calculated model performance metric is below the threshold.

12. The method according to any one of the preceding claims, wherein, The calculated model performance metrics are based on the time distance between predictions that do not meet the evaluation criteria and / or the frequency of predictions that do not meet the evaluation criteria.

13. The method according to claim 12, wherein, The calculated model performance metrics are further classified based on the severity of predictions that do not meet the evaluation criteria.

14. The method according to any one of the preceding claims, wherein, The transmitted report includes the results of converting the calculated model performance metrics into exponential functions.

15. The method according to claim 14, wherein, The index indicates a specific range within a configured set of ranges, and the specific range includes the calculated model performance metric.

16. The method according to any one of claims 14-15, wherein, When calculating the model performance metric, the function uses the timestamps of the measurement samples to weight the measurement samples.

17. The method according to any one of the preceding claims further includes receiving a request to deactivate the model from the network node (210) in response to the transmitted report being an earlier transmitted report.

18. A user equipment (UE) in a radio network (100), the user equipment comprising: A processor, and a memory containing instructions executable by the processor, thereby enabling the user equipment to perform the method according to any one of claims 1-17.

19. A method (500) performed by a network node (210) in a radio network (100), the method comprising: Generate (310, 510) a configuration for antenna beam prediction performance monitoring, and transmit (315, 520) a signal including the configuration to the UE (220) (S1).

20. The method of claim 19, further comprising receiving (340, 360) a report indicating a model performance metric of the beam prediction model.

21. The method of claim 20, wherein, The transmitted configuration instruction is an instruction to the UE (220) to calculate a model performance metric based on a first number "X" time instances, and wherein the report is an earlier transmitted report, wherein the model performance metric of the beam prediction model is based on a second number of time instances less than the first number "X" time instances.

22. The method according to claim 21, wherein, The method includes sending an indication that performance metrics are not computed in the remaining time instances of a first number "X" time instances, wherein the indication is sent in any of the following ways: - the transmitted configuration, - the updated configuration, or - separate from the transmitted configuration and the updated configuration.

23. The method according to any one of claims 20-22, wherein, The configuration indicates a set of ranges for one or more performance metrics, wherein the report indicates a specific range within the configured set of ranges, and the specific range includes the calculated performance metrics.

24. The method of claim 23, wherein, The range is either: - uniformly distributed between the minimum and maximum values, or - non-uniformly distributed between the minimum and maximum values.

25. The method according to any one of claims 20-24, wherein, The report includes an index indicating a specific range within a configured set of ranges, and the specific range includes the model performance metric.

26. The method according to any one of claims 19-25, wherein, The method also includes transmitting (350) updated configuration.

27. The method according to any one of claims 19-26, wherein, The transmitted configuration and / or the updated configuration are transmitted via downlink control information (DCI) signaling and / or via radio resource control (RRC) signaling.

28. The method according to any one of claims 19-27, wherein, The report is received via Radio Resource Control (RRC) signaling and / or Uplink Control Information (UCI) signaling and / or Physical Uplink Shared Channel (PUSCH) signaling and / or Dedicated Scheduling Request (DSC) signaling.

29. The method according to any one of claims 19-28, wherein, The transmitted configuration indication selects from any of the following: an indication of whether early transmission of the report is allowed, a definition of a standard that the UE can trigger early transmission of the report, an indication of time / frequency resources that can be used to transmit the report, a list of uniform and / or non-uniform model performance ranges, a threshold indicating the allowable prediction error of the model, a minimum number of measurement samples and / or time instances that the UE should use before evaluating the calculated model performance, and an indication of whether the UE (220) should suspend ongoing performance monitoring if the calculated model performance metric is below the threshold.

30. The method according to any one of claims 20-29, further comprising requesting the UE (220) to deactivate the model in response to the received report being an earlier transmitted report.

31. A node (210) in a radio network (100), the node comprising: A processor, and a memory containing instructions executable by the processor, thereby enabling the node to perform the method according to any one of claims 19-30.

Images