Beam management
By acquiring error-free and error-affected reference true values, the impact of hardware errors is mitigated, thus solving the accuracy and efficiency problems of beam management in wireless communication networks using machine learning models and achieving more accurate beam prediction.
Patent Information
- Application Number
- CN202610629967.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2025-05-09
- Filing Date
- 2026-05-09
- Publication Date
- 2026-11-10
AI Technical Summary
Existing machine learning models used for beam management in wireless communication networks are limited by hardware measurement errors, leading to inaccurate performance evaluation and affecting the accuracy and efficiency of beam prediction.
Before evaluating the performance of a machine learning model, we obtain both error-free and measurement-error-affected reference ground truth measurements. We then use high signal-to-noise ratio conditions and multiple measurement averaging techniques to mitigate hardware errors, assess the impact of measurement errors, and determine the model performance based on this assessment.
This improves the accuracy and efficiency of beam management in wireless communication networks using machine learning models, reduces the impact of measurement errors on performance evaluation, and ensures the accuracy and reliability of beam prediction.
Smart Images

Figure CN123074543A_ABST
Abstract
Description
Technical Field
[0001] Various example implementations relate to beam management in wireless communication networks. Background Technology
[0002] Machine learning (ML) models are used to predict the optimal beams to use between entities in a wireless communication network. Such ML models can be trained based on signal strength data from the beams. The accuracy of these ML models' predictions can be evaluated to ensure accurate beam prediction. This evaluation can be based on ground truth. Summary of the Invention
[0003] The scope of protection sought by the various exemplary embodiments of the present invention is set forth in the independent claims. Exemplary embodiments and features (if any) described in this specification that do not fall within the scope of the independent claims are to be construed as examples useful for understanding the various embodiments of the invention.
[0004] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: at least one processor; and at least one memory storing instructions, said instructions causing the apparatus to perform at least the following when executed by said at least one processor: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; Send a report of the additional reference truth value measurement to the second device; A machine learning model is used to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; Indicators for determining the quality of the performance of the machine learning model based on at least one predicted beam and at least one of the following: The error-free reference true value measurement; and The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model is sent to the second device.
[0005] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: at least one processor; and at least one memory storing instructions, said instructions causing the apparatus to perform at least the following when executed by said at least one processor: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report of the additional reference truth value measurement from the second device; Receive from the second device an indicator of the quality of the performance of a machine learning model, the machine learning model being configured to identify at least one beam for communication between the device and the second device; and Whether the measurement error affecting the additional reference truth measurement meets the first criterion is determined at least in part based on the error-free reference truth measurement.
[0006] According to various, but not necessarily all, exemplary embodiments of the present invention, a method performed by an apparatus is provided, comprising: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; Send a report of the additional reference truth value measurement to the second device; A machine learning model is used to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; Indicators for determining the quality of the performance of the machine learning model based on at least one predicted beam and at least one of the following: The error-free reference true value measurement; and The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model is sent to the second device.
[0007] According to various, but not necessarily all, exemplary embodiments of the present invention, a method performed by an apparatus is provided, comprising: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report of the additional reference truth value measurement from the second device; Receive from the second device an indicator of the quality of the performance of a machine learning model, the machine learning model being configured to identify at least one beam for communication between the device and the second device; and Whether the measurement error affecting the additional reference truth measurement meets the first criterion is determined at least in part based on the error-free reference truth measurement.
[0008] Further specific and preferred aspects are set forth in the appended independent and dependent claims. Features of the dependent claims may be suitably combined with features of the independent claims, and in combinations other than those expressly set forth in the claims.
[0009] When a device feature is described as operable to provide a function, it should be understood that this includes device features that provide the function or are adapted or configured to provide the function. Attached Figure Description
[0010] Some exemplary embodiments will now be described with reference to the accompanying drawings, in which: Figure 1 The steps of beam thinning are shown; Figure 2A The signaling diagram for using an ML model to predict beams at the UE is shown; Figure 2B The signaling diagram shows the use of ML models to predict beams at network entities; Figure 3 The cumulative distribution function (CDF) of measurement errors in various scenarios is shown; Figure 4 A signaling diagram according to an embodiment is shown; Figure 5 A signaling diagram according to an embodiment is shown; Figure 6 A signaling diagram according to an embodiment is shown; Figure 7 The CDF of the measurement error for a varying baseband error model is shown. Figure 8 A signaling diagram according to an embodiment is shown; Figure 9The steps in the method according to the embodiment are shown; Figure 10 The steps in the method according to the embodiment are shown; Figure 11 The steps in the method according to the embodiment are shown; Figure 12 The steps in the method according to the embodiment are shown; Figure 13 An apparatus according to an embodiment is shown. Detailed Implementation
[0011] Before discussing the example embodiments in more detail, an overview will first be provided.
[0012] Machine learning (ML) model inference used to predict beams for communication between a device such as a user equipment and a second device such as a test device or network entity can be evaluated to ensure satisfactory performance of the ML model. In some cases, the reference ground truth measurement used to evaluate the performance of the machine learning model may contain measurement errors, such as errors introduced by the device's hardware (e.g., radio frequency (RF) or baseband (BB) hardware). However, if the error in the reference ground truth measurement is large, the reference ground truth measurement may be unreliable for evaluating the performance of the ML model.
[0013] The embodiment proposes assessing whether measurement errors affecting the reference truth measurement are acceptable before using it to evaluate the performance of the ML model. The impact of the measurement error is determined by performing an error-free first reference truth measurement and a second / additional reference truth measurement affected by the measurement error. The reference truth measurement may be a signal strength measurement of a beam set used for communication between a first and a second device. It should be understood that measuring the signal strength of the beams may include measuring the signal strength of a reference signal transmitted using the beams. The beams may be transmitted periodically from the second device to the first device.
[0014] Obtaining error-free reference truth measurements involves mitigating errors introduced by the device's hardware. Mitigation can be achieved using any suitable technique. Such techniques include performing measurements in high signal-to-noise ratio (SNR) environments such that the measurement introduced by the device's hardware is less significant to the measurement. This can be achieved by transmitting the beam set at maximum transmission power. Another error mitigation technique is to measure the beam set over multiple reception opportunities and take the average of the measurements.
[0015] Obtaining an additional reference true value measurement without measurement error reduction (or at least with a different degree of error reduction than obtaining an error-free reference true value measurement).
[0016] Compared to error-free reference truth measurements, additional reference truth measurements can be obtained from different reception timings of the first beam set. Specifically, the first measurement error mitigation method (using high SNR) may require separate receptions, but the second measurement error mitigation method (averaging) may or may not use overlapping reception timings.
[0017] Error-affected reference truth measurements can be compared with error-free reference truth measurements to assess the impact of hardware-introduced errors on the reference truth. Criteria can be used to determine whether the impact is significant enough that the performance of an ML model should not be evaluated based on error-affected reference truth measurements. If the error is tolerable, error-affected reference truth measurements can be used to evaluate the performance of the ML model. In some cases, measurement errors in the reference truth do not need to be considered. Therefore, error-free reference truth measurements can be used to evaluate the performance of the ML model.
[0018] According to the NR standard, in the traditional beam management (BM) process, the gNodeB (gNB) configures the User Equipment (UE) to measure a set of Reference Signals (RS), such as Synchronization Signal Blocks (SSBs) and Channel State Information (CSI) RS. The gNB uses unique RSs to transmit downlink (DL) transmit (TX) beams, and the UE uses the receive (RX) beams to measure the Reference Signal Received Power (RSRP) of each Tx beam and reports the set of strongest RS identifiers and corresponding RSRP measurements to the gNB. Current CSI reporting can be configured with high periodicity in a periodic / semi-persistent manner to obtain a complete and accurate picture of UE conditions and optimize, for example, beam switching. The traditional beam alignment process finds the optimal transmit and receive beam pairs for the connected UEs, i.e., Tx-Rx beam pairs, which maximize channel gain by performing P1, P2, and P3 processes, such as... Figure 1 As shown, it illustrates the conventional beam alignment process.
[0019] In the initial beam pair establishment process (P1), the gNB transmits SSB resources using a beam codebook, referred to as the SSB beam, and the UE uses the RX beam to measure the RSRP of each SSB beam and reports the set of strongest RS identifiers and the corresponding RSRP measurements. The TX beam refinement process (P2) finds the optimal CSI-RS beam by transmitting a CSI-RS beam that is more directional than the SSB beam and provides higher received signal strength. In the RX beam refinement process (P3), the gNB transmits the same TX beam at several time instances, and the UE uses different RX beams to measure the TX beam to determine the optimal RX beam for data transmission.
[0020] However, the large number of beams included in the TX and RX codebooks necessitates performing numerous measurements to determine the optimal TX-RX beam pair. Therefore, the traditional beamforming (BM) process becomes time-inefficient and non-scalable as the antenna array size increases. Consequently, artificial intelligence (AI) and machine learning (ML) algorithms can replace sequential beam scanning by recommending a reduced set of beams that may contain the optimal beam index for a full beam scan.
[0021] The Rel-18 study on “Artificial Intelligence (AI) / Machine Learning (ML) for the NR Air Interface (FS_NR_AIML_Air)” was completed at the RAN4#109 meeting in December 2023, and the results, including consensus and open questions, are documented in TR 38.843 (Release 18.0.0). 3GPP in Release 18 has studied support for a new AI / ML-enabled radio interface for next-generation cellular systems. BM is one of the different pilot use cases for applying AI / ML models in the NR radio interface. Different pilot use cases will also identify a public AI / ML framework, including the functional requirements of the AI / ML architecture, which can be used in subsequent releases and will pave the way for the design of native AI / ML 6G networks. The motivation behind this use case study stems from the growing need for efficient beam prediction schemes in wireless communication systems. The Release 18 Study Project (SI) aims to demonstrate the effectiveness of AI / ML-based BM models in reducing measurement overhead (closely related to UE power consumption) while maintaining end-to-end performance comparable to conventional BM procedures.
[0022] Furthermore, in December 2023, a new version-19 work item (WI) on "Artificial Intelligence (AI) / Machine Learning (ML) for the NR Air Interface (NR_AIML_Air)" was approved at RAN#102 to begin specification work on a general AI / ML framework for the air interface and to enable recommended use cases from previous studies. This Rel-19 work item (WI description RP-234039) is a continuation of Rel-18 research item FS_NR_AIML_Air (RP-213599).
[0023] Version 19's AI / ML-based beam management use cases aim to perform spatial and / or temporal beam prediction to reduce overhead and reporting latency. Spatial domain beam prediction (BM-Case 1) aims to predict the best DL TX beam or best DL TX-RX beam pair for different spatial locations, i.e., the top 1 or top K beams / beam pairs. Conversely, temporal domain beam prediction (BM-Case 2) aims to predict the best DL TX beam or best DL TX-RX beam pair to be used for future moments. It has been demonstrated that ML algorithms enable prediction of serving beams for different UE location and temporal instances, thereby avoiding the need to measure actual beam quality and saving resources used for data transmission and for increasing the length of discontinuous reception (DRX) for the UE.
[0024] The AI / ML models discussed in BM-Case 1 and BM-Case 2 can be trained and deployed at either the gNB (NW-side model) or the UE (UE-side model). Furthermore, SI considers aspects related to inference, monitoring, and data collection, which are described in Clause 7.1.3 of TR 38.843. Figures 2a and 2b illustrate the operation of the AI / ML-based beam management framework; Figure 2a shows the UE-side AI / ML mode, and Figure 2b shows the NW-side AI / ML mode operation. Note that sets A and B are defined according to TR 38.843.
[0025] During the AI / ML model inference phase, it is assumed that the UE measures each set B beam and provides the RSRP measurement of set B beams to the UE-side AI / ML model to predict the strongest (Top-1 or Top-K) beam from set A. However, in reality, due to impairments in the UE's radio frequency (RF) and baseband (BB) hardware, the UE measurements are subject to error. Such error can be referred to as measurement error.
[0026] The examples are designed to evaluate the impact of RF and / or BB errors on the performance and predictions of AI / ML models. In particular, the impact of measurement errors in the inputs and outputs of AI / ML models can be considered, and performance metrics or key performance indicators (KPIs) can be determined to assess the impact of measurement errors on model performance.
[0027] There are four different scenarios for evaluating the impact of measurement error on AI / ML models, as summarized in Table 1 below.
[0028] Table 1: Assessment of the impact of measurement errors on AI / ML models.
[0029]
[0030] Case 1 provides a clear indication of the performance of the AI / ML model in the absence of measurement error, while Case 2a covers the impact of measurement error in the input of the AI / ML model. Therefore, a comparison of Case 1 and Case 2a provides an indication of the impact of measurement error.
[0031] Cases 2b and 3 introduce the effects of measurement error into the reference truth measurement used to evaluate the performance of the ML model. Therefore, there is a problem that the reference truth measurement is no longer an ideal reference truth. Consequently, the testability of cases 2b and 3 becomes challenging. Figure 3 Performance comparisons are shown for different cases where the error in the reference truth is not considered, demonstrating the impact of measurement error on exhaustive beam search with and without measurement error, in addition to the evaluations of cases 1, 2a, and 3 (where measurement error in the reference truth is not considered). The RSRP error is the difference between the L1-RSRP value between the best predicted / selected beam and the best reference truth beam.
[0032] according to Figure 3 Compared to an exhaustive search with no error in the reference ground truth, an exhaustive search with an error in the reference ground truth (affected by the error) shows a 5 dB L1-RSRP difference at the 95th percentile. Therefore, the evaluation in cases 2b and 3, which take into account the measurement error at the reference ground truth, becomes challenging because the reference ground truth may already have a 5 dB L1-RSRP error in addition to the prediction error that may be introduced by the AI / ML model.
[0033] Based on the agreement reached at RAN WG4 meeting #114-bis, it is reasonable to assume that different test modes are possible. One mode considers a reference truth value without measurement error (similar to cases 1 and 2a), while another mode considers a reference truth value with measurement error (similar to cases 2b and 3). The challenge lies in distinguishing which assumptions the UE uses during testing from the perspective of the network / test equipment (NW / TE), as this can significantly impact requirements. Therefore, the implementation considers the following unresolved issues: Question 1: How will NW / TE determine the impact of measurement error on the reference true value and whether the reference true value meets traditional requirements?
[0034] Question 2: Considering that the impact of measurement error on the reference true value is reasonable, how can we ensure prediction accuracy in the presence of measurement error?
[0035] Therefore, a method is needed to minimize the impact of measurement errors on the true reference value during the testability process.
[0036] Before assessing the impact of measurement errors on AI / ML model functionality through the following steps, a test framework for the UE and test equipment (TE) or network (NW) can allow testing of different scenarios while verifying whether the impact of measurement errors on the reference truth is acceptable: Step 1: Configure the TE / NW for AI / ML-based model inference and for reference truth extraction and reporting.
[0037] Step 2: The UE acquires error-free reference truth L1-RSRP measurements for beams in set A and / or set B using at least one of the following methods and reports the reference truth values to the TE. Note that set A and set B may be referred to as the first beam set and the second beam set, respectively. Each set contains one or more beams. Beams in set A can be used to obtain the reference truth measurements, and beams in set B can be used to obtain the input to the ML model.
[0038] Method 1: Obtain the reference true value via L1-RSRP measurement under high SNR conditions.
[0039] Method 2: Obtain the reference true value for each beam as the average / sample mean of N L1-RSRP measurements, without changing other relevant settings for the N measurements at the beam and TE / NW.
[0040] Method 3: Obtain the reference true value by combining Method 1 and Method 2, wherein N L1-RSRP measurements are obtained for each A-beam set under high SNR conditions, and then the average RSRP is calculated, which becomes the reference true value.
[0041] After step 2, the L1-RSRP measurement, which is "close" to the error-free reference truth, is known at both the UE and TE / NW.
[0042] Step 3: The TE / NW transmits beam sets A and / or B, and the UE performs L1-RSRP measurements affected by measurement errors.
[0043] Step 4 is only valid for cases 2b and 3 to assess the impact of measurement error on the true reference value.
[0044] Step 4: The UE reports the L1-RSRP measurement of set A beam to the TE / NW, and the TE / NW uses KPI to compare the L1-RSRP value measured and reported in Step 3 with the error-free reference true value L1-RSRP measurement obtained in Step 2.
[0045] KPI can be the L1-RSRP difference between the best beam in the error-free reference true value and the best beam in the measurement reference true value affected by measurement error.
[0046] After step 4, TE / NW determines whether the impact of the measurement error is acceptable for cases 2b and 3. If acceptable, proceed to the next step.
[0047] Step 5: Use the AI / ML model input from set B for inference: For case 1: Error-free L1-RSRP measurements of the B-beam array can be obtained as described in step 2.
[0048] For cases 2a / 2b / 3: L1-RSRP measurements of the B-beam set affected by measurement errors can be obtained as described in step 3.
[0049] Step 6: UE uses the following reference truth values to calculate KPIs based on AI / ML model predictions: For case 1 / 2a: the error-free reference true value L1-RSRP measurement of the set A beam obtained in step 2.
[0050] For case 2b / 3: L1-RSRP measurement of the A-beam set affected by measurement error obtained in step 3.
[0051] Step 7: The UE reports the KPIs to the NW / TE. If the KPIs meet the requirements, for example, if the L1-RSRP difference between the best predicted beam and the best measured beam is within the range of (-Y dB, +Y dB), then the performance of the AI / ML model with measurement errors is acceptable. Otherwise, the performance of the AI / ML model is unacceptable.
[0052] The above process can be covered in two alternative embodiments: 1. A two-stage operation in which the UE reports the affected reference truth value and the affected KPI to the TE / NW.
[0053] 2. Single-stage operation where the UE reports the affected reference truth value and the affected KPIs to the TE / NW.
[0054] Two-stage operation Figure 4 The signaling diagram for an embodiment of KPI calculation on the UE side is shown. Details are described below: Step 1: Configure the UE in the TE / NW configuration for AI / ML model inference and for reference truth extraction and reporting, in order to test model performance in the presence of measurement errors.
[0055] Step 2: TE / NW configures two types of operating modes based on the situation of measuring error-related tests.
[0056] For cases 1 and 2a, only operation mode 2 (MoO2) is configured because a reference truth value with error is not required.
[0057] For cases 2b and 3, both operation modes 1 (MoO1) and MoO2 are configured, where MoO1 evaluates the impact on the reference true value, while MoO2 evaluates the impact of measurement error on the AI / ML model in the presence of measurement error.
[0058] Step 3: Use method 1, 2, or 3 (see description below) Figure 5 and 6 Extract error-free reference true value L1-RSRP measurement.
[0059] Step 4 / 5: The TE / NW sends the selected RS corresponding to set A and / or set B, and the UE performs a measurement including the effects of measurement error.
[0060] Step 7: After performing the measurement (assuming the measurement includes the effects of error), the UE reports the L1-RSRP measurement of the A-beam set to the TE / NW in MoO1 to assess the impact of the measurement error on the reference true value.
[0061] KPI-based: RSRP error assessment of the impact of measurement error on the reference true value: Step 8: Select the optimal beam based on the L1-RSRP measurement, including measurement error. Assume the L1-RSRP of the optimal measurement beam is R2.
[0062] Step 9: Calculate the effect of measurement error on the reference true value as R3 = R2 - R1.
[0063] Step 10: If R3 is within the range of [-X dB, +X dB], then the effect of the measurement error on the true reference value is acceptable.
[0064] If TE / NW needs to consider the negligible measurement effects of the reference truth, then X can be a small value such as 1.
[0065] If TE / NW requires the reference true value measurement to follow the conventional requirements for measurement accuracy [specified in 138.133], then X can be ±6.5.
[0066] Note that any other KPIs agreed upon in RAN1 / RAN4 can be used to assess the impact of measurement error on the reference true value. If the impact on the reference true value is acceptable for Case 2b / Case 3, then MoO2 is triggered.
[0067] Step 11 / 12: Optional request for MoO2 from TE / NW to UE.
[0068] Step 13: Considerations for AI / ML model inference: For case 1: Error-free L1-RSRP measurement of the B-beam set obtained in step 3.
[0069] For cases 2a / 2b / 3: L1-RSRP measurement of the set B beam affected by measurement error obtained in step 5.
[0070] Step 14: Calculate the KPI based on the reference truth value, which may or may not have an error, depending on the definition of the situation.
[0071] Step 15 / 16: Report the calculated KPIs to TE / NW for evaluation according to the specified requirements.
[0072] For example, if the KPI is the RSRP error (Re) and if Re is within the range (-Y dB, +Y dB), then the performance of the AI / ML model is acceptable in the presence of measurement error, otherwise it is unacceptable.
[0073] In case 2a, Y can be a larger value than X, for example, ±8.5. In cases 2b or 3, Y can be a smaller value than X, for example, ±2.5.
[0074] Let's reconsider the three methods mentioned in step 3: Method 1: Obtain error-free L1-RSRP measurements under high SNR conditions.
[0075] Figure 5 The processing flow for extracting error-free reference truth values under high SNR conditions is shown.
[0076] Typically, NW / TE sets the power level of the TX beam to achieve the desired SNR (e.g., -3dB) at the receiver. However, under low SNR conditions, the effect of measurement error is dominant due to the low signal strength. Therefore, measuring L1-RSRP under high SNR conditions reduces the impact of measurement error on L1-RSRP measurements.
[0077] The TE / NW can increase the power of all TX beams by a factor of P. The value of P can be determined such that maximum transmit power can be achieved by introducing P. Therefore, it can be assumed that all TX beams transmit with the same power boost, and the UE measures and reports the L1-RSRP measurement back to the TE / NW. Next, the TE / NW removes the effect of the power boost by reducing all measurements by P to obtain the corresponding L1-RSRP measurement without power boost, which becomes the true reference value.
[0078] Method 2: Without changing the beam and other related settings at TE / NW, obtain an error-free L1-RSRP measurement by averaging the measurements of N beams.
[0079] Figure 6The process flow for extracting error-free reference truth values using averaged L1-RSRP measurements is shown.
[0080] First, obtain multiple L1-RSRP reports corresponding to the same settings / configurations at the TE / NW (fixed test signal transmission, channel emulator conditions, test probe power level, UE orientation, etc.). Next, take the average of all RSRP values.
[0081] For example, suppose that for a given (fixed) test setting, the error-free L1-RSRP is R_a dB. Therefore, collecting N RSRP reports for this setting (configuration #1) yields the following expression:
[0082] Where R(t) is the measured RSRP (in dB) affected by measurement error, R_a is the error-free reference truth, and n(t) models the measurement error in the tth measurement instance. Now, taking the sample mean, we obtain the following expression:
[0083]
[0084] Since the measurement error is modeled as a real-valued Gaussian independent and identically distributed (iid) random variable, a good approximation of the error-free L1-RSRP reference truth can be obtained as N increases.
[0085] Therefore, confidence in the accuracy of the reference truth L1-RSRP can be increased, but at the cost of calculating an average from multiple L1-RSRP reports.
[0086] Method 3: A combination of Method 1 and Method 2.
[0087] This method involves performing RSRP measurements using a power ramp and under high SNR conditions, as described in Method 1, before the UE averages N samples and reports these samples to the TE / NW.
[0088] Figure 7 The combined effect of Method 1 and Method 2 is shown. The Link-Level Simulation (LLS) (samples = 3, SINR = -4 dB) considers N = 3 to average at SINR = -4 dB. However, considering the LLS (samples = 5, SINR = 0 dB), the BB error distribution exhibits a smaller change as N increases to 5 and SINR increases to 0 dB. It is assumed that the error variation will narrow as N and SINR increase, and that it will become a good estimate of the error-free case (black solid line).
[0089] Single-stage operation Figure 8 The signaling diagram illustrates an embodiment of KPI calculation on the UE side. This is related to two-phase operations (such as...). Figure 4 The different details are described below: Step 2: TE / NW configures UE to report L1-RSRP measurements and KPIs together.
[0090] Step 7: (Only for cases 2b and 3) The UE reports the L1-RSRP measurement affected by the measurement error and the calculated KPI to the TE / NW.
[0091] Step 10: Report the KPIs predicted by the AI / ML model.
[0092] Step 11 / 12: (Only for cases 2b and 3) Evaluate the impact of measurement error on the measurement of the reference true value L1-RSRP.
[0093] Step 13: Evaluate whether both the requirements for the reference truth L1-RSRP and KPI are met simultaneously in order to determine whether the performance of the AI / ML model is acceptable in the presence of measurement errors.
[0094] The advantage is that, in addition to evaluations based on AI / ML KPIs, it can also evaluate traditional measurement error requirements.
[0095] Figure 9 The steps in a method according to an embodiment are shown. This method can be performed by a device such as a user equipment.
[0096] In step S1, the method includes measuring the signal strength of a first beam set from a second device to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam.
[0097] In step S2, the method includes sending a report of an error-free reference true value measurement to a second device.
[0098] In step S3, the method includes measuring the signal strength of a first beam set from a second device to obtain an additional reference true value measurement affected by measurement error.
[0099] In step S4, the method includes sending a report of an additional reference true value measurement affected by the measurement error to the second device.
[0100] Figure 10 The steps of a method according to an embodiment are shown. This method can be performed by means such as a network entity or testing equipment.
[0101] In step S10, the method includes sending a first beam set including at least one beam to a second device, the first beam set being used to obtain an error-free reference true value measurement.
[0102] In step S20, the method includes receiving a report of an error-free reference truth measurement from the second device.
[0103] In step S30, the method includes sending a first beam set to a second device, the first beam set being used to obtain an additional reference true value measurement affected by measurement error.
[0104] In step S40, the method includes receiving a report from the second device of an additional reference true value measurement affected by measurement error.
[0105] In step S50, the method includes determining, at least in part, whether the measurement error affecting the additional reference truth measurement meets the criteria based on an error-free reference truth measurement.
[0106] Figure 11 Steps in a method according to another embodiment are shown. This method can be performed by a device such as a user equipment.
[0107] In step S15, the method includes measuring the signal strength of a first beam set from a second device to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam.
[0108] In step S25, the method includes sending a report of an error-free reference true value measurement to the second device.
[0109] In step S35, the method includes measuring the signal strength of a first beam set from a second device to obtain an additional reference true value measurement affected by measurement error.
[0110] In step S45, the method includes sending a report of an additional reference truth measurement to the second device.
[0111] In step S55, the method includes using a machine learning model to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device.
[0112] In step S65, the method includes determining an indicator of the quality of the performance of a machine learning model based on at least one predicted beam and at least one of the following: an error-free reference truth measurement; and an additional reference truth measurement affected by measurement errors.
[0113] In step S75, the method includes sending an indicator of the performance quality of the machine learning model to a second device.
[0114] Figure 12 Steps in a method according to another embodiment are shown. This method can be performed by means such as a network entity or testing equipment.
[0115] In step S105, the method includes sending a first beam set, including at least one beam, to a second device to obtain an error-free reference true value measurement.
[0116] In step S205, the method includes receiving a report of an error-free reference truth measurement from a second device.
[0117] In step S305, the method includes sending the first beam set to a second device to obtain an additional reference true value measurement affected by measurement error.
[0118] In step S405, the method includes receiving a report of an additional reference truth value measurement from the second device.
[0119] In step S505, the method includes receiving an indicator of the quality of the performance of a machine learning model from the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device.
[0120] In step S605, the method includes determining, at least in part, whether a measurement error affecting an additional reference truth measurement meets a first criterion based on an error-free reference truth measurement.
[0121] Figure 13 An exemplary system comprising an apparatus according to an exemplary embodiment of the invention in a wireless communication network is illustrated. The apparatus can be configured to perform methods and actions according to the exemplary embodiments disclosed herein. The system includes a user equipment 410 and a network entity 420 including a base station. The user equipment 410 may be within a geographic cell provided and controlled by the network entity 420, such that the network entity 420 may transmit signals to the user equipment 410 and vice versa. It should be understood that the network entity 420 may alternatively be a test device.
[0122] User equipment 410 includes at least one memory 412 that can store instructions executable by at least one processor 411 of user equipment 410. User equipment 410 may include a transceiver 413 for transmitting and receiving radio signals. User equipment refers to any mobile or terminal device capable of wireless communication. By way of example and not limitation, user equipment UE may also be referred to as communication equipment, terminal equipment, or mobile station (MS). User equipment may include, but is not limited to, mobile phones, cellular phones, smartphones, tablets, wearable terminal devices, personal digital assistants (PDAs), in-vehicle wireless terminal devices, smart devices, etc. Network entity 420 includes at least one memory 422 that can store instructions executable by at least one processor 421 of network entity 420. Network entity 420 may include a transceiver 423 for transmitting and receiving radio signals. Network entity may refer to at least one of the following non-limiting examples: access node, base station, gNodeB, and eNodeB.
[0123] It should be understood that user equipment 410 and network entity 420 may include any other suitable means or circuitry configured to perform steps of the methods and actions according to the example embodiments disclosed herein.
[0124] Other aspects and exemplary embodiments of the invention are described below.
[0125] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: at least one processor; and at least one memory storing instructions, said instructions causing the apparatus to perform at least the following when executed by said at least one processor: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; A report on the additional reference true value measurement affected by the measurement error is sent to the second device.
[0126] Additional reference truth measurements can be used to evaluate the performance of a machine learning model configured to identify at least one beam used for communication between the device and a second device.
[0127] The second device can use error-free reference truth measurements to evaluate the quality of the additional reference truth measurements used for this purpose. In other words, the impact of measurement errors can be assessed, which helps determine whether the additional reference truth measurements have acceptable quality for evaluating the performance of machine learning models.
[0128] In some example embodiments, the device is further made to perform: Receive instructions for using a machine learning model to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; In response to receiving the instruction, the machine learning model is used to predict at least one beam for communication between the device and the second device.
[0129] In this way, ML model inference is used to predict one or more beams for communication between the device and the second device only if the criteria for evaluating the additional reference truth measurement are met, that is, when the measurement error in the additional reference truth measurement is tolerable, such that the additional reference truth measurement can still be used to evaluate the quality of the inference of the ML model.
[0130] In some example embodiments, the received instruction indicates that the measurement error affecting the additional reference truth measurement meets at least in part the criteria based on the error-free reference truth measurement.
[0131] ML models can take signal strength measurements as input. ML inference can use the input to predict one or more beams used for communication between a first device and a second device.
[0132] Error-free measurements of the second beamset can be performed in a manner similar to error-free reference true value measurements, i.e., including measurement error mitigation. In contrast, additional measurements of the second beamset can be performed in a manner similar to additional reference true value measurements, i.e., without measurement error mitigation, and therefore the measurements are affected by measurement errors. Measurement errors can be errors introduced by the device's hardware.
[0133] In some example embodiments, the second beam set is a subset of the first beam set. It should be understood that the first and second beam sets can be subsets or can include different beams. For example, both the first and second sets can be CSI RS beams. Alternatively, the first set can be a CSI RS beam, while the second set can be an SSB beam. When the second beam set is a subset of the first beam set, the corresponding measurements and additional reference truth measurements performed to obtain error-free results can also be used as input to the ML model.
[0134] In some example embodiments, the device is also caused to perform at least one of the following: Measuring the signal strength of a second beam set from the second device to obtain an error-free measurement for input into the machine learning model, wherein the second beam set includes at least one beam, and using the machine learning model to predict the at least one beam includes inputting the error-free measurement into the machine learning model; and Measuring the signal strength of the second beam set from the second device to obtain additional measurements affected by measurement errors for input into the machine learning model, wherein using the machine learning model to predict the at least one beam includes inputting the additional measurements affected by the measurement errors into the machine learning model.
[0135] In some example embodiments, the device is further made to perform: Indicators for determining the quality of the performance of the machine learning model based on at least one predicted beam and at least one of the following: The error-free reference true value measurement; and The additional reference truth value measurement.
[0136] In some example embodiments, indicators of the performance quality of the machine learning model include at least one of the following: Absolute reference signal received power error; Relative reference signal received power error; Pre-K / N accuracy; and Pre-K / N accuracy with margin.
[0137] In some example embodiments, the device is further made to perform: The indicator of the quality of the performance of the machine learning model is sent to the second device.
[0138] In some example embodiments, the device is further made to perform: When no instruction is received to predict at least one beam for communication between the device and the second device, the machine learning model is avoided from being used to predict at least one beam for communication between the device and the second device.
[0139] In some embodiments, the device is further configured to: receive configuration information from a second device, the configuration information including an indication for the device to obtain at least one of the following: an error-free reference truth measurement; and an additional reference truth measurement. The device may obtain the error-free reference truth measurement and the additional reference truth measurement in response to receiving the corresponding indication.
[0140] In some example embodiments, the device is further made to perform: The device receives configuration information from the second device, the configuration information including an indication as to whether the device is configured to report to the second device the additional reference true value measurement affected by the measurement error. If the device is configured to report the additional reference truth value measurement to the second device, then the device performs the instruction to send the additional reference truth value measurement to the second device; and If the device is not configured to report the additional reference truth value measurement to the second device, then the device avoids sending the instruction for the additional reference truth value measurement to the second device.
[0141] In some example embodiments, obtaining an error-free reference truth measurement includes mitigating measurement errors, wherein mitigation includes at least one of the following: The first beam set has increased power compared to the conventional power of the first beam set; and The signal strength of multiple beams in the first beam set is measured, and the average value of the measured signal strength of the multiple beams in the first beam set is calculated.
[0142] Typically, the power of the beam from the second device is set to achieve the desired SNR at that device. However, under low SNR conditions, the impact of measurement errors introduced by the device's hardware is dominant due to the low signal strength. Therefore, by increasing the beam power, a high SNR is achieved, which reduces the importance of measurement errors introduced by the device's hardware. Thus, measurement errors can be mitigated. In some embodiments, the first beam set is transmitted at maximum power.
[0143] Alternatively or additionally, the average beam strength measurement on multiple first beam sets can be calculated, thereby obtaining an error-free approximation. The larger the number of first beam sets measured, the better the approximation can be. Therefore, measurement errors can be reduced. It should be understood that the first beam sets can be periodically transmitted by the second device. Similarly, the second beam sets can be periodically transmitted by the second device.
[0144] In some example embodiments, one, more, or all of the beams in the first beam set have increased power, enabling high SNR conditions and allowing the results to be averaged. In this way, multiple methods can be combined to mitigate measurement errors. In some example embodiments, measuring signal strength to obtain an error-free reference truth measurement includes mitigating measurement errors not mitigated when measuring signal strength to obtain an additional reference truth measurement.
[0145] In some embodiments, measuring signal strength includes measuring at least one of the following: RSRP, RSRQ, and RSSI.
[0146] In some example embodiments, the first beam set includes multiple beams. In some example embodiments, the second beam set includes multiple beams.
[0147] In some example embodiments, the device is a user equipment or is included within a user equipment.
[0148] In some example embodiments, the second device is or is included in at least one of the following: Test equipment; Network entities; Access node; Base station; gNodeB.
[0149] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: A unit for measuring the signal strength of a first beam set from a second device to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; A unit for sending a report of the error-free reference truth value measurement to the second device; A component for measuring the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; A component for sending a report to the second device on the additional reference true value measurement affected by the measurement error.
[0150] The device can perform the optional features described in the above description.
[0151] Processors, memory, and sample algorithms encoded as instructions, programs, or code can be components used to provide or cause the execution of operations.
[0152] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: A circuit configured to measure the signal strength of a first beam set from a second device to obtain an error-free reference truth measurement, wherein the first beam set includes at least one beam; A circuit configured to send a report of the error-free reference truth value measurement to the second device; A circuit configured to measure the signal strength of the first beam set from the second device to obtain an additional reference truth measurement affected by measurement error; A circuit system configured to send a report of the additional reference true value measurement affected by the measurement error to the second device.
[0153] The circuit can be configured to perform the optional features described with respect to the above-described device.
[0154] According to various, but not necessarily all, exemplary embodiments of the present invention, a method is provided, comprising: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; A report on the additional reference true value measurement affected by the measurement error is sent to the second device.
[0155] In some example embodiments, the method further includes: Receive instructions for using a machine learning model to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; In response to receiving the instruction, the machine learning model is used to predict at least one beam for communication between the device and the second device.
[0156] In some example embodiments, the received instruction indicates that the measurement error affecting the additional reference truth measurement meets at least in part the criteria based on the error-free reference truth measurement.
[0157] In some example embodiments, the method further includes at least one of the following: Measuring the signal strength of a second beam set from the second device to obtain an error-free measurement for input into the machine learning model, wherein the second beam set includes at least one beam, and using the machine learning model to predict the at least one beam includes inputting the error-free measurement into the machine learning model; and Measuring the signal strength of the second beam set from the second device to obtain additional measurements affected by measurement errors for input into the machine learning model, wherein using the machine learning model to predict the at least one beam includes inputting the additional measurements affected by the measurement errors into the machine learning model.
[0158] In some example embodiments, the method further includes: Indicators for determining the quality of the performance of the machine learning model based on at least one predicted beam and at least one of the following: The error-free reference true value measurement; and The additional reference truth value measurement.
[0159] In some example embodiments, indicators of the performance quality of the machine learning model include at least one of the following: Absolute reference signal received power error; Relative reference signal received power error; Pre-K / N accuracy; and Pre-K / N accuracy with margin.
[0160] In some example embodiments, the method further includes: The indicator of the quality of the performance of the machine learning model is sent to the second device.
[0161] In some example embodiments, the method further includes: When no instruction is received to predict at least one beam for communication between the device and the second device, the machine learning model is avoided from being used to predict at least one beam for communication between the device and the second device.
[0162] In some example embodiments, the method further includes: The device receives configuration information from the second device, the configuration information including an indication as to whether the device is configured to report to the second device the additional reference true value measurement affected by the measurement error. If the device is configured to report the additional reference truth value measurement to the second device, then the device performs the instruction to send the additional reference truth value measurement to the second device; and If the device is not configured to report the additional reference truth value measurement to the second device, then the device avoids sending the instruction for the additional reference truth value measurement to the second device.
[0163] In some example embodiments, obtaining an error-free reference truth measurement includes mitigating measurement errors, wherein mitigation includes at least one of the following: The first beam set has increased power compared to the conventional power of the first beam set; and The signal strength of multiple beams in the first beam set is measured, and the average value of the measured signal strength of the multiple beams in the first beam set is calculated.
[0164] According to various, but not necessarily all, exemplary embodiments of the present invention, a computer program comprising instructions that, when executed by a device, cause the device to perform at least the following operations: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; A report on the additional reference true value measurement affected by the measurement error is sent to the second device.
[0165] The instructions can be used to execute the optional features described in the above methods.
[0166] According to various, but not necessarily all, exemplary embodiments of the present invention, a computer program is provided, which includes instructions for performing at least the following operations: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; A report on the additional reference true value measurement affected by the measurement error is sent to the second device.
[0167] The instructions can be used to execute the optional features described in the above methods.
[0168] According to various, but not necessarily all, exemplary embodiments of the present invention, a non-transitory computer-readable medium is provided, comprising program instructions stored thereon for at least performing the following operations: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; A report on the additional reference true value measurement affected by the measurement error is sent to the second device.
[0169] The instructions can be used to execute the optional features described in the above methods.
[0170] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: at least one processor; and at least one memory storing instructions, said instructions causing the apparatus to perform at least the following when executed by said at least one processor: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report from the second device regarding the additional reference true value measurement affected by the measurement error; Whether the measurement error affecting the additional reference truth measurement meets the criteria is determined at least in part based on the error-free reference truth measurement.
[0171] Additional reference truth measurements can be used to evaluate the performance of a machine learning model configured to identify at least one beam used for communication between the device and a second device.
[0172] Error-free reference truth measurements can be used by the equipment to evaluate the quality of additional reference truth measurements used for this purpose. In other words, the impact of measurement errors can be assessed, which helps determine whether the additional reference truth measurements have acceptable quality for evaluating the performance of machine learning models.
[0173] In some example embodiments, the device is further made to perform: Send an instruction to the second device for using a machine learning model to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device.
[0174] In this way, ML model inference is used to predict one or more beams for communication between the device and the second device only if the criteria for evaluating the additional reference truth measurement are met, that is, when the measurement error in the additional reference truth measurement is tolerable, such that the additional reference truth measurement can still be used to evaluate the quality of the inference of the ML model.
[0175] In some example embodiments, a command is sent to indicate that the measurement error affecting the measurement of the additional reference truth value meets a criterion.
[0176] In some example embodiments, the device avoids sending requests for indicators of the performance quality of the machine learning model when the criteria are not met.
[0177] In some example embodiments, the device is also caused to perform at least one of the following: A second beam set for obtaining error-free measurements is sent to the second device for input into a machine learning model configured to identify at least one beam for communication between the device and the second device, wherein the second beam set includes at least one beam; and The second beam set is sent to the second device to obtain additional measurements affected by measurement errors for input into the machine learning model.
[0178] In some example embodiments, the device is further made to perform: Receive from the second device an indicator of the quality of the performance of the machine learning model; and Determine whether the indicator of the quality of the performance of the machine learning model meets the second criterion.
[0179] In some example embodiments, the device is further made to perform: The configuration information is sent to the second device, the configuration information including an indication of whether the second device is configured to report the additional reference true value measurement affected by the measurement error to the device.
[0180] In some example embodiments, obtaining an error-free reference truth measurement includes mitigating measurement errors, wherein mitigation includes at least one of the following: The first beam set is transmitted with increased power compared to its normal power; and Multiple beams from the first beam set are transmitted to calculate the average signal strength of the multiple beams in the first beam set.
[0181] Typically, the power of the beam transmitted from the device is set to achieve the desired SNR at the second device. However, under low SNR conditions, the impact of measurement errors introduced by the hardware of the second device is significant due to the low signal strength. Therefore, by increasing the beam power, a high SNR is achieved, which reduces the importance of measurement errors introduced by the hardware of the second device. Thus, measurement errors can be mitigated. In some embodiments, the first beam set is transmitted at maximum power.
[0182] Additionally or alternatively, average beam strength measurements on multiple first beam sets can be calculated, allowing for the acquisition of error-free approximations. The larger the number of first beam sets measured, the better the approximation. Therefore, measurement errors can be reduced. It should be understood that the first beam sets can be transmitted periodically by the device. Similarly, the second beam sets can be transmitted periodically by the device.
[0183] In some example embodiments, one, more, or all of the beams in the first beam set have increased power, enabling high SNR conditions and allowing the results to be averaged. In this way, multiple methods can be combined to mitigate measurement errors. In some example embodiments, measuring signal strength to obtain an error-free reference truth measurement includes mitigating measurement errors not mitigated when measuring signal strength to obtain an additional reference truth measurement.
[0184] In some example embodiments, when transmitting the first beam set at increased power, the device also performs the following: The error-free reference true value measurement is descaled based on the increased power.
[0185] In some example embodiments, the second device is a user equipment or is included within a user equipment.
[0186] In some example embodiments, the device is or is included in at least one of the following: Test equipment; Network entities; Access node; Base station; gNodeB.
[0187] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: Used to transmit a first beam set including at least one beam to a second device, the first beam set being used to obtain an error-free reference true value measurement unit; A unit for receiving a report of the error-free reference truth value measurement from the second device; The component is used to send the first beam set to the second device, the first beam set being used to obtain an additional reference true value measurement affected by measurement error; A component for receiving a report from the second device of the additional reference true value measurement affected by the measurement error; Components for determining, at least in part, whether the measurement error affecting the additional reference truth measurement meets a standard based on the error-free reference truth measurement.
[0188] The device can perform the optional features described in the above description.
[0189] Processors, memory, and sample algorithms encoded as instructions, programs, or code can be components used to provide or cause the execution of operations.
[0190] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: A circuit configured to transmit a first beam set including at least one beam to a second device, the first beam set being used to obtain an error-free reference truth measurement; A circuit configured to receive a report of the error-free reference truth value measurement from the second device; A circuit configured to send the first beam set to the second device, the first beam set being used to obtain an additional reference truth measurement affected by measurement error; A circuit configured to receive from the second device a report of the additional reference true value measurement affected by the measurement error; A circuit configured to determine, at least in part, whether the measurement error affecting the additional reference truth measurement meets a standard based on the error-free reference truth measurement.
[0191] The circuit can be configured to perform the optional features described with respect to the above-described device.
[0192] According to various, but not necessarily all, exemplary embodiments of the present invention, a method performed by an apparatus is provided, comprising: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report from the second device regarding the additional reference true value measurement affected by the measurement error; Whether the measurement error affecting the additional reference truth measurement meets the criteria is determined at least in part based on the error-free reference truth measurement.
[0193] In some example embodiments, the method further includes: Send an instruction to the second device for using a machine learning model to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device.
[0194] In some example embodiments, a command is sent to indicate that the measurement error affecting the measurement of the additional reference truth value meets a criterion.
[0195] In some example embodiments, the method further includes at least one of the following: A second beam set for obtaining error-free measurements is sent to the second device for input into a machine learning model configured to identify at least one beam for communication between the device and the second device, wherein the second beam set includes at least one beam; and The second beam set is sent to the second device to obtain additional measurements affected by measurement errors for input into the machine learning model.
[0196] In some example embodiments, the method further includes: Receive from the second device an indicator of the quality of the performance of the machine learning model; and Determine whether the indicator of the quality of the performance of the machine learning model meets the second criterion.
[0197] In some example embodiments, the method further includes: The configuration information is sent to the second device, the configuration information including an indication of whether the second device is configured to report the additional reference true value measurement affected by the measurement error to the device.
[0198] In some example embodiments, obtaining an error-free reference truth measurement includes mitigating measurement errors, wherein mitigation includes at least one of the following: The first beam set is transmitted with increased power compared to its normal power; and Multiple beams from the first beam set are transmitted to calculate the average signal strength of the multiple beams in the first beam set.
[0199] In some example embodiments, when transmitting the first beam set at increased power, the method further includes: The error-free reference true value measurement is descaled based on the increased power.
[0200] According to various, but not necessarily all, exemplary embodiments of the present invention, a computer program comprising instructions that, when executed by a device, cause the device to perform at least the following operations: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report from the second device regarding the additional reference true value measurement affected by the measurement error; Whether the measurement error affecting the additional reference truth measurement meets the criteria is determined at least in part based on the error-free reference truth measurement.
[0201] The instructions can be used to execute the optional features described in the above methods.
[0202] According to various, but not necessarily all, exemplary embodiments of the present invention, a computer program is provided, which includes instructions for performing at least the following operations: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report from the second device regarding the additional reference true value measurement affected by the measurement error; Whether the measurement error affecting the additional reference truth measurement meets the criteria is determined at least in part based on the error-free reference truth measurement.
[0203] The instructions can be used to execute the optional features described in the above methods.
[0204] According to various, but not necessarily all, exemplary embodiments of the present invention, a non-transitory computer-readable medium is provided, comprising program instructions stored thereon for at least performing the following operations: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report from the second device regarding the additional reference true value measurement affected by the measurement error; Whether the measurement error affecting the additional reference truth measurement meets the criteria is determined at least in part based on the error-free reference truth measurement.
[0205] The instructions can be used to execute the optional features described in the above methods.
[0206] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: at least one processor; and at least one memory storing instructions, said instructions causing the apparatus to perform at least the following when executed by said at least one processor: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; Send a report of the additional reference truth value measurement to the second device; A machine learning model is used to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; Indicators for determining the quality of the performance of the machine learning model based on at least one predicted beam and at least one of the following: The error-free reference true value measurement; and The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model is sent to the second device.
[0207] In some example embodiments, the device is also caused to perform at least one of the following: Measuring the signal strength of a second beam set from the second device to obtain an error-free measurement for input into the machine learning model, wherein the second beam set includes at least one beam, and using the machine learning model to predict the at least one beam includes inputting the error-free measurement into the machine learning model; and Measuring the signal strength of the second beam set from the second device to obtain additional measurements affected by measurement errors for input into the machine learning model, wherein using the machine learning model to predict the at least one beam includes inputting the additional measurements affected by the measurement errors into the machine learning model.
[0208] In some example embodiments, the second beam set is a subset of the first beam set.
[0209] In some example embodiments, the device is further made to perform: Receive configuration information from the second device, the configuration information including an instruction for the device to report at least one of the following: The error-free reference true value measurement; The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model.
[0210] The device is configured to acquire error-free reference true value measurements, additional reference true value measurements, and determine a quality indicator in response to receiving a corresponding instruction. These are then reported by sending a corresponding report to a second device. Alternatively, they can be included in a single report.
[0211] In some example embodiments, obtaining an error-free reference truth measurement includes mitigating measurement errors, wherein mitigation includes at least one of the following: The first beam set has increased power compared to the conventional power of the first beam set; and The signal strength of multiple beams in the first beam set is measured, and the average value of the measured signal strength of the multiple beams in the first beam set is calculated.
[0212] In some example embodiments, indicators of the performance quality of the machine learning model include at least one of the following: Absolute reference signal received power error; Relative reference signal received power error; Pre-K / N accuracy; and Pre-K / N accuracy with margin.
[0213] In some embodiments, measuring signal strength includes measuring at least one of the following: RSRP, RSRQ, and RSSI.
[0214] In some example embodiments, the first beam set includes multiple beams. In some example embodiments, the second beam set includes multiple beams.
[0215] In some example embodiments, the device is a user equipment or is included within a user equipment.
[0216] In some example embodiments, the second device is or is included in at least one of the following: Test equipment; Network entities; Access node; Base station; gNodeB.
[0217] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: A unit for measuring the signal strength of a first beam set from a second device to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; A unit for sending a report of the error-free reference truth value measurement to the second device; A component for measuring the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; A component for sending a report of the additional reference truth value measurement to the second device; A unit for using a machine learning model to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; A unit for determining the quality of the performance of the machine learning model based on at least one predicted beam and at least one of the following: The error-free reference true value measurement; and The additional reference true value measurement affected by the measurement error; and A unit for sending the indicator of the quality of the performance of the machine learning model to the second device.
[0218] The device can perform the optional features described in the above description.
[0219] Processors, memory, and sample algorithms encoded as instructions, programs, or code can be components used to provide or cause the execution of operations.
[0220] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: A circuit configured to measure the signal strength of a first beam set from a second device to obtain an error-free reference truth measurement, wherein the first beam set includes at least one beam; A circuit configured to send a report of the error-free reference truth value measurement to the second device; A circuit configured to measure the signal strength of the first beam set from the second device to obtain an additional reference truth measurement affected by measurement error; A circuit system configured to send a report of the additional reference truth value measurement to the second device; A circuit system configured to use a machine learning model to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; A circuit configured to determine the quality of the performance of the machine learning model based on at least one predicted beam and at least one of the following: The error-free reference true value measurement; and The additional reference true value measurement affected by the measurement error; and A circuit configured to send the indicator of the quality of the performance of the machine learning model to the second device.
[0221] The circuit can be configured to perform the optional features described with respect to the above-described device.
[0222] According to various, but not necessarily all, exemplary embodiments of the present invention, a method is provided, comprising: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; Send a report of the additional reference truth value measurement to the second device; A machine learning model is used to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; Indicators for determining the quality of the performance of the machine learning model based on at least one predicted beam and at least one of the following: The error-free reference true value measurement; and The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model is sent to the second device.
[0223] In some example embodiments, the method further includes at least one of the following: Measuring the signal strength of a second beam set from the second device to obtain an error-free measurement for input into the machine learning model, wherein the second beam set includes at least one beam, and using the machine learning model to predict the at least one beam includes inputting the error-free measurement into the machine learning model; and Measuring the signal strength of the second beam set from the second device to obtain additional measurements affected by measurement errors for input into the machine learning model, wherein using the machine learning model to predict the at least one beam includes inputting the additional measurements affected by the measurement errors into the machine learning model.
[0224] In some example embodiments, the second beam set is a subset of the first beam set.
[0225] In some example embodiments, the method further includes: Receive configuration information from the second device, the configuration information including an instruction for the device to report at least one of the following: The error-free reference true value measurement; The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model.
[0226] In some example embodiments, obtaining an error-free reference truth measurement includes mitigating measurement errors, wherein mitigation includes at least one of the following: The first beam set has increased power compared to the conventional power of the first beam set; and The signal strength of multiple beams in the first beam set is measured, and the average value of the measured signal strength of the multiple beams in the first beam set is calculated.
[0227] In some example embodiments, indicators of the performance quality of the machine learning model include at least one of the following: Absolute reference signal received power error; Relative reference signal received power error; Pre-K / N accuracy; and Pre-K / N accuracy with margin.
[0228] According to various, but not necessarily all, exemplary embodiments of the present invention, a computer program comprising instructions that, when executed by a device, cause the device to perform at least the following operations: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; Send a report of the additional reference truth value measurement to the second device; A machine learning model is used to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; Indicators for determining the quality of the performance of the machine learning model based on at least one predicted beam and at least one of the following: The error-free reference true value measurement; and The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model is sent to the second device.
[0229] The instructions can be used to execute the optional features described in the above methods.
[0230] According to various, but not necessarily all, exemplary embodiments of the present invention, a computer program is provided, which includes instructions for performing at least the following operations: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; Send a report of the additional reference truth value measurement to the second device; A machine learning model is used to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; Indicators for determining the quality of the performance of the machine learning model based on at least one predicted beam and at least one of the following: The error-free reference true value measurement; and The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model is sent to the second device.
[0231] The instructions can be used to execute the optional features described in the above methods.
[0232] According to various, but not necessarily all, exemplary embodiments of the present invention, a non-transitory computer-readable medium is provided, comprising program instructions stored thereon for at least performing the following operations: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; Send a report of the additional reference truth value measurement to the second device; A machine learning model is used to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; Indicators for determining the quality of the performance of the machine learning model based on at least one predicted beam and at least one of the following: The error-free reference true value measurement; and The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model is sent to the second device.
[0233] The instructions can be used to execute the optional features described in the above methods.
[0234] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: at least one processor; and at least one memory storing instructions, said instructions causing the apparatus to perform at least the following when executed by said at least one processor: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report of the additional reference truth value measurement from the second device; Receive from the second device an indicator of the quality of the performance of a machine learning model, the machine learning model being configured to identify at least one beam for communication between the device and the second device; and Whether the measurement error affecting the additional reference truth measurement meets the first criterion is determined at least in part based on the error-free reference truth measurement.
[0235] In some example embodiments, when the measurement error affecting the additional reference truth value meets a first criterion, the method further includes: Determine whether the quality of the performance of the machine learning model meets the second criterion.
[0236] In some example embodiments, the method further includes at least one of the following: A second beamset is sent to the second device, the second beamset being used to obtain error-free measurements for input into the machine learning model, wherein the second beamset includes at least one beam; and The second beam set is sent to the second device to obtain additional measurements affected by measurement errors for input into the machine learning model.
[0237] In some example embodiments, the method further includes: Send configuration information to the second device, the configuration information including an instruction for the second device to report at least one of the following: The error-free reference true value measurement; The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model.
[0238] In some example embodiments, obtaining an error-free reference truth measurement includes mitigating measurement errors, wherein mitigation includes at least one of the following: The first beam set is transmitted with increased power compared to its normal power; and Multiple beams from the first beam set are transmitted to calculate the average signal strength of the multiple beams in the first beam set.
[0239] In some example embodiments, when transmitting the first beam set at increased power, the method further includes: The error-free reference true value measurement is descaled based on the increased power.
[0240] In some example embodiments, the second device is a user equipment or is included within a user equipment.
[0241] In some example embodiments, the device is or is included in at least one of the following: Test equipment; Network entities; Access node; Base station; gNodeB.
[0242] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: Used to transmit a first beam set including at least one beam to a second device, the first beam set being used to obtain an error-free reference true value measurement unit; A unit for receiving a report of the error-free reference truth value measurement from the second device; The component is used to send the first beam set to the second device, the first beam set being used to obtain an additional reference true value measurement affected by measurement error; Components for receiving reports of the additional reference truth value measurements from the second device; A unit for receiving an indicator of the quality of the performance of a machine learning model from the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; and Components for determining, at least in part, whether the measurement error affecting the additional reference truth measurement meets a first criterion based on the error-free reference truth measurement.
[0243] The device can perform the optional features described in the above description.
[0244] Processors, memory, and sample algorithms encoded as instructions, programs, or code can be components used to provide or cause the execution of operations.
[0245] According to various, but not necessarily all, exemplary embodiments of the present invention, an apparatus is provided, comprising: A circuit configured to transmit a first beam set including at least one beam to a second device, the first beam set being used to obtain an error-free reference truth measurement; A circuit configured to receive a report of the error-free reference truth value measurement from the second device; A circuit configured to transmit the first beam set to the second device, wherein the first beam set obtains an additional reference truth measurement affected by measurement error; A circuit configured to receive a report of the additional reference truth value measurement from the second device; A circuit system configured to receive an indicator of the performance quality of a machine learning model from the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; and A circuit configured to determine, at least in part, whether the measurement error affecting the additional reference truth measurement meets a first criterion based on the error-free reference truth measurement.
[0246] The circuit can be configured to perform the optional features described with respect to the above-described device.
[0247] According to various, but not necessarily all, exemplary embodiments of the present invention, a method performed by an apparatus is provided, comprising: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report of the additional reference truth value measurement from the second device; Receive from the second device an indicator of the quality of the performance of a machine learning model, the machine learning model being configured to identify at least one beam for communication between the device and the second device; and Whether the measurement error affecting the additional reference truth measurement meets the first criterion is determined at least in part based on the error-free reference truth measurement.
[0248] In some example embodiments, when the measurement error affecting the additional reference truth value meets a first criterion, the method further includes: Determine whether the quality of the performance of the machine learning model meets the second criterion.
[0249] In some example embodiments, the method further includes: A second beamset is sent to the second device, the second beamset being used to obtain error-free measurements for input into the machine learning model, wherein the second beamset includes at least one beam; and The second beam set is sent to the second device to obtain additional measurements affected by measurement errors for input into the machine learning model.
[0250] In some example embodiments, the method further includes: Send configuration information to the second device, the configuration information including an instruction for the second device to report at least one of the following: The error-free reference true value measurement; The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model.
[0251] In some example embodiments, obtaining an error-free reference truth measurement includes mitigating measurement errors, wherein mitigation includes at least one of the following: The first beam set is transmitted with increased power compared to its normal power; and Multiple beams from the first beam set are transmitted to calculate the average signal strength of the multiple beams in the first beam set.
[0252] In some example embodiments, when transmitting the first beam set at increased power, the method further includes: The error-free reference true value measurement is descaled based on the increased power.
[0253] According to various, but not necessarily all, exemplary embodiments of the present invention, a computer program comprising instructions that, when executed by a device, cause the device to perform at least the following operations: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report of the additional reference truth value measurement from the second device; Receive from the second device an indicator of the quality of the performance of a machine learning model, the machine learning model being configured to identify at least one beam for communication between the device and the second device; and Whether the measurement error affecting the additional reference truth measurement meets the first criterion is determined at least in part based on the error-free reference truth measurement.
[0254] The instructions can be used to execute the optional features described in the above methods.
[0255] According to various, but not necessarily all, exemplary embodiments of the present invention, a computer program is provided, which includes instructions for performing at least the following operations: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report of the additional reference truth value measurement from the second device; Receive from the second device an indicator of the quality of the performance of a machine learning model, the machine learning model being configured to identify at least one beam for communication between the device and the second device; and Whether the measurement error affecting the additional reference truth measurement meets the first criterion is determined at least in part based on the error-free reference truth measurement.
[0256] The instructions can be used to execute the optional features described in the above methods.
[0257] According to various, but not necessarily all, exemplary embodiments of the present invention, a non-transitory computer-readable medium is provided, comprising program instructions stored thereon for at least performing the following operations: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report of the additional reference truth value measurement from the second device; Receive from the second device an indicator of the quality of the performance of a machine learning model, the machine learning model being configured to identify at least one beam for communication between the device and the second device; and Whether the measurement error affecting the additional reference truth measurement meets the first criterion is determined at least in part based on the error-free reference truth measurement.
[0258] The instructions can be used to execute the optional features described in the above methods.
[0259] Furthermore, various implementations of this disclosure can be described with reference to the following entries, and their features can be combined in any reasonable manner.
[0260] Item 1. An apparatus comprising: At least one processor; and At least one memory storing instructions that, when executed by the at least one processor, cause the device to perform at least the following: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; Send a report of the additional reference truth value measurement to the second device; A machine learning model is used to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; Indicators for determining the quality of the machine learning model's performance based on the predicted at least one beam and at least one of the following: The error-free reference true value measurement; and The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model is sent to the second device.
[0261] Item 2. The apparatus according to Item 1, wherein the apparatus is further caused to perform at least one of the following: Measuring the signal strength of a second beam set from the second device to obtain an error-free measurement for input into the machine learning model, wherein the second beam set includes at least one beam, and using the machine learning model to predict the at least one beam includes inputting the error-free measurement into the machine learning model; and Measuring the signal strength of the second beam set from the second device to obtain additional measurements affected by measurement errors for input into the machine learning model, wherein using the machine learning model to predict the at least one beam includes inputting the additional measurements affected by the measurement errors into the machine learning model.
[0262] Item 3. The apparatus according to Item 2, wherein the second beam set is a subset of the first beam set.
[0263] Item 4. The apparatus according to any of the preceding items, wherein the apparatus is further caused to perform: Receive configuration information from the second device, the configuration information including an instruction for the device to report at least one of the following: The error-free reference true value measurement; The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model.
[0264] Item 5. The apparatus according to any of the preceding items, wherein obtaining the error-free reference true value measurement includes mitigating the measurement error, wherein the mitigation includes at least one of the following: The first beam set has increased power compared to the conventional power of the first beam set; and The signal strength of multiple beams in the first beam set is measured, and the average value of the measured signal strength of the multiple beams in the first beam set is calculated.
[0265] Item 6. The apparatus according to any of the preceding items, wherein the indicator of the quality of the performance of the machine learning model includes at least one of the following: Absolute reference signal received power error; Relative reference signal received power error; Pre-K / N accuracy; and Pre-K / N accuracy with margin.
[0266] Item 7. An apparatus comprising: At least one processor; and At least one memory storing instructions that, when executed by the at least one processor, cause the device to perform at least the following: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report of the additional reference truth value measurement from the second device; Receive from the second device an indicator of the quality of the performance of a machine learning model, the machine learning model being configured to identify at least one beam for communication between the device and the second device; and Whether the measurement error affecting the additional reference truth measurement meets the first criterion is determined at least in part based on the error-free reference truth measurement.
[0267] Item 8. The apparatus according to Item 7, wherein when the measurement error affecting the additional reference truth value satisfies the first criterion, the apparatus further performs: Determine whether the quality of the performance of the machine learning model meets the second criterion.
[0268] Item 9. The apparatus according to Item 7 or 8, wherein the apparatus is further caused to perform at least one of the following: Sending a second beamset to the second device, the second beamset being used to obtain error-free measurements for input into the machine learning model, wherein the second beamset includes at least one beam; and The second beam set is sent to the second device to obtain additional measurements affected by measurement errors for input into the machine learning model.
[0269] Item 10. The apparatus according to any one of items 7 to 9, wherein the apparatus is further caused to perform: Send configuration information to the second device, the configuration information including an instruction for the second device to report at least one of the following: The error-free reference true value measurement; The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model.
[0270] Item 11. The apparatus according to any one of items 7 to 10, wherein obtaining the error-free reference true value measurement includes mitigating the measurement error, wherein the mitigation includes at least one of the following: The first beam set is transmitted with increased power compared to its normal power; and Multiple beams from the first beam set are transmitted to calculate the average signal strength of the multiple beams in the first beam set.
[0271] Item 12. The apparatus according to Item 11, wherein when the first beam set is transmitted at increased power, the apparatus also performs: The error-free reference true value measurement is descaled based on the increased power.
[0272] Item 13. A method comprising: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; Send a report of the additional reference truth value measurement to the second device; A machine learning model is used to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; Indicators for determining the quality of the machine learning model's performance based on the predicted at least one beam and at least one of the following: The error-free reference true value measurement; and The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model is sent to the second device.
[0273] Item 14. A method comprising: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report of the additional reference truth value measurement from the second device; Receive from the second device an indicator of the quality of the performance of a machine learning model, the machine learning model being configured to identify at least one beam for communication between the device and the second device; and Whether the measurement error affecting the additional reference truth measurement meets the first criterion is determined at least in part based on the error-free reference truth measurement.
[0274] Those skilled in the art will readily recognize that the steps of the various methods described above can be performed by a programmed computer. In this document, some embodiments are also intended to cover program storage devices, such as digital data storage media, which are machine- or computer-readable and encode machine-executable or computer-executable instructions, said instructions performing some or all of the steps of the methods described above. Program storage devices can be, for example, digital memories, magnetic storage media such as disks and tapes, hard disk drives, or optically readable digital data storage media. Embodiments are also intended to cover computers programmed to perform the steps of the methods described above. As used herein, non-transitory is a limitation of the medium itself (i.e., tangible, not signal-based), not a limitation of the persistence of data storage (e.g., RAM and ROM).
[0275] As used in this application, the term "circuit" may refer to one or more of the following: Hardware circuit implementation only (such as implementations in analog, digital, and / or quantum circuits) and A combination of hardware circuitry and software, such as (if applicable): The combination of analog, digital, and / or quantum hardware circuitry with software / firmware, and (ii) Any or all portions of a hardware processor (including digital and / or quantum processors with software) and a memory that works together to enable a device such as a mobile device, computing device, or server to perform various functions, and (c) Any or all parts of the hardware circuitry, such as microprocessors, processors and / or quantum processors, that require software (e.g., firmware) to operate, but which may not exist when no software is required to operate.
[0276] This definition of "circuit" applies to all uses of the term in this application (including in any claim). As another example, as used herein, the term "circuit" also encompasses only hardware circuitry or a processor (or multiple processors) or a portion thereof and its accompanying software and / or firmware implementation. The term "circuit" also encompasses, for example and if applicable to a particular claim element, baseband integrated circuits or processor integrated circuits for mobile devices or similar integrated circuits in servers, cellular network devices, or other computing or network devices.
[0277] As used herein, “at least one of the following: a list of two or more elements” and “at least one of the following: a list of two or more elements” and similar wording (where the list of two or more elements is connected by “and” or “or”) means at least any one of the elements, or at least any two or more of the elements, or at least all of the elements.
[0278] The order of the steps described above may not be critical or fixed, and the exact order of the steps may vary appropriately.
[0279] Although exemplary embodiments of the invention have been described with reference to various examples in the preceding paragraphs, it should be understood that modifications may be made to the given examples without departing from the scope of the claimed invention.
[0280] The features described above can be used in combinations other than those explicitly stated.
[0281] Although some features have been described with reference to certain characteristics, these functions can be performed by other features, whether or not they are described.
[0282] Although features have been described with reference to certain embodiments, these features may also exist in other embodiments, whether or not they are described.
[0283] While efforts have been made in the foregoing description to draw attention to those features of the invention that are considered particularly important, it should be understood that the applicant claims protection for any patentable feature or combination of features referenced above and / or shown in the drawings, whether or not they have been specifically emphasized.
[0284] List of abbreviations AI (Artificial Intelligence) BB baseband BWP bandwidth portion CDF cumulative distribution function CSI Channel Status Information DL downlink DRX discontinuous reception gNBgNodeB KPIs (Key Performance Indicators) ML machine learning MoO Operation Mode NR New Radio NW Network RB resource blocks RF radio frequency RSRP reference signal received power RX receiver SI Research Project SINR (Signal-to-Interference-Ratio) SNR signal-to-noise ratio TE testing equipment TX sent UE User Equipment WI work items
Claims
1. A device for communication, comprising: At least one processor; as well as At least one memory storing instructions that, when executed by the at least one processor, cause the device to perform at least the following: The signal strength from a first beam set from a second device is measured to obtain an error-free reference true value measurement, wherein the first beam set includes at least one beam; Send a report of the error-free reference true value measurement to the second device; Measure the signal strength of the first beam set from the second device to obtain an additional reference true value measurement affected by measurement error; Send a report of the additional reference truth value measurement to the second device; A machine learning model is used to predict at least one beam for communication between the device and the second device, the machine learning model being configured to identify at least one beam for communication between the device and the second device; Indicators for determining the quality of the machine learning model's performance based on the predicted at least one beam and at least one of the following: The error-free reference true value measurement; as well as The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model is sent to the second device.
2. The apparatus of claim 1, wherein the apparatus is further caused to perform at least one of the following: Measuring the signal strength of a second beam set from the second device to obtain an error-free measurement for input into the machine learning model, wherein the second beam set includes at least one beam, and using the machine learning model to predict the at least one beam includes inputting the error-free measurement into the machine learning model; and Measuring the signal strength of the second beam set from the second device to obtain additional measurements affected by measurement errors for input into the machine learning model, wherein using the machine learning model to predict the at least one beam includes inputting the additional measurements affected by the measurement errors into the machine learning model.
3. The apparatus of claim 2, wherein the second beam set is a subset of the first beam set.
4. The apparatus according to any one of claims 1 to 3, wherein the apparatus is further caused to perform: Receive configuration information from the second device, the configuration information including an instruction for the device to report at least one of the following: The error-free reference true value measurement; The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model.
5. The apparatus according to any one of claims 1 to 3, wherein obtaining the error-free reference true value measurement includes mitigating the measurement error, wherein the mitigation includes at least one of the following: The first beam set has increased power compared to the conventional power of the first beam set; and The signal strength of multiple beams in the first beam set is measured, and the average value of the measured signal strength of the multiple beams in the first beam set is calculated.
6. The apparatus according to any one of claims 1 to 3, wherein the indicator of the quality of the performance of the machine learning model comprises at least one of the following: Absolute reference signal received power error; Relative reference signal received power error; Pre-K / N accuracy; and Pre-K / N accuracy with margin.
7. A device for communication, comprising: At least one processor; as well as At least one memory storing instructions that, when executed by the at least one processor, cause the device to perform at least the following: Send a first beam set including at least one beam to the second device, the first beam set being used to obtain an error-free reference true value measurement; Receive a report of the error-free reference truth value measurement from the second device; The first beam set is sent to the second device, the first beam set being used to obtain additional reference true value measurements affected by measurement errors; Receive a report of the additional reference truth value measurement from the second device; The second device receives an indicator of the quality of the performance of a machine learning model, which is configured to identify at least one beam used for communication between the device and the second device. as well as Whether the measurement error affecting the additional reference truth measurement meets the first criterion is determined at least in part based on the error-free reference truth measurement.
8. The apparatus according to claim 7, wherein, When the measurement error affecting the additional reference truth value meets the first criterion, the device further performs the following: Determine whether the quality of the performance of the machine learning model meets the second criterion.
9. The apparatus according to claim 7 or 8, wherein the apparatus is further caused to perform at least one of the following: Sending a second beamset to the second device, the second beamset being used to obtain error-free measurements for input into the machine learning model, wherein the second beamset includes at least one beam; and The second beam set is sent to the second device to obtain additional measurements affected by measurement errors for input into the machine learning model.
10. The apparatus according to any one of claims 7 to 9, wherein the apparatus is further caused to perform: Send configuration information to the second device, the configuration information including an instruction for the second device to report at least one of the following: The error-free reference true value measurement; The additional reference true value measurement affected by the measurement error; and The indicator of the quality of the performance of the machine learning model.
