Protocol and signaling for artificial intelligence and machine learning model performance monitoring

Abstract

Wireless communication devices, systems, and methods are provided that relate to monitoring performance of artificial intelligence (AI) and / or machine learning (ML) models, including associated protocols and signaling. For example, a method of wireless communication performed by a user equipment (UE) may include receiving a machine learning (ML) model monitoring configuration; evaluating one or more values associated with a prediction of the ML model as one or more measured values based at least in part on the ML model monitoring configuration; and sending an indication of a fault of the ML model in response to detecting the fault of the ML model based on the evaluation.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of International PCT Patent Application No. PCT / CN2022 / 112010 filed on August 12, 2022, the disclosure of which is incorporated herein by reference in its entirety as if fully set forth below and for all applicable purposes. Technical Field

[0003] The present application relates to wireless communication systems, and more particularly to methods for monitoring the performance of artificial intelligence (AI) and / or machine learning (ML) models and associated devices and systems, including associated protocols and signaling. Background Art

[0004] Wireless communication systems are widely deployed to provide various types of communication content, such as voice, video, packet data, messaging, broadcast, and the like. These systems may be able to support communication with multiple users by sharing available system resources (e.g., time, frequency, and power). A wireless multiple access communication system may include multiple base stations (BSs), each of which simultaneously supports communication with multiple communication devices, which may be further referred to as user equipment (UE). Examples of such multiple access systems include fourth generation (4G) systems (such as long term evolution (LTE) systems, advanced LTE (LTE-A) systems, or LTE-A Pro systems), and fifth generation (5G) systems, which may be referred to as new radio (NR) systems. These systems may employ techniques such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal frequency division multiple access (OFDMA), or discrete Fourier transform spread orthogonal frequency division multiplexing (DFT-S-OFDM).

[0005] To meet the growing demand for expanded mobile broadband connections, wireless communication technology is evolving from Long Term Evolution (LTE) technology to next-generation New Radio (NR) technology (which may be referred to as fifth generation (5G)). For example, NR is designed to provide lower latency, higher bandwidth or higher throughput, and higher reliability compared to LTE. NR is designed to operate on a wide array of spectrum bands, for example, from low frequency bands below about 1 gigahertz (GHz) and mid-frequency bands from about 1 GHz to about 6 GHz to high frequency bands such as millimeter wave (mmWave) bands. NR is also designed to operate across different spectrum types, from licensed spectrum to unlicensed spectrum and shared spectrum. Spectrum sharing enables operators to have the opportunity to aggregate spectrum to dynamically support high-bandwidth services. Spectrum sharing can extend the benefits of NR technology to operating entities that may not have access to licensed spectrum.

[0006] In a wireless communication network, a BS may communicate with a UE in both an uplink direction and a downlink direction. The radio frequency channel through which the BS and the UE communicate may have several channel properties that are considered for proper channel performance. The BS and the UE may perform channel sounding to better understand these channel properties by measuring and / or estimating various parameters of the channel (such as delay, path loss, absorption, multipath, reflection, fading, Doppler effect, etc.). These channel measurements may also be used for channel estimation and channel equalization. Summary of the invention

[0007] The following summarizes some aspects of the present disclosure to provide a basic understanding of the technology discussed. This summary is not an exhaustive overview of all expected features of the present disclosure, and is neither intended to identify key or important elements of all aspects of the present disclosure, nor to delineate the scope of any or all aspects of the present disclosure. The sole purpose of this summary is to present some concepts of one or more aspects of the present disclosure in a summarized form as a prelude to more specific embodiments presented later.

[0008] In one aspect of the disclosure, a method of wireless communication performed by a user equipment (UE) includes: receiving a machine learning (ML) model monitoring configuration; evaluating one or more values ​​associated with a prediction of the ML model as one or more measured values ​​based at least in part on the ML model monitoring configuration; and sending an indication of a failure of the ML model in response to detecting the failure of the ML model based on the evaluation. Associated devices, systems, components, and / or non-transitory computer-readable media having one or more instructions for execution by one or more processors of the UE are also provided.

[0009] In an additional aspect of the present disclosure, a method of wireless communication performed by a network element includes: sending a machine learning (ML) model monitoring configuration to a user equipment (UE), wherein the ML model monitoring configuration enables the UE to detect a failure of the ML model based on one or more measured values; and receiving an indication of a failure of the ML model from the UE based on the ML model monitoring configuration. An associated device, system, component, and / or non-transitory computer-readable medium having one or more instructions for execution by one or more processors of a network element is also provided.

[0010] In an additional aspect of the present disclosure, a user equipment (UE) includes a memory device; a transceiver; and a processor in communication with the processor and the transceiver, wherein the UE is configured to: receive a machine learning (ML) model monitoring configuration; evaluate one or more values ​​associated with a prediction of the ML model as one or more measured values ​​based at least in part on the ML model monitoring configuration; and send an indication of a failure of the ML model in response to detecting the failure of the ML model based on the evaluation.

[0011] In additional aspects of the present disclosure, a network unit includes a memory device; a transceiver; and a processor that communicates with the processor and the transceiver, wherein the network unit is configured to: send a machine learning (ML) model monitoring configuration to a user equipment (UE), wherein the ML model monitoring configuration enables the UE to detect a failure of the ML model based on one or more measured values; and receive an indication of a failure of the ML model from the UE based on the ML model monitoring configuration.

[0012] After reviewing the following description of specific exemplary aspects of the present invention in conjunction with the accompanying drawings, other aspects and features of the present invention will become apparent to those of ordinary skill in the art. Although features of the present invention may be discussed below with respect to certain aspects and drawings, all aspects of the present invention may include one or more of the advantageous features discussed herein. In other words, although one or more aspects may be discussed as having certain advantageous features, one or more of such features may also be used according to the various aspects of the present invention discussed herein. In a similar manner, although exemplary aspects may be discussed below as aspects of devices, systems, or methods, it should be understood that such exemplary aspects may be implemented in various devices, systems, and methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A wireless communication network according to one or more aspects of the present disclosure is illustrated.

[0014] Figure 2 A diagram illustrating an example decomposed base station architecture in accordance with one or more aspects of the present disclosure.

[0015] Figure 3 A time-domain beam prediction scheme according to some aspects of the present disclosure is illustrated.

[0016] Figure 4 A spatial beam prediction scheme according to some aspects of the present disclosure is illustrated.

[0017] Figure 5 A spatial beam prediction scheme according to one or more aspects of the present disclosure is illustrated.

[0018] Figure 6 A signaling diagram illustrating a machine learning (ML) model monitoring scheme according to one or more aspects of the present disclosure is illustrated.

[0019] Figure 7 A flowchart illustrating an ML model monitoring scheme according to one or more aspects of the present disclosure is illustrated.

[0020] Figure 8 A block diagram illustrating a user equipment (UE) according to one or more aspects of the present disclosure is illustrated.

[0021] Fig. 9 A block diagram of a network element according to one or more aspects of the present disclosure is illustrated.

[0022] Fig.10 A flow chart illustrating a method of wireless communication according to some aspects of the present disclosure is shown.

[0023] Fig.11 A flow chart illustrating a method of wireless communication according to some aspects of the present disclosure is shown. DETAILED DESCRIPTION

[0024] The detailed description set forth below in conjunction with the accompanying drawings is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. In order to provide a comprehensive understanding of the various concepts, the detailed description includes specific details. However, it is apparent to those skilled in the art that these concepts may be practiced without these specific details. In some aspects, well-known structures and components are shown in block diagram form to avoid obscuring such concepts.

[0025] The present disclosure as a whole relates to wireless communication systems, which are also referred to as wireless communication networks. In various aspects, these techniques and devices can be used in wireless communication networks, such as code division multiple access (CDMA) networks, time division multiple access (TDMA) networks, frequency division multiple access (FDMA) networks, orthogonal FDMA (OFDMA) networks, single carrier FDMA (SC-FDMA) networks, LTE networks, global mobile communication systems (GSM) networks, fifth generation (5G) or new radio (NR) networks, and other communication networks. As described herein, the terms "network" and "system" can be used interchangeably.

[0026] OFDMA networks may implement radio technologies such as Evolved UTRA (E-UTRA), Institute of Electrical and Electronics Engineers (IEEE) 802.11, IEEE 802.16, IEEE 802.20, flash-OFDM, etc. UTRA, E-UTRA, and GSM are part of the Universal Mobile Telecommunications System (UMTS). Specifically, Long Term Evolution (LTE) is a version of UMTS that uses E-UTRA. UTRA, E-UTRA, GSM, UMTS, and LTE are described in documents provided by an organization named "3rd Generation Partnership Project" (3GPP), and cdma2000 is described in documents from an organization named "3rd Generation Partnership Project 2" (3GPP2). These various radio technologies and standards are either known or under development. For example, the 3rd Generation Partnership Project (3GPP) is a collaboration between a group of telecommunications associations with the goal of defining globally applicable third generation (3G) mobile phone specifications. 3GPP Long Term Evolution (LTE) is a 3GPP project aimed at improving the UMTS mobile phone standard. 3GPP may define specifications for next generation mobile networks, mobile systems, and mobile devices. The present disclosure relates to the evolution from LTE, 4G, 5G, NR and beyond wireless technologies, where access to the wireless spectrum is shared between networks using a range of new and different radio access technologies or radio air interfaces.

[0027] Specifically, 5G networks consider various deployments, various spectrums, and various services and devices that can be implemented using a unified air interface based on OFDM. To achieve these goals, in addition to developing new radio technologies for 5G NR networks, further enhancements to LTE and LTE-A are also considered. 5G NR will be able to scale to (1) ultra-high density (e.g., about 1M nodes / km) 2 (1) provide coverage for the massive Internet of Things (IoT) with ultra-low complexity (e.g., about 10s of bits / second), ultra-low energy (e.g., about 10+ years of battery life), and deep coverage with the ability to reach challenging locations; (2) provide coverage including mission-critical control with strong security to protect sensitive personal, financial, or classified information, ultra-high reliability (e.g., about 99.9999% reliability), ultra-low latency (e.g., about 1 millisecond), and for users with extensive mobility or lack of mobility; and (3) provide coverage with enhanced mobile broadband (including very high capacity (e.g., about 10Tbps / km 2 ), coverage of very high data rates (e.g., multi-Gbps rates, 100+Mbps user experienced rates), and deep awareness with advanced discovery and optimization).

[0028] 5G NR can be implemented using an optimized OFDM-based waveform with scalable parameter sets and transmission time intervals (TTIs); with a common, flexible framework to efficiently multiplex services and features using dynamic low-latency time division duplex (TDD) / frequency division duplex (FDD) designs; and with advanced wireless technologies such as massive multiple-input multiple-output (MIMO), robust millimeter wave (mmWave) transmission, advanced channel decoding, and device-centric mobility. The scalability of parameter sets in 5G NR and the scaling of subcarrier spacing can effectively address the operation of various services across different spectrums and different deployments. For example, in various outdoor and macro coverage deployments of less than 3GHz FDD / TDD implementations, the subcarrier spacing may appear at 15kHz on bandwidths (BWs) such as 5MHz, 10MHz, 20MHz, etc. For other various outdoor and small cell coverage deployments of TDD greater than 3GHz, the subcarrier spacing may appear at 30kHz on 80MHz / 100MHz BW. For various other indoor broadband implementations, using TDD on the unlicensed portion of the 5 GHz band, subcarrier spacing may occur at 60 kHz over 160 MHz BW. Finally, for various deployments transmitting with the mmWave component at 28 GHz TDD, subcarrier spacing may occur at 120 kHz over 500 MHz BW.

[0029] 5G NR's scalable parameter sets facilitate scalable TTIs for different latency and quality of service (QoS) requirements. For example, shorter TTIs can be used for low latency and high reliability, while longer TTIs can be used for higher spectral efficiency. Efficient multiplexing of long and short TTIs allows transmission to start on symbol boundaries. 5G NR also contemplates a self-contained integrated subframe design with uplink (UL) / downlink (DL) scheduling information, data, and acknowledgment in the same subframe. Self-contained integrated subframes support communications in unlicensed or contention-based shared spectrum, and support adaptive UL / DL that can be flexibly configured on a per-cell basis to dynamically switch between UL and DL to meet current traffic needs.

[0030] The various other aspects and features of the present disclosure are further described below. It should be apparent that the teachings herein can be embodied in various forms, and any specific structure, function, or both disclosed herein are merely representative and not restrictive. Based on the teachings herein, it should be understood by those of ordinary skill in the art that the aspects disclosed herein can be implemented independently of any other aspects, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement a device or practice a method. In addition, such a device can be implemented or such a method can be practiced using other structures, functionality, or structures and functionality other than or different from one or more aspects of the aspects set forth herein. For example, the method can be implemented as a part of a system, device, device, and / or implemented as an instruction stored on a computer-readable medium for execution on a processor or computer. In addition, one aspect can include at least one element of a claim.

[0031] Various aspects of the present disclosure allow a UE and / or a network element to monitor the performance of one or more machine learning (ML) models, including ML models for beam prediction. A wireless channel between a network (e.g., a BS) and a UE may change over time. The BS may configure a set of beams for a UE, and the UE may use one or two serving beams at any point in time to receive DL transmissions from the BS or send UL transmissions to the BS. The BS and the UE may keep track of the serving beams and candidate beams. For example, the UE may perform one or more measurements of one or more reference signals configured for the UE, and may include the one or more measurements in a channel state information (CSI) report. If the serving beam fails, the BS may reconfigure the UE to use a candidate beam. Because the channel quality between the BS and the UE may change over time, the candidate beams may be updated periodically. It is expected that the UE updates the serving beam based on the channel state. The UE may report the link quality of the serving beam and the candidate beams to the BS in a CSI report, and the BS may process the CSI report and determine whether the serving beam or the candidate beam of the UE should be reconfigured. If the quality of the beam drops below a threshold, the BS may reconfigure the beam of the UE's serving beam or candidate beam. The BS may configure the threshold. Based on the determination, the BS may send a command to reconfigure the UE's serving beam and / or candidate beam in response to the CSI report.

[0032] The BS may configure the UE to periodically report CSI reports to the BS. The CSI report may include, for example, channel quality information (CQI) and / or reference signal received power (RSRP). CQI is an indicator that carries information about the quality of a communication channel. The BS may use CQI to assist in downlink (DL) scheduling. The BS may use RSRP to manage beams in multi-beam operation. The UE may perform different measurement combinations to include in the CSI report. Thus, the UE may send a CSI report including CQI but not RSRP, a CSI report including RSRP but not CQI, and / or a CSI report including both CQI and RSRP.

[0033] In 5G NR, machine learning (ML) algorithms are being implemented to assist cellular network performance. These ML algorithms may include neural networks implemented at different types of nodes within the wireless communication network. For example, the neural network may be implemented at a single node (e.g., UE / BS / central cloud server) or may be distributed across multiple nodes. ML algorithms may be implemented to assist different functions and / or modules between nodes of the wireless communication network. In various aspects, the neural network may be implemented as a convolutional neural network (CNN), a recurrent neural network (RNN), a deep convolutional network (DCN), and the like.

[0034] At each node where one or more ML algorithms are implemented, the ML algorithms may interact with different layers within the node. The ML algorithms may interact with one of the physical layer (PHY), the medium access control (MAC) layer, or an upper layer (e.g., the application layer) in some cases, or with multiple layers in other cases. These ML algorithms may involve various ML-related data transmissions between different layers of different nodes (e.g., UE, BS, central cloud server). The ML algorithms may be trained using training data sets generated by periodic and / or aperiodic data collection at one or more nodes. In various aspects, measurement data collection serves as input to the ML module. The operation of these ML algorithms at different nodes may be used for ML model parameter transmission and / or updating. The ML model framework within the wireless communication network has the ability to transmit feedback signals and / or reports between different nodes. In various aspects, the UE may feedback channel measurements indicating the accuracy of ML model predictions. For example, measurement data collection performed by the UE may be transmitted to the BS and / or central cloud server together with a report that may indicate that the ML model is generating prediction errors, thereby indicating that the ML model has failed and / or needs to be updated.

[0035] In various aspects, the UE may include different ML algorithms on board to predict channel properties for future use of the channel. For example, a machine learning based network may be implemented by a channel property prediction network to predict one or more properties of a channel and / or one or more beam parameters. In some aspects, the ML algorithms are tasked to predict what transmit beam to use for the BS and / or what receive beam to use for the UE. For example, a machine learning based network may be implemented by a beam selection prediction network to predict a BS transmit beam and / or a UE receive beam.

[0036] The present disclosure provides techniques for UEs and / or network elements to monitor the performance of one or more machine learning (ML) models, including ML models for beam prediction. In this regard, the UE and / or network element may stop the ML model, initiate retraining of the ML model, and / or adjust one or more operating parameters of the ML model upon detecting an event such as a failure of the ML model. Various aspects of the present disclosure provide protocols and signaling that allow UEs and network elements to coordinate as to the operation (or non-operation) of the ML model. As described herein, in addition to the coordination benefits, various aspects of the present disclosure also provide improved network efficiency, improved network resource allocation, reduced power consumption of UEs and / or network elements, and / or improved utilization of ML models.

[0037] Figure 1 A wireless communication network 100 according to one or more aspects of the present disclosure is illustrated. The network 100 may be a 5G network. The network 100 includes a plurality of BSs 105 (labeled 105a, 105b, 105c, 105d, 105e, and 105f, respectively) and other network entities. The BSs 105 may be stations that communicate with UEs 115 (labeled 115a, 115b, 115c, 115d, 115e, 115f, 115g, 115h, and 115k, respectively), and may also be referred to as evolved B nodes (eNBs), next generation eNBs (gNBs), access points, and the like. Each BS 105 may provide communication coverage for a particular geographic area. In 3GPP, the term "cell" may refer to that particular geographic coverage area of ​​the BS 105 and / or a BS subsystem serving that coverage area, depending on the context in which the term is used.

[0038] The BS 105 may provide communication coverage for macro cells or small cells (such as pico cells or femto cells) and / or other types of cells. A macro cell generally covers a relatively large geographic area (e.g., several kilometers in radius) and may allow unrestricted access by UEs with a service subscription with a network provider. A small cell (such as a pico cell) will generally cover a relatively small geographic area and may allow unrestricted access by UEs with a service subscription with a network provider. A small cell (such as a femto cell) will generally also cover a relatively small geographic area (e.g., a home) and, in addition to unrestricted access, may also provide restricted access by UEs associated with the femto cell (e.g., UEs in a closed subscriber group (CSG), UEs of users in a home, etc.). A BS for a macro cell may be referred to as a macro BS. A BS for a small cell may be referred to as a small cell BS, a pico BS, a femto BS, or a home BS. In Figure 1 , BS 105d and 105e may be conventional macro BSs, while BS 105a-105c may be macro BSs with one of three-dimensional (3D), full-dimensional (FD), or massive MIMO capabilities. BS 105a-105c may utilize their higher-dimensional MIMO capabilities to employ 3D beamforming in both elevation and azimuth beamforming to increase coverage and capacity. BS 105f may be a small cell BS, which may be a home node or a portable access point. BS 105 may support one or more (e.g., two, three, four, etc.) cells.

[0039] In some aspects, the term "base station" (e.g., base station 105) or "network entity" may refer to a converged base station, a decomposed base station, an integrated access and backhaul (IAB) node, a relay node, and / or one or more components thereof. For example, in some aspects, a "base station" or "network entity" may refer to a central unit (CU), a distributed unit (DU), a radio unit (RU), a near real-time (near-RT) RAN intelligent controller (RIC), or a non-real-time (non-RT) RIC, or a combination thereof. In some aspects, the term "base station" or "network entity" may refer to one device configured to perform one or more functions, such as those described herein in conjunction with base station 105. In some aspects, the term "base station" or "network entity" may refer to multiple devices configured to perform one or more functions. For example, in some distributed systems, each of multiple different devices (which may be located in the same geographical location or different geographical locations) may be configured to perform at least a portion of a function, or to repeat the performance of at least a portion of the function, and the term "base station" or "network entity" may refer to any one or more of these different devices. In some aspects, the term "base station" or "network entity" may refer to one or more virtual base stations and / or one or more virtual base station functions. For example, in some aspects, two or more base station functions may be instantiated on a single device. In some aspects, the term "base station" or "network entity" may refer to one of the base station functions but not another of the base station functions. In this way, a single device may include more than one base station.

[0040] Network 100 may support synchronous operation or asynchronous operation. For synchronous operation, the BSs may have similar frame timing, and transmissions from different BSs may be approximately aligned in time. For asynchronous operation, the BSs may have different frame timing, and transmissions from different BSs may not be aligned in time.

[0041] UE 115 is dispersed throughout the wireless network 100, and each UE 115 may be stationary or mobile. UE 115 may also be referred to as a terminal, a mobile station, a subscriber unit, a station, etc. UE 115 may be a cellular phone, a personal digital assistant (PDA), a wireless modem, a wireless communication device, a handheld device, a tablet computer, a laptop computer, a cordless phone, a wireless local loop (WLL) station, etc. In one aspect, UE 115 may be a device including a universal integrated circuit card (UICC). In another aspect, UE may be a device that does not include a UICC. In some aspects, UE 115 that does not include a UICC may also be referred to as an IoT device or an Internet of Everything (IoE) device. UE 115a-115d is an example of a mobile smart phone type device that accesses the network 100. UE 115 may also be a machine that is specifically configured for connected communications (including machine type communications (MTC), enhanced MTC (eMTC), narrowband IoT (NB-IoT), etc.). UE 115e-115h are examples of various machines configured for communication to access network 100. UE 115i-115k are examples of vehicles equipped with wireless communication devices configured for communication to access network 100. UE 115 may be able to communicate with any type of BS (whether macro BS, small cell, etc.). Figure 1 , a lightning symbol (e.g., a communication link) indicates wireless transmission between UE 115 and a serving BS 105 (which is a BS designated to provide service to UE 115 on DL and / or UL), desired transmission between BSs 105, backhaul transmission between BSs, or sidelink transmission between UEs 115.

[0042] In operation, BS 105a-105c may use 3D beamforming and cooperative spatial techniques such as coordinated multi-point (CoMP) or multiple connections to serve UEs 115a and 115b. Macro BS 105d may perform backhaul communications with BS 105a-105c and small cell BS 105f. Macro BS 105d may also transmit multicast services that UEs 115c and 115d subscribe to and receive. Such multicast services may include mobile TV or streaming video, or may include other services for providing community information, such as weather emergencies or alerts, such as Amber alerts or gray alerts.

[0043] The BSs 105 may also communicate with a core network. The core network may provide user authentication, access authorization, tracking, Internet Protocol (IP) connectivity, and other access, routing, or mobility functions. At least some of the BSs 105 (e.g., which may be instances of gNBs or access node controllers (ANCs)) may interface with the core network via a backhaul link (e.g., NG-C, NG-U, etc.) and may perform radio configuration and scheduling for communications with the UEs 115. In various cases, the BSs 105 may communicate with each other directly or indirectly (e.g., through the core network) via a backhaul link (e.g., X1, X2, etc.), which may be a wired communication link or a wireless communication link.

[0044] The network 100 may also support mission-critical communications using ultra-reliable and redundant links for mission-critical devices, such as UE 115e, which may be a drone. The redundant communication links with UE 115e may include links from macro BSs 105d and 105e, and a link from a small cell BS 105f. Other machine-type devices, such as UE 115f (e.g., a thermometer), UE 115g (e.g., a smart meter), and UE 115h (e.g., a wearable device), may communicate through the network 100 directly with BSs such as small cell BS 105f and macro BS 105e, or through communication with another user device, such as UE 115f that communicates temperature measurement information to a smart meter (UE 115g), which then reports to the network through the small cell BS 105f, which relays its information to the network, in a multi-action size configuration. The network 100 may also provide additional network efficiency through dynamic low latency TDD / FDD communications, such as V2V, V2X, C-V2X communications between UE 115i, 115j, or 115k and other UEs 115 and / or vehicle-to-infrastructure (V2I) communications between UE 115i, 115j, or 115k and BS 105.

[0045] In some implementations, the network 100 utilizes an OFDM-based waveform for communication. An OFDM-based system may divide the system BW into multiple (K) orthogonal subcarriers, which are also commonly referred to as subcarriers, tones, frequency bins, etc. Each subcarrier may be modulated with data. In some aspects, the subcarrier spacing between adjacent subcarriers may be fixed, and the total number of subcarriers (K) may depend on the system BW. The system BW may also be divided into sub-bands. In other aspects, the subcarrier spacing and / or the duration of the TTI may be scalable.

[0046] In some aspects, BS 105 may assign or schedule transmission resources (e.g., in the form of time-frequency resource blocks (RBs)) for DL ​​and UL transmissions in network 100. DL refers to the transmission direction from BS 105 to UE 115, while UL refers to the transmission direction from UE 115 to BS 105. The communication may be in the form of a radio frame. A radio frame may be divided into a plurality of subframes or time slots, for example, about 10. Each time slot may also be divided into micro-time slots. In FDD mode, simultaneous UL transmission and DL transmission may occur in different frequency bands. For example, each subframe includes a UL subframe in a UL frequency band and a DL subframe in a DL frequency band. In TDD mode, UL transmission and DL transmission occur in different time periods using the same frequency band. For example, a subset of subframes in a radio frame (e.g., a DL subframe) may be used for DL ​​transmission, and another subset of subframes in a radio frame (e.g., a UL subframe) may be used for UL transmission.

[0047] DL subframes and UL subframes may also be divided into several zones. For example, each DL subframe or UL subframe may have a predefined zone for the transmission of reference signals, control information, and data. Reference signals are predetermined signals that facilitate communication between BS 105 and UE 115. For example, reference signals may have a specific pilot pattern or structure, wherein pilot tones may span an operational BW or frequency band, and each pilot tone is located at a predefined time and a predefined frequency. For example, BS 105 may send a cell-specific reference signal (CRS) and / or a channel state information-reference signal (CSI-RS) to enable UE 115 to estimate a DL channel. Similarly, UE 115 may send a sounding reference signal (SRS) to enable BS 105 to estimate an UL channel. Control information may include resource assignments and protocol controls. Data may include protocol data and / or operational data. In some aspects, BS 105 and UE 115 may communicate using self-contained subframes. Self-contained subframes may include a portion for DL ​​communication and a portion for UL communication. A self-contained subframe may be DL-centric or UL-centric. A DL-centric subframe may include a DL communication duration that is longer than a UL communication duration. A UL-centric subframe may include a UL communication duration that is longer than a DL communication duration.

[0048] In some aspects, the network 100 may be an NR network deployed on a licensed spectrum. The BS 105 may send synchronization signals (e.g., including a primary synchronization signal (PSS) and a secondary synchronization signal (SSS)) in the network 100 to facilitate synchronization. The BS 105 may broadcast system information associated with the network 100 (e.g., including a master information block (MIB), residual system information (RMSI), and other system information (OSI)) to facilitate initial network access. In some aspects, the BS 105 may broadcast the PSS, SSS, and / or MIB in the form of a synchronization signal block (SSB), and may broadcast the RMSI and / or OSI on a physical downlink shared channel (PDSCH). The MIB may be sent on a physical broadcast channel (PBCH).

[0049] In some aspects, a UE 115 attempting to access the network 100 may perform an initial cell search by detecting the PSS from the BS 105. The PSS may enable synchronization of period timing and may indicate a physical layer identification value. The UE 115 may then receive the SSS. The SSS may enable radio frame synchronization and may provide a cell identification value that may be combined with the physical layer identification value to identify a cell. The PSS and SSS may be located in the center portion of a carrier or at any suitable frequency within a carrier.

[0050] After receiving the PSS and SSS, the UE 115 may receive the MIB. The MIB may include system information for initial network access and scheduling information for RMSI and / or OSI. After decoding the MIB, the UE 115 may receive the RMSI and / or OSI. The RMSI and / or OSI may include radio resource control (RRC) information related to random access channel (RACH) procedures, paging, control resource sets (CORESET) for physical downlink control channel (PDCCH) monitoring, physical UL control channel (PUCCH), physical UL shared channel (PUSCH), power control, and SRS.

[0051] After obtaining the MIB, RMSI, and / or OSI, the UE 115 may perform a random access procedure to establish a connection with the BS 105. In some cases, the random access procedure may be a four-step random access procedure. For example, the UE 115 may send a random access preamble, and the BS 105 may respond with a random access response. The random access response (RAR) may include a detected random access preamble identifier (ID) corresponding to the random access preamble, timing advance (TA) information, UL authorization, a temporary cell radio network temporary identifier (C-RNTI), and / or a backoff indicator. Upon receiving the random access response, the UE 115 may send a connection request to the BS 105 and the BS 105 may respond with a connection response. The connection response may indicate contention resolution. In some cases, the random access preamble, the RAR, the connection request, and the connection response may be referred to as message 1 (MSG1), message 2 (MSG2), message 3 (MSG3), and message 4 (MSG4), respectively. In some cases, the random access procedure may be a two-step random access procedure, where the UE 115 may send a random access preamble and a connection request in a single transmission, and the BS 105 may respond by sending a random access response and a connection response in a single transmission.

[0052] After establishing the connection, UE 115 and BS 105 may enter a normal operation phase, in which operational data may be exchanged. For example, BS 105 may schedule UE 115 for UL and / or DL ​​communication. BS 105 may send UL scheduling grant and / or DL ​​scheduling grant to UE 115 via PDCCH. Scheduling grant may be sent in the form of DL control information (DCI). BS 105 may send DL communication signals (e.g., carrying data) to UE 115 via PDSCH according to the DL scheduling grant. UE 115 may send UL communication signals to BS 105 via PUSCH and / or PUCCH according to the UL scheduling grant. The connection may be referred to as an RRC connection. When UE 115 actively exchanges data with BS 105, UE 115 is in an RRC connection state.

[0053] In some aspects, after establishing a connection with BS 105, UE 115 may initiate an initial network attachment process with network 100. BS 105 may coordinate with various network entities or fifth generation core (5GC) entities, such as access and mobility function (AMF), serving gateway (SGW) and / or packet data network gateway (PGW) to complete the network attachment process. For example, BS 105 may coordinate with network entities in 5GC to identify UE, authenticate UE and / or authorize UE to transmit and / or receive data in network 100. In addition, AMF may assign a set of tracking areas (TAs) to UE. Once the network attachment process is successful, a context is established in AMF for UE 115. After successfully attaching to the network, UE 115 may move around the current TA. For tracking area update (TAU), BS 105 may request UE 115 to periodically update network 100 with the location of UE 115. Alternatively, UE 115 may report the location of UE 115 to network 100 only when entering a new TA. The TAU allows the network 100 to quickly locate and page the UE 115 when an incoming data packet or call for the UE 115 is received.

[0054] In some aspects, BS 105 may communicate with UE 115 using HARQ technology to improve communication reliability, for example, to provide URLLC services. BS 105 may schedule UE 115 for PDSCH communication by sending DL grants in PDCCH. BS 105 may send DL data packets to UE 115 according to the scheduling in PDSCH. DL data packets may be sent in the form of transport blocks (TBs). After receiving the DL data packets, UE 115 may send a feedback message for the DL data packets to BS 105. In some cases, UE 115 may send feedback on confirmation resources. The feedback may be an acknowledgment (ACK) indicating that UE 115 successfully receives the DL data packet (e.g., DL data is received without error) or may be a negative acknowledgment (NACK) indicating that UE 115 does not successfully receive the DL data packet (e.g., including errors or error correction failures). In some aspects, if UE 115 successfully receives the DL data packet, UE 115 may send a HARQ ACK to BS 105. Conversely, if UE 115 fails to successfully receive the DL transmission, UE 115 may send a HARQ NACK to BS 105. Upon receiving the HARQ NACK from UE 115, BS 105 may retransmit the DL data packet to UE 115. The retransmission may include a decoded version of the DL data that is the same as the initial transmission. Alternatively, the retransmission may include a decoded version of the DL data that is different from the initial transmission. UE 115 may apply soft combining to combine the coded data received from the initial transmission and the retransmission for decoding. BS 105 and UE 115 may also apply HARQ to UL communications using a mechanism substantially similar to DL HARQ.

[0055] In some aspects, the network 100 may operate on a system BW or a component carrier (CC) BW. The network 100 may divide the system BW into multiple BWPs (e.g., parts). The BS 105 may dynamically assign the UE 115 to operate on a specific BWP (e.g., a specific part of the system BW). The assigned BWP may be referred to as an active BWP. The UE 115 may monitor the active BWP to find signaling information from the BS 105. The BS 105 may schedule the UE 115 to perform UL or DL ​​communications in the active BWP. In some aspects, the BS 105 may assign a pair of BWPs within the CC to the UE 115 for UL and DL communications. For example, the BWP pair may include one BWP for UL communication and one BWP for DL ​​communication.

[0056] The deployment of a communication system (such as a 5G New Radio (NR) system) can be arranged with various components or constituent parts in a variety of ways. In a 5G NR system or network, a network node, a network entity, a mobility element of a network, a radio access network (RAN) node, a core network node, a network element or a network equipment (such as a base station (BS), or one or more units (or one or more components) performing base station functionality) can be implemented in an aggregated architecture or a decomposed architecture. For example, a BS (such as a Node B (NB), an evolved NB (eNB), an NR BS, a 5G NB, an access point (AP), a transmit receive point (TRP) or a cell, etc.) can be implemented as an aggregated base station (also known as an independent BS or a monolithic BS) or a decomposed base station.

[0057] A converged base station may be configured to utilize a radio protocol stack that is physically or logically integrated within a single RAN node. A decomposed base station may be configured to utilize a protocol stack that is physically or logically distributed between two or more units, such as one or more central or centralized units (CUs), one or more distributed units (DUs), or one or more radio units (RUs). In some aspects, a CU may be implemented within a RAN node, and one or more DUs may be co-located with the CU, or alternatively, may be geographically or virtually distributed in one or more other RAN nodes. A DU may be implemented to communicate with one or more RUs. Each of the CU, DU, and RU may also be implemented as a virtual unit, i.e., a virtual central unit (VCU), a virtual distributed unit (VDU), or a virtual radio unit (VRU).

[0058] Base station type operations or network designs may take into account the aggregated nature of base station functionality. For example, a disaggregated base station may be used in an integrated access backhaul (IAB) network, an open radio access network (O-RAN (network configurations such as those initiated by the O-RAN Alliance)), or a virtualized radio access network (vRAN, also known as a cloud radio access network (C-RAN)). Disaggregation may include distributing functionality across two or more units at various physical locations, as well as virtually distributing functionality of at least one unit, which may enable flexibility in network design. Individual units of a disaggregated base station or disaggregated RAN architecture may be configured for wired or wireless communication with at least one other unit.

[0059] Figure 2A diagram illustrating an example disaggregated base station 200 architecture is shown. The disaggregated base station 200 architecture may include one or more central units (CUs) 210 that may communicate directly with a core network 220 via a backhaul link, or indirectly with the core network 220 through one or more disaggregated base station units, such as a near real-time (near-RT) RAN intelligent controller (RIC) 225 via an E2 link, or a non-real-time (non-RT) RIC 215 associated with a service management and orchestration (SMO) framework 205, or both. The CU 210 may communicate with one or more distributed units (DUs) 230 via respective midhaul links, such as an F1 interface. The DU 230 may communicate with one or more radio units (RUs) 240 via respective fronthaul links. The RU 240 may communicate with respective UEs 115 via one or more radio frequency (RF) access links. In some implementations, a UE 115 may be served simultaneously by multiple RUs 240.

[0060] Each of the units (i.e., CU 210, DU 230, RU 240, and near-RT RIC 225, non-RT RIC 215, and SMO framework 205) may include one or more interfaces, or be coupled to one or more interfaces configured to receive or send signals, data, or information (collectively referred to as signals) via a wired or wireless transmission medium. Each of the units or an associated processor or controller that provides instructions to the communication interface of these units may be configured to communicate with one or more of the other units via a transmission medium. For example, these units may include a wired interface that is configured to receive or send signals to one or more of the other units via a wired transmission medium. Additionally, these units may include a wireless interface that may include a receiver, transmitter, or transceiver (such as a radio frequency (RF) transceiver) that is configured to receive or send signals, or both, to one or more of the other units on a wireless transmission medium.

[0061] In some aspects, the CU 210 may host one or more higher layer control functions. Such control functions may include radio resource control (RRC), packet data convergence protocol (PDCP), service data adaptation protocol (SDAP), etc. Each control function may be implemented using an interface that is configured to communicate signals with other control functions hosted by the CU 210. The CU 210 may be configured to handle user plane functionality (i.e., central unit-user plane (CU-UP)), control plane functionality (i.e., central unit-control plane (CU-CP)), or a combination thereof. In some specific implementations, the CU 210 may be logically split into one or more CU-UP units and one or more CU-CP units. When implemented in an O-RAN configuration, the CU-UP unit may communicate bidirectionally with the CU-CP unit via an interface (such as an E1 interface). The CU 210 may be implemented to communicate with the DU 230 as needed for network control and signaling.

[0062] DU 230 may correspond to a logical unit that includes one or more base station functions for controlling the operation of one or more RUs 240. In some aspects, DU 230 may host one or more of a radio link control (RLC) layer, a medium access control (MAC) layer, and one or more high physical (PHY) layers (such as modules for forward error correction (FEC) encoding and decoding, scrambling, modulation and demodulation, etc.) at least in part according to a functional split such as those defined by the Third Generation Partnership Project (3GPP). In some aspects, DU 230 may further host one or more low PHY layers. Each layer (or module) may be implemented using an interface that is configured to communicate signals with other layers (and modules) hosted by DU 230 or with control functions hosted by CU 210.

[0063] The lower layer functionality may be implemented by one or more RUs 240. In some deployments, the RU 240 controlled by the DU 230 may correspond to a logical node that hosts RF processing functions or low PHY layer functions (such as performing fast Fourier transform (FFT), inverse FFT (iFFT), digital beamforming, physical random access channel (PRACH) extraction and filtering, etc.), or both, based at least in part on functional splitting (such as lower layer functional splitting). In such an architecture, the RU 240 may be implemented to handle over-the-air (OTA) communications with one or more UEs 115. In some specific implementations, real-time and non-real-time aspects of control plane and user plane communications with the RU 240 may be controlled by the corresponding DU 230. In some scenarios, this configuration may enable the DU 230 and CU 210 to be implemented in a cloud-based RAN architecture (such as a vRAN architecture).

[0064] The SMO framework 205 may be configured to support RAN deployment and provisioning of non-virtualized network elements and virtualized network elements. For non-virtualized network elements, the SMO framework 205 may be configured to support the deployment of dedicated physical resources for RAN coverage requirements, which may be managed via an operation and maintenance interface (such as an O1 interface). For virtualized network elements, the SMO framework 205 may be configured to interact with a cloud computing platform (such as an open cloud (O-Cloud) 290) to perform network element lifecycle management (such as instantiating virtualized network elements) via a cloud computing platform interface (such as an O2 interface). Such virtualized network elements may include, but are not limited to, CU 210, DU 230, RU 240, and near-RT RIC 225. In some specific implementations, the SMO framework 205 may communicate with hardware aspects of the 4G RAN (such as an open eNB (O-eNB) 211) via the O1 interface. Additionally, in some specific implementations, the SMO framework 205 may communicate directly with one or more RUs 240 via the O1 interface. The SMO framework 205 may also include a non-RT RIC 215 configured to support the functionality of the SMO framework 205 .

[0065] The non-RT RIC 215 may be configured to include logic functions that enable non-real-time control and optimization of RAN elements and resources, artificial intelligence / machine learning (AI / ML) workflows including model training and updating, or policy-based guidance of applications / features in the near-RT RIC 225. The non-RT RIC 215 may be coupled to or in communication with the near-RT RIC 225 (such as via an A1 interface). The near-RT RIC 225 may be configured to include logic functions that enable near-real-time control and optimization of RAN elements and resources via data collection and actions through an interface (such as via an E2 interface) that connects one or more CUs 210, one or more DUs 230, or both, and the O-eNB with the near-RT RIC 225.

[0066] In some implementations, in order to generate an AI / ML model to be deployed in the near-RT RIC 225, the non-RT RIC 215 may receive parameters or external enrichment information from an external server. Such information may be utilized by the near-RT RIC 225 and may be received from a non-network data source or from a network function at the SMO framework 205 or the non-RT RIC 215. In some examples, the non-RT RIC 215 or the near-RT RIC 225 may be configured to tune RAN behavior or performance. For example, the non-RT RIC 215 may monitor long-term trends and patterns of performance and employ AI / ML models to perform corrective actions through the SMO framework 205 (such as via reconfiguration of O1) or via creation of RAN management policies (such as A1 policies).

[0067] Figure 3 A time domain beam prediction scheme 300 is illustrated in accordance with some aspects of the present disclosure. The time domain beam prediction scheme 300 illustrates aspects of using a machine learning (ML) model to predict one or more beam characteristics in accordance with the present disclosure. In this regard, aspects of the time domain beam prediction scheme 300 may be utilized in the context of the wireless communication network 100 and with other aspects of the present disclosure, including the spatial domain beam prediction schemes 400 and 500 and the ML model monitoring schemes 600 and 700.

[0068] As shown, the BS 105 may periodically transmit one or more reference signals (e.g., downlink reference signals, CSI-RS, CRS, SSB, etc.). In the illustrated example, the BS 105 transmits a nominal reference signal group 305a, 305b, 305c at a period 310 (e.g., 10ms, 20ms, 40ms, or any other suitable period). In some aspects, the period 310 may be longer than a standard reference signal period (e.g., twice the period, or other), which may enable power savings, reduced network congestion, and / or reduced interference by omitting one or more transmissions of reference signals relative to a standard reference signal period.

[0069] The UE 115 may utilize one or more ML models 312 to predict one or more beam parameters based on the reference signal received from the BS 105. Unless otherwise specified, it should be understood that references to ML models in the present disclosure include any type of program that relies on machine learning, including but not limited to ML models, artificial intelligence (AI) models, AI / ML models, supervised learning models, unsupervised learning models, reinforcement learning models, semi-supervised learning models, self-supervised learning models, multiple instance learning models, inductive learning models, deductive reasoning models, direct learning models, multi-task learning models, active learning models, online learning models, transfer learning models, holistic learning models, and / or combinations thereof. In addition, the ML model may include neural networks implemented at different types of nodes within the wireless communication network. For example, the neural network may be implemented at a single node (e.g., UE / BS / central cloud server) or may be distributed across multiple nodes. The ML algorithm may be implemented to assist different functions and / or modules between nodes of the wireless communication network. In various aspects, the neural network may be implemented as a convolutional neural network (CNN), a recurrent neural network (RNN), a deep convolutional network (DCN), and the like.

[0070] The UE 115 may utilize the ML model 312 to predict one or more beam parameters of a predicted beam group (e.g., predicted beam group 315a, 315b, or 315c) based on a nominal reference signal group (e.g., nominal reference signal group 305a, 305b, or 305c). In this regard, the ML model 312 executed by the UE 115 may utilize measurements and / or other information associated with the nominal reference signal group (e.g., 305a, 305b, or 305c) along with other relevant parameters (e.g., UE mobility, UE location, etc.) and / or previously acquired data to determine one or more beam parameters (e.g., predicted beam measurements, predicted beam ranking order, etc.) of the predicted beam group (e.g., 315a, 315b, or 315c).

[0071] The predicted beam groups may be associated with future reference signal monitoring opportunities (eg, reference signal monitoring opportunities associated with the nominal reference signal groups 305b or 305c) and / or between reference signal monitoring opportunities. Figure 3An example is illustrated in which a predicted beam group (e.g., predicted beam groups 315a and 315b) is associated with a time period between reference signal monitoring opportunities (e.g., between reference signal monitoring opportunities associated with nominal reference signal groups 305a and 305b for predicted beam group 315a and between reference signal monitoring opportunities associated with nominal reference signal groups 305b and 305c for predicted beam group 315b). In some cases, the predicted beam group is associated with a time period between reference signal monitoring opportunities in which reference signal transmission is omitted but would occur if standard reference signal periodicity were used.

[0072] The BS 105 may periodically transmit one or more auxiliary reference signal groups 320a, 320b, 320c during the ML model evaluation period 325. In some cases, the auxiliary reference signal groups 320a, 320b, 320c are separated from the nominal reference signal groups 305i, 305j, 305k by a period 330 (e.g., 5 ms, 10 ms, 20 ms, or any other suitable period). In some aspects, the period 330 may be a standard reference signal period. For example, in some cases, the auxiliary reference signal groups 320a, 320b, 320c may be transmitted in standard reference signal transmission opportunities that are omitted outside of the ML model evaluation period 325. Additionally, in some cases, the ML model evaluation period 325 may occur periodically (e.g., based on a period 335 (e.g., 100 ms, 500 ms, or other suitable period)) and / or ad hoc. In some cases, the BS 105 may indicate the timing of the ML model evaluation period 325 in an ML model monitoring configuration, a radio resource control (RRC) message, and / or other suitable communications.

[0073] The auxiliary reference signal groups 320a, 320b, 320c may be used by the UE 115 to evaluate the performance of the ML model 312. For example, the UE 115 may utilize the ML model 312 to predict one or more beam parameters of a predicted beam group (e.g., predicted beam group 315i, 315j, or 315k) based on a nominal reference signal group (e.g., nominal reference signal group 305i, 305j, or 305k). Each predicted beam group (e.g., predicted beam group 315i, 315j, or 315k) may be associated with an auxiliary reference signal group (e.g., auxiliary reference signal group 320a, 320b, or 320c). In this regard, the performance of the ML model may be evaluated using one or more measurements of the auxiliary reference signal group with which the predicted beam group is associated. For example, as discussed further below, measurements of the auxiliary reference signal group may be compared to measurements of predicted beams of the ML model 312 and / or compared to predicted measurements of predicted beams of the ML model 312 to evaluate the performance of the ML model 312. In this regard, the comparison may indicate that an ML model failure instance (MFI) has occurred. If a sufficient number of MFIs occur over a period of time (e.g., based on an ML model failure detection (MFD) timer), this may be an indication that the ML model 312 has failed. In this manner, the occurrence of one or more events associated with the ML model (e.g., MFI and / or model failure) may be detected.

[0074] As discussed further below, in response to detecting an event such as a failure of the ML model 312, the UE may stop the ML model 312, initiate retraining of the ML model 312, and / or send an indication of the failure of the ML model 312 to the BS 105. In some cases, the UE may determine that the ML model 312 is operating properly, and may continue to run the ML model 312 and / or send an indication of proper operation of the ML model 312 to the BS 105.

[0075] Figure 4 A spatial beam prediction scheme 400 according to some aspects of the present disclosure is illustrated. The spatial beam prediction scheme 400 illustrates various aspects of using a machine learning (ML) model to predict one or more beam characteristics according to the present disclosure. In this regard, various aspects of the spatial beam prediction scheme 400 can be utilized in the context of the wireless communication network 100 and with other aspects of the present disclosure, including the time domain beam prediction scheme 300, the spatial beam prediction scheme 500, and the ML model monitoring schemes 600 and 700.

[0076] As shown, BS 105 may periodically transmit one or more reference signals (e.g., downlink reference signals, CSI-RS, CRS, SSB, etc.). In the illustrated example, BS 105 transmits nominal reference signal groups 405a, 405b, 405c in a period 410 (e.g., 10ms, 20ms, 40ms, or any other suitable period). In some aspects, period 410 may be longer than a standard reference signal period (e.g., twice the period, or other), which may enable power savings, reduced network congestion, and / or reduced interference by omitting one or more transmissions of reference signals relative to a standard reference signal period. As shown, for each nominal reference signal group 405a, 405b, and 405c, the BS may transmit reference signals using less than all available and / or active beam directions. In this regard, the beam direction transmitted for each of the nominal reference signal groups 405a, 405b, and 405c may include the same and / or different beam directions as the other nominal reference signal groups 405a, 405b, and 405c.

[0077] The UE 115 may utilize one or more ML models 412 to predict one or more beam parameters based on the nominal reference signal received from the BS 105. The UE 115 may utilize the ML model 412 to predict one or more beam parameters of a predicted beam group (e.g., predicted beam group 415a, 415b, or 415c) based on the nominal reference signal group (e.g., nominal reference signal group 405a, 305b, or 305c). In this regard, the ML model 412 executed by the UE 115 may utilize measurements and / or other information associated with the nominal reference signal group (e.g., 405a, 405b, or 405c) along with other relevant parameters (e.g., UE mobility, UE location, etc.) and / or previously acquired data to determine one or more beam parameters (e.g., predicted beam measurements, predicted beam ranking order, etc.) of the predicted beam group (e.g., 415a, 415b, or 415c). More specifically, the UE 115 may utilize the ML model 412 to estimate one or more beam parameters for beam directions that are not transmitted as part of reference signals in the corresponding nominal reference signal groups 405a, 405b, and 405c. In this regard, the solid lines in the predicted beam groups 415a, 415b, and 415c, the UE 115 represent beam directions that are transmitted as part of reference signals in the corresponding nominal reference signal groups 405a, 405b, and 405c, while the dashed lines represent beam directions that are not transmitted by the corresponding nominal reference signal groups 405a, 405b, and 405c.

[0078] The BS 105 may periodically transmit one or more auxiliary reference signal groups 420a, 420b, 420c during the ML model evaluation period 425. In some cases, the auxiliary reference signal groups 420a, 420b, 420c are separated from the nominal reference signal groups 405i, 405j, 405k by a period 430 (e.g., 5 ms, 10 ms, 20 ms, or any other suitable period). In some aspects, the period 430 may be a standard reference signal period. For example, in some cases, the auxiliary reference signal groups 420a, 420b, 420c may be transmitted in standard reference signal transmission opportunities that are omitted outside of the ML model evaluation period 425. Additionally, in some cases, the ML model evaluation period 425 may occur periodically (e.g., based on a period 435 (e.g., 100 ms, 500 ms, or other suitable period)) and / or ad hoc. In some cases, the BS 105 may indicate the timing of the ML model evaluation period 425 in an ML model monitoring configuration, a radio resource control (RRC) message, and / or other suitable communications.

[0079] The auxiliary reference signal groups 420a, 420b, 420c may be used by the UE 115 to evaluate the performance of the ML model 412. In this regard, as shown, the auxiliary reference signal groups 420a, 420b, 420c may transmit reference signals in all available and / or active beam directions, or at least in more beam directions than the nominal reference signal groups 405i, 405j, 405k. One or more measurements of the auxiliary reference signal groups may be utilized to evaluate the performance of the ML model 412 based on the predicted beam groups 415i, 415j, and 415k. For example, when generating the predicted beam groups 415i, 415j, and 415k, the UE 115 may utilize the ML model 412 to predict one or more beam parameters for beam directions that are not transmitted as part of the nominal reference signal groups (e.g., the nominal reference signal groups 405i, 405j, or 405k). As discussed further below, the measurements of the auxiliary reference signal group may be compared to the measurements of the predicted beams of the ML model 412 and / or compared to the predicted measurements of the predicted beams of the ML model 412 to evaluate the performance of the ML model 412. In this regard, the comparison may indicate that an ML model failure instance (MFI) has occurred. If a sufficient number of MFIs occur over a period of time (e.g., based on an ML model failure detection (MFD) timer), this may be an indication that the ML model 412 has failed. In this manner, the occurrence of one or more events associated with the ML model (e.g., MFI and / or model failure) may be detected.

[0080] As discussed further below, in response to detecting an event such as a failure of the ML model 412, the UE may stop the ML model 412, initiate retraining of the ML model 412, and / or send an indication of the failure of the ML model 412 to the BS 105. In some cases, the UE may determine that the ML model 412 is operating properly, and may continue to run the ML model 412 and / or send an indication of proper operation of the ML model 412 to the BS 105.

[0081] Figure 5 A spatial beam prediction scheme 500 in accordance with one or more aspects of the present disclosure is illustrated. The spatial beam prediction scheme 500 illustrates various aspects of using a machine learning (ML) model to predict one or more beam characteristics in accordance with the present disclosure. In this regard, various aspects of the spatial beam prediction scheme 500 may be utilized in the context of the wireless communication network 100 and with other aspects of the present disclosure, including the time domain beam prediction scheme 300, the spatial beam prediction scheme 400, and the ML model monitoring schemes 600 and 700.

[0082] As shown, BS 105 may periodically transmit one or more reference signals (e.g., downlink reference signals, CSI-RS, CRS, SSB, etc.). In the illustrated example, BS 105 transmits nominal reference signal groups 505a, 505b, 505c in a period 510 (e.g., 10ms, 20ms, 40ms, or any other suitable period). In some aspects, period 510 may be longer than a standard reference signal period (e.g., twice the period, or other), which may enable power savings, reduced network congestion, and / or reduced interference by omitting one or more transmissions of reference signals relative to a standard reference signal period. As shown, for each nominal reference signal group 505a, 505b, and 505c, the BS uses a wideband beam. In this regard, the wideband beam transmitted for each of the nominal reference signal groups 505a, 505b, and 505c may include the same and / or different wideband beams as the other nominal reference signal groups 505a, 505b, and 505c.

[0083] The UE 115 may utilize one or more ML models 512 to predict one or more beam parameters based on the nominal reference signals received from the BS 105. The UE 115 may utilize the ML models 512 to predict one or more beam parameters of a predicted beam group (e.g., predicted beam group 515a, 515b, or 515c) based on the nominal reference signal group (e.g., nominal reference signal group 505a, 505b, or 505c). In this regard, the ML models 512 executed by the UE 115 may utilize measurements and / or other information associated with the nominal reference signal group (e.g., 505a, 505b, or 505c) along with other relevant parameters (e.g., UE mobility, UE location, etc.) and / or previously acquired data to determine one or more beam parameters (e.g., predicted beam measurements, predicted beam ranking order, etc.) of the predicted beam group (e.g., 515a, 515b, or 515c). More specifically, the UE 115 may utilize the ML model 512 to estimate one or more beam parameters of a narrowband beam that is not transmitted as part of the wideband reference signal in the corresponding nominal reference signal groups 505a, 505b, and 505c. Figure 5 The transmission of a wideband reference signal and the prediction of a narrowband beam are shown, but in other aspects of the disclosure, a narrowband reference signal may be transmitted and an ML model may be utilized to predict one or more beam parameters of the wideband beam based on the narrowband reference signal.

[0084] The BS 105 may periodically transmit one or more auxiliary reference signal groups 520a, 520b, 520c during the ML model evaluation period 525. In some cases, the auxiliary reference signal groups 520a, 520b, 520c are separated from the nominal reference signal groups 505i, 505j, 505k by a period 530 (e.g., 5 ms, 10 ms, 20 ms, or any other suitable period). In some aspects, the period 530 may be a standard reference signal period. For example, in some cases, the auxiliary reference signal groups 520a, 520b, 520c may be transmitted in standard reference signal transmission opportunities that are omitted outside of the ML model evaluation period 525. Additionally, in some cases, the ML model evaluation period 525 may occur periodically (e.g., based on a period 535 (e.g., 100 ms, 500 ms, or other suitable period)) and / or ad hoc. In some cases, the BS 105 may indicate the timing of the ML model evaluation period 525 in an ML model monitoring configuration, a radio resource control (RRC) message, and / or other suitable communications.

[0085] The auxiliary reference signal groups 520a, 520b, 520c may be used by the UE 115 to evaluate the performance of the ML model 512. In this regard, as shown, the auxiliary reference signal groups 520a, 520b, 520c may transmit narrowband reference signals based on the nominal reference signal groups 505i, 505j, 505k in one or more of the available and / or active beam directions and / or in one or more of the beam directions predicted by the ML model 512 for the predicted beam groups 515i, 515j, 515k. In this regard, one or more measurements of the auxiliary reference signal groups may be utilized to evaluate the performance of the ML model 512 based on the predicted beam groups 515i, 515j, and 515k. For example, when generating predicted beam groups 515i, 515j, and 515k, the UE 115 may utilize the ML model 512 to predict one or more beam parameters of narrowband beam directions that are not transmitted as part of a nominal reference signal group (e.g., nominal reference signal groups 505i, 505j, or 505k). As discussed further below, measurements of the auxiliary reference signal group may be compared to measurements of predicted beams of the ML model 512 and / or compared to predicted measurements of predicted beams of the ML model 512 to evaluate the performance of the ML model 512. In this regard, the comparison may indicate that an ML model failure instance (MFI) has occurred. If a sufficient number of MFIs occur within a period of time (e.g., based on an ML model failure detection (MFD) timer), this may be an indication that the ML model 412 has failed. In this manner, the occurrence of one or more events associated with the ML model (e.g., MFI and / or model failure) may be detected.

[0086] As discussed further below, in response to detecting an event such as a failure of the ML model 512, the UE may stop the ML model 512, initiate retraining of the ML model 512, and / or send an indication of the failure of the ML model 512 to the BS 105. In some cases, the UE may determine that the ML model 512 is operating properly, and may continue to run the ML model 512 and / or send an indication of proper operation of the ML model 512 to the BS 105.

[0087] Figure 6 A signaling diagram of a machine learning (ML) model monitoring scheme 600 according to one or more aspects of the present disclosure is illustrated. The ML model monitoring scheme 600 illustrates various aspects of monitoring the performance of an ML model according to the present disclosure. In this regard, various aspects of the ML model monitoring scheme 600 may be utilized in the context of the wireless communication network 100 and with other aspects of the present disclosure, including the time domain beam prediction scheme 300, the spatial domain beam prediction schemes 400 and 500, and the ML model monitoring scheme 700.

[0088] At action 605, the UE 115 runs an ML model. The ML model run by the UE may include any type of program that relies on machine learning, including but not limited to an ML model, an artificial intelligence (AI) model, an AI / ML model, a supervised learning model, an unsupervised learning model, a reinforcement learning model, a semi-supervised learning model, a self-supervised learning model, a multi-instance learning model, an inductive learning model, a deductive reasoning model, a direct learning model, a multi-task learning model, an active learning model, an online learning model, a transfer learning model, an overall learning model, and / or a combination thereof. In addition, the ML model may include a neural network implemented at different types of nodes within the wireless communication network. For example, the neural network may be implemented at a single node (e.g., UE / BS / central cloud server) or may be distributed across multiple nodes. The ML algorithm may be implemented to assist different functions and / or modules between nodes of the wireless communication network. In various aspects, the neural network may be implemented as a convolutional neural network (CNN), a recursive neural network (RNN), a deep convolutional network (DCN), and the like.

[0089] In some aspects of the present disclosure, the ML model is configured to predict one or more beam parameters, including but not limited to predicted beam measurements (e.g., RSRP (e.g., L1-RSRP), RSRQ, RSSI, SINR, etc.), predicted beam ranking order, narrowband beams, wideband beams, etc. In some cases, the ML model may include an ML model used by the UE for beam prediction. For example, the ML model may be configured to provide time domain beam prediction (e.g., as described with respect to Figure 3 ) and / or spatial beam prediction (e.g., as discussed in Figure 4 and Figure 5 In some cases, the UE 115 may run multiple ML models. The multiple ML models may perform similar and / or different functions.

[0090] At act 610, BS 105 sends the ML model monitoring configuration to UE 115. In some cases, act 610 occurs before act 605. That is, in some cases, UE 115 may receive the ML model monitoring configuration before starting the ML model. In other cases, UE 115 may start the ML model before receiving the ML model monitoring configuration. In some aspects, UE 115 may receive an initial ML model monitoring configuration and subsequently receive an updated ML model monitoring configuration. In this regard, BS 105 may update one or more parameters of the ML model monitoring configuration and send the updated ML model monitoring configuration and / or an indication of the updated parameters.

[0091] The UE may receive the ML model monitoring configuration and / or any updates to the ML model monitoring configuration from the network element via a radio resource control (RRC) message or other suitable communication. In some aspects, the ML model monitoring configuration may be included as an information element of the communication. The ML model monitoring configuration may include one or more parameters associated with ML model monitoring and / or reporting. For example, the ML model monitoring configuration may include an indication of one or more of the following: ML model fault detection (MFD) timer duration, the maximum number of ML model fault instances (MFI), one or more values ​​associated with one or more MFI criteria, and / or other parameters associated with ML model monitoring and / or reporting. That is, the ML model monitoring configuration may include an indication of one or more parameters and / or criteria utilized to detect the occurrence of one or more events associated with the ML model. The UE may utilize information from the ML model monitoring configuration to evaluate the performance of one or more ML models.

[0092] At act 615, UE 115 performs ML model monitoring based on the ML model monitoring configuration. In some cases, at act 620, UE 115 performs ML model monitoring based on one or more ML monitoring signals sent by BS 105. The ML monitoring signals may include one or more reference signals, such as downlink reference signals, CSI-RS, CRS, SSB, etc. In some cases, BS 105 may periodically send nominal reference signals and / or auxiliary reference signals, as described above with respect to Figures 3 to 5 described.

[0093] In some cases, at act 615, the UE evaluates one or more values ​​associated with the prediction of the ML model as one or more measured values. In some aspects, the UE evaluates the values ​​associated with the prediction of the ML model as measured values ​​based at least in part on the ML model monitoring configuration received at act 610. In this regard, the UE may utilize information from the ML model monitoring configuration to evaluate the performance of the ML model. For example, the UE may evaluate one or more values ​​associated with the prediction of the ML model based on an indication of an ML model failure detection (MFD) timer duration, an indication of a maximum number of ML model failure instances (MFIs), an indication of one or more values ​​associated with one or more MFI criteria, and / or other parameters indicated in the ML model monitoring configuration.

[0094] In some aspects, the UE may detect one or more events, such as one or more ML model failure instances (MFIs) and / or failures of the ML model based on the evaluation. In this way, the UE may detect events (MFIs and / or failures of the ML model) based on the ML model monitoring configuration. In some cases, the UE detects an initial MFI based on information from the ML model monitoring configuration. In this regard, the UE may determine whether the MFI criteria are met for each of the plurality of monitoring occasions. In some aspects, the UE determines whether the MFI criteria are met by evaluating one or more values ​​associated with the prediction of the ML model relative to one or more measured values.

[0095] In some cases, at act 615, the UE evaluates one or more measured values ​​associated with the ML model's prediction as one or more measured values ​​associated with the measured beam set. In some cases, the UE may determine whether the ML model's 1st ranked predicted beam is included in the set of top K ranked beams in the measured beam set. For example, referring back to Figure 3 , the UE may determine whether measurements (e.g., RSRP (e.g., L1-RSRP), RSRQ, RSSI, SINR, etc.) associated with the 1st-ranked predicted beam in the predicted beam group 315i are within the top K beams (e.g., the top 1, 2, 3, 4, 5, etc. beams) in the measured beam group of the auxiliary reference signal group 320a. In this regard, the measurements associated with the 1st-ranked predicted beam in the predicted beam group 315i may be measurements of corresponding beams in the measured beam group of the auxiliary reference signal group 320a. For example, if the ML model predicts that a particular beam (e.g., beam index 2) is the 1st-ranked predicted beam, measurements of the particular beam (e.g., beam index 2) in the measured beam group of the auxiliary reference signal group 320a may be used as measurements associated with the 1st-ranked predicted beam in the predicted beam group 315i.

[0096] In some cases, the MFI criterion may be met if the ML model's 1st-ranked predicted beam is not included in the set of top K beams in the measured beam group. That is, if the ML model's 1st-ranked predicted beam is not included in the set of top K beams in the measured beam group, then MFI exists. For example, if measurements associated with the ML model's 1st-ranked predicted beam (e.g., beam index 2) are not included in the top K (e.g., 1, 2, 3, 4, etc.) beams in the measured beam group of the auxiliary reference signal group 320a, then MFI exists. On the other hand, if the ML model's 1st-ranked predicted beam is included in the set of top K beams in the measured beam group, then the ML model may be considered to be operating properly and MFI does not exist. For example, if the measurements associated with the 1st ranked predicted beam (e.g., beam index 2) of the ML model are included in the top K (e.g., 1, 2, 3, 4, etc.) beams in the measured beam group of the auxiliary reference signal group 320a, then there is no MFI.

[0097] In some cases, the UE may determine whether the 1st ranked measured beam in the measured beam group is included in the set of top K ranked beams in the predicted beam group associated with the prediction of the ML model. Figure 3 , the UE may determine whether the 1st ranked measured beam from the measured beam group of the auxiliary reference signal group 320a is among the top K beams (e.g., the top 1, 2, 3, 4, 5, etc. beams) in the predicted beam group 315i. The determination of the ranking of the beam may be based on RSRP (e.g., L1-RSRP), RSRQ, RSSI, SINR, etc. In this regard, the measurement of the 1st ranked measured beam from the measured beam group of the auxiliary reference signal group 320a may be based on the measurement of the beams in the measured beam group of the auxiliary reference signal group 320a. The measurements associated with the top K predicted beams in the predicted beam group 315i may be the measurements of the corresponding beams in the measured beam group of the auxiliary reference signal group 320a. For example, if the ML model predicts that particular beams (e.g., beam indices 2, 3, and 4) are the top 3 predicted beams, then measurements of those particular beams (e.g., beam indices 2, 3, and 4) in the measured beam group from auxiliary reference signal group 320a can be used as measurements associated with the top K predicted beams in predicted beam group 315i.

[0098] In some cases, the MFI criterion may be met if the 1st ranked measured beam in the measured beam group is not included in the set of top K ranked beams in the predicted beam group associated with the prediction of the ML model. That is, if the 1st ranked measured beam of the ML model is not included in the set of top K ranked beams in the predicted beam group of the ML model, then MFI exists. For example, if the measurement associated with the 1st ranked measured beam (e.g., beam index 2) in the measured beam group of the auxiliary reference signal group 320a is not included in the top K ranked (e.g., 1, 2, 3, 4, etc.) beams in the predicted beam group 315i, then MFI exists. On the other hand, if the 1st ranked measured beam in the measured beam group of the auxiliary reference signal group 320a is included in the set of top K ranked beams in the predicted beam group 315i, then the ML model may be considered to be operating properly and there is no MFI. For example, if measurements associated with the 1st ranked measured beam (e.g., beam index 2) in the measured beam group of the auxiliary reference signal group 320a are included in the top K (e.g., 1, 2, 3, 4, etc.) ranked beams in the predicted beam group 315i, then there is no MFI.

[0099] In some cases, the UE may determine whether the Layer 1 Reference Signal Received Power (L1-RSRP) and / or other measurements (e.g., RSRQ, RSSI, SNIR, etc.) of the ML model's 1st ranked predicted beam is within a threshold difference of the L1-RSRP of the 1st ranked measured beam in the measured beam group. Figure 3 , the UE may determine whether the L1-RSRP associated with the 1st ranked predicted beam in the predicted beam group 315i is within a threshold difference of the measured L1-RSRP of the 1st ranked beam in the measured beam group of the auxiliary reference signal group 320a. In this regard, the L1-RSRP associated with the 1st ranked predicted beam in the predicted beam group 315i may be a measurement of the corresponding beam in the measured beam group of the auxiliary reference signal group 320a. For example, if the ML model predicts that a particular beam (e.g., beam index 2) is the 1st ranked predicted beam, the measurement of the particular beam (e.g., beam index 2) in the measured beam group of the auxiliary reference signal group 320a may be used as the measurement associated with the 1st ranked predicted beam in the predicted beam group 315i.

[0100] In some cases, the MFI criterion is met if the L1-RSRP and / or other measurements of the ML model's 1st ranked predicted beam (e.g., beam index 2) are not within a threshold difference (e.g., 0.5 dB, 1.0 dB, 1.5 dB, or other) of the L1-RSRP of the 1st ranked measured beam in the measured beam group. On the other hand, if the L1-RSRP of the ML model's 1st ranked predicted beam is within a threshold difference of the L1-RSRP of the 1st ranked beam in the measured beam group, the ML model may be considered to be operating properly and MFI does not exist. In some cases, the UE may receive an indication of the threshold difference from the network element. In some aspects, the indication of the threshold difference may be included in the ML model monitoring configuration.

[0101] In some cases, the UE evaluates one or more predicted values ​​associated with the prediction of the ML model as one or more measured values ​​associated with the measured beam group. In some cases, the UE determines whether the predicted layer 1 reference signal received power (L1-RSRP) and / or other measurements (e.g., RSRQ, RSSI, SNIR, etc.) of the ML model's 1st ranked predicted beam is within a threshold difference of the measured L1-RSRP of the 1st ranked measured beam in the measured beam group. For example, re-referencing Figure 3 , the UE may determine whether the predicted L1-RSRP associated with the first-ranked predicted beam in the predicted beam group 315i is within a threshold difference of the measured L1-RSRP of the first-ranked beam in the measured beam group of the auxiliary reference signal group 320a. In this regard, the L1-RSRP associated with the first-ranked predicted beam in the predicted beam group 315i may be a measurement of a corresponding beam in the measured beam group of the auxiliary reference signal group 320a.

[0102] In some aspects, the MFI criterion is met if the predicted L1-RSRP of the 1st ranked predicted beam (e.g., beam index 2) of the ML model is not within a threshold difference (e.g., 0.5dB, 1.0dB, 1.5dB, or other) of the measured L1-RSRP of the 1st ranked measured beam in the measured beam group. On the other hand, if the predicted L1-RSRP of the 1st ranked predicted beam of the ML model is within a threshold difference of the measured L1-RSRP of the 1st ranked beam in the measured beam group, the ML model may be considered to be operating properly and MFI does not exist. In some cases, the UE may receive an indication of a threshold difference between the predicted L1-RSRP and the measured L1-RSRP from a network element. In some aspects, the indication of the threshold difference between the predicted L1-RSRP and the measured L1-RSRP may be included in the ML model monitoring configuration.

[0103] In response to detecting an initial MFI (e.g., based on one or more of the criteria discussed above), the UE may start an ML model failure detection (MFD) timer as part of action 615. In some aspects, the ML model monitoring configuration includes an indication of the duration of the MFD timer. In some aspects, the UE determines that the ML model has failed based on detecting a certain number of MFIs before the end of the MFD timer. For example, if the UE detects a number of MFIs exceeding a threshold before the MFD timer expires (e.g., based on one or more of the criteria discussed above), the UE may determine that the ML model has failed. In this regard, the UE may increment an MFI counter for the initial MFI and / or each of the number of MFIs before the end of the MFD timer. If the MFI counter reaches a threshold, the UE may determine and / or declare that the ML model has failed. In some cases, the ML model monitoring configuration includes an indication of a threshold (e.g., 2, 3, 4, 5, 6, 7, etc.) number of MFIs associated with the ML model failure.

[0104] At action 625, the UE sends an ML model monitoring report. The ML model monitoring report may indicate that the ML model is operating properly, has failed, and / or include one or more operating parameters associated with the ML model (e.g., the number of MFIs detected in one or more ML model evaluation periods (e.g., 335, 435, 535)). For example, in response to detecting a failure of the ML model at action 615, the UE may send an indication of the failure of the ML model at action 625. The UE may send the indication of the failure of the ML model to the BS 105 via an RRC message, a PUCCH communication, a PUSCH communication, and / or other suitable communication.

[0105] At act 630, the UE may adjust the operation of the ML model. For example, in some cases, in response to UE 115 sending an indication of a failure of the ML model to BS 105, UE 115 receives an instruction from BS 105 to deactivate the ML model, retrain the ML model, and / or adjust one or more operating parameters of the ML model. Upon receiving the instruction to deactivate the ML model, retrain the ML model, and / or adjust one or more operating parameters of the ML model, the UE adjusts the operation of the ML model (e.g., stops, retrains, and / or adjusts one or more operating parameters) based on the instruction received from base station 105. In some cases, at act 630, the UE deactivates the ML model, initiates retraining of the ML model, and / or adjusts one or more operating parameters of the ML model based on detecting the failure of the ML model at act 615.

[0106] At act 635, in some aspects, BS 105 may update the reference signal configuration based on the ML model monitoring report received at act 625. In some cases, if the ML model monitoring report notifies BS 105 of a failure of the ML model, BS 105 may update the reference signal configuration for the UE. For example, if BS 105 is transmitting a downsampled set of reference signals (e.g., less than all reference signal beam directions and / or less than all standard reference signal transmission opportunities) and relies on the beam prediction capability of the ML model of UE 115 in some beam management phases, BS 105 may update the reference signal configuration for UE 115 to transmit more reference signals (e.g., more reference signal beam directions and / or more reference signal transmission opportunities, up to and including all reference signal beam directions and / or all standard reference signal transmission opportunities). At act 640, BS 105 may send the updated reference signal configuration to UE 115. UE 115 may then monitor the reference signal from BS 105 using the updated reference signal configuration.

[0107] In some cases, at act 615, the UE may not detect a failure of the ML model. That is, the UE may determine that the ML model is operating properly based on the ML model monitoring at act 615. In some aspects, the UE may send an indication that the ML model is operating properly as part of the ML model monitoring report sent to the BS 105. For example, the UE may send an indication that the ML model is operating properly to the network element via an RRC message, a PUCCH communication, a PUSCH communication, and / or other suitable communication. At act 630, if it is determined at act 615 that the ML model is performing adequately, the UE may continue to operate the ML model as is. In some aspects, even if it is determined at act 615 that the ML model is performing adequately, the UE may adjust one or more operating parameters of the ML model at act 630 in an effort to optimize the accuracy and / or benefit of the ML model.

[0108] Figure 7A flow chart of an ML model monitoring scheme according to one or more aspects of the present disclosure is illustrated. The ML model monitoring scheme 700 illustrates various aspects of a user equipment (UE) monitoring the performance of an ML model according to the present disclosure. In this regard, various aspects of the ML model monitoring scheme 700 may be utilized in the context of the wireless communication network 100 and in conjunction with other aspects of the present disclosure, including the time domain beam prediction scheme 300, the spatial domain beam prediction schemes 400 and 500, and the ML model monitoring scheme 600. Because various aspects of the ML model monitoring scheme 700 may be similar to those discussed above with respect to the ML model monitoring scheme 600 and below with respect to the method 1000, some details are omitted in the following description. For additional details, see the description of the ML model monitoring scheme 600 and below with respect to the method 1000 (or other aspects of the present disclosure).

[0109] At action 705, the UE (e.g., UE 115 or UE 800) runs the ML model. The ML model may be configured to predict one or more beam parameters, including but not limited to predicted beam measurements (e.g., RSRP (e.g., L1-RSRP), RSRQ, RSSI, SINR, etc.), predicted beam ranking order, narrowband beams, wideband beams, etc. In some cases, the ML model may include an ML model used by the UE for beam prediction. For example, the ML model may be configured to provide time domain beam prediction (e.g., as for Figure 3 ) and / or spatial beam prediction (e.g., as discussed in Figure 4 and Figure 5 In some cases, the UE 115 may run multiple ML models. The multiple ML models may perform similar and / or different functions.

[0110] At action 710, the UE receives an ML model monitoring configuration from a network element. The ML model monitoring configuration may include one or more parameters associated with ML model monitoring and / or reporting. For example, the ML model monitoring configuration may include an indication of one or more of: an ML model fault detection (MFD) timer duration, a maximum number of ML model fault instances (MFIs), one or more values ​​associated with one or more MFI criteria, and / or other parameters associated with ML model monitoring and / or reporting. The UE may utilize information from the ML model monitoring configuration to evaluate the performance of one or more ML models.

[0111] At action 715, UE 115 performs ML model monitoring based on the ML model monitoring configuration. In some cases, UE 115 performs ML model monitoring based on one or more ML monitoring signals sent by the network element. The ML monitoring signal may include one or more reference signals, such as a downlink reference signal, CSI-RS, CRS, SSB, etc.

[0112] At action 720, the UE determines whether an initial ML model failure instance (MFI) has been detected. In some cases, the UE determines whether an MFI has occurred based on an evaluation of one or more values ​​associated with the prediction of the ML model becoming one or more measured values. The UE may make this determination for one or more monitoring occasions during the ML model monitoring period. If the UE determines at action 720 that an initial MFI has not been detected, the scheme 700 returns to action 715, where the UE continues ML model monitoring.

[0113] If, at act 720, the UE determines that an initial MFI has been detected, then the scheme 700 continues to act 725, in which the UE starts an ML model failure detection (MFD) timer. In some cases, the length of the MFD timer may be indicated in the ML model monitoring configuration received at act 710. While the MFD timer is running, at act 730, the UE determines whether an additional MFI is detected. If, at act 730, no additional MFI is detected, then the scheme 700 returns to act 715, in which the UE continues ML model monitoring.

[0114] If, at act 730, the UE detects additional MFIs, then the scheme 700 continues to act 740, in which the UE determines whether the number of detected MFIs exceeds a threshold. The value of the threshold may be preconfigured and / or indicative of the ML model monitoring configuration received at act 710. If, at act 740, the number of detected MFIs is less than the threshold (or in some cases equal to the threshold), then the scheme 700 continues to act 735, in which the MFD timer is reset. After resetting the MFD timer at act 735, the scheme 700 then returns to act 715, in which the UE continues ML model monitoring. If, at act 740, the number of detected MFIs is equal to or greater than the threshold (or in some cases merely greater than the threshold), then the scheme 700 continues to act 745, in which the UE sends an indication of an ML model failure to the network element.

[0115] At action 750, the UE may stop and / or retrain the ML model. In some cases, the UE may automatically stop the ML model and / or initiate retraining of the ML model in response to detecting a failure of the ML model. In other cases, the UE may rely on instructions from a network element to determine whether to stop and / or retrain the ML model.

[0116] Figure 8 800 is a block diagram of a UE 800 according to one or more aspects of the present disclosure. The UE 800 may be, for example, Figures 1 to 7UE 115 discussed. As shown, UE 800 may include a processor 802, a memory 804, a machine learning (ML) model monitoring module 808, a transceiver 810 including a modem subsystem 812 and an RF unit 814, and one or more antennas 816. These elements may be coupled to each other. The term "coupled" may refer to being directly or indirectly coupled or connected to one or more intermediate elements. For example, these elements may communicate directly or indirectly with each other, such as via one or more buses.

[0117] The processor 802 may include a CPU, a DSP, an ASIC, a controller, an FPGA device, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 802 may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0118] The memory 804 may include cache memory (e.g., cache memory of the processor 802), RAM, MRAM, ROM, PROM, EPROM, EEPROM, flash memory, solid-state memory devices, hard drives, other forms of volatile and non-volatile memory, or a combination of different types of memory. In one aspect, the memory 804 includes a non-transitory computer-readable medium. The memory 804 may store or have recorded thereon instructions 806. The instructions 806 may include instructions that, when executed by the processor 802, cause the processor 802 to perform operations in conjunction with aspects of the present disclosure (e.g., Figures 3 to 7 and Fig.10 Instructions 806 may also be referred to as program code. The program code may be used to cause the wireless communication device to perform these operations, for example, by causing one or more processors (such as processor 802) to control or command UE 800 to do so. The terms "instructions" and "code" should be broadly interpreted to include any type of computer-readable statements. For example, the terms "instructions" and "code" may refer to one or more programs, routines, subroutines, functions, procedures, etc. "Instructions" and "code" may include a single computer-readable statement or multiple computer-readable statements.

[0119] The ML model monitoring module 808 may be implemented via hardware, software, or a combination thereof. For example, the ML model monitoring module 808 may be implemented as a processor, circuit, and / or instructions 806 stored in the memory 804 and executed by the processor 802. In some aspects, the ML model monitoring module 808 may be integrated within the modem subsystem 812. For example, the ML model monitoring module 808 may be implemented by a combination of software components (e.g., executed by a DSP or a general purpose processor) and hardware components (e.g., logic gates and circuits) within the modem subsystem 812. The ML model monitoring module 808 may communicate with one or more components of the UE 800 to implement various aspects of the present disclosure, such as, Figures 3 to 7 and Fig.10 all aspects.

[0120] In some aspects, the ML model monitoring module 808, along with other components of the UE 800, may be configured to receive a machine learning (ML) model monitoring configuration. In some aspects, the ML model monitoring module 808, along with other components of the UE 800, may be configured to evaluate one or more values ​​associated with a prediction of the ML model as one or more measured values ​​based at least in part on the ML model monitoring configuration. In some aspects, the ML model monitoring module 808, along with other components of the UE 800, may be configured to send an indication of a failure of the ML model in response to detecting a failure of the ML model (e.g., based on the evaluation). In some aspects, the ML model monitoring module 808, along with other components of the UE 800, may be configured to detect a failure of the ML model based on the ML model monitoring configuration. In some aspects, the ML model monitoring module 808, along with other components of the UE 800, may be configured to deactivate the ML model based on detecting a failure of the ML model and / or receiving an instruction to deactivate the ML model from a network element. In some aspects, the ML model monitoring module 808 may be configured along with other components of the UE 800 to initiate retraining of the ML model based on detecting a failure of the ML model and / or receiving an instruction to retrain the ML model from a network element.

[0121] In some aspects, the ML model monitoring module 808 is further configured to run one or more ML models. In this regard, the ML model monitoring module 808, together with other components of the UE 800, may be configured to execute any type of program that relies on machine learning, including but not limited to ML models, artificial intelligence (AI) models, AI / ML models, supervised learning models, unsupervised learning models, reinforcement learning models, semi-supervised learning models, self-supervised learning models, multiple instance learning models, inductive learning models, deductive reasoning models, direct learning models, multi-task learning models, active learning models, online learning models, transfer learning models, holistic learning models, and / or combinations thereof. In addition, the ML model may include a neural network implemented at different types of nodes within the wireless communication network. For example, the neural network may be implemented at a single node (e.g., UE / BS / central cloud server) or may be distributed across multiple nodes. The ML algorithm may be implemented to assist different functions and / or modules between nodes of the wireless communication network. In various aspects, the neural network may be implemented as a convolutional neural network (CNN), a recursive neural network (RNN), a deep convolutional network (DCN), and the like.

[0122] As shown, the transceiver 810 may include a modem subsystem 812 and an RF unit 814. The transceiver 810 may be configured to communicate bidirectionally with other devices (such as the BS 105 and / or network elements). The modem subsystem 812 may be configured to modulate and / or encode data from the memory 804 and / or the ML model monitoring module 808 according to an MCS (e.g., an LDPC decoding scheme, a turbo decoding scheme, a convolutional decoding scheme, a digital beamforming scheme, etc.). The RF unit 814 may be configured to process (e.g., perform analog-to-digital conversion or digital-to-analog conversion, etc.) the modulated / encoded data (e.g., communication signals, data signals, control signals, capability reports, ML model monitoring reports, ML model failure indications, etc.) from the modem subsystem 812 (for outbound transmission). The RF unit 814 may be further configured to perform analog beamforming in conjunction with digital beamforming. Although shown as being integrated together in the transceiver 810, the modem subsystem 812 and the RF unit 814 may be separate devices that are coupled together at the UE 800 to enable the UE 800 to communicate with other devices.

[0123] The RF unit 814 may provide modulated and / or processed data (e.g., data packets (or more generally, data messages that may include one or more data packets and other information)) to the antenna 816 for transmission to one or more other devices. The antenna 816 may further receive data messages sent from other devices. The antenna 816 may provide the received data messages for processing and / or demodulation at the transceiver 810. The transceiver 810 may provide the demodulated and decoded data (e.g., communication signals, data signals, control signals, machine learning (ML) model monitoring configurations, ML model monitoring requests, instructions to deactivate and / or retrain ML models, etc.) to the ML model monitoring module 808 for processing. The antenna 816 may include multiple antennas of similar design or different designs in order to maintain multiple transmission links.

[0124] Fig. 9 is a block diagram of a network unit 900 according to one or more aspects of the present disclosure. The network unit 900 may be as follows: Figures 1 to 7 BS 105, CU 210, DU 230 and / or RU 240 discussed. Therefore, the network unit 900 may include a BS. The BS may be an aggregated BS or a decomposed BS as described above. As shown, the network unit 900 may include a processor 902, a memory 904, a machine learning (ML) model monitoring configuration 908, a transceiver 910 including a modem subsystem 912 and a radio frequency (RF) unit 914, and one or more antennas 916. These elements may be coupled to each other. The term "coupled" may refer to direct or indirect coupling or connection to one or more intermediate elements. For example, these elements may communicate directly or indirectly with each other, for example, via one or more buses.

[0125] The processor 902 may have various characteristics as a specific type of processor. For example, these may include a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a controller, a field programmable gate array (FPGA) device, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 902 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.

[0126] The memory 904 may include cache memory (e.g., cache memory of the processor 902), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state memory devices, one or more hard disk drives, memristor-based arrays, other forms of volatile and non-volatile memory, or a combination of different types of memory. In some aspects, the memory 904 may include a non-transitory computer-readable medium. The memory 904 may store instructions 906. The instructions 906 may include instructions that, when executed by the processor 902, cause the network element 900 to perform the operations described herein (e.g., Figures 3 to 7 and Fig.11 Instructions 906 may also be referred to as program code. The program code may be used to cause the wireless communication device to perform these operations, for example, by causing one or more processors (such as processor 902) to control or command the network unit 900 to do so. The terms "instructions" and "code" should be broadly interpreted to include any type of computer-readable statements. For example, the terms "instructions" and "code" may refer to one or more programs, routines, subroutines, functions, procedures, etc. "Instructions" and "code" may include a single computer-readable statement or multiple computer-readable statements.

[0127] The ML model monitoring module 908 may be implemented via hardware, software, or a combination thereof. For example, the ML model monitoring module 908 may be implemented as a processor, circuit, and / or instructions 906 stored in the memory 904 and executed by the processor 902. In some cases, the ML model monitoring module 908 may be integrated within the modem subsystem 912. For example, the ML model monitoring module 908 may be implemented by a combination of software components (e.g., executed by a DSP or a general purpose processor) and hardware components (e.g., logic gates and circuits) within the modem subsystem 912. The ML model monitoring module 908 may communicate with one or more components of the network unit 900 to implement various aspects of the present disclosure, such as, Figures 3 to 7 and Fig.11 all aspects.

[0128] In some aspects, the ML model monitoring module 908, along with other components of the network unit 900, may be configured to send a machine learning (ML) model monitoring configuration to a user equipment (UE). The ML model monitoring configuration may enable the UE to detect a failure of the ML model based on one or more measured values. In some aspects, the ML model monitoring module 908, along with other components of the network unit 900, may be configured to receive an indication of a failure of the ML model from the UE based on the ML model monitoring configuration. In some aspects, the ML model monitoring module 908, along with other components of the network unit 900, may be configured to set one or more parameters of the ML model monitoring configuration. In some aspects, the ML model monitoring module 908, along with other components of the network unit 900, may be configured to send at least one reference signal for each of a plurality of monitoring occasions associated with the ML model monitoring configuration.

[0129] As shown, the transceiver 910 may include a modem subsystem 912 and an RF unit 914. The transceiver 910 may be configured to communicate bidirectionally with other devices (such as UE 115, UE 800, and / or another network element). The modem subsystem 912 may be configured to modulate and / or encode data according to a modulation and coding scheme (MCS) (e.g., a low-density parity check (LDPC) decoding scheme, a turbo decoding scheme, a convolutional decoding scheme, a digital beamforming scheme, etc.). The RF unit 914 may be configured to process (e.g., perform analog-to-digital conversion or digital-to-analog conversion, etc.) the modulated / encoded data (e.g., communication signals, data signals, control signals, capability reports, ML model monitoring reports, ML model fault indications, etc.) from the modem subsystem 912 (for outbound transmission). The RF unit 914 may be further configured to perform analog beamforming in conjunction with digital beamforming. Although shown as being integrated together in the transceiver 910, the modem subsystem 912 and / or the RF unit 914 may be separate devices that are coupled together at the network unit 900 to enable the physical unit 900 to communicate with other devices.

[0130] The RF unit 914 may provide modulated and / or processed data (e.g., data packets (or more generally, data messages that may include one or more data packets and other information)) to the antenna 916 for transmission to one or more other devices. The antenna 916 may further receive data messages transmitted from other devices and provide the received data messages for processing and / or demodulation at the transceiver 910. The transceiver 910 may provide the demodulated and decoded data (e.g., communication signals, data signals, control signals, capability reports, ML model monitoring reports, ML model fault indications, etc.) to the ML model monitoring module 908 for processing. The antenna 916 may include multiple antennas of similar or different designs in order to maintain multiple transmission links.

[0131] Fig.10 1 is a flowchart illustrating a wireless communication method 1000 according to one or more aspects of the present disclosure. Aspects of the method 1000 may be performed by a computing device (e.g., a processor, a processing circuit, and / or other suitable components) of a wireless communication device or other suitable means for performing blocks. For example, the wireless communication device may be a UE (e.g., UE 115 or UE 800). The UE may utilize one or more components (such as a processor 802, a memory 804, an ML model monitoring module 808, a transceiver 810, a modem subsystem 812, an RF unit 814, and / or one or more antennas 816) to perform the blocks of the method 1000. The method 1000 may be implemented in accordance with Figures 3 to 7 As illustrated, method 1000 includes a plurality of enumerated blocks, but aspects of method 1000 may include additional blocks before, after, and between these enumerated blocks. In some aspects, one or more of the enumerated blocks may be omitted or executed in a different order.

[0132] At block 1010, a UE (e.g., UE 115 and / or UE 800) receives a machine learning (ML) model monitoring configuration. The UE may receive the ML model monitoring configuration from a network element (e.g., network element 900, BS 105, CU 210, DU 230, and / or RU 240). The UE may receive the ML model monitoring configuration from the network element via a radio resource control (RRC) message or other suitable communication. The ML model monitoring configuration may be included as an information element of the communication.

[0133] The ML model monitoring configuration may include one or more parameters associated with ML model monitoring and / or reporting. For example, the ML model monitoring configuration may include an indication of one or more of: an ML model fault detection (MFD) timer duration, a maximum number of ML model fault instances (MFIs), one or more values ​​associated with one or more MFI criteria, and / or other parameters associated with ML model monitoring and / or reporting. That is, the ML model monitoring configuration may include an indication of one or more parameters and / or criteria utilized to detect the occurrence of one or more events associated with the ML model. As further discussed below, the UE may utilize information from the ML model monitoring configuration to evaluate the performance of one or more ML models. In some cases, the ML model may include an ML model used by the UE for beam prediction. For example, the ML model may be configured to provide time domain beam prediction (e.g., as described with respect to Figure 3 ) and / or spatial beam prediction (e.g., as discussed in Figure 4 and Figure 5 discussed).

[0134] At block 1020, the UE evaluates one or more values ​​associated with the prediction of the ML model as one or more measured values. In some aspects, the UE evaluates the values ​​associated with the prediction of the ML model as measured values ​​based at least in part on the ML model monitoring configuration. In this regard, the UE may utilize information from the ML model monitoring configuration to evaluate the performance of the ML model. For example, the UE may evaluate the one or more values ​​associated with the prediction of the ML model based on an indication of an ML model failure detection (MFD) timer duration, an indication of a maximum number of ML model failure instances (MFIs), an indication of one or more values ​​associated with one or more MFI criteria, and / or other parameters indicated in the ML model monitoring configuration.

[0135] In some aspects, the UE may detect one or more events, such as one or more ML model failure instances (MFIs) and / or failures of the ML model based on the evaluation. In this way, the UE may detect events (MFIs and / or failures of the ML model) based on the ML model monitoring configuration. In some cases, the UE detects an initial MFI based on information from the ML model monitoring configuration. In this regard, the UE may determine whether the MFI criteria are met for each of the plurality of monitoring occasions. In some aspects, the UE determines whether the MFI criteria are met by evaluating one or more values ​​associated with the prediction of the ML model relative to one or more measured values.

[0136] In some cases, the UE evaluates one or more measured values ​​associated with the ML model's prediction as one or more measured values ​​associated with the measured beam set. In some cases, the UE may determine whether the ML model's 1st ranked predicted beam is included in the set of top K ranked beams in the measured beam set. For example, referring back to Figure 3 , the UE may determine whether measurements (e.g., RSRP (e.g., L1-RSRP), RSRQ, RSSI, SINR, etc.) associated with the 1st-ranked predicted beam in the predicted beam group 315i are within the top K beams (e.g., the top 1, 2, 3, 4, 5, etc. beams) in the measured beam group of the auxiliary reference signal group 320a. In this regard, the measurements associated with the 1st-ranked predicted beam in the predicted beam group 315i may be measurements of corresponding beams in the measured beam group of the auxiliary reference signal group 320a. For example, if the ML model predicts that a particular beam (e.g., beam index 2) is the 1st-ranked predicted beam, measurements of the particular beam (e.g., beam index 2) in the measured beam group of the auxiliary reference signal group 320a may be used as measurements associated with the 1st-ranked predicted beam in the predicted beam group 315i.

[0137] In some cases, the MFI criterion may be met if the ML model's 1st-ranked predicted beam is not included in the set of top K beams in the measured beam group. That is, if the ML model's 1st-ranked predicted beam is not included in the set of top K beams in the measured beam group, then MFI exists. For example, if measurements associated with the ML model's 1st-ranked predicted beam (e.g., beam index 2) are not included in the top K (e.g., 1, 2, 3, 4, etc.) beams in the measured beam group of the auxiliary reference signal group 320a, then MFI exists. On the other hand, if the ML model's 1st-ranked predicted beam is included in the set of top K beams in the measured beam group, then the ML model may be considered to be operating properly and MFI does not exist. For example, if the measurements associated with the 1st ranked predicted beam (e.g., beam index 2) of the ML model are included in the top K (e.g., 1, 2, 3, 4, etc.) beams in the measured beam group of the auxiliary reference signal group 320a, then there is no MFI.

[0138] In some cases, the UE may determine whether the 1st ranked measured beam in the measured beam group is included in the set of top K ranked beams in the predicted beam group associated with the prediction of the ML model. Figure 3, the UE may determine whether the 1st ranked measured beam from the measured beam group of the auxiliary reference signal group 320a is among the top K beams (e.g., the top 1, 2, 3, 4, 5, etc. beams) in the predicted beam group 315i. The determination of the ranking of the beam may be based on RSRP (e.g., L1-RSRP), RSRQ, RSSI, SINR, etc. In this regard, the measurement of the 1st ranked measured beam from the measured beam group of the auxiliary reference signal group 320a may be based on the measurement of the beams in the measured beam group of the auxiliary reference signal group 320a. The measurements associated with the top K predicted beams in the predicted beam group 315i may be the measurements of the corresponding beams in the measured beam group of the auxiliary reference signal group 320a. For example, if the ML model predicts that particular beams (e.g., beam indices 2, 3, and 4) are the top 3 predicted beams, then measurements of those particular beams (e.g., beam indices 2, 3, and 4) in the measured beam group from auxiliary reference signal group 320a can be used as measurements associated with the top K predicted beams in predicted beam group 315i.

[0139] In some cases, the MFI criterion may be met if the 1st ranked measured beam in the measured beam group is not included in the set of top K ranked beams in the predicted beam group associated with the prediction of the ML model. That is, if the 1st ranked measured beam of the ML model is not included in the set of top K ranked beams in the predicted beam group of the ML model, then MFI exists. For example, if the measurement associated with the 1st ranked measured beam (e.g., beam index 2) in the measured beam group of the auxiliary reference signal group 320a is not included in the top K ranked (e.g., 1, 2, 3, 4, etc.) beams in the predicted beam group 315i, then MFI exists. On the other hand, if the 1st ranked measured beam in the measured beam group of the auxiliary reference signal group 320a is included in the set of top K ranked beams in the predicted beam group 315i, then the ML model may be considered to be operating properly and there is no MFI. For example, if measurements associated with the 1st ranked measured beam (e.g., beam index 2) in the measured beam group of the auxiliary reference signal group 320a are included in the top K (e.g., 1, 2, 3, 4, etc.) ranked beams in the predicted beam group 315i, then there is no MFI.

[0140] In some cases, the UE may determine whether the Layer 1 Reference Signal Received Power (L1-RSRP) and / or other measurements (e.g., RSRQ, RSSI, SNIR, etc.) of the ML model's 1st ranked predicted beam is within a threshold difference of the L1-RSRP of the 1st ranked measured beam in the measured beam group. Figure 3, the UE may determine whether the L1-RSRP associated with the 1st ranked predicted beam in the predicted beam group 315i is within a threshold difference of the measured L1-RSRP of the 1st ranked beam in the measured beam group of the auxiliary reference signal group 320a. In this regard, the L1-RSRP associated with the 1st ranked predicted beam in the predicted beam group 315i may be a measurement of the corresponding beam in the measured beam group of the auxiliary reference signal group 320a. For example, if the ML model predicts that a particular beam (e.g., beam index 2) is the 1st ranked predicted beam, the measurement of the particular beam (e.g., beam index 2) in the measured beam group of the auxiliary reference signal group 320a may be used as the measurement associated with the 1st ranked predicted beam in the predicted beam group 315i.

[0141] In some cases, the MFI criterion is met if the L1-RSRP and / or other measurements of the ML model's 1st ranked predicted beam (e.g., beam index 2) are not within a threshold difference (e.g., 0.5 dB, 1.0 dB, 1.5 dB, or other) of the L1-RSRP of the 1st ranked measured beam in the measured beam group. On the other hand, if the L1-RSRP of the ML model's 1st ranked predicted beam is within a threshold difference of the L1-RSRP of the 1st ranked beam in the measured beam group, the ML model may be considered to be operating properly and MFI does not exist. In some cases, the UE may receive an indication of the threshold difference from the network element. In some aspects, the indication of the threshold difference may be included in the ML model monitoring configuration.

[0142] In some cases, the UE evaluates one or more predicted values ​​associated with the prediction of the ML model as one or more measured values ​​associated with the measured beam group. In some cases, the UE determines whether the predicted layer 1 reference signal received power (L1-RSRP) and / or other measurements (e.g., RSRQ, RSSI, SNIR, etc.) of the ML model's 1st ranked predicted beam is within a threshold difference of the measured L1-RSRP of the 1st ranked measured beam in the measured beam group. For example, re-referencing Figure 3 , the UE may determine whether the predicted L1-RSRP associated with the first-ranked predicted beam in the predicted beam group 315i is within a threshold difference of the measured L1-RSRP of the first-ranked beam in the measured beam group of the auxiliary reference signal group 320a. In this regard, the L1-RSRP associated with the first-ranked predicted beam in the predicted beam group 315i may be a measurement of a corresponding beam in the measured beam group of the auxiliary reference signal group 320a.

[0143] In some aspects, the MFI criterion is met if the predicted L1-RSRP of the 1st ranked predicted beam (e.g., beam index 2) of the ML model is not within a threshold difference (e.g., 0.5dB, 1.0dB, 1.5dB, or other) of the measured L1-RSRP of the 1st ranked measured beam in the measured beam group. On the other hand, if the predicted L1-RSRP of the 1st ranked predicted beam of the ML model is within a threshold difference of the measured L1-RSRP of the 1st ranked beam in the measured beam group, the ML model may be considered to be operating properly and MFI does not exist. In some cases, the UE may receive an indication of a threshold difference between the predicted L1-RSRP and the measured L1-RSRP from a network element. In some aspects, the indication of the threshold difference between the predicted L1-RSRP and the measured L1-RSRP may be included in the ML model monitoring configuration.

[0144] In response to detecting an initial MFI (e.g., based on one or more of the standards discussed above), the UE may start an ML model failure detection (MFD) timer. In some aspects, the ML model monitoring configuration includes an indication of the duration of the MFD timer. In some aspects, the UE determines that the ML model has failed based on detecting a certain number of MFIs before the end of the MFD timer. For example, if the UE detects a number of MFIs exceeding a threshold before the MFD timer expires (e.g., based on one or more of the standards discussed above), the UE may determine that the ML model has failed. In this regard, the UE may increment an MFI counter for each of the initial MFI and / or the number of MFIs before the end of the MFD timer. If the MFI counter reaches a threshold, the UE may determine and / or declare that the ML model has failed. In some cases, the ML model monitoring configuration includes an indication of a threshold (e.g., 2, 3, 4, 5, 6, 7, etc.) number of MFIs associated with the ML model failure.

[0145] At block 1030, in response to detecting a failure of the ML model, the UE sends an indication of the failure of the ML model. That is, in response to detecting an event, the UE may send an indication of the detected event. In some cases, the UE may detect the failure of the ML model based on the evaluation at block 1020. The UE may send an indication of the failure of the ML model to the network element. The UE may send the indication of the failure of the ML model via an RRC message, a PUCCH communication, a PUSCH communication, and / or other suitable communication.

[0146] In some cases, the UE receives an instruction to deactivate the ML model. For example, in response to the UE sending an indication of a failure of the ML model, the UE may receive an instruction to deactivate the ML model from the network element. In some cases, the UE receives an instruction to retrain the ML model. For example, in response to the UE sending an indication of a failure of the ML model, the UE may receive an instruction to retrain the ML model from the network element.

[0147] In some cases, the UE deactivates the ML model based on detecting a failure of the ML model. In some cases, the UE initiates retraining of the ML model based on detecting a failure of the ML model.

[0148] In some cases, the UE may not detect a failure of the ML model based on the evaluation at block 1020. That is, the UE may determine that the ML model is operating properly based on the evaluation at block 1020. In some aspects, the UE may send an indication to the network element that the ML model is operating properly. For example, the UE may send an indication to the network element that the ML model is operating properly via an RRC message, a PUCCH communication, a PUSCH communication, and / or other suitable communication.

[0149] Fig.11 1 is a flow chart illustrating a wireless communication method 1100 according to one or more aspects of the present disclosure. Aspects of the method 1100 may be performed by a computing device (e.g., a processor, a processing circuit, and / or other suitable components) of a wireless communication device or other suitable means for performing blocks. For example, the wireless communication device may include a network unit (e.g., a network unit 900, a BS 105, a CU 210, a DU 230, and / or a RU 240). The network unit 900 may utilize one or more components (such as a processor 902, a memory 904, an ML model monitoring module 908, a transceiver 910, a modem subsystem 912, an RF unit 914, and / or one or more antennas 916) to perform blocks of the method 1100. The method 1100 may be performed in accordance with Figures 3 to 7 Similar mechanisms described. As illustrated, method 1100 includes multiple enumerated blocks, but aspects of method 1100 may include additional blocks before, after, and between these enumerated blocks. In some aspects, one or more of the enumerated blocks may be omitted or executed in a different order.

[0150] At block 1110, a network element (network element 900, BS 105, CU 210, DU 230, and / or RU 240) sends a machine learning (ML) model monitoring configuration. The network element may send the ML model monitoring configuration to a user equipment (UE) (e.g., UE 115 and / or UE 800). The network element may send the ML model monitoring configuration to the UE via a radio resource control (RRC) message or other suitable communication. The network element may include the ML model monitoring configuration as an information element of the communication. The ML model monitoring configuration may enable the UE to monitor the performance of the ML model based on one or more measured values, including detection of a failure of the ML model.

[0151] The ML model monitoring configuration may include one or more parameters associated with ML model monitoring and / or reporting. For example, the ML model monitoring configuration may include an indication of one or more of: an ML model fault detection (MFD) timer duration, a maximum number of ML model fault instances (MFIs), one or more values ​​associated with one or more MFI criteria (e.g., whether the ML model's 1st ranked predicted beam is included in the set of top K beams in the measured beam group ... Whether the predicted layer 1 reference signal received power (L1-RSRP) or other measurement is within a threshold difference of the measured L1-RSRP of the first-ranked measured beam in the measured beam group, whether the predicted layer 1 reference signal received power (L1-RSRP) or other measurement (e.g., RSRQ, RSSI, SNIR, etc.) of the first-ranked predicted beam of the ML model is within a threshold difference of the measured L1-RSRP of the first-ranked measured beam in the measured beam group, etc.) and / or other parameters associated with ML model monitoring and / or reporting. In this regard, the network element may set one or more of the parameters included in the ML model monitoring configuration based on network conditions (e.g., traffic patterns, network loads, latency requirements, etc.), network capabilities, UE capabilities, and / or other factors. That is, the ML model monitoring configuration may include an indication of one or more parameters and / or criteria utilized to detect the occurrence of one or more events associated with the ML model. The UE may utilize information from the ML model monitoring configuration to evaluate the performance of one or more ML models, including reporting an indication of the performance to the network element in some cases.

[0152] At block 1120, the network element receives an indication of a failure of the ML model from the UE. That is, the network element may receive an indication of an event detected by the UE. In some cases, the indication of a failure of the ML model is received based on the ML model monitoring configuration. In this regard, the UE may determine that the ML model has failed based on one or more of the parameters indicated in the ML model monitoring configuration, and in response, send an indication of the failure to the network element.

[0153] In some aspects, the network unit transmits at least one reference signal for each of a plurality of monitoring occasions associated with the ML model monitoring configuration. For example, the network unit may transmit reference signals (e.g., downlink reference signals, CSI-RS, CRS, SSB, etc.) associated with a plurality of monitoring occasions, as described above with respect to Figures 3 to 5 In this regard, the network element may transmit a nominal reference signal (eg, 305, 405, 505) and / or an auxiliary reference signal (eg, 320, 420, 520) during normal operation and / or during an ML model evaluation period (eg, 335, 435, 535).

[0154] In some cases, the network element sends an instruction to the UE to deactivate the ML model. For example, the network element may send an instruction to deactivate the ML model in response to receiving an indication of a failure of the ML model from the UE. In some cases, the network element sends an instruction to retrain the ML model. For example, the network element may send an instruction to retrain the ML model in response to receiving an indication of a failure of the ML model from the UE.

[0155] Other aspects of the present disclosure include:

[0156] Clause 1. A method of wireless communication performed by a user equipment, the method comprising:

[0157] Receive machine learning (ML) model monitoring configuration;

[0158] evaluating one or more values ​​associated with a prediction of the ML model as one or more measured values ​​based at least in part on the ML model monitoring configuration; and

[0159] In response to detecting a failure of the ML model based on the evaluating, an indication of the failure of the ML model is sent.

[0160] Clause 2. The method according to clause 1, further comprising:

[0161] The failure of the ML model is detected based on the ML model monitoring configuration.

[0162] Clause 3. The method of clause 2, wherein detecting the failure of the ML model based on the ML model monitoring configuration comprises:

[0163] A ML model fault detection (MFD) timer is started in response to detecting an initial ML model fault instance (MFI).

[0164] Clause 4. The method of clause 3, wherein the ML model monitoring configuration includes an indication of a duration of the MFD timer.

[0165] Clause 5. The method of clause 3, wherein detecting the failure of the ML model based on the ML model monitoring configuration further comprises:

[0166] A number of MFIs is detected before an end of the MFD timer, wherein the number of MFIs satisfies a threshold.

[0167] Clause 6. The method of clause 5, wherein detecting the failure of the ML model based on the ML model monitoring configuration further comprises:

[0168] An MFI counter is incremented for each of the initial MFI and the number of MFIs before the end of the MFD timer.

[0169] Clause 7. The method of clause 5, wherein the ML model monitoring configuration includes an indication of the threshold.

[0170] Clause 8. The method of clause 2, wherein detecting the failure of the ML model based on the ML model monitoring configuration comprises:

[0171] A determination is made based on the evaluating whether an ML model failure instance (MFI) criterion is satisfied for each of a plurality of monitoring opportunities.

[0172] Clause 9. The method of clause 8, wherein the evaluating comprises:

[0173] One or more measured values ​​associated with the prediction of the ML model are evaluated as one or more measured values ​​associated with the measured beam set.

[0174] Clause 10. The method of clause 9, wherein evaluating the one or more measured values ​​associated with the prediction of the ML model as the one or more measured values ​​associated with the measured beam set comprises:

[0175] determining whether the ML model's 1st ranked predicted beam is included in a set of top K beams in the measured beam set; and

[0176] Wherein the MFI criterion is met if the 1st ranked predicted beam of the ML model is not included in the set of top K ranked beams in the measured beam group.

[0177] Clause 11. The method of clause 9, wherein evaluating the one or more measured values ​​associated with the prediction of the ML model as the one or more measured values ​​associated with the measured beam set comprises:

[0178] determining whether a 1st ranked measured beam in a measured beam group is included in a set of top K ranked beams in a predicted beam group associated with the prediction of the ML model; and

[0179] Wherein the MFI criterion is met if the 1st ranked measured beam in a measured beam set is not included in the set of top K ranked beams in a predicted beam set associated with the prediction of the ML model.

[0180] Clause 12. The method of clause 9, wherein evaluating the one or more measured values ​​associated with the prediction of the ML model as the one or more measured values ​​associated with the measured beam set comprises:

[0181] determining whether a layer 1 reference signal received power (L1-RSRP) of a 1st ranked predicted beam of the ML model is within a threshold difference of the L1-RSRP of a 1st ranked measured beam in a measured beam group; and

[0182] Wherein the MFI criterion is met if the L1-RSRP of the 1st ranked predicted beam of the ML model is not within the threshold difference of the L1-RSRP of the 1st ranked measured beam in the measured beam group.

[0183] Clause 13. The method of clause 8, wherein the evaluating comprises:

[0184] One or more predicted values ​​associated with the prediction of the ML model are evaluated as one or more measured values ​​associated with the measured beam set.

[0185] Clause 14. The method of clause 13, wherein evaluating the one or more predicted values ​​associated with the prediction of the ML model as the one or more measured values ​​associated with the measured beam set comprises:

[0186] determining whether a predicted layer 1 reference signal received power (L1-RSRP) of a 1st ranked predicted beam of the ML model is within a threshold difference of a measured L1-RSRP of a 1st ranked measured beam in a measured beam group; and

[0187] Wherein the MFI criterion is met if the predicted L1-RSRP of the 1st ranked predicted beam of the ML model is not within the threshold difference of the measured L1-RSRP of the 1st ranked measured beam in a measured beam set.

[0188] Clause 15. The method of any one of clauses 1 to 14, wherein receiving the ML model monitoring configuration comprises:

[0189] The ML model monitoring configuration is received from a network element via a radio resource control (RRC) message.

[0190] Clause 16. The method of any one of clauses 1 to 15, wherein receiving the ML model monitoring configuration comprises:

[0191] Receiving the ML model monitoring configuration including an indication of one or more of:

[0192] ML Model Fault Detection (MFD) timer duration;

[0193] The maximum number of ML Model Failure Instances (MFIs); or

[0194] One or more values ​​associated with the MFI standard.

[0195] Clause 17. The method according to any one of clauses 1 to 16, further comprising at least one of the following:

[0196] deactivating the ML model based on detecting the failure of the ML model; or

[0197] Retraining of the ML model is initiated based on detecting the failure of the ML model.

[0198] Clause 18. The method according to any one of clauses 1 to 17, further comprising:

[0199] receiving, based on the indication of the failure of the ML model, from a network element at least one of:

[0200] an instruction to deactivate the ML model; or

[0201] Instructions for retraining the ML model.

[0202] Clause 19. A method of wireless communication performed by a network element, the method comprising:

[0203] sending a machine learning (ML) model monitoring configuration to a user equipment (UE), wherein the ML model monitoring configuration enables the UE to detect a failure of the ML model based on one or more measured values; and

[0204] An indication of a failure of the ML model is received from the UE based on the ML model monitoring configuration.

[0205] Clause 20. The method according to clause 19, further comprising:

[0206] One or more parameters of the ML model monitoring configuration are set.

[0207] Clause 21. The method of clause 20, wherein the one or more parameters of the ML model monitoring configuration include a duration of an ML model fault detection (MFD) timer.

[0208] Clause 22. The method of clause 20, wherein the one or more parameters of the ML model monitoring configuration include a maximum number of ML model fault instances (MFIs).

[0209] Clause 23. The method of clause 20, wherein the one or more parameters of the ML model monitoring configuration include one or more values ​​associated with an ML Model Fault Instance (MFI) criterion.

[0210] Clause 24. The method of clause 23, wherein the MFI criterion is based on whether the ML model's 1st ranked predicted beam is included in a set of top K beams in the measured beam set.

[0211] Clause 25. The method of clause 23, wherein the MFI criterion is based on whether a 1st ranked measured beam in a measured beam set is included in a set of top K ranked beams in a predicted beam set of the ML model.

[0212] Clause 26. A method according to clause 23, wherein the MFI criterion is based on whether the layer 1 reference signal received power (L1-RSRP) of the 1st ranked predicted beam of the ML model is within a threshold difference of the L1-RSRP of the 1st ranked measured beam in the measured beam group.

[0213] Clause 27. A method according to clause 23, wherein the MFI criterion is based on whether the predicted layer 1 reference signal received power (L1-RSRP) of the 1st ranked predicted beam of the ML model is within a threshold difference of the measured L1-RSRP of the 1st ranked measured beam in the measured beam group.

[0214] Clause 28. The method according to any one of clauses 19 to 27, further comprising:

[0215] At least one reference signal is sent for each monitoring occasion of a plurality of monitoring occasions associated with the ML model monitoring configuration.

[0216] Clause 29. The method of any one of clauses 19 to 28, wherein sending the ML model monitoring configuration comprises:

[0217] The ML model monitoring configuration is sent via a radio resource control (RRC) message.

[0218] Clause 30. The method according to any one of clauses 19 to 29, further comprising:

[0219] sending, based on the indication of the failure of the ML model, at least one of the following to the UE:

[0220] an instruction to deactivate the ML model; or

[0221] Instructions for retraining the ML model.

[0222] Clause 31. A non-transitory computer-readable medium storing one or more instructions for wireless communication, the one or more instructions comprising one or more instructions that, when executed by one or more processors of a UE, cause the UE to perform any of clauses 1 to 18.

[0223] Clause 32. A non-transitory computer-readable medium storing one or more instructions for wireless communication, the one or more instructions comprising one or more instructions that, when executed by one or more processors of a network element, cause the network element to perform any one of the aspects described in clauses 19 to 30.

[0224] Clause 33. A user equipment (UE) comprising one or more components for performing any one or more of the aspects of clauses 1 to 18.

[0225] Clause 34. A network element comprising one or more components for performing any one or more aspects of clauses 19 to 30.

[0226] Clause 35. A user equipment (UE), the user equipment (UE) comprising: a memory; a transceiver; and at least one processor, the at least one processor coupled to the memory and the transceiver, wherein the UE is configured to perform any one or more aspects of clauses 1 to 18.

[0227] Clause 36. A network unit, comprising: a memory; a transceiver; and at least one processor, the at least one processor coupled to the memory and the transceiver, wherein the network unit is configured to perform any one or more aspects of clauses 19 to 30.

[0228] The various illustrative blocks and modules described herein in conjunction with the present disclosure may be implemented or executed using a general purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in an alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, a combination of one or more microprocessors and a DSP core, or any other such configuration).

[0229] The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. When implemented in software executed by a processor, the functions may be stored on or sent via a computer-readable medium as one or more instructions or codes. Other aspects and specific implementations are within the scope of the present disclosure and the appended claims. For example, due to the nature of software, the functions described above may be implemented using software executed by a processor, hardware, firmware, hard wiring, or any combination thereof. The features of the implementation functions may also be physically located at different locations, including being distributed so that the various parts of the functions are implemented at different physical locations. In addition, as used herein (including in the claims), "or" as used in an item enumeration (e.g., followed by a list of items such as "at least one of" or "one or more of") indicates an inclusive enumeration, such that, for example, an enumeration of [at least one of A, B, or C] means: A or B or C or AB or AC or BC or ABC (e.g., A and B and C).

[0230] As those skilled in the art will now appreciate and depending on the particular application at hand, many modifications, substitutions and changes may be made to the materials, devices, configurations and methods of use of the apparatus of the present disclosure without departing from the spirit and scope of the present disclosure. In view of this, the scope of the present disclosure should not be limited to the specific aspects illustrated and described herein (as they are only some aspects of the present disclosure), but should be fully equivalent to the appended claims and their functional equivalents.

Claims

1. A method of wireless communication performed by a user equipment, the method comprising: Receive machine learning (ML) model monitoring configuration; evaluating one or more values ​​associated with a prediction of the ML model as one or more measured values ​​based at least in part on the ML model monitoring configuration; and In response to detecting a failure of the ML model based on the evaluating, an indication of the failure of the ML model is sent.

2. The method according to claim 1, further comprising: The failure of the ML model is detected based on the ML model monitoring configuration.

3. The method of claim 2, wherein detecting the failure of the ML model based on the ML model monitoring configuration comprises: A ML model fault detection (MFD) timer is started in response to detecting an initial ML model fault instance (MFI).

4. The method of claim 3, wherein the ML model monitoring configuration includes an indication of a duration of the MFD timer.

5. The method of claim 3, wherein detecting the failure of the ML model based on the ML model monitoring configuration further comprises: A number of MFIs is detected before an end of the MFD timer, wherein the number of MFIs satisfies a threshold.

6. The method of claim 5, wherein detecting the failure of the ML model based on the ML model monitoring configuration further comprises: An MFI counter is incremented for each of the initial MFI and the number of MFIs before the end of the MFD timer.

7. The method of claim 5, wherein the ML model monitoring configuration includes an indication of the threshold.

8. The method of claim 2, wherein detecting the failure of the ML model based on the ML model monitoring configuration comprises: A determination is made based on the evaluating whether an ML model failure instance (MFI) criterion is satisfied for each of a plurality of monitoring opportunities.

9. The method of claim 8, wherein the evaluating comprises: One or more measured values ​​associated with the prediction of the ML model are evaluated as one or more measured values ​​associated with the measured beam set.

10. The method of claim 9, wherein evaluating the one or more measured values ​​associated with the prediction of the ML model as the one or more measured values ​​associated with the measured beam set comprises: determining whether a 1st ranked predicted beam of the ML model is included in a set of top K ranked beams in the measured beam group; and Wherein the MFI criterion is met if the 1st ranked predicted beam of the ML model is not included in the set of top K ranked beams in the measured beam group.

11. The method of claim 9, wherein evaluating the one or more measured values ​​associated with the prediction of the ML model as the one or more measured values ​​associated with the measured beam set comprises: determining whether a 1st ranked measured beam in a measured beam group is included in a set of top K ranked beams in a predicted beam group associated with the prediction of the ML model; and Wherein the MFI criterion is met if the 1st ranked measured beam in a measured beam set is not included in the set of top K ranked beams in a predicted beam set associated with the prediction of the ML model.

12. The method of claim 9, wherein evaluating the one or more measured values ​​associated with the prediction of the ML model as the one or more measured values ​​associated with the measured beam set comprises: determining whether a layer 1 reference signal received power (L1-RSRP) of a 1st ranked predicted beam of the ML model is within a threshold difference of the L1-RSRP of a 1st ranked measured beam in a measured beam group; and Wherein the MFI criterion is met if the L1-RSRP of the 1st ranked predicted beam of the ML model is not within the threshold difference of the L1-RSRP of the 1st ranked measured beam in the measured beam group.

13. The method of claim 8, wherein the evaluating comprises: One or more predicted values ​​associated with the prediction of the ML model are evaluated as one or more measured values ​​associated with the measured beam set.

14. The method of claim 13, wherein evaluating the one or more predicted values ​​associated with the prediction of the ML model as the one or more measured values ​​associated with the measured beam set comprises: determining whether a predicted layer 1 reference signal received power (L1-RSRP) of a 1st ranked predicted beam of the ML model is within a threshold difference of a measured L1-RSRP of a 1st ranked measured beam in a measured beam group; and Wherein the MFI criterion is met if the predicted L1-RSRP of the 1st ranked predicted beam of the ML model is not within the threshold difference of the measured L1-RSRP of the 1st ranked measured beam in a measured beam set.

15. The method of claim 1, wherein receiving the ML model monitoring configuration comprises: The ML model monitoring configuration is received from a network element via a radio resource control (RRC) message.

16. The method of claim 1, wherein receiving the ML model monitoring configuration comprises: Receiving the ML model monitoring configuration including an indication of one or more of: ML Model Fault Detection (MFD) timer duration; The maximum number of ML Model Failure Instances (MFIs); or One or more values ​​associated with the MFI standard.

17. The method of claim 1, further comprising at least one of the following: deactivating the ML model based on detecting the failure of the ML model; or Retraining of the ML model is initiated based on detecting the failure of the ML model.

18. The method according to claim 1, further comprising: receiving, based on the indication of the failure of the ML model, from a network element at least one of: an instruction to deactivate the ML model; or Instructions for retraining the ML model.

19. A method of wireless communication performed by a network element, the method comprising: sending a machine learning (ML) model monitoring configuration to a user equipment (UE), wherein the ML model monitoring configuration enables the UE to detect a failure of the ML model based on one or more measured values; and An indication of a failure of the ML model is received from the UE based on the ML model monitoring configuration.

20. The method according to claim 19, further comprising: One or more parameters of the ML model monitoring configuration are set.

21. The method of claim 20, wherein the one or more parameters of the ML model monitoring configuration include a duration of an ML model fault detection (MFD) timer.

22. The method of claim 20, wherein the one or more parameters of the ML model monitoring configuration include a maximum number of ML model fault instances (MFIs).

23. The method of claim 20, wherein the one or more parameters of the ML model monitoring configuration include one or more values ​​associated with an ML Model Fault Instance (MFI) criterion.

24. The method of claim 23, wherein the MFI criterion is based on whether the ML model's 1st ranked predicted beam is included in a set of top K beams in the measured beam set.

25. The method of claim 23, wherein the MFI criterion is based on whether a 1st ranked measured beam in a measured beam set is included in a set of top K ranked beams in a predicted beam set of the ML model.

26. The method of claim 23, wherein the MFI criterion is based on whether a Layer 1 Reference Signal Received Power (L1-RSRP) of a 1st ranked predicted beam of the ML model is within a threshold difference of the L1-RSRP of a 1st ranked measured beam in a measured beam group.

27. The method of claim 23, wherein the MFI criterion is based on whether a predicted layer 1 reference signal received power (L1-RSRP) of a 1st ranked predicted beam of the ML model is within a threshold difference of a measured L1-RSRP of a 1st ranked measured beam in a measured beam group.

28. The method of claim 19, further comprising: At least one reference signal is sent for each monitoring occasion of a plurality of monitoring occasions associated with the ML model monitoring configuration.

29. The method of claim 19, wherein the sending the ML model monitoring configuration comprises: The ML model monitoring configuration is sent via a radio resource control (RRC) message.

30. The method of claim 19, further comprising: sending, based on the indication of the failure of the ML model, at least one of the following to the UE: an instruction to deactivate the ML model; or Instructions for retraining the ML model.

31. A user equipment (UE), the user equipment (UE) comprising: Memory devices; Transceiver; and a processor in communication with the processor and the transceiver, wherein the UE is configured to: Receive machine learning (ML) model monitoring configuration; evaluating one or more values ​​associated with a prediction of the ML model as one or more measured values ​​based at least in part on the ML model monitoring configuration; and In response to detecting a failure of the ML model based on the evaluating, an indication of the failure of the ML model is sent.

32. The UE according to claim 31, wherein the UE is further configured to: The failure of the ML model is detected based on the ML model monitoring configuration.

33. The UE according to claim 32, wherein the UE is further configured to: Initiating a ML model fault detection (MFD) timer in response to detecting an initial ML model fault instance (MFI); and A number of MFIs is detected before an end of the MFD timer, wherein the number of MFIs satisfies a threshold.

34. The UE according to claim 33, wherein the UE is further configured to: An MFI counter is incremented for each of the initial MFI and the number of MFIs before the end of the MFD timer.

35. The UE according to claim 32, wherein the UE is further configured to: A determination is made based on the ML model monitoring configuration whether an ML model fault instance (MFI) criterion is satisfied for each of a plurality of monitoring opportunities.

36. The UE according to claim 35, wherein the UE is further configured to: evaluating one or more measured values ​​associated with the prediction of the ML model as one or more measured values ​​associated with the measured beam set based on at least one of: determining whether a 1st ranked predicted beam of the ML model is included in a set of top K ranked beams in the measured beam group; determining whether a 1st ranked measured beam in a measured beam group is included in a set of top K ranked beams in a predicted beam group associated with the prediction of the ML model; or A determination is made as to whether a layer 1 reference signal received power (L1-RSRP) of a 1st ranked predicted beam of the ML model is within a threshold difference of the L1-RSRP of a 1st ranked measured beam in a measured beam group.

37. The UE according to claim 35, wherein the UE is further configured to: Based on determining whether a predicted layer 1 reference signal received power (L1-RSRP) of the 1st ranked predicted beam of the ML model is within a threshold difference of a measured L1-RSRP of the 1st ranked measured beam in the measured beam group, one or more predicted values ​​associated with the prediction of the ML model are evaluated as one or more measured values ​​associated with the measured beam group.

38. The UE according to claim 31, wherein the UE is further configured to: deactivating the ML model based on detecting the failure of the ML model; or Retraining of the ML model is initiated based on detecting the failure of the ML model.

39. The UE according to claim 31, wherein the UE is further configured to: receiving, based on the indication of the failure of the ML model, from a network element at least one of: an instruction to deactivate the ML model; or Instructions for retraining the ML model.

40. A network unit, comprising: Memory devices; Transceiver; and a processor in communication with the processor and the transceiver, wherein the network unit is configured to: sending a machine learning (ML) model monitoring configuration to a user equipment (UE), wherein the ML model monitoring configuration enables the UE to detect a failure of the ML model based on one or more measured values; and An indication of a failure of the ML model is received from the UE based on the ML model monitoring configuration.

41. The network unit according to claim 40, wherein the network unit is further configured to: Setting one or more parameters of the ML model monitoring configuration, wherein the one or more parameters of the ML model monitoring configuration include at least one of the following: The duration of the MFD timer; The maximum number of ML Model Failure Instances (MFIs); or One or more values ​​associated with the MFI standard.

42. The network unit of claim 41, wherein: The one or more parameters of the ML model monitoring configuration include one or more values ​​associated with the MFI criterion; and The MFI standard is based on at least one of the following: whether the ML model's 1st ranked predicted beam is included in the set of top K beams in the measured beam set; whether a 1st ranked measured beam in the measured beam set is included in a set of top K ranked beams in the predicted beam set of the ML model; whether the Layer 1 Reference Signal Received Power (L1-RSRP) of the 1st ranked predicted beam of the ML model is within a threshold difference of the L1-RSRP of the 1st ranked measured beam in the measured beam group; or Whether a predicted layer 1 reference signal received power (L1-RSRP) of a 1st-ranked predicted beam of the ML model is within a threshold difference of a measured L1-RSRP of a 1st-ranked measured beam in a measured beam group.

43. The network unit of claim 40, wherein the network unit is further configured to: sending, based on the indication of the failure of the ML model, at least one of the following to the UE: an instruction to deactivate the ML model; or Instructions for retraining the ML model.

Images