Methods for monitoring the performance of wtru-side ai / ml models

Abstract

A wireless transmit / receive unit (WTRU) may be configured to receive a message comprising out-of-distribution (OOD) measurement and reporting configuration information, to receive a plurality of reference signals (RSs), and to measure a plurality of channel responses based on the received RSs to determine a plurality of channel state information (CSI) samples. The WTRU may be configured to perform an inference on the plurality of CSI samples using an OOD classifier mode, to determine a plurality of energy scores of the plurality of CSI samples, and to determine that one or more CSI samples of the plurality of CSI samples is OOD with respect to a training dataset based on the OOD measurement configuration information. The WTRU may be configured to determine an OOD frequency metric based on a number of CSI samples that are determined to be OOD, and to determine that an OOD event occurred.

Description

METHODS FOR MONITORING THE PERFORMANCE OF WTRU-SIDE AI / ML MODELSCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 531 ,191, filed August 7, 2023, the contents of which are incorporated herein by reference.BACKGROUND

[0002] Channel state information (CSI) reporting may include codebook-based precoding with feedback information. The feedback information may include a precoding matrix index (PMI) which may be referred to as a codeword index in the codebook. Artificial Intelligence / Machine Learning (AI / MJL based CSI feedback may use Autoencoders (AE) for CSI compression, which is a two-sided system, where the estimated CSI is compressed at the WTRU side, fed back to a gNB, and then decompressed at the gNB.SUMMARY

[0003] A wireless transmit / receive unit (WTRU) may be configured to perform a method for out of distribution (OOD) detection and model monitoring based on input distribution for CSI feedback. The WTRU may be configured to receive a message comprising out-of-distribution (OOD) measurement configuration information and reporting configuration information. The WTRU may be configured to receive a plurality of reference signals (RSs). The WTRU may be configured to measure a plurality of channel responses based on the received RSs to determine a plurality of channel state information (CSI) samples. The WTRU may be configured to perform an inference on the plurality of CSI samples. The WTRU may be configured to use an OOD classifier model to perform an inference on the plurality of CSI samples. The WTRU may be configured to determine a plurality of energy scores of the plurality of CSI samples. The WTRU may be configured to determine that one or more CSI samples of the plurality of CSI samples is OOD with respect to a training dataset based on the OOD measurement configuration information. The WTRU may be configured to determine an OOD frequency metric based on a number of CSI samples that are determined to be OOD. The WTRU may be configured to determine that an OOD event occurs, based on a comparison of the OOD frequency metric and a OOD event detection threshold value. The WTRU may be configured to report OOD information, based on the reporting configuration information. The OOD information may comprise at least one of: the energy score of one or more CSI samples, the OOD frequency metric; and an indication that an OOD event occurred. The WTRU may be configured to receive a configuration comprising an artificial intelligence I machine learning (AI / ML) model for CSI feedback and the OOD classifier model. The AI / ML model and the OOD classifier model are pre-trained with a training dataset comprised of a number of clusters (K). Each cluster may correspond to a different distribution. The OOD measurement configuration information may comprise at least one of: a temperature parameter (T), a metric quantifying a distance between a CSI sample and a training dataset distribution, an energy score threshold value, the OOD event detection threshold value, and a confidenceinterval. The metric quantifying a distance between a CSI sample and a training dataset distribution may be at least one of: a Z-score, first-order statistics, or an energy score. The message comprising OOD measurement configuration information and reporting configuration information may be received via radio resource control (RRC) signaling. The plurality of reference signals may be received during an OOD measurement window The determining a plurality of energy scores of the plurality of CSI samples may be activated by receiving a trigger indication for OOD monitoring. The trigger indication for OOD monitoring may be received via a downlink control information (DOI), a medium access control (MAC) control element(CE), or radio resource control (RRC) signaling.BRIEF DESCRIPTION OF THE DRAWINGS

[0004] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings, wherein like reference numerals in the figures indicate like elements, and wherein:

[0005] FIG. 1A is a system diagram illustrating an example communications system in which one or more disclosed embodiments may be implemented;

[0006] FIG. 1 B is a system diagram illustrating an example wireless transmit / receive unit (WTRU) that may be used within the communications system illustrated in FIG. 1A according to an embodiment;

[0007] FIG. 1C is a system diagram illustrating an example radio access network (RAN) and an example core network (CN) that may be used within the communications system illustrated in FIG. 1A according to an embodiment;

[0008] FIG. 1D is a system diagram illustrating a further example RAN and a further example CN that may be used within the communications system illustrated in FIG. 1A according to an embodiment;

[0009] FIG. 2 shows an example of codebook-based precoding with feedback information; and

[0010] FIG. 3 shows an example AI / ML framework for CSI feedback;

[0011] FIG. 4 shows examples of WTRU-based measurements in support of model monitoring;

[0012] FIG. 5 shows an example method for out of distribution (OOD) detection and model monitoring based on input distribution for CSI feedback;

[0013] FIG. 6 shows an example of WTRU measurement and reporting for OOD detection; and

[0014] FIG. 7 shows an example method for out of distribution (OOD) detection and model monitoring based on input distribution for CSI feedback.DETAILED DESCRIPTION

[0015] FIG. 1A is a diagram illustrating an example communications system 100 in which one or more disclosed embodiments may be implemented. The communications system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. The communications system 100 may enable multiple wireless users to access such content through thesharing of system resources, including wireless bandwidth. For example, the communications systems 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), singlecarrier FDMA (SC-FDMA), zero-tail unique-word discrete Fourier transform Spread OFDM (ZT-UW-DFT-S- OFDM), unique word OFDM (UW-OFDM), resource block-filtered OFDM, filter bank multicarrier (FBMC), and the like.

[0016] As shown in FIG. 1A, the communications system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, a radio access network (RAN) 104, a core network (ON) 106, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 112, though itwill be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment By way of example, the WTRUs 102a, 102b, 102c, 102d, any of which may be referred to as a station (STA), may be configured to transmit and / or receive wireless signals and may include a user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (loT) device, a watch or other wearable, a head-mounted display (HMD), a vehicle, a drone, a medical device and applications (e.g., remote surgery), an industrial device and applications (e.g., a robot and / or other wireless devices operating in an industrial and / or an automated processing chain contexts), a consumer electronics device, a device operating on commercial and / or industrial wireless networks, and the like. Any of the WTRUs 102a, 102b, 102c and 102d may be interchangeably referred to as a UE.

[0017] The communications systems 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the CN 106, the Internet 110, and / or the other networks 112. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a NodeB, an eNode B (eNB), a Home Node B, a Home eNode B, a next generation NodeB, such as a gNode B (gNB), a new radio (NR) NodeB, a site controller, an access point (AP), a wireless router, and the like. While the base stations 114a, 114b are each depicted as a single element, it will be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0018] The base station 114a may be part of the RAN 104, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, and the like. The base station 114a and / or the base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies, which may be referred to as a cell (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum A cell may provide coverage for a wireless service to a specific geographical areathat may be relatively fixed or that may change over time. The cell may further be divided into cell sectors. For example, the cell associated with the base station 114a may be divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, i.e., one for each sector of the cell. In an embodiment, the base station 114a may employ multiple-input multiple output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in desired spatial directions.

[0019] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0020] More specifically, as noted above, the communications system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, the base station 114a in the RAN 104 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 116 using wideband CDMA (WCDMA). WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High-Speed Downlink (DL) Packet Access (HSDPA) and / or High-Speed Uplink (UL) Packet Access (HSUPA).

[0021] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro).

[0022] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR Radio Access , which may establish the air interface 116 using NR.

[0023] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may implement LTE radio access and NR radio access together, for instance using dual connectivity (DC) principles. Thus, the air interface utilized by WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g , an eNB and a gNB).

[0024] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.11 (i.e , Wireless Fidelity (WiFi), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.

[0025] The base station 114b in FIG 1A may be a wireless router, Home Node B, Home eNode B, or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localizedarea, such as a place of business, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a roadway, and the like. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In an embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR etc.) to establish a picocell or femtocell. As shown in FIG. 1A, the base station 114b may have a direct connection to the Internet 110. Thus, the base station 114b may not be required to access the Internet 110 via the CN 106.

[0026] The RAN 104 may be in communication with the CN 106, which may be any type of network configured to provide voice, data, applications, and / or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have varying quality of service (QoS) requirements, such as differing throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, and the like. The CN 106 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and / or perform high-level security functions, such as user authentication. Although not shown in FIG. 1A, it will be appreciated that the RAN 104 and / or the CN 106 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 104 or a different RAT. For example, in addition to being connected to the RAN 104, which may be utilizing a NR radio technology, the CN 106 may also be in communication with another RAN (not shown) employing a GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0027] The CN 106 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or the other networks 112. The PSTN 108 may include circuit-switched telephone networks that provide plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and / or the internet protocol (IP) in the TCP / IP internet protocol suite. The networks 112 may include wired and / or wireless communications networks owned and / or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs, which may employ the same RAT as the RAN 104 or a different RAT.

[0028] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links). For example, the WTRU 102c shown in FIG. 1 A may be configured to communicate with the base station 114a, which may employ a cellularbased radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.

[0029] FIG. 1B is a system diagram illustrating an example WTRU 102. As shown in FIG. 1B, the WTRU 102 may include a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138, among others. It will be appreciated that the WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment.

[0030] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), any other type of integrated circuit (IC), a state machine, and the like. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. While FIG. 1 B depicts the processor 118 and the transceiver 120 as separate components, it will be appreciated that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0031] The transmit / receive element 122 may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station 114a) over the air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. In an embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals, for example. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive both RF and light signals. It will be appreciated that the transmit / receive element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0032] Although the transmit / receive element 122 is depicted in FIG. 1 B as a single element, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0033] The transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit / receive element 122 and to demodulate the signals that are received by the transmit / receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers for enabling the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11 , for example.

[0034] The processor 118 of the WTRU 102 may be coupled to, and may receive user input data from, the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit) The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 130 and / or the removable memory 132. The non-removable memory 130 may include random-accessmemory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0035] The processor 118 may receive power from the power source 134, and may be configured to distribute and / or control the power to the other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li- ion), etc.), solar cells, fuel cells, and the like.

[0036] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to, or in lieu of, the information from the GPS chipset 136, the WTRU 102 may receive location information over the air interface 116 from a base station (e.g., base stations 114a, 114b) and / or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by way of any suitable location-determination method while remaining consistent with an embodiment

[0037] The processor 118 may further be coupled to other peripherals 138, which may include one or more software and / or hardware modules that provide additional features, functionality and / or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs and / or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a handsfree headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a Virtual Reality and / or Augmented Reality (VR / AR) device, an activity tracker, and the like. The peripherals 138 may include one or more sensors. The sensors may be one or more of a gyroscope, an accelerometer, a hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor; a geolocation sensor, an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, a humidity sensor and the like.

[0038] The WTRU 102 may include a full duplex radio for which transmission and reception of some or all of the signals (e g., associated with particular subframes for both the UL (e.g., for transmission) and DL (e.g., for reception) may be concurrent and / or simultaneous. The full duplex radio may include an interference management unit to reduce and or substantially eliminate self-interference via either hardware (e.g., a choke) or signal processing via a processor (e.g., a separate processor (not shown) or via processor 118). In an embodiment, the WTRU 102 may include a half-duplex radio for which transmission and reception of some or all of the signals (e.g., associated with particular subframes for either the UL (e g., for transmission) or the DL (e g., for reception)).

[0039] FIG. 1C is a system diagram illustrating the RAN 104 and the CN 106 according to an embodiment. As noted above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0040] The RAN 104 may include eNode-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, the eNode-B 160a, for example, may use multiple antennas to transmit wireless signals to, and / or receive wireless signals from, the WTRU 102a.

[0041] Each of the eNode-Bs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, and the like. As shown in FIG. 1 C, the eNode-Bs 160a, 160b, 160c may communicate with one another over an X2 interface.

[0042] The CN 106 shown in FIG. 1C may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (PGW) 166. While the foregoing elements are depicted as part of the CN 106, it will be appreciated that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0043] The MME 162 may be connected to each of the eNode-Bs 162a, 162b, 162c in the RAN 104 via an S1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation / deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, and the like. The MME 162 may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and / or WCDMA

[0044] The SGW 164 may be connected to each of the eNode Bs 160a, 160b, 160c in the RAN 104 via the S1 interface. The SGW 164 may generally route and forward user data packets to / from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions, such as anchoring user planes during inter-eNode B handovers, triggering paging when DL data is available for the WTRUs 102a, 102b, 102c, managing and storing contexts of the WTRUs 102a, 102b, 102c, and the like.

[0045] The SGW 164 may be connected to the PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0046] The CN 106 may facilitate communications with other networks For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. Inaddition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to the other networks 112, which may include other wired and / or wireless networks that are owned and / or operated by other service providers.

[0047] Although the WTRU is described in FIGS. 1A-1 D as a wireless terminal, it is contemplated that in certain representative embodiments that such a terminal may use (e.g., temporarily or permanently) wired communication interfaces with the communication network.

[0048] In representative embodiments, the other network 112 may be a WLAN.

[0049] A WLAN in Infrastructure Basic Service Set (BSS) mode may have an Access Point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have access or an interface to a Distribution System (DS) or another type of wired / wireless network that carries traffic in to and / or out of the BSS. Traffic to STAs that originates from outside the BSS may arrive through the AP and may be delivered to the STAs. Traffic originating from STAs to destinations outside the BSS may be sent to the AP to be delivered to respective destinations. Traffic between STAs within the BSS may be sent through the AP, for example, where the source STA may send traffic to the AP and the AP may deliver the traffic to the destination STA The traffic between STAs within a BSS may be considered and / or referred to as peer-to-peer traffic. The peer-to- peer traffic may be sent between (e.g., directly between) the source and destination STAs with a direct link setup (DLS). In certain representative embodiments, the DLS may use an 802.11e DLS or an 802.11z tunneled DLS (TDLS). A WLAN using an Independent BSS (IBSS) mode may not have an AP, and the STAs (e.g., all of the STAs) within or using the IBSS may communicate directly with each other. The IBSS mode of communication may sometimes be referred to herein as an “ad-hoc” mode of communication.

[0050] When using the 802.11 ac infrastructure mode of operation or a similar mode of operations, the AP may transmit a beacon on a fixed channel, such as a primary channel. The primary channel may be a fixed width (e.g., 20 MHz wide bandwidth) or a dynamically set width. The primary channel may be the operating channel of the BSS and may be used by the STAs to establish a connection with the AP. In certain representative embodiments, Carrier Sense Multiple Access with Collision Avoidance (CSMA / CA) may be implemented, for example in 802.11 systems. For CSMA / CA, the STAs (e.g., every STA), including the AP, may sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, the particular STA may back off. One STA (e.g., only one station) may transmit at any given time in a given BSS.

[0051] High Throughput (HT) STAs may use a 40 MHz wide channel for communication, for example, via a combination of the primary 20 MHz channel with an adjacent or nonadjacent 20 MHz channel to form a 40 MHz wide channel.

[0052] Very High Throughput (VHT) STAs may support 20MHz, 40 MHz, 80 MHz, and / or 160 MHz wide channels The 40 MHz, and / or 80 MHz, channels may be formed by combining contiguous 20 MHz channels. A 160 MHz channel may be formed by combining 8 contiguous 20 MHz channels, or by combining two noncontiguous 80 MHz channels, which may be referred to as an 80+80 configuration. For the 80+80 configuration, the data, after channel encoding, may be passed through a segment parser that may divide the data into twostreams. Inverse Fast Fourier Transform (IFFT) processing, and time domain processing, may be done on each stream separately The streams may be mapped on to the two 80 MHz channels, and the data may be transmitted by a transmitting STA. At the receiver of the receiving STA, the above described operation for the 80+80 configuration may be reversed, and the combined data may be sent to the Medium Access Control (MAC).

[0053] Sub 1 GHz modes of operation are supported by 802.11 af and 802.11 ah. The channel operating bandwidths, and carriers, are reduced in 802.11 af and 802.11ah relative to those used in 802.11n, and 802.11ac. 802.11 af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV White Space (TVWS) spectrum, and 802.11 ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11 ah may support Meter Type Control / Machine- Type Communications (MTC), such as MTC devices in a macro coverage area. MTC devices may have certain capabilities, for example, limited capabilities including support for (e.g. , only support for) certain and / or limited bandwidths The MTC devices may include a battery with a battery life above a threshold (e.g., to maintain a very long battery life).

[0054] WLAN systems, which may support multiple channels, and channel bandwidths, such as 802 11 n, 802.11ac, 802.11af, and 802.11 ah, include a channel which may be designated as the primary channel. The primary channel may have a bandwidth equal to the largest common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and / or limited by a STA, from among all STAs in operating in a BSS, which supports the smallest bandwidth operating mode. In the example of 802.11 ah, the primary channel may be 1 MHz wide for STAs (e.g., MTC type devices) that support (e.g., only support) a 1 MHz mode, even if the AP, and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or Network Allocation Vector (NAV) settings may depend on the status of the primary channel. If the primary channel is busy, for example, due to a STA (which supports only a 1 MHz operating mode) transmitting to the AP, all available frequency bands may be considered busy even though a majority of the available frequency bands remains idle.

[0055] In the United States, the available frequency bands, which may be used by 802.11 ah, are from 902 MHz to 928 MHz. In Korea, the available frequency bands are from 917.5 MHz to 923.5 MHz. In Japan, the available frequency bands are from 916.5 MHz to 927.5 MHz. The total bandwidth available for 802.11 ah is 6 MHz to 26 MHz depending on the country code.

[0056] FIG. 1 D is a system diagram illustrating the RAN 104 and the CN 106 according to an embodiment. As noted above, the RAN 104 may employ an NR radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0057] The RAN 104 may include gNBs 180a, 180b, 180c, though it will be appreciated that the RAN 104 may include any number of gNBs while remaining consistent with an embodiment. The gNBs 180a, 180b, 180c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the gNBs 180a, 180b, 180c may implement MIMO technology. For example,gNBs 180a, 108b may utilize beamforming to transmit signals to and / or receive signals from the gNBs 180a, 180b, 180c. Thus, the gNB 180a, for example, may use multiple antennas to transmit wireless signals to, and / or receive wireless signals from, the WTRU 102a. In an embodiment, the gNBs 180a, 180b, 180c may implement carrier aggregation technology. For example, the gNB 180a may transmit multiple component carriers to the WTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum while the remaining component carriers may be on licensed spectrum. In an embodiment, the gNBs 180a, 180b, 180c may implement Coordinated Multi-Point (CoMP) technology. For example, WTRU 102a may receive coordinated transmissions from gNB 180a and gNB 180b (and / or gNB 180c).

[0058] The WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using transmissions associated with a scalable numerology. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing may vary for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using subframe or transmission time intervals (TTIs) of various or scalable lengths (e.g., containing a varying number of OFDM symbols and / or lasting varying lengths of absolute time).

[0059] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In the standalone configuration, WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c without also accessing other RANs (e.g., such as eNode-Bs 160a, 160b, 160c). In the standalone configuration, WTRUs 102a, 102b, 102c may utilize one or more of gNBs 180a, 180b, 180c as a mobility anchor point. In the standalone configuration, WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using signals in an unlicensed band. In a non-standalone configuration WTRUs 102a, 102b, 102c may communicate with / connect to gNBs 180a, 180b, 180c while also communicating with / connecting to another RAN such as eNode-Bs 160a, 160b, 160c. For example, WTRUs 102a, 102b, 102c may implement DC principles to communicate with one or more gNBs 180a, 180b, 180c and one or more eNode-Bs 160a, 160b, 160c substantially simultaneously. In the non- standalone configuration, eNode-Bs 160a, 160b, 160c may serve as a mobility anchor for WTRUs 102a, 102b, 102c and gNBs 180a, 180b, 180c may provide additional coverage and / or throughput for servicing WTRUs 102a, 102b, 102c.

[0060] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, support of network slicing, DC, interworking between NR and E-UTRA, routing of user plane data towards User Plane Function (UPF) 184a, 184b, routing of control plane information towards Access and Mobility Management Function (AMF) 182a, 182b and the like. As shown in FIG. 1D, the gNBs 180a, 180b, 180c may communicate with one another over an Xn interface.

[0061] The CN 106 shown in FIG. 1 D may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN)185a, 185b. While the foregoing elements are depicted as part of the CN 106, it will be appreciated that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0062] The AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 104 via an N2 interface and may serve as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, support for network slicing (e.g., handling of different protocol data unit (PDU) sessions with different requirements), selecting a particular SMF 183a, 183b, management of the registration area, termination of non-access stratum (NAS) signaling, mobility management, and the like. Network slicing may be used by the AMF 182a, 182b in order to customize CN support for WTRUs 102a, 102b, 102c based on the types of services being utilized WTRUs 102a, 102b, 102c. For example, different network slices may be established for different use cases such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, services for MTC access, and the like The AMF 182a, 182b may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies such as WiFi.

[0063] The SMF 183a, 183b may be connected to an AMF 182a, 182b in the CN 106 via an N11 interface. The SMF 183a, 183b may also be connected to a UPF 184a, 184b in the CN 106 via an N4 interface. The SMF 183a, 183b may select and control the UPF 184a, 184b and configure the routing of traffic through the UPF 184a, 184b. The SMF 183a, 183b may perform other functions, such as managing and allocating UE IP address, managing PDU sessions, controlling policy enforcement and QoS, providing DL data notifications, and the like. A PDU session type may be IP-based, non-IP based, Ethernet-based, and the like.

[0064] The UPF 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 104 via an N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPF 184, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering DL packets, providing mobility anchoring, and the like.

[0065] The CN 106 may facilitate communications with other networks For example, the CN 106 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to the other networks 112, which may include other wired and / or wireless networks that are owned and / or operated by other service providers In one embodiment, the WTRUs 102a, 102b, 102c may be connected to a local DN 185a, 185b through the UPF 184a, 184b via the N3 interface to the UPF 184a, 184b and an N6 interface between the UPF 184a, 184b and the DN 185a, 185b.

[0066] In view of FIGs. 1A-1 D, and the corresponding description of FIGs. 1A-1 D, one or more, or all, of the functions described herein with regard to one or more of: WTRU 102a-d, Base Station 114a-b, eNode-B 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF 182a-b, UPF 184a-b, SMF 183a-b, DN 185a-b,and / or any other device(s) described herein, may be performed by one or more emulation devices (not shown). The emulation devices may be one or more devices configured to emulate one or more, or all, of the functions described herein. For example, the emulation devices may be used to test other devices and / or to simulate network and / or WTRU functions.

[0067] The emulation devices may be designed to implement one or more tests of other devices in a lab environment and / or in an operator network environment. For example, the one or more emulation devices may perform the one or more, or all, functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network in order to test other devices within the communication network. The one or more emulation devices may perform the one or more, or all, functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network The emulation device may be directly coupled to another device for purposes of testing and / or performing testing using over-the-air wireless communications.

[0068] The one or more emulation devices may perform the one or more, including all, functions while not being implemented / deployed as part of a wired and / or wireless communication network. For example, the emulation devices may be utilized in a testing scenario in a testing laboratory and / or a non-deployed (e.g., testing) wired and / or wireless communication network in order to implement testing of one or more components. The one or more emulation devices may be test equipment. Direct RF coupling and / or wireless communications via RF circuitry (e.g., which may include one or more antennas) may be used by the emulation devices to transmit and / or receive data.

[0069] Embodiments are described herein to monitor the performance of WTRU-side artificial intelligence / machine learning (AI / ML) models for systems using 2-sided AI / ML for CSI feedback, including new measurements and procedures for out of distribution (OOD) detection based on statistical measurements, as well as detection of lack of model generalization. WTRU behavior upon OOD detection is also described.

[0070] FIG. 2 shows a basic concept of codebook-based precoding with feedback information. The feedback information may include a precoding matrix index (PM I) which may be referred to as a codeword index in the codebook as shown in FIG. 2.

[0071] As shown in FIG. 2, a codebook includes a set of precoding vectors / matrices for each rank and the number of antenna ports, and each precoding vectors / matrices has its own index so that a receiver may inform a preferred precoding vector / matrix index to a transmitter. The codebook-based precoding may have performance degradation due to its finite number of precoding vector / matrix as compared with non-codebook- based precoding. However, a major advantage of a codebook-based precoding may be lower control signaling / feedback overhead.

[0072] AI / ML based CSI feedback may use Autoencoders (AE) for CSI compression. This is a two-sided system, where the estimated CSI is compressed at the WTRU side, fed back to a gNB, and then decompressed at the gNB. An example AI / ML framework for CSI feedback compression is shown in FIG. 3. A main advantage of AI / ML based CSI feedback compression is performance improvement compared to legacy CSI feedbackusing a similar payload size. However, a disadvantage of AI / ML based CSI feedback compression may be a compression error that may occasionally lead to significant mismatch between the precoder computed at the WTRU (X) and the decompressed precoder at the network (NW) (i.e. gNB), X.

[0073] Detecting whether a given data sample is out-of-distribution (OOD) is a critical component of ML model performance monitoring. OOD detection may be achieved using dedicated ML models, which may be referred to as OOD classifiers, for example deep neural network (DNN) models which require supervised training or AE models which require unsupervised training.

[0074] An energy function may provide an indication of the distribution of the data at the ML model input at inference time, when the model is deployed, compared to the distribution of the training data. It assumes that the model is pre-trained to classify K different classes / dusters, each with a different distribution. The energy function attributes a high score to out-of-distribution (OOD) samples, while lower scores are attributed to indistribution samples. The tolerance of OOD inputs can be monitored and controlled by varying the hyperparameters.

[0075] The energy function maps an input data sample, which may be a vector or a matrix, to a real scalarK- value, for attributing a score to the sample x given a AI / ML classifier / : E(x, f) = -r. log^ e r , where T is a temperature parameter that may be used to improve the OOD detection performance, K is the number of classes, and / (x) is the ithreal-valued output of the ML model used for OOD detection. E(x, / ) expresses the confidence level of the AI / ML model decision associated to the input sample x, aligned with the probability density of the input samples. Samples with higher energies may be interpreted as data with a lower likelihood of occurrence (OOD samples).

[0076] Based on the energy score value, a OOD binary decision may be calculated, taking the following , , , , (out of distribution if — E(x; F) < T , . form, for examp rle: G (X T, f) = , where T IS the energy ( in distribution if - E(x; / ) > T threshold.

[0077] For systems using two-sided AI / ML models (e.g., for CSI feedback), the models may be pre-trained using training dataset(s). Upon deploying the AI / ML models, the new cases and environments experienced at inference time may have different distributions compared to the training datasets, which may lead to AI / ML model performance degradation.

[0078] Conventional approaches for model monitoring based on intermediate or system key performance indicators (KPIs) are not accurate in the sense that performance degradation of the AI / ML model is not clear whether it is caused by outdated CSI or by the model itself. Henceforth, performance monitoring based on input distribution and WTRU specific conditions seems to be more adequate, where monitoring may be performed online during the inference phase.

[0079] This following issues need to be addresses. How to monitor a two-sided model based on input distribution and WTRU specific conditions. How to calculate and report statistical measurements based on theinput distribution of samples during inference. How OOD samples are detected based on statistical measurements and how to quantify the generalization capability of the model. How to detect CSI model mismatch and lack of generalization.

[0080] In an embodiment, Out of Distribution (OOD) Detection and Model Monitoring based on Input Distribution for CSI Feedback is proposed A WTRU in a system using 2-sided AI / ML models for CSI feedback may be configured to measure and report OOD measurements (e.g. OOD detection flag, OOD frequency, energy score) during the OOD measurement mode and may report if a WTRU-side OOD event was detected. The WTRU may receive an OOD event confirmation from the NW (e.g. gNB). When the WTRU receives an OOD event confirmation from the NW, the WTRU may report historical input distribution statistics and may recommend model regularization and / or finetuning.

[0081] Out of Distribution (OOD) may refer to the distribution of the data at the ML model input at inference is different from the distribution of the training dataset. OOD classifier / OOD classifier model may be a ML model used to estimate whether an input data sample is in-distribution or OOD. OOD detection flag may be a binary decision at the output of the OOD classifier that indicates whether a current input sample is in-distribution or OOD, with respect to the training dataset OOD measurement mode may refer to, when enabled, a mode or state or time period for the WTRU to perform measurements to determine whether the input data is indistribution or OOD with respect to the model training dataset, and to provide assistance information for the detection of OOD events. An OOD measurement window may comprise multiple measurement occasions (e.g. associated with CSI-RS transmissions), and multiple reporting occasions (e.g., associated with the CSI feedback reports). OOD frequency / OOD rate may refer to a number of OOD detections relative to the number of measurements in the measurement window. An OOD event may refer to the OOD frequency, or the number of OOD detections within for example a window, exceeding a configured threshold.

[0082] The embodiments herein assume that the WTRU and NW (e g. gNB) are using a two-sided model (e g., an Autoencoder) for the CSI feedback compression (or set of AI / ML models). The encoder is assumed at the WTRU side and the decoder is assumed at the NW side. The WTRU may also be equipped with a proxy decoder for one-sided performance monitoring for each of these AI / ML models.

[0083] The models may be pre-trained offline using a mixed dataset. The dataset may be comprised of K clusters. Each cluster may be associated to a different statistical distribution of the input samples.

[0084] The WTRU may be equipped with a pre-trained OOD classifier (e.g. a function of input CSI samples and training clusters), which may be an AE or any other DNN or function that takes as input the CSI sample(s) and outputs an energy score of the input sample(s) with respect to the training clusters.

[0085] A WTRU may be configured with a model for performing OOD measurements (e.g., an OOD classifier). The model may be pre-trained using a dataset composed of K clusters. The K clusters may correspond to different distributions related to various applicable conditions impacting the statistical distribution of the CSI samples. Such applicable conditions may be, for example: different deployments, ranges of delayspread, ranges of angular speed, channel conditions, ranges of sparsity level, and / or different scenarios (e.g., indoor / outdoor).

[0086] The K clusters may be interchangeably referred to as component clusters or as training clusters herein.

[0087] The WTRU may be configured with a pre-trained OOD classifier for classifying CSI samples that do not belong to any distribution of the pre-defined K clusters. The OOD classifier may be trained on the same training dataset as the WTRU-side AI / ML CSI encoder model, for example on the same dataset of clusters.

[0088] In an embodiment, the OOD classifier may use an AE architecture. It may be preferable to use the same architecture of the WTRU-side encoder and the proxy WTRU-side decoder In this embodiment, the encoder of the OOD classifier has a similar structure to that of the WTRU encoder, and the OOD decoder may incorporate the same layers and architecture in addition to a fully connected (FC) or dense layer with several processing units (e.g. neurons) equal to K+1 , as shown in Example #1 of FIG. 4. This configuration setting of the OOD classifier enables the classification of the compressed version (bottleneck) of the input CSI sample with respect to K clusters, which may be seen as classes here, of the relevant dataset plus one cluster corresponding to out of distribution samples.

[0089] The OOD classification based on the output of the AE-based classifier may help to reduce the dimensionality of the CSI feedback, especially for configurations in which high frequencies are used in addition to massive multiple antenna arrays. The WTRU-side encoder may keep (e.g., only keep) the relevant information in the bottleneck, for example when the AE-based OOD classifier is optimized for the current conditions.

[0090] In addition to the FC layer, in an option, a Softmax output activation function may be used to produce probabilities that the input sample belongs to each of the K+1 distributions of the clusters. The Softmax activation function, when applied to the output of the WTRU-side decoder, enables to determine the confidence level of the model with respect to the clusters and may give an indication of which cluster the CSI sample is close to. In an embodiment, the energy-based activation function (which is a function of the K clusters) may be also used for activating the OOD classifier outputs, which enables a more accurate differentiation between indistribution and OOD samples. Furthermore, the energy function allows to optimize some hyperparameters that control the degree of tolerance for OOD samples for the specific encoder / decoder pair and for the WTRU applicable conditions. Hyperparameters may be for example manually set parameters such as batch size, learning rate, or number of layers, which may be configured, and may need to be optimized manually, for model training.

[0091] In an embodiment, the OOD classifier may use a specific DNN, or any function that has the property to map an input sample to a single scalar metric indicating whether the input is in-distribution or OOD, as shown in Example #2 of FIG. 4.

[0092] The WTRU may receive an OOD measurement mode activation message (e.g. trigger message) or indication (e.g. activation flag). In a measurement mode or state or a measurement time period, the WTRU may activate the OOD classifier associated to the configured WTRU-side AI / ML encoder and may perform an OOD measurement. When the OOD measurement and reporting mode results in a pair switching or mismatch decision, the WTRU may receive an OOD measurement mode deactivation message or indication, in which the WTRU may report its CSI feedback without any statistical measurements for OOD detection.

[0093] The OOD measurement mode activation / deactivation message or indication may be received via, for example, a downlink control information (DCI), a MAC control element (CE), or RRC signaling. The WTRU may keep the mode activated until it receives a deactivation message (e.g. deactivation flag).

[0094] After receiving the trigger message or activation flag, the WTRU may receive parameters and configuration information related to OOD measurements and reporting, as well as additional parameters used for statistical measurements of the input CSI.

[0095] The OOD measurement parameters may include a temperature parameter T, which may be used for the energy score calculation, and may play a key role in monitoring the tolerance of OOD samples. This parameter should be carefully optimized for a specific encoder / decoder pair and relevant dataset clusters. Another OOD measurement parameter is an energy score threshold for OOD detection, which may be used to configure the scaling of the OOD detection with respect to the energy score. It may define the ranges of energy scores that are considered in / out of distribution. In an option, a set of thresholds may be defined to enable different levels of OOD detection (i.e., decisions within ranges of confidence level). For the detection of an OOD event (i.e., whether the model lacks for generalizing), the threshold for OOD event detection may be configured, wherein this threshold may be configured adaptively based on the employed AIML encoder / decoder pair and OOD classifier.

[0096] The WTRU may also calculate or determine statistical measurements, performed directly on the input CSI sample. These measurements may include first-order statistics (e.g., mean and variance). Other measurements may be the Z-Test for producing the Z-scores. For the Z-Test, only one parameter may be required, which may be the critical value a, acting as a confidence interval is configured for each AIML encoder / decoder pair.

[0097] A WTRU may be configured with one or more measurement resources on which to perform one or more measurements to obtain data that may be used with an AI / ML model. The WTRU may determine whether at least one of the one or more measurements or data are in-distribution or out-of-distribution (OOD). A distribution may be a set of data or parameters thereof that may be used to train an AI / ML model. The WTRU may determine that at least one of the one or more measurements or data are in-distribution if the measurements (or parameters thereof) are similar or equal to or the same as that of a set of data (or parameters thereof) used to train an AI / ML model. The WTRU may determine that at least one of the one or moremeasurements or data are OOD if the measurements (or parameters thereof) are not similar or not equal to or not the same as that of all the sets of data (or parameters thereof) used to train an AI / ML model.

[0098] A WTRU may monitor (e.g. consistently monitor) the measurements or data or the inputs of an AI / ML model for OOD detection In an embodiment, a WTRU may be triggered to monitor the measurements or the inputs of an AI / ML model for OOD monitoring or detection. The WTRU may be triggered to start or stop OOD monitoring or detection

[0099] The WTRU may be triggered to start or stop OOD monitoring or detection based on reception of a message or an indication. For example, a WTRU may receive a message or an indication from a gNB to activate OOD monitoring. The indication may be received via, for example, a DCI, a MAC CE, or radio resource control (RRC) signaling.

[0100] The WTRU may be triggered to start or stop OOD monitoring or detection based on performance of the AI / ML model. For example, the WTRU may be triggered to start or stop OOD monitoring or detection based on a comparison of reconstructed feedback to uncompressed feedback. For example, the WTRU may be triggered to start or stop OOD monitoring or detection based on a KPI (e.g., compression rate, cosine similarity.) determined from the output of an AI / ML model or an AI / ML model encoder. For example, the WTRU may be triggered to start or stop OOD monitoring or detection based on a comparison to another AI / ML model or non- AI / ML function.

[0101] The WTRU may be triggered to start or stop OOD monitoring or detection based on performance of an associated transmission. For example, the WTRU may be triggered to start or stop OOD monitoring or detection based on HARQ ACK-NACK rate, or BLER of one or more associated transmissions.

[0102] The WTRU may be triggered to start or stop OOD monitoring or detection based on one or more measurements. The WTRU may be triggered for OOD monitoring based on a measurement. For example, if a measurement (e.g., reference signal received power (RSRP), received signal strength indicator (RSSI), reference signal received quality (RSRQ), rank indicator (Rl), precoding matrix indicator (PMI), channel quality indicator (CQI), signal to interference noise ratio (SINR), doppler spread, doppler shift, angle of arrival (AoA), angle of departure (AoD), delay spread, average delay, position coordinates) is below or above a threshold value, the WTRU may be triggered to perform OOD monitoring. A WTRU may be triggered to perform OOD monitoring if a measurement changes by more than a threshold value from a previous measurement of the same type. For example, if RSRP measured in two instances changes by more than a threshold value, the WTRU may be triggered to perform OOD monitoring). The measurements may be determined from a set of measurement resources and a set of instances of the measurement resources. For example, a measurement may represent a single instance (possibly obtained from multiple measurement resources) or may represent multiple instances (possibly each obtained from multiple measurement resources). The measurement whose value may be used as a trigger for OOD monitoring may be comprised of a statistic obtained from multiple instances of another measurement.

[0103] The WTRU may be triggered to start or stop OOD monitoring or detection based on a change in scenario or configuration For example, the WTRU may be triggered for OOD monitoring if it determines a change in scenario (e.g., bandwidth part (BWP), beam or transmission configuration indicator (TCI) state, coverage, line of sight (LOS) / non-line of sight (NLOS) and the like) or a change in configuration (e.g., number of transmit or receive antenna elements, panel, transmission / reception point (TRP)) from either the WTRU or gNB Coverage may refer to cell edge or cell center. For example, the AI / ML models may be trained for a range of SNR values. When the WTRU moves toward the cell edge, the SNR and the SINR may drop below the range the models were trained for. Panel may refer to antenna panels (e.g a number of antenna elements, geometry of the layout such as number of rows and columns, or distance between elements).

[0104] The WTRU may be triggered to start or stop OOD monitoring or detection based on mobility or position. For example, the WTRU may be triggered for OOD monitoring based on whether it is moving, the speed at which it is moving or the WTRU’s position (e.g. absolute position or relative position to at least one network element).

[0105] The WTRU may be triggered to start or stop OOD monitoring or detection based on a change or update of an AI / ML model. For example, the WTRU may be triggered for OOD monitoring based on reception of a new or updated AI / ML model at the WTRU or gNB or both (e g., triggered by change in AI / ML encoder, or AI / ML decoder, or change of both AI / ML encoder and AI / ML decoder). In an example, the WTRU may be triggered for OOD monitoring as a function of the WTRU-based or gNB-based and indicated selection of an AI / ML model.

[0106] The WTRU may be triggered to start or stop OOD monitoring or detection based on AI / ML model Life-Cycle Management (LCM) or LCM state. For example, the WTRU may be triggered for OOD monitoring as a function of a current, previous, or future LCM state of an AI / ML model. For example, when the WTRU determines the validity of an AI / ML model, it may start or stop OOD monitoring.

[0107] The WTRU may be triggered to start or stop OOD monitoring or detection based on time For example, a WTRU may be configured with periodic time instances for OOD monitoring. The periodic time instances may be configured with a period and an offset. In an example, the WTRU may be triggered for OOD monitoring based on a configurable time period since a last OOD monitoring period ended.

[0108] The WTRU may be triggered to start or stop OOD monitoring or detection based on reception of an aperiodic indication from, for example, a gNB. For example, a WTRU may receive an indication to perform aperiodic or one-shot OOD monitoring. For example, an aperiodic indication may include a set of measurement resources (e.g., aperiodic indication of measurement resources) on which the WTRU may perform OOD monitoring.

[0109] The WTRU may be triggered to start or stop OOD monitoring or detection based on a change in an OOD monitoring measurement resource. For example, the WTRU may be triggered to perform OOD monitoring when an OOD monitoring measurement resource is changed, removed, or added. An OOD monitoringmeasurement resource may include one or more of: an reference signal (RS) configuration, a beam or TCI state, a set of resource elements (REs), a set of time symbols, and a set of subcarriers.

[0110] The WTRU may be triggered to start or stop OOD monitoring or detection based on a determination that data from a first sub-band, cell, TRP, BWP, beam, or TCI state is OOD which may trigger a WTRU to perform OOD monitoring for a second sub-band, cell, TRP, BWP, beam, or TCI state.

[0111] A trigger or a configuration of a trigger or a configuration for OOD monitoring may include at least one of the following: duration of OOD monitoring, AI / ML model(s), set or subset of inputs to the AI / ML model(s), sets of training data (or parameters thereof) for determination of in-distribution or OOD, measurement resources, measurement type or statistic for OOD monitoring (e.g. energy score), threshold values to determine OOD (e.g., to use with a determined measurement type or statistic), triggers to report in-distribution or OOD, and resources to report in-distribution or OOD.

[0112] A WTRU may be configured with multiple triggers for OOD monitoring. A WTRU may be configured with triggers associated to distributions or datasets used to train an Al / M L model. For example, the WTRU may be configured with different measurement thresholds (possibly based on different measurements) per distributions or datasets used to train an Al / M L model.

[0113] A WTRU may select or determine an OOD monitoring type or an OOD measurement. The selection or determination may be based on the condition that triggered the WTRU to perform OOD monitoring.

[0114] A WTRU may report the outcome of an OOD monitoring procedure. The report may indicate at least one of: data used as input for an Al / M L is in-distribution, data used for an AI / ML model is OOD, or the procedure is ongoing (e.g., the data is not yet determined to be in-distribution or OOD).

[0115] The WTRU may be triggered to report the outcome of an OOD monitoring procedure

[0116] The WTRU may be triggered to report the outcome of an OOD monitoring procedure based on any of the aforementioned triggers to perform OOD monitoring. Any of the aforementioned triggers may also trigger a WTRU to report the outcome of the OOD monitoring procedure. For example, a WTRU may report (e.g. periodically report) the OOD status or the outcome of an OOD procedure determined from one or more measurement resources.

[0117] The WTRU may be triggered to report the outcome of an OOD monitoring procedure based on an OOD monitoring measurement or statistic. For example, the WTRU may be configured to determine an OOD monitoring measurement or statistic associated with a set of data. The WTRU may compare the measurement or statistic to one or more threshold values and based on a comparison, the WTRU may be triggered to report the outcome of an OOD monitoring procedure. For example, the WTRU may compare a Z-score to a threshold value and report the OOD monitoring procedure outcome if it is above or below the threshold value. In another example, the WTRU may determine the number of Z-scores that are above or below a threshold value, and if the number of Z-scores is above or below the threshold value, the WTRU may be triggered to report the OOD monitoring procedure outcome.

[0118] The WTRU may be triggered to report the outcome of an OOD monitoring procedure based on detection of OOD. In an example, a WTRU may report the outcome of an OOD monitoring procedure when an OOD event is detected An OOD event may be detected when a measurement or statistic determined from a set of data indicates that the data is OOD. In another example, the WTRU may report the outcome of an OOD monitoring procedure when x OOD events are detected, where x may be configurable. In another example, the WTRU may report the outcome of an OOD monitoring procedure when x OOD events are detected in a time period (e.g. a configurable time period). For example, the WTRU may use a sliding time window and be triggered to report the outcome of an OOD monitoring procedure when the number of OOD events in the time window exceed a value (e.g. a configurable value). In an example, the WTRU may be configured with a counter (e g. a counter value) and a timer (e.g. a time value) and upon detecting an OOD event, the WTRU may increment the counter and (re)start the timer. When the timer elapses or reaches a certain value (e.g. zero), the WTRU may decrement or reset the counter. The WTRU may be triggered to report the outcome of an OOD monitoring procedure if the counter exceeds a configurable value. In an example, the WTRU may count the number of OOD events and when the number of OOD events exceeds a first threshold value, the WTRU may start a timer. While the timer is running, the WTRU may count the number of in-distribution events. When the number of in-distribution events exceeds a second threshold value, the WTRU may stop the timer If the timer elapses without reaching a second threshold value of in-distribution events, the WTRU may determine the data is OOD and may be triggered to report the outcome of an OOD monitoring procedure (e.g., report that the data is OOD).

[0119] The WTRU may be triggered to report the outcome of an OOD monitoring procedure based on an OOD detected. An OOD detected in a first sub-band, cell, TRP, BWP, beam, or TCI state may trigger the WTRU to report an OOD status of a second sub-band, cell, TRP, BWP, beam, or TCI state.

[0120] A WTRU may determine, calculate, or compute the first-order measurements of the current CSI sample, including the mean and variance. These measurements may be calculated per each sub-band, and optionally may be averaged over the sub-bands.

[0121] The WTRU may determine or compute the Z-score for each cluster of the relevant dataset and AI / ML model. The Z-score may provide a soft metric of whether the input CSI distribution of the current sample matches or is close to the distributions of each of the dataset clusters. Given that the critical value for Z-test is configured, the WTRU may calculate the Z score zkassociated to the kthcluster, which is given by: zk= / r0, / 2kandkdenote the mean of the input CSI sample, the mean and standard deviation ofthe kthcluster, respectively, for k = 1, ... , K + 1.

[0122] If zk> a, then the input CSI is considered as an anomaly with respect to the K clusters (i.e., the input sample is OOD).

[0123] The final Z-Test based score may be obtained in different ways. In one example, the raw Z-score may be considered. In another example, the anomaly rate (i.e., number of anomalies relative to the number ofclusters K) may be considered as the final score. In an example, the K scores may be converted to binary decisions (K bits) (i.e., whether the input sample is an OOD with respect to each cluster). Another example is that the WTRU may calculate a probability distribution of the Z-scores with respect to each cluster (e.g , using a Softmax function on each Z-score).

[0124] A WTRU may generate the current CSI sample and feed it as input to the OOD classifier associated to the activated AI / ML encoder / decoder pair. Based on the output of the OOD classifier, the WTRU may collect the Energy score associated to the sample and generates an OOD detection flag, by comparing the negative energy with the energy threshold. The OOD detection flag may be equal to 1 if the negative energy score is under the energy threshold, otherwise it may be equal to 0.

[0125] Based on the OOD detection flag, the WTRU may update the OOD frequency associated to the employed AI / ML encoder / decoder pair, for the current OOD measurement window. The WTRU may determine whether an OOD event occurred, based on the measured OOD frequency and the configured OOD event detection threshold. In another example, the WTRU may determine that an OOD event occurred when the counter of OOD detection exceeds a preconfigured threshold value in a given time window.

[0126] In an embodiment, the WTRU may improve the decision of detecting an OOD (e.g. OOD detection flag) by combining the energy score and the calculated Z-score. Combining the energy score and the Z-score enables considering the input distribution based on the bottleneck (compressed version of the CSI produced by the encoder) and the input distribution from the raw CSI sample. In this embodiment, a pre-configured scaling parameter may be used to control the importance of the Z-score based scores and energy-based scores

[0127] A WTRU in a system using two-sided models for CSI feedback may be configured to report measurements (e.g., statistical measurements and / or OOD metrics), for example for model monitoring and / or life cycle management.

[0128] A WTRU may report statistical measurements to support NW-side model monitoring. The measurements of the current input CSI sample may include at least one of following.

[0129] The measurements of the current input CSI sample may include first order statistics (e.g. sample mean and standard deviation) of the current input CSI sample. The input CSI sample may be the raw channel matrix, or may be eigenvectors of the channel matrix (e.g., all eigenvectors or a subset of eigenvectors associated with the strongest eigenvalues). In an example, the WTRU may report first order statistics for each configured sub-band In another example, the WTRU may report first order statistics for the entire allocated bandwidth or bandwidth part(s) (BPW). The measurements of the current input CSI sample may include channel delay spread. The measurements of the current input CSI sample may include doppler spread. The measurements of the current input CSI sample may include angular spread. The measurements of the current input CSI sample may include statistics of the input CSI sample relative to the component clusters of the training dataset. For example: Z-scores of the input CSI sample relative to one or more (e.g., all) component clustersof the training dataset, where the Z-scores may be reported per sub-band, or may be wideband; Z-test outcome for input CSI sample relative to one or more (e g., all) of the component clusters; binary decisions for the input CSI sample relative to the component clusters, where the binary decisions may be based on statistical tests (e g., Z-test, t-test and the like); and average of the binary decisions over all component clusters.

[0130] The WTRU may report OOD metrics to support NW-side model monitoring. The OOD metrics may include at least one of following: energy score of the current CSI input sample, where the energy score may be reported per sub-band or may be wideband; OOD detection flag, where the OOD detection flag may be determined by the OOD classifier model; OOD frequency / OOD rate, which may indicate the number of OOD detections normalized to the total number of measurements in the current OOD measurement window; and OOD event flag (e.g., when the current OOD frequency exceeds a configured threshold value).

[0131] In an embodiment, a WTRU may (e.g., may additionally) report metrics related to the CSI reconstruction error (referred to as WTRU-side reconstruction metrics), for example when the WTRU uses an autoencoder (AE) based OOD classifier. When the WTRU uses an AE-based OOD classifier, the WTRU may use the decoder part of the AE (WTRU-side ML decoder) to determine additional metrics such as the CSI reconstruction error metrics. The WTRU may measure the CSI reconstruction metrics as a function of the input CSI and the reconstructed CSI at the output of the WTRU-side ML decoder. Examples of CSI reconstruction error metric include: cosine similarity (or squared cosine similarity, or generalized squared cosine similarity) and normalized mean square error (NMSE).

[0132] A WTRU may report the measurements (e.g , the statistics and / or the OOD metrics) in a periodic, semi-persistent, or aperiodic fashion. The WTRU may be configured for OOD measurement reporting via RRC configuration. The WTRU may receive the OOD measurement configuration via, for example: initial access, handover, reconfiguration of the bandwidth part, or it may be network controlled due, for example, to a change of channel conditions at the WTRU-side. Additional example may include life cycle management (LCM) of the AI / ML model, including WTRU-side ML model switch.

[0133] For periodic reporting, if configured, the WTRU may report the measurements with each CSI reporting occasion while in the OOD measurement mode. For example, when configured for periodic reporting, the WTRU may report the OOD detection flag and / or the OOD event flag using a PUCCH, and may report the statistics and / or the remaining OOD metrics and / or the WTRU-side reconstruction metrics using a PUSCH when the reporting occasion coincides with an uplink resource allocation. In an example, the WTRU may be configured with a time period and an offset from the start of the OOD measurement mode, for example different from the CSI reporting occasions, for reporting the statistics and / or OOD metrics. In this example, the WTRU may report the metrics averaged over the number of received CSI-RS transmissions since the last report

[0134] If configured for semi-persistent reporting using a PUCCH, the WTRU may report the OOD detection flag and / or the OOD event flag, for example when the WTRU receives a report activation via, for example, a MAC CE. The WTRU may report the OOD detection flag and / or the OOD event flag while in the OODmeasurement mode, until it receives a report deactivation message or indication (e.g., via a MAC CE). If configured for semi-persistent reporting using a PUSCH, the WTRU may report the statistical measurements and / or the OOD metrics and / or the WTRU-side reconstruction metrics, for example when the WTRU receives a measurement request, for example as a field in the DCI.

[0135] When configured for aperiodic reporting, the WTRU may send a message or an indication to the NW when it determines that an OOD event occurred. The WTRU may report the statistical measurements, and / or the OOD metrics, and / or the WTRU-side reconstruction metrics when it receives a message or an indication from the NW, for example via a DCI, as a result of the OOD event report.

[0136] A WTRU may behave a certain way after receiving an OOD event confirmation message or indication.

[0137] Upon or after reception of an OOD event confirmation message or indication from the network, a WTRU may be configured to do one or more of the following actions. In any of the examples listed herein, the ‘model’ may refer to the AI / ML CSI encoder or the pre-trained OOD classifier model or both.

[0138] The WTRU may be configured to report historical Z-score measurements. The WTRU may be configured to report historical input distribution statistics. The WTRU may be configured to send a message or an indication to the network requesting for a model training / retraining / fine-tu nin g. The WTRU may be configured to send a message or an indication to the network requesting for a new / updated / different dataset to retrai n / fi ne- tune the model. The WTRU may be configured to send a request message to the network to request for a dataset enhancement, for example to improve the model generalization performance (e.g., to avoid future OOD events). In an example, the WTRU may transmit the dataset used to train the WTRU-side model to the network. In another example, the network may already have the dataset and the WTRU may send a dataset cleaning request, and expect to receive an updated dataset in the downlink. The WTRU may be configured to perform data collection. For example, the WTRU may perform data collection and train / retrain / fine-tune the WTRU-side model to avoid future OOD events. The WTRU may send a request message to the network indicating the start of the data collection process (e.g. autonomous data collection). The WTRU may also indicate the purpose of the data collection process (e.g., to avoid future occurrences of OOD events). In a network-controlled scenario, the WTRU may send a request to the network to start the data collection, and the network may respond with a configuration (e.g. configuration message) for the data collection, which may include a start / end time for the data collection. The WTRU may be configured to perform model training / retraining / fine-tuning, for example on the same or a different / updated dataset, for example, received from the network. The WTRU may be configured to perform model regularization for example, by retraining on a more general dataset, by retraining on a clean dataset (excluding outlier data), and / or by modifying its objective function accordingly. The WTRU may be configured to perform model switching. For example, the WTRU may switch to another model for the same functionality (e.g., another CSI encoder and / or another OOD classifier model). The WTRU may be configured to send a message or request for model switching and / or activation to switch to another model from the network. For example, the WTRU may perform model switch upon receiving a model switch message or command fromthe NW (e.g., to ensure that the WTRU-side and NW-side models are synchronized). The WTRU may be configured to send a message or request to the network for a new model from the network. The WTRU may be configured for fallback to a default procedure (where the default procedure may be non-AI / ML CSI reporting). For example, the WTRU may switch to a default functionality for the model for which the OOD event was detected. For example, the WTRU may switch back to default CSI reporting and send a message or an indication of the switchback to the default procedure to the network. The WTRU may be configured for dataset enhancement For example, if the AI / ML CSI encoder or the OOD classifier model are trained at the WTRU, the WTRU may perform dataset enhancement before retraining / fine-tuning the models to avoid OOD events. The WTRU may be configured for a shortened prediction window (e.g., when the model is used for CSI prediction). For example, the WTRU may determine to shorten the length of the prediction window following detection of OOD events. With a shorter prediction window, the WTRU may be able to do more accurate prediction, despite detection of OOD events The WTRU may be configured to repeat the OOD measurement to make sure there has not been erroneous reporting of OOD detection.

[0139] In an embodiment, a WTRU may perform any one or more of the aforementioned actions upon reception of an OOD event confirmation message or indication from the network.

[0140] In an embodiment, the WTRU may associate the detection of the OOD event with other conditions (e g., radio conditions, such as channel conditions or SNR) and determine to do any one or more of the aforementioned actions when the condition is met. For example, the WTRU may determine that OOD events are primarily detected / measured when on cell edge and as such, only decide to do any one of the aforementioned actions (e.g., model switching or model fine-tuning) when detecting OOD events on cell edge.

[0141] FIG. 5 shows an example method for out-of-distribution (OOD) detection and model monitoring based on input distribution for CSI feedback. A WTRU may be configured with an AI / ML model, or a set of AI / ML models, for CSI feedback 510. The WTRU may be configured with a pre-trained OOD classifier model, or a set of pre-trained OOD classifier models, for each AI / ML CSI model 510. The models (i.e. AI / ML CSI model and OOD classifier model) may be trained (e.g., pre-trained or off-line trained) with a dataset comprised of K clusters. Each cluster may correspond to a different distribution. The distributions may be different due to, for example, different deployments, channel conditions, delay spread, Doppler spread, or angular spread. The WTRU may be pre-configured with the models (i.e. AI / ML CSI model and / or OOD classifier model). The WTRU may receive a configuration of the models from a network node (e.g. gNB).

[0142] The WTRU may receive a message comprising information regarding an OOD measurement configuration and / or a reporting configuration 520. The configuration information may comprise a temperature parameter T, which may be used for an energy score calculation. The configuration information may comprise an energy score threshold value for OOD detection. The configuration information may comprise a threshold value for OOD event detection. The configuration information may comprise a confidence interval for the statistical distribution measurements. The OOD measurement configuration information and the reporting configuration information may be received in a same message or a different message. The WTRU may receivethe information regarding an OOD measurement configuration and a reporting configuration via, for example, RRC signaling.

[0143] The WTRU may receive a message or trigger or indication to activate (e.g. enter / start) an OOD measurement mode (e.g. OOD measurement mode activation indication) 530. The OOD measurement mode may be activated via, for example, a MAC CE or via a field in the DCI The measurement mode may be a state or time period where the WTRU may perform OOD measurements A trigger for WTRU OOD measurements may include, for example activation of the OOD measurement mode and / or event based triggers. The OOD measurement mode may be periodic or aperiodic. The OOD measurement mode may be based on factors such as a change in a cell (e.g. cell (re)configuration) or a change in channel conditions. Event based triggers may include: performance of the AI / ML model, for example evaluated using WTRU-side intermediate KPI measurements, such as SGCS or NMSE. The OOD measurement may be triggered when the WTRU-side intermediate KPIs do not meet configured performance requirement(s).

[0144] The WTRU may receive a CSI-RS and measure a channel response (CSI) based on the received CSI-RS 540.

[0145] The WTRU may use the measured (e g. current) CSI to calculate or determine a metric to quantify a distance between the input sample and the training dataset distribution 550. The metric may be based on statistical measurements such as 1st order statistics, Z-scores or a combination thereof.

[0146] The WTRU may perform inference on the current CSI sample 560. For example, the current measured CSI (either the raw channel matrix, or the eigenvectors of the channel matrix), is applied at the input of the ML model (OOD classifier model) for inference. The WTRU may perform inference on the current CSI sample using the OOD classifier model. The WTRU may measure the OOD metrics, including, for example an energy score (e.g. output of the OOD classifier model) of the current CSI sample. The WTRU may set an OOD detection flag for the current CSI sample. The OOD detection flag may be an indication (e.g. binary flag) to indicate whether the current CSI sample is OOD. The WTRU may set the OOD detection flag based on a comparison of the energy score and a threshold value (e.g. energy score threshold value). For example, if the negative energy score is smaller than the threshold value, the WTRU may set the OOD detection flag (e.g. set to 1 or "true”) to indicate that the current CSI sample is OOD with respect to a cluster of the training data set. For example, if the negative energy score is higher than the threshold value, the WTRU may set the OOD detection flag (e.g. set to 0 or "false") to indicate that the current CSI sample is not OOD (i.e. in-distribution) with respect to a cluster of the training data set.

[0147] The WTRU may determine a number of OOD detections for a current OOD measurement window 570 (e.g. OOD frequency). The WTRU may determine whether an OOD event occurred, based on the OOD frequency (i.e the number of times an OOD detection occurred during the OOD measurement window or the number of times the OOD detection flag was set) and a configured OOD event detection threshold value. For example, if the OOD frequency, for a measurement window, is greater than the OOD event detection thresholdvalue, an OOD event may be considered to have occurred. In this case, an OOD event flag may be set to indicate that an OOD event has occurred.

[0148] The WTRU may report OOD measurements and measured statistics 580 The reported OOD measurements and measured statistics may include, for example: 1st order statistics, energy score of the current CSI sample, OOD detection flag for the current CSI sample, intermediate KPIs (e.g., SGCS and / or NMSE for the WTRU-side AI / ML CSI encoder), OOD frequency (if configured to be reported for each reporting occasion, otherwise reported at the end of the OOD measurement window), and OOD event if detected by the WTRU The WTRU may send the OOD measurements and / or measured statistics to the NW (i.e. gNB).

[0149] The WTRU may report historical Z-score measurements, with respect to each cluster, and / or may recommend model regularization and / or finetuning when it receives an OOD event confirmation message or indication from the NW 590. The WTRU may send the historical KZ-score measurements to the NW (i.e. gNB). The WTRU may perform model switching, for both the AI / ML CSI encoder and the OOD classifier model, when it receives a model switch message / command / configuration from the NW (e.g., when the NW detects a model mismatch).

[0150] The WTRU may exit or stop the OOD measurement mode when it receives an OOD measurement deactivation message or indication 595.

[0151] FIG. 6 shows an example method for a WTRU for determining and reporting OOD metrics for OOD detection and OOD event detection.

[0152] A WTRU may measure a channel response for a current received CSI-RS and may perform CSI compression 610. The WTRU may measure statistics of the current channel response (e.g. input CSI) 620. The WTRU may measure OOD metrics for the input CSI (e.g. using an OOD classifier), for example, an energy score 630. The WTRU may determine whether an OOD occurred. The WTRU may determine whether an OOD occurred based on a comparison of the energy score and an energy score threshold 640. If an OOD is determined to have occurred, the WTRU may set an OOD detection flag to indicate that an OOD has occurred. The WTRU may calculate or determine an OOD rate (e.g. OOD frequency) The OOD rate may be determined based on the number of times an OOD has occurred in a particular OOD measurement window or time period. The WTRU may count the number of times the OOD detection flag has been set or may use a counter (e.g. OOD detection counter) which may be incremented each time an OOD is determined to have occurred or an OOD detection flag has been set The WTRU may determine whether an OOD event occurred 650. The WTRU may determine whether an OOD event occurred based on the OOD frequency or rate (or OOD detection counter) and an OOD event detection threshold value. An OOD event may be determined to have occurred if the OOD frequency or rate is greater than the OOD event detection threshold value. The WTRU may report OOD information such as, for example, channel statistics, WTRU-side CSI reconstruction metrics, OOD metrics and compressed CSI 660 The WTRU may also report, as part of the OOD metrics, the energy scores for each measurement instance within the measurement window. The WTRU may determine whether it receives anOOD event confirmation message or indication 670. If the WTRU receives an OOD event confirmation, the WTRU may perform model switching or finetuning as configured 680 and may repeat the procedure for all CSI- RS samples during an OOD measurement mode 690. If the WTRU does not receive an OOD event confirmation, the WTRU may repeat the procedure for all CSI-RS samples during an OOD measurement mode 690.

[0153] FIG. 7 shows an example method for out-of-distribution (OOD) detection and model monitoring based on input distribution for CSI feedback.

[0154] The WTRU may receive a message comprising information regarding an OOD measurement configuration and / or a reporting configuration 710. The configuration information may comprise a temperature parameter T, which may be used for an energy score calculation. The configuration information may comprise an energy score threshold value for OOD detection. The configuration information may comprise a threshold value for OOD event detection. The configuration information may comprise a metric quantifying a distance between a CSI sample and a training dataset distribution. The metric quantifying a distance between a CSI sample and a training dataset distribution may be a Z-score, first-order statistics, and / or an energy score .The configuration information may comprise a confidence interval for the statistical distribution measurements The OOD measurement configuration information and the reporting configuration information may be received in a same message or a different message. The WTRU may receive the information regarding an OOD measurement configuration and a reporting configuration via, for example, RRC signaling. The OOD measurement configuration information and the reporting configuration information may be received in a same or a different message.

[0155] The WTRU may be configured with an AI / ML model, or a set of AI / ML models, for CSI feedback. The WTRU may be configured with a pre-trained OOD classifier model, or a set of pre-trained OOD classifier models, for each AI / ML CSI model. The models (i.e. AI / ML CSI model and OOD classifier model) may be trained (e.g., pre-trained or off-line trained) with a dataset comprised of K clusters. Each cluster may correspond to a different distribution. The distributions may be different due to, for example, different deployments, channel conditions, delay spread, Doppler spread, or angular spread The WTRU may be pre-configured with the models (i.e. AI / ML CSI model and / or OOD classifier model). The WTRU may receive a configuration of the models from a network node (e.g. gNB).

[0156] The WTRU may receive a message or trigger or indication to activate (e.g. enter / start) an OOD measurement mode (e.g. OOD measurement mode activation indication). The OOD measurement mode may be activated via, for example, a MAC CE or via a field in the DCI The measurement mode may be a state or time period where the WTRU may perform OOD measurements. A trigger for WTRU OOD measurements may include, for example activation of the OOD measurement mode and / or event based triggers The OOD measurement mode may be periodic or aperiodic. The OOD measurement mode may be based on factors such as a change in a cell (e.g. cell (re)configuration) or a change in channel conditions. Event based triggers may include: performance of the AI / ML model, for example evaluated using WTRU-side intermediate KPImeasurements, such as SGCS or NMSE. The 00D measurement may be triggered when the WTRU-side intermediate KPIs do not meet configured performance requirement(s).

[0157] The WTRU may receive a reference signal (RS) (e.g. CSI-RS) 720. The reference signals may be received during a measurement window.

[0158] The WTRU may measure a plurality of channel responses based on the received RSs to determine a plurality of CSI samples 730

[0159] The WTRU may perform inference on the plurality of CSI samples 740. The WTRU may perform an inference on the plurality of CSI samples, using the COD classifier model. For example, the CSI samples (e g. the raw channel matrix, or the eigenvectors of the channel matrix), may be applied at the input of the ML model (OOD classifier model) for inference.

[0160] The WTRU may determine one or more OOD metrics 750. For example the WTRU may determine a plurality of energy scores of the plurality of CSI samples. The determining a plurality of energy scores of the plurality of CSI samples may be activated or started by receiving a trigger indication for OOD monitoring. The trigger indication for OOD monitoring may be received via a downlink control information (DCI), a medium access control (MAC) control element(CE), or radio resource control (RRC) signaling. The trigger indication for OOD monitoring may indicate at least one of: a time duration for OOD monitoring, an AI / ML model, a set or subset of AI / ML model inputs, a set of training data, measurement resources, a measurement type for OOD monitoring, one or more thresholds, triggers for OOD reporting, or resources for reporting OOD.

[0161] The WTRU may determine whether one or more CSI samples of the plurality of CSI samples is OOD 760. The WTRU may determine whether one or more CSI samples of the plurality of CSI samples is OOD with respect to a training dataset based on the OOD measurement configuration information. The WTRU may determine whether one or more CSI samples of the plurality of CSI samples is OOD based on a comparison of the energy score and a threshold value (e.g. energy score threshold value). For example, if the negative energy score is smaller than the threshold value, the WTRU may determine that the current CSI sample is OOD with respect to a cluster of the training data set. For example, if the negative energy score is higher than the threshold value, the WTRU may determine that the current CSI sample is in-distribution with respect to a cluster of the training data set. The WTRU may set a metric (e.g. OOD detection flag) to indicate an OOD for the CSI samples. The OOD detection flag may be an indication (e g. binary flag) to indicate whether the CSI samples are OOD For example, the WTRU may set the OOD detection flag (e g. set to 1 or “true”) to indicate an OOD occurred and may set the OOD detection flag (e.g. set to 0 or “false”) to indicate that an OOD did not occur.

[0162] The WTRU may determine an OOD frequency metric based on a number of CSI samples that are determined to be OOD 770. The OOD frequency metric may be an indication of how many (e.g. frequency or rate) CSI samples are OOD (e.g. OOD detections). For example the WTRU may keep track of how many OOD detection flags were set or may use a counter to keep track of the number of OODs and may increment thecounter each time an OOD is determined to have occurred. The WTRU may determine the frequency metric for a current OOD measurement window.

[0163] The WTRU may determine whether an OOD event occurs or occurred 780 The WTRU may determine whether an OOD event occurred based on a comparison of the OOD frequency metric (e.g. OOD frequency or rate or counter) and a OOD event detection threshold value. For example if the OOD frequency metric is greater than or larger than the OOD event detection threshold value, then an OOD event may be determined to have occurred. The WTRU may set an OOD event flag to indicate that an OOD event has occurred.

[0164] The WTRU may report OOD information 790. The WTRU may report the OOD information based on the received reporting configuration information. The OOD information may comprise at least one of: the energy score of one or more CSI samples, the OOD frequency metric; and an indication that an OOD event occurred. The OOD information may further include for example: 1st order statistics, OOD detection flag for the CSI samples, intermediate KPIs (e.g., SGCS and / or NMSE for the WTRU-side AI / ML CSI encoder), OOD frequency metric (if configured to be reported for each reporting occasion, otherwise reported at the end of the OOD measurement window The WTRU may send the OOD information to the network (i.e. gNB).

[0165] The WTRU may report historical K Z-score measurements, with respect to each cluster, and / or may recommend model regularization and / or finetuning when it receives an OOD event confirmation message or indication from the network. The WTRU may send the historical K Z-score measurements to the network (i.e. gNB). The WTRU may perform model switching, for both the AI / ML CSI encoder and the OOD classifier model, when it receives a model switch message / command / configuration from the network (e.g., when the network detects a model mismatch).

[0166] The WTRU may exit or stop OOD measurements when it receives an OOD measurement deactivation message or indication

[0167] Although features and elements are described above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element can be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magnetooptical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

CLAIMSWhat is Claimed:

1. A method implemented by a wireless transmit / receive unit (WTRU), the method comprising: receiving a message comprising out-of-distribution (OOD) measurement configuration information and reporting configuration information; receiving a plurality of reference signals (RSs); measuring a plurality of channel responses based on the received RSs to determine a plurality of channel state information (CSI) samples; performing an inference on the plurality of CSI samples, using an OOD classifier model; determining a plurality of energy scores of the plurality of CSI samples; determining that one or more CSI samples of the plurality of CSI samples is OOD with respect to a training dataset based on the OOD measurement configuration information; determining an OOD frequency metric based on a number of CSI samples that are determined to be OOD; determining that an OOD event occurs, based on a comparison of the OOD frequency metric and a OOD event detection threshold value; and reporting OOD information, based on the reporting configuration information, wherein the OOD information comprises at least one of: the energy score of one or more CSI samples, the OOD frequency metric; and an indication that an OOD event occurred.

2. The method of claim 1, further comprising: receiving a configuration comprising an artificial intelligence / machine learning (Al / M L) model for CSI feedback and the OOD classifier model3. The method of claim 2, wherein the AI / ML model and the OOD classifier model are pretrained with a training dataset comprised of a number of clusters (K), wherein each cluster corresponds to a different distribution.

4. The method of claim 1 , wherein the OOD measurement configuration information comprises at least one of: a temperature parameter (T), a metric quantifying a distance between a CSI sample and a training dataset distribution, an energy score threshold value, the OOD event detection threshold value, and a confidence interval.

5. The method of claim 4, wherein the metric quantifying a distance between a CSI sample and a training dataset distribution is at least one of: a Z-score, first-order statistics, or an energy score.

6. The method of claim 1 , wherein the message comprising OOD measurement configuration information and reporting configuration information is received via radio resource control (RRC) signaling7. The method of claim 1 , wherein the plurality of reference signals are received during an OOD measurement window.

8. The method of claim 1 , wherein the determining a plurality of energy scores of the plurality of CSI samples is activated by receiving a trigger indication for OOD monitoring.

9. The method of claim 8, wherein the trigger indication for OOD monitoring is received via a downlink control information (DCI), a medium access control (MAC) control element(CE), or radio resource control (RRC) signaling.

10. The method of claim 8, wherein the trigger indication for OOD monitoring indicates at least one of: a time duration for OOD monitoring, an AI / L model, a set or subset of AI / L model inputs, a set of training data, measurement resources, a measurement type for OOD monitoring, one or more thresholds, triggers for OOD reporting, or resources for reporting OOD.

11. A wireless transmit / receive unit (WTRU) comprising: a receiver; a transmitter; and a processor, wherein: the receiver is configured to receive a message comprising out-of-distribution (OOD) measurement configuration information and reporting configuration information; the receiver is further configured to receive a plurality of reference signals (RSs); the processor is configured to measure a plurality of channel responses based on the received RSs to determine a plurality of channel state information (CSI) samples; the processor is further configured to perform an inference on the plurality of CSI samples, using an OOD classifier model; the processor is further configured to determine a plurality of energy scores of the plurality of CSI samples; the processor is further configured to determine that one or more CSI samples of the plurality of CSI samples is OOD with respect to a training dataset based on the OOD measurement configuration information; the processor is further configured to determine an OOD frequency metric based on a number of CSI samples that are determined to be OOD; determining that an OOD event occurs, based on a comparison of the OOD frequency metric and a OOD event detection threshold value; and the transmitter is configured to report OOD information, based on the reporting configuration information, wherein the OOD information comprises at least one of: the energy score of one or more CSI samples, the OOD frequency metric; and an indication that an OOD event occurred.

12. The WTRU of claim 11 , wherein: the receiver is further configured to receive a configuration comprising an artificial intelligence / machine learning (AI / ML) model for CSI feedback and the OOD classifier model.

13. The WTRU of claim 12, wherein the AI / ML model and the OOD classifier model are pretrained with a training dataset comprised of a number of clusters (K), wherein each cluster corresponds to a different distribution.

14. The WTRU of claim 11 , wherein the OOD measurement configuration information comprises at least one of: a temperature parameter (T), a metric quantifying a distance between a CSI sample and a training dataset distribution, an energy score threshold value, the OOD event detection threshold value, and a confidence interval.

15. The WTRU of claim 14, wherein the metric quantifying a distance between a CSI sample and a training dataset distribution is at least one of: a Z-score, first-order statistics, or an energy score.

16. The WTRU of claim 11 , wherein the message comprising OOD measurement configuration information and reporting configuration information is received via radio resource control (RRC) signaling17. The WTRU of claim 11 , wherein the plurality of reference signals are received during an OOD measurement window.

18. The WTRU of claim 11 , wherein the determining a plurality of energy scores of the plurality of CSI samples is activated by receiving a trigger indication for OOD monitoring.

19. The WTRU of claim 18, wherein the trigger indication for OOD monitoring is received via a downlink control information (DCI), a medium access control (MAC) control element(CE), or radio resource control (RRC) signaling.

20. The WTRU of claim 18, wherein the trigger indication for OOD monitoring indicates at least one of: a time duration for OOD monitoring, an AI / L model, a set or subset of AI / ML model inputs, a set of training data, measurement resources, a measurement type for OOD monitoring, one or more thresholds, triggers for OOD reporting, or resources for reporting OOD.

Images