Ml model monitoring using machine learning (ML)-based monitoring processes

Abstract

A method of wireless communication performed by a user equipment (UE) includes operating, by the UE, a first machine learning model of a plurality of first machine learning models. The first machine learning model is associated with wireless communications. The UE monitors the first machine learning model using the second machine learning model. The second machine learning model is configured to monitor the first machine learning model based on one or more distance measures between at least a first data set for training the first machine learning model and one or more inference data instances observed during operation of the first machine learning model.

Description

Technical Field

[0001] All aspects of this disclosure relate to ML model monitoring using a machine learning (ML)-based monitoring process. Background Technology

[0002] Wireless communication systems are widely deployed to provide a variety of telecommunications services such as telephone, video, data, messaging, and broadcasting. Typical wireless communication systems employ multiple access technologies that can support communication with multiple users by sharing available system resources (e.g., bandwidth, transmit power, etc.). Examples of such multiple access technologies include Code Division Multiple Access (CDMA) systems, Time Division Multiple Access (TDMA) systems, Frequency Division Multiple Access (FDMA) systems, Orthogonal Frequency Division Multiple Access (OFDMA) systems, Single Carrier Frequency Division Multiple Access (SC-FDMA) systems, Time Division Synchronous Code Division Multiple Access (TD-SCDMA) systems, and Long Term Evolution (LTE). LTE / LTE-Advanced is an enhancement set of the Universal Mobile Telecommunications System (UMTS) mobile standard released by the 3rd Generation Partnership Project (3GPP). Narrowband (NB) Internet of Things (IoT) and Enhanced Machine-Type Communications (eMTC) are enhancement sets of LTE for machine-type communications.

[0003] A wireless communication network may include multiple base stations (BSs) capable of supporting communication for multiple user equipment (UEs). UEs can communicate with the base stations (BSs) via downlinks and uplinks. A downlink (or forward link) refers to the communication link from the BS to the UE, while an uplink (or reverse link) refers to the communication link from the UE to the BS. As will be described in more detail, a BS may be referred to as a Node B, Evolved Node B (eNB), gNB, Access Point (AP), Radio Headend, Transmit and Receive Point (TRP), New Radio (NR) BS, 5G Node B, etc.

[0004] The above multiple access technologies have been adopted in various telecommunications standards to provide a common protocol that enables different user equipment to communicate at the city, country, region, and even global levels. New Radio (NR) (also known as 5G) is a set of enhancements to the LTE mobile standard released by the 3rd Generation Partnership Project (3GPP). NR is designed to better support mobile broadband internet access by using Orthogonal Frequency Division Multiplexing (OFDM) with a Cyclic Prefix (CP) on the downlink (DL), and CP-OFDM and / or SC-FDM (e.g., also known as Discrete Fourier Transform Extended OFDM (DFT-s-OFDM)) on the uplink (UL), as well as supporting beamforming, multiple-input multiple-output (MIMO) antenna technologies and carrier aggregation to improve spectral efficiency, reduce costs, improve service, utilize new spectrum, and better integrate with other open standards.

[0005] Artificial neural networks can include interconnected groups of artificial neurons (e.g., neuron models). Artificial neural networks can be computing devices or represented as methods performed by computing devices. Convolutional neural networks (such as deep convolutional neural networks) are a type of feedforward artificial neural network. Convolutional neural networks can include individual neuron layers that can be configured in tiled receptive fields. Applying neural network processing to wireless communication to achieve greater efficiency would be desirable.

[0006] Machine learning (ML) monitoring involves monitoring the mismatch between the data and the ML model. This mismatch can include, for example, data and concept drift after the ML model has been deployed. The ML model can be associated with a desired data distribution, such as the distribution of the training dataset on which the ML model is designed to operate. The relationship between input and output data can change over time, which can then affect the underlying mapping function. Therefore, the accuracy of the ML model may decrease.

[0007] For example, if an ML model operates in an environment or under different operating conditions than the specific environment and conditions in which it was trained, concept drift may occur. As a result, performance may be significantly reduced. Summary of the Invention

[0008] In some aspects of this disclosure, a method of wireless communication performed by a user equipment (UE) includes having the UE operate a first ML model among a plurality of first machine learning (ML) models. The first ML model is associated with wireless communication. The method also includes having the UE monitor the first ML model using a second ML model. The second ML model is configured to monitor the first ML model based on one or more distance measures between at least a first dataset used to train the first ML model and one or more inference data instances observed during the operation of the first ML model.

[0009] Various aspects of this disclosure relate to an apparatus including components for a first ML model of a plurality of first machine learning (ML) models to be operated by a UE. The first ML model is associated with wireless communication. The apparatus also includes components for the UE to monitor the first ML model using a second ML model. The second ML model is configured to monitor the first ML model based on one or more distance measures between at least a first dataset used to train the first ML model and one or more inference data instances observed during the operation of the first ML model.

[0010] In some aspects of this disclosure, a non-transitory computer-readable medium having non-transitory program code recorded thereon is disclosed. This program code is executed by a processor and includes program code for a first ML model among a plurality of first machine learning (ML) models to be operated by a UE. The first ML model is associated with wireless communication. The program code also includes program code for the UE to monitor the first ML model using a second ML model. The second ML model is configured to monitor the first ML model based on one or more distance measures between at least a first dataset used to train the first ML model and one or more inference data instances observed during the operation of the first ML model.

[0011] Various aspects of this disclosure relate to an apparatus having a memory and one or more processors coupled to the memory. The processors are configured for a UE to operate a first ML model among a plurality of first machine learning (ML) models. The first ML model is associated with wireless communication. The apparatus also includes components for the UE to monitor the first ML model using a second ML model. The second ML model is configured to monitor the first ML model based on one or more distance measures between at least a first dataset used to train the first ML model and one or more inference data instances observed during the operation of the first ML model.

[0012] In some aspects of this disclosure, a method of wireless communication performed by a network device includes receiving distance metric information from a user equipment (UE) associated with a first machine learning (ML) model of the wireless communication. The method also includes monitoring the first ML model using a second ML model based on the distance metric information. Furthermore, the method includes determining, based on the distance metric information, whether to select a different first ML model from a plurality of first ML models for UE operation, fine-tune the first ML model, or revert to a non-machine learning model.

[0013] Various aspects of this disclosure relate to an apparatus including components for receiving distance metric information from a user equipment (UE) associated with a first machine learning (ML) model in relation to wireless communication. The apparatus also includes components for monitoring the first ML model using a second ML model based on the distance metric information. Furthermore, the apparatus includes components for determining, based on the distance metric information, whether to select a different first ML model from a plurality of first ML models for UE operation, fine-tune the first ML model, or revert to a non-machine learning model.

[0014] In some aspects of this disclosure, a non-transitory computer-readable medium having non-transitory program code recorded thereon is disclosed. This program code is executed by a processor and includes program code for receiving distance metric information from a user equipment (UE) associated with a first machine learning (ML) model in relation to wireless communication. The program code also includes program code for monitoring the first ML model using a second ML model based on the distance metric information. Furthermore, the program code includes program code for determining, based on the distance metric information, whether to select a different first ML model from a plurality of first ML models for UE operation, fine-tune the first ML model, or revert to a non-machine learning model.

[0015] Another aspect of this disclosure relates to an apparatus having at least one memory and one or more processors coupled to the at least one memory. The processors are configured to receive distance metric information from a user equipment (UE) associated with a first machine learning (ML) model in connection with wireless communication. The apparatus also includes components for monitoring the first ML model using a second ML model based on the distance metric information. The apparatus further includes components for determining, based on the distance metric information, whether to select a different first ML model from a plurality of first ML models for UE operation, fine-tune the first ML model, or revert to a non-machine learning model.

[0016] The aspects as a whole include, as described substantially with reference to the accompanying drawings and description, and as illustrated in the accompanying drawings and description, methods, apparatus, systems, computer program products, non-transitory computer-readable media, user equipment, base stations, wireless communication devices, and processing systems.

[0017] The features and technical advantages of the examples according to this disclosure have been summarized rather broadly above in order to better understand the detailed description that follows. Additional features and advantages will be described. The disclosed concepts and specific examples can be readily used as the basis for modifying or designing other structures for achieving the same purpose as this disclosure. Such equivalent constructions do not depart from the scope of the appended claims. The characteristics of the disclosed concepts, in both their organization and manner of operation, and the associated advantages, will be better understood by considering the following description in conjunction with the accompanying drawings. Each drawing in the accompanying drawings is for illustrative and descriptive purposes and not as a definition of limitation of the claims. Attached Figure Description

[0018] To gain a detailed understanding of the features of this disclosure, reference can be made to various aspects, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only certain aspects of this disclosure and should therefore not be considered as limiting its scope, as the description may allow for other equivalent aspects. The same reference numerals in different drawings may identify the same or similar elements.

[0019] Figure 1 It is a block diagram that conceptually illustrates examples of wireless communication networks according to various aspects of this disclosure.

[0020] Figure 2 This is a block diagram that conceptually illustrates examples of communication between a base station and a user equipment (UE) in a wireless communication network according to various aspects of this disclosure.

[0021] Figure 3 This is a block diagram illustrating an example decomposed base station architecture according to various aspects of this disclosure.

[0022] Figure 4 Example implementations of designing neural networks using a system-on-a-chip (SoC) including a general-purpose processor, according to certain aspects of this disclosure, are illustrated.

[0023] Figure 5A , Figure 5B and Figure 5C These are illustrations of neural networks according to various aspects of this disclosure.

[0024] Figure 5D This is a diagram illustrating exemplary deep convolutional networks (DCNs) according to various aspects of this disclosure.

[0025] Figure 6 This is a block diagram illustrating exemplary deep convolutional networks (DCNs) according to various aspects of this disclosure.

[0026] Figure 7 This is a diagram illustrating examples of machine learning (ML) based monitoring using K-nearest neighbors (KNN) according to various aspects of this disclosure.

[0027] Figure 8 This is a diagram illustrating examples of ML models that monitor complexity reduction by diluting training samples according to various aspects of this disclosure.

[0028] Figure 9 This is a diagram illustrating examples of ML models that monitor complexity reduction by clustering training samples according to various aspects of this disclosure.

[0029] Figure 10 This is a flowchart illustrating an example process of ML model monitoring performed by a UE according to various aspects of this disclosure.

[0030] Figure 11 This is a flowchart illustrating an example process of ML model monitoring performed by a base station according to various aspects of this disclosure. Detailed Implementation

[0031] Various aspects of this disclosure are described more fully below with reference to the accompanying drawings. However, this disclosure may be embodied in many different forms and should not be construed as limited to any particular structure or function presented throughout this disclosure. Rather, these aspects are provided so that this disclosure will be comprehensive and complete, and will fully convey the scope of protection of this disclosure to those skilled in the art. Based on the teachings, those skilled in the art will recognize that the scope of this disclosure is intended to cover any aspect of this disclosure, whether implemented independently of or in combination with any other aspect of this disclosure. For example, an apparatus or method may be implemented using any number of the aspects set forth. Furthermore, the scope of this disclosure is intended to cover such apparatus or methods practiced using other structures, functions, or structures and functions other than or supplementing the various aspects of this disclosure set forth. It should be understood that any aspect of this disclosure may be embodied by one or more elements of the claims.

[0032] Several aspects of a telecommunications system will now be presented with reference to various devices and techniques. These devices and techniques will be described in detail below and illustrated in the accompanying drawings by various blocks, modules, components, circuits, steps, processes, algorithms, etc. (collectively, “elements”). These elements can be implemented using hardware, software, or a combination thereof. Whether such elements are implemented as hardware or software depends on the specific application and the design constraints imposed on the system as a whole.

[0033] It should be noted that although the aspects may be described using terms commonly associated with 5G, 6G and other related wireless technologies, the aspects of this disclosure may be applied in other generation-based communication systems, such as and including (but not limited to) 3G and / or 4G technologies.

[0034] As described, machine learning model concept drift involves a mismatch between the data (such as the training dataset) and the machine learning model, which may develop after the machine learning model has been deployed. Therefore, it may be desirable to use certain metrics to monitor the machine learning (ML) model in order to keep the ML model in the desired operating scenario.

[0035] When a data model mismatch is detected (e.g., concept drift), possible actions may include (but are not limited to) switching between different ML models (i.e., selecting a different ML model), reverting to a non-ML model, training a global ML model that generalizes well in different environments, or retraining or fine-tuning the ML model online.

[0036] When training an ML model in a specific environment (e.g., indoor vs. outdoor, busy vs. sparse, urban vs. rural), or configuration, the distribution of the ML model's inputs and outputs (such as ground reality labels) can be affected. By observing the distribution of the input or output data after inference, the monitoring process can determine different actions, such as switching to an ML model or reverting to a non-ML operation.

[0037] One approach to monitoring ML models is to measure distribution similarity. For example, the similarity between the distribution of inference input data and the input distribution of different ML models can be measured using the Kolmogorov-Smirnov (KS) distance, Earth mover distance, Kullback–Leibler (KL) divergence, or other similarity measures.

[0038] The monitoring process can select an ML model (e.g., the current ML model or a different ML model) based on the similarity between the input data distribution and the inference data distribution (e.g., the input data distribution with the highest similarity to the inference data distribution). The monitoring process can set a threshold for distribution similarity before determining whether to revert to a non-ML operation. This method advantageously provides a simple, low-complexity approach for ML model monitoring. However, distribution similarity methods rely on the availability of previously recent inference data to generate the inference data distribution and compare it with the training data distribution.

[0039] The ML monitoring process compares the similarity between the input and output data distributions of the training and inference data. If there is a high similarity between the training and inference data distributions, the monitoring process determines that the ML model is suitable for inference. Conversely, if there is a low similarity between the training and inference data distributions, the monitoring process determines that the ML model is not suitable for inference. In this case, the monitoring process can be used to determine whether to switch to an ML model or revert to a non-ML operation.

[0040] However, monitoring ML models based on data distributions assumes the availability of inference data distributions, which may not always be available. For example, if the UE or network has not performed any recent inference, the monitoring process may lack any inference data distributions to assess the suitability of a particular ML model. In such scenarios, conventional methods may not be able to use data distributions to monitor ML models.

[0041] The performance of data distribution-based monitoring can be further enhanced by performing additional processing on the input data, for example, using ML monitoring procedures. Detection of data or concept drift can be improved by performing additional processing on the data distribution (e.g., by using a classic ML model).

[0042] Therefore, to address the challenge of distribution similarity, aspects of this disclosure relate to using ML-based monitoring procedures to monitor ML models, which can be applied to as few as a single data instance. For example, in some aspects, the ML-based monitoring procedure may rely on K-nearest neighbor (KNN) information (e.g., KNN distance, KNN majority vote, K-mean distance, etc.) to monitor the ML model. The proximity between newly received inference data instances and training data can be used to monitor the suitability of the ML model, select an ML model from a set of ML models, revert to non-ML operations, or determine whether to fine-tune the ML model.

[0043] Specific aspects of the subject matter described in this disclosure can be implemented to achieve one or more of the following potential advantages. In some examples, the described techniques reduce concept drift and increase the accuracy and usability of ML models.

[0044] Figure 1 This is an illustration of a network 100 in which various aspects of the present disclosure may be practiced. Network 100 may be a 5G or NR network, or some other wireless network (such as an LTE network). Wireless network 100 may include multiple BS 110s (shown as BS 110a, BS 110b, BS 110c, and BS 110d) and other network entities. A BS is an entity that communicates with a user equipment (UE) and may also be referred to as a base station, NR BS, Node B, gNB, 5G Node B, access point, Transmit and Receive Point (TRP), network node, network entity, etc. A BS may be implemented as an aggregated base station, a decomposed base station, an Integrated Access and Backhaul (IAB) node, a relay node, a sidelink node, etc. A BS may be implemented in an aggregated or monolithic base station architecture, or alternatively in a decomposed base station architecture, and may include one or more of a central unit (CU), a distributed unit (DU), a radio unit (RU), a near real-time (near RT) RAN intelligent controller (RIC), or a non-real-time (non-RT) RIC. Each BS can provide communication coverage for a specific geographic area. In 3GPP, depending on the context in which the term is used, the term "cell" can refer to the coverage area of ​​a BS and / or the BS subsystem serving that coverage area.

[0045] A BS can provide communication coverage for macrocells, picocells, femtocells, and / or another type of cell. A macrocell can cover a relatively large geographic area (e.g., a radius of several kilometers) and can allow unrestricted access for UEs with a service subscription. A picocell can cover a relatively small geographic area and can allow unrestricted access for UEs with a service subscription. A femtocell can cover a relatively small geographic area (e.g., a home) and can allow restricted access for UEs associated with the femtocell (e.g., UEs in a Closed Subscriber Group (CSG)). A BS used for macrocells can be referred to as a macro BS. A BS used for picocells can be referred to as a pico BS. A BS used for femtocells can be referred to as a femtocell BS or a home BS. Figure 1 In the example shown, BS 110a can be a macro BS for macro cell 102a, BS 110b can be a pico BS for pico cell 102b, and BS 110c can be a femto BS for femto cell 102c. A BS can support one or more (e.g., three) cells. The terms “eNB,” “base station,” “NR BS,” “gNB,” “AP,” “node B,” “5G NB,” “TRP,” and “cell” are used interchangeably.

[0046] In some respects, the cell does not need to be stationary, and the geographical area of ​​the cell can be moved depending on the location of the mobile BS. In some respects, the BS can use any suitable transport network to interconnect with each other and / or with one or more other BSs or network nodes (not shown) in the wireless network 100 via various types of backhaul interfaces (such as direct physical connections, virtual networks, etc.).

[0047] The wireless network 100 may also include a relay station. A relay station is an entity that can receive data transmissions from an upstream station (e.g., a BS or a UE) and forward those data transmissions to a downstream station (e.g., a UE or a BS). A relay station can also be a UE that can relay transmissions for other UEs. Figure 1 In the example shown, relay station 110d can communicate with macro BS 110a and UE 120d to facilitate communication between BS 110a and UE 120d. A relay station can also be referred to as a relay BS, relay base station, repeater, etc.

[0048] Wireless network 100 can be a heterogeneous network comprising different types of Base Stations (BSs) (e.g., macro BSs, pico BSs, femto BSs, relay BSs, etc.). These different types of BSs can have different transmit power levels, different coverage areas, and different effects on interference in wireless network 100. For example, macro BSs can have high transmit power levels (e.g., 5 watts to 40 watts), while pico BSs, femto BSs, and relay BSs can have lower transmit power levels (e.g., 0.1 watts to 2 watts).

[0049] Network controller 130 can be coupled to a group of base stations (BSs) and can provide coordination and control for these BSs. Network controller 130 can communicate with the BSs via backhaul. The BSs can also communicate with each other via wireless or wired backhaul (e.g., directly or indirectly).

[0050] UEs 120 (e.g., 120a, 120b, 120c) may be distributed throughout the wireless network 100, and each UE may be stationary or mobile. UEs may also be referred to as access terminals, terminals, mobile stations, subscriber units, stations, etc. UEs may be cellular phones (e.g., smartphones), personal digital assistants (PDAs), wireless modems, wireless communication devices, handheld devices, laptops, cordless phones, wireless local loop (WLL) stations, tablets, cameras, gaming devices, netbooks, smartbooks, ultrabooks, medical devices or equipment, biometric sensors / devices, wearable devices (smartwatches, smart clothing, smart glasses, smart wristbands, smart jewelry (e.g., smart rings, smart bracelets)), entertainment devices (e.g., music or video devices or satellite radios), components or sensors of vehicles, smart meters / sensors, industrial manufacturing equipment, GPS devices, or any other suitable device configured to communicate via wireless or wired media.

[0051] Some UEs can be considered Machine-Type Communication (MTC) or Evolved or Enhanced Machine-Type Communication (eMTC) UEs. For example, MTC and eMTC UEs include robots, drones, remote devices, sensors, meters, monitors, location tags, etc., that can communicate with a base station, another device (e.g., a remote device), or some other entity. Wireless nodes can provide connectivity to or to a network (such as the Internet or a wide area network with cellular networks) via wired or wireless communication links. Some UEs can be considered Internet of Things (IoT) devices and / or can be implemented as NB-IoT (Narrowband Internet of Things) devices. Some UEs can be considered Customer Premises Equipment (CPE). UE 120 can be included in a housing that houses the components of UE 120, such as processor components, memory components, etc.

[0052] Generally, any number of wireless networks can be deployed in a given geographical area. Each wireless network can support a specific Radio Access Technology (RAT) and can operate on one or more frequencies. A RAT can also be referred to as a radio technology, air interface, etc. A frequency can also be referred to as a carrier, frequency channel, etc. Within a given geographical area, each frequency can support a single RAT to avoid interference between wireless networks using different RATs. In some cases, NR or 5G RAT networks can be deployed.

[0053] In some respects, two or more UEs 120 (e.g., shown as UE 120a and UE 120e) may communicate directly using one or more sidelink channels (e.g., without using base station 110 as an intermediary). For example, UEs 120 may communicate using peer-to-peer (P2P) communication, device-to-device (D2D) communication, vehicle-to-everything (V2X) protocols (e.g., which may include vehicle-to-vehicle (V2V) protocols, vehicle-to-infrastructure (V2I) protocols, mesh networks, etc.). In this case, UEs 120 may perform scheduling operations, resource selection operations, and / or other operations described elsewhere herein, performed by base station 110. For example, base station 110 may configure UEs 120 via downlink control information (DCI), radio resource control (RRC) signaling, media access control-control element (MAC-CE), or via system information (e.g., system information block (SIB)).

[0054] UE 120 may include a machine learning (ML) monitoring module 140. For simplicity, only one UE 120d is shown as including the ML monitoring module 140. The ML monitoring module 140 may be used by the UE to monitor a first machine learning model to detect concept drift using a second machine learning model, the second machine learning model being configured to monitor the first machine learning model based on one or more distance measures between a first dataset used to train the first machine learning model and one or more inference data instances generated by the first machine learning model.

[0055] Network controller 130 or base station 110 or any other network device (e.g., such as...) Figure 3 The (as seen) may include an ML monitoring module 138, which is used to receive distance metric information from a user equipment (UE) associated with a first machine learning (ML) model for wireless communication, monitor the first ML model using a second ML model based on the distance metric information, and determine, based on the distance metric information, whether to select a different first ML model from a plurality of first ML models for UE operation, fine-tune the first ML model, or revert to a non-machine learning model.

[0056] As indicated above, Figure 1 This is provided merely as an example. Other examples are available in conjunction with [the relevant documentation / information]. Figure 1 The examples described are different.

[0057] Figure 2 A block diagram of a design 200 for a base station 110 and a UE 120 is shown. The base station and the UE can be... Figure 1 One of the base stations in the base station and Figure 1 One of the UEs in the UE. Base station 110 may be equipped with T antennas 234a to 234t, and UE 120 may be equipped with R antennas 252a to 252r, where generally, T≥1 and R≥1.

[0058] At base station 110, transmitting processor 220 can receive data for one or more UEs from data source 212, select one or more modulation and decoding schemes (MCS) for each UE based at least in part on the channel quality indicator (CQI) received from each UE, process (e.g., encode and modulate) the data for each UE based at least in part on the MCS selected for each UE, and provide data symbols for all UEs. Reducing the MCS decreases throughput but increases transmission reliability.

[0059] The transmit processor 220 can also process system information (e.g., semi-static resource partitioning information (SRPI) and control information (e.g., CQI requests, grants, upper-layer signaling, etc.) and provide overhead symbols and control symbols. The transmit processor 220 can also generate reference symbols for reference signals (e.g., cell-specific reference signals (CRS)) and synchronization signals (e.g., primary synchronization signal (PSS) and secondary synchronization signal (SSS)). The transmit (TX) multiple-input multiple-output (MIMO) processor 230 can perform spatial processing (e.g., pre-decoding) on ​​data symbols, control symbols, overhead symbols, and / or reference symbols where applicable, and can provide T output symbol streams to T modulators (MODs) 232a to 232t.

[0060] Each modulator 232 can process the corresponding output symbol stream (e.g., for Orthogonal Frequency Division Multiplexing (OFDM)) to obtain an output sample stream. Each modulator 232 can further process (e.g., convert to analog, amplify, filter, and up-convert) the output sample stream to obtain a downlink signal. The T downlink signals from modulators 232a to 232t can be transmitted via T antennas 234a to 234t respectively. Position coding can be used to generate synchronization signals to convey additional information, according to various aspects described in more detail below.

[0061] At UE 120, antennas 252a to 252r can receive downlink signals from base station 110 and / or other base stations, and can provide the received signals to demodulators (DEMODs) 254a to 254r respectively. Each demodulator 254 can adjust (e.g., filter, amplify, down-convert, and digitize) the received signal to obtain an input sample. Each demodulator 254 can further process these input samples (e.g., for OFDM, etc.) to obtain the received symbols. MIMO detector 256 can obtain the received symbols from all R demodulators 254a to 254r, perform MIMO detection on the received symbols where applicable, and provide the detected symbols. Receiver processor 258 can process (e.g., demodulate and decode) the detected symbols, provide decoded data for UE 120 to data sink 260, and provide decoded control information and system information to controller / processor 280. The channel processor can determine the Reference Signal Received Power (RSRP), Received Signal Strength Indicator (RSSI), Reference Signal Received Quality (RSRQ), and / or Channel Quality Indicator (CQI), etc. In some aspects, one or more components of the UE 120 may be included in a housing.

[0062] On the uplink, at UE 120, the transmitting processor 264 can receive data from data source 262 and control information (e.g., for reports including RSRP, RSSI, RSRQ, CQI, etc.) from controller / processor 280, and process the data and control information. The transmitting processor 264 can also generate reference symbols for one or more reference signals. Symbols from the transmitting processor 264 can be pre-decoded by the TX MIMO processor 266, where applicable, further processed by modulators 254a to 254r (e.g., for Discrete Fourier Transform Extended OFDM (DFT-s-OFDM), CP-OFDM, etc.), and transmitted to base station 110. At base station 110, uplink signals from UE 120 and other UEs can be received by antenna 234, processed by demodulator 254, detected by MIMO detector 236 (where applicable), and further processed by receiving processor 238 to obtain decoded data and control information transmitted by UE 120. The receiver processor 238 can provide the decoded data to the data sink 239 and the decoded control information to the controller / processor 240. The base station 110 may include a communication unit 244 and communicates with the network controller 130 via the communication unit 244. The network controller 130 may include a communication unit 294, a controller / processor 290, and a memory 292.

[0063] The controller / processor 240 of base station 110, the controller / processor 280 of UE 120 and / or Figure 2Any other component may perform one or more techniques associated with determining membership in region-based joint learning, as described in more detail elsewhere. For example, the controller / processor 240 of base station 110, the controller / processor 280 of UE 120, and / or Figure 2 Any other component that can execute or direct, for example Figures 9 to 10 The operation of the process and / or other processes as described. Memory 242 and memory 282 may store data and program code for base station 110 and UE 120, respectively. Scheduler 246 may schedule UEs for data transmission on downlink and / or uplink.

[0064] In some aspects, UE 120 and / or base station 110 may include components for operation, components for monitoring, components for receiving, and components for using a second ML model and / or components for determination. Such components may include combinations of... Figure 2 The UE 120 or one or more components of the base station 110 described.

[0065] As indicated above, Figure 2 This is provided merely as an example. Other examples are available in conjunction with [the relevant documentation / information]. Figure 2 The examples described are different.

[0066] In some cases, different types of devices supporting different types of applications and / or services can coexist in a cell. Examples of different types of devices include UE handsets, Customer Premises Equipment (CPE), vehicles, Internet of Things (IoT) devices, etc. Examples of different types of applications include Ultra Reliable Low Latency Communication (URLLC) applications, Massive Machine-Type Communication (mMTC) applications, Enhanced Mobile Broadband (eMBB) applications, Vehicle-to-Everything (V2X) applications, etc. Furthermore, in some cases, a single device can simultaneously support different applications or services.

[0067] Communication systems, such as 5G New Radio (NR) systems, can be deployed with various components or parts in multiple ways. In a 5G NR system or network, network nodes, network entities, network mobility elements, radio access network (RAN) nodes, core network nodes, network elements or network equipment (such as base stations (BS)) or one or more units (or components) performing base station functions can be implemented in aggregated or decomposed architectures. For example, BSs (such as Node B (NB), evolved NB (eNB), NR BS, 5G NB, access point (AP), transmit and receive point (TRP), or cell, etc.) can be implemented as aggregated base stations (also known as standalone BS or monolithic BS) or decomposed base stations.

[0068] Aggregated base stations can be configured to utilize a radio protocol stack that is physically or logically integrated within a single RAN node. Decomposed base stations can be configured to utilize a protocol stack that is physically or logically distributed across two or more units, such as one or more central or centralized units (CUs), one or more distributed units (DUs), or one or more radio units (RUs). In some respects, the CU may be implemented within a RAN node, and one or more DUs may co-located with the CU, or alternatively, may be geographically or virtually distributed across one or more other RAN nodes. DUs may be implemented to communicate with one or more RUs. Each of the CUs, DUs, and RUs may also be implemented as a virtual unit, namely a virtual centralized unit (VCU), a virtual distributed unit (VDU), or a virtual radio unit (VRU).

[0069] Base station type operation or network design can take into account the aggregation characteristics of base station functionality. For example, decomposed base stations can be utilized in Integrated Access Backhaul (IAB) networks, Open Radio Access Networks (O-RAN (such as network configurations initiated by the O-RAN Alliance)), or Virtualized Radio Access Networks (vRAN, also known as Cloud Radio Access Networks (C-RAN)). Decomposition can include distributing functionality across two or more units in various physical locations, as well as virtually distributing the functionality of at least one unit, which enables flexibility in network design. The various units in a decomposed base station or decomposed RAN architecture can be configured for wired or wireless communication with at least one other unit.

[0070] Figure 3 A diagram illustrating an example of a decomposed base station 300 architecture is shown. The decomposed base station 300 architecture may include one or more central units (CUs) 310, which may communicate directly with the core network 320 via a backhaul link, or indirectly with the core network 320 via one or more decomposed base station units, such as a near real-time (near-RT) RAN Intelligent Controller (RIC) 325 via an E2 link, or a non-real-time (non-RT) RIC 315 associated with a Service Management and Orchestration (SMO) framework 305, or both. CUs 310 may communicate with one or more distributed units (DUs) 330 via appropriate midhaul links (such as F1 interfaces). DUs 330 may communicate with one or more radio units (RUs) 340 via appropriate fronthaul links. RUs 340 may communicate with corresponding UEs 120 via one or more radio frequency (RF) access links. In some implementations, a UE 120 may be served simultaneously by multiple RUs 340.

[0071] Each of these units (e.g., CU 310, DU 330, RU 340, and near-RT RIC 325, non-RT RIC 315, and SMO frame 305) may include one or more interfaces, or may be coupled to one or more interfaces configured to receive or transmit signals, data, or information (collectively, signals) via wired or wireless transmission media. Each of the units, or an associated processor or controller providing instructions to the communication interfaces of these units, may be configured to communicate with one or more other units via transmission media. For example, these units may include wired interfaces configured to receive signals or transmit signals to one or more other units via wired transmission media. Additionally, these units may include wireless interfaces that may include receivers, transmitters, or transceivers (such as radio frequency (RF) transceivers) configured to receive signals or transmit signals to one or more other units via wireless transmission media, or both.

[0072] In some aspects, the CU 310 can host one or more higher-level control functions. Such control functions may include Radio Resource Control (RRC), Packet Data Convergence Protocol (PDCP), Serving Data Adaptation Protocol (SDAP), etc. Each control function can be implemented using an interface configured to signal to other control functions hosted by the CU 310. The CU 310 can be configured to handle user plane functionality (i.e., Central Unit-User Plane (CU-UP)), control plane functionality (i.e., Central Unit-Control Plane (CU-CP)), or a combination thereof. In some implementations, the CU 310 can be logically divided into one or more CU-UP units and one or more CU-CP units. When implemented in an O-RAN configuration, the CU-UP units can communicate bidirectionally with the CU-CP units via an interface such as an E1 interface. The CU 310 can be implemented to communicate with the DU 330 for network control and signaling, as needed.

[0073] DU 330 may correspond to a logical unit comprising one or more base station functions for controlling the operation of one or more RU 340s. In some aspects, DU 330 may, at least in part, host one or more of the Radio Link Control (RLC) layer, the Media Access Control (MAC) layer, and one or more high physical (PHY) layers (such as modules for forward error correction (FEC) encoding and decoding, scrambling, modulation and demodulation, etc.) depending on functional splits (such as those defined by the 3rd Generation Partnership Project (3GPP). In some aspects, DU 330 may also host one or more low PHY layers. Each layer (or module) may be implemented using an interface configured to communicate signals with other layers (and modules) hosted by DU 330 or with control functions hosted by CU310.

[0074] Lower-layer functionality can be implemented by one or more RU 340s. In some deployments, an RU340 controlled by a DU 330 may correspond to a logical node that at least partially hosts RF processing functions or low-PHY layer functions (such as performing Fast Fourier Transform (FFT), Inverse FFT (iFFT), digital beamforming, Physical Random Access Channel (PRACH) extraction and filtering, etc.) based on functional decomposition such as lower-layer functional decomposition, or both. In this architecture, the RU 340 can be implemented to handle over-the-air (OTA) communications with one or more UE 120s. In some specific implementations, the real-time and non-real-time aspects of communication with the control plane and user plane of the RU 340 may be controlled by the corresponding DU 330. In some scenarios, this configuration allows the DU330 and CU 310 to be implemented in cloud-based RAN architectures such as vRAN architectures.

[0075] The SMO framework 305 can be configured to support RAN deployment and provisioning of both non-virtualized and virtualized network elements. For non-virtualized network elements, the SMO framework 305 can be configured to support the deployment of dedicated physical resources for RAN coverage requirements, which can be managed via operation and maintenance interfaces such as the O1 interface. For virtualized network elements, the SMO framework 305 can be configured to interact with a cloud computing platform such as the Open Cloud (O-Cloud) 390 to perform network element lifecycle management (such as instantiating virtualized network elements) via a cloud computing platform interface such as the O2 interface. Such virtualized network elements may include, but are not limited to, CU 310, DU 330, RU 340, and near-RT RIC 325. In some implementations, the SMO framework 305 can communicate with the hardware aspects of the 4G RAN (such as the Open eNB (O-eNB) X11) via the O1 interface. Additionally, in some implementations, the SMO framework 305 can communicate directly with one or more RU 340s via the O1 interface. The SMO framework 305 may also include a non-RT RIC 315 configured to support the functionality of the SMO framework 305.

[0076] The non-RT RIC 315 can be configured to include logical functions that enable non-real-time control and optimization of RAN elements and resources, including machine learning (ML) workflows for model training and updates, or policy-based guidance for applications / features in the near-RT RIC 325. The non-RT RIC 315 can be coupled to or communicate with the near-RT RIC 325, such as via an A1 interface. The near-RT RIC 325 can be configured to include logical functions that enable near real-time control and optimization of RAN elements and resources via data collection and actions on interfaces connecting one or more CUs 310s, one or more DUs 330s, or both, and O-eNBs to the near-RT RIC 325, such as via an E2 interface.

[0077] In some implementations, to generate ML models to be deployed in the near-RT RIC 325, the non-RT RIC 315 may receive parameters or external enrichment information from an external server. This information can be utilized by the near-RT RIC 325 and may be received from non-network data sources or network functions at the SMO framework 305 or the non-RT RIC 315. In some examples, the non-RT RIC 315 or near-RT RIC 325 may be configured to tune RAN behavior or performance. For example, the non-RT RIC 315 may monitor long-term trends and patterns in performance and employ ML models to perform corrective actions via the SMO framework 305 (such as reconfiguration via O1) or by creating RAN management policies (such as A1 policies).

[0078] Figure 4 An example implementation of a System-on-Chip (SOC) 400 according to certain aspects of this disclosure is illustrated, which may include a Central Processing Unit (CPU) 402 or a multi-core CPU configured to generate gradients for training a neural network. The SOC 400 may be included in a base station 110 or a UE 120. Variables (e.g., neural signals and synaptic weights), system parameters associated with the computing device (e.g., a weighted neural network), latency, frequency window (bin) information, and task information may be stored in a memory block associated with a Neural Processing Unit (NPU) 408, a memory block associated with the CPU 402, a memory block associated with a Graphics Processing Unit (GPU) 404, a memory block associated with a Digital Signal Processor (DSP) 406, a memory block 418, or may be distributed across multiple blocks. Instructions executed at the CPU 402 may be loaded from the program memory associated with the CPU 402 or may be loaded from memory block 418.

[0079] The SOC 400 may also include additional processing blocks tailored for specific functions, such as a GPU 404, a DSP 406, a connectivity block 410 (which may include fifth-generation (5G) connectivity, fourth-generation LTE (4G LTE) connectivity, Wi-Fi connectivity, USB connectivity, Bluetooth connectivity, etc.), and a multimedia processor 412 capable of, for example, detecting and recognizing gestures. In one implementation, the NPU is implemented within a CPU, DSP, and / or GPU. The SOC 400 may also include a sensor processor 414, an image signal processor (ISP) 416, and / or a navigation module 420, which may include a global positioning system.

[0080] The SOC 400 may be based on the ARM instruction set. In this disclosure, the instructions loaded into the general-purpose processor 402 may include code for the UE to operate a first ML model among a plurality of first machine learning (ML) models. The first ML model is associated with wireless communication. The general-purpose processor 402 may also include code for the UE to monitor the first ML model using a second ML model. The second ML model is configured to monitor the first ML model based on one or more distance measures between at least a first dataset used to train the first ML model and one or more inference data instances observed during the operation of the first ML model.

[0081] In various aspects of this disclosure, instructions loaded into the general-purpose processor 402 may include code for receiving distance metric information from a user equipment (UE) associated with a first machine learning (ML) model related to wireless communication. The general-purpose processor 402 may also include code for monitoring the first ML model using a second ML model based on the distance metric information. The general-purpose processor 402 may also include code for determining, based on the distance metric information, whether to select a different first ML model from a plurality of first ML models for UE operation, fine-tune the first ML model, or revert to a non-machine learning model.

[0082] Deep learning architectures perform object recognition tasks by learning to represent inputs at progressively higher levels of abstraction in each layer, thereby constructing useful feature representations of the input data. In this way, deep learning addresses a major bottleneck of traditional machine learning. Before deep learning, machine learning methods for object recognition problems often relied heavily on human-designed features, possibly in conjunction with shallow classifiers. Shallow classifiers could be two-class linear classifiers, where a weighted sum of feature vector components is compared to a threshold to predict which class the input belongs to. Human-designed features could be templates or kernels customized for a specific problem domain by engineers with domain expertise. In contrast, while deep learning architectures can learn to represent features similar to those that human engineers might design, this requires training. Furthermore, deep networks can learn to represent and recognize novel types of features that humans might not have considered.

[0083] Deep learning architectures can learn hierarchical structures of features. For example, if presented with visual data, the first layer can learn to recognize relatively simple features in the input stream, such as edges. In another example, if presented with auditory data, the first layer can learn to recognize spectral power at specific frequencies. The second layer, taking the output of the first layer as input, can learn to recognize combinations of features, such as simple shapes in visual data or combinations of sounds in auditory data. For example, higher layers can learn to represent complex shapes in visual data or words in auditory data. Even higher layers can learn to recognize common visual objects or spoken phrases.

[0084] Deep learning architectures perform particularly well when applied to problems with a natural hierarchical structure. For example, the classification of motorized vehicles can benefit from first learning to identify features such as wheels, windshields, and others. These features can then be combined in different ways at higher levels to identify cars, trucks, and airplanes.

[0085] Neural networks can be designed to have multiple connectivity patterns. In feedforward networks, information is passed from lower layers to higher layers, where each neuron in a given layer communicates with neurons in higher layers. As described above, hierarchical representations can be built in successive layers of a feedforward network. Neural networks can also have recurrent or feedback (also known as top-down) connections. In recurrent connections, the output from a neuron in a given layer can be passed to another neuron in the same layer. Recurrent architectures can help identify patterns across more than one block of input data that is sequentially delivered to the neural network. Connections from neurons in a given layer to neurons in lower layers are called feedback (or top-down) connections. Networks with many feedback connections can be helpful when the recognition of higher-level concepts can aid in discerning specific lower-level features of the input.

[0086] The connections between layers in a neural network can be fully connected or locally connected. Figure 5A An example of a fully connected neural network 502 is illustrated. In the fully connected neural network 502, neurons in the first layer can pass their outputs to each neuron in the second layer, so that each neuron in the second layer will receive inputs from each neuron in the first layer. Figure 5B An example of a locally connected neural network 504 is illustrated. In the locally connected neural network 504, neurons in the first layer can connect to a finite number of neurons in the second layer. More generally, the locally connected layers of the locally connected neural network 504 can be configured such that each neuron in the layer will have the same or similar connectivity pattern, but the connection strength can have different values ​​(e.g., 510, 512, 514, and 516). The connectivity pattern of locally connected layers can produce spatially different receptive fields in higher layers because higher-layer neurons in a given region can receive inputs that are tuned to the characteristics of a restricted portion of the total input to the network through training.

[0087] An example of a locally connected neural network is a convolutional neural network. Figure 5C An example of a convolutional neural network 506 is illustrated. A convolutional neural network 506 can be configured such that the connection strength associated with the input for each neuron in the second layer is shared (e.g., 508). Convolutional neural networks may be well-suited for problems where the spatial location of the input is meaningful.

[0088] One type of convolutional neural network is the deep convolutional network (DCN). Figure 5DA detailed example of a DCN 500 designed to recognize visual features from an image 526 input by an image capture device 530 (such as an in-vehicle camera) is illustrated. The DCN 500 in this example can be trained to identify traffic signs and the numbers provided on them. Of course, the DCN 500 can be trained for other tasks, such as identifying lane markings or traffic lights.

[0089] Supervised learning can be used to train the DCN 500. During training, an image (such as image 526 of a speed limit sign) can be presented to the DCN 500, and the forward pass can then be computed to produce output 522. The DCN 500 may include a feature extraction part and a classification part. Upon receiving image 526, convolutional layer 532 may apply a convolutional kernel (not shown) to image 526 to generate a first set 518 of feature maps. As an example, the convolutional kernel used for convolutional layer 532 may be a 5x5 kernel that generates a 28x28 feature map. In this example, because four different feature maps are generated in the first set of feature maps 518, four different convolutional kernels are applied to image 526 at convolutional layer 532. Convolutional kernels may also be referred to as filters or convolutional filters.

[0090] The first set of feature maps 518 can be subsampled by a max-pooling layer (not shown) to generate the second set of feature maps 520. The max-pooling layer reduces the size of the first set of feature maps 518. That is, the size of the second set of feature maps 520 (e.g., 14×14) is smaller than the size of the first set of feature maps 518 (e.g., 28×28). The reduced size provides similar information to subsequent layers while reducing memory consumption. The second set of feature maps 520 can be further convolved via one or more subsequent convolutional layers (not shown) to generate one or more subsequent sets of feature maps (not shown).

[0091] exist Figure 5D In the example, the second set of feature maps 520 is convolved to generate a first feature vector 524. Furthermore, the first feature vector 524 is further convolved to generate a second feature vector 528. Each feature of the second feature vector 528 may include a number corresponding to a possible feature of image 526, such as "sign", "60", and "100". A softmax function (not shown) can convert the numbers in the second feature vector 528 into probabilities. Thus, the output 522 of DCN 500 can be the probability that image 526 includes one or more features.

[0092] In this example, the probabilities for "sign" and "60" in output 522 are higher than the probabilities for other numbers in output 522 (such as "30", "40", "50", "70", "80", "90", and "100"). Before training, output 522 generated by DCN 500 may be incorrect. Therefore, the error between output 522 and the target output can be calculated. The target output is the ground truth of image 526 (e.g., "sign" and "60"). The weights of DCN 500 can then be adjusted so that output 522 of DCN 500 is more closely aligned with the target output.

[0093] To adjust the weights, the learning process computes the gradient vector of the weights. The gradient indicates by how much the error will increase or decrease as the weights are adjusted. At the top layers, the gradient corresponds directly to the values ​​of the weights connecting the activated neurons in the penultimate layer to the neurons in the output layer. In lower layers, the gradient depends on the values ​​of the weights and the error gradient computed in the higher layers. The weights can then be adjusted to reduce the error. This method of adjusting weights is called "backpropagation" because it involves "passing backward" through the neural network.

[0094] In practice, the error gradient of the weights can be calculated using a small number of examples to make the calculated gradient approximate the true error gradient. This approximation method is called stochastic gradient descent. Stochastic gradient descent can be repeated until the achievable error rate of the entire system stops decreasing or until the error rate reaches a target level. After learning, a new image (e.g., a speed limit sign in image 526) can be presented to the DCN 500, and output 522 can be generated through the forward pass of the DCN 500. This output can be considered as the inference or prediction of the DCN 500.

[0095] Deep Belief Networks (DBNs) are probabilistic models that include multiple layers of hidden nodes. DBNs can be used to extract hierarchical representations of training datasets. DBNs are obtained by stacking layers of Restricted Boltzmann Machines (RBMs). An RBM is a type of artificial neural network that learns a probability distribution from a set of inputs. Because RBMs can learn a probability distribution without information about the class each input should be classified into, they are often used for unsupervised learning. Using a hybrid paradigm of supervised and unsupervised learning, the bottom RBM of a DBN can be trained unsupervised and used as a feature extractor, while the top RBM can be trained supervisedly (on the joint distribution of inputs from the previous layer and the target class) and used as a classifier.

[0096] DCN is a network of convolutional networks configured with additional pooling and normalization layers. DCN has achieved state-of-the-art performance on many tasks. DCN can be trained using supervised learning, where both the input and output targets are known for many paradigms and are used to modify the network's weights using gradient descent.

[0097] DCNs can be feedforward networks. Furthermore, as described above, connections from neurons in the first layer of a DCN to a set of neurons in the next higher layer are shared across neurons in the first layer. The feedforward and shared connections of a DCN can be used for fast processing. For example, the computational cost of a DCN may be much smaller than that of a similarly sized neural network that includes recurrent or feedback connections.

[0098] The processing at each layer of a convolutional network can be thought of as a spatially invariant template or base projection. If the input is first decomposed into multiple channels, such as the red, green, and blue channels of a color image, then a convolutional network trained on that input can be thought of as three-dimensional, with two spatial dimensions along the axes of the image and a third dimension capturing color information. The output of the convolutional connections can be thought of as forming a feature map in the next layer, where each element in the feature map (e.g., 220) receives input from a range of neurons in the previous layer (e.g., feature map 218) and from each of the multiple channels. The values ​​in the feature map can be further processed with nonlinearities (such as correction, max(0,x)). Values ​​from neighboring neurons can be further pooled, which corresponds to downsampling and provides additional local invariance and dimensionality reduction. Normalization corresponding to whitening can also be applied through lateral inhibition between neurons in the feature map.

[0099] Figure 6 This is a block diagram illustrating the DCN 650. The DCN 650 can include multiple different types of layers based on connectivity and weight sharing. Figure 6 As shown, DCN 650 includes convolutional blocks 654A and 654B. Each convolutional block in convolutional blocks 654A and 654B can be configured with a convolutional layer (CONV) 656, a normalization layer (LNorm) 658, and a max pooling layer (MAX POOL) 660.

[0100] Although only two convolutional blocks 654A and 654B are shown, this disclosure is not limited thereto, and any number of convolutional blocks 654A and 654B may be included in the DCN 650 according to design preferences.

[0101] Convolutional layer 656 may include one or more convolutional filters that can be applied to the input data to generate feature maps. Normalization layer 658 can normalize the output of the convolutional filters. For example, normalization layer 658 can provide whitening or lateral suppression. Max pooling layer 660 can provide spatial downsampling aggregation to achieve local invariance and dimensionality reduction.

[0102] Parallel filter banks of deep convolutional networks can be loaded onto an SOC 400 (e.g., Figure 4 The CPU 402 or GPU 404 of the SOC 400 can be used to achieve high performance and low power consumption. In an alternative implementation, a parallel filter bank can be loaded onto the DSP 406 or ISP 416 of the SOC 400. Additionally, the DCN 650 can access other processing blocks that may exist on the SOC 400, such as the sensor processor 414 and navigation module 420, which are dedicated to sensors and navigation, respectively.

[0103] The DCN 650 may also include one or more fully connected layers 662 (FC1 and FC2). The DCN 650 may also include logistic regression (LR) layers 664. Weights (not shown) to be updated are present between each of the layers 656, 658, 660, 662, and 664 of the DCN 650. The output of each layer (e.g., 656, 658, 660, 662, and 664) can be used as input to the next layer in the DCN 650 (e.g., 656, 658, 660, 662, and 664) to learn hierarchical feature representations from the input data 652 (e.g., images, audio, video, sensor data, and / or other input data) supplied at the first convolutional block in convolutional block 654A. The output of the DCN 650 is a classification score 666 of the input data 652. The classification score 666 may be a set of probabilities, where each probability is a probability of the input data, including features from a feature set.

[0104] As described, aspects of this disclosure relate to ML-based monitoring processes for monitoring ML models. That is, ML models can be configured to be monitored to detect and / or resolve concept drift. For example, ML-based monitoring processes can rely on K-nearest neighbor (KNN) information (e.g., KNN distance, KNN majority vote, K-mean distance, etc.) to monitor the ML model. In some aspects, a single instance of inference data can be used for monitoring.

[0105] Figure 7 Figure 700 illustrates examples of KNN-based ML-based monitoring according to various aspects of this disclosure. Reference Figure 7The diagram illustrates training data for a first dataset and training data for a second dataset. The first and second datasets may correspond to corresponding trained ML models for spatial beam estimation of a user equipment (UE). The UE (e.g., UE120) may, for example, choose an ML model trained on the first dataset or an ML model trained on the second dataset.

[0106] The second ML model can monitor the ML model used for beam estimation operated by the UE. For example, the second ML model can monitor the selected ML model based on the proximity between the new inference data instance 702 and the training data (e.g., dataset 1 and dataset 2). Thus, for example, the second ML model can use proximity information to monitor the suitability of the ML model, select a different ML model from a set of ML models, fall back to non-ML operations, or determine whether to fine-tune the ML model.

[0107] exist Figure 7 In the example, KNN distance can be used for spatial beam prediction. In this example, two fixed beams (e.g., two reference signal received power (RSRP) measurements) can be present as inputs to the ML model for model monitoring. However, this is merely for illustrative purposes and is not limited to this. Instead, any number of inputs can be used.

[0108] Therefore, the KNN distance of the new inference data instance 702 can be calculated. When the UE (e.g., UE 120) receives the new inference data instance 702, the UE (e.g., UE 120) can measure the KNN distance (or KNN majority vote, K-means distance, or other proximity metric) between the new inference data instance 702 and the two datasets (e.g., dataset 1 and dataset 2) used to train the two ML models. The UE (e.g., UE 120) or the network (e.g., base station 110) can select the ML model with the lowest KNN distance.

[0109] To compute the KNN distance for a single inference data instance (e.g., 702), the K nearest neighbor instances in each dataset can be identified. The KNN distance can then be computed as follows: KNN distance of the dataset (1) in Indicates a distance measurement. Represents the dataset The index of the nearest neighbor of the inference instance in the inference.

[0110] exist Figure 7In this context, the UE (e.g., UE 120) can select ML model 1 (e.g., trained with dataset 1) because the KNN distance between the new inference data instance 702 and the data instances in dataset 1 is smaller than the KNN distance between the new inference data instance 702 and the data instances in dataset 2. By utilizing KNN information for ML model monitoring, the availability of previous recent inference data may not be required to generate the inference data distribution.

[0111] In various ways, the network can configure the UE (e.g., UE 120) to report, for example, new inference data instances (e.g., 702) in a specified training dataset, K-means distance, and KNN information such as KNN distance or KNN majority vote among the K nearest neighbors. Doing so allows the network to monitor the ML model operating at the UE.

[0112] In some aspects, a hybrid ML model monitoring approach can be employed. For example, the UE (e.g., UE 120) can perform ML model monitoring, and the network can determine whether to select a different ML model from a set of ML models, revert to non-ML operation, or fine-tune the current ML model. In other examples, the UE can operate an ML model for spatial beam prediction. The network can operate a second ML model to monitor the ML model operated by the UE.

[0113] The network can receive proximity information from the UE. In some examples, the network (e.g., via a network device such as base station 110) can configure the UE (e.g., UE 120) to report statistical characteristics of KNN distance or KNN majority voting over multiple inference data instances. The network can then determine whether to select a different ML model from a set of ML models, revert to a non-ML operation, or fine-tune the current ML model. The network can send messages to the UE to configure the UE based on the determination.

[0114] In some aspects, a second ML model can monitor the ML model operated by the UE and / or determine modifications (e.g., selecting a different ML model, reverting to a non-ML model, or fine-tuning the ML model) based on distance metrics (e.g., KNN distance) determined based on one or more inference data instances. However, depending on the environment, changes in KNN distances across multiple inference data instances may not be significant. Therefore, it may be beneficial to report statistical characteristics of KNN information to reduce reporting overhead. For example, in some aspects, the network can configure the UE (e.g., UE 120) to report the mean, variance (e.g., the 5th percentile or 95th percentile), or other statistical characteristics of KNN information across multiple inference data instances.

[0115] Furthermore, the network can be configured to report the periodicity of KNN information statistics from previous data instances. For example, the periodicity can be specified in milliseconds (ms) based on the number of time slots or, for example, the time duration used in calculating the KNN information statistics. The network can also configure uplink resources for the UE (e.g., UE 120) to report the calculated KNN distance based on the configured periodicity.

[0116] The network can also configure KNN information parameters to be used by the UE (e.g., UE 120) to monitor ML models. For example, the network can configure the value of K to be used by the UE (e.g., UE 120) when calculating KNN distance / majority vote. The value of K can represent a trade-off between the complexity of KNN distance calculation (e.g., increased computation) and the accuracy of ML model selection. A larger K value indicates higher complexity in the monitoring process. A smaller K value indicates more error in selecting the best model.

[0117] In various ways, the network can also be configured with one or more thresholds regarding the KNN distance. By doing so, the network can enable the UE (e.g., UE 120) to switch between different ML models or fall back to non-ML operations based on the thresholds. For example, if the KNN distance of the dataset for the currently active ML model exceeds a predefined threshold, the UE (e.g., UE 120) can switch ML models. Otherwise, the UE (e.g., UE 120) can continue using the same ML model.

[0118] In some respects, the network can be configured to reduce the complexity of KNN information computation. One challenge in using KNN information to monitor ML models is that the ML computes the KNN distance from a new inference data instance to every other data instance in the training dataset, sorts the distances, and averages the K nearest distances. Therefore, the monitoring complexity can be high. Thus, the network can configure the UE (e.g., UE 120) to measure KNN distances along specific dimensions. For example, the input to the ML model may include two input features. For the spatial beam prediction use case, the UE (e.g., UE 120) can measure, for example, five beams. The spatial beams may have been measured at four different points in history and fed as input to the ML model.

[0119] In this scenario, the ML monitoring process calculates the 20-dimensional KNN distance between inference data instances and the training datasets of different ML models. To reduce the complexity of KNN distance calculation, the network can configure the UE (e.g., UE 120) to calculate the KNN distance based on a specific dimension / feature and exclude other dimensions. For example, the KNN distance can be calculated based on five input measurements instead of twenty. Using the network's understanding of the environment and operating conditions, the network can identify the more important features or inputs for KNN distance calculation monitoring, and which features can be excluded from ML monitoring to reduce complexity.

[0120] In some aspects, the network can configure the UE (e.g., UE 120) to dilute the training samples used for KNN information computation. Another challenge in KNN information computation may be that the UE (e.g., UE 120) retains all training data samples, which increases storage overhead and the complexity of computing the KNN distance. To address this challenge, the network can configure the UE (e.g., UE 120) to retain a diluted version of the training samples for use in monitoring ML models using KNN information.

[0121] Figure 8 Figure 800 illustrates an example of an ML model that monitors complexity reduction by diluting training samples according to various aspects of this disclosure. (See Figure 800 for reference.) Figure 8 Figure 800 includes a graph 700 illustrating an example of model monitoring using KNN distance. To reduce the complexity of ML monitoring, such as by using KNN distance instead of considering the entire training dataset (e.g., dataset 1 or dataset 2), a reduced set of training data can be used to compute the KNN distance. Figure 8 As shown, subsets of measurements from dataset 1 and dataset 2 are included in Figure 802. The subsets of measurements from dataset 1 (e.g., diluted dataset 1) and dataset 2 (e.g., diluted dataset 2) can then be used to compute the KNN distance relative to the new inference data instance 702. In some respects, diluted dataset 1 and diluted dataset 2 can be retained in memory (e.g., Figure 4 The memory (418) is used for subsequent ML monitoring tasks.

[0122] Reducing the complexity of KNN information computation may come at the cost of decreased accuracy in ML monitoring, potentially prompting the network to configure dilution operations based on its understanding of the environment. The network can configure various levels of detail within the dilution operation, such as the dilution factor (e.g., by 1). 2 or 1 4 or 1 (e.g., 8 training samples, etc.) or the number of training samples to be retained for monitoring. Dilution may include, for example, random uniform dilution on a specific dimension, and may be applied, for example, before or after the transformation. The network may also be configured that the UE (e.g., UE 120) may retain diluted training samples (e.g., diluted dataset 1 or diluted dataset 2) for computing KNN information.

[0123] In some aspects, the network can further configure the UE (e.g., UE 120) to cluster the training samples used for KNN information computation. The network can configure the UE (e.g., UE 120) to retain a clustered version of the training samples for use in monitoring ML models using KNN information. In various aspects, the network can configure various levels of detail in the dilution operation. For example, the network can configure multiple clusters (e.g., 1, 2, 4, or 10) in each dataset. Clustering schemes can include, for example, centroid-based clustering, density-based clustering, distribution-based clustering, etc.

[0124] Figure 9 Figure 900 illustrates an example of an ML model that monitors complexity reduction by clustering training samples according to various aspects of this disclosure. (See Figure 900 for reference.) Figure 9 Figure 900 includes a graph 700 illustrating examples of model monitoring using KNN distance. Dataset 1 and Dataset 2 can be represented as one or more clusters, rather than considering the complete training dataset (e.g., Dataset 1 or Dataset 2 in Figure 700). For example, as shown in Figure 902, a measurement of Dataset 1 can be represented as centroid 904, and a measurement of Dataset 2 can be represented as centroid 906. Distance metrics (e.g., KNN distance) can then be computed relative to a new inference data instance 702 using centroids 904 and 906. In some aspects, clusters (e.g., 904, 906) can be retained in memory (e.g., ...). Figure 4 The memory (418) is used for subsequent ML monitoring tasks.

[0125] The UE (e.g., UE 120) can also be configured to use KNN information (e.g., KNN distance, KNN majority vote, or K-means distance) to report its ability to monitor ML models. The UE (e.g., UE 120) can report the ability / recommendation used for calculating the K value in the KNN information. For example, the UE (e.g., UE 120) can use dimensionality reduction, training data dilution, or training data clustering to report the ability / recommendation to reduce the complexity of monitoring.

[0126] A UE (e.g., UE 120) can report recommendations regarding thresholds for KNN information used to switch ML models. In some aspects, UE capabilities can be reported using Radio Resource Control (RRC) signaling. In others, UE recommendations can be reported using RRC signaling, Media Access Control-Control Element (MAC-CE), or Uplink Control Information (UCI). On the other hand, network configuration can be static via RRC signaling, semi-static via MAC-CE, or dynamically activated via Downlink Control Information (DCI).

[0127] In some respects, the UE (e.g., UE 120) can use different ML models for ML model monitoring. For example, the UE (e.g., UE 120) can use support vector machines (SVM), random forests, decision trees, logistic regression, neural networks, and / or K-means for ML monitoring.

[0128] As indicated above, Figures 4 to 9 This is provided as an example. Other examples are available relative to... Figures 4 to 9 The examples described are different.

[0129] Figure 10 This is a flowchart illustrating an example process 1000 of machine learning (ML) model monitoring performed by a UE according to various aspects of this disclosure. The UE (e.g., UE 120) may include one or more processors, such as CPU 402, GPU 404, and / or NPU 408, which can execute instructions associated with the ML monitoring module 140 to implement the example process 1000. For example, the UE (e.g., UE 120) may also include one or more memory units, such as memory 418.

[0130] At box 1002, the UE operates a first ML model out of a set of first machine learning (ML) models. The first ML model is associated with wireless communication. For example, the UE may select and operate one ML model from the set of ML models for spatial beam estimation or other wireless communication tasks.

[0131] At box 1004, the UE uses a second ML model to monitor the first ML model. The second ML model is configured to monitor the first ML model based on one or more distance metrics between at least a first dataset used to train the first ML model and one or more inference data instances. (See reference...) Figure 7 As described, the second ML model can monitor the ML model used for beam estimation operated by the UE. For example, the second ML model can monitor the ML model based on the proximity between the new inference data instance 702 and the training data (e.g., dataset 1 and dataset 2).

[0132] In some respects, at box 1006, the UE may optionally determine, based on one or more distance metrics, whether to select a different first ML model from a plurality of first ML models for operation, fine-tune the first ML model, or revert to a non-machine learning model for wireless communication. For example, as referenced Figure 7 As described, the second ML model can use proximity information (e.g., KNN distance) to monitor the suitability of the ML model, select different ML models from a set of ML models, fall back to non-ML operations, or determine whether to fine-tune the ML model.

[0133] Figure 11 This is a flowchart illustrating an example process 1100 of machine learning (ML) model monitoring performed by a network device according to various aspects of this disclosure. The network device (e.g., base station 110) may include one or more processors, such as CPU 402, GPU 404, and / or NPU 408, which can execute instructions associated with the ML monitoring module 138 to implement the example process 1100. For example, the network device (e.g., base station 110) may also include one or more memory units, such as memory 418.

[0134] At box 1102, the network device receives distance metric information from the user equipment (UE) associated with a first machine learning (ML) model of wireless communication. (As relative to...) Figure 7 As described, the network (e.g., via a network device such as base station 110) can receive proximity information from the UE. In some examples, the network can configure the UE (e.g., UE 120) to report statistical characteristics of KNN distance or KNN majority vote over multiple inference data instances.

[0135] At box 1104, the network device uses a second ML model to monitor the first ML model based on distance metric information. For example, the network device may compare the received distance metric with a predefined threshold. In some aspects, the network device may determine that the UE should use a different ML model in response to the distance metric exceeding the predefined threshold.

[0136] In some respects, multiple different thresholds can be used to trigger different modifications. For example, in response to a distance metric exceeding a first threshold, a second ML model can determine to fine-tune the first ML model. In response to a distance metric exceeding a second threshold, the second ML model can determine to revert to a non-ML model used for wireless communication tasks (e.g., spatial beam estimation).

[0137] At box 1106, the network device determines, based on distance metric information, whether to select a different first ML model from multiple first ML models for UE operation, fine-tune the first ML model, or revert to a non-machine learning model. The network device may determine the modification based on monitoring. For example, as referenced... Figure 7 As described, the network can determine, based on distance metrics, whether to select a different ML model from a set of ML models, revert to a non-ML operation, or fine-tune the current ML model. The network can then send messages to the UE to configure the UE based on this determination.

[0138] Specific implementation examples are provided in the following numbered clauses.

[0139] 1. A method for wireless communication performed by a user equipment (UE), the method comprising: The UE operates a first ML model out of a plurality of first machine learning (ML) models, the first ML model being associated with wireless communication; and The UE uses a second ML model to monitor the first ML model, the second ML model being configured to monitor the first ML model based on one or more distance metrics between at least a first dataset used to train the first ML model and one or more inference data instances observed during the operation of the first ML model.

[0140] 2. The method according to Clause 1, the method further comprising the UE determining, based on the one or more distance metrics, whether to select a different first ML model among the plurality of first ML models for operation, fine-tune the first ML model, or revert to a non-machine learning model for the wireless communication.

[0141] 3. The method according to Clause 1 or 2, wherein determining whether to select the different first ML models among the plurality of first ML models to operate, fine-tune the first ML models, or revert to the non-machine learning model is based on one or more configurable thresholds respectively.

[0142] 4. The method according to any one of Clauses 1 to 3, wherein the one or more distance measures include nearest neighbor distance, K nearest neighbor (KNN) distance or K-mean distance, where K is an integer parameter.

[0143] 5. The method according to any one of Clauses 1 to 4, the method further comprising receiving from the network a first message including one or more configuration parameters for the second ML model.

[0144] 6. The method according to any one of Clauses 1 to 5, wherein the one or more configuration parameters include the value of the K integer parameter or the one or more configurable thresholds.

[0145] 7. The method according to any one of clauses 1 to 6, the method further comprising retaining a subset of the first dataset used to train the first ML model as a diluted training dataset, the second ML model calculating the one or more distance metrics based on the diluted training dataset.

[0146] 8. The method according to any one of Clauses 1 to 7, the method further comprising sending a second message to the network indicating monitoring capabilities, the second message including a Radio Resource Control (RRC) message, a Media Access Control-Control Element (MAC-CE) message, or an Uplink Control Information (UCI).

[0147] 9. The method according to any one of clauses 1 to 8, the method further comprising performing a clustering process on the first dataset to reduce the complexity of computing the one or more distance metrics.

[0148] 10. A method for wireless communication performed by a network device, the method comprising: Receive distance measurement information associated with a first machine learning (ML) model related to wireless communication from the user equipment (UE); The first ML model is monitored using the second ML model based on the distance metric information; and Based on the distance metric information, it is determined whether to select different first ML models from multiple first ML models for the UE to operate, fine-tune the first ML model, or revert to a non-machine learning model.

[0149] 11. The method according to Clause 10, further comprising: Receive from the UE a first message indicating the UE's capability to monitor the first ML model associated with the wireless communication; and The UE sends a set of configuration parameters for the second ML model, the set of configuration parameters being used by the UE to configure the second ML model to monitor the first ML model based on one or more distance metrics between a first dataset used to train the first ML model and one or more inference data instances generated by the first ML model.

[0150] 12. The method according to Clause 10 or 11, wherein the one or more distance metrics include nearest neighbor distance, K-nearest neighbor (KNN) distance, or K-means distance, where K is an integer parameter.

[0151] 13. The method according to any one of Clauses 10 to 12, wherein the set of configuration parameters includes the value of the K integer parameter and one or more configurable thresholds.

[0152] 14. The method according to any one of clauses 10 to 13, wherein the set of configuration parameters includes a model complexity parameter that configures the UE to compute the distance metric information using a diluted set of the first dataset or one or more clusters of the first dataset.

[0153] 15. The method according to any one of clauses 10 to 14, the method further comprising sending a second message to the UE, the second message being based on the determination to configure the UE to select the different first ML models among the plurality of first ML models to operate, fine-tune the first ML models, or revert to the non-machine learning model for the wireless communication.

[0154] 16. An apparatus for performing wireless communication by a user equipment (UE), the apparatus comprising: At least one memory; and At least one processor, coupled to the at least one memory, is configured to: The UE operates a first ML model out of a plurality of first machine learning (ML) models, the first ML model being associated with the wireless communication; and The UE uses a second ML model to monitor the first ML model, the second ML model being configured to monitor the first ML model based on one or more distance metrics between at least a first dataset used to train the first ML model and one or more inference data instances observed during the operation of the first ML model.

[0155] 17. The apparatus according to Clause 16, wherein the at least one processor is further configured to be operated by the UE based on the one or more distance metrics, either by selecting a different first ML model among the plurality of first ML models, fine-tuning the first ML model, or reverting to a non-machine learning model for the wireless communication.

[0156] 18. The apparatus according to clause 16 or 17, wherein determining whether to select the different first ML models among the plurality of first ML models to operate, fine-tune the first ML models, or revert to the non-machine learning model is based on one or more configurable thresholds respectively.

[0157] 19. The apparatus according to any one of Clauses 16 to 18, wherein the one or more distance metrics include nearest neighbor distance, K-nearest neighbor (KNN) distance, or K-mean distance, where K is an integer parameter.

[0158] 20. The apparatus according to any one of clauses 16 to 19, wherein the at least one processor is further configured to receive from the network a first message including one or more configuration parameters for the second ML model.

[0159] 21. The apparatus according to any one of clauses 16 to 20, wherein the one or more configuration parameters include the value of the K integer parameter or the one or more configurable thresholds.

[0160] 22. The apparatus according to any one of claims 16 to 21, wherein the at least one processor is further configured to retain a subset of the first dataset used to train the first ML model as a diluted training dataset, the second ML model calculating the one or more distance metrics based on the diluted training dataset.

[0161] 23. The apparatus according to any one of Clauses 16 to 22, wherein the at least one processor is further configured to send a second message to the network indicating monitoring capabilities, the second message including a Radio Resource Control (RRC) message, a Media Access Control-Control Element (MAC-CE) message, or an Uplink Control Information (UCI).

[0162] 24. The apparatus according to any one of clauses 16 to 23, wherein the at least one processor is further configured to perform a clustering process on the first dataset to reduce the complexity of computing the one or more distance metrics.

[0163] 25. An apparatus for wireless communication performed by a network device, the apparatus comprising: At least one memory; and At least one processor, coupled to the at least one memory, is configured to: Receive distance measurement information associated with a first machine learning (ML) model related to wireless communication from the user equipment (UE); The first ML model is monitored using the second ML model based on the distance metric information; and Based on the distance metric information, it is determined whether to select different first ML models from multiple first ML models for the UE to operate, fine-tune the first ML model, or revert to a non-machine learning model.

[0164] 26. The apparatus according to clause 25, wherein the at least one processor is further configured to: Receive from the UE a first message indicating the UE's capability to monitor the first ML model associated with the wireless communication; and The UE sends a set of configuration parameters for the second ML model, the set of configuration parameters being used by the UE to configure the second ML model to monitor the first ML model based on one or more distance metrics between a first dataset used to train the first ML model and one or more inference data instances generated by the first ML model.

[0165] 27. The apparatus according to clause 25 or 26, wherein the one or more distance metrics include nearest neighbor distance, K-nearest neighbor (KNN) distance, or K-mean distance, where K is an integer parameter.

[0166] 28. The method according to any one of Clauses 25 to 27, wherein the set of configuration parameters includes the value of the K integer parameter and one or more configurable thresholds.

[0167] 29. The method according to any one of Clauses 25 to 28, wherein the set of configuration parameters includes a model complexity parameter that configures the UE to compute the distance metric information using a diluted set of the first dataset or one or more clusters of the first dataset.

[0168] 30. The apparatus according to any one of claims 25 to 29, wherein the at least one processor is further configured to send a second message to the UE, the second message being based on the determination to configure the UE to select one of the plurality of first ML models to operate, fine-tune the first ML model, or revert to the non-machine learning model for the wireless communication.

[0169] The foregoing disclosure provides examples and descriptions, but is not intended to be exhaustive or to limit the aspects to the precise forms disclosed. Modifications and variations may be made based on the foregoing disclosure, or from various practices.

[0170] As used, the term "component" is intended to be interpreted broadly as hardware, firmware, and / or a combination of hardware and software. As used, a processor is implemented using hardware, firmware, and / or a combination of hardware and software.

[0171] The threshold is used to describe certain aspects. As used, depending on the context, meeting the threshold can mean that the value is greater than the threshold, greater than or equal to the threshold, less than the threshold, less than or equal to the threshold, equal to the threshold, not equal to the threshold, etc.

[0172] It will be apparent that the described systems and / or methods can be implemented in various forms, including hardware, firmware, and / or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not limiting in any way. Therefore, since the operation and performance of these systems and / or methods are described without reference to specific software code, it should be understood that the software and hardware used to implement these systems and / or methods can be designed, at least in part, based on this description.

[0173] Although specific combinations of features are set forth in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of various aspects. In fact, many of these features can be combined in ways not specifically set forth in the claims and / or not disclosed in the specification. Although each dependent claim listed below may directly depend on only one claim, the disclosure of various aspects includes each dependent claim combined with every other claim in the claim set. The phrase “at least one of” in the list of items refers to any combination of those items, including single members. As an example, “at least one of a, b, or c” is intended to cover: a, b, c, ab, ac, bc, and abc, as well as any combination having multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0174] The elements, actions, or instructions used should not be interpreted as critical or necessary unless explicitly stated otherwise. Furthermore, as used, the articles “a” and “one” are intended to include one or more items and may be used interchangeably with “one or more.” Additionally, as used, the terms “set” and “group” are intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and may be used interchangeably with “one or more.” If only one item is desired, the phrase “only one” or similar terminology will be used. Furthermore, as used, the terms “have,” “possess,” “have,” etc., are intended to be open-ended terms. Additionally, the phrase “based on” is intended to mean “at least partially based on” unless otherwise explicitly stated.

Claims

1. A method for wireless communication performed by a user equipment (UE), the method comprising: The UE operates a first ML model out of a plurality of first machine learning (ML) models, the first ML model being associated with wireless communication; as well as The UE uses a second ML model to monitor the first ML model, the second ML model being configured to monitor the first ML model based on one or more distance metrics between at least a first dataset used to train the first ML model and one or more inference data instances observed during the operation of the first ML model.

2. The method of claim 1, further comprising the UE determining, based on the one or more distance metrics, whether to select a different first ML model from the plurality of first ML models for operation, fine-tune the first ML model, or revert to a non-machine learning model for the wireless communication.

3. The method of claim 2, wherein determining whether to select the different first ML models among the plurality of first ML models to operate, fine-tune the first ML models, or revert to the non-machine learning model is based on one or more configurable thresholds.

4. The method of claim 3, wherein the one or more distance metrics include nearest neighbor distance, K-nearest neighbor (KNN) distance, or K-means distance, where K is an integer parameter.

5. The method of claim 4, further comprising receiving from the network a first message including one or more configuration parameters for the second ML model.

6. The method of claim 5, wherein the one or more configuration parameters include the value of the K integer parameter or the one or more configurable thresholds.

7. The method of claim 1, further comprising reserving a subset of the first dataset used to train the first ML model as a diluted training dataset, wherein the second ML model computes the one or more distance metrics based on the diluted training dataset.

8. The method of claim 1, further comprising sending a second message to the network indicating monitoring capabilities, the second message including a Radio Resource Control (RRC) message, a Media Access Control-Control Element (MAC-CE) message, or an Uplink Control Information (UCI).

9. The method of claim 1, further comprising performing a clustering process on the first dataset to reduce the complexity of computing the one or more distance metrics.

10. A method for wireless communication performed by a network device, the method comprising: Receive distance measurement information associated with a first machine learning (ML) model related to wireless communication from the user equipment (UE); The first ML model is monitored using the second ML model based on the distance metric information; and Based on the distance metric information, it is determined whether to select different first ML models from multiple first ML models for the UE to operate, fine-tune the first ML model, or revert to a non-machine learning model.

11. The method according to claim 10, further comprising: The UE receives a first message indicating the UE's capability to monitor the first ML model associated with the wireless communication; as well as The UE sends a set of configuration parameters for the second ML model, the set of configuration parameters being used by the UE to configure the second ML model to monitor the first ML model based on one or more distance metrics between a first dataset used to train the first ML model and one or more inference data instances generated by the first ML model.

12. The method of claim 11, wherein the one or more distance metrics include nearest neighbor distance, K-nearest neighbor (KNN) distance, or K-means distance, where K is an integer parameter.

13. The method of claim 12, wherein the set of configuration parameters includes the value of the K integer parameter and one or more configurable thresholds.

14. The method of claim 11, wherein the set of configuration parameters includes a model complexity parameter that configures the UE to use a diluted set of the first dataset or one or more clusters of the first dataset to compute the distance metric information.

15. The method of claim 10, further comprising sending a second message to the UE, the second message being based on the determination to configure the UE to select one of the plurality of first ML models to operate, fine-tune the first ML model, or revert to the non-machine learning model for the wireless communication.

16. An apparatus for performing wireless communication by a user equipment (UE), the apparatus comprising: At least one memory; and At least one processor, coupled to the at least one memory, is configured to: The UE operates a first ML model out of a plurality of first machine learning (ML) models, the first ML model being associated with the wireless communication; as well as The UE uses a second ML model to monitor the first ML model, the second ML model being configured to monitor the first ML model based on one or more distance metrics between at least a first dataset used to train the first ML model and one or more inference data instances observed during the operation of the first ML model.

17. The apparatus of claim 16, wherein the at least one processor is further configured to be operated by the UE based on the one or more distance metrics, either by selecting a different first ML model among the plurality of first ML models, fine-tuning the first ML model, or reverting to a non-machine learning model for the wireless communication.

18. The apparatus of claim 17, wherein determining whether to select the different first ML models among the plurality of first ML models to operate, fine-tune the first ML models, or revert to the non-machine learning model is based on one or more configurable thresholds.

19. The apparatus of claim 18, wherein the one or more distance metrics include nearest neighbor distance, K-nearest neighbor (KNN) distance, or K-means distance, wherein K is an integer parameter.

20. The apparatus of claim 19, wherein the at least one processor is further configured to receive from the network a first message including one or more configuration parameters for the second ML model.

21. The apparatus of claim 20, wherein the one or more configuration parameters include the value of the K integer parameter or the one or more configurable thresholds.

22. The apparatus of claim 16, wherein the at least one processor is further configured to retain a subset of the first dataset used to train the first ML model as a diluted training dataset, the second ML model calculating the one or more distance metrics based on the diluted training dataset.

23. The apparatus of claim 16, wherein the at least one processor is further configured to send a second message indicating monitoring capability to the network, the second message including a Radio Resource Control (RRC) message, a Media Access Control-Control Element (MAC-CE) message, or an Uplink Control Information (UCI).

24. The apparatus of claim 16, wherein the at least one processor is further configured to perform a clustering process on the first dataset to reduce the complexity of computing the one or more distance metrics.

25. An apparatus for wireless communication performed by a network device, the apparatus comprising: At least one memory; and At least one processor, coupled to the at least one memory, is configured to: Receive distance measurement information associated with a first machine learning (ML) model related to wireless communication from the user equipment (UE); The first ML model is monitored using the second ML model based on the distance metric information; and Based on the distance metric information, it is determined whether to select different first ML models from multiple first ML models for the UE to operate, fine-tune the first ML model, or revert to a non-machine learning model.

26. The apparatus of claim 25, wherein the at least one processor is further configured to: Receive from the UE a first message indicating the UE's capability to monitor the first ML model associated with the wireless communication; and The UE sends a set of configuration parameters for the second ML model, the set of configuration parameters being used by the UE to configure the second ML model to monitor the first ML model based on one or more distance metrics between a first dataset used to train the first ML model and one or more inference data instances generated by the first ML model.

27. The apparatus of claim 26, wherein the one or more distance metrics include nearest neighbor distance, K-nearest neighbor (KNN) distance, or K-means distance, wherein K is an integer parameter.

28. The apparatus of claim 27, wherein the set of configuration parameters includes the value of the K integer parameter and one or more configurable thresholds.

29. The apparatus of claim 26, wherein the set of configuration parameters includes a model complexity parameter that configures the UE to compute the distance metric information using a diluted set of the first dataset or one or more clusters of the first dataset.

30. The apparatus of claim 25, wherein the at least one processor is further configured to send a second message to the UE, the second message being based on the determination to configure the UE to select one of the plurality of first ML models to operate, fine-tune the first ML model, or revert to the non-machine learning model for the wireless communication.

Images