AI / ML model quality monitoring and rapid recovery under model fault detection

Abstract

AI / ML model quality monitoring and rapid recovery under model fault detection are disclosed. A UE may receive a training quality query message from a network device, wherein the training quality query message includes query information about one or more training quality metrics of one or more AI models used to train the UE; and send a training quality response message to the network device, wherein the training quality response message includes response information about the one or more training quality metrics of the one or more AI models used to train the UE.

Description

Technical Field

[0001] The present application as a whole relates to wireless communication systems, including artificial intelligence (AI) / machine learning (ML) model quality monitoring and rapid recovery under model fault detection. Background Art

[0002] Wireless mobile communication technologies use various standards and protocols to send data between base stations and wireless communication devices. Wireless communication system standards and protocols may include, for example, the 3rd Generation Partnership Project (3GPP) Long Term Evolution (LTE) (e.g., 4G), 3GPP New Radio (NR) (e.g., 5G), and the IEEE 802.11 standard for wireless local area networks (WLANs) (commonly referred to within industry organizations as WLANs). ).

[0003] As envisioned by 3GPP, different wireless communication system standards and protocols may use various radio access networks (RANs) to communicate between base stations of the RAN (which may also sometimes be referred to as RAN nodes, network nodes, or simply nodes) and wireless communication devices referred to as user equipment (UE). 3GPP RANs may include, for example, Global System for Mobile Communications (GSM), Enhanced Data Rates for GSM Evolution (EDGE) RAN (GERAN), Universal Terrestrial Radio Access Network (UTRAN), Evolved Universal Terrestrial Radio Access Network (E-UTRAN), and / or Next Generation Radio Access Network (NG-RAN).

[0004] Each RAN may use one or more radio access technologies (RATs) to perform communications between base stations and UEs. For example, GERAN implements GSM and / or EDGE RAT, UTRAN implements Universal Mobile Telecommunications System (UMTS) RAT or other 3GPP RAT, E-UTRAN implements LTE RAT (sometimes referred to as LTE), and NG-RAN implements NR RAT (sometimes referred to herein as 5G RAT, 5G NR RAT, or NR). In some deployments, E-UTRAN may also implement NR RAT. In some deployments, NG-RAN may also implement LTE RAT.

[0005] The base stations used by the RAN may correspond to the RAN. An example of an E-UTRAN base station is an Evolved Universal Terrestrial Radio Access Network (E-UTRAN) Node B (also commonly denoted as an evolved Node B, enhanced Node B, eNodeB, or eNB). An example of an NG-RAN base station is a Next Generation Node B (also sometimes referred to as a gNodeB or gNB).

[0006] The RAN provides communication services together with external entities through its connection with the Core Network (CN). For example, E-UTRAN may utilize the Evolved Packet Core (EPC) and NG-RAN may utilize the 5G Core Network (5GC).

[0007] The frequency bands of 5G NR may be divided into two or more different frequency ranges. For example, frequency range 1 (FR1) may include frequency bands operating at frequencies below 6 GHz, some of which may be used by previous standards and may potentially be expanded to cover new spectrum products from 410 MHz to 7125 MHz. Frequency range 2 (FR2) may include frequency bands from 24.25 GHz to 52.6 GHz. The frequency bands in the millimeter wave (mmWave) range of FR2 may have a smaller range but potentially higher available bandwidth than the frequency bands in FR1. The skilled person will recognize that these frequency ranges, provided by way of example, may vary over time or from region to region. Summary of the invention

[0008] The present disclosure provides an apparatus, system, and method for artificial intelligence (AI) / machine learning (ML) model quality monitoring and rapid recovery under model fault detection.

[0009] According to some embodiments, a user equipment (UE) is disclosed, which includes: at least one antenna; at least one radio component, the at least one radio component coupled to the at least one antenna; and a processor, the processor coupled to the at least one radio component; the processor is configured to: receive a training quality query message from a network device, wherein the training quality query message includes query information about one or more training quality metrics of one or more AI models used to train the UE; and send a training quality response message to the network device, wherein the training quality response message includes response information about the one or more training quality metrics of the one or more AI models used to train the UE.

[0010] According to some embodiments, a network device is disclosed, comprising: at least one antenna; at least one radio component coupled to the at least one antenna; and a processor coupled to the at least one radio component; the processor being configured to: send a training quality query message to a UE, wherein the training quality query message includes query information about one or more training quality metrics of one or more AI models used to train the UE; and receive a training quality response message from the UE, wherein the training quality response message includes response information about the one or more training quality metrics of the one or more AI models used to train the UE.

[0011] According to some embodiments, a user equipment (UE) is disclosed, comprising: at least one antenna; at least one radio component coupled to the at least one antenna; and a processor coupled to the at least one radio component; the processor being configured to: detect a failure state of a first AI model of the UE; and switch to a second AI model different from the first AI model.

[0012] According to some embodiments, a network device is disclosed, comprising: at least one antenna; at least one radio component coupled to the at least one antenna; and a processor coupled to the at least one radio component; the processor being configured to: receive a failure status of a first AI model of the UE from a UE; and transmit a switching indication to the UE to switch to a second AI model different from the first AI model.

[0013] According to some embodiments, a non-transitory computer-readable storage medium is disclosed, which stores instructions that, when executed by a processor of a device, cause the processor to perform the methods or steps described herein.

[0014] According to some embodiments, a computer program product is disclosed, which stores a computer program that, when executed by a processor of a device, causes the processor to perform the methods or steps described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] To easily identify the discussion of any particular element or action, the most significant digit(s) in a reference number refers to the drawing number that first introduces the element.

[0016] Figure 1 An example architecture of a wireless communication system according to embodiments disclosed herein is illustrated.

[0017] Figure 2 An example system for performing signaling between a wireless device and a network device according to embodiments disclosed herein is illustrated.

[0018] Figure 3 An example functional framework of AI / ML in a wireless communication system according to the embodiments disclosed herein is illustrated.

[0019] Figure 4 An example process for performing training quality negotiation between a UE and a network device according to embodiments disclosed herein is illustrated.

[0020] Figure 5An example communication process for performing training quality negotiation between a UE and a network device according to embodiments disclosed herein is illustrated.

[0021] Figure 6 An example process for performing fast recovery under model fault detection according to embodiments disclosed herein is illustrated.

[0022] Figure 7 An example communication process for performing fast recovery under model fault detection according to embodiments disclosed herein is illustrated.

[0023] Figure 8 Another example communication process for performing fast recovery under model fault detection according to embodiments disclosed herein is illustrated. DETAILED DESCRIPTION

[0024] Various embodiments are described with respect to UE. However, reference to UE is provided for illustrative purposes only. The example embodiments may be used with any electronic component that can establish a connection with a network and is configured with hardware, software, and / or firmware for exchanging information and data with the network. Therefore, UE as described herein is used to represent any suitable electronic component.

[0025] Figure 1 An example architecture of a wireless communication system 100 according to embodiments disclosed herein is illustrated. The description provided below is for an example wireless communication system 100 operating in conjunction with the LTE system standard and / or the 5G or NR system standard provided in the 3GPP technical specifications.

[0026] like Figure 1 As shown, wireless communication system 100 includes UE 102 and UE 104 (although any number of UEs may be used). In this example, UE 102 and UE 104 are illustrated as smartphones (e.g., handheld touch screen mobile computing devices capable of connecting to one or more cellular networks), but may also include any mobile or non-mobile computing device configured for wireless communication.

[0027] UE 102 and UE 104 may be configured to be communicatively coupled to RAN 106. In an embodiment, RAN 106 may be NG-RAN, E-UTRAN, etc. UE 102 and UE 104 utilize connections (or channels) (shown as connection 108 and connection 110, respectively) with RAN 106, where each connection (or channel) includes a physical communication interface. RAN 106 may include one or more base stations, such as base station 112 and base station 114, to implement connection 108 and connection 110.

[0028] In this example, connection 108 and connection 110 are air interfaces that enable such communicative coupling and may conform to the RAT used by RAN 106 , such as, for example, LTE and / or NR.

[0029] In some embodiments, UE 102 and UE 104 may also directly exchange communication data via side link interface 116. UE 104 is shown as being configured to access an access point (shown as AP 118) via connection 120. By way of example, connection 120 may include a local wireless connection, such as a connection conforming to any IEEE 802.11 protocol, wherein AP 118 may include In this example, AP 118 may be connected to another network (eg, the Internet) without going through CN 124.

[0030] In an embodiment, UE 102 and UE 104 may be configured to communicate with each other or with base station 112 and / or base station 114 over a multi-carrier communication channel using orthogonal frequency division multiplexing (OFDM) communication signals according to various communication techniques, such as but not limited to orthogonal frequency division multiple access (OFDMA) communication techniques (e.g., for downlink communication) or single carrier frequency division multiple access (SC-FDMA) communication techniques (e.g., for uplink and ProSe or sidelink communication), although the scope of the embodiment is not limited in this respect. The OFDM signal may include multiple orthogonal subcarriers.

[0031] In some embodiments, all or part of base station 112 or base station 114 may be implemented as one or more software entities running on a server computer as part of a virtual network. In addition, or in other embodiments, base station 112 or base station 114 may be configured to communicate with each other via interface 122. In an embodiment where wireless communication system 100 is an LTE system (e.g., when CN 124 is EPC), interface 122 may be an X2 interface. The X2 interface may be defined between two or more base stations (e.g., two or more eNBs, etc.) connected to EPC and / or between two eNBs connected to EPC. In an embodiment where wireless communication system 100 is an NR system (e.g., when CN 124 is 5GC), interface 122 may be an Xn interface. The Xn interface is defined between two or more base stations (e.g., two or more gNBs, etc.) connected to 5GC, between base station 112 (e.g., gNB) and eNB connected to 5GC, and / or between two eNBs connected to 5GC (e.g., CN 124).

[0032] The RAN 106 is shown as being communicatively coupled to the CN 124. The CN 124 may include one or more network elements 126 configured to provide various data and telecommunication services to customers / subscribers (e.g., UE 102 and users of UE 104) connected to the CN 124 via the RAN 106. The components of the CN 124 may be implemented in one physical device or separate physical devices including components for reading and executing instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine-readable storage medium).

[0033] In an embodiment, CN 124 may be an EPC, and RAN 106 may be connected to CN 124 via an S1 interface 128. In an embodiment, S1 interface 128 may be divided into two parts: an S1 user plane (S1-U) interface that carries service data between base station 112 or base station 114 and a serving gateway (S-GW); and an S1-MME interface that is a signaling interface between base station 112 or base station 114 and a mobility management entity (MME).

[0034] In an embodiment, CN 124 may be a 5GC, and RAN 106 may be connected to CN 124 via an NG interface 128. In an embodiment, NG interface 128 may be divided into two parts: an NG user plane (NG-U) interface that carries service data between base station 112 or base station 114 and a user plane function (UPF); and an S1 control plane (NG-C) interface that is a signaling interface between base station 112 or base station 114 and an access and mobility management function (AMF).

[0035] Generally speaking, application server 130 may be an element that provides applications (e.g., packet-switched data services) that use Internet Protocol (IP) bearer resources with CN 124. Application server 130 may also be configured to support one or more communication services (e.g., VoIP sessions, group communication sessions, etc.) for UE 102 and UE 104 via CN 124. Application server 130 may communicate with CN 124 via IP communication interface 132.

[0036] Figure 2 A system 200 for performing signaling 234 between a wireless device 202 and a network device 218 according to an embodiment disclosed herein is illustrated. The system 200 may be part of a wireless communication system as described herein. The wireless device 202 may be, for example, a UE of a wireless communication system. The network device 218 may be, for example, a base station (e.g., an eNB or a gNB) of a wireless communication system.

[0037] The wireless device 202 may include one or more processors 204. The processor 204 may execute instructions to perform various operations of the wireless device 202, as described herein. The processor 204 may include one or more baseband processors implemented using, for example, a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a controller, a field programmable gate array (FPGA) device, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein.

[0038] The wireless device 202 may include a memory 206. The memory 206 may be a non-transitory computer-readable storage medium that stores instructions 208 (which may include, for example, instructions executed by the processor 204). The instructions 208 may also be referred to as program code or a computer program. The memory 206 may also store data used by the processor 204 and results calculated by the processor.

[0039] The wireless device 202 may include one or more transceivers 210, which may include radio frequency (RF) transmitter and / or receiver circuitry that uses an antenna 212 of the wireless device 202 to facilitate signaling (e.g., signaling 234) to and / or from the wireless device 202 and other devices (e.g., network device 218) according to a corresponding RAT.

[0040] The wireless device 202 may include one or more antennas 212 (e.g., one, two, four, or more). For embodiments with multiple antennas 212, the wireless device 202 may utilize the spatial diversity of such multiple antennas 212 to transmit and / or receive multiple different data streams on the same time-frequency resource. This behavior may be referred to as, for example, multiple-input multiple-output (MIMO) behavior (referring to multiple antennas used at each of the transmitting device and the receiving device to implement this aspect). MIMO transmission by the wireless device 202 may be implemented based on precoding (or digital beamforming) applied at the wireless device 202, which multiplexes the data streams across the antennas 212 based on known or assumed channel characteristics, so that each data stream is received at an appropriate signal strength relative to the other streams and at a desired location in the spatial domain (e.g., the location of the receiver associated with the data stream). Certain embodiments may use a single-user MIMO (SU-MIMO) approach (where the data streams are all directed to a single receiver) and / or a multi-user MIMO (MU-MIMO) approach (where separate data streams may be directed to separate (different) receivers in different locations in the spatial domain).

[0041] In certain embodiments with multiple antennas, the wireless device 202 may implement analog beamforming techniques whereby the phases of signals transmitted by the antennas 212 are relatively adjusted so that the (joint) transmissions of the antennas 212 can be directed (this is sometimes referred to as beam steering).

[0042] The wireless device 202 may include one or more interfaces 214. The interfaces 214 may be used to provide input to or output from the wireless device 202. For example, the wireless device 202 (UE) may include an interface 214, such as a microphone, a speaker, a touch screen, buttons, etc., to allow a user of the UE to provide input and / or output to the UE. Other interfaces of such a UE may consist of transmitters, receivers, and other circuits (e.g., in addition to the transceiver 210 / antenna 212 described above) that allow communication between the UE and other devices, and may be based on known protocols (e.g., etc.) to perform the operation.

[0043] The wireless device 202 may include a model management module 216. The model management module 216 may be implemented via hardware, software, or a combination thereof. For example, the model management module 216 may be implemented as a processor, circuit, and / or instructions 208 stored in the memory 206 and executed by the processor 204. In some examples, the model management module 216 may be integrated within the processor 204 and / or the transceiver 210. For example, the model management module 216 may be implemented by a combination of software components (e.g., executed by a DSP or a general purpose processor) and hardware components (e.g., logic gates and circuits) within the processor 204 or the transceiver 210.

[0044] The model management module 216 may be used in various aspects of the present disclosure, for example, Figures 4 to 8 The model management module 216 is configured to perform model quality negotiation, monitoring, and rapid recovery under model failure detection for one or more AI / ML models associated with the wireless device 202.

[0045] The network device 218 may include one or more processors 220. The processor 220 may execute instructions to perform various operations of the network device 218, as described herein. The processor 204 may include one or more baseband processors implemented using, for example, a CPU, a DSP, an ASIC, a controller, an FPGA device, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein.

[0046] The network device 218 may include a memory 222. The memory 222 may be a non-transitory computer-readable storage medium that stores instructions 224 (which may include, for example, instructions executed by the processor 220). The instructions 224 may also be referred to as program code or a computer program. The memory 222 may also store data used by the processor 220 and results calculated by the processor.

[0047] The network device 218 may include one or more transceivers 226, which may include RF transmitter and / or receiver circuitry that uses an antenna 228 of the network device 218 to facilitate signaling (e.g., signaling 234) to and / or from the network device 218 and other devices (e.g., wireless device 202) according to a corresponding RAT.

[0048] The network device 218 may include one or more antennas 228 (eg, one, two, four, or more). In embodiments with multiple antennas 228, the network device 218 may perform MIMO, digital beamforming, analog beamforming, beamsteering, etc. as described.

[0049] The network device 218 may include one or more interfaces 230. The interfaces 230 may be used to provide input to or output from the network device 218. For example, the network device 218 (base station) may include an interface 230 consisting of a transmitter, a receiver, and other circuits (e.g., in addition to the transceiver 226 / antenna 228 already described) that enables the base station to communicate with other equipment in the core network and / or enables the base station to communicate with external networks, computers, databases, etc., for the purpose of performing operations, management, and maintenance of the base station or other equipment operably connected to the base station.

[0050] The network device 218 may include a model management module 232. The model management module 232 may be implemented via hardware, software, or a combination thereof. For example, the model management module 232 may be implemented as a processor, circuit, and / or instructions 224 stored in the memory 222 and executed by the processor 220. In some examples, the model management module 232 may be integrated within the processor 220 and / or the transceiver 226. For example, the model management module 232 may be implemented by a combination of software components (e.g., executed by a DSP or general purpose processor) and hardware components (e.g., logic gates and circuits) within the processor 220 or the transceiver 226.

[0051] The model management module 232 may be used in various aspects of the present disclosure, for example, Figures 4 to 8The model management module 232 is configured to perform model quality negotiation, monitoring, and rapid recovery under model failure detection for one or more AI / ML models associated with wireless devices connected to the network device 218.

[0052] The embodiments contemplated herein provide AI / ML model quality monitoring and rapid recovery under model fault detection in wireless communication systems.

[0053] The AI ​​technology described herein may refer to a method, system or machine that simulates human intelligence to perform tasks and can iteratively improve itself based on the information it collects. Machine learning (ML) technology may be a typical subfield of widely used AI technology. In the following, ML and AI are used interchangeably. The AI / ML technology that can be used with the embodiments disclosed herein may involve algorithms, processes, models and applications that have been designed, are being designed or will be designed. It is not intended to limit the present disclosure to any particular type of AI / ML technology.

[0054] AI / ML technology can be applied to wireless communication systems (e.g., cellular communication systems) to improve system performance. In some embodiments, AI / ML technology can be applied to channel state information (CSI) feedback enhancement (e.g., reducing overhead, improving accuracy and prediction). In some embodiments, AI / ML technology can be applied to beam management (BM) (e.g., beam prediction in the time domain, spatial domain to reduce overhead and delay, and improving beam selection accuracy). In some embodiments, AI / ML technology can be applied to positioning accuracy enhancement in different scenarios (including, for example, those with strong NLOS conditions). In other embodiments, AI / ML technology can be applied to other aspects of wireless communication systems. It should be easily understood that the use cases of AI / ML technology in wireless communication systems are not limited to the embodiments disclosed herein.

[0055] Figure 3 An example functional framework 300 of AI / ML in a wireless communication system according to embodiments disclosed herein is illustrated. The functional framework 300 may include a data collection function 302, a model training function 304, a model reasoning function 306, an actor function 308, and data flows between these functions.

[0056] The data collection function 302 may be a function that provides input data to the model training function 304 and the model reasoning function 306. Examples of input data may include measurement results from UEs or different network entities, feedback from actors, outputs from AI / ML models. The training data may include data required as input to the AI / ML model training function 304. The reasoning data may include data required as input to the AI / ML model reasoning function 306.

[0057] The model training function 304 may be a function that performs AI / ML model training, validation, and testing, which may generate model performance metrics as part of the model testing process. If necessary, the model training function 304 may also be responsible for data preparation (e.g., data preprocessing and cleaning, formatting, and conversion) based on the training data delivered by the data collection function 302. Model deployment / update is the initial deployment of the trained, validated, and tested AI / ML model to the model reasoning function 306 or the delivery of an updated model to the model reasoning function 306.

[0058] The model reasoning function 306 may be a function that provides an AI / ML model reasoning output (e.g., a prediction or decision). The model reasoning function 306 may provide model performance feedback to the model training function 304 when applicable. If necessary, the model reasoning function 306 may also be responsible for data preparation (e.g., data preprocessing and cleaning, formatting, and conversion) based on the reasoning data delivered by the data collection function 302. The output is the reasoning output of the AI / ML model generated by the model reasoning function 306. The details of the reasoning output may be specific to each use case. When available, model performance feedback can be used to monitor the performance of the AI / ML model.

[0059] The actor function 308 may be a function that receives output from the model reasoning function 306 and triggers or performs a corresponding action. The actor function 308 may trigger actions involving other entities or itself. Feedback is the information needed to derive training data, reasoning data, or monitor the performance of the AI / ML model and its impact on the network by updating KPIs and performance counters.

[0060] The functions of the functional framework 300 may be distributed and / or executed in various entities of the wireless communication system, including but not limited to one or more network devices (e.g., base stations), one or more servers of a mobile network operator (MNO), one or more UEs, and / or one or more servers serving the UE (e.g., servers provided by the manufacturer of the UE). For example, the data collection function 302 may be partially executed at the UE and / or partially executed at the network device to collect training data and / or reasoning data that can be used for model training and / or reasoning. The model training function 304 may be executed online (e.g., at the UE) or offline (e.g., at a server of the MNO or at a server serving the UE). The actor function 308 may involve different actions performed at different entities of the wireless communication system.

[0061] Due to the complexity of wireless communication systems, it may be challenging to apply AI / ML techniques to wireless communication systems. For example, the performance of an AI / ML model may depend on the specific implementation of an individual UE for which the AI / ML model is trained and / or the AI / ML model is to be deployed. Therefore, it may be desirable to train and / or deploy an AI / ML model specifically for an individual UE. In addition, an AI / ML model that is suitable for a specific wireless environment (e.g., trained for a specific wireless environment) may not work in different wireless environments. Due to the randomness of the wireless channel and the mobility of the UE, it is difficult for the AI / ML model to always maintain optimal performance in all scenarios, and the performance may even deteriorate sharply in some scenarios. It is expected to monitor the quality of the AI / ML model and recover quickly from model failures.

[0062] Figure 4 An example process 400 for performing training quality negotiation between a UE and a network device according to embodiments disclosed herein is illustrated. The process 400 may be performed by a UE or a module of a UE. In one example, the process 400 may be performed by a processor 204 of a wireless device 202, or by a model management module 216 of a wireless device 202. The network device may be a base station of a cellular network, such as the network device 218 described above.

[0063] Process 400 may start at step 402. In this step, the UE may be configured to receive a training quality query message from the network device. The training quality query message may include query information about one or more training quality metrics used to train one or more AI / ML models of the UE.

[0064] According to the embodiments disclosed herein, the query information may include one or more values ​​specified by the network device for the one or more training quality metrics. This allows the network device to specify the desired training quality for the one or more AI / ML models used to train the UE.

[0065] According to the embodiments disclosed herein, the one or more AI / ML models of the UE associated with the training quality query message may include various AI / ML models applicable to the UE. Examples of applicable AI / ML models may include AI / ML models involving channel state information (CSI) feedback enhancement, beam management (BM), and positioning accuracy enhancement, as described above. It should be readily understood that applicable AI / ML models are not limited to these use cases.

[0066] According to various embodiments, the one or more training quality metrics may include various metrics capable of measuring / affecting the training quality of the one or more AI / ML models of the UE. Examples of the one or more training quality metrics may include, but are not limited to:

[0067] the minimum size of a dataset used to train the one or more AI / ML models;

[0068] the maximum age of the dataset allowed to be used to train the one or more AI / ML models;

[0069] The time budget for training the one or more AI / ML models;

[0070] the allowed number of missing values ​​that need not be present in the dataset used to train the one or more AI / ML models; or

[0071] An allowed number of outliers that need not be present in the dataset used to train the one or more AI / ML models.

[0072] According to embodiments disclosed herein, the network device may include a training quality category index in the query information instead of including the one or more training quality metrics in the query information. The training quality category index may be a single value mapped to one or more expected training quality metrics selected from all possible training quality metrics based on a preconfigured mapping table.

[0073] After receiving the training quality query message, process 400 may proceed to step 404. In this step, the UE may be configured to send a training quality response message to the network device. The training quality response message may include response information about the one or more training quality metrics used to train the one or more AI / ML models of the UE.

[0074] According to the embodiments disclosed herein, the response information may include a rejection indication or an acceptance indication provided by the UE regarding at least one of the one or more values ​​specified by the network device for the one or more training quality metrics. In an optional embodiment, the response information in the training quality query message may also include one or more values ​​that the UE can accept for at least one of the one or more training quality metrics.

[0075] According to the embodiments disclosed herein, the training quality query message and the training quality response message may be a pair of radio resource control (RRC) messages.

[0076] Process 400 allows the UE and the network device to negotiate the training quality of the AI / ML model of the UE. After process 400, the one or more AI / ML models may be trained using the negotiated one or more training quality metrics, and the one or more trained AI / ML models may be stored (e.g., by the UE or other suitable entity of the system) for future use. Figure 5Additional details are described regarding performing training quality negotiation between a UE and a network device.

[0077] Figure 5 A communication process 500 for performing training quality negotiation between a UE and a network device according to an embodiment disclosed herein is illustrated. The UE may be a wireless device (such as the wireless device 202 described above) served by a network device. The network device may be a base station of a cellular network, such as the network device 218 described above. The communication process 500 may be performed by the UE and the network device, or by modules of the UE and the network device. In one example, the communication process 500 may be performed by the processor 204 of the wireless device 202 and the processor 220 of the network device 218. In another example, the communication process 500 may be performed by the model management module 216 of the wireless device 202 and the model management module 232 of the network device 218.

[0078] At step 502, the UE and the network device may be configured to exchange the AI / ML capabilities of the UE. For example, the network device may be configured to transmit a query to the UE regarding the UE's ability to apply AI / ML techniques. The UE's ability to apply AI / ML techniques may be associated with the UE's ability to train one or more AI / ML models and / or the UE's ability to run one or more trained AI / ML models. In response to the query, the UE may be configured to report the UE's AI / ML capabilities to the network device, for example via one or more capability reporting messages. Step 502 is illustrated with a dashed line because it is an optional step.

[0079] At step 504, the network device may be configured to send a training quality query message to the UE. As described above, the training quality query message may include query information about one or more training quality metrics for training one or more AI / ML models of the UE. The training quality query message may be carried in any type of message or signaling that can be transmitted from the network device to the UE. In a preferred embodiment, the training quality query message may be carried in an RRC message.

[0080] The one or more training quality metrics may include various metrics that can measure / influence the training quality of the one or more AI / ML models of the UE. In one embodiment, the one or more training quality metrics may include the minimum size of a data set to be used to train the one or more AI / ML models. The data set to be used to train the one or more AI / ML models may be referred to as a training data set. In general, a larger training data set can improve the training quality of the AI / ML model. The minimum size of the data set to be used to train the one or more AI / ML models may generally be associated with the minimum acceptable training quality of the one or more AI / ML models.

[0081] Additionally or alternatively, the one or more training quality metrics may include a maximum age of a dataset allowed for training the one or more AI / ML models. Generally speaking, training data collected a long time ago may not be suitable for training an AI / ML model for future use because such data may be outdated and therefore may not reflect the current / future state of the system. A maximum age of a dataset allowed for training the one or more AI / ML models may be specified to exclude data older than the specified maximum age.

[0082] Additionally or alternatively, the one or more training quality metrics may include a time budget for training the one or more AI / ML models. The metric may be specified to limit the duration that may be required to train the AI / ML models. Generally speaking, AI / ML models that are trained for a longer duration may have better training quality.

[0083] Additionally or alternatively, the one or more training quality metrics may include an allowed number of missing values ​​that do not need to be present in the data set used to train the one or more AI / ML models. Additionally or alternatively, the one or more training quality metrics may include an allowed number of outliers that do not need to be present in the data set used to train the one or more AI / ML models. In general, the presence of unwanted missing values ​​or outliers in the training data may reduce the accuracy of the model or cause the model to be biased. Therefore, if the relationship with other variables is not properly considered during model training, inaccurate inference output / prediction may occur. Therefore, it is critical to properly handle missing values ​​and outliers. The allowed number of undesirable outliers can be expressed as tolerances and thresholds.

[0084] It should be readily understood that the above training quality metrics are merely examples of the one or more training quality metrics. In other embodiments, other training quality metrics that can measure / affect the training quality of the one or more AI / ML models may be used without restriction.

[0085] The network device may generate and use the query information to specify a desired training quality for the one or more AI / ML models used to train the UE.

[0086] The query information may include an identifier of each of the one or more AI / ML models. For example, the query information may include an identifier of a first AI / ML model and a first set of one or more training quality metrics that may be associated with the first AI / ML model. Additionally, the query information may include an identifier of a second AI / ML model and a second set of one or more training quality metrics that may be associated with the second AI / ML model. The first set of one or more training quality metrics and the second set of one or more training quality metrics may be the same or different.

[0087] For each of the identified AI / ML model and its associated one or more training quality metrics, the query information may include one or more values ​​specified by the network device for the one or more training quality metrics. The one or more values ​​specified for the one or more training quality metrics may represent the network device's expected training quality for the UE with respect to the identified AI / ML model. In some embodiments, the value specified by the network device for the training quality metric may include a range of values ​​that includes multiple specified values.

[0088] Alternatively, the query information may include a training quality class index. The training quality class index may be a single value mapped to one or more expected training quality metrics selected from all possible training quality metrics based on a mapping table. Different training quality class indexes may be mapped to different subsets of training quality metrics in all possible training quality metrics. The mapping table may be preconfigured. The preconfigured table is known to both the network device and the UE, so that the UE may be configured to determine the expected one or more training quality metrics based on the received training quality class index.

[0089] In an embodiment where the network device has received the AI / ML capabilities of the UE (e.g., in step 502), the network device may select the one or more AI / ML models associated with the query information from all possible AI / ML models based on the AI / ML capabilities of the UE, so that the selected model can be adapted to the AI / ML capabilities of the UE. Additionally, the one or more training quality metrics may also be determined by the network device based on the AI / ML capabilities of the UE. In addition, the one or more values ​​specified by the network device for the one or more training quality metrics may also be determined by the network device based on the AI / ML capabilities of the UE.

[0090] At step 506, the UE may be configured to send a training quality response message to the network device in response to receiving the training quality query message. The training quality response message may include response information about the one or more training quality metrics used to train the one or more AI / ML models of the UE. The training quality response message may be carried in any type of message or signaling that can be transmitted from the UE to the network device. In a preferred embodiment, the training quality response message may be carried in an RRC message.

[0091] The UE may generate and use the response information in the training quality response message to indicate whether the expected training quality as indicated in the training quality query message is acceptable to the UE.

[0092] The response information may include an identifier of at least one of the one or more AI / ML models, the one or more AI / ML models being included in the training quality query message. For each of the identified AI / ML models, the response information may also include a rejection indication or an acceptance indication provided by the UE for at least one of the training quality metrics associated with the AI / ML model.

[0093] For example, if the UE determines that the value specified for one of the one or more training quality metrics in the query information is acceptable, the UE may provide an acceptance indication in the response information for the training quality metric. If the UE determines that the value specified for one of the one or more training quality metrics in the query information is unacceptable, the UE may provide a rejection indication in the response information for the training quality metric.

[0094] According to various embodiments, the UE may be configured to determine whether the value specified in the query information for one of the one or more training quality metrics is acceptable based on various factors. For example, the UE may be configured to make a determination based at least on a specific implementation of the UE and / or a current state of the UE.

[0095] In some embodiments, the response information in the training quality query message may include a rejection indication or an acceptance indication for each of the training quality metrics included in the training quality query message. In other embodiments, the response information may include only a rejection indication for each of the rejected training quality metrics, without including any acceptance indication for the acceptable training quality metrics. In an alternative embodiment, the response information may include only an acceptance indication for each of the acceptable training quality metrics, without including any rejection indication for the rejected training quality metrics.

[0096] In an optional embodiment, the response information in the training quality query message may include one or more values ​​that the UE can accept for at least one of the one or more training quality metrics. For example, for a rejected training quality metric, the UE may provide a corresponding value associated with the training quality metric in the response information, thereby indicating the provided value as a value that the UE can accept for the associated training quality metric.

[0097] At step 508, the AI / ML model training may be performed by the UE and / or the network device. The AI / ML model training may be performed under the negotiated one or more training quality metrics. In one embodiment, the negotiated one or more training quality metrics may be determined by steps 504 and 506. Specifically, after receiving the training quality response message at step 506, the network device may determine one or more values ​​for the one or more training quality metrics of the one or more AI / ML models. Therefore, the network device may configure the AI / ML model training using the values ​​determined for the one or more training quality metrics. In another embodiment, the negotiated one or more training quality metrics may be determined by repeating steps 504 and 506. For example, steps 504 and 506 may be repeated until a specific condition is met (such as no rejection indication is provided in the training quality response message). During the repetition of steps 504 and 506, the UE and the network device may be configured to dynamically modify their desired training quality or acceptable training quality so that they can eventually reach an agreement.

[0098] According to the embodiments disclosed herein, the AI / ML model training of step 508 may be performed online and / or offline, wherein the negotiated one or more training quality metrics are met. The trained one or more AI / ML models may be stored for future use. In one example, the trained AI / ML model may be stored locally at the UE. In another example, the trained AI / ML model may be uploaded to another entity of the system (e.g., an MNO server or a UE server). The trained AI / ML model herein may refer to a set of parameters of a general AI / ML model that has been specifically adjusted and improved for the UE.

[0099] Using the training quality negotiation described in process 400 and / or communication process 500, the UE and the network device may be able to negotiate a training quality metric to be applied in training one or more AI / ML models of the UE. This allows the UE and the network device to achieve a preferred training quality so that the trained AI / ML model may be specifically adapted to each individual UE.

[0100] The trained one or more AI / ML models may be applied to the UE for which the training quality has been negotiated. As shown in step 510, the network device may transmit an AI / ML model configuration message to the UE to deploy one or more trained AI / ML models intended for use with the UE. The AI / ML model configuration message may include an identifier of each of the expected AI / ML models, and optionally include one or more settings associated with the model (such as the inference quality monitor field discussed below). Upon receiving the AI / ML model configuration message, the UE may be configured to run the corresponding AI / ML model to collect inference output. At step 512, the UE may return an AI / ML model configuration completion message to the network device. The AI / ML model configuration message may be carried in an RRCReconfiguration message, and the AI / ML model configuration completion message may be carried in an RRCReconfigurationComplete message. Steps 510 and 512 are illustrated with dashed lines because they are not required for training quality negotiation.

[0101] Due to the randomness of the wireless channel and the mobility of the UE, it is difficult for AI / ML models to always maintain optimal performance, and in some scenarios the performance may even deteriorate dramatically. Specifically, when the AI / ML models no longer match the wireless environment in which they are used, the reasoning quality of the AI / ML models may degrade. As a result, the AI / ML models fail due to this mismatch. It is desirable to monitor the quality of the AI / ML models and recover quickly from model failures.

[0102] Figure 6A process 600 for performing fast recovery under model fault detection according to embodiments disclosed herein is illustrated. The process 600 may be performed by a UE or a module of a UE. In one example, the process 600 may be performed by the processor 204 of the wireless device 202 or by the model management module 216 of the wireless device 202.

[0103] Process 600 may begin at step 602. In this step, the UE may be configured to detect a failure state of a first AI / ML model of the UE. The first AI / ML model may be one of the AI / ML models currently running with the UE.

[0104] In some embodiments, the UE may detect the fault state by determining that the value of one or more inference quality metrics of the first AI / ML model does not match one or more first preconfigured thresholds of the one or more inference quality metrics. Examples of the one or more inference quality metrics may include: inference accuracy of the first AI / ML model; inference completeness of the first AI / ML model, the inference completeness being associated with the likelihood that a prediction error of the first AI / ML model exceeds a threshold; or inference latency of the first AI / ML model.

[0105] For example, the UE may be configured to calculate the value of the one or more inference quality metrics of the first AI / ML model using the collected truth labels and the inference output of the first AI / ML model. The calculated value may be compared with the one or more first preconfigured thresholds. For example, when deploying the first AI / ML model for use with the UE, the one or more first preconfigured thresholds may be preconfigured by the network device.

[0106] In a preferred embodiment, a trigger time mechanism may be used so that the fault state is detected only when the value of the one or more inference quality metrics of the first AI / ML model does not match the one or more first preconfigured thresholds within a preconfigured duration.

[0107] In response to detecting the failure state of the first AI / ML model, the UE may be configured to transmit model failure information associated with the first AI / ML model to the network device. The model failure information may be transmitted from the UE to the network device via various types of messages (e.g., RRC messages or MAC CE (MAC control element) messages). In one embodiment, the RRC message may be a UE assistance (UAI) message.

[0108] In an optional embodiment, the model failure information may also indicate one or more candidate AI / ML models preferred by the UE. The one or more candidate AI / ML models preferred by the UE may be determined based on various factors, such as the specific implementation of the UE.

[0109] After transmitting the model failure information, the UE may be configured to further receive a switching indication from the network device to switch to a second AI / ML model. The switching indication may be generated by the network device in response to the received model failure information. The switching indication may be received via various types of signaling from the network device to the UE (e.g., RRC signaling or MAC CE message from the network device to the UE).

[0110] In an optional embodiment, the second AI / ML model may be selected based at least in part on the one or more candidate AI / ML models preferred by the UE.

[0111] In an alternative embodiment, the UE is not configured to locally compare the calculated values ​​of the one or more inference quality metrics of the first AI / ML model with the one or more first preconfigured thresholds of the one or more inference quality metrics. Instead, the UE is configured to report the calculated values ​​of the one or more inference quality metrics of the first AI / ML model to the network device, and receive a switching indication from the network device to switch to the second AI / ML model that is different from the first AI / ML model. In other words, the network device may be responsible for determining a failure of the first AI / ML model based on the reported values ​​of the one or more inference quality metrics of the first AI / ML model, and then indicating the determined failure state to the UE.

[0112] In some embodiments, the value of the one or more inference quality metrics of the first AI / ML model may be reported by the UE to the network device in a periodic manner. The periodic cycle of reporting the value may be preconfigured by the network device, for example, when deploying the first AI / ML model for use with the UE.

[0113] In other embodiments, the UE may report the value of the one or more inference quality metrics of the first AI / ML model to the network device in response to one or more triggering events. These triggering events may be preconfigured by the network device, for example when the first AI / ML model is deployed for use with the UE. Examples of these triggering events may include the value of the one or more inference quality metrics of the first AI / ML model not matching one or more second preconfigured thresholds. Similarly, a trigger time mechanism may be used so that the values ​​are reported to the network device only when the value of the one or more inference quality metrics fails to match the one or more second preconfigured thresholds for a preconfigured duration.

[0114] After detecting the failure state of the first AI / ML model, process 600 may proceed to step 604. In this step, the UE may be configured to switch to a second AI / ML model different from the first AI / ML model. As described above, the switching may be performed in response to receiving a switching indication from the network device. The switching indication may be received via various types of signaling, including RRC signaling or a MAC CE message.

[0115] Combined with the following Figure 7 and Figure 8 Describes additional details about performing fast recovery under model failure detection.

[0116] Figure 7 A communication process 700 for performing fast recovery under model fault detection according to embodiments disclosed herein is illustrated. A UE may be a wireless device (such as the wireless device 202 described above) served by a network device. The network device may be a base station of a cellular network, such as the network device 218 described above. The communication process 700 may be performed by the UE and the network device, or by modules of the UE and the network device. In one example, the communication process 700 may be performed by the processor 204 of the wireless device 202 and the processor 220 of the network device 218, or by the model management module 216 of the wireless device 202 and the model management module 232 of the network device 218.

[0117] At step 702, the network device may be configured to transmit an AI / ML model configuration message to the UE to deploy one or more AI / ML models intended for use with the UE. As described above, the one or more AI / ML models may have been trained for use with the UE at a negotiated training quality for the UE. The AI / ML model configuration message may include an identifier of each of the intended AI / ML models, and optionally one or more settings associated with the model (such as an inference quality monitoring field). The AI / ML model configuration message may be carried in an RRCReconfiguration message.

[0118] exist Figure 7 In an example, the inference quality monitoring field in the AI / ML model configuration may include an inference quality threshold field (e.g., "InferenceQualityThreshold"). The inference quality threshold field may specify a first preconfigured threshold for one or more inference quality metrics as described above. The UE may compare the received first preconfigured threshold with the calculated value of the one or more inference quality metrics to determine whether the running AI / ML model has failed (or does not match the wireless environment).

[0119] When receiving the AI / ML model configuration message, the UE may be configured to deploy and run at least one AI / ML model indicated in the AI / ML model configuration message. At step 704, the UE may also return an AI / ML model configuration completion message to the network device. The AI / ML model configuration completion message may be carried in the RRCReconfigurationComplete message.

[0120] Step 702 and step 704 are illustrated with dashed lines because they are optional steps for performing fast recovery under model fault detection.

[0121] When running the AI / ML model, the UE may be configured to monitor whether the running AI / ML model fails. At step 706, the UE may be configured to detect a model failure of the AI / ML model. Specifically, the UE may detect a failure state of the AI / ML model by determining that the value of one or more inference quality metrics of the AI / ML model does not match one or more first preconfigured thresholds of the one or more inference quality metrics. The value of the one or more inference quality metrics of the AI / ML model may be calculated by the UE using the collected truth labels and the inference output of the AI / ML model. For example, when the AI / ML model is applied to CSI feedback enhancement (such as CSI channel compression), the value of the one or more inference quality metrics of the AI / ML model may be calculated by the correlation between the uncompressed original channel and the compressed channel (which may be the inference output of the AI / ML model). In step 702, the one or more first preconfigured thresholds may be configured by the network device, for example, via the "InferenceQualityThreshold" field of the AI / ML model configuration message.

[0122] In an optional embodiment, a trigger time mechanism may be used to detect the fault state. Specifically, the UE will determine that the AI / ML model has failed only when the value of the one or more inference quality metrics of the AI / ML model does not match the one or more first preconfigured thresholds within a preconfigured duration.

[0123] According to the embodiments disclosed herein, the one or more inference quality metrics may include various metrics capable of measuring the quality of the inference output of the AI / ML model. Examples of the inference quality metrics may include, but are not limited to:

[0124] The inference accuracy (mean or variance) of the AI / ML model;

[0125] the inference completeness of the AI / ML model, which is associated with the likelihood that the AI / ML model’s prediction error exceeds a threshold; or

[0126] The inference latency of the AI / ML model, which indicates how long it takes for the model to output an inference result after receiving the input.

[0127] At step 708, in response to detecting the failure state of the AI / ML model, the UE may be configured to transmit model failure information associated with the AI / ML model to the network device. The model failure information may be transmitted via various types of messages.

[0128] In one embodiment, the model failure information may be transmitted via an RRC message. The RRC message may include a failure status of the AI / ML model, indicating that the AI / ML model does not match the wireless environment. Additionally, the RRC message may also include a calculated value of an inference quality metric associated with the failed AI / ML model. In an optional embodiment, the RRC message may also indicate one or more candidate AI / ML models preferred by the UE to switch to. The one or more candidate AI / ML models preferred by the UE may be determined based on the specific implementation of the UE. In one embodiment, the RRC message may be a UE Assisted (UAI) message.

[0129] In an alternative embodiment, the model failure information may be transmitted via a MAC CE message, which may be referred to as a model failure recovery (MFR) MAC CE message. The sending of the MFR MAC CE message may be used as an indication of a failure state of the AI / ML model currently running with the UE. The MFR MAC CE message may also include an indication of one or more candidate AI / ML models preferred by the UE to switch to. For example, the MFR MAC CE message may include multiple fields, each field indicating whether a corresponding AI / ML model is preferred by the UE. In this example, the MFR MAC CE message may have a structure as shown in Table 1.

[0130]

[0131] Table 1

[0132] As shown in the example of Table 1, the MFR MAC CE message may include eight fields C0 to C7, each field corresponding to a corresponding candidate AI / ML model that can be used with the UE. If the corresponding candidate AI / ML model is preferred by the UE, the UE may set the corresponding fields in C0 to C7 to a first predetermined value (e.g., 1). In addition, if the corresponding candidate AI / ML model is not preferred by the UE, the UE may set the corresponding fields in C0 to C7 to a second predetermined value (e.g., 0). In some embodiments, the UE may set multiple fields to a first value to indicate multiple preferred AI / ML models. Optionally, the tiebreaker field may provide additional information, for example to facilitate selection between multiple preferred AI / ML models.

[0133] In this embodiment, the mapping between fields C0 to C7 and the corresponding AI / ML models can be preconfigured. For example, the RRC configuration may include identifiers of the AI / ML models, and each of these identifiers may be configured to map to one of fields C0 to C7 for fast model recovery. It should be readily appreciated that any suitable number of fields may be used, not limited to eight fields C0 to C7.

[0134] In this embodiment where the MFR MAC CE message is used to carry model failure information, during the logical channel prioritization (LCP) process, the priority of the MFR MAC CE message may be lower than the priority of the MAC CE message for multiple expected protection symbols and higher than the priority of the MAC CE message for the preemptive buffer status report (BSR).

[0135] At step 710, after receiving the model failure information, the network device may be configured to transmit a switching indication to the UE. In some embodiments, the switching indication may be transmitted via an RRC message or a MAC CE message. For example, the switching indication may be included in an RRCReconfiguration message. For example, the network device may transmit an AI / ML model configuration message associated with a new AI / ML model to instruct the UE to deploy the new AI / ML model to replace the AI / ML model that has been detected as a failure.

[0136] At step 712, in response to receiving the switching indication, the UE may be configured to switch to the new AI / ML model. For example, the UE may be configured to feed input to the new AI / ML model and collect reasoning output from the new model. The AI / ML model may be discarded by not providing input to the previous AI / ML model that has been detected as a failure. Additionally, the UE may transmit a success indication to the network device. After switching to the new AI / ML model, the UE may keep monitoring the reasoning quality of the new AI / ML model.

[0137] Figure 8 Another communication process 800 for performing fast recovery under model fault detection according to the embodiments disclosed herein is illustrated. The UE may be a wireless device (such as the wireless device 202 described above) served by a network device. The network device may be a base station of a cellular network, such as the network device 218 described above. The communication process 800 may be performed by the UE and the network device, or by modules of the UE and the network device. In one example, the communication process 800 may be performed by the processor 204 of the wireless device 202 and the processor 220 of the network device 218, or by the model management module 216 of the wireless device 202 and the model management module 232 of the network device 218.

[0138] At step 802, the network device may transmit an AI / ML model configuration message to the UE to deploy one or more AI / ML models intended for use with the UE. As described above, the one or more AI / ML models may have been trained for use with the UE at a negotiated training quality for the UE. The AI / ML model configuration message may include an identifier of each of the intended AI / ML models and, optionally, one or more settings associated with the model (such as an inference quality monitoring field).

[0139] exist Figure 8 In an example, the inference quality monitoring field in the AI / ML model configuration message may include an inference quality reporting configuration field (e.g., "ReportConfigInferenceQuality"), which may specify attributes of reporting the inference quality from the UE to the network device. The specified attributes may include, but are not limited to, a trigger type (periodic or event triggered), a length of a periodic cycle, and / or one or more triggering events.

[0140] When receiving the AI / ML model configuration message, the UE may be configured to deploy and run at least one AI / ML model indicated in the AI / ML model configuration message. Additionally, at step 804, the UE may return an AI / ML model configuration completion message to the network device.

[0141] The AI / ML model configuration message may be carried in an RRCReconfiguration message, and the AI / ML model configuration completion message may be carried in an RRCReconfigurationComplete message.

[0142] Step 802 and step 804 are illustrated with dashed lines because they are optional steps for performing fast recovery under model fault detection.

[0143] When running the AI / ML model, the UE may be configured to monitor the AI / ML model and calculate the value of one or more inference quality metrics of the AI / ML model. As discussed above, examples of the inference quality metrics may include, but are not limited to: the inference accuracy (mean or variance) of the AI / ML model; the inference completeness of the AI / ML model, which is associated with the likelihood that the prediction error of the AI / ML model exceeds a threshold; or the inference latency of the AI / ML model, which indicates how long it takes for the model to output an inference result after receiving the input.

[0144] The calculation of the value of the one or more inference quality metrics of the AI / ML model may be similar to that described above. Specifically, the UE may be configured to calculate the value of one or more inference quality metrics of the first AI / ML model using the collected truth labels and the inference output of the AI / ML model.

[0145] exist Figure 8 In the example of , the UE may be configured to report the calculated values ​​of one or more inference quality metrics to the network device in response to one or more triggering events or in a periodic manner. At step 806, the occurrence of the triggering event or the expiration of the periodic cycle may be detected.

[0146] In an embodiment where the UE reports in a periodic manner, the UE may maintain a periodic timer. Whenever the periodic timer expires at step 806, the UE may be triggered to report the calculated values ​​of one or more inference quality metrics to the network device at step 808.

[0147] In another embodiment, the UE reports in response to the occurrence of one or more predetermined triggering events. For example, a triggering event may occur when the calculated value of one or more inference quality metrics of the current AI / ML model does not match one or more preconfigured thresholds. In another example, a triggering event may occur when the calculated value of the quality metric does not match the threshold for a preconfigured duration. Other triggering events may also be used. When at least one of the triggering events is detected at step 806, the UE may be triggered to report the calculated value of one or more inference quality metrics to the network device at step 808.

[0148] At step 810, after receiving the reported values ​​of one or more inference quality metrics of the AI / ML model from the UE, the network device may be configured to detect a fault state of the AI / ML model. The detection of the fault state may be similar to the detection in step 706, but performed by the network device. Specifically, the network device may detect the fault state of the AI / ML model by determining that the reported calculated values ​​of one or more inference quality metrics of the AI / ML model do not match one or more first preconfigured thresholds of the one or more inference quality metrics. Similarly, a trigger time mechanism may be optionally applied.

[0149] The one or more first preconfigured thresholds may be different from the one or more preconfigured thresholds for triggering reporting. More specifically, the one or more first preconfigured thresholds may be higher than the one or more preconfigured thresholds for triggering reporting.

[0150] If the network device does not detect any failure of the AI / ML model at step 810, the network device may take no further action, and the communication process 800 may return to step 806 to wait for the next trigger event or the next expiration of the periodic timer. If the network device detects the failure state of the AI / ML model at step 810, then at step 812, the network device may be configured to transmit a switching indication to the UE. In some embodiments, the switching indication may be transmitted via an RRC message or a MAC CE message. For example, the switching indication may be included in an RRCReconfiguration message. More specifically, the network device may transmit an AI / ML model configuration message associated with the new AI / ML model to instruct the UE to switch the new AI / ML model.

[0151] At step 814, in response to receiving the switching indication, the UE may be configured to switch to the new AI / ML model. Additionally, the UE may transmit a success indication to the network device. After switching to the new AI / ML model, the UE may continue to monitor and report the inference quality of the new AI / ML model.

[0152] By utilizing the methods described in process 600 and / or communication processes 700 / 800, the UE can recover quickly under model failure detection.

[0153] Embodiments contemplated herein include an apparatus including components for performing one or more elements of processes 400 and / or 600 and / or communication processes 500, 700, and / or 800. The apparatus may be, for example, an apparatus of a UE (such as wireless device 202 (UE), as described herein).

[0154] Embodiments contemplated herein include one or more non-transitory computer-readable media including instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform one or more elements of processes 400 and / or 600 and / or communication processes 500, 700, and / or 800. The non-transitory computer-readable medium may be, for example, a memory of a UE (such as memory 206 of wireless device 202 (UE), as described herein).

[0155] Embodiments contemplated herein include an apparatus including logical components, modules, or circuits for performing one or more elements of process 400 and / or 600 and / or communication processes 500, 700, and / or 800. The apparatus may be, for example, an apparatus of a UE (such as wireless device 202 (UE), as described herein).

[0156] Embodiments contemplated herein include an apparatus comprising: one or more processors and one or more computer-readable media comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform one or more elements of processes 400 and / or 600 and / or communication processes 500, 700, and / or 800. The apparatus may be, for example, an apparatus of a UE (such as wireless device 202 (UE), as described herein).

[0157] Embodiments contemplated herein include a signal as described in or associated with one or more elements of processes 400 and / or 600 and / or communication processes 500 , 700 , and / or 800 .

[0158] Embodiments contemplated herein include a computer program or computer program product that includes instructions that, when executed by a processor, cause the processor to perform one or more elements of process 400 and / or 600 and / or communication process 500, 700, and / or 800. The processor may be a processor of a UE (such as processor 204 of wireless device 202 (UE), as described herein). These instructions may be located, for example, in a processor and / or on a memory of a UE (such as memory 206 of wireless device 202 (UE), as described herein).

[0159] Embodiments contemplated herein include an apparatus including means for performing one or more elements of communication processes 500, 700, and / or 800. The apparatus may be, for example, an apparatus of a base station (such as network device 218 (base station), as described herein).

[0160] Embodiments contemplated herein include one or more non-transitory computer-readable media including instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform one or more elements of communication processes 500, 700, and / or 800. The non-transitory computer-readable medium may be, for example, a memory of a base station (such as memory 222 of network device 218 (base station), as described herein).

[0161] Embodiments contemplated herein include an apparatus comprising logical components, modules, or circuits for performing one or more elements of communication processes 500, 700, and / or 800. The apparatus may be, for example, an apparatus of a base station (such as network device 218 (base station), as described herein).

[0162] Embodiments contemplated herein include an apparatus comprising: one or more processors and one or more computer-readable media comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform one or more elements of communication processes 500, 700, and / or 800. The apparatus may be, for example, an apparatus of a base station (such as network device 218 (base station), as described herein).

[0163] Embodiments contemplated herein include a signal as described in or associated with one or more elements of communication processes 500 , 700 , and / or 800 .

[0164] Embodiments contemplated herein include a computer program or computer program product that includes instructions that, when executed by a processing element, cause the processing element to perform one or more elements of communication processes 500, 700, and / or 800. The processor may be a processor of a base station (such as processor 220 of network device 218 (base station), as described herein). These instructions may be located, for example, in the processor and / or on a memory of a UE (such as memory 222 of network device 218 (base station), as described herein).

[0165] For one or more embodiments, at least one of the components set forth in one or more of the foregoing figures may be configured to perform one or more operations, techniques, processes, and / or methods as described herein. For example, a baseband processor as described herein in conjunction with one or more of the foregoing figures may be configured to operate according to one or more of the examples set forth herein. For another example, circuits associated with a UE, a base station, a network element, etc. as described above in conjunction with one or more of the foregoing figures may be configured to operate according to one or more of the examples set forth herein.

[0166] Unless otherwise expressly stated, any of the above embodiments may be combined with any other embodiment (or combination of embodiments). The foregoing description of one or more specific implementations provides illustration and description, but is not intended to be exhaustive or to limit the scope of the embodiments to the precise form disclosed. In view of the above teachings, modifications and variations are possible or can be obtained from the practice of various embodiments.

[0167] Embodiments and implementations of the systems and methods described herein may include various operations that may be embodied in machine executable instructions to be executed by a computer system. A computer system may include one or more general or special purpose computers (or other electronic devices). A computer system may include hardware components that include specific logic components for performing operations; or may include a combination of hardware, software, and / or firmware.

[0168] It should be appreciated that the systems described herein include descriptions of specific embodiments. These embodiments may be combined into a single system, partially combined into other systems, separated into multiple systems, or otherwise divided or combined. In addition, it is contemplated that parameters, attributes, aspects, etc. of another embodiment may be used in one embodiment. For clarity, these parameters, attributes, aspects, etc. are described only in one or more embodiments, and it should be appreciated that these parameters, attributes, aspects, etc. may be combined with or substituted for parameters, attributes, aspects, etc. of another embodiment unless expressly stated herein.

[0169] It is understood that the use of personally identifiable information should be subject to privacy policies and practices that are generally recognized to meet or exceed industry or government requirements for maintaining user privacy. Specifically, personally identifiable information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use, and the nature of the authorized use should be clearly stated to users.

[0170] Although the foregoing has been described in considerable detail for the sake of clarity, it will be apparent that certain changes and modifications may be made without departing from the principles of the invention. It should be noted that there are many alternative ways to implement both the processes and the apparatus described herein. Therefore, the embodiments of the present invention should be regarded as illustrative rather than restrictive, and the specification is not limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.

Claims

1. A user equipment (UE), the user equipment (UE) comprising: at least one antenna; at least one radio component coupled to the at least one antenna; and a processor coupled to the at least one radio; The processor is configured to: receiving a training quality query message from a network device, wherein the training quality query message includes query information about one or more training quality metrics used to train one or more artificial intelligence (AI) models of the UE; as well as A training quality response message is sent to the network device, wherein the training quality response message includes response information about the one or more training quality metrics used to train the one or more AI models of the UE.

2. The UE of claim 1 , wherein the one or more training quality metrics associated with one or more AI models include one or more of the following: the minimum size of a dataset used to train the one or more AI models; the maximum age of a dataset allowed to be used to train the one or more AI models; a time budget for training the one or more AI models; the allowed number of missing values ​​that need not be present in the dataset used to train the one or more AI models; or An allowed number of outliers that need not be present in a dataset used to train the one or more AI models. 3 . The UE of claim 1 , wherein the query information comprises one or more values ​​specified by the network device for the one or more training quality metrics. 4 . The UE of claim 1 , wherein the query information comprises a training quality category index, and the training quality category index is mapped to the one or more training quality metrics based on a preconfigured mapping table. 5 . The UE according to claim 1 , wherein the response information comprises a rejection indication or an acceptance indication provided by the UE regarding at least one of the one or more training quality metrics. 6 . The UE according to claim 1 , wherein the response information comprises one or more values ​​acceptable to the UE for at least one of the one or more training quality metrics. 7 . The UE of claim 1 , wherein the training quality query message and the training quality response message are communicated via a radio resource control (RRC) message pair.

8. The UE of claim 1, wherein the processor is further configured to exchange the AI ​​capability of the UE with the network device.

9. The UE of claim 8, wherein the one or more training quality metrics are determined based on an AI capability of the UE.

10. The UE of claim 1, wherein the processor is configured to train the one or more AI models under the one or more training quality metrics.

11. A user equipment (UE), the user equipment (UE) comprising: at least one antenna; at least one radio component coupled to the at least one antenna; and a processor coupled to the at least one radio; The processor is configured to: detecting a failure state of a first artificial intelligence (AI) model of the UE; and Switch to a second AI model different from the first AI model.

12. The UE according to claim 11, wherein the processor is configured to detect the failure state of the first AI model by at least the following steps: It is determined, by the processor of the UE, that values ​​of one or more inference quality metrics of the first AI model do not match one or more first preconfigured thresholds of the one or more inference quality metrics.

13. The UE of claim 12, wherein the first preconfigured thresholds of the one or more inference quality metrics are configured by a network device.

14. The UE of claim 12, wherein the one or more inference quality metrics of the first AI model include at least one of the following: the reasoning accuracy of the first AI model; the inference completeness of the first AI model, the inference completeness being associated with a likelihood that a prediction error of the first AI model exceeds a threshold; or The inference latency of the first AI model.

15. The UE of claim 12, wherein the processor is configured to calculate the values ​​of the one or more inference quality metrics of the first AI model using the collected truth labels and the inference output of the first AI model.

16. The UE of claim 12, wherein the failure state is detected when the values ​​of the one or more inference quality metrics of the first AI model do not match the one or more first preconfigured thresholds for a preconfigured duration.

17. The UE according to claim 12, wherein the processor is configured to: In response to detecting the failure state of the first AI model, transmitting model failure information associated with the first AI model to a network device; and A switching instruction to switch to the second AI model is received from the network device.

18. The UE of claim 17, wherein the model failure information indicates one or more candidate AI models preferred by the UE.

19. The UE of claim 18, wherein the one or more candidate AI models preferred by the UE are determined based on a specific implementation of the UE.

20. The UE of claim 17, wherein the model failure information is transmitted via an RRC message or a MAC CE message.

21. The UE of claim 20, wherein the MAC CE message comprises a plurality of fields, each field indicating whether a corresponding AI model among a plurality of AI models is preferred by the UE.

22. The UE according to claim 17, wherein: During a logical channel prioritization (LCP) procedure, the priority of the MAC CE message for the model fault information is lower than the priority of the MAC CE message for multiple desired protection symbols and higher than the priority of the MAC CE message for a preemptive buffer status report (BSR).

23. The UE of claim 17, wherein the handover indication is received via radio resource control (RRC) signaling or a MAC CE message.

24. The UE according to claim 11, wherein the processor is configured to detect the failure state of the first AI model by at least the following steps: reporting values ​​of one or more inference quality metrics of the first AI model to a network device; and A switching instruction is received from the network device to switch to the second AI model different from the first AI model.

25. The UE of claim 24, wherein the reporting is configured by the network device.

26. The UE of claim 24, wherein the one or more inference quality metrics include at least one of the following: the reasoning accuracy of the first AI model; the inference completeness of the first AI model, the inference completeness being associated with a likelihood that a prediction error of the first AI model exceeds a threshold; or The inference latency of the first AI model.

27. The UE of claim 24, wherein the values ​​of the one or more inference quality metrics of the first AI model are reported in a periodic manner.

28. The UE of claim 24, wherein the values ​​of the one or more inference quality metrics of the first AI model are reported in response to one or more triggering events.

29. The UE of claim 28, wherein the one or more triggering events include values ​​of the one or more inference quality metrics of the first AI model not matching one or more second preconfigured thresholds for a preconfigured duration.

30. The UE of claim 24, wherein the handover indication is received via radio resource control (RRC) signaling or a MAC CE message.

Images