Artificial intelligence / machine learning model mechanisms for beam management

Abstract

Exemplary embodiments of the present disclosure relate to artificial intelligence / machine learning (AI / ML) model mechanisms for beam management. In an aspect, a terminal device transmits, to a first network device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed. The terminal device further receives, from the first network device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

Description

ARTIFICIAL INTELLIGENCE / MACHINE LEARNING MODEL MECHANISMS FOR BEAM MANAGEMENTCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority from, and the benefit of, EP Application No. 25156182.5, filed February 6, 2025, the contents of which are hereby incorporated by reference in their entirety.FIELD

[0002] Exemplary embodiments of the present disclosure generally relate to the field of communications, and in particular, to devices, apparatuses, methods and a computer-readable storage medium for improving artificial intelligence / machine learning model (AI / ML) mechanisms for beam management.BACKGROUND

[0003] A communication network can be seen as a facility that enables communications between two or more communication devices, or provides communication devices access to a data network. A mobile or wireless communication network is one example of a communication network.

[0004] Such communication networks operate in according with standards such as those provided by 3GPP (Third Generation Partnership Project) or ETSI (European Telecommunications Standards Institute). Examples of standards are the so-called 5G (5th Generation) standards provided by 3GPP.SUMMARY

[0005] In general, exemplary embodiments of the present disclosure provide a solution for improving AI / ML model mechanisms for beam management.

[0006] In a first aspect, there is provided a terminal device. The terminal device comprises: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the terminal device at least to: transmit, to a first network device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; and receive, from the first network device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

[0007] In a second aspect, there is provided a first network device. The first network device comprises: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the first network device at least to: receive, from a terminal device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; and transmit, to the terminal device, a result about whether the AI / ML model fails, wherein the result is based on comparison between theperformance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

[0008] In a third aspect, there is provided a second network device, the second network device comprises: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the second network device at least to: receive, from a first network device, an operational condition of a terminal device and performance metrics of an artificial intelligence / machine learning (AI / ML) within a monitoring window over which performance metrics of the AI / ML model are aggregated and assessed; determine, based on historical performance data of the AI / ML model and the operational condition, at least one threshold for assessing performance metrics of the AI / ML model; determine, based on comparison between the performance metrics and the at least one threshold, whether the AI / ML model fails; and transmit, to the first network device, a result about whether the AI / ML model fails.

[0009] In a fourth aspect, there is provided a method performed by a terminal device. The method comprises: transmitting, to a first network device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; and receiving, from the first network device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

[0010] In a fifth aspect, there is provided a method performed by a first network device. The method comprises: receiving, from a terminal device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; and transmitting, to the terminal device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

[0011] In a sixth aspect, there is provided a method performed by a second network device. The method comprises: receiving, from a first network device, an operational condition of a terminal device and performance metrics of an artificial intelligence / machine learning (AI / ML) within a monitoring window over which performance metrics of the AI / ML model are aggregated and assessed; determining, based on historical performance data of the AI / ML model and the operational condition, at least one threshold for assessing performance metrics of the AI / ML model; determining, based on comparison between the performance metrics and the at least one threshold, whether the AI / ML model fails; and transmitting, to the first network device, a result about whether the AI / ML model fails.

[0012] In a seventh aspect, there is provided an apparatus. The apparatus comprises: means for transmitting, to a first network device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; and means for receiving, from the first network device, a result about whether theAI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

[0013] In an eighth aspect, there is provided an apparatus. The apparatus comprises: means for receiving, from a terminal device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; and means for transmitting, to the terminal device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

[0014] In a ninth aspect, there is provided an apparatus. The apparatus comprises: means for receiving, from a first network device, an operational condition of a terminal device and performance metrics of an artificial intelligence / machine learning (AI / ML) within a monitoring window over which performance metrics of the AI / ML model are aggregated and assessed; means for determining, based on historical performance data of the AI / ML model and the operational condition, at least one threshold for assessing performance metrics of the AI / ML model; means for determining, based on comparison between the performance metrics and the at least one threshold, whether the AI / ML model fails; and means for transmitting, to the first network device, a result about whether the AI / ML model fails.

[0015] In a tenth aspect, there is provided a non-transitory computer readable medium comprising program instructions for causing an apparatus to perform at least the method according to any of fourth to sixth aspects.

[0016] In an eleventh aspect, there is provided a computer program comprising instructions, which, when executed by an apparatus, cause the apparatus at least to perform at least the method according to any of fourth to sixth aspects.

[0017] In a twelfth aspect, there is provided a terminal device. The terminal device comprises: transmitting circuitry configured to transmit, to a first network device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; and receiving circuitry configured to receive, from the first network device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

[0018] In a thirteenth aspect, there is provided a first network device. The first network device comprises: receiving circuitry configured to receive, from a terminal device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; and transmitting circuitry configured to transmit, to the terminal device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

[0001] In a fourteenth aspect, there is provided a second network device. The second network device comprises: receiving circuitry configured to receive, from a first network device, an operational condition of a terminal device and performance metrics of an artificial intelligence / machine learning (AI / ML) within a monitoring window over which performance metrics of the AI / ML model are aggregated and assessed; first determining circuitry configured to determine, based on historical performance data of the AI / ML model and the operational condition, at least one threshold for assessing performance metrics of the AI / ML model; second determining circuitry configured to determine, based on comparison between the performance metrics and the at least one threshold, whether the AI / ML model fails; and transmitting circuitry configured to transmit, to the first network device, a result about whether the AI / ML model fails.

[0002] In a fifteenth aspect, there is provided a terminal device. The terminal device may comprise at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the terminal device at least to: transmit, to a network device, an operational condition of the terminal device within a monitoring window over which an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed; and receive a result of the anomaly from the network device, wherein the result is based on comparison between performance metrics of the AI / ML model and historical data.

[0003] In a sixteenth aspect, there is provided a network device. The network device may comprise at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the network device at least to: receive, from a terminal device, an operational condition of the terminal device within a monitoring window over which an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed; and transmit a result of the anomaly detection to the terminal device, wherein the result is based on comparison between performance metrics of the AI / ML model and historical data.

[0004] In a seventeenth aspect, there is provided a method. The method may comprise: transmitting, to a network device, an operational condition of the terminal device within a monitoring window over which an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed; and receiving a result of the anomaly from the network device, wherein the result is based on comparison between performance metrics of the AI / ML model and historical data.

[0005] In an eighteenth aspect, there is provided a method. The method may comprise: receiving, from a terminal device, an operational condition of the terminal device within a monitoring window over which an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed; and transmitting a result of the anomaly detection to the terminal device, wherein the result is based on comparison between performance metrics of the AI / ML model and historical data.

[0006] In a nineteenth aspect, there is provided an apparatus. The apparatus may comprise: means for transmitting, to a network device, an operational condition of the terminal device within a monitoring windowover which an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed; and means for receiving a result of the anomaly from the network device, wherein the result is based on comparison between performance metrics of the AI / ML model and historical data.

[0007] In a twentieth aspect, there is provided an apparatus. The apparatus may comprise: means for receiving, from a terminal device, an operational condition of the terminal device within a monitoring window over which an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed; and means for transmitting a result of the anomaly detection to the terminal device, wherein the result is based on comparison between performance metrics of the AI / ML model and historical data.

[0008] In a twenty-first aspect, there is provided a non-transitory computer readable medium comprising program instructions for causing an apparatus to perform at least the method according to seventeenth or eighteenth aspect.

[0009] In a twenty-second aspect, there is provided a computer program comprising instructions, which, when executed by an apparatus, cause the apparatus at least to perform at least the method according to seventeenth or eighteenth aspect.

[0010] In a twenty-third aspect, there is provided a terminal device. The terminal device may comprise: transmitting circuitry configured to transmit, to a network device, an operational condition of the terminal device within a monitoring window over which an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed; and receiving circuitry configured to receive a result of the anomaly from the network device, wherein the result is based on comparison between performance metrics of the AI / ML model and historical data.

[0011] In a twenty-fourth aspect, there is provided a network device. The network device may comprise: receiving circuitry configured to receive, from a terminal device, an operational condition of the terminal device within a monitoring window over which an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed; and transmitting circuitry configured to transmit a result of the anomaly detection to the terminal device, wherein the result is based on comparison between performance metrics of the AI / ML model and historical data.

[0012] In a twenty-fifth aspect, there is provided a terminal device. The terminal device may comprise at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the terminal device at least to: obtain an operational condition of the terminal device; and determine, based on the operational condition, a size of a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are assessed.

[0013] In a twenty-sixth aspect, there is provided a network device. The network device may comprise at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the network device at least to: receive, from a terminal device, an operational condition of the terminal device within a determined monitoring window over which performance metrics of an artificialintelligence / machine learning (AI / ML) model are assessed.

[0014] In a twenty-seventh aspect, there is provided a method. The method may comprise: obtaining an operational condition of the terminal device; and determining, based on the operational condition, a size of a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are assessed.

[0015] In a twenty-eighth aspect, there is provided a method. The method may comprise: receiving, from a terminal device, an operational condition of the terminal device within a determined monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are assessed.

[0016] In a twenty-ninth aspect, there is provided an apparatus. The apparatus may comprise: means for obtaining an operational condition of the terminal device; and means for determining, based on the operational condition, a size of a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are assessed.

[0017] In a thirtieth aspect, there is provided an apparatus. The apparatus may comprise means for receiving, from a terminal device, an operational condition of the terminal device within a determined monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are assessed.

[0018] In a thirty-first aspect, there is provided a non-transitory computer readable medium comprising program instructions for causing an apparatus to perform at least the method according to twenty-seventh or twenty-eighth aspect.

[0019] In a thirty-second aspect, there is provided a computer program comprising instructions, which, when executed by an apparatus, cause the apparatus at least to perform at least the method according to twenty-seventh or twenty-eighth aspect.

[0020] In a thirty-third aspect, there is provided a terminal device. The terminal device may comprise: obtaining circuitry configured to obtain an operational condition of the terminal device; and determining circuitry configured to determine, based on the operational condition, a size of a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are assessed.

[0021] In a thirty-fourth aspect, there is provided a network device. The network device may comprise: receiving circuitry configured to receive, from a terminal device, an operational condition of the terminal device within a determined monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are assessed.

[0022] It is to be understood that the summary section is not intended to identify key or essential features of embodiments of the present disclosure, nor is it intended to be used to limit the scope of the present disclosure. Other features of the present disclosure will become easily comprehensible through the following description.BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Some exemplary embodiments will now be described with reference to the accompanying drawings, in which:

[0024] Fig. 1 illustrates an example of a network environment in which some exemplary embodiments of the present disclosure may be implemented;

[0025] Fig. 2 illustrates an example signaling process in accordance with some embodiments of the present disclosure;

[0026] Fig. 3 illustrates an example signaling process in accordance with some embodiments of the present disclosure;

[0027] Fig. 4 illustrates an example signaling process in accordance with some embodiments of the present disclosure;

[0028] Fig. 5 illustrates an example signaling process in accordance with some embodiments of the present disclosure;

[0029] Fig. 6 illustrates an example signaling process in accordance with some embodiments of the present disclosure;

[0030] Fig. 7A illustrates a flowchart of an example method implemented at a terminal device in accordance with some exemplary embodiments of the present disclosure;

[0031] Fig. 7B illustrates a flowchart of an example method implemented at a first network device in accordance with some exemplary embodiments of the present disclosure;

[0032] Fig. 7C illustrates a flowchart of an example method implemented at a second network device in accordance with some exemplary embodiments of the present disclosure;

[0033] Fig. 8A illustrates a flowchart of an example method implemented at a terminal device in accordance with some exemplary embodiments of the present disclosure;

[0034] Fig. 8B illustrates a flowchart of an example method implemented at a network device in accordance with some exemplary embodiments of the present disclosure;

[0035] Fig. 9A illustrates a flowchart of an example method implemented at a terminal device in accordance with some exemplary embodiments of the present disclosure;

[0036] Fig. 9B illustrates a flowchart of an example method implemented at a network device in accordance with some exemplary embodiments of the present disclosure;

[0037] Fig. 10 illustrates a simplified block diagram of a device that is suitable for implementing some exemplary embodiments of the present disclosure; and

[0038] Fig. 11 illustrates a block diagram of an example of a computer-readable medium in accordance with some exemplary embodiments of the present disclosure.

[0039] Throughout the drawings, the same or similar reference numerals represent the same or similar elements.DETAILED DESCRIPTION

[0040] Principles of the present disclosure will now be described with reference to some exemplary embodiments. It is to be understood that these embodiments are described only for the purpose of illustration and help those skilled in the art to understand and implement the present disclosure, without suggesting any limitation as to the scope of the disclosure. The disclosure described herein may be implemented in various manners other than the ones described below.

[0041] In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skills in the art to which this disclosure belongs.

[0042] References in the present disclosure to “one embodiment,” “an embodiment,” “an example embodiment,” and the like indicate that the embodiment described may include a particular feature, structure, or characteristic, but it is not necessary that every embodiment includes the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0043] It shall be understood that although the terms “first” and “second” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and similarly, a second element could be termed a first element, without departing from the scope of exemplary embodiments. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0044] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises”, “comprising”, “has”, “having”, “includes” and / or “including”, when used herein, specify the presence of stated features, elements, and / or components etc., but do not preclude the presence or addition of one or more other features, elements, components and / or combinations thereof. As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or”, mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements.

[0045] As used in this application, the term “circuitry” may refer to one or more or all of the following:(a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry) and(b) combinations of hardware circuits and software, such as (as applicable):(i) a combination of analog and / or digital hardware circuit(s) with software / firmware and (ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and(c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (for example, firmware) for operation, but the software may not be present when it is not needed for operation.

[0046] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

[0047] As used herein, the term “communication network” refers to a network following any suitable communication standards, such as Long Term Evolution (LTE), LTE-Advanced (LTE-A), Wideband Code Division Multiple Access (WCDMA), High-Speed Packet Access (HSPA), Narrow Band Internet of Things (NB-loT) and so on. Furthermore, the communications between a terminal device and a network device in the communication network may be performed according to any suitable generation communication protocols, including, but not limited to, the fourth generation (4G), 4.5G, the future fifth generation (5G) communication protocols, and / or any other protocols either currently known or to be developed in the future. Embodiments of the present disclosure may be applied in various communication systems. Given the rapid development in communications, there will of course also be future type communication technologies and systems with which the present disclosure may be embodied. It should not be seen as limiting the scope of the present disclosure to only the aforementioned system.

[0048] As used herein, the term “network device” (also referred to as “network node”) refers to a node in a communication network via which a terminal device accesses the network and receives services therefrom. The network device may refer to a base station (BS) or an access point (AP), for example, a node B (NodeB or NB), an evolved NodeB (eNodeB or eNB), a NR NB (also referred to as a gNB), a Remote Radio Unit (RRU), a radio header (RH), a remote radio head (RRH), a relay, a low power node such as a femto, a pico, and so forth, depending on the applied terminology and technology.

[0049] The term “terminal device” refers to any end device that may be capable of wireless communication. By way of example rather than limitation, a terminal device may also be referred to as a communication device, user equipment (UE), a Subscriber Station (SS), a Portable Subscriber Station, a Mobile Station (MS), or an Access Terminal (AT). The terminal device may include, but not limited to, a mobile phone, a cellularphone, a smart phone, voice over IP (VoIP) phones, wireless local loop phones, a tablet, a wearable terminal device, a personal digital assistant (PDA), portable computers, desktop computer, image capture terminal devices such as digital cameras, gaming terminal devices, music storage and playback appliances, vehiclemounted wireless terminal devices, wireless endpoints, mobile stations, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), USB dongles, smart devices, wireless customer-premises equipment (CPE), an Internet of Things (loT) device, a watch or other wearable, a head-mounted display (HMD), a vehicle, a drone, a medical device and applications (for example, remote surgery), an industrial device and applications (for example, a robot and / or other wireless devices operating in an industrial and / or an automated processing chain contexts), a consumer electronics device, a device operating on commercial and / or industrial wireless networks, and the like. In the following description, the terms “terminal device”, “communication device”, “terminal”, “user equipment” and “UE” may be used interchangeably.

[0050] The application of AI / ML techniques to NR air interface has been studied. The reliability of AI / ML models in beam management is critical, as failures can lead to degraded service quality. In other words, the reliability of AI / ML models in beam management is essential for maintaining service quality in 3GPP networks. Failures in these models can lead to significant performance degradation. Therefore, there is need for reliable AI / ML models in beam management within 3GPP networks and method for ensuring that model failures are detected promptly and accurately and allowing for seamless transitions to other methods to uphold service quality and minimize disruptions.

[0051] For illustrative purposes, principles and example embodiments of the present disclosure for improving AI / ML model for beam management will be described below with reference to Figs. 1-11. However, it is to be noted that these embodiments are given to enable the skilled in the art to understand concepts of the present disclosure and implement the solution as proposed herein, and not intended to limit scope of the present disclosure in any way.

[0052] Fig. 1 illustrates an example of a network environment 100 in which some exemplary embodiments of the present disclosure may be implemented. In the descriptions of the exemplary embodiments of the present disclosure, the network environment 100 may also be referred to as a communication system 100 (for example, a portion of a communication network). For illustrative purposes only, various aspects of exemplary embodiments will be described in the context of one or more terminal devices and network devices that communicate with one another. It should be appreciated, however, that the description herein may be applicable to other types of apparatus or other similar apparatuses that are referenced using other terminology.

[0053] The network device 102 may provide services to the terminal device 101, and the network device 102 and the terminal device 101 may communicate data and control information with each other. In some exemplary embodiments, the network device 102 and the terminal device 101 may communicate with direct links / channels. In the communication system 100, a link from the network device 102 to the terminal device101 is referred to as a downlink (DL), while a link from the terminal device 101 to the network device 102 is referred to as an uplink (UL). In downlink, the network device 102 is a transmitting (TX) device (or a transmitter) and the terminal device 101 is a receiving (RX) device (or a receiver). In uplink, the terminal device 101 is a transmitting (TX) device (or a transmitter) and the network device 102 is a RX device (or a receiver). As shown in Fig. 1, the network device 102 may also communicate with another element, for example, a network function entity 103.

[0054] Communications in the network environment 100 may be implemented according to any proper communication protocol(s), comprising, but not limited to, cellular communication protocols of the fourth generation (4G) and the fifth generation (5G) and on the like, wireless local network communication protocols such as Institute for Electrical and Electronics Engineers (IEEE) 802.11 and the like, and / or any other protocols currently known or to be developed in the future. Moreover, the communication may utilize any proper wireless communication technology, comprising but not limited to: Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Frequency Division Duplex (FDD), Time Division Duplex (TDD), Multiple-Input Multiple-Output (MIMO), Orthogonal Frequency Division Multiple (OFDM), Discrete Fourier Transform spread OFDM (DFT-s-OFDM) and / or any other technologies currently known or to be developed in the future.

[0055] It is to be understood that the number of devices and their connection relationships and types shown in Fig. 1 are for illustrative purposes only without suggesting any limitation. The communication system 100 may comprise any suitable number of devices adapted for implementing embodiments of the present disclosure.

[0056] The application of AI / ML techniques to NR air interface has been studied. The normative support for the general framework for AI / ML for air interface as well as enabling the recommended use cases has been discussed in the preceding study. In addition, a number of study objectives will tackle some outstanding issues identified during the study in an attempt to deepen the understanding in view of future normative work. For example, it needs to provide specification support for beam management - downlink transmitter (DL TX) beam prediction for both UE-sided model and NW-sided model from the following aspects: (1) spatial-domain DL Tx beam prediction for Set A of beams based on measurement results of Set B of beams (“BM-Case1”); (2) temporal DL Tx beam prediction for Set A of beams based on the historic measurement results of Set B of beams (“BM-Case2”); (3) specify necessary signalling / mechanism(s) to facilitate LCM operations specific to the Beam Management use cases, if any; and (4) enabling method(s) to ensure consistency between training and inference regarding NW-side additional conditions (if identified) for inference at UE. Accordingly, it strives for common framework design to support both BM-Case1 and BM-Case2.

[0057] For the technical specification group radio access network, there was study on AI / ML for NR air interface in Release 18. The problem statement emphasizes the need for reliable AI / ML models in beammanagement within 3GPP networks, highlighting the importance of fallback mechanisms when these models fail. There were some proposed solutions to reveal various approaches to address the challenges outlined in the problem statement. While the proposed solutions collectively address various aspects of the problem statement, there are notable gaps in specificity regarding the implementation of thresholds, monitoring mechanisms, and fallback strategies. The solutions vary in their strengths, with some providing robust frameworks for monitoring and switching, while others lack detail on practical execution.

[0058] In summary, the reliability of AI / ML models in beam management is critical, as failures can lead to degraded service quality. In other words, the reliability of AI / ML models in beam management is essential for maintaining service quality in 3GPP networks. Failures in these models can lead to significant performance degradation, necessitating the implementation of robust fallback mechanisms to transition to non-AI / ML methods when model performance falls below defined thresholds. Therefore, there is need for robust fallback mechanisms to switch to other methods for beam management when model performance is inadequate and a method for ensuring that model failures are detected promptly and accurately and allowing for seamless switching to uphold service quality and minimize disruptions. Future improvements may focus on a system that ensures seamless switching, particularly in dynamic environments.

[0059] In view of the foregoing, an example signaling process 200 in accordance with some embodiments of the present disclosure will be described with reference to Fig. 2. For the purpose of discussion, the communication process 200 will be described with reference to Fig. 1. It would be appreciated that although the communication process 200 has been described referring to the network environment 100 of Fig. 1, this communication process 200 may be likewise applied to other similar communication scenarios. 102103

[0060] As shown in Fig. 2, the terminal device 101 transmits (205), to a first network device 102, an operational condition of the terminal device within a monitoring window over which performance metrics of an AI / ML model are aggregated and assessed. The first network device 102 receives (210) the operational condition of the terminal device 101 within the monitoring window. The first network device 102 transmits (215) the operational condition along with the performance metrics of the AI / ML model to a second network device 103 within the monitoring window. The second network device 103 receives (220) the operational condition along with the performance metrics of the AI / ML model within the monitoring window.

[0061] Then, the second network device 103 determines (225), based on historical performance data of the AI / ML model and the operational condition, at least one threshold for assessing performance metrics of the AI / ML model. The second network device 103 determines (230), based on comparison between the performance metrics and the at least one threshold, whether the AI / ML model fails. Then, the second network device 103 transmits (235) to the first network device 102, a result about whether the AI / ML model fails, and the result is based on comparison between the performance metrics and at least one threshold. The first network device 102 receives (240) the result about whether the AI / ML model fails. Then, the first network device 102 transmits (245) to the terminal device 101 the result about whether the AI / ML model fails.Then, the terminal device 101 receives (250) from the first network device 102 the result about whether the AI / ML model fails.

[0062] By the process 200, an operation condition of the terminal device may be transmitted to the network device within a monitoring window over which performance metrics of the AI / ML model are aggregated and assessed. Instead of a single observation, the performance metrics of the AI / ML model are aggregated and assessed over the monitoring window. Thresholds for assessing the performance metrics of the AI / ML model may be dynamically determined by a network function entity (for example, the second network device 103) based on the current operation condition and historical performance data of the AI / ML model. Thus, instead of static threshold for model failure, the solution will implement a calibration mechanism that adjusts thresholds based on historical performance data and current operational conditions. For example, if the UE is in a high-mobility scenario, the threshold for switching to non-AI / ML methods may be set lower to account for the increased likelihood of model inaccuracies. Then, a result about whether the AI / ML model fails can be determined, for example, by the network function entity, based on the comparison between the dynamically determined threshold and the performance metrics. Then, the result may be transmitted to the terminal device.

[0063] Since the reliability of AI / ML models in beam management is critical, as failures can lead to degraded service quality, this process provides a method for ensuring that the UE can effectively validate model suitability based on multiple observations by including dynamically defining thresholds for model failures and monitoring performance over time. Then, a switch to other methods (for example, non-AI / ML method) may be made by the terminal device based on the resultthatthe AI / ML fails. Based on the prompt and accurate detection of model failure, the process 200 may provide robust fallback mechanisms to switch to other methods (for example, non-AI / ML method) when model performance is inadequate. The switching logic may be designed to allow for seamless transitions, where the UE can switch back to AI / ML methods once performance improves, thus ensuring minimal disruption in service quality.

[0064] In some embodiments, the operational condition comprises a mobility state; a configuration from the first network device; an environment type; or a network condition and the like. It should be appreciated that the operational condition can be selected according to the requirement.

[0065] Further, embodiments of the present disclosure may define a set of performance metrics, such as prediction accuracy, latency, and resource utilization, which will be monitored over a sliding window. Embodiments of the present disclosure may utilize a combination of statistical methods and machine learning techniques to continuously assess the performance of AI / ML models. Therefore, a comprehensive list of performance metrics will be monitored, including prediction accuracy, latency, and resource utilization. Establish baseline values for these metrics under various operational conditions.

[0066] In some embodiments, the performance metrics comprise prediction accuracy of the AI / ML model; a latency metric of the AI / ML model; a confidence score of the AI / ML model; an error rate of the AI / ML model;or resource utilization of the AI / ML model. In some embodiments, the performance metric further comprises a metric associated with beam coverage and capacity; a metric associated with radio resource utilization and efficiency; a metric associated with reliability and a quality of service; or a metric associated with complexity, power consumption and a hardware requirement, and any combination thereof.

[0067] In some embodiments, the metric associated with beam coverage and capacity comprises a beamforming gain which is a metric that quantifies the improvement in signal strength achieved by focusing the signal towards an intended terminal device; a coverage area which represents an area covered by a beam; a terminal device throughput which represents an average data rate achieved by terminal devices within the beam; spectral efficiency which represents an amount of data transmitted per unit of bandwidth; or cell edge performance which represents a performance of terminal devices located at an edge of the cell, and any combination thereof..

[0068] For example, the metrics for the coverage and the capacity comprise: beamforming gain, which is a metric that quantifies the improvement in signal strength achieved by focusing the signal towards the intended user and is typically measured in dB; coverage area, which is the area covered by the beam, which can be measured in square meters or kilometers; user throughput which is the average data rate achieved by users within the beam; spectral efficiency which is the amount of data transmitted per unit of bandwidth, measured in bits per second per Hertz (bps / Hz); and cell edge performance which is the performance of users located at the edge of the cell, which is often a challenging area for signal reception, and any combination thereof..

[0069] In some embodiments, the metric associated with radio resource utilization and efficiency comprises a beam switching speed for switching between different beams; a beam overlap which represents an extent to which beams overlap; resource allocation efficiency which represents an effectiveness of allocating resources to different beams; or beam tracking accuracy which represents ability of a system to accurately track a location of the terminal device and adjust a beam, and any combination thereof.

[0070] For example, the metrics for the resource utilization and efficiency comprises: beam switching speed which is the time it takes to switch between different beams, which is important for supporting user mobility and dynamic channel conditions; beam overlap which is the extent to which beams overlap, which can lead to interference and reduced efficiency; resource allocation efficiency which is the effectiveness of allocating resources (e.g., power, bandwidth) to different beams; or beam tracking accuracy which is the ability of the system to accurately track the user's location and adjust the beam accordingly, and any combination thereof..

[0071] In some embodiments, the metric associated with reliability and quality of service comprises a block error rate which represents a rate of errors in data transmission; a latency which represents a delay experienced by data packets; a packet loss rate which represents a percentage of data packets that are lost during transmission; or a terminal device experience which represents subjective measures of the terminaldevice satisfaction with the quality of service, and any combination thereof.

[0072] For example, the metrics for the reliability and quality of service comprises: block error rate (BLER) which is the rate of errors in data transmission, which is a measure of the reliability of the link; latency, which is the delay experienced by data packets, which is important for real-time applications; packet loss rate which is the percentage of data packets that are lost during transmission; or user experience which is subjective measures of user satisfaction with the quality of service, such as perceived signal strength and data rate.

[0073] I n some embodiments, the metrics associated with complexity, power consumption and a hardware requirement comprise complexity which represents computational complexity of a beam management algorithm; power consumption which represents an amount of power consumed by a beamforming system; or a hardware requirement which represents hardware resources needed to support beam management, and any combination thereof.

[0074] For example, other metrics comprises complexity which is the computational complexity of the beam management algorithms, which can impact the implementation cost and energy consumption; power consumption which is the amount of power consumed by the beamforming system; and hardware requirements, which is hardware resources needed to support beam management, such as the number of antennas and processing units.

[0075] In some embodiments, in order to obtain the performance metric of the AI / ML model, the first network device transmits, to an entity including the AI / ML model, a request for beam prediction and performance data of the AI / ML model and the received operational condition within the monitoring window; and receive, from the entity including the AI / ML model, a result of the beam prediction and the performance metric of the AI / ML model within the monitoring window.

[0076] In some embodiments, in order to assess the performance metric of the AI / ML model, the first network device transmits, to a threshold logic entity, the received performance metrics of the AI / ML model that are to be compared with the at least one threshold at the threshold logic entity within the monitoring window. If the performance metric of the AI / ML model is below the threshold, the second network device 103 (for example, threshold logic entity) transmits, to the first network device, an indication to continue AI / ML-based beam management, and first network device 102 receives from for example the threshold logic entity, an indication to continue AI / ML-based beam management; and transmits, to the terminal device 101, an AI / ML-based beam reporting instruction. Therefore, if the AI / ML model does not fail, the terminal device 101 receives the result by receiving an AI / ML-based beam reporting instruction.

[0077] If the performance metric of the AI / ML model is above the threshold, the second network device 103 (for example, threshold logic entity) transmits, to a fallback mechanism entity, a request for initiating fallback process, and the first network device 102 receives, from the fallback mechanism entity, an indication of fallback-based beam reporting; and transmits, to the terminal device 101 , a fallback-based beam reporting instruction. Therefore, if the AI / ML model fails, the terminal device 101 receives the result by receiving afallback-based beam reporting instruction.

[0078] As mentioned above, the threshold for assessing the performance metrics of the AI / ML model may be dynamically determined by a network function entity (for example, the threshold logic entity) based on the current operation condition and historical performance data of the AI / ML model, embodiments of the present disclosure may design a calibration process that utilizes historical performance data to set and adjust thresholds for switching between AI / ML and non-AI / ML methods. This may include simulations to test the effectiveness of different threshold settings under varying conditions.

[0079] In some embodiments, the at least one threshold comprises a composite threshold having a weighted sum of items of the initial performance metric included in the operational condition. In some embodiments, a first threshold for a first operational condition is different from a second threshold for a second operational condition.

[0080] For example, based on the initial performance metrics included in the operational condition, a threshold needs to be calibrated to optimize the switching between AI / ML method and the traditional methods. The threshold may include a weighted metric for key selected metrics in the operational condition. For example, instead of relying on a single metric like accuracy, some embodiments of the present disclosure define a composite threshold (e.g., weighted sum or weighted average) that captures the relative importance of accuracy, latency, and resource use as shown below:threshold = w1 xaccuracy+w2x|atency+w3xresourceUsage\text{Threshold} = w_1 \times \text{accuracy} +w_2 \times \text{latency} +w_3\times\text{resourceUsage}.

[0081] Threshold=w1 xaccuracy+w2xlatency+w3xresourceUsage,with w1+w2+w3=1w_1 +w_2 +w_3 = 1 w1 +w2+w3=1.

[0082] Further, a hysteresis may be included in the threshold and calibration to avoid over-frequent switching. Frequent switching between AI / ML and non-AI / ML methods can cause additional signaling overhead and potential performance dips (e.g., re-initializing beam selection).

[0083] For example, some embodiments of the present disclosure may introduce separate entry and exit thresholds (or use a threshold plus a margin) to avoid oscillations. Therefore, in some embodiments, the at least one threshold comprises: a threshold value and a margin value; or an entry threshold and an exit threshold of a threshold range. For example, the terminal device switches to fallback if composite metric falls below T1T_1T1; and the terminal device switches back to AI / ML if composite metric rises above T2T.2T2 (T2>T1T_2 > T_1T2>T1 ).

[0084] Further, in some embodiments of the present disclosure, in scenarios where AI / ML models are unavailable due to resource constraints or power-saving measures, a fallback reporting mechanism may be defined to utilize predefined heuristics based on historical beam patterns and environmental context. This mechanism will guide the UE in selecting the most appropriate beams to report, ensuring that even in the absence of AI / ML predictions, the UE can maintain effective communication with the gNB. For example, itmay develop a set of heuristics for beam reporting that can be employed when AI / ML models are unavailable. This may involve analyzing historical beam patterns and environmental factors to guide the UE in selecting the most effective beams.

[0085] In some embodiments, the at least one historical beam usage pattern comprises one or more beams that are selected for a corresponding operational condition; and the at least one environmental factor comprises (i) measurements from neighboring network devices or beams for refining fallback selection, or (ii) factors in obstacles or known channel impairments.

[0086] For example, the historical beam usage patterns may be e.g., which beams performed best in similar locations or mobility states. The environmental context incorporates measurements such as Received Signal Strength Indicator (RSSI), Signal to Interference plus Noise Ratio (SINR), or Reference Signal Time Difference (RSTD) from neighboring beams or gNBs to refine fallback choices; and incorporates factor in obstacles or known channel impairments if such data is available.

[0087] In some embodiments, the set of heuristics for beam reporting comprises ranking beams based on recent average signal strength; rotating through candidate beams that historically have performance over a corresponding threshold; using a rule-based system that includes context variables; or any combination thereof.

[0088] For example, the heuristics may comprise simple heuristics and enhance heuristics. The simple heuristics comprises ranking beams by their most recent average signal strength, or rotating through candidate beams that historically had good performance. The advantage of the simple heuristics may be low overhead and easy to implement. The enhanced heuristics may use a lightweight, possibly rule-based system that includes context variables such as speed, environment type (urban vs. rural), or historical interference levels. The advantage of the enhanced heuristics comprises more accurate fallback, but slightly higher implementation complexity.

[0089] The role of UE in the fallback may comprises UE autonomy and the signaling from the network. If the role of the UE in the fallback is that UE autonomy, the terminal device may store at least one historical beam usage pattern of a set of heuristics for beam reporting that are predefined based on the at least one historical beam usage pattern and at least one environmental factor; based on determining that the AI / ML model fails, autonomously switch to the fallback-based beam reporting; and select an effective beam based on the at least one stored historical beam usage pattern. For example, the UE can store historical beam usage patterns (e.g., which beams performed best in similar locations or mobility states). When AI / ML predictions are unavailable or unreliable, this historical insight guides the next best beam selection. In this scenario, the UE may store minimal data (e.g., beam ID, performance) to promptly switch to fallback in case the AI / ML model fails.

[0090] If the role of the UE in the fallback is getting signaling from the network, the NW can confirm or override the UE’s fallback heuristics based on broader network intelligence (e.g., sector loading, neighborgNB beam states). In this scenario, the terminal device may receive, from the first network device, an instruction to switch to fallback-based beam reporting, the predefined set of heuristics for beam reporting are confirmed or overridden by the first network device based on network intelligence. Then, the terminal device switches to the fallback-based beam reporting based on the instruction.

[0091] Therefore, some embodiments of the present disclosure establish a comprehensive framework for monitoring AI / ML model performance, including defining failure thresholds and implementing fallback strategies to non-AI / ML methods when necessary. This comprehensive framework may involve dynamic switching capabilities based on operational conditions and a mechanism for reporting beam predictions when AI / ML models are unavailable. To address the challenges regarding the reliability of AI / ML models in beam management within 3GPP networks, embodiments of the present disclosure provide a solution that integrates a multi-tiered performance monitoring and dynamic fallback mechanism. This solution builds upon the areas of threshold determination, adaptability to varying operational conditions, and seamless transitions between AI / ML and traditional methods.

[0092] Hereinafter, the exemplary signaling process according to the embodiment of Fig. 2 will be described with reference to Fig. 3. It should be appreciated that the UE is an example of the terminal device 101, the NW is an example of the network device 102, and threshold logic entity and the fallback mechanism entity may be examples of the network function entity 103. The ML model entity may be deployed at the UE side or at the NW side.

[0093] As shown in Fig. 3, at step 1, the UE transmits its operational context (e.g., mobility state, serving gNB configuration) to the NW. It will be appreciated that the operational context may be adapted according to the actual requirement. As shown in Fig. 3, at step 2, the NW requests a beam prediction (and any associated performance metrics) from an MLModel entity. At step 3, the MLModel entity returns the predicted beams along with performance metrics (accuracy, confidence scores, latency, etc.) to the NW. At step 4, the NW forwards these performance metrics to the ThresholdLogic entity to evaluate whether the performance metrics meet required criteria. At step 5, A decision is made on whether the metrics are above or below the defined threshold(s) at the threshold logic entity.

[0094] If the threshold is not exceeded, the NW continues to rely on AI / ML-based beam management. If the threshold is exceeded, indicating possible model performance degradation, the fallback mechanism is triggered. If the threshold is not exceeded, that is in the normal case (threshold not exceeded), at step 6a, the NW sends AI / ML-based beam reporting instructions to the UE. If the threshold is exceeded, the FallbackMechanism entity (or module) is activated at step 6b. The FallbackMechanism entity provides fallback reporting instructions to the NW at step 7, which are then delivered to the UE at step 8. Then, the UE updates its beam management approach— either continuing with AI / ML-based predictions or switching to fallback heuristics— based on the instructions received from the NW.

[0095] Through the process 300, by focusing on the UE and NW interactions (Steps 1, 6a, 6b / 8), it mayreport the UE’s initial context reporting; the network entities may make decision to continue using AI / ML or switch to fallback; and a final instruction may be provided to UE to follow to manage its beam selection.

[0096] The reliability of AI / ML models in beam management is essential for maintaining service quality in 3GPP networks. Failures in these models can lead to significant performance degradation, necessitating the implementation of robust fallback mechanisms to transition to non-AI / ML methods when model performance falls below defined thresholds. By the process, clear performance monitoring metrics may be established to account for multiple observations overtime to accurately assess model validity. The process 300 may further include dynamically defining a failure threshold based on cumulative performance data, ensuring that the UE can dynamically switch between AI / ML-based beam prediction and traditional methods in response to changing operational conditions, such as variations in UE speed or transitions between different gNBs with distinct downlink transmission beam configurations. Additionally, a fallback mechanism is proposed to guide the UE in beam reporting when AI / ML models are unavailable due to resource constraints or power-saving measures. Overall, embodiments of the present disclosure may ensure that model failures are detected promptly and accurately, allowing for seamless transitions to non-AI / ML methods to uphold service quality and minimize disruptions.

[0097] Embodiments of the present disclosure, compared to conventional solutions, are the integration of adaptive monitoring, dynamic threshold calibration, and a robust fallback reporting mechanism that collectively enhance the reliability and responsiveness of beam management in 3GPP networks. This holistic approach addresses the limitations of existing solutions by ensuring that performance assessments are context-aware and that transitions between AI / ML and traditional methods are seamless and efficient.

[0098] An example signaling process 400 in accordance with some embodiments of the present disclosure will be described with reference to Fig. 4. For the purpose of discussion, the communication process 400 will be described with reference to Fig. 1. It would be appreciated that although the communication process 400 has been described referring to the network environment 100 of Fig. 1, this communication process 400 may be likewise applied to other similar communication scenarios.

[0099] As shown in Fig. 4, the terminal device 101 transmits (405) to a network device 102, an operational condition of the terminal device within a monitoring window over which an anomaly detection of an active artificial i ntelligence / machi ne learning (AI / ML) model is performed. The network device 102 receives (410) the operational condition, and the operational condition may be used for requesting the performance metrics of the active AI / ML model. Then, the network device 102 transmits (415) to the terminal device 102 a result of the anomaly detection, and the result is based on comparison between performance metrics of the AI / ML model and historical data. The terminal device 102 receives (420) the result of the anomaly detection.[OO1OO] In the process 400, the operational condition for requesting the performance metric of the active AI / ML model is transmitted to the network device within a monitoring window, and an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed over the monitoring window.Then, a result of the anomaly detection can be determined over time based on the comparison between performance metrics of the AI / ML model and historical data. Based on the accurate and prompt anomaly detection, the terminal device may switch to a new model or continue using the active AI / ML model.

[0101] Therefore, to ensure optimal performance in beam management within the 3GPP framework, it is critical to implement a comprehensive monitoring and validation system for AI / ML models used in this context. A detection mechanism can be established by the process 400 to determine when a UE-sided model or functionality is no longer valid, particularly in scenarios where the UE transitions between gNBs with differing downlink transmission beam configurations. The process 400 may accurately detect model failures by analyzing performance metrics of the AI / ML model overtime (over a monitoring window), rather than relying on single observations, which can lead to erroneous conclusions due to the inherent variability in wireless communication environments. Thus, the process may mitigate unnecessary service disruptions and enhance the reliability of beam prediction, ultimately maintaining high service quality in dynamic network conditions.

[0102] Therefore, to address the problem statement regarding optimal performance in beam management within the 3GPP framework, this process integrates a dynamic, context-aware monitoring (for example, the operational condition) and validation system for AI / ML models. This system will leverage a combination of temporal performance analysis and contextual awareness of network conditions to ensure robust beam management, particularly during transitions between gNBs with varying downlink transmission beam configurations.

[0103] In some embodiments, if the anomaly of the active AI / ML model is not detected, the terminal device may receive the result by receiving an instruction to continue using the active AI / ML model. If the anomaly of the active AI / ML model is detected, the terminal device may receive the result by receiving an instruction to activate an inactive model that is identified to be suitable for beam management. Therefore, there is also a robust process for monitoring inactive models to assess whether conditions for their activation are met, ensuring that the most suitable model is selected for switching when performance degradation is detected in the currently active model. In addition to the temporal performance analysis and contextual awareness of network conditions, this system may further leverage adaptive model selection mechanisms to ensure robust beam management.

[0104] In some embodiments, the terminal device may adjust a size of the monitoring window based on historical performance data of the active AI / ML model and the operational condition of the terminal device. These embodiments involve a continuous performance assessment of the active model using a rolling window mechanism that can adapt based on real-time network conditions. For example, it may utilize a combination of historical performance metrics (e.g., accuracy, latency, and error rates) and contextual data (e.g., signal strength, user mobility patterns, and gNB configurations) to dynamically adjust the monitoring window. This adaptability addresses the limitations of struggling with rapidly changing conditions due to itsstatic monitoring window. It should be noted that the size of the monitoring window may also be adjusted by the network device based on the historical performance data of the active AI / ML model and the operational condition of the terminal device.

[0105] In order to obtain the performance metric of the active AI / ML model, the network device may transmit, to a first entity including the active AI / ML model, a request for current performance data of the active AI / ML model within the monitoring window; and receive, from the first entity, a current performance metric of the active AI / ML model within the monitoring window.

[0106] In order to obtain the result of anomaly detection, the network device may transmit, to an anomaly detection entity, the received current performance metric that is to be compared with historical data at the anomaly detection entity to detect the anomaly of the active AI / ML model; and receive, from the anomaly detection entity, an output of the anomaly detection entity representing result of the anomaly detection.

[0107] For example, to detect model failures, the anomaly detection entity may employ a machine learningbased anomaly detection algorithm that analyzes the performance metrics over time, rather than relying solely on predefined thresholds. This algorithm will learn from historical data to identify patterns indicative of model degradation, allowing for more nuanced and timely responses to performance issues.

[0108] In addition to anomaly detection of the active AI / ML model, some embodiments of the present disclosure may also monitor an inactive model. In order to manage the inactive model, the network device may transmit, to a second entity including an inactive AI / ML model, a request for proactively monitoring current performance data of the inactive AI / ML model within the monitoring window; and receive, from the second entity, a readiness data of the inactive AI / ML model within the monitoring window, and the readiness data represents whether the inactive AI / ML model has sufficient recent training data for the operational condition.

[0109] For example, the system may implement a proactive monitoring strategy that assesses the performance of inactive models based on a set of predefined criteria, such as environmental conditions and user behavior patterns. By utilizing a lightweight data exchange protocol, the network device can receive real-time updates regarding the performance of inactive models without impacting the operation of the active model. This approach builds on non-activation monitoring and ensures timely and efficient dataset delivery through a prioritized data queuing system that considers the urgency of model activation based on current network conditions.

[0110] In order to determine whether to continue use the active AI / ML model or switch to the inactive model, the network device may determine, based on the output of the anomaly detection entity and the readiness data of the inactive AI / ML model, whether the terminal device is to continue using the active AI / ML model or switch to the inactive model. Then, the network device may transmit, to the terminal device, an instruction to continue using the active model based on determining that the anomaly of the active model is not detected or determining that the inactive model is not suitable for beam management; or may transmit,to the terminal device, an instruction to switch to the inactive model based on determining that the anomaly of the active model is detected or determining that the inactive model is identified to be suitable for beam management.

[0111] Hereinafter, the exemplary signaling process 500 according to the embodiment of Fig. 4 will be described with reference to Fig. 5. It should be appreciated that the UE is an example of the terminal device 101, the NW is an example of the network device 102, and anomaly detection entity may be examples of the network function entity 103. The active and inactive ML model entity may be located at the UE side or at the NW side, and preferably at the UE side.

[0112] As shown in Fig. 5, at step 1, the UE sends its operational context (e.g., mobility state, gNB configuration) and initial performance metrics to the NW. At step 2, the NW queries an ActiveModel entity for updated performance data (e.g., accuracy, latency) within a rolling window. This rolling window duration is dynamically adjusted based on network conditions (for example, mobility state of the UE, signal quality, etc.). At step 3, the ActiveModel entity returns its current performance metrics over that rolling window to the NW. At step 4a, the NW forwards these metrics to the AnomalyDetection module, and the AnomalyDetection module compares them against historical data to detect potential degradation patterns, rather than relying on single-shot threshold checks, and then notifies the NW if performance degradation is detected.

[0113] At step 5, in parallel, the NW proactively monitors one or more InactiveModel entities within the rolling window. By using a lightweight data exchange protocol, the NW obtains real-time updates about readiness and suitability of the inactive model(s), ensuring that UE-side transitions can be triggered smoothly if the active model degrades. At step 6, the InactiveModel entities respond with readiness data (e.g., whether it has sufficient recent training data for the current environment).

[0114] At step 7, the NW processes the AnomalyDetection output and the InactiveModel readiness data to decide whether the UE is to continue using the active model or switch to an inactive model. Then, if performance is acceptable and no better model is found, the NW directs the UE to continue using the active model. Otherwise, if the AnomalyDetection module flags a significant performance drop or if an inactive model is identified as more suitable, the NW instructs the UE to activate the inactive model for beam management. Then, the UE updates its beam management strategy accordingly, seamlessly switching models where necessary, thus maintaining optimal service quality.

[0115] The process 500 may leverage a combination of temporal performance analysis, contextual awareness of network conditions, and adaptive model selection mechanisms to ensure robust beam management, particularly during transitions between gNBs with varying downlink transmission beam configurations. This dual-layer (active + inactive) monitoring approach highlights how the UE and NW collaborate to maintain robust beam management, using time-based anomaly detection and proactive evaluation of inactive models for fast switching. This approach will mitigate unnecessary service disruptionsand enhance the reliability of beam prediction, ultimately maintaining high service quality in dynamic network conditions.

[0116] An example signaling process 600 in accordance with some embodiments of the present disclosure will be described with reference to Fig. 6. For the purpose of discussion, the communication process 600 will be described with reference to Fig. 1. It would be appreciated that although the communication process 600 has been described referring to the network environment 100 of Fig. 1, this communication process 600 may be likewise applied to other similar communication scenarios.

[0117] As shown in Fig. 6, the terminal device 101 obtains (605) an operation condition of the terminal device. The terminal device 101 determines (610) based on the operational condition, a size of a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are assessed. The terminal device 101 transmits (615) to the network device 102 the operational condition within the determined monitoring window. The network device 102 receives (620) from the terminal device the operational condition of the terminal device within the monitoring window.

[0118] By the process 600, the monitoring window size will be dynamically adjusted based on at least the UE's operational conditions (e.g., speed, gNB configuration). For instance, a shorter window may be used when the UE is moving rapidly between gNBs, while a longer window may be appropriate in stable conditions. This adaptive approach ensures that the monitoring is responsive to real-time changes in the network environment. It may dynamically adjust the monitoring window size based on operational condition of the UE, for example, UE speed and gNB configuration. By dynamic adjustment of the monitoring window, the process may have dynamic adaptability in varying conditions, and thus may have adaptive monitoring for the performance metric of the AI / ML model over the dynamically determined monitoring windows.

[0119] For example, the UE may implement the monitoring window based on the estimated UE speed and the gNB configuration, e.g. number of beams, beam widths, etc. That is to say, the monitoring window can be dynamically adjusted based on the operational condition of the terminal device.

[0120] A window in time (or over a certain number of beam-management cycles) may be period over which performance metrics are aggregated and assessed. Rather than using a fixed-size window, the algorithm adjusts window length according to real-time UE and network parameters. For example, the UE Speed and gNB Configuration may be used as Key Inputs. The UE can estimate its own speed (e.g., via GPS data, Doppler shift, or IMU sensors) and factor in the gNB beam configuration (number of beams, beam width, etc.) to resize the monitoring window. For the high speed, it may have a shorter window, for example, rapid channel changes require more frequent performance checks to capture potential AI / ML degradation quickly. For the low speed or stationary state, it may have a longer window, for example, less frequent changes in channel conditions allow for more extended observation intervals, reducing overhead.

[0121] For example, in some embodiments, the UE may periodically estimate a speed of the terminal device based on the mobility state; periodically estimate a factor indicated in the configuration from thenetwork device; and periodically resize the size of the monitoring window based on the estimated speed and the estimated factor. For example, the UE periodically re-evaluates conditions and may shift to a smaller or larger window if it detects a change in speed, network load, or beam availability. For example, the UE may evaluate the conditions before transmitting the operational condition to the network device at step 1 of Fig. 3 or at step 1 of Fig. 5, or the UE may evaluate the condition after receiving the instruction of the continuing AI / ML-based beam reporting at step 6a of Fig. 3 which may comprises the configuration for UE and network parameter, for example, UE speed, network load, and beam availability.

[0122] Further, this may involve using machine learning techniques to analyze historical performance data and predict optimal window sizes, and it may leverage historical performance logs to train a model (e.g., regression or reinforcement learning) that predicts an optimal window size under certain conditions (speed ranges, environment types). For example, in some embodiments, the terminal device may determine the size of the monitoring window by: training a model by using historical performance data of the AI / ML model and historical operational condition of the terminal device; and predicting a window size based on the obtained operational condition of the terminal device by using the trained model.

[0123] For the process 300 as shown in Fig.3, the monitoring window may be a sliding window, and the window size may be dynamically adjusted by the terminal device or the network device based on the UE's operational conditions (e.g., speed, gNB configuration) and the historical performance logs. For the process 500 as shown in Fig. 5, the monitoring window may be a rolling window, and a combination of historical performance metrics (e.g., accuracy, latency, and error rates) and contextual data (e.g., signal strength, user mobility patterns, and gNB configurations) may be utilized by the terminal device or network device to dynamically adjust the monitoring window.

[0124] Fig. 7A illustrates another flowchart of an example method 700A implemented at a terminal device in accordance with some other embodiments of the present disclosure. For the purpose of discussion, the method 700A will be described from the perspective of the terminal device 101 with reference to Fig. 1.

[0125] At block 71 OA, the terminal device transmits, to a first network device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed. At block 720A, the terminal device receives, from the first network device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

[0126] In some embodiments, the AI / ML model does not fail, and the terminal device receives the result by receiving an AI / ML-based beam reporting instruction; or the AI / ML model fails, and the terminal device receives the result by receiving a fallback-based beam reporting instruction.

[0127] In some embodiments, the operational condition comprises an initial performance metric; and the at least one threshold comprises a composite threshold having a weighted sum of items of the initialperformance metric included in the operational condition. In some embodiments, the at least one threshold comprises: a threshold value and a margin value; or an entry threshold and an exit threshold of a threshold range. In some embodiments, a first threshold for a first operational condition is different from a second threshold for a second operational condition.

[0128] In some embodiments, the terminal device may further store at least one historical beam usage pattern of a set of heuristics for beam reporting that are predefined based on the at least one historical beam usage pattern and at least one environmental factor; based on determining that the AI / ML model fails, autonomously switch to the fallback-based beam reporting; and select an effective beam based on the at least one stored historical beam usage pattern. In some embodiments, the at least one historical beam usage pattern comprises one or more beams that are selected for a corresponding operational condition; and the at least one environmental factor comprises (i) measurements from neighboring network devices or beams for refining fallback selection, or (ii) factors in obstacles or known channel impairments.

[0129] In some embodiments, the set of heuristics for beam reporting comprises at least one of the following: ranking beams based on recent average signal strength; rotating through candidate beams that historically have performance over a corresponding threshold; or using a rule-based system that includes context variables. In some embodiments, the context variable comprises at least one of the following: a speed of the terminal device, an environment type of the terminal device, or a historical interference level.

[0130] In some embodiments, the terminal device may further receive, from the first network device, an instruction to switch to fallback-based beam reporting, wherein a set of heuristics for beam reporting, that are predefined based on at least one historical beam usage pattern and at least one environmental factor, are confirmed or overridden by the first network device based on network intelligence; and switch to the fallbackbased beam reporting based on the instruction. In some embodiments, the terminal device may further adjust the sliding window based on the operational condition.

[0131] In some embodiments, the operational condition comprises at least one of the following: a mobility state; a configuration from the first network device; an environment type; or a network condition. In some embodiments, the performance metrics comprise at least one of the following: prediction accuracy of the AI / ML model; a latency metric of the AI / ML model; a confidence score of the AI / ML model; an error rate of the AI / ML model; or resource utilization of the AI / ML model.

[0132] In some embodiments, the performance metric further comprises at least one of the following: a metric associated with beam coverage and capacity; a metric associated with radio resource utilization and efficiency; a metric associated with reliability and a quality of service; or a metric associated with complexity, power consumption and a hardware requirement.

[0133] In some embodiments, the metric associated with beam coverage and capacity comprises at least one of the following: a beamforming gain which is a metric that quantifies the improvement in signal strength achieved by focusing the signal towards an intended terminal device; a coverage area which represents anarea covered by a beam; a terminal device throughput which represents an average data rate achieved by terminal devices within the beam; spectral efficiency which represents an amount of data transmitted per unit of bandwidth; or cell edge performance which represents a performance of terminal devices located at an edge of the cell.

[0134] In some embodiments, the metric associated with radio resource utilization and efficiency comprises at least one of the following: a beam switching speed for switching between different beams; a beam overlap which represents an extent to which beams overlap; resource allocation efficiency which represents an effectiveness of allocating resources to different beams; or beam tracking accuracy which represents ability of a system to accurately track a location of the terminal device and adjust a beam.

[0135] In some embodiments, the metric associated with reliability and quality of service comprises at least one of the following: a block error rate which represents a rate of errors in data transmission; a latency which represents a delay experienced by data packets; a packet loss rate which represents a percentage of data packets that are lost during transmission; or a terminal device experience which represents subjective measures of the terminal device satisfaction with the quality of service.

[0136] In some embodiments, the metric associated with complexity, power consumption and a hardware requirement comprises at least one of the following: complexity which represents computational complexity of a beam management algorithm; power consumption which represents an amount of power consumed by a beamforming system; or a hardware requirement which represents hardware resources needed to support beam management.

[0137] Fig. 7B illustrates another flowchart of an example method 700B implemented at a first network device in accordance with some other embodiments of the present disclosure. For the purpose of discussion, the method 700B will be described from the perspective of the first network device 102 with reference to Fig. 1.

[0138] As shown in Fig. 7B, at block 71 OB, the first network device receives, from a terminal device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed. At block 720B, the first network device transmits, to the terminal device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

[0139] In some embodiments, the first network device may further transmit, to an entity including the AI / ML model, a request for beam prediction and performance data of the AI / ML model and the received operational condition within the monitoring window; and receive, from the entity including the AI / ML model, a result of the beam prediction and the performance metric of the AI / ML model within the monitoring window.

[0140] In some embodiments, the first network device may further transmit, to a threshold logic entity, the received performance metrics of the AI / ML model that are to be compared with the at least one threshold at the threshold logic entity within the monitoring window.

[0141] In some embodiments, the performance metric of the AI / ML model is below the threshold, and the first network device may further receive, from the threshold logic entity, an indication to continue AI / ML-based beam management; and transmit, to the terminal device, an AI / ML-based beam reporting instruction. In some embodiments, the performance metric of the AI / ML model is above the threshold, and the first network device may further receive, from a fallback mechanism entity, an indication of fallback-based beam reporting; and transmit, to the terminal device, a fallback-based beam reporting instruction.

[0142] Fig. 7C illustrates another flowchart of an example method 700C implemented at a second network device in accordance with some other embodiments of the present disclosure. For the purpose of discussion, the method 700C will be described from the perspective of the network functional entity 103 with reference to Fig. 1.

[0143] As shown in Fig. 7C, at block 71 OC, the second network device receives from a first network device, an operational condition of a terminal device and performance metrics of an artificial intelligence / machine learning (AI / ML) within a monitoring window over which performance metrics of the AI / ML model are aggregated and assessed. At block 720C, the second network device determines, based on historical performance data of the AI / ML model and the operational condition, at least one threshold for assessing performance metrics of the AI / ML model. At block 730C, the second network device determines, based on comparison between the performance metrics and the at least one threshold, whether the AI / ML model fails. At block 740C, the second network device transmits, to the first network device, a result about whether the AI / ML model fails.

[0144] In some embodiments, based on determining that the performance metrics do not exceed the at least one determined threshold, the second network device transmits, to the first network device, an indication to continue AI / ML-based beam management. In some embodiments, based on determining that the performance metrics exceed the at least one determined threshold, the second network device transmits, to a fallback mechanism entity, a request for initiating fallback process.

[0145] Fig. 8A illustrates another flowchart of an example method 800A implemented at a terminal device in accordance with some other embodiments of the present disclosure. For the purpose of discussion, the method 800A will be described from the perspective of the terminal device 101 with reference to Fig. 1.

[0146] At block 81 OA, the terminal device transmits, to a network device, an operational condition of the terminal device within a monitoring window over which an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed. At block 820A, the terminal device receives a result of the anomaly from the network device, wherein the result is based on comparison between performance metrics of the AI / ML model and historical data.

[0147] In some embodiments, the anomaly of the active AI / ML model is not detected, and the terminal device receives the result by receiving an instruction to continue using the active AI / ML model. In some embodiments, the anomaly of the active AI / ML model is detected, and the terminal device receives the result by receiving an instruction to activate an inactive model that is identified to be suitable for beam management.

[0148] In some embodiments, the terminal device may further adjust a size of the monitoring window based on historical performance data of the active AI / ML model and the operational condition of the terminal device.

[0149] In some embodiments, the operational condition comprises at least one of the following: a mobility state; a configuration from the first network device; an environment type; or a network condition. In some embodiments, the performance metrics comprise at least one of the following: prediction accuracy of the AI / ML model; a latency metric of the AI / ML model; a confidence score of the AI / ML model; an error rate of the AI / ML model; or resource utilization of the AI / ML model. In some embodiments, the performance metrics further comprise at least one of the following: a metric associated with beam coverage and capacity; a metric associated with radio resource utilization and efficiency; a metric associated with reliability and a quality of service; or a metric associated with complexity, power consumption and a hardware requirement. In some embodiments, the AI / ML model is deployed at the terminal device.

[0150] Fig. 8B illustrates another flowchart of an example method 800B implemented at a network device in accordance with some other embodiments of the present disclosure. For the purpose of discussion, the method 800B will be described from the perspective of the first network device 102 with reference to Fig. 1.

[0151] As block 81 OB, the network device receives, from a terminal device, an operational condition of the terminal device within a monitoring window over which an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed. At block 820B, the network device transmits a result of the anomaly detection to the terminal device, wherein the result is based on comparison between performance metrics of the AI / ML model and historical data.

[0152] In some embodiments, the network device may further transmit, to a first entity including the active AI / ML model, a request for current performance data of the active AI / ML model within the monitoring window; and receive, from the first entity, a current performance metric of the active AI / ML model within the monitoring window. In some embodiments, the network device may further: transmit, to an anomaly detection entity, the received current performance metric that is to be compared with historical data at the anomaly detection entity to detect the anomaly of the active AI / ML model; and receive, from the anomaly detection entity, an output of the anomaly detection entity representing result of the anomaly detection.

[0153] In some embodiments, the network device may further transmit, to a second entity including a n inactive AI / ML model, a request for proactively monitoring current performance data of the inactive AI / ML model within the monitoring window; and receive, from the second entity, a readiness data of the inactiveAI / ML model within the monitoring window, wherein the readiness data represents whether the inactive AI / ML model has sufficient recent training data for the operational condition.

[0154] In some embodiments, the request is transmitted by using a lightweight data exchange protocol; and the readiness data is received by using the lightweight data exchange protocol.

[0155] In some embodiments, the network device may further determine, based on the output of the anomaly detection entity and the readiness data of the inactive AI / ML model, whether the terminal device is to continue using the active AI / ML model or switch to the inactive model.

[0156] In some embodiments, the network device may further transmit, to the terminal device, an instruction to continue using the active model based on determining that the anomaly of the active model is not detected or determining that the inactive model is not suitable for beam management; or transmit, to the terminal device, an instruction to switch to the inactive model based on determining that the anomaly of the active model is detected or determining that the inactive model is identified to be suitable for beam management.

[0157] In some embodiments, the network device may further adjust a size of the monitoring window based on historical performance data of the active AI / ML model and the operational condition of the terminal device.

[0158] Fig. 9A illustrates another flowchart of an example method 900A implemented at a terminal device in accordance with some other embodiments of the present disclosure. For the purpose of discussion, the method 900A will be described from the perspective of the terminal device 101 with reference to Fig. 1.

[0159] At block 91 OA, the terminal device obtains an operational condition of the terminal device. At block 920A, the terminal device determines, based on the operational condition, a size of a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are assessed.

[0160] In some embodiments, the operational condition comprises at least one of the following: a mobility state; a configuration from a network device; an environment type; or a network condition.

[0161] In some embodiments, the terminal device determines the size of the monitoring window by: periodically estimating a speed of the terminal device based on the mobility state; periodically estimating a factor indicated in the configuration from the network device; and periodically resizing the size of the monitoring window based on the estimated speed and the estimated factor. In some embodiments, the configuration from the network device comprises at least one of the following: a number of the beams, or a beam width.

[0162] In some embodiments, the terminal device determines the size of the monitoring window by: training a model by using historical performance data of the AI / ML model and historical operational condition of the terminal device; and predicting a window size based on the obtained operational condition of the terminal device by using the trained model.

[0163] In some embodiments, a first size of the monitoring window for a first operational condition is different from a second size of the monitoring window for a second operational condition, and in the event that the first operational condition has a mobility state higher than that in the second operational condition, the first size is shorter than the second size.

[0164] In some embodiments, the monitoring window in time is associated with a number of beammanagement cycles. In some embodiments, the historical performance data of the AI / ML model comprises at least one of the following: prediction accuracy of the AI / ML model; a latency metric of the AI / ML model; a confidence score of the AI / ML model; an error rate of the AI / ML model; or resource utilization of the AI / ML model.

[0165] In some embodiments, the monitoring window comprises a sliding window over which the performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed. In some embodiments, the monitoring window comprises a rolling window over which the performance metrics of an active AI / ML model at the terminal device are assessed over time to detect an anomaly of the active AI / ML model.

[0166] In some embodiments, the performance metric further comprises at least one of the following: a metric associated with beam coverage and capacity; a metric associated with radio resource utilization and efficiency; a metric associated with reliability and a quality of service; or a metric associated with complexity, power consumption and a hardware requirement.

[0167] Fig. 9B illustrates another flowchart of an example method 900B implemented at a network device in accordance with some other embodiments of the present disclosure. For the purpose of discussion, the method 900B will be described from the perspective of the first network device 102 with reference to Fig. 1.

[0168] At block 91 OB, the network device receives, from a terminal device, an operational condition of the terminal device within a determined monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are assessed.

[0169] In some embodiments, an apparatus (for example, the terminal device 101) capable of performing the method 700A may comprise means for performing the respective steps of the method 700A. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module.

[0170] In some embodiments, the apparatus comprises means for transmitting to a first network device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; and means for receiving, from the first network device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

[0171] In some embodiments, the AI / ML model does not fail, and the means for receiving the result comprises means for receiving an AI / ML-based beam reporting instruction; or the AI / ML model fails, and the means for receiving the result comprises means for receiving a fallback-based beam reporting instruction.

[0172] In some embodiments, the apparatus further comprises means for storing at least one historical beam usage pattern of a set of heuristics for beam reporting that are predefined based on the at least one historical beam usage pattern and at least one environmental factor; means for based on determining that the AI / ML model fails, autonomously switching to the fallback-based beam reporting; and means for selecting an effective beam based on the at least one stored historical beam usage pattern.

[0173] In some embodiments, the apparatus further comprises means for receiving, from the first network device, an instruction to switch to fallback-based beam reporting, wherein a set of heuristics for beam reporting, that are predefined based on at least one historical beam usage pattern and at least one environmental factor are confirmed or overridden by the first network device based on network intelligence; and means for switching to the fallback-based beam reporting based on the instruction. In some embodiments, the apparatus further comprises means for adjusting the sliding window based on the operational condition.

[0174] In some embodiments, the apparatus further comprises means for performing other steps in some embodiments of the method 700A. In some embodiments, the means comprises at least one processor and at least one memory including computer program code, the at least one memory and computer program code configured to, with the at least one processor, cause the performance of the apparatus.

[0175] In some embodiments, an apparatus (for example, the first network device 102) capable of performing the method 700B may comprise means for performing the respective steps of the method 700B. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module.

[0176] In some embodiments, the apparatus comprises means for receiving, from a terminal device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; and means for transmitting, to the terminal device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

[0177] In some embodiments, the apparatus further comprises means for transmitting, to an entity including the AI / ML model, a request for beam prediction and performance data of the AI / ML model and the received operational condition within the monitoring window; and means for receiving, from the entity including the AI / ML model, a result of the beam prediction and the performance metric of the AI / ML model within the monitoring window.

[0178] In some embodiments, the apparatus further comprises means for transmitting, to a threshold logic entity, the received performance metrics of the AI / ML model that are to be compared with the at least one threshold at the threshold logic entity within the monitoring window.

[0179] In some embodiments, the performance metric of the AI / ML model is below the threshold, and the apparatus further comprises means for receiving, from the threshold logic entity, an indication to continue AI / ML-based beam management; and means for transmitting, to the terminal device, an AI / ML-based beam reporting instruction. In some embodiments, the performance metric of the AI / ML model is above the threshold, and the apparatus further comprises means for receiving, from a fallback mechanism entity, an indication of fallback-based beam reporting; and means for transmitting, to the terminal device, a fallbackbased beam reporting instruction.

[0180] In some embodiments, the apparatus further comprises means for performing other steps in some embodiments of the method 700B. In some embodiments, the means comprises at least one processor and at least one memory including computer program code, the at least one memory and computer program code configured to, with the at least one processor, cause the performance of the apparatus.

[0181] In some embodiments, an apparatus (for example, the second network device 103) capable of performing the method 700C may comprise means for performing the respective steps of the method 700C. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module.

[0182] In some embodiments, the apparatus comprises means for receiving from a first network device, an operational condition of a terminal device and performance metrics of an artificial intelligence / machine learning (AI / ML) within a monitoring window over which performance metrics of the AI / ML model are aggregated and assessed; means for determining, based on historical performance data of the AI / ML model and the operational condition, at least one threshold for assessing performance metrics of the AI / ML model; means for determining, based on comparison between the performance metrics and the at least one threshold, whether the AI / ML model fails; and means for transmitting, to the first network device, a result about whether the AI / ML model fails.

[0183] In some embodiments, based on determining that the performance metrics do not exceed the at least one determined threshold, means for transmitting a result comprises means for transmitting, to the first network device, an indication to continue AI / ML-based beam management. In some embodiments, based on determining that the performance metrics exceed the at least one determined threshold, the apparatus further comprises means for transmitting, to a fallback mechanism entity, a request for initiating fallback process.

[0184] In some embodiments, the apparatus further comprises means for performing other steps in some embodiments of the method 700C. In some embodiments, the means comprises at least one processorand at least one memory including computer program code, the at least one memory and computer program code configured to, with the at least one processor, cause the performance of the apparatus.

[0185] In some embodiments, an apparatus (for example, the terminal device 101) capable of performing the method 800A may comprise means for performing the respective steps of the method 800A. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module.

[0186] In some embodiments, the apparatus comprises means for transmitting to a network device, an operational condition of the terminal device within a monitoring window over which an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed; and means for receiving a result of the anomaly from the network device, wherein the result is based on comparison between performance metrics of the AI / ML model and historical data.

[0187] In some embodiments, the anomaly of the active AI / ML model is not detected, and the means for receiving the result comprises means for receiving an instruction to continue using the active AI / ML model. In some embodiments, the anomaly of the active AI / ML model is detected, and the means for receiving the result comprises means for receiving an instruction to activate an inactive model that is identified to be suitable for beam management.

[0188] In some embodiments, the apparatus further comprises means for adjusting a size of the monitoring window based on historical performance data of the active AI / ML model and the operational condition of the terminal device.

[0189] In some embodiments, the apparatus further comprises means for performing other steps in some embodiments of the method 800A. In some embodiments, the means comprises at least one processor and at least one memory including computer program code, the at least one memory and computer program code configured to, with the at least one processor, cause the performance of the apparatus.

[0190] In some embodiments, an apparatus (for example, the first network device 102) capable of performing the method 800B may comprise means for performing the respective steps of the method 800B. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module.

[0191] In some embodiments, the apparatus comprises means for receiving, from a terminal device, an operational condition of the terminal device within a monitoring window over which an anomaly detection of an active artificial intelligence / machine learning (AI / ML) model is performed; and means for transmitting a result of the anomaly detection to the terminal device, wherein the result is based on comparison between performance metrics of the AI / ML model and historical data.

[0192] In some embodiments, the apparatus further comprises means for transmitting, to a first entity including the active AI / ML model, a request for current performance data of the active AI / ML model within the monitoring window; and means for receiving, from the first entity, a current performance metric of theactive AI / ML model within the monitoring window. In some embodiments, the apparatus further comprises means for transmitting, to an anomaly detection entity, the received current performance metric that is to be compared with historical data at the anomaly detection entity to detect the anomaly of the active AI / ML model; and means for receiving, from the anomaly detection entity, an output of the anomaly detection entity representing result of the anomaly detection.

[0193] In some embodiments, the apparatus further comprises means for transmitting, to a second entity including a n inactive AI / ML model, a request for proactively monitoring current performance data of the inactive AI / ML model within the monitoring window; and means for receiving, from the second entity, a readiness data of the inactive AI / ML model within the monitoring window, wherein the readiness data represents whether the inactive AI / ML model has sufficient recent training data for the operational condition.

[0194] In some embodiments, the request is transmitted by using a lightweight data exchange protocol; and the readiness data is received by using the lightweight data exchange protocol.

[0195] In some embodiments, the apparatus further comprises means for determining, based on the output of the anomaly detection entity and the readiness data of the inactive AI / ML model, whether the terminal device is to continue using the active AI / ML model or switch to the inactive model.

[0196] In some embodiments, the apparatus further comprises means for transmitting, to the terminal device, an instruction to continue using the active model based on determining that the anomaly of the active model is not detected or determining that the inactive model is not suitable for beam management; or means for transmitting, to the terminal device, an instruction to switch to the inactive model based on determining that the anomaly of the active model is detected or determining that the inactive model is identified to be suitable for beam management.

[0197] In some embodiments, the apparatus further comprises means for adjusting a size of the monitoring window based on historical performance data of the active AI / ML model and the operational condition of the terminal device.

[0198] In some embodiments, the apparatus further comprises means for performing other steps in some embodiments of the method 800B. In some embodiments, the means comprises at least one processor and at least one memory including computer program code, the at least one memory and computer program code configured to, with the at least one processor, cause the performance of the apparatus.

[0199] In some embodiments, an apparatus (for example, the terminal device 101) capable of performing the method 900A may comprise means for performing the respective steps of the method 900A. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module.

[0200] In some embodiments, the apparatus comprises means for obtaining an operational condition of the terminal device; and means for determining, based on the operational condition, a size of a monitoringwindow over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are assessed.

[0201] In some embodiments, the means for determining the size of the monitoring window comprises means for periodically estimating a speed of the terminal device based on the mobility state; means for periodically estimating a factor indicated in the configuration from the network device; and means for periodically resizing the size of the monitoring window based on the estimated speed and the estimated factor. In some embodiments, the configuration from the network device comprises at least one of the following: a number of the beams, or a beam width.

[0202] In some embodiments, the means for determining the size of the monitoring window comprises means fortraining a model by using historical performance data of the AI / ML model and historical operational condition of the terminal device; and means for predicting a window size based on the obtained operational condition of the terminal device by using the trained model.

[0203] In some embodiments, the apparatus further comprises means for performing other steps in some embodiments of the method 900A. In some embodiments, the means comprises at least one processor and at least one memory including computer program code, the at least one memory and computer program code configured to, with the at least one processor, cause the performance of the apparatus.

[0204] In some embodiments, an apparatus (for example, the first network device 102) capable of performing the method 900B may comprise means for performing the respective steps of the method 900B. The means may be implemented in any suitable form. For example, the means may be implemented in a circuitry or software module.

[0205] In some embodiments, the apparatus comprises means for receiving, from a terminal device, an operational condition of the terminal device within a determined monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are assessed.

[0206] In some embodiments, the apparatus further comprises means for performing other steps in some embodiments of the method 900B. In some embodiments, the means comprises at least one processor and at least one memory including computer program code, the at least one memory and computer program code configured to, with the at least one processor, cause the performance of the apparatus.

[0207] Fig. 10 illustrates a simplified block diagram of a device 1000 that is suitable for implementing some exemplary embodiments of the present disclosure. The device 1000 may be provided to implement a communication device, for example, the network device 102, 103, or the terminal device 101 as shown in Fig. 1. As shown, the device 1000 includes one or more processors 1010, one or more memories 1020 coupled to the processor 1010, and one or more communication modules 1040 coupled to the processor 1010.

[0208] The communication module 1040 is for bidirectional communications. The communication module 1040 has at least one antenna to facilitate communication. The communication interface may represent any interface that is necessary for communication with other network elements.

[0209] The processor 1010 may be of any type suitable to the local technical network and may include one or more of the following: general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs) and processors based on multicore processor architecture, as non-limiting examples. The device 1000 may have multiple processors, such as an application specific integrated circuit chip that is slaved in time to a clock which synchronizes the main processor.

[0210] The memory 1020 may include one or more non-volatile memories and one or more volatile memories. Examples of the non-volatile memories include, but are not limited to, a Read Only Memory (ROM) 1024, an electrically programmable read only memory (EPROM), a flash memory, a hard disk, a compact disc (CD), a digital video disk (DVD), and other magnetic storage and / or optical storage. Examples of the volatile memories include, but are not limited to, a random access memory (RAM) 1022 and other volatile memories that will not last in the power-down duration.

[0211] A computer program 1030 includes computer executable instructions that are executed by the associated processor 1010. The program 1030 may be stored in the ROM 1024. The processor 1010 may perform any suitable actions and processing by loading the program 1030 into the RAM 1022.

[0212] The embodiments of the present disclosure may be implemented by means of the program 1030 so that the device 1000 may perform any process of the disclosure as discussed above. The embodiments of the present disclosure may also be implemented by hardware or by a combination of software and hardware.

[0213] In some exemplary embodiments, the program 1030 may be tangibly contained in a computer-readable medium which may be included in the device 1000 (such as in the memory 1020) or other storage devices that are accessible by the device 1000. The device 1000 may load the program 1030 from the computer-readable medium to the RAM 1022 for execution. The computer-readable medium may include any types of tangible non-volatile storage, such as ROM, EPROM, a flash memory, a hard disk, CD, DVD, and the like.

[0214] Fig. 11 illustrates a block diagram of an example of a computer-readable medium 1100 in accordance with some exemplary embodiments of the present disclosure. The computer-readable medium 1100 has the program 1030 stored thereon. It is noted that although the computer-readable medium 1100 is depicted in form of CD or DVD, the computer-readable medium 1100 may be in any other form suitable for carry or hold the program 1030.

[0215] Generally, various embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by acontroller, microprocessor or other computing device. While various aspects of embodiments of the present disclosure are illustrated and described as block diagrams, flowcharts, or using some other pictorial representations, it is to be understood that the block, apparatus, system, technique or method described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0216] The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computerexecutable instructions, such as those included in program modules, being executed in a device on a target real or virtual processor, to carry out the method as described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, or the like that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or split between program modules as desired in various embodiments. Machine-executable instructions for program modules may be executed within a local or distributed device. In a distributed device, program modules may be located in both local and remote storage media.

[0217] Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0218] In the context of the present disclosure, the computer program codes or related data may be carried by any suitable carrier to enable the device, apparatus or processor to perform various processes and operations as described above. Examples of the carrier include a signal, computer-readable medium, and the like.

[0219] The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium may include but not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium would include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. The term “non-transitory,” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM).

[0220] Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are contained in the above discussions, these should not be construed as limitations on the scope of the present disclosure, but rather as descriptions of features that may be specific to particular embodiments. Certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination.

[0221] Although the present disclosure has been described in languages specific to structural features and / or methodological acts, it is to be understood that the present disclosure defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

WHAT IS CLAIMED IS:

1. A terminal device comprising:at least one processor; andat least one memory storing instructions that, when executed by the at least one processor, cause the terminal device at least to:transmit, to a first network device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; andreceive, from the first network device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

2. The terminal device of claim 1, wherein:the AI / ML model does not fail, and the terminal device is caused to receive the result by receiving an AI / ML-based beam reporting instruction; orthe AI / ML model fails, and the terminal device is caused to receive the result by receiving a fallbackbased beam reporting instruction.

3. The terminal device of claims 1 or 2, wherein the operational condition comprises an initial performance metric; andthe at least one threshold comprises a composite threshold having a weighted sum of items of the initial performance metric included in the operational condition.

4. The terminal device of any of claims 1 to 3, wherein the at least one threshold comprises: a threshold value and a margin value; oran entry threshold and an exit threshold of a threshold range.

5. The terminal device of any of claims 1 to 4, wherein a first threshold for a first operational condition is different from a second threshold for a second operational condition.

6. The terminal device of any of claims 1 to 5, wherein the terminal device is further caused to: store at least one historical beam usage pattern of a set of heuristics for beam reporting that are predefined based on the at least one historical beam usage pattern and at least one environmental factor;based on determining that the AI / ML model fails, autonomously switch to the fallback-based beam reporting; andselect an effective beam based on the at least one stored historical beam usage pattern.

7. The terminal device of claim 6, wherein the at least one historical beam usage pattern comprises one or more beams that are selected for a corresponding operational condition; andthe at least one environmental factor comprises (i) measurements from neighboring network devices or beams for refining fallback selection, or (ii) factors in obstacles or known channel impairments.

8. The terminal device of claim 6 or 7, wherein the set of heuristics for beam reporting comprises at least one of the following:ranking beams based on recent average signal strength;rotating through candidate beams that historically have performance over a corresponding threshold; orusing a rule-based system that includes context variables.

9. The terminal device of claim 8, wherein the context variable comprises at least one of the following: a speed of the terminal device,an environment type of the terminal device, ora historical interference level.

10. The terminal device of any of claims 1 to 9, wherein the terminal device is further caused to: receive, from the first network device, an instruction to switch to fallback-based beam reporting, wherein a set of heuristics for beam reporting, that are predefined based on at least one historical beam usage pattern and at least one environmental factor, are confirmed or overridden by the first network device based on network intelligence; andswitch to the fallback-based beam reporting based on the instruction.

11. The terminal device of any of claims 1 to 10, wherein the terminal device is further caused to: adjust the sliding window based on the operational condition.

12. The terminal device of any of claims 1 to 11 , wherein the operational condition comprises at least one of the following:a mobility state;a configuration from the first network device;an environment type; ora network condition.

13. The terminal device of any of claims 1 to 12, wherein the performance metrics comprise at least one of the following:prediction accuracy of the AI / ML model;a latency metric of the AI / ML model;a confidence score of the AI / ML model;an error rate of the AI / ML model; orresource utilization of the AI / ML model.

14. The terminal device of any of claim 1 to 13, wherein the performance metric further comprises at least one of the following:a metric associated with beam coverage and capacity;a metric associated with radio resource utilization and efficiency;a metric associated with reliability and a quality of service; ora metric associated with complexity, power consumption and a hardware requirement.

15. The terminal device of claim 14, wherein the metric associated with beam coverage and capacity comprises at least one of the following:a beamforming gain which is a metric that quantifies the improvement in signal strength achieved by focusing the signal towards an intended terminal device;a coverage area which represents an area covered by a beam;a terminal device throughput which represents an average data rate achieved by terminal devices within the beam;spectral efficiency which represents an amount of data transmitted per unit of bandwidth; or cell edge performance which represents a performance of terminal devices located at an edge of the cell.

16. The terminal device of claim 14 or 15, wherein the metric associated with radio resource utilization and efficiency comprises at least one of the following:a beam switching speed for switching between different beams;a beam overlap which represents an extent to which beams overlap;resource allocation efficiency which represents an effectiveness of allocating resources to different beams; orbeam tracking accuracy which represents ability of a system to accurately track a location of the terminal device and adjust a beam.

17. The terminal device of any of claims 14 to 16, wherein the metric associated with reliability and quality of service comprises at least one of the following:a block error rate which represents a rate of errors in data transmission;a latency which represents a delay experienced by data packets;a packet loss rate which represents a percentage of data packets that are lost during transmission; ora terminal device experience which represents subjective measures of the terminal device satisfaction with the quality of service.

18. The terminal device of any of claims 14 to 17, wherein the metric associated with complexity, power consumption and a hardware requirement comprises at least one of the following:complexity which represents computational complexity of a beam management algorithm; power consumption which represents an amount of power consumed by a beamforming system; or a hardware requirement which represents hardware resources needed to support beam management.

19. A first network device comprising:at least one processor; andat least one memory storing instructions that, when executed by the at least one processor, cause the first network device at least to:receive, from a terminal device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; andtransmit, to the terminal device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

20. The first network device of claim 19, wherein the first network device is further caused to: transmit, to an entity including the AI / ML model, a request for beam prediction and performance data of the AI / ML model and the received operational condition within the monitoring window; andreceive, from the entity including the AI / ML model, a result of the beam prediction and the performance metric of the AI / ML model within the monitoring window.

21. The first network device of claim 19 or 20, wherein the first network device is further caused to: transmit, to a threshold logic entity, the received performance metrics of the AI / ML model that are to be compared with the at least one threshold at the threshold logic entity within the monitoring window.

22. The first network device of claim 21, wherein the performance metric of the AI / ML model is below the threshold, and the first network device is further caused to:receive, from the threshold logic entity, an indication to continue AI / ML-based beam management; andtransmit, to the terminal device, an AI / ML-based beam reporting instruction.

23. The first network device of claim 21, wherein the performance metric of the AI / ML model is above the threshold, and the first network device is further caused to:receive, from a fallback mechanism entity, an indication of fallback-based beam reporting; and transmit, to the terminal device, a fallback-based beam reporting instruction.

24. A second network device comprising:at least one processor; andat least one memory storing instructions that, when executed by the at least one processor, cause the second network device at least to:receive, from a first network device, an operational condition of a terminal device and performance metrics of an artificial intelligence / machine learning (AI / ML) within a monitoring window over which performance metrics of the AI / ML model are aggregated and assessed;determine, based on historical performance data of the AI / ML model and the operational condition, at least one threshold for assessing performance metrics of the AI / ML model;determine, based on comparison between the performance metrics and the at least one threshold, whether the AI / ML model fails; andtransmit, to the first network device, a result about whether the AI / ML model fails.

25. The second network device of claim 24, wherein the second network device is further caused to: based on determining that the performance metrics do not exceed the at least one determined threshold, transmit, to the first network device, an indication to continue AI / ML-based beam management.

26. The second network device of claim 24, wherein the second network device is further caused to:based on determining that the performance metrics exceed the at least one determined threshold, transmit, to a fallback mechanism entity, a request for initiating fallback process.

27. A method at a terminal device comprising:transmitting, to a first network device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; andreceiving, from the first network device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

28. A method at a first network device comprising:receiving, from a terminal device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; andtransmitting, to the terminal device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

29. A method at a second network device comprising:receiving, from a first network device, an operational condition of a terminal device and performance metrics of an artificial intelligence / machine learning (AI / ML) within a monitoring window over which performance metrics of the AI / ML model are aggregated and assessed;determining, based on historical performance data of the AI / ML model and the operational condition, at least one threshold for assessing performance metrics of the AI / ML model;determining, based on comparison between the performance metrics and the at least one threshold, whether the AI / ML model fails; andtransmitting, to the first network device, a result about whether the AI / ML model fails.

30. An apparatus comprising:means for transmitting, to a first network device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; andmeans for receiving, from the first network device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

31. An apparatus comprising:means for receiving, from a terminal device, an operational condition of the terminal device within a monitoring window over which performance metrics of an artificial intelligence / machine learning (AI / ML) model are aggregated and assessed; andmeans for transmitting, to the terminal device, a result about whether the AI / ML model fails, wherein the result is based on comparison between the performance metrics and at least one threshold that is based on historical performance data of the AI / ML model and the operational condition.

32. An apparatus comprising:means for receiving, from a first network device, an operational condition of a terminal device and performance metrics of an artificial intelligence / machine learning (AI / ML) within a monitoring window over which performance metrics of the AI / ML model are aggregated and assessed;means for determining, based on historical performance data of the AI / ML model and the operational condition, at least one threshold for assessing performance metrics of the AI / ML model;means for determining, based on comparison between the performance metrics and the at least one threshold, whether the AI / ML model fails; andmeans for transmitting, to the first network device, a result about whether the AI / ML model fails.

33. A computer readable medium comprising program instructions for causing an apparatus to perform at least the method of any of claims 27 to 29.

Images