AI / ML model applicability mechanism

Abstract

Example embodiments of the present disclosure relate to an artificial intelligence (AI) / machine learning (ML) model applicability mechanism. In accordance with an example embodiment, a device may be configured to evaluate suitability of an AI / ML model associated with a mobile communication network, and then generate a report message including the evaluated suitability. The device may then provide the report message to at least one network element (NE) of the mobile telecommunications network.

Description

Cross-reference to related applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 541,015, filed with the U.S. Patent and Trademark Office on September 28, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to mechanisms for the applicability of artificial intelligence (AI) / machine learning (ML) models. Background Technology

[0003] The information disclosed in this Background section is intended only to enhance the understanding of the general background of this disclosure and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art.

[0004] With the development of telecommunications technology, the role of artificial intelligence (AI) and / or machine learning (ML) models (referred to as "AI / ML models" in this paper) and features enabled by AI / ML models (referred to as "AI / ML enabled features" in this paper) is becoming more critical in telecommunications networks, because the integration and implementation of AI / ML models and associated features can enable potential use cases such as optimizing network performance, enhancing user experience, and enabling advanced functions such as autonomous lifecycle management (LCM) of network elements. Summary of the Invention

[0005] Example embodiments of this disclosure provide systems, apparatuses, methods, etc., that utilize various novel features or mechanisms associated with the applicability of AI / ML models.

[0006] According to an example embodiment, the device can be configured to evaluate the suitability of an AI / ML model associated with a mobile telecommunications network and then generate a report message containing the evaluated suitability. The device can then provide the report message to at least one network element (NE) of the mobile telecommunications network.

[0007] According to an example embodiment, the method may include: evaluating the suitability of an AI / ML model associated with a mobile telecommunications network; generating a report message containing the evaluated suitability; and providing the report message to at least one NE of the mobile telecommunications network.

[0008] A non-transitory computer-readable recording medium may have instructions thereon that can be executed by a device, which, when executed, cause the device to perform a method comprising: evaluating the suitability of an AI / ML model associated with a mobile telecommunications network; generating a report message containing the evaluated suitability; and providing the report message to at least one NE of the mobile telecommunications network.

[0009] Additional aspects will be set forth in part in the description which follows, and will be apparent in part from the description, or may be realized by means of the embodiments presented in this disclosure. Attached Figure Description

[0010] The features, aspects, and advantages of embodiments of the present disclosure will now be described with reference to the accompanying drawings, in which similar reference numerals denote similar elements, and wherein:

[0011] Figure 1 A block diagram illustrating an example system architecture according to one or more example embodiments is shown;

[0012] Figure 2 A block diagram illustrating an example method for transmitting AI / ML model suitability information according to one or more example embodiments;

[0013] Figure 3 A flowchart illustrating example use cases associated with proactive reporting based on one or more example embodiments;

[0014] Figure 4 A flowchart illustrating example use cases associated with reactive reporting based on one or more example embodiments;

[0015] Figure 5 A block diagram illustrating an example method for dynamic threshold reporting according to one or more example embodiments is shown;

[0016] Figure 6A and Figure 6B Each illustrates a flowchart of an example use case associated with dynamic threshold adjustment according to one or more example embodiments;

[0017] Figure 7 A block diagram illustrating an example method for real-time monitoring and reporting according to one or more example embodiments is shown;

[0018] Figure 8 Flowcharts illustrating example use cases associated with real-time monitoring and reporting according to one or more example embodiments; and

[0019] Figure 9 A device for implementing one or more example embodiments is shown. Detailed Implementation

[0020] The following detailed description of exemplary embodiments is provided with reference to the accompanying drawings. While the foregoing disclosure provides illustrations and descriptions, it is not intended to be exhaustive or to limit implementations to the precise forms disclosed. Modifications and variations can be made based on the foregoing disclosure, or can be obtained from practice of implementations. Furthermore, one or more features or components of one embodiment may be combined or incorporated into another embodiment (or one or more features of another embodiment). Additionally, the flowcharts and operational descriptions provided below relate to one of various embodiments. It should be noted that other embodiments that do not perfectly match the flowcharts and their descriptions are also possible. It should be understood that in other embodiments, one or more operations may be omitted, one or more operations may be added, or one or more operations may be performed simultaneously (at least partially).

[0021] It is evident that the systems and / or methods described herein can be implemented using various forms of hardware, firmware, or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not limited to the implementations described. Therefore, this document describes the operation and behavior of the systems and / or methods without reference to specific software code. It should be understood that software and hardware can be designed to implement the system and / or method based on the descriptions herein.

[0022] Although specific combinations of features are disclosed in the claims and / or specification, these combinations are not intended to limit the disclosure of the implementation. In fact, many of these features can be combined in ways not specifically stated in the claims and / or not disclosed in the specification. Although each dependent claim listed below may directly depend on only one claim, the disclosure of the implementation includes each dependent claim combined with each other claim in the claim set.

[0023] No element, action, or instruction used herein should be construed as critical or necessary unless explicitly stated otherwise. Furthermore, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” Also, as used herein, the terms “having,” “having,” “with,” “including,” “comprising,” etc., are open-ended terms. Additionally, unless explicitly stated otherwise, the word “based on” means “at least partially based on.” Moreover, expressions such as “at least one of [A] and [B],” “[A] and / or [B],” or “at least one of [A] or [B]” should be understood to include only A, only B, or both A and B.

[0024] It should be noted that the description of exemplary embodiments of this disclosure may include terms and names defined in one or more standardization organizations, such as the 3rd Generation Partnership Project (3GPP) standardization organization, the European Telecommunications Standards Institute (ETSI) standardization organization, etc. For example, unless otherwise described, the terms “RRC,” “UAI,” “NAS,” “MAC CE,” “IE,” etc., and the associated features and operations, should be interpreted as consistent with the features and operations specified in one or more technical specifications.

[0025] Furthermore, as used herein, the term "AI / ML suitability" can indicate whether an AI / ML technology is suitable or unsuitable for a particular condition, scenario, or use case. The term "AI / ML model suitability" can indicate whether an AI / ML model is suitable or unsuitable for a particular condition, scenario, or use case. The term "AI / ML-enabled feature suitability" can indicate whether a feature associated with an AI / ML model or AI / ML technology is suitable or unsuitable for a particular condition, scenario, or use case. In this regard, unless otherwise described, the terms "AI / ML suitability," "AI / ML model suitability," and "AI / ML-enabled feature suitability" can be used interchangeably herein, as all of these terms can indicate whether AI / ML is suitable.

[0026] As mentioned above, in recent years, the implementation and utilization of artificial intelligence (AI) and / or machine learning (ML) in telecommunications systems have been significantly improved, especially due to the emergence of new advanced telecommunications technologies such as fifth-generation (5G) network systems and open radio access network (O-RAN) architectures.

[0027] In this regard, multiple AI / ML models for the same features can be pre-trained based on datasets associated with different aspects of network behavior and performance. For example, a first AI / ML model associated with beamforming can be pre-trained based on data obtained from dense urban areas, while a second AI / ML model associated with beamforming can be pre-trained based on data obtained from rural environments. These pre-trained AI / ML models can be stored in an AI / ML server that manages the deployment and updates of the AI / ML models. In operation, user equipment (UE) and / or network elements (NE) can communicate with the AI / ML server to load the AI / ML models and then implement the associated features based on them.

[0028] In the implementation of AI / ML features in telecommunications systems, the applicability of the AI / ML model (e.g., whether the AI / ML model is applicable under specific conditions) is crucial, as this information is needed to ensure accurate AI / ML model implementation. For example, an AI / ML model may vary depending on the UE's location (e.g., an implementation of an AI / ML model developed for a densely populated urban area might result in inaccurate AI / ML performance when the UE is in a rural environment). Similarly, AI / ML models can also vary depending on various types of conditions (e.g., network conditions, UE configuration / settings, application requirements, etc.).

[0029] In related technologies, there is no interaction between the UE and NE when implementing AI / ML-enabled features. Instead, the AI / ML model to be implemented is selected by the UE or NE. For example, the UE or NE can select the AI / ML model and activate / deactivate AI / ML-enabled features based on its own observation and evaluation of applicability conditions.

[0030] As a result, the selected AI / ML model is not always accurate, and the implemented AI / ML features may not function as expected. Furthermore, this can increase the complexity of lifecycle management (LCM) and the burden on the UE or network side to evaluate the suitability of the AI / ML model and apply the features. Therefore, the UE and NE may experience inconsistencies or misalignments in their AI / ML functional and performance expectations.

[0031] This disclosure provides example embodiments of systems, methods, devices, etc., that implement a novel suitability framework and mechanism for managing AI / ML model suitability information within a mobile telecommunications network. Specifically, through the suitability framework and associated mechanisms of the example embodiments, UEs and / or NEs within the network can effectively and efficiently report and notify users of AI / ML model suitability information. Furthermore, UEs and / or NEs can dynamically adjust or configure the thresholds and conditions used to evaluate the suitability of AI / ML models. Moreover, UEs and / or NEs can perform real-time monitoring of AI / ML model performance metrics and report them to other network components. Additionally, UEs and / or NEs can dynamically select appropriate AI / ML models.

[0032] It is conceivable that the features, advantages, and significance of the exemplary embodiments described above are only a part of this disclosure and are not intended to be exhaustive or to limit the scope of this disclosure. Further descriptions of features, components, configurations, and operations will be provided below. Example System Architecture

[0033] Figure 1A block diagram illustrating an example system architecture according to one or more example embodiments is shown. Figure 1 As shown, the system architecture may include at least one user equipment (UE) 110 and multiple network elements (NEs) 120. The multiple NEs 120 may include a base station 120-1, a non-access stratum (NAS) 120-2, and an AI / ML model server 120-3. It is conceivable that... Figure 1 The system architecture shown is simplified for descriptive purposes, and the scope of this disclosure should not be limited thereto. For example, without departing from the scope of this disclosure, the plurality of NEs 120 may include any other suitable NE associated with a mobile telecommunications network.

[0034] UE 110 may include one or more devices or equipment that can be used by one or more network users (e.g., network operators, end users such as network subscribers) to access the network system through interoperability with one or more of NE 120. For example, UE 120-2 may include one or more of the following: static devices (e.g., computing devices such as desktop computers or servers, fixed wireless access (FWA) routers, internet modems, Internet of Things (IoT) devices such as security cameras, sensors, and smart home devices), mobile devices (e.g., smartphones, cordless phones, laptops, tablets, handheld computers, smart devices, wearable devices such as smart glasses or smartwatches, portable hotspots), mobile devices that have been in a static state for a predetermined period of time (e.g., mobile devices that have been left on a table or charging station for a predetermined period of time), and / or any other suitable devices or equipment.

[0035] Base station 120-1 may include at least one of the following: fourth-generation (4G) Long Term Evolution (LTE) eNodeB, fifth-generation (5G) gNodeB, etc. According to an example embodiment, base station 120-1 may include a radio unit (RU), a distribution unit (DU), and a central unit (CU). According to an example embodiment, base station 120-1 may include a base station based on an Open Radio Access Network (O-RAN).

[0036] On the other hand, NAS 120-2 can be implemented as a network function (NF), such as a virtualized NF (VNF), a containerized / cloudified NF (CNF), etc. According to an example embodiment, NAS 120-2 can be part of a core network element, such as a component or function of the Mobility Management Entity (MME) in LTE, or a component or function of the Access and Mobility Management Function (AMF) in 5G. According to an example embodiment, NAS 120-2 can communicatively couple base station 120-1 to AI / ML model server 120-3.

[0037] Further as described below, in some example embodiments, NAS 120-1 may participate in signaling of AI / ML model suitability information between UE 110 and one or more NE 120s. It is conceivable that in example implementations where AI / ML model suitability information is transmitted via other types of mechanisms (e.g., RRC message signaling, PHY layer message signaling, etc.), NAS 120-1 may be optional and may be excluded from the system architecture.

[0038] AI / ML model server 120-3 may include one or more servers that host, manage, and process one or more AI / ML models associated with a mobile telecommunications network. AI / ML model server 120-3 can interact with UE 110 and other NEs 120 to transmit information associated with one or more AI / ML models for various network-related tasks. For example, server 120-3 may receive data from UE 110, base station 120-1, and / or NAS 120-2, and then use AI / ML models to process the received data. Therefore, server 120-3 can provide UE 110, base station 120-1, and / or NAS 120-2 with actionable insights or information associated with AI / ML model processing.

[0039] UE 110 can interoperate with one or more NE 120s to implement the AI / ML model suitability framework and associated mechanisms. For example, UE 110 and / or NE 120 can be configured to: select an appropriate AI / ML model relevant to the mobile telecommunications network, evaluate the suitability of the AI / ML model, report AI / ML model suitability information to other components within the network system, monitor and report real-time performance metrics, and dynamically adjust thresholds associated with AI / ML model suitability. Further descriptions of the suitability framework and the mechanisms provided therefrom are provided below. Example AI / ML applicability frameworks and mechanisms

[0040] According to an example embodiment, the system (such as...) Figure 1 The (shown) can be configured to implement an applicability framework that enables the UE and NE to interact with each other, thereby transmitting information related to the applicability of the AI / ML model (which may be referred to as "AI / ML model applicability information" in this document).

[0041] According to the example embodiment, AI / ML model suitability information may include information on one or more suitability conditions. In this regard, the suitability framework may include the following three main components associated with suitability conditions: suitability condition management, suitability notification, and suitability condition evaluation.

[0042] Applicability condition management can include applicability condition definitions. Specifically, applicability conditions can be defined by at least one criterion or threshold that must be met to make the associated AI / ML model or feature applicable in a specific scenario or use case. In this regard, applicability conditions and associated criterions or thresholds can be predefined by one or more standardized technical specifications (e.g., 3GPP specifications, etc.) or by network operators and / or UE manufacturers based on the implementation of a specific UE and / or NE. Furthermore, applicability conditions can be static or dynamic, meaning that the UE and / or NE can change or adjust the applicability conditions in real time (or near real time) depending on various factors such as user capabilities, network environment, and specific use cases.

[0043] Applicability assessment can include determining whether an AI / ML-enabled feature or model meets one or more applicability conditions. For example, the UE and / or NE may evaluate available information or measurement results based on collaboration level, model type, etc., to determine whether the AI / ML-enabled feature or model meets the applicability conditions. Further, as described below, the applicability assessment can trigger a proactive reporting mechanism, enabling AI / ML model applicability information to be autonomously transmitted between the UE and NE based on the assessment results.

[0044] Applicability notifications may include informing whether an AI / ML-enabled feature or model is applicable, based on evaluation results. This notification may be executed by the UE, NE, or both the UE and NE, depending on the collaboration level and model type.

[0045] In addition to the features and mechanisms associated with applicability conditions, the applicability framework can also provide features and mechanisms for: sending and reporting AI / ML model applicability results via signaling, monitoring and reporting AI / ML model performance metrics, and dynamically adjusting thresholds associated with AI / ML model applicability. Several example features and their associated mechanisms will be described below. Enhance reporting mechanisms

[0046] According to example embodiments, the suitability framework can provide or implement several reporting mechanisms for information exchange between the UE and the NE. In some example embodiments, the UE can autonomously evaluate the suitability of an AI / ML model based on one or more predefined conditions (e.g., suitability conditions) and then proactively report the suitability of the AI / ML model to one or more NEs. This approach may be referred to herein as a "proactive reporting mechanism," which effectively reduces latency, enhances responsiveness, and reduces network dependency.

[0047] According to the example embodiment, the UE can be configured to perform proactive reporting based on a predefined algorithm. This algorithm can be designed by, for example, network operators and / or UE manufacturers, and can be used by the UE and / or NE to evaluate the suitability conditions of AI / ML models. Example algorithms according to the example embodiment are presented in Table 1 below. Table 1: Example Algorithms for Active Evaluation of AI / ML Model Applicability

[0048] In the example in Table 1, information associated with several conditions (e.g., channel conditions, UE mobility status, application requirements, UE battery level, etc.) is obtained by the UE and / or NE, and these conditions are processed by the UE and / or NE (e.g., evaluation, inspection, analysis, verification, etc.) to determine an applicability result (e.g., applicable, inapplicable, unknown). As a non-limiting example, the UE and / or NE may compare the UE's battery level to a predefined threshold, and then determine that the AI / ML model is applicable based on the determination that the battery level is below the predefined threshold. Therefore, these conditions may also be referred to herein as "applicability conditions".

[0049] According to example embodiments, proactive reporting can be triggered or initiated based on at least one predefined condition. Therefore, the predefined condition may also be referred to herein as a "triggering condition." In this regard, the triggering condition can be predefined by the network operator and / or UE manufacturer to ensure that proactive reporting is triggered in a standardized and uniform manner, thereby ensuring that applicability evaluation and proactive reporting are consistent and reliable, and adapted to specific network requirements. In some example implementations, the triggering condition may include at least one predefined activation condition associated with the functionality of the UE and / or the functionality of the NE. This predefined activation condition can explicitly define the conditions required for activating the function (e.g., evaluation of the applicability of the AI / ML model, etc.), thereby ensuring that the expected conditions are met and verified before activation, and enhancing operational reliability. It is conceivable that the terms "triggering condition" and "activation condition" may be used interchangeably without departing from the scope of this disclosure.

[0050] Several example triggering conditions according to the example embodiments are presented in Table 2 below. Table 2: Example Triggering Conditions

[0051] Table 2 illustrates several triggering conditions, namely conditions 1-3. Condition 1 is defined by comparing a change in the channel condition to a first predefined threshold (i.e., the change in the channel condition is greater than the first predefined threshold), condition 2 is defined by comparing a change in the UE's mobility status to a second predefined threshold (i.e., the change in the mobility status is equal to a threshold associated with high speed), and condition 3 is defined by comparing the UE's battery level to a third predefined threshold (i.e., the battery level is lower than a threshold associated with low battery level). It is conceivable that the triggering conditions in Table 2 are merely examples, and the scope of this disclosure should not be limited thereto.

[0052] According to example embodiments, the UE and / or NE can be configured to generate a reporting message (referred to herein as a "proactive reporting message") during proactive reporting. Therefore, the UE and / or NE can provide this reporting message to the NE and / or UE via various types of signaling mechanisms (described further below), thereby enabling the transmission of AI / ML model applicability information therebetween. Example proactive reporting messages according to one or more example embodiments are presented in Table 3 below. Table 3: Example Proactive Reporting Messages

[0053] As shown in Table 3, the proactive reporting message may include an identifier (ID) associated with the AI / ML feature (e.g., an integer between 1 and 1024), the AI / ML model applicability result (e.g., applicable, not applicable, unknown), and at least one evaluation parameter (e.g., channel conditions, mobility status, battery level, etc.) for the UE and / or NE to perform an applicability evaluation.

[0054] According to an example embodiment where the UE is configured to perform proactive reporting, the NE can provide the UE with information associated with triggering conditions. For example, the NE can generate a configuration message containing triggering condition information (which may be referred to herein as a "trigger configuration message") and then provide the configuration message to the UE, thereby specifying the triggering conditions for proactive reporting and providing a network configuration trigger that ensures the UE's reporting is consistent with network requirements and conditions. Example trigger configuration messages according to one or more example embodiments are presented in Table 4 below. Table 4: Example Triggered Configuration Messages

[0055] As shown in Table 4, the configuration message may contain details associated with one or more trigger conditions, such as the trigger condition ID (e.g., an integer between 1 and 256), the trigger condition type (e.g., channel, mobility, battery, application, etc.), the threshold value associated with the trigger condition (e.g., an integer), and the evaluation interval associated with the time when the trigger condition should be evaluated.

[0056] According to an example embodiment, the UE can receive a trigger configuration message from the NE and then perform an evaluation process (e.g., periodically, continuously, etc.) based on trigger conditions to determine whether a proactive report should be initiated. In this regard, the evaluation process can be predefined by the network operator and / or UE manufacturer to ensure a consistent and reliable process for initiating proactive reports. Example structures of the evaluation process according to one or more example embodiments are presented in Table 5 below. Table 5: Example Structure of Condition Evaluation Process

[0057] As shown in Table 5, the evaluation procedure may include: receiving a configuration message containing triggering conditions and associated configurations; and evaluating the triggering conditions. For example, upon receiving the configuration message, the UE may evaluate the triggering conditions to determine whether one or more conditions for initiating an active reporting procedure are met. Therefore, based on the determination that one or more conditions are met, the UE may determine the suitability of the AI / ML model, generate a report message containing information associated with the suitability of the AI / ML model, and send the report message to the NE.

[0058] According to example embodiments, the UE and / or NE can perform proactive reporting via Radio Resource Control (RRC) signaling. Specifically, the UE and / or NE can include information related to the suitability of the AI / ML model in the RRC Information Element (IE), and provide the NE and / or UE with an RRC message containing the RRC IE via RRC signaling, thereby ensuring that the suitability information is transmitted efficiently and accurately. Examples of RRC IEs for proactive reporting according to one or more example embodiments (which may be referred to herein as "RRC Proactive Reporting IEs") are presented in Table 6 below. Table 6: Example of RRC Proactive Reporting IE

[0059] As shown in Table 6, the RRC proactive reporting IE may include an ID (e.g., an integer between 1 and 1024) associated with the features of the AI / ML model, an applicability result (e.g., applicable, inapplicable, unknown, etc.), and at least one evaluation parameter (e.g., channel conditions, UE mobility status such as stationary, moving, and high-speed, UE battery level such as high, medium, and low, etc.). It is conceivable that, without departing from the scope of this disclosure, the RRC proactive reporting IE may include more or fewer information or parameters than those shown in Table 6. For example, the RRC proactive reporting IE may also include applicability conditions associated with the applicability result, applicability results associated with the applicability result, action suggestions associated with the applicability result, etc.

[0060] In addition to or as an alternative to RRC signaling, the UE can also be configured to perform proactive reporting via UE Assist Information (UAI) signaling. Specifically, the UE can include information related to the suitability of AI / ML models in the UAI IE within the extended UAI message, and provide the UAI message to the UE via UAI signaling, thereby providing detailed suitability information and assisting network optimization. Examples of UAI IEs for proactive reporting (which may be referred to herein as "UAI Proactive Reporting IEs") according to one or more example embodiments are presented in Table 7 below. Table 7: Example of UAI Proactive Reporting IE

[0061] Similar to the RRC proactive reporting IE in Table 6, the UAI proactive reporting IE in Table 7 may include an ID (e.g., an integer between 1 and 1024) associated with the features of the AI / ML model, an applicability result (e.g., applicable, inapplicable, unknown, etc.), and at least one evaluation parameter (e.g., channel conditions, UE mobility state such as stationary, moving, and high speed, UE battery level such as high, medium, and low, etc.). It is conceivable that, without departing from the scope of this disclosure, the UAI proactive reporting IE may contain more or fewer information or parameters than those shown in Table 7. For example, the UAI proactive reporting IE may also include performance metrics associated with the AI / ML model, the training process associated with the AI / ML model, inference accuracy associated with the AI / ML model, information related to data security mechanisms and privacy regulations, etc.

[0062] According to example embodiments, the UE can be configured to perform proactive reporting via RRC and UAI signaling. In this regard, the UE can generate and transmit proactive reporting messages via RRC and UAI through procedures predefined by the network operator and / or UE manufacturer, thereby ensuring timely and accurate applicability reporting and enhancing network responsiveness. Example procedures for proactive reporting via RRC and UAI according to one or more example embodiments are presented in Table 8 below. Table 8: Example RRC and UAI Proactive Reporting Process

[0063] As shown in Table 8, during the reporting process, the UE can first evaluate the triggering conditions to determine whether RRC and UAI reports should be triggered. Therefore, based on the determination that the triggering conditions are met, the UE can trigger RRC and UAI reports by generating an RRC proactive report IE (or an RRC message containing an RRC proactive report IE) and a UAI proactive report IE (or a UAI message containing a UAI proactive report IE) and sending them to the NE.

[0064] In addition to or as an alternative to proactive reporting, the UE can evaluate and report AI / ML model suitability information in response to network queries. Similarly, the NE can evaluate and report AI / ML model suitability information in response to UE queries. This approach, referred to herein as a "reactive reporting mechanism," effectively ensures accurate information exchange and enables the UE and / or NE to provide efficient and accurate AI / ML suitability reports in response to network-initiated queries.

[0065] According to an example embodiment, the UE can be configured to perform a reactive reporting in response to a network query message. This network query message can be provided to the UE by at least one NE (e.g., a base station) to trigger or initiate a reactive applicability report. Example network query messages according to one or more example embodiments are presented in Table 9 below. Table 9: Example Web Query Messages

[0066] As shown in Table 9, the network query message may contain an ID (e.g., an integer between 1 and 1024) associated with a feature of the AI / ML model, as well as multiple query parameters. These query parameters include a condition type (e.g., channel, mobility, battery, application, etc.) and a threshold (e.g., an integer, etc.) associated with that condition. It is conceivable that, without departing from the scope of this disclosure, the network query message may contain more or fewer information or parameters than those shown in Table 9. For example, the network query message may contain any network-side additional conditions or network configurations that may affect the applicability of the AI / ML model and influence the UE's decision-making process and reporting behavior.

[0067] Upon receiving the network query message from the NE, the UE can be configured to generate a response message (referred to herein as a "UE response message") containing information related to the applicability of the AI / ML model. Example UE response messages according to one or more example embodiments are presented in Table 10 below. Table 10: Example UE Response Messages

[0068] As shown in Table 10, this UE response message may contain parameters or information similar to those in the RRC Proactive Reporting IE (in Table 6) and the UAI Proactive Reporting IE (in Table 7). Therefore, for the sake of brevity, the repetitive descriptions associated with it can be omitted in this document.

[0069] According to example embodiments, the UE can be configured to perform reactive reporting through a process predefined by the network operator and / or UE manufacturer, thereby ensuring a consistent and reliable process for reactive reporting. Example flows of reactive reporting flows according to one or more example embodiments are presented in Table 11 below. Table 11: Example Reactive Reporting Process

[0070] As shown in Table 11, the reactive reporting process on the UE side may include: the UE receiving a network query message (e.g., the network query message in Table 9), and evaluating the conditions defined by the query parameters included in the network query message. Therefore, the UE can generate a response message (e.g., the UE response message in Table 10) and send the response message to the NE.

[0071] According to an example embodiment, the UE can be configured to perform reactive reporting via RRC signaling. In this regard, the UE can be configured to generate an RRC message (which may be referred to herein as an "RRC AI / ML Applicability Message") containing information associated with the suitability of an AI / ML model, and then provide it to the NE to provide detailed suitability information and assist in network optimization. Example RRC messages according to one or more example embodiments are presented in Table 12 below. Table 12: Example RRC Messages

[0072] As shown in Table 12, the RRC message may include an ID associated with a feature of the AI / ML model (e.g., an integer between 1 and 1024), an applicability result (e.g., applicable, not applicable, unknown, etc.), and at least one evaluation parameter (e.g., channel conditions, UE mobility state such as stationary, moving, and high speed, UE battery level such as high, medium, and low, etc.). It is conceivable that, without departing from the scope of this disclosure, the RRC message may contain more or fewer information or parameters than those shown in Table 12. For example, the RRC message may also include applicability conditions associated with the applicability result, action suggestions associated with the applicability result, etc.

[0073] According to an example embodiment, the UE can be configured to perform reactive reporting via UAI signaling. In this regard, the UE can be configured to generate a UAI report message (which may be referred to herein as a "UAI AI / ML Applicability Report") containing information related to the suitability of the AI / ML model, and then provide it to the NE, thereby ensuring a comprehensive and clear suitability report. Example UAI messages according to one or more example embodiments are presented in Table 13 below. Table 13: Example UAI Messages

[0074] Similar to the RRC messages in Table 12, the UAI messages in Table 13 may also include features of the AI / ML model (e.g., integers between 1 and 1024) and applicability results (e.g., applicable, not applicable, unknown, etc.). Additionally, the UAI message may include an ID associated with the evaluated condition (e.g., an integer between 1 and 256) and information such as the type of deviation and the value of deviation associated with the performance of the AI / ML features relative to that condition. It is conceivable that, without departing from the scope of this disclosure, the UAI message may contain more or fewer information or parameters than those shown in Table 13. For example, the UAI message may also include performance metrics associated with the AI / ML model, the training process associated with the AI / ML model, inference accuracy associated with the AI / ML model, information associated with data security mechanisms and privacy regulations, etc.

[0075] According to example embodiments, the UE can be configured to perform reactive reporting via Media Access Control (MAC) Control Element (CE) signaling. In this regard, the UE can be configured to include information related to the suitability of the AI / ML model (which may be referred to herein as a “MAC CE AI / ML Suitability Query” or a “MAC CE message”) in the MAC CE (or a message containing a MAC CE) to ensure efficient and accurate suitability reporting. Examples of MAC CE messages according to one or more example embodiments are presented in Table 14 below. Table 14: Example MAC CE Message

[0076] Similar to the RRC messages in Table 12, the MAC CE messages in Table 14 may also include an ID (e.g., an integer between 1 and 1024) associated with the features of the AI / ML model, an applicability result (e.g., applicable, not applicable, unknown, etc.), and at least one evaluation parameter (e.g., channel conditions, UE mobility state such as stationary, moving, and high speed, UE battery level such as high, medium, and low, etc.). Additionally, the MAC CE message may include an identifier associated with the applicability query message provided by the NE and the associated response. It is conceivable that, without departing from the scope of this disclosure, the MAC CE message may contain more / fewer information or parameters than those shown in Table 14. For example, the MAC CE message may also include applicability reasons associated with the applicability result, information related to data security mechanisms and privacy regulations, etc.

[0077] According to an example embodiment, the UE can be configured to perform reactive reporting via physical (PHY) layer signaling. In this regard, the UE can include information associated with AI / ML model suitability into one or more PHY layer measurements (which may be referred to herein as “PHY AI / ML suitability signaling” or “PHY layer message”), thereby enhancing the accuracy and reliability of suitability reporting. PHY layer messages according to one or more example embodiments are presented in Table 15 below. Table 15: Example PHY Layer Messages

[0078] As shown in Table 15, the PHY layer message may contain an ID associated with the PHY layer measurement (e.g., an integer between 1 and 256), parameters associated with channel conditions (e.g., integers), the UE's mobility state (e.g., stationary, moving, high-speed, etc.), and information associated with the beam of the PHY layer signaling, such as the beam ID (e.g., an integer) and parameters associated with the beam strength (e.g., an integer between 0 and 100, etc.). It is conceivable that, without departing from the scope of this disclosure, the PHY layer message may contain more or fewer information or parameters than those shown in Table 15.

[0079] According to an example embodiment, the UE can be configured to perform reactive reporting via Non-Access Stratum (NAS) signaling. In this regard, the UE can be configured to receive a request for AI / ML suitability information from the NE via NAS signaling (referred to herein as a "NAS AI / ML suitability request" or "NAS request message"), then generate a response message containing the requested suitability information (referred to herein as a "NAS AI / ML suitability response," "NAS response message," or "NAS layer message"), and provide it to the NE, thereby ensuring timely and accurate suitability reporting and supporting efficient network management. Example NAS AI / ML request messages and example NAS response messages according to one or more example embodiments are presented in Tables 16 and 17 below, respectively. Table 16: Example NAS Request Message

[0080] As shown in Table 16, the NAS request message may contain an ID (e.g., an integer between 1 and 1024) associated with a feature of the AI / ML model, as well as multiple query parameters, including condition types (e.g., channel, mobility, battery, application, etc.) and thresholds to be used to evaluate those conditions. It is conceivable that, without departing from the scope of this disclosure, the NAS request message may contain more or fewer information or parameters than those shown in Table 16. Table 17: Example NAS Response Messages

[0081] As shown in Table 17, similar to the RRC messages in Table 12, the NAS response message may also include an ID associated with a feature of the AI / ML model (e.g., an integer between 1 and 1024), an applicability result (e.g., applicable, inapplicable, unknown, etc.), and at least one evaluation parameter (e.g., channel conditions, UE mobility state such as stationary, moving, and high speed, UE battery level such as high, medium, and low, etc.). It is conceivable that, without departing from the scope of this disclosure, the NAS response message may contain more or fewer information or parameters than those shown in Table 17.

[0082] In view of the foregoing, by applying the applicability framework of the example embodiments, the UE and / or NE can be configured to perform proactive reporting, reactive reporting, or a combination thereof, thereby ensuring timely and accurate exchange of AI / ML applicability information. Signaling enhancement for applicability reporting

[0083] As described above, the UE and / or NE can be configured to perform proactive reporting, reactive reporting, or a combination thereof via RRC signaling, MAC CE signaling, and UAI signaling. Further or alternative descriptions relating to these signaling and associated messages are provided below.

[0084] Table 18 below shows an example RRC message. Table 18: Example RRC Messages

[0085] The example RRC messages in Table 18 can be used in proactive reporting, reactive reporting, or both proactive and reactive reporting to ensure accurate and efficient transmission of applicability information between the UE and NE.

[0086] As shown in Table 18, this example RRC message may include: an ID associated with a feature of the AI / ML model (e.g., an integer between 1 and 1024), an applicability condition, an applicability result, an applicability reason, and an action suggestion. It is conceivable that, without departing from the scope of this disclosure, the RRC message may contain more or fewer information or parameters than those shown in Table 18. For example, the RRC message may also include parameters associated with the UE's state, parameters associated with the UE's capabilities, etc. By including a detailed description of these reporting metrics and parameters in the RRC message, it is possible to ensure that the UE's state and capabilities, as well as information associated with the AI / ML model's applicability, are comprehensively and transparently communicated to the network, thereby providing better insight and control.

[0087] As mentioned above, applicability conditions can refer to conditions under which an AI / ML model (or associated features) is applicable, such as channel conditions, UE speed, UE battery level, etc. By providing detailed applicability conditions in this RRC message, accurate applicability reporting can be ensured, which is necessary for network optimization. Example structures of applicability conditions according to one or more example embodiments are presented in Table 19 below. Table 19: Example Structure of Applicability Conditions

[0088] As shown in Table 19, the information associated with the applicability conditions may include: the ID associated with the applicability condition (e.g., an integer between 1 and 256), the type of applicability condition (e.g., channel, mobility, battery, application, etc.), and the threshold associated with the applicability condition (e.g., an integer).

[0089] On the other hand, applicability results can refer to information indicating whether an AI / ML function is applicable, inapplicable, or unknown. Providing clear applicability results in RRC messages simplifies network decision-making. Example structures of applicability results according to one or more example embodiments are presented in Table 20 below. Table 20: Example Structure of Applicability Results

[0090] Furthermore, applicability reasons can define the cause or origin of an applicability result, which can assist the Network Entity (NE) in network decision-making. Providing applicability reasons in the RRC message enhances the transparency and traceability of applicability measurements. Example structures for applicability reasons according to one or more example embodiments are presented in Table 21 below. Table 21: Example Structure of Applicability Reasons

[0091] As shown in Table 21, the information associated with the applicability reason may include: the ID associated with the reason for the applicability result (e.g., an integer between 1 and 256), the type associated with the reason (e.g., channel conditions, mobility status, battery level, application requirements, etc.), and a detailed description of the reason (e.g., a string description, etc.).

[0092] Furthermore, action recommendations may include one or more recommendations relating to one or more actions that can (or should) be taken based on applicability results, applicability reasons, and / or applicability conditions. By providing action recommendations in RRC messages, actionable insights can be provided, enabling NEs to dynamically optimize AI / ML functions. Example structures of action recommendations according to one or more example embodiments are presented in Table 22 below. Table 22: Example Structure of Action Suggestions

[0093] As shown in Table 22, the information associated with the action suggestion may include: the ID associated with the suggested action (e.g., an integer between 1 and 256), the type of the suggested action (e.g., activate the AI / ML model, deactivate the AI / ML model, update the AI / ML model, keep the AI / ML model, etc.), and the applicability reason (e.g., one or more parameters or information presented in Table 21, etc.).

[0094] The following description provides an explanation of MAC CE used for applicability reporting. According to an example embodiment, the UE can perform proactive reporting via MAC CE signaling. For example, the UE can generate a MAC CE message (which may be referred to herein as a "proactive MAC CE message" or "MAC CE proactive applicability report") during proactive reporting. This MAC CE message can be designed to enable the UE to proactively report AI / ML applicability based on predefined conditions, thereby ensuring that the network is always notified of the AI / ML applicability status and enhancing responsiveness. Example proactive MAC CE messages according to one or more example embodiments are presented in Table 23 below. Table 23: Example of an active MAC CE message

[0095] As shown in Table 23, the proactive MAC CE message may include: an ID associated with a feature of the AI / ML model (e.g., an integer between 1 and 1024), information associated with applicability conditions, information associated with applicability results, and information associated with applicability reasons. This information has already been described above with reference to Tables 19-21; therefore, for the sake of brevity, repetitive descriptions associated with it can be omitted below.

[0096] According to example embodiments, the UE can perform reactive reporting via MAC CE signaling. For example, the UE can generate a MAC CE message (which may be referred to herein as a "reactive MAC CE message" or "MAC CE reactive applicability report") during reactive reporting. This MAC CE message can be designed to enable the UE to report AI / ML applicability in response to network queries, thereby allowing the network (or NE) to request specific applicability information as needed and improve decision-making accuracy. Example reactive MAC CE messages according to one or more example embodiments are presented in Table 24 below. Table 24: Example Reactive MAC CE Message

[0097] As shown in Table 24, the reactive MAC CE message can include: the ID associated with the AI / ML feature, information associated with the applicability result, and information associated with the applicability reason. The associated descriptions have already been provided above with reference to Tables 19-21 and 23; therefore, for the sake of brevity, repetitive descriptions associated with it can be omitted below.

[0098] It is conceivable that, without departing from the scope of this disclosure, the MAC CE message may contain more or fewer information or parameters than those shown in Tables 23-24. For example, in some example implementations, the MAC CE message may also include mechanisms designed to protect user data and ensure compliance with applicable privacy regulations. By including such information or parameters in the MAC CE message, potential privacy issues can be addressed, and data security in the AI / ML applicability report can be enhanced, while ensuring compliance with applicable privacy regulations and protecting user data.

[0099] The following description provides an explanation of the UAI signaling used for suitability reporting. According to an example embodiment, the UE can be configured to provide the NE with detailed information related to AI / ML suitability (e.g., suitability conditions, real-time performance metrics, model status, etc.) via UAI signaling.

[0100] For example, the UE can generate a report message containing applicability information (which may be referred to herein as a "UAI message" or "UAI AI / ML applicability report") and then provide it to the NE, thereby providing granular UAI reports in real time (or near real time). Example UAI messages according to one or more example embodiments are presented in Table 25 below. Table 25: Example UAI Messages

[0101] As shown in Table 25, the UAI message may contain information associated with one or more performance metrics (presented as "UAI Performance Metrics"), information associated with the training process of the AI / ML model (presented as "UAI Training Progress"), and information associated with the inference accuracy of the AI / ML model ("UAI Inference Accuracy"). It is conceivable that, without departing from the scope of this disclosure, the UAI message may contain more or fewer information or parameters than those shown in Table 25. For example, in some example implementations, the UAI message may also include mechanisms designed to protect user data and ensure compliance with relevant privacy regulations. By including such information or parameters in the UAI message, potential privacy issues can be addressed, and data security in the AI / ML suitability report can be enhanced, while ensuring compliance with relevant privacy regulations and protecting user data.

[0102] Performance metrics can define the real-time (or near-real-time) performance associated with an AI / ML model, such as latency, accuracy, and power consumption. By including performance metrics in UAI messages, real-time (or near-real-time) insights into model performance are provided, enabling timely adjustments to the AI / ML model if needed. Example structures of performance metrics, based on one or more example embodiments, are presented in Table 26 below. Table 26: Example Structure of Performance Metrics

[0103] As shown in Table 26, the information associated with a performance metric may include: the ID associated with the performance metric (e.g., an integer between 1 and 256), the type of the performance metric (e.g., latency, accuracy, power consumption), and the value associated with the performance metric (e.g., an integer).

[0104] Furthermore, the training process of an AI / ML model can define the model's ready state. Providing the training process of the AI / ML model in the UAI message helps the NE understand the current capabilities of the AI / ML model. Example structures of the training process according to one or more example embodiments are presented in Table 27 below. Table 27: Example Structure of the Training Process

[0105] As shown in Table 27, information associated with the training process may include: IDs associated with features of the AI / ML model (e.g., integers between 1 and 1024), and the percentage of the training process (e.g., integers between 0 and 100).

[0106] Furthermore, the inference accuracy of an AI / ML model can define model performance. Providing the inference accuracy of an AI / ML model in the UAI message helps the NE determine the inference performance of the AI / ML model, which is crucial for maintaining high quality of service. Example structures for inference accuracy based on one or more example embodiments are presented in Table 28 below. Table 28: Example Structure of Reasoning Accuracy

[0107] As shown in Table 28, information associated with inference accuracy may include: the ID associated with the features of the AI / ML model (e.g., an integer between 1 and 1024), and the percentage of inference accuracy (e.g., an integer between 0 and 100). Dynamic threshold adjustment

[0108] According to example embodiments, UEs and / or NEs can be configured to dynamically adjust thresholds associated with the suitability of AI / ML models based on real-time (or near-real-time) conditions, thereby ensuring optimal performance and suitability of AI / ML functions under changing network and environmental conditions, and enhancing the reliability and efficiency of AI / ML deployment in the network. Example algorithms for dynamic threshold adjustment according to one or more example embodiments are presented in Table 29 below. Table 29: Example Algorithms for Dynamic Threshold Adjustment

[0109] As shown in Table 29, the dynamic threshold adjustment includes: inputting real-time (or near-real-time) conditions (e.g., network load, UE mobility, environmental factors, etc.), processing (e.g., evaluating, assessing, analyzing, etc.) the input conditions to calculate a new threshold (presented in this paper as "adjusted threshold"), and outputting the new threshold.

[0110] According to the example embodiments, the UE and / or NE can be configured to adjust the threshold based on one or more parameters predefined by the network operator and / or UE manufacturer (referred to herein as "threshold adjustment parameters"), thereby ensuring that the threshold adjustment is accurate and context-dependent. Furthermore, using context-specific parameters and rule-based adjustment intervals enables a degree of granularity and responsiveness. Example structures associated with the threshold adjustment parameters according to one or more example embodiments are presented in Table 30 below. Table 30: Example structure of threshold adjustment parameters

[0111] As shown in Table 30, the threshold adjustment parameter may include: an ID associated with the threshold adjustment parameter (e.g., an integer between 1 and 256 specific to each parameter), a parameter type (e.g., network load, UE mobility, environmental factors, etc.), a threshold associated with the threshold adjustment parameter (e.g., an integer, etc.), and an adjustment interval associated with the time interval used to adjust the threshold.

[0112] According to example embodiments, when adjusting a threshold, the UE and / or NE can be configured to report the adjusted threshold, thereby ensuring the transparency and traceability of performance adjustments. For example, the UE and / or NE can generate a reporting message (which may be referred to herein as an "adjusted threshold report") containing information associated with the adjusted threshold, and then provide the reporting message to the NE and / or UE, thereby ensuring that the adjustment is transmitted and archived in real time (or near real time) and allows for immediate analysis and further optimization when needed. Example adjusted threshold reporting messages according to one or more example embodiments are presented in Table 31 below. Table 31: Example Adjusted Threshold Report

[0113] As shown in Table 31, the adjusted threshold report may include: an ID associated with the threshold adjustment parameter (e.g., an integer between 1 and 256 specific to each parameter), the adjusted threshold as the new threshold after adjustment, and a timestamp associated with the time the threshold was adjusted. Real-time monitoring and reporting

[0114] According to example embodiments, the UE and / or NE can be configured to monitor and report AI / ML-related parameters in real-time (or near real-time). For example, the UE and / or NE can continuously (or periodically) monitor and report one or more AI / ML performance metrics, thereby ensuring the reliability and accuracy of AI / ML functionality and providing a proactive approach to maintain optimal AI / ML performance. Furthermore, real-time (or near real-time) monitoring and reporting can facilitate immediate decision-making and adaptive responses to performance issues, thereby enhancing overall system resilience.

[0115] Example real-time monitoring processes based on one or more example embodiments are presented in Table 32 below. Table 32: Example Real-time Monitoring Process

[0116] As shown in Table 32, the real-time monitoring process may include a list of metrics to be monitored (e.g., latency, accuracy, power consumption, etc.), a frequency for defining the time intervals used to monitor the metrics in the list, and steps involved in the monitoring process (e.g., collecting metrics, logging data, generating reports, etc.). Examples of report messages containing real-time monitoring information or data (which may be referred to herein as "real-time report messages") according to one or more example embodiments are presented in Table 33 below. Table 33: Example Real-Time Reporting Messages

[0117] As shown in Table 33, the real-time report message may include: an ID associated with the monitored performance metric (e.g., an integer between 1 and 256 unique to each performance metric), a metric value that may contain an integer associated with the monitored performance metric, and a timestamp associated with the time the performance metric was monitored and recorded. AI / ML model selection

[0118] In addition to identifying and reporting the suitability of AI / ML models, the suitability framework and mechanisms of the example implementation can also be used to select one or more appropriate AI / ML models based on one or more conditions.

[0119] Non-limiting example use cases for selecting multiple AI / ML models according to one or more example embodiments are presented in Table 34 below. Table 34: Example Use Cases

[0120] As shown in Table 34, the AI / ML model selection process may include: inputting multiple applicability conditions, processing (e.g., evaluating, assessing, checking, etc.) each of the applicability conditions, and then selecting an AI / ML model. The following sections provide a description of each of the applicability conditions and the operations or steps used to process them.

[0121] As shown in Table 34, the applicability conditions may include: UE location type (e.g., urban, rural, etc.), UE speed (e.g., high speed, low speed, etc.), UE capabilities (e.g., UE storage capacity, UE processing capacity, UE battery level, etc.), network conditions (e.g., network congestion level, network processing load, etc.), constraints on user privacy settings, UE model version and network model version, time factors such as time of day and season, environmental factors or conditions, service type or requirements (e.g., Ultra Reliable Low Latency Communication (URLLC), Enhanced Mobile Broadband (eMBB), etc.), Quality of Service (QoS) requirements or parameters, performance metrics used for performance degradation checks, and dataset characteristics (e.g., dataset size, dataset quality, dataset bias, dataset obsolescence, etc.).

[0122] The applicability of an AI / ML model can depend on the location type of the UE. For example, the beamforming AI / ML model may differ for a UE located in a densely populated urban location versus a UE located in a low-density rural location. Therefore, if the UE is located in a rural location, a beamforming AI / ML model developed for densely populated urban locations may not be applicable. Thus, in the example in Table 34, when the applicability condition includes the UE's location type, the UE and / or NE can evaluate whether the location type is urban or rural and then select the appropriate AI / ML model based on the UE's location type. For example, based on the determination that the UE is located in a rural location, the UE and / or NE can select an AI / ML model developed for rural locations, and vice versa.

[0123] Furthermore, the applicability of the AI / ML model can also depend on the UE's speed. For example, the channel estimation AI / ML model may differ when the UE is at a high speed versus when it is at a low speed. Therefore, in the example in Table 34, when the applicability condition includes the UE's speed, the UE and / or NE can evaluate whether the UE's speed is greater than a predefined threshold (e.g., a threshold indicating whether the UE is at a high / low speed) and then select an appropriate AI / ML model based on the UE's speed. For example, based on determining that the UE is at a high speed, the UE and / or NE can select an AI / ML model developed for high-speed UEs, and vice versa.

[0124] Furthermore, the suitability of AI / ML models can vary depending on the capabilities of the UE (e.g., storage capacity associated with available data storage in the UE, processing capabilities such as available computing power or processing units, and battery level associated with the UE's power consumption). Therefore, when suitability conditions include one or more of these UE capabilities, the UE and / or NE can compare these UE capabilities with associated thresholds and then select the appropriate AI / ML model optimized according to the UE's capabilities.

[0125] Furthermore, the applicability of AI / ML models can be affected by network conditions such as network congestion levels and network processing load. Therefore, when the applicability criteria include one or more of these network conditions, the UE and / or NE can compare these network conditions with associated thresholds and then select the appropriate AI / ML model optimized according to the network conditions.

[0126] Furthermore, the applicability of an AI / ML model can also depend on privacy constraints imposed by user privacy settings. For example, an AI / ML model that requires access to the user's location may not be suitable for a user who has disabled location sharing in their privacy settings. Therefore, when applicability conditions include user privacy settings, the UE and / or NE can consider whether any constraints arise from these settings and then select an appropriate AI / ML model accordingly. For instance, based on determining the constraints or limitations imposed by user privacy settings, the UE and / or NE can select a privacy-preserving AI / ML model, and vice versa.

[0127] Furthermore, the suitability of AI / ML models can also depend on the model versions of the UE and the network. For example, the network may need to adjust the parameters of the AI / ML models on the UE to optimize their performance for different scenarios or conditions. Therefore, when suitability conditions include the model versions of the UE and the network, the UE and / or NE can evaluate the model versions to determine if the UE model version is consistent with the network model version, and then select an appropriate AI / ML model based on this. For example, if it is determined that the UE model version is inconsistent with the network model version, the UE and / or NE can select an AI / ML model compatible with either the UE model version or the network model version.

[0128] Furthermore, the applicability of AI / ML models can also depend on time and environmental factors, such as time of day, season, traffic forecasts, demand forecasts, and weather forecasts. Therefore, when applicability conditions include one or more time and / or environmental conditions, the UE and / or NE can evaluate these conditions and then select the AI / ML model optimized for those time and / or environmental conditions.

[0129] Furthermore, the applicability of an AI / ML model can also depend on service requirements or QoS requirements. For example, an AI / ML model developed for URLCC services may not be applicable to eMBB services. Therefore, when the applicability conditions include one or more service requirements and / or QoS requirements, the UE and / or NE can determine the type of service (e.g., URLLC, eMBB, etc.) and / or QoS requirements (e.g., high, medium, low, etc.) and then select an appropriate AI / ML model based on this.

[0130] Furthermore, the applicability of AI / ML models may change over time due to changes in the environment or user needs. For example, the accuracy of AI / ML models used for channel prediction may decrease as channel conditions change. Therefore, when applicability conditions include one or more performance metrics, the UE and / or NE can check whether those performance metrics have degraded over time and then select an appropriate AI / ML model accordingly. For example, based on the determination that the performance metric has degraded over time, the UE and / or NE can select an updated AI / ML model.

[0131] Furthermore, the suitability of an AI / ML model can also depend on dataset characteristics, such as dataset size, dataset quality, dataset bias, and dataset staleness. Therefore, when suitability conditions include one or more dataset characteristics, the UE and / or NE can evaluate these characteristics by comparing them to associated thresholds, and then select an appropriate AI / ML model optimized for the dataset.

[0132] It is conceivable that, without departing from the scope of this disclosure, the UE and / or NE may obtain any other suitable data (e.g., real-time data, etc.) and then select one or more AI / ML models based on this data. When selecting an AI / ML model, the UE and / or NE may generate a report message containing information associated with the selected AI / ML model (e.g., the ID of the selected model, the name of the selected model, thresholds associated with the selected model, etc.), and then provide this report message to the NE and / or UE.

[0133] In view of the above, the example embodiments of this disclosure provide the following framework and mechanism for effectively and efficiently evaluating the suitability of AI / ML models, selecting appropriate AI / ML models, reporting AI / ML model suitability information, monitoring parameters / indicators associated with AI / ML model suitability in real time (or near real time), and dynamically adjusting thresholds associated with AI / ML model suitability.

[0134] It is conceivable that the example parameters and contents presented in Tables 1-34 above are merely examples of possible embodiments or use cases, and the scope of this disclosure should not be limited thereto. Example operations and use cases

[0135] The following sections will describe example operations that implement one or more features of the applicability framework described above with reference to Tables 1-33, as well as example use cases associated with them.

[0136] Figure 2 A block diagram is shown of an example method 200 for transmitting AI / ML model suitability information according to one or more example embodiments. One or more operations of method 200 may be performed by a device (e.g., UE, base station, etc.) to transmit AI / ML model suitability to another device (e.g., base station, UE, etc.). For example, the operations may be performed by at least one processor of the device when executing computer-readable instructions stored in the device's storage. Reference will be made below. Figure 9 This will provide further description of the processor and storage.

[0137] like Figure 2As shown, in operation S210, the device can be configured to evaluate the suitability of an AI / ML model related to a mobile telecommunications network. According to an example embodiment, the device can evaluate the suitability of the AI / ML model by comparing at least one suitability condition with a predefined threshold. This condition may include one or more of the following: network conditions (e.g., network load, channel conditions, etc.), device (e.g., UE) mobility, and device (e.g., UE) battery level. Therefore, based on determining that the suitability condition meets the conditions defined by the predefined threshold (e.g., battery level is higher than the predefined threshold, indicating that the battery level meets the "high battery level" condition, etc.), the device can determine that the AI / ML model is suitable. Otherwise, based on determining that the suitability condition does not meet the conditions defined by the predefined threshold, the device can determine that the AI / ML model is unsuitable.

[0138] According to an example embodiment, the device can evaluate the applicability of an AI / ML model by determining whether the AI / ML model is correctly configured. For example, the device can compare at least one applicability condition (e.g., network conditions, UE mobility, UE battery level, etc.) with an associated predefined threshold. Based on the determination that the applicability condition meets the conditions defined by the predefined threshold, the device can determine that the AI / ML model is correctly configured. Otherwise, based on the determination that the applicability condition does not meet the conditions defined by the predefined threshold, the device can determine that the AI / ML model is not correctly configured. Therefore, based on the determination that the AI / ML model is correctly configured, the device can determine that the AI / ML model is applicable. Conversely, based on the determination that the AI / ML model is not correctly configured, the device can determine that the AI / ML model is inapplicable. In this way, the device can assess the availability and readiness of the AI / ML model while ensuring that the AI / ML model is correctly configured and free from internal problems. Ultimately, the reliability of functional applicability can be enhanced.

[0139] Subsequently, in operation S220, the device can be configured to generate a report message containing the evaluated applicability. According to an example embodiment, the report message may contain at least one of the following: an RRC message, a UAI message, a MAC CEW message, a PHY layer message, and a NAS layer message.

[0140] According to an example embodiment of the report message containing an RRC message, the RRC message may contain at least one of the following: applicability conditions for the AI / ML model, applicability results indicating whether the features of the AI / ML model are applicable, applicability reasons associated with the reasons for the applicability results, action recommendations associated with at least one NE that should take action based on the applicability results, parameters associated with the state of the UE, and parameters associated with the capabilities of the UE.

[0141] According to an example embodiment of the report message containing a MAC CE message, the MAC CE message may contain at least one of the following: an ID associated with a feature of the AI / ML model, an applicability result indicating whether the feature of the AI / ML model is applicable, an applicability reason associated with the reason for the applicability result, and a mechanism (or information associated therewith) designed to protect user data and ensure compliance with relevant privacy regulations.

[0142] According to an example embodiment of the report message containing a UAI message, the UAI message may contain at least one of the following: an ID associated with a feature of the AI / ML model, an applicability result indicating whether the feature of the AI / ML model is applicable, an applicability reason associated with the reason for the applicability result, and a mechanism (or information associated therewith) designed to protect user data and ensure compliance with relevant privacy regulations.

[0143] Subsequently, in operation S230, the device can be configured to provide the reporting message to at least one NE of the mobile telecommunications network. For example, the device can provide the reporting message to at least one NE via RRC signaling, UAI signaling, MAC CE signaling, PHY layer signaling, and NAS layer signaling.

[0144] The above has already provided further descriptions related to the evaluation of the suitability of AI / ML models, the generation of report messages, the delivery of report messages, and the types and example content of report messages, with reference to at least a portion of Tables 1-34. Therefore, for the sake of brevity, the repetitive descriptions associated with these descriptions can be omitted below.

[0145] According to an example embodiment, the device can be configured to autonomously perform evaluations of the suitability of AI / ML models, generation of report messages, and provision of report messages based on triggering conditions. In some example implementations, the triggering conditions may include predefined activation conditions associated with a function of at least one of the device and the NE. Example use cases associated with this will be described below.

[0146] Figure 3 This diagram illustrates a flowchart of an example use case associated with proactive reporting, according to one or more example embodiments; the example use case may involve Figure 2 One or more operations of method 200 in the example. Furthermore, for the purposes of description only, it is assumed that in this use case, UE 110 has received one or more triggering conditions from at least one NE (e.g., base station 120-1, etc.).

[0147] like Figure 3As shown, in step 1, UE 110 can evaluate at least one triggering condition to determine whether an evaluation of the suitability of the AI / ML model should be triggered (e.g., whether the desired conditions are met, whether the desired AI / ML is correctly configured, etc.). Therefore, based on the determination that an evaluation of the suitability of the AI / ML model should be triggered, in step 2, UE 110 can evaluate or determine the suitability of the AI / ML model. Subsequently, in step 3, UE 110 can generate a report message and send the report message to base station 120-1 (e.g., via RRC signaling, UAI signaling, etc.).

[0148] Upon receiving an active reporting message from UE 110, in step 4, base station 120-1 may forward the active reporting message to NAS 120-2. Subsequently, in step 5, NAS 120-2 may send data related to AI / ML model applicability to AI / ML model server 120-3. Next, in step 6, AI / ML model server 120-3 may provide a response message to NSA 120-2 to acknowledge receipt of the data. Therefore, in step 7, NAS 120-2 may provide a response message to base station 120-1 to acknowledge receipt of the active reporting message. Similarly, in step 8, base station 120-1 may provide a response message to UE 110 to acknowledge receipt of the active reporting message.

[0149] In light of the above, UE 110 can automatically initiate an AI / ML model suitability evaluation process and report AI / ML model suitability to NEs (e.g., base station 120-1, NAS 120-2, AI / ML model server 120-3). Further descriptions of the process and related messages have already been provided above with reference to the "proactive reporting" mechanism; therefore, for the sake of brevity, repetitive descriptions can be omitted below.

[0150] According to an example embodiment, the device can be configured to perform an evaluation of the suitability of an AI / ML model, generate a report message, and provide a report message in response to receiving a query message from at least one NE. In some example implementations, the query message may contain information associated with at least one of network configuration and network conditions.

[0151] Figure 4 A flowchart illustrating an example use case associated with reactive reporting, based on one or more example embodiments, is provided. This example use case may involve... Figure 2 One or more operations of method 200 in the code. Furthermore, Figure 4 One or more steps in the example use case can be with Figure 3One or more steps in the example use case are similar. The example use cases associated with it will be described below.

[0152] like Figure 4 As shown, in step 1, base station 120-1 can provide a query message to UE 110 (e.g., via RRC signaling, etc.). Therefore, UE 110 can evaluate the applicability of the AI / ML model (in step 2) and send a reactive report message to base station 120-1 (in step 3).

[0153] In this regard, Figure 4 Steps 2-8 in the text can be compared with Figure 3 Steps 2-8 in [the original text] are similar. These steps can be compared with... Figure 3 The steps differ from those in the example use case, in that UE 110 can be configured to generate reactive reporting messages (rather than proactive reporting messages) and send them to the NE. Furthermore, Figure 4 Steps 2-8 in the example are triggered in response to receiving a query message, not as... Figure 3 As shown, it is automatically initiated based on the evaluation of the triggering conditions.

[0154] In light of the above, UE 110 can, in response to receiving a query message from base station 120-1, initiate an AI / ML model suitability evaluation process and report the AI / ML model suitability to the NE (e.g., base station 120-1, NAS 120-2, AI / ML model server 120-3). Further descriptions of the processes and messages involved in the "reactive reporting" mechanism have already been provided above; therefore, for the sake of brevity, repetitive descriptions can be omitted below.

[0155] According to an example embodiment, the device can be configured to dynamically adjust at least one predefined threshold associated with the suitability of an AI / ML model based on real-time (or near-real-time) data. For example, the device can adjust a predefined threshold associated with a triggering condition (e.g., a predefined threshold for evaluating the triggering condition). Additionally or alternatively, the device can adjust a predefined threshold associated with suitability conditions (e.g., a predefined threshold for evaluating the suitability of an AI / ML model).

[0156] Figure 5 A block diagram is shown of an example method 500 for dynamic threshold reporting according to one or more example embodiments. One or more operations of method 500 may be performed by a device (e.g., UE, base station, etc.). Furthermore, one or more operations of method 500 may be independent of / dependent on... Figure 2Method 200 is executed. Moreover, for purposes of description only, it may be assumed that method 500 can be executed to adjust a predefined threshold associated with the applicability conditions.

[0157] like Figure 5 As shown, in operation S510, the device can be configured to acquire real-time data. In this regard, the real-time data may include at least one of the following: current network conditions (e.g., network load, network congestion level, etc.), current UE mobility (e.g., static, high-speed movement, low-speed movement, etc.), and environmental factors (e.g., season, etc.).

[0158] Subsequently, in operation S520, the device can be configured to calculate a new threshold for the applicability conditions based on the acquired real-time data. Next, in operation S530, the device can be configured to generate a report message containing information associated with the new threshold. Therefore, in operation S540, the device can be configured to provide the report message.

[0159] Figure 6A and Figure 6B Each example illustrated is a flowchart of an example use case associated with dynamic threshold adjustment, according to one or more example embodiments. The example use case may involve... Figure 5 One or more operations of method 500 in the middle.

[0160] First refer to Figure 6A This is related to the example use case where the new threshold is calculated by UE 110 and applied / adjusted by base station 120-1.

[0161] like Figure 6A As shown, in step 1, UE 110 can be configured to obtain real-time (or near-real-time) network condition data from base station 120-1. Therefore, in step 2, UE 110 can evaluate the network conditions, and in step 3, UE 110 can calculate a new threshold. Subsequently, in step 4, UE 110 can provide base station 120-1 with a report message containing information associated with the new threshold. Therefore, in step 5, base station 120-1 can adjust or modify a predefined threshold to apply the new threshold.

[0162] Next, refer to Figure 6B The result of the calculation of the new threshold is calculated by base station 120-1 and the new threshold is associated with the example use case of UE 110 application / adjustment.

[0163] like Figure 6BAs shown, in step 1, base station 120-1 can be configured to obtain real-time (or near-real-time) UE mobility data from UE 110. Therefore, in step 2, base station 120-1 can evaluate the UE mobility, and in step 3, base station 120-1 can calculate a new threshold. Subsequently, in step 4, base station 120-1 can provide UE 110 with a report message containing information associated with the new threshold. Therefore, in step 5, UE 110 can adjust or modify a predefined threshold to apply the new threshold.

[0164] In light of the above, the device can dynamically adjust the thresholds associated with the suitability of the AI / ML model based on real-time (or near-real-time) data. Further descriptions of the processes and messages involved have already been provided above with reference to at least a portion of Tables 1-34; therefore, for the sake of brevity, repetitive descriptions may be omitted below.

[0165] According to an example embodiment, the device can be configured to perform real-time monitoring and reporting of performance metrics associated with the suitability of AI / ML models.

[0166] Figure 7 A block diagram is shown of an example method 700 for real-time monitoring and reporting according to one or more example embodiments. One or more operations of method 700 may be performed by a device (e.g., UE, base station, etc.). Furthermore, one or more operations of method 700 may be independent of / dependent on... Figure 2 Method 200 and / or Figure 5 Method 500 in the code is executed.

[0167] like Figure 7 As shown, in operation S710, the device can be configured to monitor at least one performance metric of the AI / ML model. This performance metric may include, for example, latency, accuracy, power consumption, etc.

[0168] Subsequently, in operation S720, the device can be configured to generate a report message that includes at least one of the following: an ID associated with the monitored performance metric, a metric value associated with the monitored performance metric, and a timestamp associated with the time the performance metric was monitored. Therefore, in operation S730, the device can be configured to provide the report message.

[0169] Figure 8 A flowchart illustrating an example use case associated with real-time monitoring and reporting, according to one or more example embodiments, is provided. This example use case may involve... Figure 7 One or more operations of method 700 in the middle.

[0170] like Figure 8As shown, in step 1, UE 110 can monitor at least one performance metric of the AI / ML model. Subsequently, in step 2, UE 110 can record or log data associated with the monitored performance metric. Next, in step 3, UE 110 can generate a report message containing information associated with the monitored performance metric, and then provide the report message to base station 120-1.

[0171] Therefore, in step 4, base station 120-1 can forward the report message to NAS 120-2. Furthermore, in step 5, NAS 120-2 can send data related to the performance of the AI / ML model to AI / ML model server 120-3. Subsequently, in step 6, AI / ML model server 120-3 can provide a response message to NSA 120-2 to acknowledge receipt of the performance data. Therefore, in step 7, NAS 120-2 can provide a response message to base station 120-1 to acknowledge receipt of the report message. Similarly, in step 8, base station 120-1 can provide a response message to UE 110 to acknowledge receipt of the report message.

[0172] In view of the above, UE 110 can monitor the real-time performance metrics of the AI / ML model and report them to the NE (e.g., base station 120-1, NAS 120-2, AI / ML model server 120-3). Further descriptions relating to the processes and messages involved in Tables 1-34 have already been provided above with reference to at least a portion thereof; therefore, for the sake of brevity, repetitive descriptions may be omitted below.

[0173] According to an example embodiment, the device can be configured to select one or more AI / ML models. For example, the device can obtain real-time data (e.g., current network conditions, current UE capabilities, environmental factors, etc.) and then select an AI / ML model based on this data (e.g., selecting an AI / ML model that is correctly configured and free from internal problems, selecting an AI / ML mode optimized for current network conditions / current UE capabilities, etc.). When selecting an AI / ML model, the device can generate a report message containing information associated with the selected AI / ML model and then provide the report message (e.g., to the UE, to the NE, etc.).

[0174] In light of the above, the device can select an appropriate AI / ML model based on real-time and dynamic conditions and then report it to other network components. Further descriptions relating to the processes and messages involved in Tables 1-34 have already been provided above with reference to at least a portion thereof; therefore, for the sake of brevity, repetitive descriptions can be omitted below. Examples of hardware components

[0175] One or more components of the system in the example embodiment (e.g., UE, base station, etc.) and their associated operations can be implemented in one or more device or hardware components. For example, one or more components / operations of the base station can be implemented in one or more servers, etc.

[0176] The following description will provide a device in which example embodiments can be implemented. It is conceivable that the above references... Figures 1 to 8 The described one or more features, operations, and methods can be performed by the device. For example, one or more operations or methods can be performed by at least one processor of the device when executing machine-readable or computer-readable instructions stored in the memory or storage component of the device.

[0177] Figure 9 An embodiment of device 900 is shown. For example... Figure 9 As shown, device 900 may include processor 910, memory 920, storage component 930, input component 940, output component 950, communication interface 960, and bus 970.

[0178] As used herein, processor 910 refers to any type of computing circuit that may include hardware and software elements. Processor 910 may be implemented as a multi-core processor, a single-core processor, or a combination of one or more multi-core processors and / or one or more single-core processors, a distributed processing system, etc. Processor 910 may be a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), an application-specific integrated circuit (ASIC), or other types of processing components.

[0179] Memory 920 includes a non-transitory computer-readable medium. Memory 920 includes random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic storage, and / or optical storage) that stores information and / or instructions for use by processor 910. Memory 920 includes machine-readable instructions executable by processor 910. When executed by processor 910, these machine-readable instructions cause processor 910 to perform one or more method steps of the embodiments described above.

[0180] Storage component 930 stores information and / or software related to the operation and use of device 900. For example, storage component 930 may include hard disks (e.g., magnetic disks, optical disks and / or magneto-optical disks, and / or solid-state drives), compact discs (CDs), digital universal discs (DVDs), floppy disks, cassette tapes, magnetic tapes, and / or other types of non-transitory machine-readable media and corresponding drives.

[0181] Input component 940 is configured to receive information, such as user input. For example, input component 940 may include, but is not limited to, a touchscreen display, keyboard, keypad, mouse, button, switch, and / or microphone. Additionally or alternatively, input component 940 may include sensors for sensing information (e.g., Global Positioning System (GPS), accelerometer, gyroscope, and / or actuator).

[0182] Output component 950 is configured to provide output information from device 900. For example, output component 950 may be, but is not limited to, a display, a speaker, instructions to an external device, and / or one or more light-emitting diodes (LEDs).

[0183] Communication interface 960 is an interface that provides communication connections to other devices, such as external and internal devices. Connections via communication interface 960 can be wired, wireless, or a combination of wired and wireless connections, and can be direct or indirect connections via a communication network existing between device 900 and other devices. In other words, the standard of communication interface 960 is unrestricted.

[0184] Bus 970 serves as an interconnection between processor 910, memory 920, storage component 930, input component 940, output component 950, and communication interface 960 in device 900. Bus 970 may include wired or wireless interconnection.

[0185] Figure 9 The number and arrangement of components shown are provided as an example. In practice, device 900 may include additional components, fewer components, different components, or components with... Figure 9 The components shown are arranged differently. Additionally or alternatively, a group of components of device 900 (e.g., one or more components) may perform one or more functions described as being performed by another group of components of device 900. Furthermore, one or more method steps described in any of the embodiments may be performed using multiple devices 900 communicating with each other. Various aspects of the embodiments

[0186] The above reference is worth considering. Figures 1 to 9 The described example embodiments are merely examples of possible embodiments of this disclosure and are not intended to limit or restrict the scope of this disclosure.

[0187] Specifically, while the foregoing disclosure provides examples and descriptions, it is not intended to be exhaustive or to limit the implementation to the precise form disclosed. Modifications and variations can be made based on the foregoing disclosure, or derived from the practice of the implementation.

[0188] Some embodiments may relate to devices (e.g., network nodes, etc.), systems, methods, and / or computer-readable media at any possible level of technical detail integration. Furthermore, one or more of the foregoing components may be implemented as instructions stored on a computer-readable medium and executable by at least one processor (and / or may include at least one processor). The computer-readable medium may include one or more computer-readable non-transitory storage media having computer-readable program instructions on it for causing the processor to perform operations.

[0189] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital universal disc (DVD), memory sticks, floppy disks, mechanical encoding devices such as punched cards or recessed protrusions on which instructions are recorded, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through metal wires.

[0190] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network such as the Internet, local area network, wide area network, and / or wireless network. This network may include copper cables, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. Network adapter cards or network interfaces in each computing / processing device receive the computer-readable program instructions from the network and forward them to a computer-readable storage medium within the corresponding computing / processing device.

[0191] Computer-readable program code / instructions used to perform operations can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and procedural programming languages ​​such as the "C" programming language or similar programming languages.

[0192] The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet provided by an Internet service provider). In some embodiments, electronic circuits, such as those including programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can execute the computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuits for performing aspects or operations.

[0193] These computer-readable program instructions may be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, form means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium capable of directing a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of manufacture comprising the instructions that implement aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0194] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be executed on the computer, other programmable apparatus, or other equipment, thereby producing a process implemented by the computer, such that the instructions executed on the computer, other programmable apparatus, or other equipment perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0195] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, the blocks in the flowcharts or block diagrams may represent portions of modules, segments, or instructions, including one or more executable instructions for implementing the specified logical function. Methods, computer systems, and computer-readable media may include additional blocks, fewer blocks, different blocks, or blocks arranged differently from those depicted in the figures. In some alternative implementations, the functions indicated in the blocks may occur in a different order than indicated in the figures. For example, depending on the functions involved, two blocks shown consecutively may actually be executed simultaneously or substantially simultaneously, or these blocks may sometimes be executed in reverse order. It should also be noted that the blocks in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified function or action, or performs a combination of dedicated hardware and computer instructions.

[0196] It is evident that the systems and / or methods described herein can be implemented using various forms of hardware, firmware, or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not limited to these implementations. Therefore, since this document describes the operation and behavior of the systems and / or methods without reference to specific software code, it is understood that software and hardware can be designed to implement the system and / or methods based on the descriptions herein.

[0197] In view of the foregoing, various other corresponding aspects and features of embodiments of this disclosure may be defined by the following items: Project [1]: An apparatus configured to: evaluate the suitability of an artificial intelligence (AI) / machine learning (ML) model associated with a mobile telecommunications network; generate a report message including the evaluated suitability; and provide the report message to at least one network element (NE) of the mobile telecommunications network. Project [2]: The device according to Project [1], wherein the device is configured to autonomously perform, based on triggering conditions, evaluation of the suitability of AI / ML models, generation of report messages, and provision of report messages, and wherein the triggering conditions include predefined activation conditions associated with at least one of the following: the device and the NE. Item [3]: A device according to any one of Items [1]-[2], wherein the device is configured to: in response to receiving a query message from at least one NE, perform: evaluation of the suitability of an AI / ML model, generation of a report message, and provision of a report message, and wherein the query message includes information associated with at least one of: network configuration and network conditions. Item [4]: ​​A device according to any one of Items [1]-[3], wherein the reported message includes at least one of the following: Radio Resource Control (RRC) message, User Equipment Assistance Information (UAI) message, Media Access Control (MAC) Control Element (CE) message, Physical PHY layer message, and Non-Access Stratum (NAS) layer message. Project [5]: According to Project [4], the device wherein the reporting message includes an RRC message and the RRC message includes at least one of the following: an applicability condition for the AI / ML model, an applicability result indicating whether the features of the AI / ML model are applicable, an applicability reason associated with the reason for the applicability result, an action suggestion associated with at least one NE that should take action based on the applicability result, a parameter associated with the state of the UE, and a parameter associated with the capabilities of the UE. Project [6]: The device according to Project [4], wherein the reported message includes a MAC CE message, and the MAC CE message includes at least one of the following: an identifier ID associated with a feature of the AI / ML model, an applicability result indicating whether the feature of the AI / ML model is applicable, and an applicability reason associated with the reason for the applicability result. Project [7]: According to Project [4], the device in which the reported message includes a UAI message, the UAI message including at least one of the following: an identifier ID associated with a feature of the AI / ML model, a performance metric associated with the AI / ML model, a training process associated with the AI / ML model, and an inference accuracy associated with the AI / ML model. Item [8]: A device according to any one of Items [1]-[7], wherein the device is configured to evaluate applicability by: determining whether the AI / ML model is correctly configured; determining that the AI / ML model is applicable based on the determination that the AI / ML model is correctly configured; and determining that the AI / ML model is not applicable based on the determination that the AI / ML model is not correctly configured. Project [9]: According to the device of Project [8], the device is configured to determine whether the AI / ML model is correctly configured by: comparing an applicability condition with a predefined threshold, wherein the applicability condition includes at least one of the following: network conditions, UE mobility, and UE battery level; determining that the AI / ML model is correctly configured based on the determination that the applicability condition meets the conditions defined by the predefined threshold; and determining that the AI / ML model is not correctly configured based on the determination that the applicability condition does not meet the conditions defined by the predefined threshold. Project

[10] : The device according to Project [9], wherein the device is further configured to: obtain real-time data, wherein the real-time data includes at least one of the following: current network conditions, current UE mobility, and environmental factors; calculate a new threshold for applicability conditions based on the real-time data; generate a second report message including the new threshold; and provide the second report message to at least one NE. Item

[11] : A device according to any one of Items [1]-

[10] , wherein the device is further configured to: monitor performance metrics of AI / ML models; generate a third report message including: an ID associated with the monitored performance metric, a metric value associated with the monitored performance metric, and a timestamp associated with when the performance metric was monitored; and provide the third report message to at least one NE. Item

[12] : A device according to any one of Items [1]-

[11] , wherein the device is further configured to: obtain real-time data, wherein the real-time data includes at least one of the following: current network conditions, current UE capabilities, and environmental factors; select an AI / ML model based on the real-time data; generate a fourth report message, the fourth report message including information associated with the selected AI / ML model; and provide the fourth report message to at least one NE. Project

[13] : A method comprising: evaluating the suitability of an artificial intelligence (AI) / machine learning (ML) model in relation to a mobile telecommunications network; generating a report message including the evaluated suitability; and providing the report message to at least one network element (NE) of the mobile telecommunications network. Project

[14] : According to the method of Project

[13] , the method includes: automatically performing, based on triggering conditions, evaluation of the suitability of AI / ML models, generation of report messages, and provision of report messages, and wherein the triggering conditions include predefined activation conditions associated with at least one of the following: device and NE. Item

[15] : The method according to any one of Items

[13] -

[14] , wherein the method includes: in response to receiving a query message from at least one NE, performing: evaluation of the suitability of an AI / ML model, generation of a report message, and provision of a report message, and wherein the query message includes information associated with at least one of: network configuration and network conditions. Project

[16] : The method according to any one of Projects

[13] -

[15] , wherein evaluating applicability includes: determining whether the AI / ML model is correctly configured; determining that the AI / ML model is applicable based on determining that the AI / ML model is correctly configured; and determining that the AI / ML model is not applicable based on determining that the AI / ML model is not correctly configured, wherein determining whether the AI / ML model is correctly configured includes comparing applicability conditions with predefined thresholds, wherein the applicability conditions include at least one of the following: network conditions, UE mobility, and UE battery level; determining that the AI / ML model is correctly configured based on determining that the applicability conditions meet the conditions defined by the predefined thresholds; and determining that the AI / ML model is not correctly configured based on determining that the applicability conditions do not meet the conditions defined by the predefined thresholds. Project

[17] : According to the method of Project

[16] , the method further includes: obtaining real-time data, wherein the real-time data includes at least one of the following: current network conditions, current UE mobility, and environmental factors; calculating a new threshold for applicability conditions based on the real-time data; generating a second report message including the new threshold; and providing the second report message to at least one NE. Project

[18] : The method of any one of Projects

[13] -

[17] , wherein the method further includes: monitoring the performance metrics of the AI / ML model; generating a third report message, the third report message including an ID associated with the monitored performance metric, a metric value associated with the monitored performance metric, and a timestamp associated with when the performance metric was monitored; and providing the third report message to at least one NE. Project

[19] : The method according to any one of Projects

[13] -

[18] , wherein the method further comprises: obtaining real-time data, wherein the real-time data includes at least one of the following: current network conditions, current UE capabilities, and environmental factors; selecting an AI / ML model based on the real-time data; generating a fourth report message, the fourth report message including information associated with the selected AI / ML model; and providing the fourth report message to at least one NE. Project

[20] : A non-transitory computer-readable recording medium having instructions recorded thereon that can be executed by a device to cause the device to perform a method comprising: evaluating the suitability of an artificial intelligence (AI) / machine learning (ML) model in relation to a mobile telecommunications network; generating a report message including the evaluated suitability; and providing the report message to at least one network element (NE) of the mobile telecommunications network.

[0198] It should be understood that many modifications and variations of this disclosure can be made based on the teachings above. It is evident that, to the extent of the appended terms, this disclosure can be practiced in ways other than those specifically described herein.

Claims

1. A device configured to: Evaluate the applicability of artificial intelligence (AI) / machine learning (ML) models related to mobile telecommunications networks; Generate a report message including the evaluated applicability; and The reporting message is provided to at least one network element NE of the mobile telecommunications network.

2. The device according to claim 1, The device is configured to autonomously perform, based on triggering conditions, the evaluation of the suitability of the AI / ML model, the generation of the report message, and the provision of the report message. The triggering conditions include predefined activation conditions associated with at least one of the following: the device and the NE.

3. The device according to claim 1, The device is configured to perform, in response to receiving a query message from the at least one NE, the following actions: evaluating the suitability of the AI / ML model, generating the report message, and providing the report message. The query message includes information associated with at least one of the following: network configuration and network conditions.

4. The device according to claim 1, wherein the reporting message includes at least one of the following: Radio Resource Control (RRC) message, User Equipment Assistance Information (UAI) message, Media Access Control (MAC) Control Element (CE) message, Physical PHY layer message, and Non-Access Stratum (NAS) layer message.

5. The device of claim 4, wherein the reporting message includes the RRC message, and the RRC message includes at least one of the following: applicability conditions to which the AI / ML model applies, an applicability result indicating whether the features of the AI / ML model are applicable, an applicability reason associated with the reason for the applicability result, an action suggestion associated with the action that the at least one NE should take based on the applicability result, a parameter associated with the state of the UE, and a parameter associated with the capabilities of the UE.

6. The device of claim 4, wherein the reporting message includes the MAC CE message, and the MAC CE message includes at least one of the following: an identifier ID associated with a feature of the AI / ML model, an applicability result indicating whether the feature of the AI / ML model is applicable, and an applicability reason associated with the reason for the applicability result.

7. The device of claim 4, wherein the reporting message includes the UAI message, the UAI message including at least one of the following: an identifier ID associated with a feature of the AI / ML model, a performance metric associated with the AI / ML model, a training process associated with the AI / ML model, and inference accuracy associated with the AI / ML model.

8. The device of claim 1, wherein the device is configured to evaluate the suitability by: Determine whether the AI / ML model is configured correctly; Based on determining that the AI / ML model is correctly configured, it is determined that the AI / ML model is applicable; and Based on the determination that the AI / ML model is not configured correctly, it is determined that the AI / ML model is unsuitable.

9. The device of claim 8, wherein the device is configured to determine whether the AI / ML model is correctly configured by: The applicability conditions are compared with predefined thresholds, wherein the applicability conditions include at least one of the following: network conditions, UE mobility, and UE battery level; Based on the determination that the applicability conditions satisfy the conditions defined by the predefined threshold, it is determined that the AI / ML model is correctly configured; as well as Based on the determination that the applicability conditions do not meet the conditions defined by the predefined threshold, it is determined that the AI / ML model is not configured correctly.

10. The device according to claim 9, wherein the device is further configured to: Obtain real-time data, wherein the real-time data includes at least one of the following: current network conditions, current UE mobility, and environmental factors; Based on the real-time data, calculate a new threshold for the applicability conditions; Generate a second report message including the new threshold; and The second report message is provided to at least one NE.

11. The device according to claim 1, wherein the device is further configured to: Monitor the performance metrics of the AI / ML model; Generate a third report message, the third report message including: The ID associated with the monitored performance metric, the metric value associated with the monitored performance metric, and the timestamp associated with when the performance metric was monitored; as well as The third report message is provided to at least one NE.

12. The device according to claim 1, wherein the device is further configured to: Obtain real-time data, wherein the real-time data includes at least one of the following: current network conditions, current UE capabilities, and environmental factors; Based on the real-time data, select an AI / ML model; Generate a fourth report message, which includes information associated with the selected AI / ML model; and The fourth report message is provided to at least one NE.

13. A method comprising: Evaluate the applicability of artificial intelligence (AI) / machine learning (ML) models related to mobile telecommunications networks; Generate a report message that includes the evaluated applicability; as well as The reporting message is provided to at least one network element NE of the mobile telecommunications network.

14. The method of claim 13, wherein the method comprises: The following actions are performed automatically based on triggering conditions: the evaluation of the suitability of the AI / ML model, the generation of the report message, and the provision of the report message. The triggering conditions include predefined activation conditions associated with at least one of the following: the device and the NE.

15. The method of claim 13, wherein the method comprises: In response to receiving a query message from the at least one NE, the following actions are performed: the evaluation of the suitability of the AI / ML model, the generation of the report message, and the provision of the report message. The query message includes information associated with at least one of the following: network configuration and network conditions.

16. The method according to claim 13, The evaluation of the applicability includes: Determine whether the AI / ML model is configured correctly; Based on determining that the AI / ML model is correctly configured, it is determined that the AI / ML model is applicable; and Based on the determination that the AI / ML model is not configured correctly, it is determined that the AI / ML model is unsuitable. The determination of whether the AI / ML model is correctly configured includes: The applicability conditions are compared with predefined thresholds, wherein the applicability conditions include at least one of the following: network conditions, UE mobility, and UE battery level; Based on the determination that the applicability conditions satisfy the conditions defined by the predefined threshold, it is determined that the AI / ML model is correctly configured; and Based on the determination that the applicability conditions do not meet the conditions defined by the predefined threshold, it is determined that the AI / ML model is not configured correctly.

17. The method of claim 16, wherein the method further comprises: Obtain real-time data, wherein the real-time data includes at least one of the following: current network conditions, current UE mobility, and environmental factors; Based on the real-time data, calculate a new threshold for the applicability conditions; Generate a second report message that includes the new threshold; as well as The second report message is provided to at least one NE.

18. The method of claim 13, wherein the method further comprises: Monitor the performance metrics of the AI / ML model; Generate a third report message, which includes an ID associated with the monitored performance metric, a metric value associated with the monitored performance metric, and a timestamp associated with when the performance metric was monitored. as well as The third report message is provided to at least one NE.

19. The method of claim 13, wherein the method further comprises: Obtain real-time data, wherein the real-time data includes at least one of the following: current network conditions, current UE capabilities, and environmental factors; Based on the real-time data, select an AI / ML model; Generate a fourth report message, which includes information associated with the selected AI / ML model; as well as The fourth report message is provided to at least one NE.

20. A non-transitory computer-readable recording medium having instructions recorded thereon that can be executed by a device to cause the device to perform a method, the method comprising: Evaluate the applicability of artificial intelligence (AI) / machine learning (ML) models related to mobile telecommunications networks; Generate a report message that includes the evaluated applicability; as well as The reporting message is provided to at least one network element NE of the mobile telecommunications network.

Images