Configuration of performance monitoring for ai / ml based positioning

Abstract

Example embodiments of the present disclosure relate to configuration of performance monitoring for positioning based on artificial intelligence / machine learning (AI / ML). A method comprises: receiving, from a second apparatus, configuration information comprising a set of one or more conditions for triggering performance monitoring of an AI / ML model, wherein the AI / ML model is deployed for positioning of the first apparatus, and wherein the set of one or more conditions is based on at least one of: a first error margin associated with position information of the first apparatus determined using the AI / ML model, or a second error margin associated with an intermediate feature for positioning of the first apparatus; and in accordance with a determination that at least one condition from the set of one or more conditions is satisfied, sending, to the second apparatus, a performance monitoring result for the AI / ML model based on the configuration information.

Description

Cross-reference to related applications

[0001] This application claims priority and benefit to U.S. Provisional Application No. 63 / 755571, filed February 7, 2025, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0002] Various example embodiments of this disclosure generally relate to the telecommunications field, and more specifically to methods, apparatuses, devices, and computer-readable storage media for configuring performance monitoring of artificial intelligence / machine learning (AI / ML) based positioning. Background Technology

[0003] In the telecommunications industry, artificial intelligence / machine learning (AI / ML) has already been adopted in telecom systems to improve performance. The 3rd Generation Partnership Project (3GPP) Release 18 initiated research into AI / ML for the New Radio (NR) air interface. The goal is to explore features that enhance AI / ML to better support AI / ML-based algorithms, thereby improving performance and / or reducing complexity and overhead. Several use cases are considered to identify a general AI / ML framework, including the functional requirements of the AI / ML architecture, which can be used in subsequent projects. Summary of the Invention

[0004] In a first aspect of this disclosure, a first apparatus is provided. The first apparatus includes at least one processor; and at least one memory storing instructions, which, when executed by the at least one processor, cause the first apparatus to at least: receive configuration information from a second apparatus, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed for localization of the first apparatus, and wherein the set of one or more conditions is based on at least one of: a first error margin associated with location information of the first apparatus determined using the AI / ML model, or a second error margin associated with intermediate features used for localization of the first apparatus; and, based on the configuration information, send the performance monitoring results of the AI / ML model to the second apparatus.

[0005] In a second aspect of this disclosure, a second apparatus is provided. The second apparatus includes: at least one processor; and at least one memory storing instructions, which, when executed by the at least one processor, cause the second apparatus to at least: send configuration information to a first apparatus, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed at the first apparatus for localization of the first apparatus, and wherein the set of one or more conditions is based on at least one of: a first error margin associated with location information of the first apparatus determined using the AI / ML model, or a second error margin associated with intermediate features of localization of the first apparatus; and receive performance monitoring results of the AI / ML model from the first apparatus based on the configuration information, wherein the performance monitoring results are received based on the satisfaction of at least one condition from the set of one or more conditions.

[0006] In a third aspect of this disclosure, a method is provided. The method includes: receiving configuration information from a second device, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed for localization of the first device, and wherein the set of one or more conditions is based on at least one of: a first error margin associated with location information of the first device determined using the AI / ML model, or a second error margin associated with intermediate features of localization for the first device; and transmitting performance monitoring results of the AI / ML model to the second device based on the configuration information, according to determining that at least one condition from the set of one or more conditions is satisfied.

[0007] In a fourth aspect of this disclosure, a method is provided. The method includes: sending configuration information to a first device, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed at the first device for localization of the first device, and wherein the set of one or more conditions is based on at least one of: a first error margin associated with location information of the first device determined using the AI / ML model, or a second error margin associated with intermediate features of localization for the first device; and receiving, based on the configuration information, a performance monitoring result of the AI / ML model from the first device, wherein the performance monitoring result is received based on the satisfaction of at least one condition from the set of one or more conditions.

[0008] In a fifth aspect of this disclosure, a first apparatus is provided. The first apparatus includes: components for receiving configuration information from a second apparatus, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed for the localization of the first apparatus, and wherein the set of one or more conditions is based on at least one of: a first error margin associated with location information of the first apparatus determined using the AI / ML model, or a second error margin associated with intermediate features of localization of the first apparatus; and components for sending performance monitoring results of the AI / ML model to the second apparatus based on the configuration information, according to determining that at least one condition from the set of one or more conditions is satisfied.

[0009] In a sixth aspect of this disclosure, a second apparatus is provided. The second apparatus includes: components for sending configuration information to a first apparatus, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed at the first apparatus for localization of the first apparatus, and wherein the set of one or more conditions is at least one of: a first error margin associated with location information of the first apparatus determined using the AI / ML model, or a second error margin associated with intermediate features used for localization of the first apparatus; and components for receiving performance monitoring results for the AI / ML model from the first apparatus based on the configuration information, wherein the performance monitoring results are received based on the satisfaction of at least one condition from the set of one or more conditions.

[0010] In a seventh aspect of this disclosure, a computer-readable medium is provided. The computer-readable medium includes instructions stored thereon for causing a device to at least execute the method according to a third aspect.

[0011] In an eighth aspect of this disclosure, a computer-readable medium is provided. The computer-readable medium includes instructions stored thereon for causing a device to at least execute the method according to the fourth aspect.

[0012] It should be understood that the summary portion is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] Some exemplary embodiments will now be described with reference to the accompanying drawings, in which: Figure 1 An example of a communication network to which the examples disclosed herein can be applied is shown; Figure 2The signaling flow for performance monitoring on an AI / ML model of a positioning terminal device is illustrated according to some exemplary embodiments of the present disclosure; Figure 3 A flowchart is shown illustrating a method implemented at a first device according to some exemplary embodiments of the present disclosure; Figure 4 A flowchart is shown illustrating a method implemented at a second device according to some example embodiments of the present disclosure; Figure 5 A simplified block diagram of a device suitable for implementing example embodiments of the present disclosure is shown; and Figure 6 A block diagram of an example computer-readable medium according to some example embodiments of the present disclosure is shown.

[0014] Throughout the accompanying drawings, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0015] The principles of this disclosure will now be described with reference to some exemplary embodiments. It should be understood that these embodiments are described for illustrative purposes only and to assist those skilled in the art in understanding and implementing this disclosure, without imposing any limitation on the scope of this disclosure. The embodiments described herein can be implemented in various ways other than those described below.

[0016] In the following description and claims, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0017] References to "an embodiment," "embodiment," "example embodiment," etc., in this disclosure indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment needs to include that particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Moreover, when a particular feature, structure, or characteristic is described in connection with an embodiment, whether explicitly described or not, it is believed that its influence on such feature, structure, or characteristic in conjunction with other embodiments is within the knowledge of those skilled in the art.

[0018] It should be understood that although various elements may be described herein using prefixes such as “first,” “second,” etc., these elements should not be limited by these terms. These terms are used only to distinguish one element from another, and they do not restrict the order of the terms. For example, without departing from the scope of the exemplary embodiments, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. As used herein, the term “and / or” includes any and all combinations of one or more of the listed terms.

[0019] As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements is connected by “and” or “or”, means at least any one of the elements, or at least any two or more of the elements, or at least all of the elements.

[0020] As used herein, unless explicitly stated otherwise, the action “in response to A” does not indicate that the action is performed immediately after “A” occurs and may include one or more intervention steps.

[0021] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that the terms “comprising,” “including,” “having,” “containing,” and / or “including” as used herein specify the presence of the stated features, elements, and / or components, etc., but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0022] As used in this application, the term "circuit system" may refer to one or more of the following: (a) Hardware circuit implementation only (such as implementation in analog and / or digital circuits only), and (b) A combination of hardware circuitry and software, such as (if applicable): (i) A combination of analog and / or digital hardware circuitry with software / firmware, and (ii) Any part of a hardware processor having software (including digital signal processor(s) working together to enable a device (such as a mobile phone or server) to perform various functions), software, and memory), and (c) One or more hardware circuits and / or one or more processors, such as one or more microprocessors or a portion thereof, that require software (e.g., firmware) to operate, but the software may not be present when operation is not required.

[0023] This definition of circuit system applies to all uses of the term in this application, including in any claim. As another example, as used herein, the term circuit system also covers implementations of only hardware circuitry or processors (or processors in general) or a portion thereof and their accompanying software and / or firmware. For example, and if applicable to a particular claim element, the term circuit system also covers baseband integrated circuits or processor integrated circuits for use in mobile devices or servers, cellular network devices, or other computing or networking devices.

[0024] As used herein, the term "communication network" refers to a network that conforms to any suitable communication standard, such as New Radio (NR), Long Term Evolution (LTE), LTE-A Advanced (LTE-A), Wideband Code Division Multiple Access (WCDMA), High-Speed ​​Packet Access (HSPA), Narrowband Internet of Things (NB-IoT), etc. Furthermore, communication between terminal devices and network devices in a communication network can be performed according to any suitable generated communication protocol, including but not limited to first-generation (1G), second-generation (2G), 2.5G, 2.75G, third-generation (3G), fourth-generation (4G), 4.5G, fifth-generation (5G), sixth-generation (6G) communication protocols and / or any other currently known or future-developed protocols. Embodiments of this disclosure can be applied to a variety of communication systems. Given the rapid development in communications, there will naturally be future types of communication technologies and systems that can implement this disclosure. The scope of this disclosure should not be limited to the aforementioned systems only.

[0025] As used herein, the term "network device" refers to a node in a communications network through which terminal devices access the network and receive services. Network devices can refer to base stations (BS) or access points (APs), such as Node B (NodeB or NB), evolved Node B (eNodeB or eNB), NR NB (also known as gNB), Remote Radio Unit (RRU), Radio Head (RH), Remote Radio Head (RRH), repeater, Integrated Access and Backhaul (IAB) node, low-power node (such as femtoseconds, picoseconds), non-terrestrial network (NTN) or non-terrestrial network equipment (such as satellite network equipment, low Earth orbit (LEO) satellites, and geostationary Earth orbit (GEO) satellites), spacecraft network equipment, etc., depending on the terminology and technology applied. In some example embodiments, the Radio Access Network (RAN) split architecture includes a centralized unit (CU) and a distributed unit (DU) at the IAB donor node. An IAB node includes a mobile terminal (IAB-MT) portion that behaves like a UE toward its parent node, and a DU portion that behaves like a base station toward the next-hop IAB node. In some examples, network devices may include core network (CN) devices. A core network includes one or more core network devices or equipment constructed with hardware and software components. The characteristics of these components may be substantially similar to those described with respect to the UE, network device, and / or host, such that the description generally applies to the corresponding components of the core network device. Example core network devices include functions such as Location Management Function (LMF), Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier Dehiding Function (SIDF), Unified Data Management (UDM), Secure Edge Protection Agent (SEPP), Network Exposure Function (NEF), and / or User Plane Function (UPF).

[0026] The term "terminal device" refers to any terminal device capable of wireless communication. As an example and not a limitation, a terminal device may also be referred to as a communication device, user equipment (UE), subscriber station (SS), portable subscriber station, mobile station (MS), or access terminal (AT). Terminal devices can include, but are not limited to, mobile phones, cellular phones, smartphones, Voice over IP (VoIP) phones, wireless local loop phones, tablets, wearable terminal devices, personal digital assistants (PDAs), portable computers, desktop computers, image capture terminal devices (such as digital cameras), gaming terminal devices, music storage and playback devices, in-vehicle wireless terminal devices, wireless endpoints, mobile stations, laptop embedded devices (LEEs), laptop devices (LMEs), USB dongles, smart devices, wireless customer premises equipment (CPEs), Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in the context of industrial and / or automated processing chains), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. The terminal device may also correspond to the mobile terminal (MT) portion of an IAB node (e.g., a relay node). In the following description, the terms "terminal device," "communication device," "terminal," "user equipment," and "UE" are used interchangeably.

[0027] As used herein, the terms “resource,” “transmission resource,” “resource block,” “physical resource block” (PRB), “uplink resource,” or “downlink resource” can refer to any resource used to perform communication, such as communication between a terminal device and a network device, including resources in the time domain, frequency domain, spatial domain, code domain, or any other combination of time, frequency, spatial, and / or code domain resources used to implement communication. In the following, unless explicitly stated otherwise, resources in both the frequency and time domains will be used as examples of transmission resources used to describe some exemplary embodiments of this disclosure. Note that the exemplary embodiments of this disclosure are equally applicable to other resources in other domains.

[0028] Artificial intelligence (AI) can be broadly defined as enabling computers to perform tasks that mimic the human brain. Machine learning (ML) is a class of AI technologies: computer algorithms that can automatically improve their performance without explicit programming. AI algorithms were first conceived in the 1850s, but only in recent years has AI / ML been applied in a wide range of real-world fields, thanks in part to advancements in computing power and the availability of storage capacity for data.

[0029] AI / ML can assist in adjusting and optimizing Radio Access Network (RAN) parameters and settings by monitoring and predicting network performance, quality, and demand usage in real time. Additionally, AI / ML can identify and diagnose network performance degradation and provide protection against network attacks. To name a few, AI / ML can be used for energy saving, load balancing, mobility optimization, link adaptation, and security.

[0030] It is conceivable that AI / ML will enable real-time analysis and automated operation and control beyond 5G and RAN. This requires the timely acquisition of data from wireless devices, especially in applications with extremely time-sensitive requirements such as real-time video surveillance and extended reality (XR). This can be reflected in network architecture, such as by deploying and moving ML agents to desired locations within the network, for example, for data collection. User equipment (mobile devices) can assist network decision-making in resource management, thus acting as infrastructure resources.

[0031] As networks evolve towards programmable and flexible cloud-native implementations, AI / ML-based network automation will be used to simplify network management and optimization. It is anticipated that certain signal processing algorithms will support portions of the air interface, and may even eventually replace them with machine learning models. Therefore, 6G wireless communication standards will inherently support AI-based air interfaces.

[0032] To facilitate understanding of the terminology, some definitions for the AI / ML terminology list are provided below.

[0033] AI / ML Models: Data-driven algorithms that apply AI / ML techniques to generate output sets based on input sets.

[0034] AI / ML Model Delivery: A general term referring to the delivery of AI / ML models from one entity to another in any way. Note: Entities can refer to network nodes / functions (e.g., gNB, LMF, etc.), UEs, proprietary servers, etc.

[0035] AI / ML model inference: The process of using a trained AI / ML model to generate a set of outputs based on a set of inputs.

[0036] AI / ML Model Testing: A sub-process of training used to evaluate the performance of the final AI / ML model using a different dataset than that used for model training and validation. Unlike AI / ML model validation, testing does not assume subsequent tuning of the model.

[0037] AI / ML model training: The process of training AI / ML models in a data-driven manner (by learning input / output relationships) and obtaining trained AI / ML models for inference.

[0038] Data collection: The process by which network nodes, management entities, or UEs collect data for the purpose of AI / ML model training, data analysis, and inference.

[0039] Function Identification: The process / method for identifying AI / ML functions for consensus between the network and the UE. Note: Information about the AI / ML functions can be shared during function identification. The specific AI / ML functions depend on the specific use case and sub-use cases.

[0040] Network-side (AI / ML) model: An AI / ML model in which inference is performed entirely at the network.

[0041] Semi-supervised learning: The process of training a model using a mixture of labeled and unlabeled data.

[0042] Supervised learning: The process of training a model based on inputs and their corresponding labels.

[0043] UE-side (AI / ML) model: Inference is performed entirely on the terminal device or UE using the AI / ML model.

[0044] Unsupervised learning: The process of training a model without labeled data.

[0045] Figure 1 An example of a communication network 100 to which the examples disclosed herein can be applied is shown. It should be understood that the elements shown in the communication network 100 are intended to represent the main functions provided within the system. Therefore, Figure 1 The block references shown provide specific elements within the communication network for these main functions. However, some or all of the main functions represented can be implemented using other network elements. Furthermore, it should be understood that... Figure 1 This document does not depict all the functions of the communication network, but only presents those functions that help illustrate exemplary embodiments. Furthermore, Figure 1 The number of elements shown is for illustrative purposes only, and any number of elements may be present.

[0046] The communication network 100 or cellular communication network 100 may include network nodes 130 providing one or more cells. For example, each cell may be a macro cell, micro cell, femtocell, or picocell. A cell may define the coverage area or service area of ​​a corresponding access node.

[0047] Network node 130 can provide wireless access to the communication network to terminal device 110. Wireless access may include downlink (DL) communication from the network node to terminal device 110 and uplink (UL) communication from terminal device 110 to the network node. Examples of uplink channels include a Physical Uplink Control Channel (PUCCH) for transmitting control information and a Physical Uplink Shared Channel (PUSCH) for transmitting data to the network. Examples of downlink channels include a Physical Downlink Control Channel (PDCCH) for transmitting control information and a Physical Downlink Shared Channel (PDSCH) for transmitting data to user equipment.

[0048] There can be multiple terminal devices in the system. Each of them can be served by the same or different network nodes 1. The terminal devices can be configured with dual connectivity (DC), where a terminal device (e.g., terminal device 110) can connect to multiple network nodes. Terminal devices 110 can communicate with each other when a device-to-device (D2D) communication interface is established between them via a so-called side link (SL). For example, such D2D communication can be referred to as machine-to-machine, peer-to-peer (P2P) communication, or vehicle-to-vehicle (V2V) communication.

[0049] In a communication network with multiple network nodes, these nodes can connect to each other via interfaces. The LTE specification refers to this interface as the X2 interface. The interface between an LTE node and a 5G node, or between two 5G nodes, can be called the Xn interface.

[0050] Network node 130 can also connect to the core network of the communication network via another interface. The LTE specification designates the core network as the Evolved Packet Core (EPC), and the core network may include, for example, a Mobility Management Entity (MME) and gateway nodes. The MME can handle the mobility of terminal devices in a tracking area containing multiple cells and handle signaling connections between the terminal devices and the core network. Gateway nodes can handle data routing in the core network to / from terminal devices. The 5G specification designates the core network as the 5G Core (5GC). The 5G Core may include, for example, Access and Mobility Management Functions (AMF) and User Plane Functions / Gateways (UPF), and other functions. The AMF can handle Non-Access Stratum (NAS) signaling, NAS encryption & integrity protection, registration management, connection management, mobility management, access authentication and authorization, and termination of security context management. For example, a UPF node can support packet routing and forwarding, packet inspection, and Quality of Service (QoS) processing.

[0051] In some example embodiments, one or more AI / ML models 105-1, 105-2, ..., 105-N (collectively or individually referred to as AI / ML model 105) may be deployed at the terminal device 110, referred to as UE-side AI / ML models. In some example embodiments, one or more AI / ML models 125-1, 125-2, ..., 125-M (collectively or individually referred to as AI / ML model 125) may be deployed at network nodes (such as LMF 120 or network node 130), referred to as network-side AI / ML models. AI / ML models may sometimes be simply referred to as AI models or ML models.

[0052] Different AI / ML models can be configured to implement the same different algorithms. In a positioning scenario, AI / ML model 105 or 125 can be configured for direct or assisted positioning of the terminal device within the communication network. Inference, testing, training, and / or validation of the AI / ML model can be performed at terminal device 110, network node 130, and / or other entities (such as LMF 120). LMF 120 can be an entity implementing location management services. LMF 120 can be a network node in the RAN or core network, or it can be an external entity. In some example embodiments, LMF 120 may be able to communicate with network node 130 (e.g., RAN network equipment).

[0053] AI / ML models can be delivered from one entity to another in any way. AI / ML models can be delivered over the air interface in a manner agnostic to 3GPP signaling, whether the parameters of the model structure are known at the receiving end or the model itself is a new model with parameters. Delivery can contain a complete model or a partial model.

[0054] In some example embodiments, the AI / ML model is configured for AI / ML-based localization or AI / ML-enabled localization. AI / ML-enabled localization is one of the selected use cases for research projects in communication networks. In some example embodiments, two localization methods have been proposed. The first method is direct AI / ML localization, where the output of the AI / ML model inference is the UE location. Several options exist for the model's input, including channel observations such as channel impulse response (CIR), power delay distribution (PDP), reference signal received power (RSRP), reference signal received path power (RSRPP), etc. The second method is AI / ML-assisted localization, where the output of the AI / ML model inference is a new measurement and / or enhancement to existing measurements, and this new measurement and / or enhancement can be referred to as an intermediate feature as it is input to a second function to ultimately estimate the UE location. These measurements include, for example, line-of-sight (LOS) / non-line-of-sight (NLOS) identification, time of arrival (ToA), time difference of arrival (TDoA), path phase, reference signal time difference (RSTD), etc.

[0055] The enhancements to positioning accuracy are expected to cover both direct AI / ML positioning and AI / ML-assisted positioning. Positioning accuracy enhancements related to AI / ML positioning are listed in Table 1 below.

[0056] Table 1

[0057] AI / ML model lifecycle management (LCM) can include model training, model deployment, model inference, model monitoring, and model updates.

[0058] As part of the LCM of the AI / ML model, performance monitoring of the UE-side model may be required. If model monitoring of the UE-side model is required and ground reality labels (or their approximations) are provided, monitoring metrics may include statistics on the differences between the model output and the provided ground reality labels, examples of which may include mean, standard deviation, instantaneous values, and a threshold (or approximation) for the ground reality labels. To monitor the UE-side model for AI / ML-based positioning, signaling from the monitoring entity to request real labels (if needed), signaling from the monitoring entity to request model output (if needed), and / or signaling for potential requests / reporting of monitoring metrics (if needed) may be required.

[0059] Performance monitoring of the model on the specified UE side has been discussed. For model performance monitoring in AI / ML positioning scenario 1, regarding the calculation of model performance monitoring metrics in label-based model monitoring, one option for generating information about truth labels is for the target UE side to perform the monitoring metric calculation. For example, at least the information about the truth labels of the target UE is generated by the LMF and provided to the target UE. In one example, the target UE and / or gNB send measurements (e.g., conventional measurements) to the LMF, enabling the LMF to derive information about ground truth labels. In some examples, the LMF provides at least location calculation auxiliary data (e.g., existing information based on the UE's positioning method) to the target UE. In some examples, the auxiliary data can be transmitted from the LMF to the target UE, where Positioning Reference Unit (PRU) measurements (e.g., conventional measurements) and the corresponding PRU positions are sent to the target UE via the LMF.

[0060] Currently, it is unclear how the UE can trigger performance monitoring and report its results for the AI / ML model on the UE side.

[0061] While the decision to trigger performance monitoring may be entirely left to the UE, this could result in unnecessary processing and signaling overhead on both the UE and the network (NW) sides, depending on the specific circumstances. Specifically, for monitoring, the UE may require the assistance of the Downlink Location Reference Signal (LMF), which may involve the configuration and transmission of the Downlink Location Reference Signal (DL PRS), as well as the collection and signaling of measurements and associated ground fact tags (for tag-based monitoring). Additionally, the UE will perform further monitoring-related operations, such as executing new DL PRS measurements, running its ML model, and calculating performance metrics.

[0062] Therefore, when monitoring is triggered unnecessarily, i.e. when monitoring does not affect LCM decisions (e.g., feature switching or rollback), such as when it is repeated too frequently, and when the results consistently indicate the same performance level or show no significant change compared to previous monitoring results, such monitoring results in additional overhead.

[0063] Conversely, if the UE decides not to trigger any monitoring to save processing and / or signaling resources, it may lead to performance degradation when the UE performs inference using the same model / function, even if it is needed (e.g., due to significant changes in radio conditions).

[0064] Furthermore, there is currently no consensus on when (e.g., how often) and in what way (e.g., in what format) the UE can provide the results of its performance monitoring to the LMF. Too frequent monitoring incurs additional overhead, while less frequent monitoring may lead to performance degradation.

[0065] Therefore, it is necessary to establish a proactive mechanism that allows the UE to trigger monitoring and report monitoring results in a timely manner, so as to enable the LMF to make efficient LCM decisions while saving unnecessary processing and signaling overhead.

[0066] According to some example embodiments of this disclosure, a solution for configuring performance monitoring for AI / ML-based positioning is provided. A terminal device is configured with a set of one or more conditions for triggering performance monitoring of an AI / ML model deployed on the terminal device side. The terminal device can monitor whether one or more configured conditions are met to determine whether performance monitoring should be triggered. Such conditions are based at least on a first error margin associated with location information of a first device determined using the AI / ML model, and / or a second error margin associated with intermediate features used to locate the terminal device. If at least one condition from the set of one or more conditions is met, the terminal device determines and sends the performance monitoring result of the AI / ML model.

[0067] This solution enables terminal devices to receive specific conditions for timely triggering of performance monitoring and reporting of their performance results, thereby supporting efficient LCM (Low-Terminal Computation) decisions. The solution allows for timely and efficient triggering of performance monitoring and reporting of results for UE-side AI / ML models / functions. Furthermore, it saves unnecessary signaling and processing overhead on both the terminal device and network sides while ensuring effective LCM decisions.

[0068] Example embodiments of this disclosure will now be described in detail with reference to the accompanying drawings. It should be understood that although an "AI / ML model" is used, the example embodiments can be similarly applied to AI / ML functions, AI / ML enabling features, AI / ML enabling feature groups, or any other AI / ML used for localization.

[0069] Figure 2 A flowchart of a signaling flow 200 for configuring performance monitoring for AI / ML-based location services, according to some example embodiments of this disclosure, is shown. Signaling flow 200 relates to terminal device 110 and LMF 120, which can be configured to manage location services. Terminal device 110 may have one or more AI / ML models 105 deployed for positioning terminal device 110. The deployed AI / ML models 105 may or may not be activated for positioning at the side of terminal device 110. It should be understood that, instead of and / or in addition to LMF 120, any suitable network entity that can communicate with terminal device 110 for configuring performance monitoring for AI / ML-based location services may exist.

[0070] In signaling stream 200, LMF 120 sends (3010) configuration information, and terminal device 110 receives (3015) configuration information from LMF 120, which includes a set of one or more conditions for triggering performance monitoring on an AI / ML model. In some examples, the conditions can be specifically configured for an AI / ML model deployed at terminal device 110. In other examples, the conditions can be applied to any AI / ML model deployed at terminal device 110.

[0071] Terminal device 110 determines (3020) whether at least one condition from a set of one or more conditions is met. If at least one condition from a set of one or more conditions is met, terminal device 110 may determine to trigger performance monitoring. In some examples, performance monitoring may be triggered if any one of the conditions or any particular combination of conditions is met (this may depend on the configuration).

[0072] When performance monitoring is triggered, terminal device 110 performs performance monitoring (3050) on the AI / ML model and sends the performance monitoring results (3060) to LMF 120 based on the configuration information. LMF 120 then receives the performance monitoring results (3065) for the AI / ML model.

[0073] In some examples, terminal device 110 can report performance monitoring results to LMF 120 based on the corresponding configuration. For example, terminal device 110 can aggregate the results of multiple monitoring results and report them in a single message.

[0074] In some example embodiments, auxiliary data may be required to assist in performance monitoring of the AI / ML model. The auxiliary data may include, for example, a DL PRS configuration. According to the DL positioning method, the terminal device 110 can measure the DL PRS configuration to obtain timing, power, phase, or frequency measurement information, or combinations thereof. Some examples of such measurement information may include, for example, CIR, PDP, RSRP, RSRPP, DL-TDOA, and / or DL-A oD. Measurements on the DL PRS can then be used as model inputs for the AI / ML model. Furthermore, the auxiliary data may alternatively or additionally include ground truth labels for performance monitoring of the AI / ML model. The terminal device 110 can compare model inference results with the ground truth labels to calculate performance monitoring results.

[0075] Still referencing Figure 2In some example embodiments, after determining that at least one condition from a set of one or more conditions is met, terminal device 110 may first send a request (3030) to LMF 120 for auxiliary data for performance monitoring on the AI / ML model. In response to receiving (3035) the request, LMF 120 may send (3040) the requested auxiliary data to terminal device 110. Terminal device 110 receives (3045) the auxiliary data and uses the received auxiliary data to perform monitoring on the AI / ML model in order to determine the performance monitoring results of the AI / ML model based on the received auxiliary data.

[0076] In some example embodiments, alternatively, terminal device 110 may receive (3045) auxiliary data from LMF 120 without sending an explicit request for auxiliary data. In some example embodiments, LMF 120 may determine when to provide auxiliary data to terminal device 110 based on conditions configured in its configuration information. For example, LMF 120 may provide auxiliary data to terminal device 110 when terminal device 110 connects to a new cell, or LMF 120 may adjust the periodicity of auxiliary data based on monitoring trigger periodicity, etc.

[0077] In some example embodiments, the configuration information received at 3015 may include an indication of whether LMF 120 will provide auxiliary data for performance monitoring on the AI / ML model, or an indication of whether terminal device 110 needs to request auxiliary data. In this way, LMF 120 can provide necessary auxiliary data for model performance monitoring in a timely manner based on requests from the terminal device or according to its configured conditions.

[0078] In some example embodiments, auxiliary data or configuration information received at 3015 may also instruct the terminal device 110 to calculate a monitoring metric as a performance monitoring metric. Examples of monitoring metrics may include, for example, the ratio of correct predictions, the statistical difference between the training dataset and the measurement, the confidence level of the inference result, etc. The terminal device 110 may determine the performance monitoring result based on the configured metric.

[0079] In some example embodiments, terminal device 110 may trigger additional data collection for performance monitoring and / or for training AI / ML models based on the determination that at least one condition from a set of one or more conditions is not met. In cases where none of these conditions are met or some conditions are not met, terminal device 110 may not trigger monitoring, but instead may trigger additional data collection to add data matching the requirements.

[0080] According to an example embodiment of this disclosure, it enables the network (i.e., LMF) to configure conditions at the terminal device to trigger performance monitoring of the UE-side AI / ML model / function and report the results. Since the LMF 120 knows the terminal device's past location QoS in time and space, it predicts that conditions affecting the performance of AI / ML-based location involving the UE-side model are in a better position. These could be, for example, certain areas or times in the environment affecting channel conditions, the availability of transmit / receive points (TRPs), network load, etc. Therefore, by proactively notifying the terminal device of such conditions to trigger performance monitoring and configure the reporting of its results, efficient LCM decisions can be made while saving unnecessary signaling and processing overhead on both sides.

[0081] The following section describes in detail the set of conditions used to trigger performance monitoring.

[0082] Typically, the set of one or more conditions used to trigger performance monitoring is configured based at least on the confidence or quality of the inference results associated with the AI / ML model. Performance monitoring is required if it detects unexpected inference errors.

[0083] Specifically, the set of one or more conditions used to trigger performance monitoring is based on at least one of the following: a first error margin associated with the location information of the terminal device 110 determined using an AI / ML model, or a second error margin associated with intermediate features used to locate the terminal device 110.

[0084] For direct AI / ML localization, the output of the AI / ML model inference is the location information of the terminal device. For AI / ML assisted localization, the output of the AI / ML model inference is new measurements and / or enhancements to existing measurements, and such new measurements and / or enhancements can be referred to as intermediate features, as they are input into a second function to ultimately estimate the location information of the terminal device. Intermediate features may include, but are not limited to, timing measurements (e.g., ToA, TDoA, RSTD, etc.) used for locating the terminal device 110, angle measurements (e.g., angle of arrival, AoA, angle of departure, AoD, etc.) used for locating the terminal device 110, phase measurements (e.g., path phase) used for locating the terminal device 110, power measurements used for locating the terminal device 110, and / or line-of-sight (LOS) / non-line-of-sight (NLOS) indicators used for locating the terminal device 110.

[0085] A first error margin associated with the location information of the terminal device 110 determined using an AI / ML model, or a second error margin associated with intermediate features used to locate the terminal device, can indicate the inference error margin of the AI / ML model.

[0086] In some example embodiments, a condition configured based on a first error margin is satisfied if the accuracy of the location information determined using the AI / ML model is determined to be less than a first configured threshold. Terminal device 110 may trigger performance monitoring in response to the satisfaction of this condition. In some examples, the first error margin associated with the location information may be based on the horizontal and / or vertical accuracy of the location information reported to the LMF. Performance monitoring may be triggered if the horizontal and / or vertical accuracy of the location information reported to the LMF is less than the configured horizontal and / or vertical accuracy threshold. Alternatively or additionally, the first error margin associated with the location information may be based on other positioning parameters, such as parameters of the uncertain ellipse or circle used for positioning estimation.

[0087] In some example embodiments, a condition based on a second error margin is satisfied if the quality of an intermediate feature used to locate the terminal device 110 is determined to be less than a second configured threshold. The terminal device 110 may trigger performance monitoring in response to the satisfaction of this condition. As described above, the intermediate feature may include one or more positioning-related measurements associated with the AI / ML model, such as timing measurements, angle measurements, etc. Performance monitoring may be triggered if the quality of one or more positioning-related measurements is less than the configured threshold.

[0088] In some example embodiments, alternative or additional conditions for triggering performance monitoring can be based on changes to a list of cells that are valid for ancillary data and detectable by terminal device 110. Ancillary data (such as DL PRS configuration) can always be associated with a valid region. In some examples, the valid region can be specified as a list of cells, such as the Physical Cell ID (PCID), Global Cell ID (GCID), and PRS ID based on the TRP within the cell. As described above, monitoring can be triggered based on changes in cell-specific information.

[0089] When the terminal device is within these cells, it can measure DL PRS and perform location estimation. Otherwise, if the terminal device leaves the effective area, the measurement may become problematic, or at least the terminal device may not be able to adequately measure DL PRS. In this case, model inference for location may degrade because there are not enough DL PRS measurements available. Therefore, if there is any change in the effective area, the terminal device 110 can trigger model performance monitoring. If the terminal device 110 detects a large degree of change in the cell list configured in the effective area, it means that the terminal device 110 is moving out of the effective area, and therefore performance monitoring of the AI / ML model can be triggered. Therefore, the condition for triggering performance monitoring based on changes in the cell list can be configured such that if the change exceeds a certain level, the condition is met and performance monitoring is triggered.

[0090] Terminal device 110 can determine the degree of change in the configured cell list or validity area by detecting the number of cells or TRPs not included in the cell list. If all detectable cells are included in the cell list, monitoring is not required. Otherwise, if the number of new cells (i.e., cells not in the list of validity areas) exceeds a (preset) threshold, terminal device 110 is triggered to perform monitoring (because the detection of new cells or TRPs indicates that terminal device 110 has moved to another location or that the environment around terminal device 110 has changed). When the validity area or DL ​​PRS configuration is provided to terminal device 110, the threshold used to determine the degree of change in the validity area can be indicated by LMF.

[0091] As an alternative to the cell list, the network (e.g., LMF 120) can be configured with validity timers for validity areas or for DLPRS configuration. The condition for triggering performance monitoring can be when a configured timer expires.

[0092] In some example embodiments, one or more additional conditions for triggering performance monitoring may be based on characteristics associated with the measurements used for performance monitoring or inference of the AI / ML model, as described below. Conditions may be based on the availability of measurement samples to perform monitoring or inference. For example, performance monitoring may be triggered if terminal device 110 has obtained a specific (minimum and / or maximum) number of measurement samples (e.g., to calculate a monitoring metric averaged across multiple measurements associated with ground fact tags) and / or a certain distribution of data samples (e.g., the average distance between measurements). LMF 120 may set certain requirements on the characteristics of the measurements available at terminal device 110 (e.g., the ratio of LOS to NLOS measurements) before terminal device 110 can use these measurements to perform monitoring.

[0093] Specifically, some alternative or additional conditions for triggering performance monitoring could be the ratio of Loss samples to NLoS samples associated with measurements used to locate terminal device 110. Conditions can be configured based on thresholds. A condition is met if the ratio of Loss samples to NLoS samples is below or above a configured threshold. For example, if terminal device 110 is expected to enter an NLoS-dominated area, it might not utilize monitoring measurements that primarily contain LOS measurements to perform monitoring, as it would be meaningless.

[0094] Alternatively or additionally, the condition for triggering performance monitoring may be the number of Loss samples associated with measurements used to locate terminal device 110, and / or the number of NLoS samples associated with measurements used to locate terminal device 110. This condition may also be configured based on a threshold. For example, the condition is met if the number of Loss samples and / or the number of NLoS samples is lower than or higher than a configured threshold. The comparison result may indicate whether terminal device 110 is currently in an NLoS-dominated region or a LosS-dominated region.

[0095] In addition to the Loss / NLoS samples, other measurement characteristics may include the number of paths, the total number of samples associated with the measurements used to locate the terminal device 110, the timestamp range associated with the measurements used to locate the terminal device 110, and the timing error group (TEG) margin associated with the measurements performed by the terminal device 110. Alternative or additional conditions for triggering performance monitoring can be further configured based on those measurement characteristics. Whether a condition is met or not can be determined based on a comparison of the corresponding value of the measurement characteristic with a configured threshold. As described above, measurements may include various characteristics obtained by measuring the DL PRS.

[0096] In the above example embodiment, the satisfaction or non-satisfaction of the above conditions is determined based on a comparison between the measured features and the corresponding thresholds. Such thresholds can be configured by the LMF, enabling the terminal device to trigger monitoring.

[0097] In some example embodiments, alternative or additional conditions for triggering performance monitoring can be based on the difference between the measurement features to be used for performance monitoring or inference and the previous measurement features used to train the AI / ML model. The difference between the current measurement features and the (average) measurements used for training can also be configured with a threshold. If the difference is higher than the threshold, performance monitoring can be triggered because the current measurement used for AI / ML-based localization has deviated from the measurements used to train the AI / ML model. For example, if the average TEG of measurements in the training dataset is 8Tc, and the average TEG of the new measurement is 64Tc, then the terminal device 110 can better trigger performance monitoring.

[0098] This type of condition can be configured for each or some of the measurement characteristics described above. Specifically, some alternative or additional conditions can be based on one or more of the following: - The difference between the ratio of Loss samples to NLoS samples to be used for monitoring or inference and the ratio of Loss samples to NLoS samples used to train the AI / ML model. - The difference between the number of Loss samples used for performance monitoring or inference and the number of Loss samples used to train the AI / ML model. - The difference between the number of NLoS samples used for performance monitoring or inference and the number of NLoS samples used to train the AI / ML model. - The difference between the number of paths used for performance monitoring or inference and the number of paths used to train the AI / ML model. - The difference between the number of samples used for performance monitoring or inference and the number of samples used to train the AI / ML model. - The difference between the timestamp when collecting DL PRS measurements for performance monitoring or inference and the timestamp when collecting DL PRS measurements for training the AI / ML model, or The difference between the TEG margin associated with measurements used for performance monitoring or inference and the TEG margin associated with measurements used to train the AI / ML model.

[0099] It should be understood that other measurement features may also exist that can be used to determine monitoring triggers.

[0100] In some cases, when deploying a new AI / ML model or feature at terminal device 110, performance monitoring is required to validate / test the AI / ML model or feature in order to make it available for use. Therefore, the additional conditions for triggering performance monitoring can be based on the deployment of the AI / ML model on terminal device 110.

[0101] It should be noted that one or more of the above conditions can be combined with each other. For example, terminal device 110 can check the confidence level of its inference output when it enters a new cell or periodically. It should be understood that various conditions for triggering performance monitoring have been described above, but other conditions may be envisioned and configured as triggering conditions in other example embodiments.

[0102] For example, some conditions can be time-based, such as monitoring triggering immediately after a condition is received, triggering at some future time, or periodically. As another example, some further conditions can be based on the mobility of terminal device 110 or the (re)selection of any / specific cell. Monitoring is triggered if the change in the UE's coarse location or speed exceeds a configured threshold, or if terminal device 110 has the ability to select a specific cell or any new cell (re-).

[0103] As another example, some conditions can be further based on a reference signal, such as DL PRS. If the measurement result (e.g., SINR or RSRP on DL PRS is below a certain threshold) can trigger monitoring.

[0104] In some example embodiments, the set of one or more conditions for triggering performance monitoring can be considered a first set of one or more conditions, and the configuration information from LMF 120 may also include a second set of one or more conditions for reporting performance monitoring results. That is, although terminal device 110 can perform performance monitoring on the AI / ML model when the conditions for triggering monitoring are met, it may not report the performance monitoring results to LMF 120 until one or more configured conditions are met. Performance monitoring results are reported if any or a combination of specific conditions from the second set of one or more conditions are met.

[0105] In some example embodiments, the conditions for reporting performance monitoring results can be configured to be reported when performance monitoring is triggered. For example, terminal device 110 can report performance monitoring results each time monitoring is triggered.

[0106] In some example embodiments, the conditions for reporting performance monitoring results may be based on a comparison of the performance metrics in the performance monitoring results with a threshold configured in a third configuration.

[0107] Terminal device 110 can determine whether a performance metric is higher or lower than a configured threshold in order to determine whether the corresponding condition is met to trigger a report. It can depend on the performance metric being measured as a result; if the metric is higher than the threshold, or if the metric is lower than the threshold, then the condition for reporting is met. For example, if the performance metric is the ratio of correct predictions, then if the ratio of correct predictions is lower than a certain threshold, then the condition for reporting is met; that is, performance monitoring indicates a greater degree of performance degradation that should be reported to the LMF.

[0108] In some example embodiments, the conditions for reporting performance monitoring results can be based on a reporting period. Reporting of performance monitoring results can be configured within a specific time window. Terminal device 110 can determine whether the current time is within the configured time window to trigger a report. For example, terminal device 110 can be configured to record and aggregate multiple monitoring results and report them within a specific time window.

[0109] In some example embodiments, terminal device 110 may report performance monitoring results with different modes (e.g., light, medium, or high reporting). For example, in a static environment, terminal device 110 may not report performance monitoring results after each time slot, while in a mobile environment, terminal device 110 may need to report its performance monitoring results frequently. In another example, based on UE positioning accuracy requirements, the LMF may configure the monitoring reporting frequency, for example, by configuring the reporting period.

[0110] In some example embodiments, LMF 120 may request terminal device 110 to report the number of samples it uses for monitoring, for example, if it differs from the configuration. This allows the network to assess the confidence level at which it can assign to the provided monitoring metrics and whether data collection needs to be triggered for monitoring.

[0111] In some example embodiments, performance monitoring reports can be configured to use a discrete, normalized format, such as indicating specific results without using numerical values. For example, terminal device 110 can indicate whether a performance monitoring result is low (or similar label), average (or similar label), or high-performance monitoring (or similar label). Of course, the granularity of discrete performance monitoring results can be configured in any other way.

[0112] In some example embodiments, after determining or receiving the performance monitoring results of the AI / ML model deployed on the terminal device side, LMF 120 (or terminal device 110) can take necessary LCM actions (e.g., function / model switching, rollback, etc.) based on the monitoring results and notify terminal device 110 (or LMF 120).

[0113] Still referencing Figure 2 In signaling stream 200, based on performance monitoring results received from terminal device 110, LMF 120 can determine (3070) an LCM decision for the AI / ML model. The LCM decision may include, for example, activating, deactivating, switching or rolling back the AI / ML model or function, or (re)training or further monitoring the AI / ML model or function. LMF 120 can notify terminal device 110 of the LCM decision (3080). Terminal device 110 receives the LCM decision (3085) and acts accordingly.

[0114] In some alternative scenarios, terminal device 110 may determine (3090) an LCM decision based on performance monitoring results. Terminal device 110 may notify LMF 120 of the LCM decision (3100). LMF 120 receives (3105) the LCM decision and acts accordingly, or may confirm with terminal device 110 whether it can act based on the LCM decision, such as deactivating or activating an AI / ML model or function, switching or reverting an AI / ML model or function, etc.

[0115] It should be understood that although AI / ML models or functions for positioning are discussed in the example embodiments, this disclosure is generally applicable to performance monitoring of other (potential) AI / ML use cases, such as beam management, mobility, etc. In this case, instead of LMF, RAN network nodes (e.g., gNBs) can be network entities, and signaling between the end device and the RAN network node occurs via other protocols (e.g., RRC, MAC, etc.).

[0116] Figure 3 A flowchart illustrating an example method 300 implemented at a first device according to some example embodiments of the present disclosure is shown. The first device may be a terminal device 110, or may be included as... Figure 1 It is part of the terminal device 110. For the purposes of discussion, method 300 will be described from the perspective of the first device.

[0117] In box 310, the first device receives configuration information from the second device, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed for the localization of the first device, and wherein the set of one or more conditions is based on at least one of: a first error margin associated with location information of the first device determined using the AI / ML model, or a second error margin associated with intermediate features of localization of the first device.

[0118] At box 320, based on the determination that at least one condition from a set of one or more conditions is satisfied, the first device sends the performance monitoring results for the AI / ML model to the second device based on the configuration information.

[0119] In some example embodiments, method 300 further includes: sending a request to a second device for auxiliary data for performance monitoring of an AI / ML model based on determining that at least one condition from a set of one or more conditions is satisfied; and receiving the auxiliary data from the second device, wherein the performance monitoring result for the AI / ML model is determined based on the received auxiliary data.

[0120] In some example embodiments, method 300 further includes: receiving auxiliary data from a second device for performance monitoring of an AI / ML model without sending a request for the auxiliary data, and wherein the performance monitoring results for the AI / ML model are determined based on the received auxiliary data.

[0121] In some example embodiments, method 300 further includes determining, in response to, that a first condition based on a first error margin is satisfied: the accuracy of the location information determined using the AI / ML model is less than a first configuration threshold.

[0122] In some example embodiments, method 300 further includes determining, in response to the following, that a second condition based on a second error margin is satisfied: the quality of an intermediate feature for locating the first device is less than a second configuration threshold.

[0123] In some example embodiments, the intermediate features include at least one of the following: timing measurement for positioning of the first device, angle measurement for positioning of the first device, phase measurement for positioning of the first device, power measurement for positioning of the first device, or line-of-sight channel or non-line-of-sight channel indicator.

[0124] In some example embodiments, the set of one or more conditions is also based on at least one of the following characteristics: the ratio of line-of-sight (LoS) samples to non-line-of-sight (NLoS) samples associated with measurements for positioning of the first device; the number of LoS samples associated with measurements for positioning of the first device; the number of NLoS samples associated with measurements for positioning of the first device; the number of paths associated with measurements for positioning of the first device; the number of samples associated with measurements for positioning of the first device; the timestamp range associated with measurements for positioning of the first device; the margin of the timing error group (TEG) associated with measurements performed by the first device; changes in the list of cells that are valid for auxiliary data and detectable by the first device; or the deployment of an AI / ML model on the first device.

[0125] In some example embodiments, the set of one or more conditions is further based on at least one of the following: the difference between the ratio of LosS samples to NLoS samples to be used for performance monitoring or inference and the ratio of LosS samples to NLoS samples to be used for training the AI / ML model; the difference between the number of LosS samples to be used for performance monitoring or inference and the number of LosS samples to be used for training the AI / ML model; the difference between the number of NLoS samples to be used for performance monitoring or inference and the number of NLoS samples to be used for training the AI / ML model; the difference between the number of paths to be used for performance monitoring or inference and the number of paths to be used for training the AI / ML model; the difference between the number of samples to be used for performance monitoring or inference and the number of samples to be used for training the AI / ML model; the difference between the timestamp in collecting DL PRS measurements to be used for performance monitoring or inference and the timestamp when collecting DL PRS measurements to be used for training the AI / ML model; and the difference between the TEG margin associated with the measurement to be used for performance monitoring or inference and the TEG margin associated with the measurement used for training the AI / ML model.

[0126] In some example embodiments, method 300 further includes triggering additional data collection for performance monitoring and / or for training AI / ML models based on determining that at least one condition from a set of one or more conditions is not met.

[0127] In some example embodiments, the set of one or more conditions is a first set of one or more conditions, and the configuration information also includes a second set of one or more conditions for reporting performance monitoring results, and the second set of one or more conditions is based on at least one of the following: reporting when performance monitoring is triggered, a comparison of the performance metric in the performance monitoring results with a threshold of a third configuration, or a reporting period.

[0128] In some example embodiments, method 300 further includes: performing performance monitoring on the AI / ML model based on determining that at least one condition from a first set of one or more conditions is satisfied; and sending the performance monitoring results for the AI / ML model to a second device based on determining that at least one condition from a second set of one or more conditions is satisfied.

[0129] In some example embodiments, the configuration information also includes an indication of whether the second device will provide auxiliary data for performance monitoring of the AI / ML model, or an indication of whether the first device needs to request auxiliary data.

[0130] In some example embodiments, the first device is a terminal device or is included in a terminal device, and the second device is or is included in a location management function (LMF) node.

[0131] Figure 4 A flowchart illustrating an example method 400 implemented at a second device according to some example embodiments of the present disclosure is shown. The second device may be an LMF 120, or may be included as... Figure 1 This is part of LMF 120. For the purposes of discussion, method 400 will be described from the perspective of the second device.

[0132] In box 410, the second device sends configuration information to the first device, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed at the first device for the localization of the first device, and wherein the set of one or more conditions is based on at least one of the following: a first error margin associated with location information of the first device determined using the AI / ML model, or a second error margin associated with intermediate features of localization of the first device.

[0133] At box 420, the second device receives performance monitoring results for the AI / ML model from the first device based on configuration information, wherein the performance monitoring results are received based on the satisfaction of at least one condition from a set of one or more conditions.

[0134] In some example embodiments, method 400 further includes: receiving from a first device a request for auxiliary data for performance monitoring of an AI / ML model; and sending the auxiliary data to the first device, wherein the performance monitoring results of the AI / ML model are based on the auxiliary data.

[0135] In some example embodiments, method 400 further includes: sending auxiliary data for performance monitoring of an AI / ML model to a first device based on determining that at least one condition from a set of one or more conditions is satisfied, and wherein the performance monitoring result of the AI / ML model is based on the received auxiliary data.

[0136] In some example embodiments, a first condition based on a first error margin is satisfied in response to the accuracy of the location information determined using an AI / ML model being less than a first configuration threshold.

[0137] In some example embodiments, in response to the quality of intermediate features for positioning of the first device being less than a second configuration threshold, a second condition based on a second error margin is satisfied.

[0138] In some example embodiments, the intermediate features include at least one of the following: timing measurement for positioning of the first device, angle measurement for positioning of the first device, phase measurement for positioning of the first device, power measurement for positioning of the first device, or line-of-sight channel or non-line-of-sight channel indicator.

[0139] In some example embodiments, the set of one or more conditions is also based on at least one of the following: the ratio of line-of-sight (LoS) samples to non-line-of-sight (NLoS) samples associated with measurements for positioning of the first device; the number of LoS samples associated with measurements for positioning of the first device; the number of NLoS samples associated with measurements for positioning of the first device; the number of paths associated with measurements for positioning of the first device; the number of samples associated with measurements for positioning of the first device; the range of timestamps associated with measurements for positioning of the first device; the margin of the timing error group (TEG) associated with measurements performed by the first device; changes in the list of cells that are valid for auxiliary data and detectable by the first device; or the deployment of an AI / ML model on the first device.

[0140] In some example embodiments, the set of one or more conditions is further based on at least one of the following: the difference between the ratio of LosS samples to NLoS samples to be used for performance monitoring or inference and the ratio of LosS samples to NLoS samples to be used for training the AI / ML model; the difference between the number of LosS samples to be used for performance monitoring or inference and the number of LosS samples to be used for training the AI / ML model; the difference between the number of NLoS samples to be used for performance monitoring or inference and the number of NLoS samples to be used for training the AI / ML model; the difference between the number of paths to be used for performance monitoring or inference and the number of paths to be used for training the AI / ML model; the difference between the number of samples to be used for performance monitoring or inference and the number of samples to be used for training the AI / ML model; the difference between the timestamp in collecting DL PRS measurements to be used for performance monitoring or inference and the timestamp when collecting DL PRS measurements to be used for training the AI / ML model; and the difference between the TEG margin associated with the measurement to be used for performance monitoring or inference and the TEG margin associated with the measurement used for training the AI / ML model.

[0141] In some example embodiments, the set of one or more conditions is a first set of one or more conditions, and the configuration information also includes a second set of one or more conditions for reporting performance monitoring results, and the second set of one or more conditions is based on at least one of the following: reporting when performance monitoring is triggered, a comparison of the performance metric in the performance monitoring results with a threshold configured in a third configuration, or a reporting period.

[0142] In some exemplary embodiments, performance monitoring results are received based on at least one condition from a set of first or more conditions and at least one condition from a second set of one or more conditions.

[0143] In some example embodiments, the configuration information also includes an indication of whether the second device will provide auxiliary data for performance monitoring of the AI / ML model, or whether the first device needs to request auxiliary data.

[0144] In some example embodiments, the first device is a terminal device or is included in a terminal device, and the second device is a Location Management Function (LMF) node or is included in a Location Management Function (LMF) node.

[0145] In some example embodiments, a first means capable of performing any of the methods in method 300 (e.g., Figure 1 The terminal device 110 may include components for performing the corresponding operations of method 300. The device may be implemented in any suitable form. For example, the device may be implemented in a circuit or software module. The first device may be implemented as or included in... Figure 1 In terminal device 110.

[0146] In some example embodiments, the first device includes: a component for receiving configuration information from a second device, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed for the localization of the first device, and wherein the set of one or more conditions is based on at least one of: a first error margin associated with location information of the first device determined using the AI / ML model, or a second error margin associated with intermediate features of localization of the first device; and a component for sending performance monitoring results of the AI / ML model to the second device based on the configuration information, according to determining that at least one condition from the set of one or more conditions is satisfied.

[0147] In some example embodiments, the first device further includes: a component for sending a request for auxiliary data for performance monitoring of an AI / ML model to a second device based on determining that at least one condition from a set of one or more conditions is satisfied; and a component for receiving the auxiliary data from the second device, wherein the performance monitoring result for the AI / ML model is determined based on the received auxiliary data.

[0148] In some example embodiments, the first device further includes a component for receiving auxiliary data for performance monitoring of an AI / ML model from the second device without sending a request for the auxiliary data, and wherein the performance monitoring result for the AI / ML model is determined based on the received auxiliary data.

[0149] In some example embodiments, the first device further includes: a component for determining, in response to, that a first condition based on a first error margin is satisfied: the accuracy of the location information determined using the AI / ML model is less than a first configuration threshold.

[0150] In some example embodiments, the first device further includes: a component for determining, in response to the following, that a second condition based on a second error margin is satisfied: the quality of an intermediate feature of the positioning of the first device is less than a second configuration threshold.

[0151] In some example embodiments, the intermediate features include at least one of the following: timing measurement for positioning of the first device, angle measurement for positioning of the first device, phase measurement for positioning of the first device, power measurement for positioning of the first device, or line-of-sight channel or non-line-of-sight channel indicator.

[0152] In some example embodiments, the set of one or more conditions is also based on at least one of the following characteristics: the ratio of line-of-sight (LoS) samples to non-line-of-sight (NLoS) samples associated with the measurement of the location of the first device, the number of LoS samples associated with the measurement of the location of the first device, the number of NLoS samples associated with the measurement of the location of the first device, the number of paths associated with the measurement of the location of the first device, the number of samples associated with the measurement of the location of the first device, the timestamp range associated with the measurement of the location of the first device, the timing error group (TEG) margin associated with the measurement performed by the first device, a change in the list of cells that are valid for auxiliary data and detectable by the first device, or the deployment of an AI / ML model on the first device.

[0153] In some example embodiments, the set of one or more conditions is further based on at least one of the following: the difference between the ratio of LosS samples to NLoS samples to be used for performance monitoring or inference and the ratio of LosS samples to NLoS samples to be used for training the AI / ML model; the difference between the number of LosS samples to be used for performance monitoring or inference and the number of LosS samples to be used for training the AI / ML model; the difference between the number of NLoS samples to be used for performance monitoring or inference and the number of NLoS samples to be used for training the AI / ML model; the difference between the number of paths to be used for performance monitoring or inference and the number of paths to be used for training the AI / ML model; the difference between the number of samples to be used for performance monitoring or inference and the number of samples to be used for training the AI / ML model; the difference between the timestamp in collecting DL PRS measurements to be used for performance monitoring or inference and the timestamp when collecting DL PRS measurements to be used for training the AI / ML model; and the difference between the TEG margin associated with the measurement to be used for performance monitoring or inference and the TEG margin associated with the measurement used for training the AI / ML model.

[0154] In some example embodiments, the first device further includes a component for triggering additional data collection for performance monitoring and / or for training an AI / ML model based on determining that at least one condition from a set of one or more conditions is not met.

[0155] In some example embodiments, the set of one or more conditions is a first set of one or more conditions, and the configuration information also includes a second set of one or more conditions for reporting performance monitoring results, and the second set of one or more conditions is based on at least one of the following: reporting when performance monitoring is triggered, a comparison of performance metrics in the performance monitoring results with a threshold configured in a third configuration, or a reporting period.

[0156] In some example embodiments, the first device further includes: a component for performing performance monitoring on the AI / ML model based on determining that at least one condition from a first set of one or more conditions is satisfied; and a component for sending the performance monitoring results of the AI / ML model to the second device based on determining that at least one condition from a second set of one or more conditions is satisfied.

[0157] In some example embodiments, the configuration information also includes an indication of whether the second device will provide auxiliary data for performance monitoring of the AI / ML model, or an indication of whether the first device is required to request auxiliary data.

[0158] In some example embodiments, the first device is a terminal device or is included in a terminal device, and the second device is a Location Management Function (LMF) node or is included in a Location Management Function (LMF) node.

[0159] In some example embodiments, a second means capable of performing any of the methods in method 400 (e.g., Figure 1 The LMF 120 in the device may include components for performing the corresponding operations of method 400. The device may be implemented in any suitable form. For example, the device may be implemented in a circuit or software module. The second device may be implemented as or included in... Figure 1 In LMF 120.

[0160] In some example embodiments, the second device includes: a component for sending configuration information to the first device, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed at the first device for the localization of the first device, and wherein the set of one or more conditions is based on at least one of: a first error margin associated with location information of the first device determined using the AI / ML model, or a second error margin associated with intermediate features used for the localization of the first device; and a unit for receiving performance monitoring results for the AI / ML model from the first device based on the configuration information, wherein the performance monitoring results are received based on at least one condition from the set of one or more conditions being met.

[0161] In some example embodiments, the second device further includes: a component for receiving from the first device a request for auxiliary data for performance monitoring of an AI / ML model; and a component for sending the auxiliary data to the first device, wherein the performance monitoring results for the AI / ML model are based on the auxiliary data.

[0162] In some example embodiments, the second device further includes: a component for sending auxiliary data for performance monitoring of an AI / ML model to the first device based on determining that at least one condition from a set of one or more conditions is satisfied, and wherein the performance monitoring results for the AI / ML model are based on the received auxiliary data.

[0163] In some example embodiments, a first condition based on a first error margin is satisfied in response to the accuracy of the location information determined using an AI / ML model being less than a first configuration threshold.

[0164] In some example embodiments, a second condition based on a second error margin is satisfied in response to the quality of an intermediate feature used to locate the first device being less than a second configuration threshold.

[0165] In some example embodiments, the intermediate features include at least one of the following: timing measurement for positioning of the first device, angle measurement for positioning of the first device, phase measurement for positioning of the first device, power measurement for positioning of the first device, or line-of-sight channel or non-line-of-sight channel indicator.

[0166] In some example embodiments, the set of one or more conditions is also based on at least one of the following: the ratio of line-of-sight (LoS) samples to non-line-of-sight (NLoS) samples associated with the measurement of positioning for the first device; the number of LoS samples associated with the measurement of positioning for the first device; the number of NLoS samples associated with the measurement of positioning for the first device; the number of paths associated with the measurement of positioning for the first device; the number of samples associated with the measurement of positioning for the first device; the range of timestamps associated with the measurement of positioning for the first device; the margin of the timing error group (TEG) associated with the measurement performed by the first device; a change in the list of cells that are valid for auxiliary data and can be detected by the first device; or the deployment of an AI / ML model on the first device.

[0167] In some example embodiments, the set of one or more conditions is further based on at least one of the following: the difference between the ratio of LosS samples to NLoS samples to be used for performance monitoring or inference and the ratio of LosS samples to NLoS samples to be used for training the AI / ML model; the difference between the number of LosS samples to be used for performance monitoring or inference and the number of LosS samples to be used for training the AI / ML model; the difference between the number of NLoS samples to be used for performance monitoring or inference and the number of NLoS samples to be used for training the AI / ML model; the difference between the number of paths to be used for performance monitoring or inference and the number of paths to be used for training the AI / ML model; the difference between the number of samples to be used for performance monitoring or inference and the number of samples to be used for training the AI / ML model; the difference between the timestamp in collecting DL PRS measurements to be used for performance monitoring or inference and the timestamp when collecting DL PRS measurements to be used for training the AI / ML model; and the difference between the TEG margin associated with the measurement to be used for performance monitoring or inference and the TEG margin associated with the measurement used for training the AI / ML model.

[0168] In some example embodiments, the set of one or more conditions is a first set of one or more conditions, and the configuration information also includes a second set of one or more conditions for reporting performance monitoring results, and the second set of one or more conditions is based on at least one of the following: reporting when performance monitoring is triggered, a comparison of the performance metric in the performance monitoring results with a performance threshold configured in a third configuration, or a reporting period.

[0169] In some exemplary embodiments, performance monitoring results are received based on the satisfaction of at least one condition from a first set of one or more conditions and the satisfaction of at least one condition from a second set of one or more conditions.

[0170] In some example embodiments, the configuration information also includes an indication of whether the second device will provide auxiliary data for performance monitoring of the AI / ML model, or an indication of whether the first device needs to request auxiliary data.

[0171] In some example embodiments, the first device is a terminal device or is included in a terminal device, and the second device is or is included in a location management function (LMF) node.

[0172] Figure 5 This is a simplified block diagram of a device 500 suitable for implementing exemplary embodiments of the present disclosure. The device 500 can be provided to implement a communication device, for example, such as... Figure 1The terminal device 110, LMF 120, or network node 130 shown are illustrated. As shown, device 500 includes one or more processors 510, one or more memories 520 coupled to processor 510, and one or more communication modules 540 coupled to processor 510.

[0173] Communication module 540 is used for bidirectional communication. Communication module 540 has one or more communication interfaces to facilitate communication with one or more other modules or devices. The communication interface can represent any interface necessary for communication with other network elements. In some example embodiments, communication module 540 may include at least one antenna.

[0174] As a non-limiting example, processor 510 can be any type suitable for a local technology network and can include one or more of the following: general-purpose computer, special-purpose computer, microprocessor, digital signal processor (DSP), and processor based on a multi-core processor architecture. Device 500 can have multiple processors, such as application-specific integrated circuit chips that are time-dependent on a clock that synchronizes with the main processor.

[0175] Memory 520 may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memories include, but are not limited to, read-only memory (ROM) 524, electrically programmable read-only memory (EPROM), flash memory, hard disk, compact disc (CD), digital video disc (DVD), optical disc, laser disc, and other magnetic and / or optical storage. Examples of volatile memories include, but are not limited to, random access memory (RAM) 522 and other volatile memories that will not persist for the duration of a power outage.

[0176] Computer program 530 includes computer-executable instructions that are executed by an associated processor 510. The instructions of program 530 may include instructions for performing operations / actions of some example embodiments of this disclosure. Program 530 may be stored in memory (e.g., ROM 524). Processor 510 can perform any suitable actions and processes by loading program 530 into RAM 522.

[0177] The exemplary embodiments of this disclosure can be implemented by program 530, enabling device 500 to perform as described in the reference. Figures 2 to 4 Any process discussed in this disclosure. Exemplary embodiments of this disclosure may also be implemented by hardware or a combination of software and hardware.

[0178] In some example embodiments, program 530 may be tangibly contained in a computer-readable medium, which may be included in device 500 (such as in memory 520) or other storage device accessible by device 500. Device 500 may load program 530 from the computer-readable medium into RAM 522 for execution. In some example embodiments, the computer-readable medium may include any type of non-transitory storage medium, such as ROM, EPROM, flash memory, hard disk, CD, DVD, etc. As used herein, the term "non-transitory" is a limitation of the medium itself (i.e., tangible, not tactile), rather than a limitation of the persistence of data storage (e.g., RAM versus ROM).

[0179] Figure 6 An example of a computer-readable medium 600 is shown, which may be in the form of a CD, DVD, or other optical storage disc. The computer-readable medium 600 has a program 530 stored thereon.

[0180] Generally, the various embodiments of this disclosure can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects can be implemented in hardware, and others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. Although various aspects of the embodiments of this disclosure are shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that the blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware, or controllers or other computing devices, or some combination thereof, as non-limiting examples.

[0181] Some exemplary embodiments of this disclosure also provide at least one computer program product tangibly stored on a computer-readable medium, such as a non-transitory computer-readable medium. The computer program product includes computer-executable instructions that execute in a device on a target physical or virtual processor, such as those included in a program module, to perform any of the methods described above. Typically, a program module includes routines, programs, libraries, objects, classes, components, data structures, etc., that perform a particular task or implement a particular abstract data type. In various embodiments, the functionality of a program module can be combined or split among program modules as needed. The machine-executable instructions for a program module can execute within a local or distributed device. In a distributed device, the program module can reside on both local and remote storage media.

[0182] Program code used to perform the methods of this disclosure may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that, when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a stand-alone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0183] In the context of this disclosure, computer program code or related data may be carried by any suitable carrier wave to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carrier waves include signals, computer-readable media, etc.

[0184] Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable media can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination thereof. More specific examples of computer-readable storage media will include electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0185] Furthermore, although operations are described in a specific order, this should not be construed as requiring that such operations be performed in the specific order shown or sequentially, or requiring that all shown operations be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure, but rather as a description of features that may be specific to particular embodiments. Unless explicitly stated otherwise, certain features described in the context of a single embodiment may also be implemented in combination in a single embodiment. Conversely, unless explicitly stated otherwise, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0186] Although this disclosure has been described in language specific to structural features and / or methodological actions, it should be understood that the disclosure as defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are disclosed as exemplary forms for implementing the claims.

[0187] Furthermore, the various implementations of this disclosure can be described with reference to the following terms, and their features can be combined in any reasonable manner.

[0188] Clause 1. A first device for communication, comprising: at least one processor; and at least one memory storing instructions, the instructions, when executed by the at least one processor, causing the first device to at least: receive configuration information from a second device, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed for localization of the first device, and wherein the set of one or more conditions is based on at least one of: a first error margin associated with location information of the first device determined using the AI / ML model, or a second error margin associated with intermediate features of localization of the first device; and, based on the configuration information, send performance monitoring results for the AI / ML model to the second device.

[0189] Clause 2. The first apparatus according to Clause 1, wherein the first apparatus is configured to: send a request to the second apparatus for auxiliary data for performance monitoring of the AI / ML model based on determining that at least one condition from a set of the one or more conditions is satisfied; and receive the auxiliary data from the second apparatus, wherein the performance monitoring result for the AI / ML model is determined based on the received auxiliary data.

[0190] Clause 3. The first apparatus according to Clause 1, wherein the first apparatus is configured to: receive auxiliary data for performance monitoring of the AI / ML model from the second apparatus without sending a request for the auxiliary data, and wherein the performance monitoring result for the AI / ML model is determined based on the received auxiliary data.

[0191] Clause 4. The first device according to Clause 1, wherein the first device is configured to: determine, in response to the following, that a first condition based on the first error margin is satisfied: the accuracy of the location information determined using the AI / ML model is less than a first configuration threshold.

[0192] Clause 5. The first device according to Clause 1, wherein the first device is configured such that: in response to the following, a second condition based on the second error margin is satisfied: the quality of the intermediate feature for positioning of the first device is less than a second configuration threshold.

[0193] Clause 6. The first device according to Clause 1, wherein the intermediate feature includes at least one of the following: a timing measurement for positioning of the first device, an angle measurement for positioning of the first device, a phase measurement for positioning of the first device, a power measurement for positioning of the first device, or a line-of-sight or non-line-of-sight channel indicator.

[0194] Clause 7. The first device according to Clause 1, wherein the set of one or more conditions is further based on at least one of the following characteristics: the ratio of line-of-sight LosS samples to non-line-of-sight NLoS samples associated with measurements of positioning of the first device, the number of LosS samples associated with measurements of positioning of the first device, the number of NLoS samples associated with measurements of positioning of the first device, the number of paths associated with measurements of positioning of the first device, the number of samples associated with measurements of positioning of the first device, the timestamp range associated with measurements of positioning of the first device, the timing error group TEG margin associated with measurements performed by the first device, changes in the cell list that are valid for auxiliary data and detectable by the first device, or the deployment of the AI / ML model on the first device.

[0195] Clause 8. The first apparatus according to Clause 7, wherein the set of one or more conditions is further based on at least one of the following: the difference between the ratio of LosS samples to NLoS samples to be used for performance monitoring or inference and the ratio of LosS samples to NLoS samples to be used for training the AI / ML model; the difference between the number of LosS samples to be used for performance monitoring or inference and the number of LosS samples to be used for training the AI / ML model; the difference between the number of NLoS samples to be used for performance monitoring or inference and the number of NLoS samples to be used for training the AI / ML model; the difference between the number of paths to be used for performance monitoring or inference and the number of paths to be used for training the AI / ML model; the difference between the number of samples to be used for performance monitoring or inference and the number of samples to be used for training the AI / ML model; the difference between the timestamp when collecting DL PRS measurements to be used for performance monitoring or inference and the timestamp when collecting DL PRS measurements to be used for training the AI / ML model; and the difference between the TEG margin associated with the measurement to be used for performance monitoring or inference and the TEG margin associated with the measurement to be used for training the AI / ML model.

[0196] Clause 9. The first device according to Clause 1, wherein the first device is further configured to: trigger additional data collection for the performance monitoring and / or for training the AI / ML model based on determining that at least one condition from the set of the one or more conditions is not satisfied.

[0197] Clause 10. The first apparatus according to Clause 1, wherein the set of one or more conditions is a first set of one or more conditions, and the configuration information further includes a second set of one or more conditions for reporting the performance monitoring results, and the second set of one or more conditions is based on at least one of the following: reporting when performance monitoring is triggered, a comparison of the performance metric in the performance monitoring results with a threshold of a third configuration, or a reporting period.

[0198] Clause 11. The first apparatus according to Clause 10, wherein the first apparatus is configured to: perform performance monitoring on the AI / ML model based on determining that at least one condition from a first set of the one or more conditions is satisfied; and send performance monitoring results for the AI / ML model to the second apparatus based on determining that at least one condition from a second set of the one or more conditions is satisfied.

[0199] Clause 12. The first apparatus according to any one of claims 1 to 11, wherein the configuration information further includes an indication of whether the second apparatus will provide auxiliary data for performance monitoring of the AI / ML model, or an indication of whether the first apparatus needs to request the auxiliary data.

[0200] Clause 13. A second means for communication, comprising: at least one processor; and at least one memory storing instructions, which, when executed by the at least one processor, cause the second means to at least: send configuration information to a first means, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed at the first means for localization of the first means, and wherein the set of one or more conditions is based on at least one of: a first error margin associated with location information of the first means determined using the AI / ML model, or a second error margin associated with intermediate features of localization of the first means; and receive, based on the configuration information, performance monitoring results for the AI / ML model from the first means, wherein the performance monitoring results are received based on the satisfaction of at least one condition from the set of one or more conditions.

[0201] Clause 14. A method for communication, comprising: receiving configuration information from a second device, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed for positioning of a first device, and wherein the set of one or more conditions is based on at least one: a first error margin associated with location information of the first device determined using the AI / ML model, or a second error margin associated with intermediate features of positioning of the first device; and transmitting performance monitoring results for the AI / ML model to the second device based on the configuration information, provided that at least one condition from the set of one or more conditions is satisfied.

[0202] Clause 15. A method for communication, comprising: sending configuration information to a first device, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed at the first device for localization of the first device, and wherein the set of one or more conditions is based on at least one of: a first error margin associated with location information of the first device determined using the AI / ML model, or a second error margin associated with intermediate features of localization of the first device; and receiving, based on the configuration information, performance monitoring results for the AI / ML model from the first device, wherein the performance monitoring results are received based on the satisfaction of at least one condition from the set of one or more conditions.

Claims

1. A first device for communication, comprising: At least one processor; as well as At least one memory storing instructions, which, when executed by the at least one processor, cause the first device to at least: Configuration information is received from a second device, the configuration information including a set of one or more conditions for triggering performance monitoring of an artificial intelligence / machine learning (AI / ML) model, wherein the AI / ML model is deployed for positioning of the first device, and wherein the set of one or more conditions is based on at least one of the following: The first error margin associated with the position information of the first device determined using the AI / ML model, or A second error margin associated with intermediate features of the positioning of the first device; as well as Based on the determination that at least one condition from the set of one or more conditions is satisfied, the performance monitoring results for the AI / ML model are sent to the second device according to the configuration information.

2. The first device according to claim 1, wherein the first device is configured to: Based on the determination that at least one condition from the set of one or more conditions is satisfied, a request for auxiliary data for performance monitoring of the AI / ML model is sent to the second device; and Receive the auxiliary data from the second device, and The performance monitoring results for the AI / ML model are determined based on the received auxiliary data.

3. The first device according to claim 1, wherein the first device is configured to: Receive auxiliary data from the second device for performance monitoring of the AI / ML model, without sending a request for the auxiliary data, and The performance monitoring results for the AI / ML model are determined based on the received auxiliary data.

4. The first device according to claim 1, wherein the first device is configured to: In response to the following, it is determined that a first condition based on the first error margin is satisfied: The accuracy of the location information determined using the AI / ML model is less than a first configuration threshold.

5. The first device according to claim 1, wherein the first device is configured to: In response to the following, it is determined that the second condition based on the second error margin is satisfied: The quality of the intermediate feature for the positioning of the first device is less than the second configuration threshold.

6. The first device according to claim 1, wherein the intermediate feature comprises at least one of the following: Timing measurement for positioning of the first device, Regarding the angle measurement for positioning of the first device, Phase measurement for positioning of the first device, Power measurement for positioning of the first device, or Line-of-sight or non-line-of-sight indicator.

7. The first apparatus of claim 1, wherein the set of one or more conditions is further based on at least one of the following features: The ratio of line-of-sight (LoS) samples to non-line-of-sight (NLoS) samples associated with measurements of the positioning of the first device. The number of LoS samples associated with measurements of positioning for the first device. The number of NLoS samples associated with measurements of positioning for the first device. The number of paths associated with measurements of positioning of the first device. The number of samples associated with measurements of positioning of the first device. The time stamp range associated with measurements of the positioning of the first device. The timing error group TEG margin associated with the measurement performed by the first device. For changes in the cell list that are valid for auxiliary data and detectable by the first device, or The AI / ML model is deployed on the first device.

8. The first apparatus of claim 7, wherein the set of one or more conditions is further based on at least one of the following: The difference between the ratio of Loss samples to NLoS samples used for monitoring or inference and the ratio of Loss samples to NLoS samples used for training the AI / ML model. The difference between the number of Loss samples used for performance monitoring or inference and the number of Loss samples used to train the AI / ML model. The difference between the number of NLoS samples used for performance monitoring or inference and the number of NLoS samples used to train the AI / ML model. The difference between the number of paths used for performance monitoring or inference and the number of paths used to train the AI / ML model. The difference between the number of samples used for performance monitoring or inference and the number of samples used to train the AI / ML model. The difference between the timestamps collected when DL PRS measurements are to be used for performance monitoring or inference and the timestamps collected when DL PRS measurements are used to train the AI / ML model is analyzed. The difference between the TEG margin associated with the measurement to be used for performance monitoring or inference and the TEG margin associated with the measurement used to train the AI / ML model.

9. The first device according to claim 1, wherein the first device is further configured to: Based on the determination that at least one condition from the set of one or more conditions is not met, additional data collection for performance monitoring and / or for training the AI / ML model is triggered.

10. The first apparatus of claim 1, wherein the set of one or more conditions is a first set of one or more conditions, and the configuration information further includes a second set of one or more conditions for reporting the performance monitoring results, and the second set of one or more conditions is based on at least one of the following: Report when performance monitoring is triggered. The comparison result of the performance metric in the performance monitoring results with the threshold of the third configuration, or Reporting cycle.

Images