Method and apparatus for a device for a wireless communication system

Abstract

A method, for example a computer-implemented method, for a device for a wireless communication system, the method comprising: receiving (300), e.g., from at least one further entity, first information (I-1) characterizing reference data, e.g., ground truth, for a positioning procedure based on artificial intelligence, AI, e.g., machine learning, ML, determining (302), based at least on the first information (I-1), second information (I-2) characterizing at least one performance metric associated with the positioning procedure.

Description

[0001] R.414606

[0002] - 1 -

[0003] Method and Apparatus for a Device for a Wireless Communication System

[0004] Technical Field

[0005] The disclosure relates to a method for a device for a wireless communication system.

[0006] The disclosure further relates to an apparatus for a device for a wireless communication system.

[0007] Summary

[0008] Some examples relate to a method, for example a computer-implemented method, for a device for a wireless communication system, the method comprising: receiving, e.g., from at least one further entity, first information characterizing reference data, e.g., ground truth, for a positioning procedure based on artificial intelligence, Al, e.g., machine learning, ML, determining, based at least on the first information, second information characterizing at least one performance metric associated with the positioning procedure. In some examples, this may enable to efficiently assess a performance of the positioning procedure.

[0009] In some examples, the wireless communications system may, e.g., be a wireless, for example cellular, communications system, which is for example based on and / or adheres at least partially to at least one third generation partnership project, 3GPP, radio standard such as 4G (fourth generation), 5G (fifth generation), or 6G (sixth generation), or to another other radio access technology.

[0010] In some examples, the device for the wireless communication system may, e.g., be a terminal device, e.g., user equipment, e.g., UE. R.414606

[0011] - 2 -

[0012] In some examples, the device for the wireless communication system may, e.g., be a network device, e.g., base station, e.g., gNB or a device of a network management system.

[0013] In some examples, the at least one further entity is an entity or device, respectively, for a location management function, e.g., for and / or of the wireless communication system.

[0014] In some examples, the method further comprises: sending the second information to at least one further entity, such as, e.g., an entity for a location management function. In some examples, this enables the at least one further entity to evaluate the second information, e.g., for modifying, e.g., optimizing, at least one aspect of the positioning procedure.

[0015] In some examples, the at least one performance metric associated with the positioning procedure comprises at least one of: a) a positioning error, e.g., characterizing a difference between a predicted position as, e.g., inferred by the positioning procedure and a reference, e.g., ground truth, position as characterized by the reference data, or b) an accuracy, e.g., characterizing a number or percentage of position predictions that fall within a predetermined error threshold compared to the respective reference, e.g., ground truth, position, or c) a reliability, e.g., characterizing a consistency of position predictions, e.g., over a predetermined time interval.

[0016] In some examples, the further comprises: sending one or more measurements associated with the positioning procedure (or at least one of a plurality of positioning procedures) to at least one further entity, receiving the first information from the at least one further entity. In some examples, the at least one further entity may, e.g., determine the first information based on the one or more measurements.

[0017] In some examples, the one or more measurements comprise at least one of: a) signal strength, or b) time of arrival, or c) angle of arrival.

[0018] In some examples, the method further comprises at least one of: a) receiving, e.g., from the at least one further entity, third information characterizing position R.414606

[0019] - 3 - calculation assistance data, or b) receiving, e.g., from the at least one further entity and / or from at least one position reference unit, fourth information characterizing at least one of: b1) one or more measurements of one or more position reference units, or b2) a location of the one or more position reference units, or c) using at least one of c1) the third information or c2) the fourth information for determining at least a part of c3) the first information or c4) the second information.

[0020] Some examples relate to an apparatus for performing the method according to the disclosure, wherein for example the apparatus is configured to perform the method according to the disclosure.

[0021] Some examples relate to a device for a wireless communication system, comprising at least one apparatus according to the disclosure. In some examples, the device may, e.g., be a user equipment.

[0022] Some examples relate to a method, for example a computer-implemented method, for an entity or device for a location management function, the method comprising: sending first information characterizing reference data, e.g., ground truth, for a positioning procedure based on artificial intelligence, Al, e.g., machine learning, ML, e.g., to a device for a wireless communication system, e.g., to a device according to claim 9, receiving second information, e.g., from the device, the second information characterizing at least one performance metric associated with the positioning procedure, e.g., as determined by the device, e.g., based on the first information.

[0023] In some examples, the method further comprises: evaluating the second information.

[0024] In some examples, the method further comprises at least one of: a) sending third information characterizing position calculation assistance data to the device, or b) sending, to the device, fourth information characterizing at least one of: b1) one or more measurements of one or more position reference units, or b2) a location of the one or more position reference units. R.414606

[0025] - 4 -

[0026] Some examples relate to an apparatus for performing the method according to the disclosure, e.g., according to at least one of the claims 10 to 12, wherein for example the apparatus is configured to perform the method according to the disclosure.

[0027] Some examples relate to an entity or device for a location management function, for a wireless communication system, comprising at least one apparatus according to the disclosure.

[0028] Some examples relate to a computer program comprising instructions which, when the program is executed by a computer and / or an apparatus and / or a device and / or an entity, cause the computer and / or the apparatus and / or the device and / or the entity to perform the method according to the disclosure.

[0029] Some examples relate to a computer-readable storage medium comprising instructions which, when executed by a computer and / or an apparatus and / or a device and / or an entity, cause the computer and / or the apparatus and / or the device and / or the entity to perform the method according to the disclosure.

[0030] Some examples relate to a data carrier signal carrying and / or characterizing the computer program according to the disclosure.

[0031] Some examples relate to a use of the method according to the disclosure and / or of the apparatus according to the disclosure and / or of the device and / or entity according to the disclosure and / or of the computer program according to the disclosure and / or of the computer-readable storage medium according to the disclosure and / or of the data carrier signal according to the disclosure for at least one of: a) providing reference data, e.g., ground truth, for a positioning procedure based on artificial intelligence, Al, e.g., machine learning, ML, or b) monitoring a performance of a positioning procedure based on artificial intelligence, Al, e.g., machine learning, ML, or c) enhancing a reliability of a positioning procedure based on artificial intelligence, Al, e.g., machine learning, ML.

[0032] Brief Description of Some Example Figures R.414606

[0033] - 5 -

[0034] Some example embodiments will now be described with reference to the accompanying drawings, in which:

[0035] Fig. 1 schematically depicts a simplified flow-chart,

[0036] Fig. 2 schematically depicts a simplified block diagram,

[0037] Fig. 3 schematically depicts a simplified flow-chart,

[0038] Fig. 4 schematically depicts a simplified flow-chart,

[0039] Fig. 5 schematically depicts a simplified flow-chart,

[0040] Fig. 6 schematically depicts a simplified flow-chart,

[0041] Fig. 7 schematically depicts a simplified flow-chart,

[0042] Fig. 8 schematically depicts a simplified block diagram,

[0043] Fig. 9 schematically depicts aspects of use.

[0044] Some examples, see, for example, Fig. 1, 2, relate to a method, for example a computer-implemented method, for a device 10 (Fig. 2) for a wireless communication system 1 , the method comprising: receiving 300 (Fig. 1), e.g., from at least one further entity 20, first information 1-1 characterizing reference data RD, e.g., ground truth, for a positioning procedure POS-PROC based on artificial intelligence, Al, e.g., machine learning, ML, determining 302, based at least on the first information 1-1 , second information I-2 characterizing at least one performance metric PM-POS-PROC associated with the positioning procedure POS-PROC. In some examples, this may enable to efficiently assess a performance of the Al- and / or ML-based positioning procedure POS-PROC.

[0045] In some examples, Fig. 2, the wireless communications system 1 may, e.g., be a wireless, for example cellular, communications system, which is for example based on and / or adheres at least partially to at least one third generation partnership project, 3GPP, radio standard such as 4G (fourth generation), 5G R.414606

[0046] - 6 -

[0047] (fifth generation), or 6G (sixth generation), or to another other radio access technology.

[0048] In some examples, Fig. 2, the device 10 for the wireless communication system 1 may, e.g., be a terminal device, e.g., user equipment, e.g., UE.

[0049] In some examples, the device 10 for the wireless communication system 1 may, e.g., be a network device, e.g., base station, e.g., gNB or a device of a network management system.

[0050] In some other examples, the device 10 may, e.g., represent any other device or type of device that may be associated with, e.g., be used for, performing at least one positioning procedure that is, e.g., based on at least one of Al or ML.

[0051] In some examples, the reference data RD may characterize or represent, respectively, label information, e.g., ground truth labels indicating a ground truth, e.g., true position information, which may, in some examples, be used for training of the Al- and / or ML-based positioning procedure POS-PROC.

[0052] In some examples, Fig. 2, the at least one further entity 20 is an entity or device, respectively, for a location management function LMF, e.g., for and / or of the wireless communication system 1.

[0053] In some examples, Fig. 1 , the method further comprises: sending 304 the second information I-2 to the at least one further entity 20 (and / or to another entity (not shown)), such as, e.g., an entity for a location management function. In some examples, this enables the at least one further entity 20 to evaluate the second information I-2, e.g., for modifying, e.g., optimizing, at least one aspect of the positioning procedure POS-PROC, such as, e.g., modifying, e.g., optimizing at least one parameter and / or hyperparameter of the positioning procedure POS- PROC.

[0054] In some examples, Fig. 2, the at least one performance metric PM-POS-PROC associated with the positioning procedure POS-PROC comprises at least one of: a) a positioning error PM-1 , e.g., characterizing a difference between a predicted position as, e.g., inferred by the positioning procedure POS-PROC and a R.414606

[0055] - 7 - reference, e.g., ground truth, position as, e.g., characterized by the reference data RD, or b) an accuracy PM-2, e.g., characterizing a number or percentage of position predictions that fall within a predetermined error threshold compared to the respective reference, e.g., ground truth, position, or c) a reliability PM-3, e.g., characterizing a consistency of position predictions, e.g., over a predetermined time interval. In some examples, alternatively or additionally to the abovementioned aspects PM-1 , PM-2, PM-3, one or more further aspects may be provided as the at least one performance metric PM-POS-PROC or as a part of the at least one performance metric PM-POS-PROC.

[0056] In some examples, Fig. 3, the further comprises: sending 310 one or more measurements MEAS-POS-PROC associated with the positioning procedure POS-PROC to the at least one further entity 20 (Fig. 2), receiving 314 the first information 1-1 from the at least one further entity 20. In some examples, the at least one further entity 20 may, e.g., determine the first information 1-1 based on the one or more measurements MEAS-POS-PROC, see the dashed, optional, block 312 of Fig. 3.

[0057] In some examples, Fig. 2, , the one or more measurements MEAS-POS-PROC comprise at least one of: a) signal strength M-1 , or b) time of arrival M-2, or c) angle of arrival M-3, and / or at least one further parameter that may be used to determine at last one aspect of the first information.

[0058] In some examples, Fig. 4, the method further comprises at least one of: a) receiving 320, e.g., from the at least one further entity 20, third information I-3 characterizing position calculation assistance data, or b) receiving 322, e.g., from the at least one further entity 20 and / or from at least one position reference unit PRU, fourth information I-4 characterizing at least one of: b1) one or more measurements of one or more position reference units PRU, or b2) a location of the one or more position reference units PRU, or c) using 324 at least one of c1) the third information I-3 or c2) the fourth information I-4 for determining at least a part 1-1 ', I-2' of c3) the first information 1-1 or c4) the second information I-2.

[0059] In some examples, position calculation assistance data may refer to supplementary information, e.g., data, e.g., provided to enhance the accuracy and efficiency of determining a device's location within a wireless network. This R.414606

[0060] - 8 - data may, e.g., include at least one of a) measurement data (e.g., signal strength, time of arrival, timestamps), or b) reference data (e.g., precise locations from Position Reference Units), or c) environmental information (e.g., mapping and building structures), or d) network information (e.g., base station locations and network topology). In some examples, one or more of these aspects of position calculation assistance data may collectively support AI / ML models such as, e.g., provided for the positioning procedure POS-PROC, in calibrating, validating, and performing real-time position calculations.

[0061] Some examples, Fig. 2, relate to an apparatus 100 for performing the method according to the disclosure, wherein for example the apparatus 100 is configured to perform the method according to the disclosure. In some examples, the apparatus 100 or its functionality, respectively, may be integrated in the device 10.

[0062] Some examples, Fig. 2, relate to a device 10 for a wireless communication system 1 , comprising at least one apparatus 100 according to the disclosure. In some examples, the device 10 may, e.g., be a user equipment.

[0063] Some examples, Fig. 5, relate to a method, for example a computer-implemented method, for an entity or device 20 (Fig. 2) for a location management function LMF, the method comprising: sending 350 (Fig. 5) first information 1-1 characterizing reference data RD, e.g., ground truth, for a positioning procedure POS-PROC based on artificial intelligence, Al, e.g., machine learning, ML, e.g., to a device 10 (Fig. 2) for a wireless communication system 1 , e.g., to a device 10 according to claim 9, receiving 352 second information I-2, e.g., from the device 10, the second information I-2 characterizing at least one performance metric PM-POS-PROC associated with the positioning procedure POS-PROC, e.g., as determined by the device 10, e.g., based on the first information 1-1.

[0064] In some examples, Fig. 5, the method further comprises: evaluating 354 the second information I-2. In some examples, this may enable the device 20 and / or the location management function LMF to assess a performance of the positioning procedure POS-PROC, and / or to modify at least one parameter and / or hyperparameter, e.g., of at least one Al- or ML-based model of the R.414606

[0065] - 9 - positioning procedure POS-PROC, e.g., to improve the performance of the positioning procedure POS-PROC.

[0066] In some examples, Fig. 6, the method further comprises at least one of: a) sending 360 third information I-3 characterizing position calculation assistance data to the device 10, or b) sending 362 , to the device 10, fourth information I-4 characterizing at least one of: b1) one or more measurements of one or more position reference units, or b2) a location of the one or more position reference units.

[0067] Some examples, Fig. 2, relate to an apparatus 200 for performing the method according to the disclosure, e.g., according to at least one of the claims 10 to 12, wherein for example the apparatus 200 is configured to perform the method according to the disclosure.

[0068] Some examples, Fig. 2, relate to an entity or device 20 for a location management function LMF, for a wireless communication system 1 , comprising at least one apparatus 200 according to the disclosure.

[0069] Fig. 7 schematically depicts a simplified flow-chart according to some examples. Element E1 symbolizes collecting data by one or more devices 10 (e.g., UE and / or gNB), 20, ... of the wireless communication system 1. In some examples, the data collection E1 may comprise at least one of: a) performing measurements MEAS- POS-PROC, also see, for example, block 310 of Fig. 3, e.g., for sending the measurements MEAS-POS-PROC to at least one other device, or b) performing positioning, e.g., using the positioning procedure POS-PROC and collecting data associated with the positioning procedure POS-PROC, or c) collecting one or more, e.g., other, operational parameters of the wireless communication system 1 or at least one component associated with the wireless communication system 1 or arranged within an environment of the wireless communication system 1.

[0070] Elements E2, E5 symbolize a determination, e.g., generation, of reference data RD, e.g., ground truth data, according to different variants. R.414606

[0071] - 10 -

[0072] As an example, element E2 symbolizes reference data generation on an LMF side (e.g., based on measurements M EAS- POS- PROC as provided by the device 10 to the device 20, see, for example, block 310 of Fig. 3) and sending, e.g., transmission, of the reference data so generated from the LMF or the device 20 to the device, e.g., UE, 10, see for example block 100 of Fig. 1 or block 350 of Fig. 5.

[0073] As a further example, element E5 symbolizes reference data generation on an LMF side (e.g., based on measurements MEAS-POS-PROC as provided by the device 10 to the device 20, see, for example, block 310 of Fig. 3), e.g., without sending, e.g., transmission, of the reference data so generated from the LMF or the device 20 to the device, e.g., UE, 10, see for example block 100 of Fig. 1 or block 350 of Fig. 5. Thus, in other words, in some examples, in element E5, the reference data RD is generated on the LMF side, e.g., for further use on the LMF side.

[0074] Elements E3, E6 symbolize a determination of at least one performance metric, see, for example block PM-POS-PROC of Fig. 2 or block 102 of Fig. 1 or block 352 of Fig. 5, wherein element E3 symbolizes the determination of the at least one performance metric on a UE side (e.g., device 10 of Fig. 2, also see block 102 of Fig. 1), according to some examples. By contrast, element E6 of Fig. 7 symbolizes the determination of the at least one performance metric on the LMF side, e.g., by the further device 20 and / or by the location management function LMF (Fig. 2), according to some other examples.

[0075] Elements E4, E7 symbolize respective feedback actions. According to some examples, element E4 symbolizes a feedback, e.g., of the at least one performance metric, as, e.g., obtained according to element E3, to a model associated with the positioning procedure POS-PROC, e.g., on the UE side, e.g., device 10, also see arrow a1 of Fig. 7. According to some other examples, element E7 symbolizes a feedback, e.g., of the at least one performance metric, as, e.g., obtained according to element E6, to a model associated with the positioning procedure POS-PROC, e.g., on the LMF side, e.g., device 20, also see arrow a2 of Fig. 7. R.414606

[0076] - 11 -

[0077] Some examples, Fig. 8, relate to a configuration or apparatus 400 for performing at least one aspect of the disclosure.

[0078] In some examples, Fig. 8, the apparatus 400 comprises at least one calculating unit, e.g. processor or "computer", 402 and at least one memory unit 404 associated with (i.e. , usable by) the at least one calculating unit 402 for at least temporarily storing a computer program PRG and / or data DAT, wherein the computer program PRG is e.g. configured to at least temporarily control an operation of the apparatus 400 and / or the device 10, 20, e.g. an execution of a method according to the disclosure.

[0079] In some examples, Fig. 8, the at least one calculating unit 402 comprises at least one core (see the dashed rectangle) for executing the computer program PRG or at least parts thereof, e.g. for executing the method according to the disclosure or at least one or more steps thereof.

[0080] According to some examples, the at least one calculating unit 402 may comprise at least one of the following elements: a microprocessor, a microcontroller, a digital signal processor (DSP), a programmable logic element (e.g., FPGA, field programmable gate array), an ASIC (application specific integrated circuit), hardware circuitry, a tensor processor, a graphics processing unit (GPU).

[0081] According to further preferred embodiments, any combination of two or more of these elements is also possible.

[0082] According to some examples, the memory unit 404 comprises at least one of the following elements: a volatile memory 404a, e.g., a random-access memory (RAM), a non-volatile memory 404b, e.g., a Flash-EEPROM.

[0083] In some examples, Fig. 8, the computer program PRG is at least temporarily stored in the non-volatile memory 404b. Data DAT (e.g. associated with the first information 1-1 and / or the second information I-2 and / or the third information I-3, and the like), which may, e.g. be used for performing the method according to the disclosure, may at least temporarily be stored in the RAM 404a.

[0084] In some examples, Fig. 8, an optional computer-readable storage medium SM comprising instructions, e.g. in the form of a or the computer program PRG, may R.414606

[0085] - 12 - be provided. As an example, the storage medium SM may comprise or represent a digital storage medium such as a semiconductor memory device (e.g., solid state drive, SSD) and / or a magnetic storage medium such as a disk or harddisk drive (HDD) and / or an optical storage medium such as a compact disc (CD) or DVD (digital versatile disc) or the like.

[0086] In some examples, Fig. 8, the configuration or apparatus 400 may comprise an optional data interface 406, e.g. for bidirectional data exchange with at least one further device (not shown). As an example, by means of the data interface 406, a data carrier signal DCS may be exchanged, e.g. with an external device, for example via a wired or a wireless data transmission medium, e.g. over a (virtual) private computer network and / or a public computer network such as e.g. the Internet.

[0087] Some examples, Fig. 8, relate to a computer program comprising instructions which, when the program is executed by a computer 402, cause the computer 402 to perform the method according to the disclosure.

[0088] In some examples, at least one of the devices 10, 20 or apparatus 100, 200 may comprise a configuration identical or at least similar to the configuration or apparatus 400 of Fig. 8.

[0089] In the following, further aspects and examples are disclosed that, in some examples, may be combined with each other and / or with at least one of the aspects and examples disclosed above, e.g., to form further aspects and examples illustrating the disclosure.

[0090] In some examples, one or more information elements and / or messages may be provided, e.g., for exchanging at least one of the following elements: a) the first information 1-1, or b) the second information I-2, or c) the third information I-2, or d) the fourth information I-4, or e) the measurements M EAS- POS- PROC, e.g., between at least some of the devices or entities 10, 20, LMF.

[0091] In some examples, the principle according to the disclosure enables to address at least some challenges of at least some conventional AI / ML-based positioning systems, e.g., regarding accurately monitoring a model performance, e.g., due to R.414606

[0092] - 13 - a difficulty of generating reliable ground truth labels. In some examples, e.g., in this context, ground truth labels correspond to a true position, that is the output of a positioning system.

[0093] In some examples, the principle according to the disclosure enables to enhance an accuracy and reliability of performance metrics, ultimately leading to better AI / ML model performance associated with the positioning procedure POS-PROC, e.g., for providing a positioning system, e.g., for and / or using at least some components 10, 20, LMF of the wireless communication system 1.

[0094] In some examples, the principle according to the disclosure enables to provide a robust framework, e.g., for providing, e.g., generating, and / or utilizing reference data RD, e.g., ground truth labels, e.g., to monitor AI / ML model performance associated with at least one positioning procedure POS-PROC, e.g., in positioning systems. By implementing one or more aspects of the disclosure, more accurate and reliable performance metrics may be achieved in some examples, thus, e.g., improving an overall quality and reliability of positioning services.

[0095] In some examples, the principle according to the disclosure enables to provide different options for providing, e.g., generating, reference data RD, e.g., ground truth labels, either at a) a UE, e.g., target UE, 10, or b) at an LMF or LMF side (see, for example, device 20).

[0096] In some examples, the principle according to the disclosure enables to provide mechanisms for an accurate calculation and / or efficient reporting of performance metrics.

[0097] In some examples, the principle according to the disclosure enables to ensure flexibility, e.g., enabling an integration of various different options or methods for ground truth label generation and / or metric calculation.

[0098] In some examples, the principle according to the disclosure enables to provide, e.g., generate, ground truth labels, e.g., at a UE side ("Aspect A"), e.g., at the device 10, e.g., a target UE, e.g., for the positioning procedure POS-PROC, R.414606

[0099] - 14 - and / or at an LMF side, e.g., at the device 20 or at the location management function LMF ("Aspect B").

[0100] In some examples, in the target UE side-approach (Aspect A), the location management function LMF (and / or device 20) may provide ground truth labels to the target UE 10, e.g., based on measurements, e.g., sent from the target UE 10 (and / or gNB, e.g., if the device 10 is a gNB).

[0101] Alternatively, in some examples, in the LMF side-approach (Aspect B), the device 10, e.g., target UE, may send inference results (e.g., of a positioning procedure POS-PROC (e.g., in some examples, based on Al and / or ML, or, in some other examples, not based on Al and / or ML) to the LMF side, e.g., to at least one of the device 20 or the location management function LMF, wherein at least one of element 20, LMF may then generate the ground truth labels based on the inference results and may, e.g., calculate one or more performance metrics.

[0102] Thus, in some examples, the AI / ML-based positioning procedure POS-PROC may, e.g., be trained on data obtained by other positioning methods, e.g., legacy positioning methods, such as Uu positioning and / or sidelink positioning.

[0103] In some examples, the principle according to the disclosure enables to provide advanced algorithms, e.g., based on mean square error or root mean square error between the AI / ML output and the ground truth or between quantities derived from the output and the ground truth, such as the Euclidean or Manhattan distance, e.g., for calculating performance metrics such as positioning error, accuracy, and latency, e.g., using the generated ground truth labels. In some examples, these metrics may provide a comprehensive evaluation of the AI / ML model's performance, guiding improvements and adjustments.

[0104] In some examples, the principle according to the disclosure enables to provide and / or use optimized signaling protocols, e.g., including new message types and / or information elements and / or headers, e.g., extended headers, e.g., to ensure efficient data transmission, e.g., with minimal signaling overhead.

[0105] In some examples, the principle according to the disclosure enables to provide tools, e.g., for performance monitoring, e.g., for continuous performance R.414606

[0106] - 15 - monitoring, which may, e.g., be implemented to assess a performance of at least one AI / ML model (e.g., for the positioning procedure POS-PROC), e.g., in realtime. In some examples, one or more feedback loops may be established, e.g., to refine and improve one or more algorithms, e.g., based on performance data.

[0107] In some examples, the principle according to the disclosure may at least temporarily address and / or improve at least one of the following aspects: a) enhance accuracy, e.g., by providing reliable ground truth labels, e.g., leading to precise performance metrics and improved tuning of AI / ML models, or b) enable timely adjustments, e.g., based on accurate monitoring, e.g., enhancing the reliability and performance of the models, or c) flexibility and / or adaptability, e.g., to support multiple methods for ground truth generation and metric calculation (e.g., on UE side, e.g., side of device 10, and / or on LMF side, e.g., at element(s) 20, LMF), thus, e.g., ensuring versatility and robustness.

[0108] In some examples, the principle according to the disclosure may contribute to improve an operational efficiency, e.g., through optimized signaling protocols (e.g., using at least one of efficient data packing, or selective data transmission, or compression, or adaptive signaling depending on network conditions, or edge processing) that may reduce signaling overhead, freeing up network resources. In some examples, the scalable design ensures that the system can handle increasing amounts of data and complexity, maintaining long-term effectiveness. Furthermore, by maintaining compatibility with existing 3GPP standards, the invention facilitates widespread adoption and interoperability across different networks and devices.

[0109] In some examples, the principle according to the disclosure enables to provide one or more of the following aspects related to providing, e.g., generating, the reference data RD, e.g., as ground truth labels: a) information on a ground truth label, e.g., of the target UE 10, is generated by the location management function LMF (and / or device 20), and, e.g., provided to the target UE 10, e.g., by sending the information from any of the entities 20, LMF to the device 10. As an example, the target UE 10 (and / or gNB, in some examples) may send measurements (e.g., legacy measurements, see, for example dashed block arrow MEAS-POS-PROC) to R.414606

[0110] - 16 - the LMF (or entity 20), which may, e.g., derive the ground truth label information based at least on these measurements. b) Position calculation assistance data is provided, e.g., from LMF to the target UE 10. c) PRU measurements and / or corresponding PRU location(s) are sent via LMF to the target UE 10, e.g., reusing aspects of a 3GPP Rel-18 assistance data transfer framework, e.g., from LMF to the target UE 10. d) PRU measurements (and corresponding PRU location, e.g., if not known to the UE 10) are sent from PRU to the target UE 10. In some examples, this may, e.g., be implemented in a manner transparent to specifications, e.g., if the PRU sends information to the target UE 10 in a proprietary method.

[0111] In the following, one or more aspects of performance metric calculation according to further examples are disclosed.

[0112] In some examples, the performance metric calculation may involve several steps, e.g., to ensure accurate and reliable monitoring of AI / ML model performance, e.g., in positioning systems. In some examples, a calculation process may, e.g., be implemented either at the target UE side (see Aspect A as mentioned above) or at the LMF side (see Aspect B as mentioned above), e.g., with each option or aspect offering unique advantages.

[0113] In some examples, a performance metric calculation related to Aspect A (e.g., in the sense of a "target UE side monitoring metric calculation") may comprise one or more of the following aspects:

[0114] • Ground Truth Label Generation:

[0115] • In some examples, the LMF may generate ground truth labels based on measurements received from the target UE and / or gNB 10. In some examples, these measurements may include at least one of signal strength, time of arrival, angle of arrival, etc.

[0116] • In some examples, the generated ground truth labels are then sent to the target UE, providing a reference for performance evaluation.

[0117] • Metric Calculation at Target UE: R.414606

[0118] - 17 -

[0119] • In some examples, the target UE 10 may use the received ground truth labels, e.g., to calculate performance metrics such as positioning error, accuracy, and reliability.

[0120] • Example Metrics:

[0121] • Positioning Error: The difference between the predicted position (inferred by the AI / ML model) and the ground truth position.

[0122] • Accuracy: The percentage of predictions that fall within a certain error threshold compared to the ground truth.

[0123] • Reliability: The consistency of the positioning predictions over time.

[0124] • Reporting to LMF:

[0125] • In some examples, the target UE 10 may send the calculated performance metrics to the device 20 and / or the LMF, e.g., for further analysis and optimization.

[0126] In some examples, a performance metric calculation related to Aspect B (e.g., in the sense of a " LMF Side Monitoring Metric Calculation") may comprise one or more of the following aspects:

[0127] • Inference Result Transmission:

[0128] • In some examples, the target UE 10 may perform initial positioning using its AI / ML model (e.g., the positioning procedure POS-PROC) and may send the inference results (i.e. , the model output corresponding to the target UE’s channel measurement) to the LMF.

[0129] • Alternatively, in some examples, a PRU’s channel measurement may be sent, e.g., via the LMF, e.g., to the target UE 10, and the inference result corresponding to PRU’s channel measurement may then be sent by the target UE 10 to the LMF and / or the device 20.

[0130] • Ground Truth Label Generation at LMF:

[0131] • In some examples, the device 20 and / or LMF may generate ground truth labels based on the received measurements and historical data.

[0132] • Metric Calculation at LMF:

[0133] • In some examples, the device 20 and / or LMF may use the ground truth labels and the received inference results, e.g., to calculate performance metrics.

[0134] • In some examples, example performance metrics comprise at least one of:

[0135] • Positioning Error: Calculated by comparing the AI / ML model’s predicted positions with the ground truth positions.

[0136] • Model Confidence: Evaluates the confidence level of the AI / ML model’s predictions based on the variance of prediction errors. R.414606

[0137] - 18 -

[0138] • Latency: Measures the time taken from receiving the measurements to generating the positioning output.

[0139] • Feedback to Target UE:

[0140] • In some examples, the device 20 and / or LMF may provide feedback to the target UE 10, e.g., based on the calculated metrics, e.g., suggesting adjustments to improve model performance.

[0141] In some examples, the principle according to the disclosure enables to attain at least some of the following benefits: a) Improved accuracy: More accurate performance metrics as obtained by using the principle according to the disclosure may lead to better tuning of AI / ML models, thus, e.g., resulting in enhanced positioning accuracy. b) Enhanced reliability: Reliable ground truth labels as obtained by using the principle according to the disclosure may enable to improve an overall reliability of positioning services, providing consistent and dependable performance. c) Scalability: The flexible framework as may be provided by using the principle according to the disclosure enables to adapt to future advancements and to integrate new methods for performance monitoring, ensuring long-term relevance and effectiveness.

[0142] Some examples, Fig. 9, relate to a use 500 of the method according to the disclosure and / or of the apparatus 100, 200, 400 according to the disclosure and / or of the device 10, 20 and / or entity according to the disclosure and / or of the computer program PRG according to the disclosure and / or of the computer- readable storage medium SM according to the disclosure and / or of the data carrier signal DCS according to the disclosure for at least one of: a) providing 501 reference data RD, e.g., ground truth, e.g., in the form of labels, e.g., for a positioning procedure POS-PROC based on artificial intelligence, Al, e.g., machine learning, ML, or b) monitoring 502 a performance of a positioning procedure POS-PROC based on artificial intelligence, Al, e.g., machine learning, ML, or c) enhancing 503 a reliability of a positioning procedure based on artificial intelligence, Al, e.g., machine learning, ML.

Claims

R.414606- 19 -Claims1. A method, for example a computer-implemented method, for a device (10) for a wireless communication system (1), the method comprising:- receiving (300), e.g., from at least one further entity (20), first information (1-1) characterizing reference data (RD), e.g., ground truth, for a positioning procedure (POS-PROC) based on artificial intelligence, Al, e.g., machine learning, ML,- determining (302), based at least on the first information (1-1), second information (I-2) characterizing at least one performance metric (PM- POS-PROC) associated with the positioning procedure (POS-PROC).

2. The method according to claim 1 , wherein the device (10) is a) a terminal device, e.g., user equipment, or b) a network device, e.g., gNB.

3. The method according to any of the preceding claims, wherein the method further comprises:- sending (304) the second information (I-2) to at least one further entity (20), such as, e.g., an entity (20) for a location management function (LMF).

4. The method according to any of the preceding claims, wherein the at least one performance metric (PM-POS-PROC) associated with the positioning procedure (POS-PROC) comprises at least one of: a) a positioning error (PM-1), e.g., characterizing a difference between a predicted position as, e.g., inferred by the positioning procedure (POS- PROC), and a reference, e.g., ground truth, position as characterized by the reference data (RD), or b) an accuracy (PM-2), e.g., characterizing a number or percentage of position predictions that fall within a predetermined error threshold compared to the respective reference, e.g., ground truth, position, or c) a reliability (PM-3), e.g., characterizing a consistency of position predictions, e.g., over a predetermined time interval.R.414606- 20 -5. The method according to any of the preceding claims, wherein the method further comprises:- sending (310) one or more measurements (MEAS-POS-PROC) associated with the positioning procedure (POS-PROC) to at least one further entity (20),- receiving (314) the first information (1-1) from the at least one further entity (20).

6. The method according to claim 5, wherein the one or more measurements (MEAS-POS-PROC) comprise at least one of: a) signal strength (M-1), or b) time of arrival (M-2), or c) angle of arrival (M-3).

7. The method according to any of the preceding claims, wherein the method further comprises at least one of: a) receiving (320), e.g., from the at least one further entity (20), third information (I-3) characterizing position calculation assistance data, or b) receiving (322), e.g., from the at least one further entity (20) and / or from at least one position reference unit (PRU), fourth information (I-4) characterizing at least one of: b1) one or more measurements of one or more position reference units (PRU), or b2) a location of the one or more position reference units (PRU), or c) using (324) at least one of c1) the third information (I-3) or c2) the fourth information (I-4) for determining at least a part (1-1 ', I-2') of c3) the first information (1-1) or c4) the second information (I-2).

8. An apparatus (100) for performing the method according to at least one of the preceding claims, wherein for example the apparatus (100) is configured to perform the method according to at least one of the preceding claims.

9. A device (10) for a wireless communication system (1), comprising at least one apparatus (100) according to claim 8.R.414606- 21 -10. A method, for example a computer-implemented method, for an entity (20) for a location management function (LMF), the method comprising:- sending (350) first information (1-1) characterizing reference data, e.g., ground truth, for a positioning procedure (POS-PROC) based on artificial intelligence, Al, e.g., machine learning, ML, e.g., to a device (10) for a wireless communication system (1), e.g., to a device (10) according to claim 9,- receiving (352) second information (I-2), e.g., from the device (10), the second information (I-2) characterizing at least one performance metric (PM-POS-PROC) associated with the positioning procedure (POS- PROC), e.g., as determined by the device (10), e.g., based on the first information (1-1).11 . The method of claim 10, further comprising:- evaluating (354) the second information (I-2).

12. The method according to any of the claims 10 to 11 , wherein the method further comprises at least one of: a) sending (360) third information (I-3) characterizing position calculation assistance data to the device (10), or b) sending (362), to the device (10), fourth information (I-4) characterizing at least one of: b1) one or more measurements of one or more position reference units (PRU), or b2) a location of the one or more position reference units (PRU).

13. An apparatus (200) for performing the method according to at least one of the claims 10 to 12, wherein for example the apparatus (200) is configured to perform the method according to at least one of the claims 10 to 12.

14. An entity (20) for a location management function (LMF), for a wireless communication system (1), comprising at least one apparatus (200) according to claim 13.

15. A computer program (PRG) comprising instructions which,- when the program (PRG) is executed by a computer (402) and / or the apparatus (100) of claim 9 and / or the device (10) of claim 10, cause theR.414606- 22 - computer (402) and / or the apparatus (100) and / or the device (10) to perform the method according to at least one of the claims 1 to 7, and / or- when the program (PRG) is executed by a computer (402) and / or the apparatus (200) of claim 13 and / or the entity (20) of claim 14, cause the computer (402) and / or the apparatus (200) and / or the entity (20) to perform the method according to at least one of the claims 10 to 12.

16. A computer-readable storage medium (SM) comprising instructions (PRG) which,- when executed by a computer (402) and / or the apparatus (100) of claim 9 and / or the device (10) of claim 10, cause the computer (402) and / or the apparatus (100) and / or the device (10) to perform the method according to at least one of the claims 1 to 7, and / or- when executed by a computer (402) and / or the apparatus (200) of claim 13 and / or the entity (20) of claim 14, cause the computer (402) and / or the apparatus (200) and / or the entity (20) to perform the method according to at least one of the claims 10 to 12.

17. A data carrier signal (DCS) carrying and / or characterizing the computer program (PRG) of claim 15.

18. A use (500) of the method according to any of the claims 1 to 7 or 10 to 12 and / or of the apparatus (100; 200) according to claim 8 or 13 and / or of the device (10) and / or entity (20) of any of the claims 9 or 14 and / or of the computer program (PRG) according to claim 15 and / or of the computer- readable storage medium (SM) according to claim 16 and / or of the data carrier signal (DCS) according to claim 17 for at least one of: a) providing (501) reference data (RD), e.g., ground truth, for a positioning procedure (POS-PROC) based on artificial intelligence, Al, e.g., machine learning, ML, or b) monitoring (502) a performance of a positioning procedure (POS-PROC) based on artificial intelligence, Al, e.g., machine learning, ML, or c) enhancing (503) a reliability of a positioning procedure (POS-PROC) based on artificial intelligence, Al, e.g., machine learning, ML.

Images