Traceability error indication after AI / ML data collection and reporting

Abstract

Example embodiments of the present disclosure relate to schemes for traceability error indication after AI / ML data collection and reporting. In one aspect, data for operations are collected, where the operations include artificial intelligence or machine learning operations; reporting the data collected for operation to the network entity; determining traceability error information associated with the reported data; and sending traceability error information associated with the reported data to the network entity.

Description

TECHNICAL FIELD

[0001] Example and non-limiting example embodiments relate generally to communications, and more particularly to retroactive error indication after AI / ML data collection and reporting. BACKGROUND

[0002] Communication devices are known to obtain access to a communication network by accessing a network node. SUMMARY

[0003] According to an aspect, an apparatus comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to collect data for an operation, wherein the operation comprises an artificial intelligence or machine learning operation; report, to a network entity, data that was collected for the operation; determine retroactive error information associated with the data that was reported; and send, to the network entity, the retroactive error information associated with the data that was reported.

[0004] According to an aspect, an apparatus comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to configure a user equipment to collect data for an operation, wherein the operation comprises an artificial intelligence or machine learning operation; receive, from the user equipment, a report comprising data that was collected for the operation; receive, from the user equipment, retroactive error information associated with the data that was reported; and perform an action based on the retroactive error information associated with the data that was reported received from the user equipment.

[0005] According to an aspect, an apparatus comprises at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to receive, from a first user equipment, a report comprising data that was collected for an operation, wherein the operation comprises an artificial intelligence or machine learning operation; send, to a second user equipment, the data that was collected for the operation of the report; receive, from the second user equipment, a trigger message for performing a retroactive data error check of the data that was reported to determine retroactive error information; and send, to the first user equipment, the trigger message for performing the retroactive data error check of the data that was reported to determine the retroactive error information.

[0006] According to an aspect, an apparatus comprises at least one processor; and at least one memory that stores instructions, which when executed by the at least one processor, cause the apparatus at least to receive, from a network entity, data provided by the network entity, the data comprising data collected for an operation, wherein the operation comprises an artificial intelligence or machine learning operation; receive, from the network entity, retroactive error information associated with the data that has been provided; and perform an action based on the retroactive error information associated with the data that has been provided received from the network entity.

[0007] According to an aspect, an apparatus comprises at least one processor; and at least one memory that stores instructions, which when executed by the at least one processor, cause the apparatus at least to collect data for an operation, wherein the operation comprises an artificial intelligence or machine learning operation; provide, to a user device, the data collected for the operation; determine retroactive error information associated with the data that has been provided; and send, to the user device, the retroactive error information associated with the data that has been provided. BRIEF DESCRIPTION OF DRAWINGS

[0008] The foregoing aspects and other features are explained in the following description, taken in connection with the accompanying drawings, wherein:

[0009] Figure 1 is a block diagram of one possible and non-limiting system in which example embodiments can be practiced.

[0010] Figure 2 is an illustration of an example embodiment (Embodiment A) of the schemes described herein.

[0011] Figure 3 is an illustration of an example embodiment (Embodiment B) of the schemes described herein.

[0012] Figure 4 is an example apparatus configured to implement examples described herein.

[0013] Figure 5 shows an example representation of an example of a non-volatile memory medium used to store instructions that implement examples described herein.

[0014] Figure 6 is an example method based on examples described herein.

[0015] Figure 7 is an example method based on examples described herein.

[0016] Figure 8 is an example method based on examples described herein.

[0017] Figure 9 is an example method based on examples described herein.

[0018] Figure 10 is an example method based on examples described herein. DETAILED DESCRIPTION

[0019] Turning to Figure 1 , the figure shows a block diagram of one possible and non-limiting example in which examples can be practiced. A user equipment (UE) 110, a radio access network (RAN) node 170, and network element(s) 190 are illustrated. In Figure 1 In examples, a user equipment (UE) 110 is in wireless communication with a wireless network 100. The UE is a wireless device that can access the wireless network 100. The UE 110 includes one or more processors 120, one or more memories 125, and one or more transceivers 130 interconnected through one or more buses 127. Each of the one or more transceivers 130 includes a receiver, Rx 132, and a transmitter, Tx 133. The one or more buses 127 can be address, data, or control buses, and can include any interconnection mechanism, such as a motherboard or a series of wires, fibers, or other optical communication equipment, etc., on an integrated circuit. The one or more transceivers 130 are connected to one or more antennas 128. The one or more memories 125 include computer program code 123. The UE 110 includes modules including one or both of portions 140-1 and / or 140-2, which can be implemented in a variety of ways. The modules can be implemented in hardware as modules 140-1, such as being implemented as part of the one or more processors 120. The modules 140-1 can also be implemented as integrated circuits or through other hardware such as programmable gate arrays. In another example, the modules 140 can be implemented as modules 140-2, which are implemented as computer program code 123 and executed by the one or more processors 120. For example, the one or more memories 125 and computer program code 123 can be configured to, with the one or more processors 120, cause the user equipment 110 to perform one or more operations as described herein. The UE 110 communicates with the RAN node 170 via a wireless link 111.

[0020] The RAN node 170 in this example is a base station that provides wireless devices, such as the UE 110, access to the wireless network 100. The RAN node 170 can be, for example, a base station for 5G (also referred to as New Radio (NR)). In 5G, the RAN node 170 can be an NG-RAN node, which is defined to be a gNB or ng-eNB. A gNB is a node that terminates the NR user plane and control plane protocol and connects to a 5GC (e.g., network element(s) 190) via an NG interface, such as connection 131. An ng-eNB is a node that terminates the E-UTRA user plane and control plane protocols and connects to a 5GC via an NG interface, such as connection 131. The NG-RAN node can include multiple gNBs, which can also include a central unit (CU) (gNB-CU) 196 and distributed unit(s) (DUs) (gNB-DUs) (of which a DU 195 is shown). Note that the DU 195 can include or be coupled to and control a radio unit (RU). The gNB-CU 196 is a logical node of a gNB or en-gNB that hosts the radio resource control (RRC), SDAP, and PDCP protocols of the gNB controlling the operation of one or more gNB-DUs or the RRC and PDCP protocols of an en-gNB. The gNB-CU 196 terminates the F1 interface with the gNB-DU 195. The F1 interface is illustrated as reference 198, but the reference 198 also illustrates links between remote elements of the RAN node 170 and centralized elements of the RAN node 170, such as between a gNB-CU and a gNB-DU. The gNB-DU 195 is a logical node that hosts the RLC, MAC, and PHY layers of a gNB or en-gNB and whose operation is controlled in part by the gNB-CU 196. One gNB-CU 196 supports one or more cells. One cell can be supported with one gNB-DU 195, or one cell can be supported / shared with multiple DUs under RAN sharing. The gNB-DU 195 terminates the F1 interface 198 with the gNB-CU 196. Note that the DU 195 is considered to include the transceiver 160, for example, as part of a RU, although some examples thereof can have the transceiver 160 as part of a separate RU, for example, under the control of and connected to the DU 195. The RAN node 170 can also be an eNB (evolved Node B) base station for LTE (Long Term Evolution), or any other suitable base station or node.

[0021] The RAN node 170 includes one or more processors 152, one or more memories 155, one or more network interfaces (N / W I / F) 161, and one or more transceivers 160 interconnected through one or more buses 157. Each of the one or more transceivers 160 includes a receiver, Rx, 162 and a transmitter, Tx, 163. The one or more transceivers 160 are connected to one or more antennas 158. The one or more memories 155 include computer program code 153. The CU 196 can include the processor(s) 152, the memory 155, and the network interface 161. Note that the DU 195 can also contain its own memory and processor(s) and / or other hardware, but these are not shown.

[0022] The RAN node 170 includes modules 150, including one or both of part 150-1 and / or part 150-2, which can be implemented in a number of ways. The modules 150 can be implemented in hardware as modules 150-1, for example, as part of the processor(s) 152. The modules 150-1 can also be implemented in integrated circuits or by other hardware such as programmable gate arrays. In another example, the modules 150 can be implemented as modules 150-2 which are implemented as computer program code 153 and are executed by the processor(s) 152. For example, the memory(ies) 155 and the computer program code 153 are configured to, working with the processor(s) 152, cause the RAN node 170 to perform one or more of the operations as described herein. Note that the functionality of the modules 150 can be distributed, such as between the DU 195 and the CU 196, or implemented only in the DU 195.

[0023] The one or more network interfaces 161 communicate through networks, such as via links 176 and 131. Two or more gNBs 170 can communicate using, for example, links 176. The links 176 can be wireline or wireless or both and can implement, for example, an Xn interface for 5G, an X2 interface for LTE, or other suitable interfaces for other standards.

[0024] The one or more buses 157 can be address, data, or control buses, and can include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment, wireless channels, and the like. For example, the one or more transceivers 160 can be implemented as a remote radio head (RRH) 195 for LTE or a distributed unit (DU) 195 for a gNB implementation for 5G, where other elements of the RAN node 170 can be physically located in a different location from the RRH / DU 195, and the one or more buses 157 can be implemented in part as, for example, fiber optic cables or other suitable network connections to connect the other elements of the RAN node 170 (e.g., central unit (CU), gNB-CU 196) to the RRH / DU 195. Reference 198 also indicates those suitable network link(s).

[0025] The RAN node / gNB can comprise one or more TRPs, and the methods described herein can apply to these TRPs. Figure 1 The RAN node 170 is shown to include TRPs 51 and TRP 52 in addition to the TRP represented by transceiver 160. Like transceiver 160, TRPs 51 and 52 can each include one transmitter and one receiver. The RAN node 170 can host or include Figure 1 Other TRPs not shown.

[0026] A forwarding node in NR is referred to as an integrated access and backhaul node. The mobile termination part of an IAB node facilitates the backhaul (parent link) connection. In other words, the mobile termination part comprises functionality hosting UE functionality. The distributed unit part of an IAB node facilitates the so-called access link (child link) connection (i.e., for an access link UE, and in case of multi-hop IAB, for other IAB nodes). In other words, the distributed unit part is responsible for certain base station functionality. The IAB scenario can follow a so-called split architecture, where a central unit hosts higher layer protocols towards UEs and terminates the control plane and user plane interface to the 5G core network.

[0027] Note that the description herein indicates that a“cell” performs a function, but it should be clear that a device forming the cell can perform the function. A cell constitutes a part of a base station. That is, each base station can correspond to multiple cells. For example, for a single carrier frequency and associated bandwidth, there can be three cells each covering one third of a 360 degree region so that the coverage area of a single base station covers an approximately elliptical or circular shape. Further, each cell can correspond to a single carrier, and a base station can use multiple carriers. Thus, if each carrier corresponds to three 120 degree cells and there are two carriers, then a base station has a total of 6 cells.

[0028] The wireless network 100 can include one or more network elements 190, which can include core network functionality, and provide connectivity to further networks (e.g., a telephone network and / or a data communications network (e.g., the Internet)) via one or more links 181. Such core network functionality for 5G can include Location Management Function(s) (LMF) and / or Access and Mobility Management Function(s) (AMF) and / or User Plane Function(s) (UPF) and / or Session Management Function(s) (SMF). Such core network functionality for LTE can include MME (Mobility Management Entity) / SGW (Serving Gateway) functionality. Such core network functionality can include SON (Self-Organizing / Optimizing Network) functionality. These are merely example functions that can be supported by the network element(s) 190, and note that both 5G and LTE functionality can be supported. The RAN nodes 170 are coupled via links 131 to the network elements 190. The links 131 can be implemented, for example, as an NG interface for 5G, or an SI interface for LTE, or other suitable interfaces for other standards. The network elements 190 include one or more processors 175, one or more memories 171, and one or more network interfaces (N / W I / F) 180 interconnected through one or more buses 185. The one or more memories 171 include computer program code 173. The computer program code 173 can include SON and / or MRO functionality 172.

[0029] The wireless network 100 can implement network virtualization, which is the process of combining hardware and software network resources and network functionality into a single, software- based, managed entity, or virtual network. Network virtualization involves platform virtualization, often combined with resource virtualization. Network virtualization is classified as either external, which combines many networks, or parts of networks, into a virtual unit, or internal, which provides network-like functionality to software containers on a single system. Note that the virtualized entities resulting from network virtualization are still implemented at some level using hardware such as the processors 152 or 175 and memories 155 and 171, and such virtualized entities also produce technical effects.

[0030] The computer-readable storage media 125, 155, and 171 can be of any type suitable to the local technical environment, and can be implemented using any suitable data storage technology, such as semiconductor based memory devices, flash memory, magnetic memory devices and systems, optical memory devices and systems, non- transitory memory, transitory memory, fixed memory, and removable memory. The computer-readable storage media 125, 155, and 171 can be means for storing information and / or functionality. The processors 120, 152, and 175 can be of any type suitable to the local technical environment, and can include one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), and processors based on multi-core processor architectures, as non-limiting examples. The processors 120, 152, and 175 can be means for performing functions, such as controlling UE 110, RAN node 170, network element(s) 190, and other functions as described herein.

[0031] In general, the various example embodiments of the user equipment 110 can include, but are not limited to, cellular telephones such as smart phones, tablets, personal digital assistants (PDAs) having wireless communication capabilities, portable computers having wireless communication capabilities, image capture devices such as digital cameras having wireless communication capabilities, gaming devices having wireless communication capabilities, music storage and playback appliances having wireless communication capabilities, Internet appliances permitting wireless Internet access and browsing, tablets with wireless communication capabilities, as well as portable units or terminals that incorporate combinations of such functions. The UE 110 can also be a vehicle (such as an automobile) or a UE installed in a vehicle, a UAV (such as a drone) or a UE installed in a UAV. The user equipment 110 can be an end device, such as a phone, a mobile device, a sensor device, etc., that is used by a user or that is not used by a user.

[0032] The UE 110, RAN node 170, and / or network element(s) 190 (and associated memory, computer program code, and modules) can be configured to implement (e.g., partially implement) the methods described herein. Thus, the computer program code 123, modules 140-1, 140-2, and other elements / features shown in the computer program code 123 of the UE 110 can implement user equipment-related aspects of the examples described herein. Similarly, the computer program code 153, modules 150-1, 150-2, and other elements / features shown in the computer program code 153 of the RAN node 170 can implement gNB / TRP-related aspects of the examples described herein. The computer program code 163, modules 160-1, 160-2, and other elements / features shown in the computer program code 163 of the network element(s) 190 can implement network element-related aspects of the examples described herein. Figure 1 The computer program code 123, modules 140-1, 140-2, and other elements / features shown in the computer program code 123 of the UE 110 can implement user equipment-related aspects of the examples described herein. Similarly, the computer program code 153, modules 150-1, 150-2, and other elements / features shown in the computer program code 153 of the RAN node 170 can implement gNB / TRP-related aspects of the examples described herein. The computer program code 163, modules 160-1, 160-2, and other elements / features shown in the computer program code 163 of the network element(s) 190 can implement network element-related aspects of the examples described herein. Figure 1 The computer program code 123, modules 140-1, 140-2, and other elements / features shown in the computer program code 123 of the UE 110 can implement user equipment-related aspects of the examples described herein. Similarly, the computer program code 153, modules 150-1, 150-2, and other elements / features shown in the computer program code 153 of the RAN node 170 can implement gNB / TRP-related aspects of the examples described herein. The computer program code 163, modules 160-1, 160-2, and other elements / features shown in the computer program code 163 of the network element(s) 190 can implement network element-related aspects of the examples described herein. Figure 1The computer program code 173 and other elements / features shown in the computer 100 can be configured to implement network element related aspects of the examples described herein.

[0033] With this background in mind, which is intended to be illustrative and not limiting, the example embodiments will now be described in more detail.

[0034] So far, the application of AI / ML in wireless communication has been limited to implementation based approaches, either at the network side or at the UE side. However, enhancing the air interface with features that enable support for improvements to AI / ML based algorithms can potentially lead to enhanced performance, e.g. improved throughput, robustness, accuracy or reliability, etc. (depending on the use case), and can also reduce complexity / overhead. To this end, 3GPP is currently actively pursuing AI / ML for the air interface.

[0035] Related use cases for AI / ML technology application to the NR air interface include CSI feedback enhancement use cases, beam management use cases, and positioning accuracy enhancement use cases. Such CSI feedback enhancement use cases include spatial-frequency domain CSI compression using a two-sided AI model, or time domain CSI prediction using a UE side model. Such beam management use cases include spatial domain downlink beam prediction for beam set A based on measurement results of beam set B, and time domain downlink beam prediction for beam set A based on historical measurement results of beam set B. Such positioning accuracy enhancement use cases include direct AI / ML positioning use cases and AI / ML assisted positioning use cases. Direct AI / ML positioning use cases include AI / ML model output of UE position, e.g. fingerprinting based on channel observations as input to the AI / ML model. AI / ML assisted positioning use cases include new measurements and / or enhanced AI / ML model output of existing measurements, e.g. LOS / NLOS identification, time and / or angle of measurement, likelihood of measurement.

[0036] An AI / ML general framework for single-sided AI / ML models can be used for the examples described herein. The functions described and used herein refer to AI / ML features / feature groups (FGs) enabled by configuration(s), where the configuration(s) are supported based on conditions indicated by UE capability.

[0037] AI / ML data collection is the process of collecting data by network nodes, management entities, or UEs for the purpose of AI / ML model training, data analytics, and inference. Data collection is a function that provides input data for model training, management, and inference functions. Training data is the data required as input for AI / ML model training functions. Monitoring data is the data required as input for AI / ML model or AI / ML function management, and inference data is the data required as input for AI / ML inference functions.

[0038] For AI / ML based positioning data collection, information with potential impact can include (1-9):

[0039] 1) Ground truth labels, including reports from label data generating entities.

[0040] 2) Measurements (corresponding to model inputs), including reports from measurement data generating entities.

[0041] 3) Quality indicators, for and / or associated with ground truth labels and / or measurements, or reports from label and / or measurement data generating entities, and / or as requests from different entities (e.g., data collection, etc.).

[0042] 4) RS configuration(s), at least for deriving measurements, or as requests from data generating entities (UEs / PRUs / TRPs) to LMF, and / or as LMF assistance signaling to UEs / PRUs / TRPs (no enhancements on top of existing RS configuration(s) or no new RS configurations for positioning measurements).

[0043] 5) Timestamps, at least for and / or associated with collected data, when measurements and ground truth labels are generated by different entities, timestamps for measurements and ground truth labels are separate, or reports from data generating entities are together with collected data, and / or as LMF assistance signaling (no enhancements on top of existing positioning measurement reports or no new timestamp reports for positioning measurements, and whether and how the above information can be applied to different aspects of AI / ML LCM (e.g., training, updating, monitoring, etc.) can be different).

[0044] Transmitting data from the entity generating the data to different entities is not precluded from the perspective of RANI. If any specification impact is identified, the impact can be different between different positioning use cases. The necessity of other information for data collection (e.g., scenario identifier, LOS / NLOS status, timing error, etc.) can be specific to a particular case.

[0045] 6) Details of the request / reporting of labels and / or other training data, and enabling passing collected labels and / or other training data to the training entity when the training entity is not the same entity that acquires the labels and / or other training data.

[0046] 7) Assistance signaling indicating reference signal(s) configuration to derive labels and / or other training data.

[0047] 8) Request / reporting of training data: ground truth labels; measurements corresponding to model inputs; associated information of ground truth labels and / or measurements corresponding to model inputs.

[0048] 9) Assistance signaling and procedures to facilitate generation of training data: reference signal(s) (e.g., PRS / SRS) configuration and configuration identifier; assistance information, e.g., between LMF and UE / PRU, for label computation / generation, and label validity / quality conditions, etc. (Whether such assistance signaling and procedures can be applied to other aspects of AI / ML LCM can be specific to concrete cases).

[0049] There can be different entities to generate training data, and different types of training data, where applicable. The following two cases can be considered, where applicable: the case where the training entity is the same as the entity that generates the training data, and the case where the training entity is different from the entity that generates the training data.

[0050] For positioning use cases, selection, activation, switching, and fallback of models or functions can be initiated by the UE, gNB, or LMF, for which different cases applicable to UE-side models and network-side models can be distinguished.

[0051] For data collection, model transfer / delivery, and function-to-entity mapping analysis, various scenarios arise when the data generation entity and termination entity are different. For example, for model training, for UE-side models, training data can be generated by the UE, while the termination point for the training data can include the UE or a UE-side OTT server. OAM or core network can be used for UE-side model training. LMF can be used for UE-side model training.

[0052] In one case, for training data generation for AI / ML based positioning, measurements and their related data (e.g., timestamps) are generated by PRUs and / or non-PRU UEs. In another case, for training data generation for AI / ML based positioning, channel measurements and their related data (e.g., timestamps) are generated by PRUs and / or non-PRU UEs.

[0053] For training data collection for AI / ML based positioning, the collected data samples can include the following components: Part A, including channel measurements, quality indicators for channel measurements, timestamps for channel measurements; Part B, including ground truth labels (or approximation thereof), quality indicators for labels, and timestamps for labels. If a training data sample contains both Part A and Part B, it can be assumed that Part A and Part B in one training data sample are for the same UE (PRU or non-PRU UE) and for the same location associated with Part B.

[0054] For training data generation for AI / ML based positioning, labels and their related data (e.g., timestamps) can be generated by: PRU, non-PRU UE with estimated location, or LMF.

[0055] In an example, for UE-side model(s) developed (e.g., trained, updated) at UE side, for data collection, the NW signals (multiple) data collection related configurations and its / their associated ID (associated ID for each sub-use case related to NW-side additional conditions) through signaling, the (multiple) UE collects data corresponding to the (multiple) associated ID, and the AI / ML model is developed (e.g., trained, updated) at UE side based on the collected data corresponding to the (multiple) associated ID.

[0056] The gNB-centric and OAM-centric approaches for NW-side data collection can be considered, which are related to beam management use cases. The data collection procedure from UE to NW (e.g., gNB, LMF, or OAM) can be considered for NW-side model LCM, including training, inference, and management. RRC configuration can be used to configure radio measurements and their related reporting to enable data collection for NW-side training.

[0057] Accordingly, various data collection aspects for AI / ML can be implemented in a wireless communication network, such as a data collection procedure from UE to NW (e.g., gNB, LMF, or OAM) for NW-side model LCM (including training, inference, management). Examples described herein relate to retroactive error reporting for AI / ML data collection.

[0058] Consider a scenario where a network configures a UE for data collection for AI / ML operation (e.g., training an AI / ML model). In this case, the UE that collects the data generates a dataset that consists of, for example, measurements and / or estimations performed at the UE and any associated ground truth labels. For example, for a positioning use case (e.g., fingerprint-based direct UE positioning), a UE with a known location (using 3GPP-based or non-3GPP-based positioning methods) can measure channel impulse responses (CIRs) to create a dataset of CIRs versus associated locations (e.g., estimated locations using some positioning technique). Subsequently, the UE reports the collected data to the network for AI / ML operation, e.g., for AI / ML model training. Note that the AI / ML operation can be performed at the network or at another UE. When performed at other UEs, the network can provide the collected data to the other UE.

[0059] However, any errors that can exist in the collected data can adversely affect the AI / ML operation. For example, in a fingerprint-based direct positioning use case, when the labels (e.g., locations) associated with the input data (e.g., CIRs) are inaccurate, the performance of an AI / ML model trained using such labeled input data can be affected. Therefore, it is necessary to ensure the quality of the data (e.g., accurate labeling and / or discarding erroneous data sample(s)) before the dataset is used for the intended AI / ML operation to ensure high performance of the operation. Moreover, even when a dataset with erroneous data samples has already been used, awareness of the errors can still allow, for example, retraining or updating the model to avoid poor model performance.

[0060] At the UE, the presence of errors in the collected data can not be known when reporting the data. Therefore, the UE cannot filter or correct the erroneous data samples before reporting the collected data. However, there is currently no mechanism for the UE to indicate any errors related to the dataset that it has reported to the network.

[0061] Examples described herein relate to data collection for AI / ML. Specifically, examples described herein involve a UE retroactively indicating to the network erroneous information for data that has already been reported. This retroactive error information is sent when / if the UE identifies an error in data that has already been reported. The retroactive error identification at the UE can be triggered by the network or can be performed autonomously by the UE. The retroactive error information can include one or more of: a unique identification of the (potentially) erroneous data sample, or a unique identification of a group of data samples that includes the (potentially) erroneous data sample, a (potentially) erroneous component(s) of the data sample(s), a correction of the error component(s) of the data sample(s), and / or a level of error in the error component(s) of the data sample(s).

[0062] The network receives from the UE retroactive error information for data that has been collected from the UE. If the collected data has not yet been used for AI / ML operations (e.g. model training), the network can correct the data samples based on the received retroactive error information for the collected data (e.g. by applying corrections to labels and / or discarding error data sample(s)). If a data set with error data samples has already been used, e.g. to train a model, the network can utilize the error report to perform, e.g. a re-training or model update of the model to avoid poor model performance if the identified error is deemed to impact model performance.

[0063] At the time of reporting the data, the presence of errors in the collected data can not be known at the UE. Therefore, the UE cannot filter or correct error data samples before reporting the collected data. However, at a later time, the UE can realize that there is an error in one or more collected data samples in the set of data that has already been reported. Some example scenarios (A-D) are listed next in which the UE can realize the error in the data samples late after the data has already been reported.

[0064] A) When a UE sensor is used for data generation, e.g. for a positioning use case, the training data label UE position is determined using a non-3GPP based solution, after the sensor is recalibrated at the UE, the UE can determine that the label (position) in the training data collected for a certain time period can be erroneous. Further, based on the recalibration, an error correction for the label can be determined at the UE.

[0065] B) Similarly, for GNSS or TBS positioning, the UE can obtain the necessary correction data (e.g. GNSS differential corrections as part of the assistance data in LPP) long after the positioning estimate is performed using these positioning technologies.

[0066] C) The UE can realize that the reference time it used to timestamp its measurements was / is wrong, e.g., after selecting a different synchronization source. The UE can also realize the amount of time offset that caused the error and determine a value to compensate for the error in the timestamps it previously provided in the dataset.

[0067] D) The UE can realize that the algorithm and / or AI / ML model it used to perform the measurements and / or estimates (e.g., using AI / ML inference) contains, e.g., systematic errors, flaws, etc. This realization can occur, e.g., after the UE obtains feedback from the network on its estimates (e.g., based on AI / ML performance monitoring) and after internally debugging / inspecting its algorithm. Note that this realization can occur at the application layer or at any external server (e.g., a server of the vendor the UE belongs to).

[0068] By indicating the retroactive error information for the reported data to the network, the UE can create at the network the realization of the error in the data (from the UE) that has been collected. This allows the network to, e.g., correct the data samples (e.g., by applying a correction to the labels) and / or discard the error data sample(s) before using the dataset for the intended AI / ML operation to ensure high performance of that operation. Furthermore, if the dataset with the error data sample has already been used, the network can perform, e.g., a retraining or a model update of the model to avoid poor model performance if the identified error is deemed to impact the model performance.

[0069] Figure 2 Embodiments of the schemes described herein are illustrated showing a signaling exchange between a UE-A 110-A and a network (e.g., a RAN node 170 or one or more network elements 190). The steps are shown as follows:

[0070] Step 0: The NW determines the configuration needed for data collection and provides it to the UE, such as the configuration of the reference signals (e.g., DL PRS) that need to be transmitted, and the configuration of the measurements and reporting (e.g., the positioning-related RSRP measurements to be performed by the UE, their reporting frequency, etc.). The NW also transmits any reference signals needed for data collection, e.g., DL PRS, via the TRPs.

[0071] Step 1: The UE collects data according to the network’s configuration.

[0072] Step 2: The UE reports the collected data after completing the data collection, e.g., according to the reporting configuration it received from the NW. The UE adds a unique identifier for each data sample in the dataset. This unique identifier allows the UE and the network to pinpoint the data sample for which the UE has provided the retroactive error information (see steps 6 and 7).

[0073] Step 3: The NW performs AI / ML operations with the reported data set, e.g. (re)training or monitoring of AI / ML models on the LMF side or gNB side, or storing the data set for later final use in AI / ML operations. The network can also forward the data to other UEs or network entities for AI / ML operations. Step 4 (or Step 5) can happen immediately or later after Step 3. In other words, Step 3 is not necessarily followed immediately by Step 4 or Step 5.

[0074] Step 4 (optional): The network triggers the UE to perform a retrospective data sample check for the data that has been reported. This step is only performed by the network when the network suspects that the collected data samples have errors or are corrupted.

[0075] In one example, the network determines the need based on monitoring the performance of a pre-trained AI / ML model using the provided data set. For example, by taking a previously trained high-performance AI / ML model, the measurements in the data set can be taken as input for inference, and then the inference output is compared with the labels in the data set to verify whether the labels are close to the expected ground truth.

[0076] In another example, the network can determine the need for error checking based on comparing the data set with other similar data sets, e.g. collected by other UEs or network entities, and involving the same area, having close time stamps, etc.

[0077] In another example, the network can determine the need for retrospective error checking by observing the consistency between the collected data and previously collected data from the same device and identifying potential outliers. For example, the network can evaluate each provided data set (e.g. a set of, e.g. 20K bytes, of collected data) on its probability of being an outlier data set. An outlier data set can be, for example, a data set that includes at least one value that is far beyond the values provided by the device so far; or a data set that does not conform to the trend of the last n provided data sets (n being an integer).

[0078] The trigger message can include a unique identification of the data sample that the network believes to be potentially erroneous, or a unique identification of a group of data samples that includes the potentially erroneous data sample, or a component part of the potentially erroneous data sample(s) that the network believes to be potentially erroneous, where the data sample(s) can consist of measurements and / or ground truth labels (or approximations thereof) and associated quality indicators and associated time stamps.

[0079] Step 5: The UE performs a retrospective data sample error check for the data that has been reported in Step 2.

[0080] The check can be triggered by the network (step 4) or autonomously at the UE. The autonomous triggering at the UE can be based on e.g. re-calibration of sensors / RF components at the UE, change of data collection method (e.g. change of positioning technique at the UE), etc. by which the UE becomes aware of errors in previously reported data. See the scenarios described earlier for more examples. The UE can become aware of the error through a number of ways described earlier.

[0081] Step 6: The UE reports the retroactive error information for the reported data.

[0082] The retroactive error information for the reported data can comprise one or more of (A-D):

[0083] A) A unique identification of the (potentially) erroneous data sample, or a unique identification of a group of data samples comprising the (potentially) erroneous data sample. A timestamp of the collected and / or reported data can be used as an identifier. Each data sample in a data set can e.g. have a unique record ID. Each sample can also be identified by the ID(s) of the protocol session used for data collection, e.g. a session ID, transaction ID, message ID, etc. in LPP or RRC.

[0084] B) The (potentially) erroneous component(s) of the data sample(s).

[0085] C) A correction of the erroneous component(s) of the data sample(s).

[0086] D) A level of the error in the erroneous component(s) of the data sample(s).

[0087] Step 7: The network utilizes the retroactive error information indicated by the UE for the collected data from the UE. If the collected data has not been used for AI / ML operations (e.g. model training), or if the data set is stored in a repository for future use, the network can correct the data samples (e.g. by applying a correction to the labels) and / or discard the erroneous data sample(s) based on the received retroactive error information for the collected data. If the data set with the erroneous data samples has already been used for e.g. model training, the network can perform e.g. re-training or model update of the model to avoid poor model performance if the identified error is deemed to affect the model performance.

[0088] In additional embodiments, the UE collecting the data is trained to identify potential errors with respect to the collected data, with assistance from the network; or equivalently, the UE is trained to assess the probability that the collected data contains an error.

[0089] In one variant of this embodiment, the UE is provided with conditions or criteria to define abnormal data. The provision of this condition or criteria is achieved by an additional IE (compared to Figure 2 / the main embodiment) at step 4. Examples of such conditions (criteria) are (1a - 1b):

[0090] 1a. Correlation with synthetic data. This involves generating synthetic data from data that has been collected at the UE, and an acceptable range of the data that is actually collected. For example, the UE is configured to generate synthetic data for a collection time instant x based on data collected at time instants x-n;... x-1; x+1;... x+n. The UE is also configured with a range, and if the data that is actually collected at time instant x is outside this range, then potentially abnormal data is declared. In this case, the UE can indicate to the network retrospectively that the erroneous data that was collected / reported for past time instant x.

[0091] 1b. In one variant of this approach, the UE employs an active approach to replace or complement the passive approach described above. In the active approach, the UE uses previous data to create synthetic data, which is then compared with the currently collected data to determine if there is a potential error. However, the drawback of this approach is that it requires a relatively high computational capability at the UE, and hence is only suitable for limited types of UEs.

[0092] Depending on the use case, the network can be a core network entity (e.g. LMF for positioning), or a RAN node, e.g. gNB (e.g. for use cases of beam management).

[0093] In another embodiment, Figure 2 The roles of the UE and the network as shown can be reversed. That is, the network can collect the AI / ML data set and provide it to the UE, as Figure 3 shown. For convenience (to maintain consistency between Figure 2 and Figure 3 ), the data provided by the network to the UE is referred to as a data report.

[0094] Figure 3 Signaling exchange between UE-B 110-B and the network (e.g. RAN node 170 or one or more network elements 190) is shown.

[0095] In another embodiment, the signaling and the behavior of the network and the UE as Figure 3 shown can be an extension of Figure 2 where the collected data from one UE (UE-A) is provided by the network to another UE (UE-B), as Figure 3 shown. In this case, the trigger received by the network from UE-B at step 4 of Figure 3 may in turn cause the network to triggerFigure 2 UE-A performs Figure 2 the retrospective data sample error check in step 4. Further, step 5 (as shown in step 5 of Figure 2 ).

[0096] Advantages and technical effects of the examples described herein include that the solution enables the network to be aware of errors in the data that has been collected from the UE. This allows the network to correct the data samples (e.g. by applying corrections to the labels) and / or discard the error data sample(s) to ensure high performance of the intended AI / ML operation, e.g. before using the data set for the intended AI / ML operation. Furthermore, if a data set with error data samples has already been used, the network can perform e.g. re-training of the model or model update to avoid poor model performance if the identified errors are considered to affect the model performance.

[0097] The examples described herein are thus related to AI / ML data collection aspects and to AI / ML air interface use cases (CSI, beam management, positioning) e.g. in Rel-19. Furthermore, the AI / ML data collection enhancements described herein can be part of 6G and beyond, including aspects related to signaling.

[0098] Figure 4 is an example apparatus 400 that can be implemented in hardware configured to implement the examples described herein. The apparatus 400 comprises at least one processor 402 (e.g. FPGA and / or CPU), one or more memories 404 comprising computer program code 405 having instructions for performing the methods described herein, wherein the at least one memory 404 and the computer program code 405 are configured to, with the at least one processor 402, cause the apparatus 400 to implement circuitry, processes, components, modules or functions (with control module 406) to implement the examples described herein. The one or more memories 404 can comprise non-transitory memory, transitory memory, volatile memory (e.g. RAM) or non-volatile memory (e.g. ROM).

[0099] The retrospective error indication or determination 430 implements the examples described herein related to retrospective error indication after AI / ML data collection and reporting.

[0100] The apparatus 400 includes a display and / or I / O interface 408, which includes user interface (UI) circuitry and elements, which can be used to display aspects or status of the methods described herein (e.g., when one of the methods is being performed or at a later time), or to receive input from a user, such as through the use of a keyboard, camera, touchscreen, touchpad, microphone, biometric, one or more sensors, etc. The apparatus 400 includes one or more communication interfaces (e.g., network (N / W) interface (I / F)) 410. The communication interface(s) 410 can be wired and / or wireless, and transmit over one or more links 424 on the Internet / (other) network(s) via any communication technology. The link(s) 424 can be Figure 1 the link(s) 131 and / or 176 in FIG. 1. Figure 1 The link(s) 131 and / or 176 in FIG. 1 can also be implemented using the transceiver(s) 416 and corresponding wireless link(s) 426. The communication interface(s) 410 can include one or more transmitters or one or more receivers.

[0101] The transceiver 416 includes one or more transmitters 418 and one or more receivers 420. The transceiver 416 and / or the communication interface(s) 410 can include standard, well-known components such as amplifiers, filters, frequency converters, (de)modulators, encoding / decoding circuitry, and one or more antennas, such as the antenna 414 for communicating over the wireless link 426.

[0102] The control module 406 of the apparatus 400 includes one or both of portions 406-1 and / or 406-2, and can be implemented in a variety of ways. The control module 406 can be implemented in hardware as control module 406-1, such as part of the processor(s) 402. The control module 406-1 can also be implemented as an integrated circuit, or through other hardware such as a programmable gate array. In another example, the control module 406 can be implemented as control module 406-2, which is implemented as computer program code (with corresponding instructions) 405, and executed by the processor(s) 402. For example, the memory(ies) 404 store instructions that, when executed by the processor(s) 402, cause the apparatus 400 to perform one or more of the operations described herein. Moreover, the processor(s) 402, the memory(ies) 404, and the example algorithms (e.g., flowcharts and / or signaling diagrams) encoded as instructions, programs, or code are means for causing performance of the operations described herein.

[0103] The apparatus 400 for implementing the control 406 functionality can be the UE 110, the RAN node 170 (e.g., gNB), or the network element(s) 190 (e.g., LMF 190). Thus, the processor 402 can correspond to the processor(s) 120, the processor(s) 152, and / or the processor(s) 175; the memory 404 can correspond to the memory or memories 125, the memory or memories 155, and / or the memory or memories 171; the computer program code 405 can correspond to the computer program code 123, the computer program code 153, and / or the computer program code 173; the control module 406 can correspond to the module 140-1, the module 140-2, the module 150-1, and / or the module 150-2; the communication interface(s) 410 and / or the transceiver 416 can correspond to the transceiver 130, the antenna(s) 128, the transceiver 160, the antenna(s) 158, the network interface(s) 161, and / or the network interface(s) 180. Alternatively, the apparatus 400 and its elements can not correspond to the UE 110, the RAN node 170, or the network element(s) 190 and their respective elements, as the apparatus 400 can be part of a self-organizing / optimizing network (SON) node or other node (e.g., in the cloud).

[0104] The apparatus 400 can also be distributed throughout the network (e.g., 100) including within the apparatus 400 and between the apparatus 400 and any of the network elements (e.g., network control element (NCE) 190 and / or RAN node 170 and / or UE 110).

[0105] The interface 412 enables data communication and signaling between the various items of the apparatus 400, as shown by the arrows. Figure 4 The interface 412 can be one or more buses, such as an address bus, a data bus, or a control bus, and can include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics, or other interconnection mechanisms. The computer program code (e.g., instructions) 405 (including the control 406) can include object-oriented software configured to pass data or messages between objects within the computer program code 405, or the computer program code (e.g., instructions) 405 (including the control 406) can include functional, scripted, or procedural code. The apparatus 400 need not include every feature mentioned, and can include other features. The various components of the apparatus 400 can be located within a common housing 428, or a subset of the various components of the apparatus 400 can be located within a different housing, which can include the housing 428.

[0106] Figure 5Schematic representations of non-volatile storage media 500a (e.g., computer / optical disc (CD) or digital versatile optical disc (DVD)), 500b (e.g., a Universal Serial Bus (USB) memory stick), and 500c (e.g., cloud storage for downloading instructions and / or parameters 502 or receiving email transmission instructions and / or parameters 502) are shown, on which instructions and / or parameters 502 are stored, which, when executed by a processor, cause the processor to perform one or more steps of the methods described herein. Instructions and / or parameters 502 may represent computer-readable media.

[0107] Figure 6 This is an example method 600 based on the examples described herein. At 610, the method includes collecting data for operation, wherein the operation includes artificial intelligence or machine learning operation. At 620, the method includes reporting the data collected for operation to a network entity. At 630, the method includes determining traceability error information associated with the reported data. At 640, the method includes sending the traceability error information associated with the reported data to a network entity. Method 600 can be performed using UE 110, UE-A 110-A, UE-B 110-B, or device 400.

[0108] Figure 7 This is an example method 700 based on the examples described herein. At 710, the method includes configuring a user equipment to collect data for operation, wherein the operation includes artificial intelligence or machine learning operations. At 720, the method includes receiving a report from the user equipment, the report including the data collected for operation. At 730, the method includes receiving traceability error information associated with reported data from the user equipment. At 740, the method includes performing an action based on the traceability error information associated with the reported data received from the user equipment. Method 700 can be performed using RAN node 170, one or more network elements 190, or device 400.

[0109] Figure 8This is an example method 800 based on the examples described herein. At 810, the method includes receiving a report from a first user equipment, the report including data collected for operation, wherein the operation includes artificial intelligence or machine learning operations. At 820, the method includes sending the reported, collected data for operation to a second user equipment. At 830, the method includes receiving a trigger message from the second user equipment for performing a traceability data error check on the reported data to determine traceability error information. At 840, the method includes sending a trigger message to the first user equipment for performing a traceability data error check on the reported data to determine traceability error information. Method 800 can be performed using RAN node 170, one or more network elements 190, or apparatus 400.

[0110] Figure 9 This is an example method 900 based on the examples described herein. At 910, the method includes receiving data provided by a network entity, including data collected for operation, wherein the operation includes artificial intelligence or machine learning operations. At 920, the method includes receiving traceability error information associated with the provided data from the network entity. At 930, the method includes performing an action based on the traceability error information associated with the provided data received from the network entity. Method 900 can be performed using UE 110, UE-A 110-A, UE-B 110-B, or device 400.

[0111] Figure 10 This is an example method 1000 based on the examples described herein. At 1010, the method includes collecting data for operation, wherein the operation includes artificial intelligence or machine learning operations. At 1020, the method includes providing the data collected for operation to a user equipment. At 1030, the method includes determining traceability error information associated with the provided data. At 1040, the method includes sending the traceability error information associated with the provided data to the user equipment. Method 1000 can be performed using RAN node 170, one or more network elements 190, or device 400.

[0112] The following embodiments are provided and described herein.

[0113] Example 1. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus to at least: collect data for operation, wherein the operation includes artificial intelligence or machine learning operation; report the data collected for operation to a network entity; determine traceability error information associated with the reported data; and send the traceability error information associated with the reported data to the network entity.

[0114] Example 2. According to the apparatus of Example 1, the data sample of the reported data includes at least one of the following: measurement, truth label, quality indicator, timestamp.

[0115] Example 3. An apparatus according to any one of Examples 1 to 2, wherein the traceability error information includes at least one or more of the following: a unique identifier of an erroneous data sample of the reported data, a unique identifier of a set of data samples including the erroneous data sample of the reported data, an erroneous component of a data sample of the reported data, a correction of the erroneous component of a data sample of the reported data, the level of error in the erroneous component of the data sample of the reported data, the actual error in the erroneous component of the data sample of the reported data, an indication that at least one data sample or other aspect of the reported data is potentially erroneous, and a unique identifier of at least one data sample or other aspect of the reported data that is potentially erroneous.

[0116] Example 4. According to the apparatus of Example 3, wherein an erroneous data sample of the reported data, or a set of data samples including an erroneous data sample of the reported data, or an erroneous component of a data sample of the reported data, or at least one potentially erroneous data sample of the reported data, or other aspects thereof, are identified by at least one or more of the following: a timestamp, a unique record identifier, an identifier of a protocol session for the collection of the reported data, and an identifier of a protocol message for the collection of the reported data.

[0117] Example 5. An apparatus according to any one of Examples 1 to 4, wherein the apparatus is further configured to: send a unique data identifier to a network entity for each data sample of the reported data collected for operation.

[0118] Example 6. An apparatus according to any one of Examples 1 to 5, wherein the apparatus is further configured to: receive from a network entity a trigger message for performing a traceability data error check on the reported data to determine traceability error information; wherein the traceability data error check for determining traceability error information is performed in response to receiving the trigger message from the network entity.

[0119] Example 7. According to the apparatus of Example 6, the trigger message received from the network entity for performing a retrospective data error check on the reported data includes one of the following: a unique identifier of a potentially erroneous data sample of the reported data, or a unique identifier of a set of data samples including a potentially erroneous data sample of the reported data, or a component of a potentially erroneous data sample of the reported data.

[0120] Example 8. An apparatus according to any one of Examples 1 to 7, wherein the apparatus is further configured to determine traceability error information based on one of the following: recalibration of at least one sensor of the apparatus, recalibration of at least one radio frequency component of the apparatus, change of method for collecting data for operation, change of positioning technology of the apparatus, receiving correction data after collecting the reported data, determination of systematic errors in collecting the reported data.

[0121] Example 9. An apparatus according to any one of Examples 1 to 8, wherein the traceability error information is based on an error in the reported data, and wherein the apparatus becomes aware of the error in the reported data based on: recalibration of at least one sensor of the apparatus, or receiving correction data after the reported data has been collected, or determining that the reference time used to timestamp the measurement is incorrect, wherein the collected data includes the measurement, or determining a systematic error used to collect the reported data.

[0122] Example 10. An apparatus according to any of Examples 1 to 9, wherein the apparatus is trained to identify errors in the collected data.

[0123] Example 11. An apparatus according to any one of Examples 1 to 10, wherein the apparatus is trained to determine the probability that the collected data has errors.

[0124] Example 12. An apparatus according to any one of Examples 1 to 11, wherein the apparatus is further configured to: receive from a network at least one criterion for determining anomalous data; wherein traceability error information associated with reported data is determined based on at least one criterion received from the network for determining anomalous data.

[0125] Example 13. The apparatus according to Example 12, wherein the apparatus is further configured to: perform a traceability data error check to determine traceability error information in response to receiving a trigger message from a network entity; wherein at least one criterion for determining anomalous data is received from the network together with the trigger message to perform a traceability data error check on the reported data.

[0126] Example 14. An apparatus according to any one of Examples 12 to 13, wherein the apparatus is further configured to: determine that an anomalous data sample of the reported data is outside the range of values, wherein at least one criterion for determining the anomalous data includes a range of values; wherein the traceability error information associated with the reported data is based on determining that an anomalous data sample of the reported data is outside the range of values.

[0127] Example 15. An apparatus according to any one of Examples 1 to 14, wherein the apparatus is further configured to: receive a first threshold from a network entity, the first threshold being used to determine that a data sample of the reported data is an erroneous data sample.

[0128] Example 16. The apparatus according to Example 15, wherein the apparatus is further configured to: generate a synthetic data sample at the moment when a data sample of the reported data is collected; determine the difference between the synthetic data sample and the data sample of the reported data; and determine that the data sample of the reported data is an erroneous data sample in response to the difference between the synthetic data sample and the data sample of the reported data being greater than a first threshold; wherein the traceability error information includes information relating to the data sample that is an erroneous data sample.

[0129] Example 17. An apparatus according to any one of Examples 1 to 16, wherein the apparatus is further configured to: generate a synthetic data sample at the moment the data sample of the reported data is collected; determine the difference between the synthetic data sample and the data sample of the reported data; and determine that the data sample of the reported data is an erroneous data sample in response to the difference between the synthetic data sample and the data sample of the reported data being greater than a second threshold; wherein the traceability error information includes information relating to the data sample that is an erroneous data sample. The first threshold and the second threshold may be the same or different.

[0130] Example 18. An apparatus according to any one of Examples 1 to 17, wherein: the apparatus is a user equipment, or the apparatus includes a user equipment, or the user equipment includes an apparatus.

[0131] Example 19. An apparatus according to any one of Examples 1 to 18, wherein the network entity is or includes: a core network entity, or a radio access network node, or a user equipment.

[0132] Example 20. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus to at least: configure a user equipment to collect data for operation, wherein the operation includes artificial intelligence or machine learning operation; receive a report from the user equipment, the report including data collected for operation; receive traceability error information associated with reported data from the user equipment; and perform an action based on the traceability error information associated with the reported data received from the user equipment.

[0133] Example 21. According to the apparatus of Example 20, the data sample of the reported data includes at least one of the following: measurement, truth label, quality indicator, timestamp.

[0134] Example 22. An apparatus according to any one of Examples 20 to 21, wherein the traceability error information includes at least one or more of the following: a unique identifier of an erroneous data sample of the reported data, a unique identifier of a set of data samples including the erroneous data sample of the reported data, an erroneous component of a data sample of the reported data, a correction of the erroneous component of a data sample of the reported data, the level of error in the erroneous component of the data sample of the reported data, the actual error in the erroneous component of the data sample of the reported data, an indication that at least one data sample or other aspect of the reported data is potentially erroneous, and a unique identifier of at least one data sample or other aspect of the reported data that is potentially erroneous.

[0135] Example 23. The apparatus according to Example 22, wherein an erroneous data sample of the reported data, or a set of data samples including an erroneous data sample of the reported data, or an erroneous component of a data sample of the reported data, or at least one potentially erroneous data sample of the reported data, or other aspects thereof, is identified by at least one or more of the following: a timestamp, a unique record identifier, an identifier of the protocol session for the collection of the reported data, and an identifier of the protocol message for the collection of the reported data.

[0136] Example 24. An apparatus according to any one of Examples 20 to 23, wherein the apparatus is further configured to: receive from a user equipment a unique data identifier for each data sample of the reported, collected data for operation; wherein an action performed based on traceability error information for the reported data is performed based on at least one unique data identifier among the unique data identifiers of the data samples of the collected data that have been reported for operation.

[0137] Example 25. An apparatus according to any one of Examples 20 to 24, wherein the apparatus is further configured to: send a trigger message to a user equipment for performing a traceability data error check on the reported data; wherein traceability error information associated with the reported data is received from the user equipment based on the sending of the trigger message.

[0138] Example 26. The apparatus according to Example 25, wherein the apparatus is further configured to: determine that there is a potential error in the reported data used for operation, or that the reported data used for operation is potentially corrupted; wherein a trigger message for performing a retrospective data error check on the reported data is sent to the user equipment in response to determining that there is a potential error in the reported data used for operation or that the reported data used for operation is potentially corrupted.

[0139] Example 27. The apparatus of Example 26, wherein determining that there is a potential error in the reported data used for operation or that the reported data used for operation is potentially corrupted is based on one of the following: using the reported data to monitor the performance of the operation, wherein the operation includes executing a pre-trained artificial intelligence or machine learning model, comparing the reported data with a dataset, and observing the consistency between the reported data and data previously provided by a user device.

[0140] Example 28. An apparatus according to any one of Examples 25 to 27, wherein a trigger message sent to a user equipment for performing a retrospective data error check on reported data includes one of the following: a unique identifier of a data sample of reported data identified as potentially erroneous, a unique identifier of a set of data samples, a set of data samples including the data sample of reported data identified as potentially erroneous, and a component of the data sample of reported data identified as potentially erroneous.

[0141] Example 29. An apparatus according to any one of Examples 20 to 28, wherein the action performed based on traceability error information associated with reported data includes one of the following: updating the reported data and performing an operation using the updated reported data; updating an error data sample having an identifier corresponding to an identifier of an error data sample received together with the traceability error information; updating a trained model and performing an operation using the updated trained model, the trained model being trained using the reported data; discarding the error data of the reported data and performing an operation using the reported data without the discarded error data; and sending the traceability error information associated with the reported data to another user equipment.

[0142] Example 30. An apparatus according to any one of Examples 20 to 29, wherein the apparatus is further configured to: send to a user equipment at least one criterion for determining anomalous data; wherein the traceability error information received from the user equipment associated with the reported data is based on at least one criterion sent to the user equipment for determining anomalous data.

[0143] Example 31. The apparatus according to Example 30, wherein at least one criterion for determining anomalous data is sent to the user equipment along with a trigger message, the trigger message being used to perform a traceability data error check on the reported data to determine traceability error information.

[0144] Example 32. An apparatus according to any one of Examples 30 to 31, wherein: at least one criterion for determining anomalous data includes a range of values; and traceability error information associated with the reported data is based on a sample of anomalous data from the reported data that is outside the range of values.

[0145] Example 33. An apparatus according to any one of Examples 20 to 32, wherein the apparatus is further configured to: send a first threshold to a user equipment, the first threshold being used to determine that a data sample of the reported data is an erroneous data sample.

[0146] Example 34. The apparatus according to Example 33, wherein: the difference between the synthetic data sample generated at the moment when the data sample of the reported data is collected and the data sample of the reported data is greater than a first threshold, the data sample of the reported data is an erroneous data sample, and the traceability error information includes information related to the data sample being an erroneous data sample.

[0147] Example 35. An apparatus according to any one of Examples 20 to 34, wherein: the difference between a synthetic data sample generated at the moment when the data sample of the reported data is collected and the data sample of the reported data is greater than a second threshold, the data sample of the reported data is an erroneous data sample, and the traceability error information includes information related to the data sample being an erroneous data sample.

[0148] Example 36. An apparatus according to any one of Examples 20 to 35, wherein: the apparatus is a core network entity, or the apparatus includes a core network entity, or the core network entity includes the apparatus, or the apparatus is a radio access network node, or the apparatus includes a radio access network node, or the radio access network node includes the apparatus, or the apparatus is a network node, or the apparatus includes a network node, or the network node includes the apparatus.

[0149] Example 37. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus to at least: receive a report from a first user equipment, the report including data collected for operation, wherein the operation includes artificial intelligence or machine learning operation; transmit the reported, collected data for operation to a second user equipment; receive from the second user equipment a trigger message for performing a traceability data error check on the reported data to determine traceability error information; and transmit to the first user equipment a trigger message for performing a traceability data error check on the reported data to determine traceability error information.

[0150] Example 38. The apparatus according to Example 37, wherein the traceability error information includes at least one or more of the following: a unique identifier of an erroneous data sample of the reported data, a unique identifier of a set of data samples including an erroneous data sample of the reported data, an erroneous component of a data sample of the reported data, a correction of an erroneous component of a data sample of the reported data, the level of error in an erroneous component of a data sample of the reported data, the actual error in an erroneous component of a data sample of the reported data, an indication that at least one data sample or other aspect of the reported data may be erroneous, and a unique identifier of at least one data sample or other aspect of the reported data that may be erroneous.

[0151] Example 39. An apparatus according to any one of Examples 37 to 38, wherein the apparatus is further configured to: receive traceability error information from a first user equipment; and send traceability error information to a second user equipment.

[0152] Example 40. The apparatus according to Example 39, wherein: a trigger message received from a second user equipment and sent to a first user equipment includes an identifier of a potentially erroneous data sample of the reported data; and traceability error information received from the first user equipment and sent to the second user equipment includes an update of the potentially erroneous data sample of the reported data.

[0153] Example 41. An apparatus according to any one of Examples 37 to 40, wherein: the apparatus is a core network entity, or the apparatus includes a core network entity, or the core network entity includes the apparatus, or the apparatus is a radio access network node, or the apparatus includes a radio access network node, or the radio access network node includes the apparatus, or the apparatus is a network node, or the apparatus includes a network node, or the network node includes the apparatus.

[0154] Example 42. A method comprising: collecting data for an operation, wherein the operation includes artificial intelligence or machine learning operations; reporting the data collected for the operation to a network entity; identifying traceability error information associated with the reported data; and sending the traceability error information associated with the reported data to the network entity.

[0155] Example 43. A method comprising: configuring a user device to collect data for operation, wherein the operation includes artificial intelligence or machine learning operation; receiving a report from the user device, the report including the data collected for the operation; receiving traceability error information associated with the reported data from the user device; and performing an action based on the traceability error information associated with the reported data received from the user device.

[0156] Example 44. A method comprising: receiving a report from a first user equipment, the report including data collected for operation, wherein the operation includes artificial intelligence or machine learning operation; sending the reported, collected data for operation to a second user equipment; receiving from the second user equipment a trigger message for performing a traceability data error check on the reported data to determine traceability error information; and sending the trigger message to the first user equipment for performing a traceability data error check on the reported data to determine traceability error information.

[0157] Example 45. An apparatus comprising: components for collecting data for operation, wherein the operation includes artificial intelligence or machine learning operation; components for reporting the collected data for operation to a network entity; components for determining traceability error information associated with the reported data; and components for sending the traceability error information associated with the reported data to the network entity.

[0158] Example 46. An apparatus comprising: components for configuring a user equipment to collect data for operation, wherein the operation includes artificial intelligence or machine learning operation; components for receiving a report from the user equipment, the report including the data collected for operation; components for receiving traceability error information associated with the reported data from the user equipment; and components for performing an action based on the traceability error information associated with the reported data received from the user equipment.

[0159] Example 47. An apparatus comprising: components for receiving a report from a first user equipment, the report including data collected for operation, wherein the operation includes artificial intelligence or machine learning operation; components for sending the reported, collected data for operation to a second user equipment; components for receiving from the second user equipment a trigger message for performing a traceability data error check on the reported data to determine traceability error information; and components for sending the trigger message to the first user equipment for performing a traceability data error check on the reported data to determine traceability error information.

[0160] Example 48. A computer-readable medium including instructions stored thereon for performing at least the following operations: collecting data for operations, wherein the operations include artificial intelligence or machine learning operations; reporting the data collected for operations to a network entity; determining traceability error information associated with the reported data; and sending the traceability error information associated with the reported data to the network entity.

[0161] Example 49. A computer-readable medium including instructions stored thereon for performing at least the following: configuring a user equipment to collect data for operation, wherein the operation includes artificial intelligence or machine learning operation; receiving a report from the user equipment, the report including data collected for operation; receiving traceability error information associated with reported data from the user equipment; and performing an action based on the traceability error information associated with reported data received from the user equipment.

[0162] Example 50. A computer-readable medium including instructions stored thereon for performing at least the following: receiving a report from a first user equipment, the report including data collected for operation, wherein the operation includes artificial intelligence or machine learning operation; sending the reported, collected data for operation to a second user equipment; receiving from the second user equipment a trigger message for performing a traceability data error check on the reported data to determine traceability error information; and sending to the first user equipment a trigger message for performing a traceability data error check on the reported data to determine traceability error information.

[0163] Example 51. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus to at least: receive data provided by a network entity, the data including data collected for operation, wherein the operation includes artificial intelligence or machine learning operations; receive traceability error information associated with the provided data from the network entity; and perform an action based on the traceability error information associated with the provided data received from the network entity.

[0164] Example 52. According to the apparatus of Example 51, the data sample of the provided data includes at least one of the following: measurement, truth label, quality indicator, timestamp.

[0165] Example 53. An apparatus according to any one of Examples 51 to 52, wherein the traceability error information includes at least one or more of the following: a unique identifier of an erroneous data sample of the provided data, a unique identifier of a set of data samples including an erroneous data sample of the provided data, an erroneous component of a data sample of the provided data, a correction of an erroneous component of a data sample of the provided data, the level of error in an erroneous component of a data sample of the provided data, the actual error in an erroneous component of a data sample of the provided data, an indication that at least one data sample or other aspect of the provided data may be erroneous, and a unique identifier of at least one data sample or other aspect of the provided data that may be erroneous.

[0166] Example 54. The apparatus according to Example 53, wherein an erroneous data sample of already provided data, or a set of data samples including an erroneous data sample of already provided data, or an erroneous component of a data sample of already provided data, or at least one potentially erroneous data sample of already provided data, or other aspects thereof, is identified by at least one or more of the following: a timestamp, a unique record identifier, an identifier of a protocol session for the collection of already provided data, and an identifier of a protocol message for the collection of already provided data.

[0167] Example 55. An apparatus according to any one of Examples 51 to 54, wherein the apparatus is further configured to: receive from a network entity a unique data identifier for each data sample of collected data provided for operation; wherein an action performed based on traceability error information for the provided data is performed based on at least one unique data identifier among the unique data identifiers for the data samples of collected data provided for operation.

[0168] Example 56. An apparatus according to any one of Examples 51 to 55, wherein the apparatus is further configured to: send a trigger message to a network entity for performing a traceability data error check on the data already provided; wherein traceability error information associated with the data already provided is received from the network entity based on the sending of the trigger message.

[0169] Example 57. The apparatus according to Example 56, wherein the apparatus is further configured to: determine that there is a potential error in the provided data for operation or that the provided data for operation is potentially corrupted; wherein a trigger message for performing a traceable data error check on the provided data is sent to the network entity in response to determining that there is a potential error in the provided data for operation or that the provided data for operation is potentially corrupted.

[0170] Example 58. The apparatus according to Example 57, wherein determining that there is a potential error in the already provided data for operation or that the already provided data for operation is potentially corrupted is based on one of the following: using the already provided data to monitor the performance of the operation, wherein the operation includes executing a pre-trained artificial intelligence or machine learning model, comparing the already provided data with a dataset, and observing the consistency between the already provided data and data previously provided by the network entity.

[0171] Example 59. An apparatus according to any one of Examples 56 to 58, wherein a trigger message sent to a network entity for performing a traceable data error check on provided data includes one of the following: a unique identifier of a data sample of provided data identified as potentially erroneous, and a unique identifier of a set of data samples, the set of data samples including the data sample of provided data identified as potentially erroneous, and the components of the data sample of provided data identified as potentially erroneous.

[0172] Example 60. An apparatus according to any one of Examples 51 to 59, wherein the action performed based on traceability error information associated with already provided data includes one of the following: updating the already provided data and performing an operation using the updated already provided data; updating an error data sample having an identifier corresponding to an identifier of an error data sample received together with the traceability error information; updating a trained model and performing an operation using the updated trained model, the trained model being trained using the already provided data; discarding error data from the already provided data and performing an operation using the already provided data without the discarded error data; and sending traceability error information associated with the already provided data to a user equipment.

[0173] Example 61. An apparatus according to any one of Examples 51 to 60, wherein the apparatus is further configured to: send to a network entity at least one criterion for determining anomalous data; wherein traceability error information received from the network entity in connection with data already provided is based on at least one criterion sent to the network entity for determining anomalous data.

[0174] Example 62. The apparatus according to Example 61, wherein at least one criterion for determining anomalous data is sent to a network entity along with a trigger message, the trigger message being used to perform a traceability data error check on the data already provided to determine traceability error information.

[0175] Example 63. An apparatus according to any one of Examples 61 to 62, wherein: at least one criterion for determining anomalous data includes a range of values; and traceability error information associated with the provided data is based on a sample of anomalous data from the provided data that is outside the range of values.

[0176] Example 64. An apparatus according to any one of Examples 51 to 63, wherein the apparatus is further configured to: send a first threshold to a network entity, the first threshold being used to determine that a data sample of the provided data is an erroneous data sample.

[0177] Example 65. The apparatus according to Example 64, wherein: the difference between the synthetic data sample generated at the moment when the data sample of the already provided data is collected and the data sample of the already provided data is greater than a first threshold, the data sample of the already provided data is an erroneous data sample, and the traceability error information includes information related to the data sample being an erroneous data sample.

[0178] Example 66. An apparatus according to any one of Examples 51 to 65, wherein: the difference between a synthetic data sample generated at the moment when a data sample of the already provided data is collected and a data sample of the already provided data is greater than a second threshold, the data sample of the already provided data is an erroneous data sample, and the traceability error information includes information related to the data sample being an erroneous data sample. The first threshold and the second threshold may be the same or different.

[0179] Example 67. An apparatus according to any one of Examples 51 to 66, wherein: the apparatus is a user equipment, or the apparatus includes a user equipment, or the user equipment includes an apparatus.

[0180] Example 68. An apparatus according to any one of Examples 51 to 67, wherein the network entity is or includes: a core network entity, or a radio access network node, or a user equipment.

[0181] Example 69. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus to at least: collect data for operation, wherein the operation includes artificial intelligence or machine learning operations; provide the collected data for operation to a user equipment; determine traceability error information associated with the provided data; and send the traceability error information associated with the provided data to the user equipment.

[0182] Example 70. The apparatus according to Example 69, wherein the data sample of the provided data includes at least one of the following: measurement, truth label, quality indicator, timestamp.

[0183] Example 71. An apparatus according to any one of Examples 69 to 70, wherein the traceability error information includes at least one or more of the following: a unique identifier of an erroneous data sample of the provided data, and a unique identifier of a set of data samples including the erroneous data sample of the provided data, and an erroneous component of a data sample of the provided data, and a correction of the erroneous component of the data sample of the provided data, and the level of error in the erroneous component of the data sample of the provided data, the actual error in the erroneous component of the data sample of the provided data, an indication that at least one data sample or other aspect of the provided data is potentially erroneous, and a unique identifier of at least one data sample or other aspect of the provided data that is potentially erroneous.

[0184] Example 72. The apparatus according to Example 71, wherein an erroneous data sample of already provided data, or a set of data samples including an erroneous data sample of already provided data, or an erroneous component of a data sample of already provided data, or at least one data sample or other aspect of a potentially erroneous data of already provided data is identified by at least one or more of the following: a timestamp, a unique record identifier, an identifier of a protocol session for the collection of already provided data, and an identifier of a protocol message for the collection of already provided data.

[0185] Example 73. An apparatus according to any one of Examples 69 to 72, wherein the apparatus is further configured to: send to a user equipment a unique data identifier for each data sample of the collected data provided for operation.

[0186] Example 74. An apparatus according to any one of Examples 69 to 73, wherein the apparatus is further configured to: receive from a user equipment a trigger message for performing a traceability data error check on the provided data to determine traceability error information; wherein, in response to receiving the trigger message from the user equipment, the traceability data error check for determining traceability error information is performed.

[0187] Example 75. The apparatus of Example 74, wherein a trigger message received from a user equipment for performing a traceable data error check on provided data includes one of the following: a unique identifier of the provided data that is identified as a potentially erroneous data sample, and a unique identifier of a set of data samples, the set of data samples including the provided data containing potentially erroneous data samples, and the components of the provided data containing potentially erroneous data samples.

[0188] Example 76. An apparatus according to any one of Examples 69 to 75, wherein the apparatus is further configured to determine traceability error information based on one of the following: recalibration of at least one sensor of the apparatus, recalibration of at least one radio frequency component of the apparatus, change of method for collecting data for operation, change of positioning technology for generating the provided data, receiving verification data after collecting the provided data, and determining a systematic error in collecting the provided data.

[0189] Example 77. An apparatus according to any one of Examples 69 to 76, wherein the traceability error information is based on an error in already provided data, and wherein the apparatus learns of the error in the already provided data based on: recalibration of at least one sensor of the apparatus, or receiving verification data after the already provided data has been collected, or determining that the reference time used to timestamp the measurement results is incorrect, wherein the already collected data includes the measurement results, or determining a systematic error in the collection of the already provided data.

[0190] Example 78. An apparatus according to any of Examples 69 to 77, wherein the apparatus is trained to identify errors in the collected data.

[0191] Example 79. An apparatus according to any of Examples 69 to 78, wherein the apparatus is trained to determine the probability that the collected data has errors.

[0192] Example 80. An apparatus according to any one of Examples 69 to 79, wherein the apparatus is further configured to: receive from a user equipment at least one criterion for determining anomalous data; wherein traceability error information associated with the provided data is determined based on at least one criterion received from the user equipment for determining anomalous data.

[0193] Example 81. The apparatus according to Example 80, wherein the apparatus is further configured to: perform a traceability data error check to determine traceability error information in response to receiving a trigger message from a user equipment; wherein at least one criterion for determining abnormal data is received from the user equipment together with the trigger message to perform a traceability data error check on the data already provided.

[0194] Example 82. An apparatus according to any one of Examples 80 to 81, wherein the apparatus is further configured to: determine that an anomalous data sample of the provided data is outside the range of values, wherein at least one criterion for determining the anomalous data includes a range of values; wherein the traceability error information associated with the provided data is based on determining that an anomalous data sample of the provided data is outside the range of values.

[0195] Example 83. An apparatus according to any one of Examples 69 to 82, wherein the apparatus is further configured to: receive a first threshold from a user equipment, the first threshold being used to determine that a data sample of the provided data is an erroneous data sample.

[0196] Example 84. The apparatus according to Example 83, wherein the apparatus is further configured to: generate a synthetic data sample at the moment when a data sample of the already provided data is collected; determine the difference between the synthetic data sample and the data sample of the already provided data; and determine that the data sample of the already provided data is an erroneous data sample in response to the difference between the synthetic data sample and the data sample of the already provided data being greater than a first threshold; wherein the traceability error information includes information related to the data sample being an erroneous data sample.

[0197] Example 85. An apparatus according to any one of Examples 69 to 84, wherein the apparatus is further configured to: generate a synthetic data sample at the moment when a data sample of the already provided data is collected; determine the difference between the synthetic data sample and the data sample of the already provided data; and determine that the data sample of the already provided data is an erroneous data sample in response to the difference between the synthetic data sample and the data sample of the already provided data being greater than a second threshold; wherein the traceability error information includes information related to the data sample being an erroneous data sample.

[0198] Example 86. An apparatus according to any one of Examples 69 to 85, wherein: the apparatus is a core network entity, or the apparatus includes a core network entity, or the core network entity includes the apparatus, or the apparatus is a radio access network node, or the apparatus includes a radio access network node, or the radio access network node includes the apparatus, or the apparatus is a network node, or the apparatus includes a network node, or the network node includes the apparatus.

[0199] Example 87. A method comprising: receiving data provided by a network entity, the data including data collected for operation, wherein the operation includes artificial intelligence or machine learning operation; receiving traceability error information associated with the provided data from the network entity; and performing an action based on the traceability error information associated with the provided data received from the network entity.

[0200] Example 88. A method comprising: collecting data for an operation, wherein the operation includes artificial intelligence or machine learning operations; providing the data collected for the operation to a user device; determining traceability error information associated with the provided data; and sending the traceability error information associated with the provided data to the user device.

[0201] Example 89. An apparatus comprising: components for receiving data provided by a network entity from the network entity, the data including data collected for operation, wherein the operation includes artificial intelligence or machine learning operation; components for receiving traceability error information associated with the provided data from the network entity; and components for performing an action based on the traceability error information associated with the provided data received from the network entity.

[0202] Example 90. An apparatus comprising: components for collecting data for operation, wherein the operation includes artificial intelligence or machine learning operation; components for providing the collected data for operation to a user device; components for determining traceability error information associated with the provided data; and components for sending the traceability error information associated with the provided data to the user device.

[0203] Example 91. A computer-readable medium including instructions stored thereon for performing at least the following operations: receiving data provided by a network entity, the data including data collected for operation, wherein the operation includes artificial intelligence or machine learning operation; receiving traceability error information associated with the provided data from the network entity; and performing an action based on the traceability error information associated with the provided data received from the network entity.

[0204] Example 92. A computer-readable medium including instructions stored thereon for performing at least the following operations: collecting data for operations, wherein the operations include artificial intelligence or machine learning operations; providing the collected data for operations to a user device; determining traceability error information associated with the provided data; and sending the traceability error information associated with the provided data to the user device.

[0205] References to "computer," "processor," etc., should be understood to include not only computers with different architectures, such as single-processor / multi-processor architectures and sequential or parallel architectures, but also special-purpose circuits, such as field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), signal processing devices, and other processing circuits. References to computer programs, instructions, code, etc., should be understood to include software or firmware used with programmable processors, such as the programmable content of hardware devices, whether it is the processor's instructions or the configuration settings of fixed-function devices, gate arrays, or programmable logic devices.

[0206] The memory described herein can be implemented using any suitable data storage technology, such as semiconductor-based memory devices, flash memory, magnetic storage devices and systems, optical storage devices and systems, non-transitory memory, transient memory, fixed memory, and removable memory. The memory may include a database for storing data.

[0207] The term “non-transient” as used in this article refers to the limitations of the medium itself (i.e., tangible, not signal-based), rather than limitations on the persistence of data storage (e.g., RAM compared to ROM).

[0208] As used herein, the term "circuit system" may refer to: (a) a hardware circuit implementation, such as an implementation in an analog circuit system and / or a digital circuit system; and (b) a combination of circuitry and software (and / or firmware), such as (if applicable): (i) a combination of processors, or (ii) a portion of a processor / software comprising a digital signal processor, software, and memory, which work together to enable a device to perform various functions; and (c) circuitry that requires software or firmware to function, such as a microprocessor or a portion thereof, even if the software or firmware is not physically present. As a further example, the term "circuit system" as used herein may also cover an implementation consisting solely of a processor (or a plurality of processors) or a portion thereof and its accompanying software and / or firmware. The term "circuit system" may also cover, for example (if applicable), baseband integrated circuits or application processor integrated circuits for mobile phones, or similar integrated circuits in servers, cellular network equipment, or other network equipment.

[0209] It should be understood that the above description is illustrative only. Those skilled in the art can devise various alternatives and modifications. For example, the features recited in the dependent claims can be combined with each other in any suitable combination. Furthermore, features of the different embodiments described above can be selectively combined to form a new embodiment. Therefore, this description is intended to cover all such alternatives, modifications, and variations falling within the scope of the appended claims.

[0210] The following abbreviations and abbreviations that may appear in this specification and / or accompanying drawings are defined as follows (these abbreviations and abbreviations may be connected to each other or combined with other characters, such as using hyphens, forward slashes, letters or numbers, and may be case-insensitive):

[0211] 3GPP - Third Generation Partnership Project

[0212] 4G – Fourth Generation

[0213] 5G – the fifth generation

[0214] 5GC – 5G Core Network

[0215] 6G – Sixth Generation

[0216] AI - Artificial Intelligence

[0217] AI / ML — Artificial Intelligence / Machine Learning

[0218] AMF – Access and Mobility Management Function

[0219] ASIC — Application-Specific Integrated Circuit

[0220] CD - Optical Disc / Computer Disk

[0221] CIR—Channel Impulse Response

[0222] CPU — Central Processing Unit

[0223] CSI – Channel State Information

[0224] CU — Central Unit or Centralized Unit

[0225] DC - Dual Connection

[0226] DL - Downlink

[0227] DSP—Digital Signal Processor

[0228] DU—Distributed Unit

[0229] DVD – Digital Multifunction Optical Disc; eNB – Evolved Node B (e.g., LTE base station)

[0230] EN-DC—E-UTRAN New Radio—Dual-Connectivity en-gNB—provides NR user plane and control plane protocol termination to the UE and acts as a secondary node in EN-DC.

[0231] EPC – Evolution Group Core

[0232] E-UTRA – Evolved UMTS Terrestrial Radio Access, also known as LTE Radio Access Technology

[0233] E-UTRAN——E-UTRA network

[0234] F1 – Interface between CU and DU

[0235] FG – Feature Group

[0236] FPGA—Field-Programmable Gate Array (gNB)—is a general-purpose node B used in 5G / NR base stations. It provides NR user plane and control plane protocol termination to the UE and connects to the 5GC node via the NG interface.

[0237] GNSS—Global Navigation Satellite System

[0238] IAB – Integrated Access and Backhaul

[0239] ID — Identifier or identifier

[0240] I / F — Interface

[0241] I / O — Input / Output

[0242] LCM – Lifecycle Management

[0243] LMF – Location Management Function

[0244] LOS - Line of Sight

[0245] LPP – LTE Location Protocol

[0246] LTE – Long Term Evolution (4G)

[0247] MAC – Media Access Control

[0248] ML - Machine Learning

[0249] MME – Mobility Management Entity

[0250] MRO – Mobility Robustness Optimization

[0251] NCE – Network Control Element; ng or NG – Next Generation ng-eNB; NG-RAN – Next Generation Radio Access Network

[0252] NLOS - Non-line-of-sight

[0253] NR - New Radio

[0254] NW - Network

[0255] N / W — Network

[0256] OAM – Operation, Management and Maintenance / Operation and Management

[0257] OTT - Over-the-top

[0258] PDA - Personal Digital Assistant

[0259] PDCP – Packet Data Convergence Protocol

[0260] PHY — Physical Layer

[0261] PRS—Positioning Reference Signal

[0262] PRU – Location Reference Unit

[0263] RAM—Random Access Memory

[0264] RAN—Radio Access Network

[0265] Rel — Version (Release)

[0266] RF - Radio Frequency

[0267] RLC – Radio Link Control

[0268] ROM - Read-Only Memory

[0269] RRC – Radio Resource Control

[0270] RS—Reference Signal

[0271] RSRPP—Receive Reference Signal Path Power

[0272] RU—Radio Unit

[0273] Rx — Receiver, or receiver

[0274] S1 – The interface between the Mobility Management Entity (MME) in EPC and the Evolved Node B in E-UTRAN.

[0275] SDAP - Service Data Adaptation Protocol

[0276] SGW - Service Gateway

[0277] SMF – Session Management Function

[0278] SON – Self-Organizing / Optimizing Network

[0279] SRS—Detection Reference Signal

[0280] TBS – Ground Beacon System

[0281] TRP - Transmitter / Receiver Point

[0282] Tx — Transmission, or transmitter

[0283] UAV - Unmanned Aerial Vehicle

[0284] UE – User Equipment (e.g., wireless, typically mobile equipment)

[0285] UI - User Interface

[0286] UMTS – Universal Mobile Telecommunications System

[0287] UPF - User Face Function

[0288] USB - Universal Serial Bus

[0289] X2 – Network interface between RAN nodes and between RAN and core network

[0290] Xn — Network interface between NG-RAN nodes.

Claims

1. An apparatus for communication, comprising: at least one processor; and at least one memory that stores instructions, which when executed by the at least one processor, cause the apparatus at least to: collect data for an operation, wherein the operation comprises an artificial intelligence or machine learning operation; report the data that was collected for the operation to a network entity; determine retroactive error information associated with the data that was reported; and send the retroactive error information associated with the data that was reported to the network entity.

2. The apparatus of claim 1, wherein a data sample of the data that was reported comprises at least one of: a measurement, a true value label, a quality indicator, a timestamp.

3. The apparatus of claim 1, wherein the retroactive error information comprises at least one or more of: a unique identification of an erroneous data sample of the data that was reported, a unique identification of a group of data samples that includes an erroneous data sample of the data that was reported, an erroneous component of a data sample of the data that was reported, a correction to an erroneous component of a data sample of the data that was reported, a level of error in an erroneous component of a data sample of the data that was reported, an actual error in an erroneous component of a data sample of the data that was reported, an indication that at least one data sample or other aspect of the data that was reported is potentially erroneous, a unique identification of at least one data sample or other aspect of the data that was reported that is potentially erroneous.

4. The apparatus of claim 3, wherein the erroneous data sample of the data that was reported, or the group of data samples that includes the erroneous data sample of the data that was reported, or the erroneous component of the data sample of the data that was reported, or the at least one data sample or other aspect of the data that was reported that is potentially erroneous is identified with at least one or more of: a timestamp, a unique record identifier, an identifier of a protocol session for the collection of the data that was reported, an identifier of a protocol message for the collection of the data that was reported.

5. The apparatus of claim 1, wherein the apparatus is further caused to: send, to the network entity, a unique data identifier for each data sample of the data that was reported that was collected for the operation.

6. The apparatus of claim 1, wherein the apparatus is further caused to: receive, from the network entity, a trigger message for performing a retroactive data error check of the data that was reported to determine the retroactive error information; wherein the retroactive data error check to determine the retroactive error information is performed in response to receiving the trigger message from the network entity.

7. The apparatus of claim 6, wherein the trigger message received from the network entity for performing the retroactive data error check of the data that was reported comprises one of: a. a trigger message for performing a retroactive data error check of the data that was reported to determine the retroactive error information; a unique identification of a potentially erroneous data sample of the data that has been reported, or a unique identification of a group of data samples of the data that has been reported, or a component of a potentially erroneous data sample of the data that has been reported.

8. The apparatus according to claim 1, wherein the apparatus is further caused to determine to perform a retroactive data error check of the data that has been reported to determine the retroactive error information based on one of: a recalibration of at least one sensor of the apparatus, a recalibration of at least one radio frequency component of the apparatus, a change of a method used to collect the data for the operation, a change of a positioning technology of the apparatus, a reception of correction data after the collection of the data that has been reported, a determination of a systematic error for collecting the data that has been reported.

9. The apparatus according to claim 1, wherein the retroactive error information is based on an error of the data that has been reported, and wherein the apparatus learns of the error of the data that has been reported based on: a recalibration of at least one sensor of the apparatus, or a reception of correction data after the collection of the data that has been reported, or a determination that a reference time used to time stamp a measurement, wherein the data that has been collected comprises the measurement, is erroneous, or a determination of a systematic error for collecting the data that has been reported.

10. The apparatus according to any one of claims 1 to 9, wherein the apparatus is further caused to: receive at least one criterion for determining abnormal data from the network; wherein the retroactive error information associated with the data that has been reported is determined based on the at least one criterion for determining abnormal data received from the network.

Images