Method of ai / ML model re-training in a wireless network

Abstract

Methods of using pre-configured AI / ML (artificial intelligence / machine learning) based model dataset adaptation in wireless mobile communication system including base stations (e.g., gNB) and mobile stations (e.g., UE). If an AI / ML model is applied to a radio access network, signaling of model dataset information exchange can be heavily congested based on different model execution environments with varying models when a model drift occurs. Therefore, model operation (e.g., model re-training / updating) can be set up between the network and the UE by selecting different subset(s) of features / datasets adaptively.

Description

[0001] TITLE

[0002] Method of AI / ML model re-training in a wireless network

[0003] TECHNNICAL FIELD

[0004] The present disclosure relates to AI / ML based model dataset adaptation, where techniques for pre-configuring and signaling the specific information about the model dataset adaptation using associations between drift information and datasets / features are presented.

[0005] BACKGROUND

[0006] In 3GPP (3rd Generation Partnership Project), one of the selected study items as of the approved Release 18 package is AI / ML (artificial intelligence / machine learning) as described in the related document (RP-213599) addressed in 3GPP TSG RAN (Technical Specification Group Radio Access Network) meeting #94e. The official title of the AI / ML study item is “Study on AI / ML for NR Air Interface”, and currently RAN WG1 and WG2 are actively working on a specification. The goal of this study item is to identify a common AI / ML framework and areas of obtaining gains using AI / ML based techniques with use cases and LCM (lifecycle management).

[0007] According to 3GPP, the main objective of this study item is to study AI / ML frameworks for air-interface with target use cases by considering performance, complexity, and potential specification impact. In particular, AI / ML model terminology and description to identify common and specific characteristics for the framework will be one of the key work scopes. Regarding AI / ML frameworks, various aspects are under consideration for investigation and one of the key items is about lifecycle management of AI / ML models where multiple stages are included as mandatory for model training, model deployment, model inference, model monitoring, model updating etc. Earlier, in 3GPP TR 37.817 for Release 17, titled as “Study on enhancement for Data Collection for NR and EN-DC”, UE (user equipment) mobility was also considered as one of the AI / ML use cases and one of the scenarios for model training / inference is that both functions are located within a RAN node. Followingly, in Release 18, the new work item of “Artificial Intelligence (Al) / Machine Learning (ML) for NG-RAN” was initiated to specify data collection enhancements and signaling support within existing NG-RAN interfaces and architectures.

[0008] For the above active standardization works, a model identification (e.g., model ID) to support RAN-based AI / ML models is considered as very significant for both the network and the UE to meet any desired model operations (e.g., model training, inference, selection, switching, update, monitoring, etc.). A model ID information can be signaled to pair both network-side and UE-side models for various lifecycle management (LCM) operations. However, if all physical models available on both sides are identified with their own model IDs separately, signaling overhead indicating the model ID information can be very high especially when model ID based LCM is processed between a base station (BS / gNB) and multiple UEs.

[0009] Currently, there is no specification defined for signaling methods and network-UE behaviors so as to identify the required datasets when the model is updating / re-training, as any activated model ID can be impacted due to a model / data drift. When AI / ML-enabled UE mobility occurs (due to moving around in different locations), the enabled AI / ML model(s) performance can be impacted due to a data / model drift. In this case, model re-training / updating can be executed.

[0010] However, if model is re-trained with the original full features / dataset configuration, high signaling overhead and / or high compute power can be very challenging.

[0011] For example, US 2016 092 786 A1 discloses an online system which simplifies modification of features used by machine learned models used by the online system, such as machined learned models with high dimensionality. The online system obtains a superset of features including features used by at least one machine learned model and may include additional features. From the superset of features, the online system generates various groups of features for a machine learned model. The groups of features may be a group including features currently used by the machine learned model, a group including all available features, and one or more intermediate groups. Intermediate groups include various numbers of features from the set selected based on measures of feature impact on the machine learned model associated with various features. A user may select a group of features, test the machine learning model using the selected group, and then launch the tested model based on the results.

[0012] US 2019 130 247 A1 discloses feature ranking within deep neural networks in a multi-task and / or multi-label setting. Approaches are described to identify features that are task-specific as well as features that are shared across multiple tasks. In addition to facilitating interpretability, the selected subset of features can be used to make efficient models leading to better stability & regularization along with reduced compute and memory.

[0013] US 2021 035 021 A1 provides a method of monitoring performance of a machine learning model externally to the machine learning model, comprising: monitoring data elements being fed into a machine learning model trained on a training dataset of historical training data elements, wherein the data elements are each associated with a respective time after the time associated with the training dataset, analyzing the data elements for identifying shift(s) between at least two subsets of the data elements, computing according to the shift(s), measurement(s) denoting an expected effect on output of the model, and detecting a misclassification event by the model when the measurement(s) exceeds a threshold of the model, wherein the monitoring, the analyzing, the computing, and the detecting are performed externally to the model, without accessing at least one of: data stored within the machine learning model, an implementation of the model, and data structures of the model.

[0014] US 2021 374 562 A1 discloses a system for streamlining machine learning. During operation, the system determines a resource overhead for a baseline version of a machine learning model that uses a set of features to produce entity rankings and a number of features to be removed to lower the resource overhead to a target resource overhead. Next, the system calculates importance scores for the features, wherein each importance score represents an impact of a corresponding feature on the entity rankings. The system then identifies a first subset of the features to be removed as the number of features with lowest importance scores and trains a simplified version of the machine learning model using a second subset of the features that excludes the first subset of the features. Finally, the system executes the simplified version to produce new entity rankings.

[0015] BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The disclosed invention will be further discussed in the following based on preferred embodiments presented in the attached drawings. However, the disclosed invention may be embodied in many different forms and should not be construed as limited to said preferred embodiments. Rather, said preferred embodiments are provided for thoroughness and completeness, and fully convey the scope of the invention to the skilled person. The following detailed description refers to the attached drawings, in which:

[0017] Figure 1 is an exemplary block diagram of generating a mapping relationship information for the indexed list of subsets of features / datasets.

[0018] Figure 2 is an exemplary table of mapping relationship information.

[0019] Figure 3 is an exemplary MAC CE format with LCID for indication of a subset of features / datasets.

[0020] Figure 4 is an exemplary flow chart of a network sided (e.g., gNB) behavior for adaptation of subset of features / datasets.

[0021] Figure 5 is an exemplary flow chart of a device sided (e.g., UE) behavior for adaptation of subset of features / datasets. Figure 6 is an exemplary block diagram of an adaptation of a subset of features / datasets with a decision by the network side.

[0022] Figure 7 is an exemplary block diagram of an adaptation of a subset of features / datasets with a decision by the device side.

[0023] Figure 8 is an exemplary signaling flow of an adaptation of a subset of features / datasets using a dataset at the device side.

[0024] Figure 9 is an exemplary signaling flow of an adaptation of a subset of features / datasets using a dataset at the network side.

[0025] Figure 10 is an exemplary signaling flow of triggering an adaptation of a subset of features / datasets at the network side.

[0026] Figure 11 is an exemplary block diagram of applying an adaptation of a subset of features / datasets using the mapping relationship information.

[0027] DETAILED DESCRIPTION

[0028] The detailed description set forth below, with reference to the annexed drawings, is intended as a description of various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. The detailed description includes specific details for the purpose of providing a thorough understanding of the various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. In particular, although terminology from 3GPP 5G NR may be used in this disclosure to exemplify embodiments herein, this should not be seen as limiting the scope of the invention.

[0029] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Other embodiments, however, are contained within the scope of the subject matter disclosed herein, the disclosed subject matter should not be construed as limited to only the embodiments set forth herein; rather, these embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art.

[0030] Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and / or is implied from the context in which it is used. All references to a / an / the element, apparatus, component, means, step, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and / or where it is implicit that a step must follow or precede another step. Any feature of any of the embodiments disclosed herein may be applied to any other embodiment, wherever appropriate. Likewise, any advantage of any of the embodiments may apply to any other embodiments, and vice versa. Other objectives, features and advantages of the enclosed embodiments will be apparent from the following description.

[0031] In some embodiments, a more general term “network node” may be used and may correspond to any type of radio network node or any network node, which communicates with a UE (directly or via another node) and / or with another network node. Examples of network nodes are NodeB, MeNB, ENB, a network node belonging to MCG or SCG, base station (BS), multi-standard radio (MSR) radio node such as MSR BS, eNodeB, gNodeB, network controller, radio network controller (RNC), base station controller (BSC), relay, donor node controlling relay, base transceiver station (BTS), access point (AP), transmission points, transmission nodes, RRU, RRH, nodes in distributed antenna system (DAS), core network node (e.g. Mobile Switching Center (MSC), Mobility Management Entity (MME), etc.), Operations & Maintenance (O&M), Operations Support System (OSS), Self Optimized Network (SON), positioning node (e.g. Evolved- Serving Mobile Location Centre (E-SMLC)), Minimization of Drive Tests (MDT), test equipment (physical node or software), another UE, etc. In some embodiments, the non-limiting term user equipment (UE) or wireless device may be used and may refer to any type of wireless device communicating with a network node and / or with another UE in a cellular or mobile communication system. Examples of UE are target device, device to device (D2D) UE, machine type UE or UE capable of machine to machine (M2M) communication, PDA, PAD, Tablet, mobile terminals, smart phone, laptop embedded equipped (LEE), laptop mounted equipment (LME), USB dongles, UE category Ml, UE category M2, ProSe UE, V2V UE, V2X UE, etc.

[0032] Additionally, terminologies such as base station / gNodeB and UE should be considered non-limiting and in particular do not imply a certain hierarchical relation between the two; in general, “gNodeB” could be considered as device 1 and “UE” could be considered as device 2 and these two devices communicate with each other over some radio channel. And in the following the transmitter or receiver could be either gNodeB (gNB), or UE.

[0033] As will be appreciated by one skilled in the art, aspects of the embodiments may be embodied as a system, apparatus, method, or computer program product.

[0034] Accordingly, embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects.

[0035] For example, the disclosed embodiments may be implemented as a hardware circuit comprising custom very-large-scale integration (“VLSI”) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. The disclosed embodiments may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, or the like. As another example, the disclosed embodiments may include one or more physical or logical blocks of executable code which may, for instance, be organized as an object, procedure, or function. Furthermore, embodiments may take the form of a computer program product embodied in one or more computer readable storage devices storing machine readable code, computer readable code, and / or program code, referred hereafter as code. The storage devices may be tangible, non- transitory, and / or non-transmission. The storage devices may not embody signals. In a certain embodiment, the storage devices only employ signals for accessing code.

[0036] Any combination of one or more computer readable medium may be utilized. The computer readable medium may be a computer readable storage medium. The computer readable storage medium may be a storage device storing the code. The storage device may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, holographic, micromechanical, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.

[0037] More specific examples (a non-exhaustive list) of the storage device would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random-access memory (“RAM”), a read-only memory (“ROM”), an erasable programmable read-only memory (“EPROM” or Flash memory), a portable compact disc read-only memory (“CD-ROM”), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0038] Code for carrying out operations for embodiments may be any number of lines and may be written in any combination of one or more programming languages including an object-oriented programming language such as Python, Ruby, Java, Smalltalk, C++, or the like, and conventional procedural programming languages, such as the “C” programming language, or the like, and / or machine languages such as assembly languages. The code may execute entirely on the user’s computer, partly on the user’s computer, as a stand-alone software package, partly on the user’s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user’s computer through any type of network, including a local area network (“LAN”), wireless LAN (“WLAN”), or a wide area network (“WAN”), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider (“ISP”)).

[0039] Furthermore, the described features, structures, or characteristics of the embodiments may be combined in any suitable manner. In the following description, numerous specific details are provided, such as examples of programming, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, etc., to provide a thorough understanding of embodiments. One skilled in the relevant art will recognize, however, that embodiments may be practiced without one or more of the specific details, or with other methods, components, materials, and so forth.

[0040] In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of an embodiment. Reference throughout this specification to “one embodiment,” “an embodiment,” or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, appearances of the phrases “in one embodiment”, “in an embodiment”, and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment, but mean “one or more but not all embodiments” unless expressly specified otherwise. The terms “including”, “comprising”, “having”, and variations thereof mean “including but not limited to”, unless expressly specified otherwise. An enumerated listing of items does not imply that any or all of the items are mutually exclusive, unless expressly specified otherwise. The terms “a”, “an”, and “the” also refer to “one or more” unless expressly specified otherwise.

[0041] Aspects of the embodiments are described below with reference to schematic flowchart diagrams and / or schematic block diagrams of methods, apparatuses, systems, and computer program products according to embodiments. It will be understood that each block of the schematic flowchart diagrams and / or schematic block diagrams, and combinations of blocks in the schematic flowchart diagrams and / or schematic block diagrams, can be implemented by code. This code may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart diagrams and / or block diagrams.

[0042] The code may also be stored in a storage device that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the storage device produce an article of manufacture including instructions which implement the function / act specified in the flowchart diagrams and / or block diagrams.

[0043] The code may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other devices to produce a computer implemented process such that the code which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart diagrams and / or block diagrams.

[0044] The flowchart diagrams and / or block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatuses, systems, methods, and program products according to various embodiments. In this regard, each block in the flowchart diagrams and / or block diagrams may represent a module, segment, or portion of code, which includes one or more executable instructions of the code for implementing the specified logical function(s).

[0045] It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. Other steps and methods may be conceived that are equivalent in function, logic, or effect to one or more blocks, or portions thereof, of the illustrated figures.

[0046] Although various arrow types and line types may be employed in the flowchart and / or block diagrams, they are understood not to limit the scope of the corresponding embodiments. Indeed, some arrows or other connectors may be used to indicate only the logical flow of the depicted embodiment. For instance, an arrow may indicate a waiting or monitoring period of unspecified duration between enumerated steps of the depicted embodiment. It will also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and code.

[0047] The description of elements in each figure may refer to elements of proceeding figures. Like numbers refer to like elements in all figures, including alternate embodiments of like elements.

[0048] The following explanation will provide the detailed description of the mechanism about pre-configuring AI / ML-based model before a handover occurrence in a wireless mobile communication system including base stations (e.g., gNB) and mobile stations (e.g., UE).

[0049] The following explanation will provide a detailed description of the mechanism about pre-configuring and signaling specific information about model selection using associations between models and index values. AI / ML based techniques are currently applied to many different applications and 3GPP also started to work on its technical investigation to apply them to multiple use cases based on the observed potential gains. An AI / ML lifecycle can be split into several stages such as data collection / pre-processing, model training, model testing / validation, model deployment / update, model monitoring etc., where in each stage it is equally important to achieve target performance with any specific model(s). In applying AI / ML models for any use cases or applications, one of the challenging issues is to manage the lifecycle of AI / ML models.

[0050] This is mainly caused a data / model drift, which occurs during model deployment / inference and which results in performance degradation of AI / ML models. Fundamentally, dataset statistical changes occur after the model is deployed and model inference capability is impacted when using unseen data as input. In a similar aspect, the statistical properties of datasets and the relationship between input and output for the trained model can be changed with drift occurrence. In this context, model selection is one of the key issues for model performance maintenance as model performance such as inferencing and / or training is dependent on different model execution environments with varying configuration parameters.

[0051] To handle this issue, a collaboration between a UE and a gNB is highly important to track model performance and re-configure models corresponding to different environments. AI / ML models needs model monitoring after deployment because model performance cannot be maintained continuously due to drift, and update feedback is then provided to re-train / update the model or select an alternative model. When an AI / ML model enabled wireless communication network is deployed, it is then important to consider how to handle AI / ML models in activation with re-configuration for wireless devices under operations such as model training, inference, updating, etc.

[0052] According to a first aspect of this invention, a method of AI / ML model re-training in a wireless network, the wireless network comprising a user equipment (UE) on which the AI / ML model is executed at least in part, the AI / ML model comprising multiple features / datasets, comprises the steps:

[0053] • Configure a mapping relationship, the mapping relationship mapping a subset of the features / datasets to a drift index, the drift index being associated with a model drift case, the mapping relationship further mapping the drift index to a threshold value or range, the threshold value or range being indicative of the model drift case, • Monitor a UE model performance by measuring the subset of features / datasets and comparing said measurements against the threshold value or range, and

[0054] • Re-train the AI / ML model if the UE model performance indicates a model drift by re-train the subset of features / datasets.

[0055] Advantageously, the mapping relationship comprises multiple subsets of the features / datasets with respective drift indices, and the steps of monitoring and re-training are implemented for each subset.

[0056] Advantageously, the multiple subsets of the features / datasets are prioritized, the prioritization determining the order in which the multiple subsets of the features / datasets are re-trained.

[0057] Advantageously, the threshold value or range is either based on a model validation measurement or pre-configured.

[0058] Advantageously, the mapping relationship is determined by the UE or a base station of the wireless network.

[0059] Advantageously, the mapping relationship is formed as one or more tables.

[0060] Advantageously, the feature is a measurable property or characteristic of input data that the AI / ML model uses to make predictions, including datasets, UE mobility data, a UE trajectory, or a UE site location.

[0061] Advantageously, the drift index is generated based on:

[0062] • An ML application, and / or

[0063] • An implementation use case, and / or

[0064] • A LCM operation, and / or

[0065] • A specific AI / ML model, and / or

[0066] • A specific data or model drift, and / or

[0067] • A UE mobility measurement. Advantageously, the UE reports the subset of features / datasets or the drift index to a base station of the wireless network , the subset of features / datasets or the drift index being signaled via:

[0068] • RRC signaling, and / or

[0069] • L1 / L2 signaling, and / or

[0070] • A reserved available LCID number, the LCID number describing the identity of the configured or selected subset of features / datasets or the drift index.

[0071] Advantageously, a base station of the wireless network determines the subset of features / datasets based on a ML measurement , the base station further signals a corresponding information to the UE.

[0072] Advantageously, the UE autonomously determines the subset of features / datasets based on the pre-configured mapping relationship, the UE further signals to a base station of the wireless network a corresponding information.

[0073] Advantageously, different subset(s) of features / datasets can be selected for model re-training / updating based on the pre-configured mapping relationship information look-up table. Multiple drift IDs in mapping relationship

[0074] Advantageously, a base station of the network provides a ML configuration to the UE with pre-configured subset identifiers of features / datasets with an index information.

[0075] Advantageously, the look-up table comprises a list of subset IDs of features / datasets which are indexed with an associated measurement ID that identifies a ML measurement, where the measurement ID is an index information that indicates the matching subset ID of features / datasets.

[0076] Advantageously, partial features / datasets are applied for the re-training step.

[0077] According to a second aspect of this invention, an apparatus for AI / ML model re-training in a wireless network comprises: • a wireless transceiver,

[0078] • a processor coupled with a memory in which computer program instructions are stored, said instructions being configured to implement afore-described steps and the apparatus being designed for the use in a base station (gNB).

[0079] According to a third aspect of this invention, an apparatus for AI / ML model re-training in a wireless network comprises:

[0080] • a wireless transceiver,

[0081] • a processor coupled with a memory in which computer program instructions are stored, said instructions being configured to implement afore-described steps and the apparatus being designed for the use in a user equipment (UE).

[0082] According to a fourth aspect of this invention, a base station (gNB) comprises an afore-described apparatus.

[0083] According to a fifth aspect of this invention, a User Equipment comprises an afore-described apparatus.

[0084] According to a sixth aspect of this invention, a wireless communication system comprises at least an afore-described base station (gNB) and at least an afore-described user equipment (UE).

[0085] This invention describes a method of configuring mapping relationship information between feature / dataset based subset information and index information for specific AI / ML model with the associated applications and / or functionalities, comprising:

[0086] • Configuring a finite set of features / datasets to produce multiple subsets of features / datasets when there are full set of features / datasets to be used as input to model;

[0087] • Applying partial features / datasets used for model re-training / updating and / or inferencing and / or monitoring based on measurements of UE mobility (e.g., UE trajectory, site location, etc.), model drift characteristics in association with the applied model properties; • Prioritizing partial features / datasets to be applied for model re-training / updating and / or inferencing and / or monitoring if necessary; and

[0088] • Splitting full set of features / datasets into subsets of features / datasets to apply use of partial features / datasets.

[0089] This invention describes a method, wherein each subsets of features / datasets are matched with index information in the mapping relationship structure, comprising:

[0090] • Index information can be identification (e.g., ID) of ML measurement data such as data / model drift and / or UE mobility (e.g., UE trajectory, site location, etc.), etc.;

[0091] • Multiple tables of the mapping relationship structures containing subsets of features / datasets and the associated matching index information can be generated for different ML applications or LCM operations with specific models.

[0092] This invention describes a method, wherein subset ID for subsets of features / datasets can be used for mapping relationship, comprising:

[0093] • Index information included in mapping relationship can be identification type (e.g., ID) of ML measurement data such as any combination of data / model drift and / or UE mobility measurement;

[0094] • Common or different threshold values for each subset of features / dataset IDs can be also configured by network side.

[0095] This invention describes a method, wherein UE reports corresponding index of subset of features / datasets to network side (e.g., via L1 / L2 signaling) after comparing ML measurement data with the configured threshold, comprising:

[0096] • Threshold values for selection of each indexed subsets can be pre-configured and shared (e.g., via RRC) based on historical data / repository or any measurement data;

[0097] • The reserved available LCID number can be used to signal index information (e.g., ML measurement data ID) to report index information;

[0098] • LCID value describes the identity of the configured / selected subset of features / datasets. This invention describes a method, wherein multiple subsets of features / datasets can be selected with prioritization of subsets depending on different implementation cases.

[0099] This invention describes a method, wherein network side can determine subset of features / datasets based on ML measurement data and the selection information is signaled to UE side.

[0100] This invention describes a method, wherein UE side autonomously can determine subset of features / datasets based on the pre-configured mapping relationship information about subset of features / dataset.

[0101] This invention describes a method, wherein different subset(s) of features / datasets can be selected for model re-training / updating based on the pre-configured mapping relationship information look-up table.

[0102] This invention describes a method, wherein prioritization for subset selection can be applied based on different implementation use cases when selecting any specific subset of features / datasets.

[0103] This invention describes a method, wherein network side can provide ML configuration to UE side with the pre-configured subset IDs of features / datasets with index information.

[0104] This invention describes a method, wherein the mapping relationship information can be implemented as look-up table, comprising:

[0105] • A list of subset IDs of features / datasets are indexed with the associated measurement ID that is equivalently identification of ML measurement data such as data / model drift and / or UE mobility (e.g., UE trajectory, site location, etc.), etc.;

[0106] • Measurement ID can be index information that indicates the matching subset ID of features / datasets; • Mapping relationship information can be only a list of subset IDs of features / datasets so that indication message of subset ID selection can be signaled based on ML measurement if necessary.

[0107] In this method, a mapping relationship information between feature / dataset based subset information and index information is configured for a specific AI / ML model with the associated applications and / or functionalities so that a finite set of features / datasets are identified to be multiple subsets of features / datasets. For example, in initial model training and / or inferencing there can be full set of features / datasets to be used as input to model. In model deployment, partial features / datasets can be used for model re-training / updating and / or inferencing and / or monitoring based on measurements of UE mobility (e.g., UE trajectory, site location, etc.), or model drift characteristics in association with the applied model properties. In addition, prioritization of partial features / datasets is also applied for model re-training / updating and / or inferencing and / or monitoring if necessary based on the same measurement results.

[0108] To use partial features / datasets, the full set of features / datasets is split into subsets of features / datasets and each subset is matched with an index information in the mapping relationship structure where the index information can be an identification (e.g., ID) of an ML measurement data such as data / model drift and / or UE mobility (e.g., UE trajectory, site location, etc.), etc. There can be multiple tables of the mapping relationship structures containing subsets of features / datasets and the associated matching index information for different ML applications or LCM operations with specific models. For subsets of features / datasets, the subset ID can be used for the mapping relationship. For index information included in mapping relationship, it can be an identification type (e.g., ID) of an ML measurement data such as any combination of data / model drift and / or UE mobility measurement.

[0109] In addition, common or different threshold values for each subset of features / dataset IDs can be configured by the network side. For example, after comparing ML measurement data with the configured threshold, the UE reports the corresponding index of subset of features / datasets to the network side (e.g., via L1 / L2 signaling) where the threshold values for selection of each indexed subsets can be also pre-configured and shared (e.g., via RRC) based on historical data / repository or any measurement data. To report the index information, the reserved available LCID number can be used to signal the index information (e.g., ML measurement data ID) where the LCID value describes the identity of the configured / selected subset of features / datasets.

[0110] Depending on different implementation cases, there can be multiple subsets of features / datasets to be selected with prioritization. For the selection decision about subset of features / datasets, there can be two scenarios as follows. In the first scenario, the network side determines the subset of features / datasets based on ML measurement data and the selection information is signaled to the UE side. In the second scenario, the UE side autonomously determines the subset of features / datasets based on the pre-configured mapping relationship information about the subset of features / dataset so that the UE side model re-training / updating can proceed.

[0111] Figure 1 shows an exemplary block diagram of generating a mapping relationship information for the indexed list of subsets of features / datasets. In this example, it describes how to generate the mapping relationship information to produce the subsets of features / dataset (e.g., in the format of IDs) and the associated index information where the index information can be an identification (e.g., ID) of an ML measurement data such as data / model drift and / or UE mobility (e.g., UE trajectory, site location, etc.), etc. In addition, the index information can then be used as a threshold value or range when the UE measures a model drift or a UE mobility to compare measurement data with the pre-configured threshold that is equivalent to index information mapped onto a subset of features / datasets. In an implementation aspect, the generation of the mapping relationship information can be performed through an offline measurement and computation processing. There can be multiple tables of the mapping relationship structures containing subsets of features / datasets and the associated matching index information for different ML applications or LCM operations with specific models. Figure 2 shows an exemplary table of the mapping relationship information. In this example, the mapping relationship information can be implemented as a look-up table and a list of subset IDs of features / datasets are indexed with the associated measurement ID that is equivalently an identification of a ML measurement data such as data / model drift and / or UE mobility (e.g., UE trajectory, site location, etc.), etc. The measurement ID can be an index information that indicates the matching subset ID of features / datasets. As another example, the mapping relationship information can be only a list of subset IDs of features / datasets so that an indication message of a subset ID selection can be signaled based on an ML measurement if necessary.

[0112] Figure 3 shows an exemplary MAC CE (MAC control element) format with LCID (logical channel ID) for an indication of a subset of features / datasets. In this example, an indication message of subset ID selection can be signaled via MAC CE where the first field can represent the LCID number from any value reserved and the second field can represent index information (e.g., measurement ID) to identify the selected subset of features / datasets based on the mapping relationship information describing a list of subset IDs of features / datasets with index information.

[0113] Figure 4 shows an exemplary flow chart of a network sided (e.g., gNB) behavior for an adaptation of a subset of features / datasets. In this example, the network side provides a ML configuration to the UE side with the pre-configured subset IDs of features / datasets with an index information. After the ML model is activated for deployment, the model is monitored and any model drift is detected with UE mobility (e.g., UE trajectory, site location, etc.). When a model drift occurs, the network side can take multiple measures such as activation of model re-training / updating, transmission of dataset transfer (e.g., subset of features / datasets) and / or transmission of an indication message for the UE-sided model re-training / updating.

[0114] Figure 5 shows an exemplary flow chart of a device sided (e.g., UE) behavior for an adaptation of a subset of features / datasets. In this example, when the UE detects a model drift (e.g., due to UE mobility), model re-training / updating is activated based on a selection of the pre-configured subset IDs of features / datasets based on a ML measurement where the ML measurement can be used to be compared with a threshold so as to determine the subset of features / datasets. With the selected subset of features / datasets, model re-training / updating is performed.

[0115] Figure 6 shows an exemplary block diagram of an adaptation of a subset of features / datasets with the decision by the network side. In this example, network side determines the subset of features / datasets based on a ML measurement data and the selection information is signaled to the UE side. The determined subset of features / datasets itself can be directly sent to the UE side from the network side if necessary, or the subset ID of features / datasets can be sent to the UE side so that locally, the indicated subset of features / datasets can be collected at the UE side.

[0116] Figure 7 shows an exemplary block diagram of an adaptation of a subset of features / datasets with the decision by the device side. In this example, the UE side autonomously determines the subset of features / datasets based on the pre-configured mapping relationship information about the subset of features / dataset so that the UE sided model re-training / updating can proceed.

[0117] Figure 8 shows an exemplary signaling flow of an adaptation of the subset of features / datasets using a dataset at device side. In this example, after detection of the triggered model re-training, the UE side autonomously determines the subset of features / datasets based on the pre-configured mapping relationship information about the subset of features / dataset so that the UE sided model re-training / updating can proceed. After model re-training, an indication message (e.g., via L1 / L2 signaling) is sent to the network side for alignment about the updated model status information.

[0118] Figure 9 shows an exemplary signaling flow of an adaptation of the subset of features / datasets using the dataset at the network side. In this example, the UE side autonomously determines the subset of features / datasets based on the pre-configured mapping relationship information about the subset of features / dataset. However, the request of the determined subset of features / datasets is sent to the network side so that the subset of features / datasets can be transferred to the UE for model re-training / updating.

[0119] Figure 10 shows an exemplary signaling flow of triggering an adaptation of the subset of features / datasets at the network side. In this example, the network side triggers the model re-training / updating so that an indication message is sent to the UE for the UE sided model re-training / updating based on the aligned mapping relationship information about a list of subsets of features / datasets. In this example, a two-sided model can be considered to apply.

[0120] Figure 11 shows an exemplary block diagram of applying an adaptation of the subset of features / datasets using the mapping relationship information. In this example, for different model drift cases, different subset(s) of features / datasets can be selected for model re-training / updating based on the pre-configured mapping relationship information look-up table. In addition, when selecting any specific subset of features / datasets, prioritization for subset selection can be applied based on different implementation use cases.

[0121] Abbreviations

[0122] BWP Bandwidth part

[0123] CBG Code block group

[0124] CLI Cross Link Interference

[0125] CP Cyclic prefix

[0126] CQI Channel quality indicator

[0127] CPU CSI processing unit

[0128] CRB Common resource block

[0129] CRC Cyclic redundancy check

[0130] CRI CSI-RS Resource Indicator

[0131] CSI Channel state information

[0132] CSI-RS Channel state information reference signal

[0133] CSI-RSRP CSI reference signal received power

[0134] CSI-RSRQ CSI reference signal received quality

[0135] CSI-SINR CSI signal-to-noise and interference ratio

[0136] CW Codeword

[0137] DCI Downlink control information

[0138] DL Downlink

[0139] DM-RS Demodulation reference signals

[0140] DRX Discontinuous Reception

[0141] EPRE Energy per resource element

[0142] IAB-MT Integrated Access and Backhaul - Mobile Terminal

[0143] L1 -RSRP Layer 1 reference signal received power

[0144] LI Layer Indicator

[0145] MCS Modulation and coding scheme

[0146] PDCCH Physical downlink control channel

[0147] PDSCH Physical downlink shared channel

[0148] PSS Primary Synchronisation signal

[0149] PUCCH Physical uplink control channel

[0150] QCL Quasi co-location

[0151] PMI Precoding Matrix Indicator

[0152] PRB Physical resource block

[0153] PRG Precoding resource block group PRS Positioning reference signal

[0154] PT-RS Phase-tracking reference signal

[0155] RB Resource block

[0156] RBG Resource block group

[0157] Rl Rank Indicator

[0158] RIV Resource indicator value

[0159] RP Resource Pool

[0160] RS Reference signal

[0161] SCI Sidelink control information

[0162] SL CR Sidelink Channel Occupancy Ratio

[0163] SL CBR Sidelink Channel Busy Ratio

[0164] SLIV Start and length indicator value

[0165] SR Scheduling Request

[0166] SRS Sounding reference signal

[0167] SS Synchronisation signal

[0168] SSS Secondary Synchronisation signal

[0169] SS-RSRP SS reference signal received power

[0170] SS-RSRQ SS reference signal received quality

[0171] SS-SINR SS signal-to-noise and interference ratio

[0172] TB Transport Block

[0173] TCI Transmission Configuration Indicator

[0174] TDM Time division multiplexing

[0175] UE User equipment

[0176] UL Uplink

Claims

CLAIMS1 . A method of AI / ML model re-training in a wireless network, the wireless network comprising a user equipment (UE) on which the AI / ML model is executed at least in part, the AI / ML model comprising multiple features / datasets, the method comprising the steps:• Configure a mapping relationship, the mapping relationship mapping a subset of the features / datasets to a drift index, the drift index being associated with a model drift case, the mapping relationship further mapping the drift index to a threshold value or range, the threshold value or range being indicative of the model drift case,• Monitor a UE model performance by measuring the subset of features / datasets and comparing said measurements against the threshold value or range, and• Re-train the AI / ML model if the UE model performance indicates a model drift by re-train the subset of features / datasets.

2. Method according to claim 1 , characterized in that the mapping relationship comprises multiple subsets of the features / datasets with respective drift indices, and the steps of monitoring and re-training are implemented for each subset.

3. Method according to claim 2, characterized in that the multiple subsets of the features / datasets are prioritized, the prioritization determining the order in which the multiple subsets of the features / datasets are re-trained.

4. Method according to one of the previous claims, characterized in that the threshold value or range is either based on a model validation measurement or pre-configured.

5. Method according to one of the previous claims, characterized in that the mapping relationship is determined by the UE or a base station of the wireless network.

6. Method according to one of the previous claims, characterized in that the mapping relationship is formed as one or more look-up tables.

7. Method according to one of the previous claims, characterized in that the feature is a measurable property or characteristic of input data that the AI / ML model uses to make predictions, including datasets, UE mobility data, a UE trajectory, or a UE site location.

8. Method according to one of the previous claims, characterized in that the drift index is generated based on:• An ML application, and / or• An implementation use case, and / or• A LCM operation, and / or• A specific AI / ML model, and / or• A specific data or model drift, and / or• A UE mobility measurement.

9. Method according to one of the previous claims, characterized in that the UE reports the subset of features / datasets or the drift index to a base station of the wireless network , the subset of features / datasets or the drift index being signaled via:• RRC signaling, and / or• L1 / L2 signaling, and / or• A reserved available LCID number, the LCID number describing the identity of the configured or selected subset of features / datasets or the drift index.

10. Method according to claim one of the previous claims, characterized in that a base station of the wireless network determines the subset of features / datasets based on a ML measurement , the base station further signals a corresponding information to the UE.11 . Method according to one of the previous claims, characterized in that the UE autonomously determines the subset of features / datasets based on the pre-configured mapping relationship, the UE further signals to a base station of the wireless network a corresponding information.

12. Method according to claim 6, characterized in that different subsets of features / datasets are selected for model re-training based on the pre-configured mapping relationship look-up table.

13. Method according to one of the previous claims, characterized in that a base station of the network provides a ML configuration to the UE with pre-configured subset identifiers of features / datasets with an index information.

14. Method according to claim 6 or 12, characterized in that the look-up table comprises a list of subset IDs of features / datasets which are indexed with an associated measurement ID that identifies a ML measurement, where the measurement ID is an index information that indicates the matching subset ID of features / datasets.

15. Method according to one of the previous claims, characterized in that, partial features / datasets are applied for the re-training step.

16. An apparatus for AI / ML model re-training in a wireless network, the apparatus comprising:• a wireless transceiver,• a processor coupled with a memory in which computer program instructions are stored, said instructions being configured to implement steps of the claims 1 to 13 and the apparatus being designed for the use in a base station (gNB).

17. An apparatus for AI / ML model re-training in a wireless network, the apparatus comprising:• a wireless transceiver,• a processor coupled with a memory in which computer program instructions are stored, said instructions being configured to implement steps of the claims 1 to 13 and the apparatus being designed for the use in a user equipment (UE).

18. A base station (gNB) comprising an apparatus according to claim 14.

19. A User Equipment comprising an apparatus according to claim 15.

20. A wireless communication system comprising at least a base station (gNB) according to claim 16 and at least a user equipment (UE) according to claim 17.

Images