Method and device for training machine learning model cluster in telecommunication network

Abstract

The disclosure relates to a 5G or 6G communication system for supporting a higher data transmission rate. The present disclosure relates to a method and device for training Machine Learning (ML) models cluster in a telecommunication network. The method performed by a Management Services (MnS) producer in a ML training function server includes a ML training request message including a clustering criteria from a MnS consumer device, sending a response message to the MnS consumer device indicating creation of a model object instance (MOI), selecting a set of ML models from a multiple of ML models that can be trained together by the ML training function server (100) based on the clustering criteria, forming a cluster of the set of selected ML models, and training the cluster of the set of selected ML models together.

Description

METHOD AND DEVICE FOR TRAINING MACHINE LEARNING MODEL CLUSTER IN TELECOMMUNICATION NETWORK

[0001] The present disclosure relates to the field of Artificial Intelligence (AI) and Machine Learning (ML) techniques. More specifically, the disclosure relates to the training of machine learning models in clusters within a telecommunication network.

[0002] 5G mobile communication technologies define broad frequency bands such that high transmission rates and new services are possible, and can be implemented not only in "Sub 6GHz" bands such as 3.5GHz, but also in "Above 6GHz" bands referred to as mmWave including 28GHz and 39GHz. In addition, it has been considered to implement 6G mobile communication technologies (referred to as Beyond 5G systems) in terahertz bands (for example, 95GHz to 3THz bands) in order to accomplish transmission rates fifty times faster than 5G mobile communication technologies and ultra-low latencies one-tenth of 5G mobile communication technologies.

[0003] At the beginning of the development of 5G mobile communication technologies, in order to support services and to satisfy performance requirements in connection with enhanced Mobile BroadBand (eMBB), Ultra Reliable Low Latency Communications (URLLC), and massive Machine-Type Communications (mMTC), there has been ongoing standardization regarding beamforming and massive MIMO for mitigating radio-wave path loss and increasing radio-wave transmission distances in mmWave, supporting numerologies (for example, operating multiple subcarrier spacings) for efficiently utilizing mmWave resources and dynamic operation of slot formats, initial access technologies for supporting multi-beam transmission and broadbands, definition and operation of BWP (BandWidth Part), new channel coding methods such as a LDPC (Low Density Parity Check) code for large amount of data transmission and a polar code for highly reliable transmission of control information, L2 pre-processing, and network slicing for providing a dedicated network specialized to a specific service.

[0004] Currently, there are ongoing discussions regarding improvement and performance enhancement of initial 5G mobile communication technologies in view of services to be supported by 5G mobile communication technologies, and there has been physical layer standardization regarding technologies such as V2X (Vehicle-to-everything) for aiding driving determination by autonomous vehicles based on information regarding positions and states of vehicles transmitted by the vehicles and for enhancing user convenience, NR-U (New Radio Unlicensed) aimed at system operations conforming to various regulation-related requirements in unlicensed bands, NR UE Power Saving, Non-Terrestrial Network (NTN) which is UE-satellite direct communication for providing coverage in an area in which communication with terrestrial networks is unavailable, and positioning.

[0005] Moreover, there has been ongoing standardization in air interface architecture / protocol regarding technologies such as Industrial Internet of Things (IIoT) for supporting new services through interworking and convergence with other industries, IAB (Integrated Access and Backhaul) for providing a node for network service area expansion by supporting a wireless backhaul link and an access link in an integrated manner, mobility enhancement including conditional handover and DAPS (Dual Active Protocol Stack) handover, and two-step random access for simplifying random access procedures (2-step RACH for NR). There also has been ongoing standardization in system architecture / service regarding a 5G baseline architecture (for example, service based architecture or service based interface) for combining Network Functions Virtualization (NFV) and Software-Defined Networking (SDN) technologies, and Mobile Edge Computing (MEC) for receiving services based on UE positions.

[0006] As 5G mobile communication systems are commercialized, connected devices that have been exponentially increasing will be connected to communication networks, and it is accordingly expected that enhanced functions and performances of 5G mobile communication systems and integrated operations of connected devices will be necessary. To this end, new research is scheduled in connection with eXtended Reality (XR) for efficiently supporting AR (Augmented Reality), VR (Virtual Reality), MR (Mixed Reality) and the like, 5G performance improvement and complexity reduction by utilizing Artificial Intelligence (AI) and Machine Learning (ML), AI service support, metaverse service support, and drone communication.

[0007] Furthermore, such development of 5G mobile communication systems will serve as a basis for developing not only new waveforms for providing coverage in terahertz bands of 6G mobile communication technologies, multi-antenna transmission technologies such as Full Dimensional MIMO (FD-MIMO), array antennas and large-scale antennas, metamaterial-based lenses and antennas for improving coverage of terahertz band signals, high-dimensional space multiplexing technology using OAM (Orbital Angular Momentum), and RIS (Reconfigurable Intelligent Surface), but also full-duplex technology for increasing frequency efficiency of 6G mobile communication technologies and improving system networks, AI-based communication technology for implementing system optimization by utilizing satellites and AI (Artificial Intelligence) from the design stage and internalizing end-to-end AI support functions, and next-generation distributed computing technology for implementing services at levels of complexity exceeding the limit of UE operation capability by utilizing ultra-high-performance communication and computing resources.

[0008] The adoption of Artificial Intelligence (AI) and Machine Learning (ML) techniques has seen a significant rise across various industries, demonstrating substantial success in enhancing operational efficiencies and decision-making processes. Recently, these advanced techniques have begun to permeate the telecommunication industry, particularly in mobile networks. Despite the general maturity of AI / ML technologies, certain aspects remain in a state of evolution, and new complementary techniques continue to emerge.

[0009] AI / ML techniques encompass a variety of learning methods including supervised learning, semi-supervised learning, unsupervised learning, and reinforcement learning. Each of these methods is tailored to specific categories of inference, such as prediction, and necessitates specific types of training data. The lifecycle management of AI / ML models is being standardized by the 3GPP SA5 working group (TS 28.105), delineating stages such as ML model training, ML testing, ML emulation, ML entity loading, and AI / ML inference.

[0010] 1. ML Model Training: This initial phase involves both the initial training and re-training of ML models or groups of models. Validation is a critical component, aimed at evaluating the performance of the ML entity on training and validation data. If the validation results are unsatisfactory (e.g., unacceptable variance), re-training of the associated ML model is necessary.

[0011] 2. ML Testing: Following validation, the ML entity undergoes testing to assess the performance of the trained ML model on testing data. Successful testing results allow progression to the next phase, while unsatisfactory results necessitate further re-training.

[0012] 3. ML Emulation: This phase involves running an ML entity for inference within an emulation environment to evaluate its performance prior to deployment in the target network or system.

[0013] 4. ML Entity Loading: This process involves making the trained ML entity available for use at the target AI / ML inference function.

[0014] 5. AI / ML Inference: The final phase involves performing inference using the trained ML entity by the AI / ML inference function.

[0015] A critical aspect of this lifecycle is the context under which the ML model is trained and inferenced. Context encompasses various conditions such as network conditions, traffic characteristics, time of day, weather, and climate. Each MLTrainingRequest specifies an expectedRunTimeContext, outlining the conditions for which the MLModel should be trained and applied.

[0016] The 3GPP TS 28.105 standard defines three types of contexts: ExpectedRunTimeContext, trainingContext, and RunTimeContext. Although ML models can provide AI / ML inference services for multiple scenarios, there are notable similarities in the contexts where these models operate. For instance, ML model instances designed for the same inference type in urban or rural areas may exhibit significant contextual overlap. Leveraging context similarity can facilitate the formation of ML model clusters, wherein models are either trained from a previously trained ML model or from a model trained for a similar context as the baseline. Training ML models for multiple contexts simultaneously offers efficiency gains through cluster training, as opposed to individually training each model.

[0017] However, the current standards only consider inferenceModelRef and dataProviderRef as ML context criteria, which limits the ability to cluster or group ML models for more efficient training. Introducing new inputs as ML context criteria to determine which ML models can be clustered or grouped for training together is essential to enhance efficiency, robustness, optimization, and overall quality of ML models.

[0018] Thus, it is desired to address the above-mentioned disadvantages or other shortcomings or at least provide a useful alternative.

[0019] The principal object of the disclosure herein is to provide a method and a system for machine learning models cluster training in a telecommunication network.

[0020] An object of the disclosure herein is to introduce a new attribute for ensuring clustering of ML models for training together based on their learning type.

[0021] An object of the disclosure herein is to introduce a new attribute for ensuring clustering of ML models which intend to achieve the same performance characteristics.

[0022] An of the disclosure herein is to introduce a new attribute for ensuring clustering of ML models having the same type of task, such as the specific inference type or capability for which an ML model can generate inference.

[0023] An object of the disclosure herein is to introduce a new attribute to ensure clustering of ML models that have same applicability domain and are designed to solve similar kinds of domain problems like text analysis and image processing.

[0024] An object of the disclosure herein is to introduce a new attribute ensuring clustering of ML models whose combined training time does not exceed the set time limit for cluster training.

[0025] An object of the disclosure herein is to introduce a new attribute ensuring clustering of ML models based on the required inference time of the final trained model.

[0026] An object of the disclosure herein is to introduce a new attribute based on the preferred clustering technique by the consumer for clustering the models for training.

[0027] An object of the disclosure herein is to introduce a new attribute based on the preferred model diversity types that can be considered for models clustering.

[0028] In an aspect, a method for training Machine Learning (ML) model cluster by a Management Services (MnS) producer in a ML training function server in a telecommunication network is disclosed. The method may comprise receiving a ML training request message from a MnS consumer device (200), wherein the ML training request message comprises a clustering criteria comprising at least one of a learning type of ML models, a performance metric of the ML models, a task type for the ML models, an applicability domain of the ML models, an allowed cluster training time of the ML models, an allowed cluster inference time of the ML models, a preferred clustering technique of the ML models, or a preferred model diversity type of the ML models; sending a response message to the MnS consumer device indicating creation of a model object instance (MOI); selecting a set of ML models from a plurality of ML models that can be trained together by the ML training function server based on the clustering criteria; forming a cluster of the set of selected ML models; and training the cluster of the set of selected ML models together.

[0029] In an aspect, g a method for training ML model cluster by a Management Services (MnS) consumer device in a telecommunication network is disclosed. The method may comprise generating a clustering criteria for clustering ML models to train together, wherein the clustering criteria comprises at least one of a learning type of the ML models, a performance metric of the ML models, a task type for the ML models, an applicability domain of the ML models, an allowed cluster training time of the ML models, an allowed cluster inference time of the ML models, a preferred clustering technique of the ML models, and a preferred model diversity type of the ML models; sending a ML training request message comprising the clustering criteria to a ML training function server; and receiving a response message from the MnS producer in the ML training function server (100) indicating creation of a model object instance (MOI).

[0030] In an aspect, a ML training function server including a Management Services (MnS) producer for training ML model cluster in a telecommunication network is disclosed. The ML training function server may comprise memory; and at least one processor operably coupled to the memory. The at least one processor may be configured to receive a ML training request message from a MnS consumer device, wherein the ML training request message comprises a clustering criteria comprising at least one of a learning type of ML models, a performance metric of the ML models, a task type for the ML models, an applicability domain of the ML models, an allowed cluster training time of the ML models, an allowed cluster inference time of the ML models, a preferred clustering technique of the ML models, or a preferred model diversity type of the ML models, send a response message to the MnS consumer device indicating creation of a model object instance (MOI), select a set of ML models from a plurality of ML models that can be trained together by the ML training function server based on the clustering criteria, form a cluster of the set of selected ML models, and train the cluster of the set of selected ML models together.

[0031] In an aspect, a MnS consumer device for training ML model cluster in a telecommunication network is disclosed. The MnS consumer device may comprise memory; and at least one processor operably coupled to the memory. The at least one processor may be configured to generate a clustering criteria for clustering ML models to train together, wherein the clustering criteria comprises at least one of a learning type of the ML models, a performance metric of the ML models, a task type for the ML models, an applicability domain of the ML models, an allowed cluster training time of the ML models, an allowed cluster inference time of the ML models, a preferred clustering technique of the ML models, and a preferred model diversity type of the ML models; send a ML training request message comprising the clustering criteria to a ML training function server; and receive a response message from the MnS producer in the ML training function server (100) indicating creation of a model object instance (MOI).

[0032] The present disclosure is illustrated in the accompanying drawings, throughout which like reference letters indicate corresponding parts in the various figures. The embodiments herein will be better understood from the following description with reference to the drawings, in which:

[0033] FIG. 1 is a block diagram that illustrates hardware components associated with a ML training function server according to embodiments as disclosed herein.

[0034] FIG. 2 is a block diagram that illustrates hardware components associated with a MnS consumer device according to embodiments as disclosed herein.

[0035] FIG. 3 is a flow diagram that illustrates a proposed method implemented by a MnS producer in an ML training function server for clustering ML models and training ML models cluster network according to embodiments as disclosed herein.

[0036] FIG. 4 is a flow diagram that illustrates a proposed method implemented by the MnS consumer device for training ML models cluster according to embodiments as disclosed herein.

[0037] FIG. 5 is a sequence diagram that illustrates an example scenario for clustering ML models and training ML models cluster according to embodiments as disclosed herein.

[0038] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein. Also, the various embodiments described herein are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments. The term "or" as used herein, refers to a non-exclusive or, unless otherwise indicated. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein can be practiced and to further enable those skilled in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.

[0039] As is existing in the field, embodiments can be described and illustrated in terms of blocks which carry out a described function or functions.　These blocks, which can be referred to herein as managers, units, modules, hardware components or the like, are physically implemented by analog and / or digital circuits such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits and the like, and can optionally be driven by firmware and software. The circuits may, for example, be embodied in one or more semiconductor chips, or on substrate supports such as printed circuit boards and the like.　The circuits constituting a block can be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware to perform some functions of the block and a processor to perform other functions of the block.　Each block of the embodiments can be physically separated into two or more interacting and discrete blocks without departing from the scope of the disclosure.　Likewise, the blocks of the embodiments can be physically combined into more complex blocks without departing from the scope of the disclosure.

[0040] The accompanying drawings facilitate understanding of various technical features. The embodiments presented are not limited by these drawings; the disclosure extends to any alterations, equivalents, and substitutes beyond those depicted. Terms like first, second, etc., are used for distinction and should not limit the elements described.

[0041] As input to the training, the clustering criteria indicating which ML models can form the cluster and be trained together is provided by the MnS consumer. This indicates similarities in context aspects like, but not limited to, type of learning paradigm, evaluation performance metrics such as same performance characteristics that ML models intend to achieve, task type, applicability domain, allowed cluster training time, allowed cluster inference time, preferred clustering technique, and preferred model diversity.

[0042] In an embodiment, the proposed disclosure includes the MnS consumer providing to the MnS producer in an ML Training function a set of requirements or criteria related to ML models in order to decide which ML models can be grouped or clustered together for training. The Following information is provided to facilitate formation of a cluster or group of ML models which are to be trained together:

[0043] learningType: This defines the criteria based on the learning paradigm, such as the type of learning. The ML models with the same type of learning, as indicated by the learning type, shall be grouped together for training. Its values can be one of, but not limited to, supervised learning, unsupervised learning, reinforcement learning. The learningType is of string type.

[0044] performanceMetric: This defines clustering criteria based on the performance metric for which the ML model is mainly evaluated. For instance, the models which intend to achieve the same performance characteristics as indicated by the performance metric are clustered together for training. It indicates the performance metric used to evaluate the performance of an ML model, such as but not limited to "accuracy," "precision," "F1 score," etc. The performanceMetric is of string type.

[0045] taskType: This defines grouping criteria based on the task the ML model is trained for and the set of ML models with same type of task as indicated by the task type is selected and clustered together for training. This can be one or more of, but not limited to, an aIMLInferenceName or capabilityName, The aIMLInferenceName is a type of inference that the ML model supports, wherein aIMLInferenceName comprises values of at least one of, but not limited to, MDA type (3GPP TS 28.104), Analytics ID(s) of NWDAF (3GPP TS 23.288), NG-RAN inference type (TS 38.300 and TS 38.401) or vendor's specific extensions type and wherein the capabilityName comprises name of a capability for which an ML model can generate inference , wherein the capabilityName comprises at least one of, but not limited to, traffic analysis capability, coverage analysis capability, mobility analysis capability or vendor specific extensions. The taskType is of string type.

[0046] applicabilityDomain: This defines grouping of ML models that are designed to solve similar kinds of domain problems as indicated by the applicability domain, such as, but not limited to, text analysis, and image processing. The applicabilityDomain is of type string.

[0047] allowedClusterTrainingTime: This defines the combined time limit within which the training of ML models cluster is completed. A cluster of ML models takes more time to train together as compared to the time taken for training an individual ML model. The criteria allow grouping only those ML models for training whose combined training time does not exceed the set time limit indicated by the allowed cluster training time. The allowedClusterTrainingTime is of type TimeWindow.

[0048] allowedClusterInferenceTime: This defines the required inference time of the trained model. This information indicates the max or minimum prediction time limits that the consumer expects for the predictions or inferences from the trained model. The criteria allow grouping only those ML models for training which result in a model whose inference time does not exceed the set time limit as indicated by the allowed cluster inference time. The allowedClusterInferenceTime is of type TimeWindow.

[0049] preferredClusteringTechnique: This defines the preferred clustering technique by the consumer for clustering the models for training. Examples of the clustering technique includes, but is not limited to, Bagging (bootstrap random forest), and Boosting (adaptive sequential decision trees, gradient boosting), stacking. The preferredClusteringTechnique is of type string.

[0050] preferredModelDiversity: This defines the consumer preferred model diversity types that can be considered for models clustering, such as, but not limited to, decision trees, neural networks, and linear regression. The preferredModelDiversity is of type string.

[0051] The disclosure proposes following attributes to enhance NRM in TS 28.105. The NRM enhancement for accommodating above mentioned information can be done in two ways:

[0052] NRM method 1: To provide above mentioned clustering criteria information by the MnS consumer, the MLTrainingRequest IOC is enhanced with attribute clusteringInfo containing information that identifies the clustering criteria for the ML Models. This attribute will be of type ClusteringCriteria <<dataType>>. This datatype is composed of all above mentioned attributes as the clustering criteria.

[0053] NRM method 2: The above mentioned clustering criteria information is provided as a part of ML context datatype such as expectedRuntimeContext. For this the MLContext <<dataType>> is enhanced with above mentioned attributes.

[0054] The disclosure proposes following attributes to enhance NRM for AL / ML management in 3GPP TS 28.105:

[0055] Attribute nameSupport QualifierisReadableisWritableisInvariantisNotifyableAttribute DescriptionAttribute PropertieslearningTypeMTTFTThis defines the criteria based on the learning paradigm i.e. the type of learning. ML models with same type of learning as indicated by the learning type shall is grouped together for training. Its values can be, but not limited to, supervised learning, unsupervised learning, and reinforcement learning.type: Stringmultiplicity: 1isOrdered: N / AisUnique: N / AdefaultValue: NoneisNullable: FalseperformanceMetricMTTFTThis defines clustering criteria based on the performance characteristics for which the ML model is mainly evaluated. That is, the models, which intend to achieve same performance characteristic as indicated by the performance metric is clustered together for training. It indicates the performance metric used to evaluate the performance of an ML model, such as, but not limited to, "accuracy", "precision", "F1 score".type: Stringmultiplicity: 1isOrdered: N / AisUnique: N / AdefaultValue: NoneisNullable: FalsetaskTypeMTTFTThis defines grouping criteria based on the task the ML model is trained for. This can be one or more of, but not limited to, aIMLInferenceName or capabilityName, wherein the aIMLInferenceName is a type of inference that the ML model supports, wherein aIMLInferenceName comprises values of at least one of, but not limited to, MDA type (3GPP TS 28.104), Analytics ID(s) of NWDAF (3GPP TS 23.288), NG-RAN inference type (TS 38.300 and TS 38.401) or vendor's specific extensions type and wherein the capabilityName comprises name of a capability for which an ML model can generate inference , wherein the capabilityName comprises at least one of, but not limited to, traffic analysis capability, coverage analysis capability, mobility analysis capability or vendor specific extensions.type: Stringmultiplicity: 1isOrdered: N / AisUnique: N / AdefaultValue: NoneisNullable: FalseapplicabilityDomainMTTFTThis defines grouping of ML models that are designed to solve similar kind of domain problems like, but not limited to, text analysis, image processing.type: Stringmultiplicity: 1isOrdered: N / AisUnique: N / AdefaultValue: NoneisNullable: FalseallowedClusterTrainingTimeMTTFTThis defines the combined time limit within which the training of ML models cluster is completed. A cluster of ML models takes more time to train together as compared to time taken for training an individual ML model. The criteria allows accommodating only those ML models whose combined training time does not exceed the set time limit.type: TimeWindow multiplicity: 1isOrdered: N / AisUnique: N / AdefaultValue: NoneisNullable: FalseallowedClusterInferenceTimeMTTFTThis defines the required inference time of the trained model. This information indicates the max or minimum prediction time limits that the consumer expects for the predictions or inferences from the trained model. The criteria allows grouping only those ML models for training which result in a model whose inference time does not exceed the set time limit.type: TimeWindowmultiplicity: 1isOrdered: N / AisUnique: N / AdefaultValue: NoneisNullable: TruepreferredClusteringTechniqueMTTFTThis defines the preferred technique by the consumer, for clustering the models for training. This include, but not limited to, Bagging (bootstrap, random forest), Boosting (adaptive, sequential decision trees, gradient boosting), and stacking.type: Stringmultiplicity: 1isOrdered: N / AisUnique: N / AdefaultValue: NoneisNullable: FalsepreferredModelDiversityMTTFTThis defines the consumer preferred model diversity types that is considered for models clustering. For instance, but not limited to, decision trees, neural networks, linear regression.type: Stringmultiplicity: 1isOrdered: N / AisUnique: N / AdefaultValue: NoneisNullable: False

[0056] Table 1 shows attributes to enhance NRM for AL / ML management.

[0057] Referring now to the drawings, and more particularly to FIGS. 1 through 5, there are shownpreferredembodiments.

[0058] FIG 1 illustrates exemplary hardware components of an ML training function server according to embodiments disclosed herein. The ML training function server (100) includes a processor (101), a memory (102), a communicator (103), and a cluster-based training controller (104). The components of the ML training function server (100) are operatively coupled to one another via one or more internal buses, interfaces, or interconnect mechanisms.

[0059] In an embodiments, the ML training function server (100) corresponds to or is implemented within a 3GPP-compliant management and control entity such as a gNodeB, an eNodeB, a centralized unit (CU), a distributed unit (DU), or any other 3GPP-compliant network function, network element, or managed entity capable of participating in training ML models, cluster, or an MnS producer entity. The processor (101) is configured to orchestrate ML training, including management of clustering criteria, model distribution, and aggregation of locally trained model updates.

[0060] The processor (101) executes instructions stored in the memory (102) and controls interactions among the memory (102), the communicator (103), and the cluster-based training controller (104). The processor (101) may include one or more processing units such as a Central Processing Unit (CPU), an Application Processor (AP), a Graphics Processing Unit (GPU), a Neural Processing Unit (NPU), or other hardware accelerators or any combination thereof to support scalable and latency-tolerant ML processing in a 3GPP network environment. The processor (101) and the cluster-based training controller (104) can be integrally referred to as at least one processor.

[0061] The memory (102) is configured to store an operating system, virtualization or container runtime components, application programs, configuration data, and operational data used during the execution of ML model cluster procedures. The memory (102) stores instructions and MnS producer that, when executed by the processor (101) and the cluster-based training controller (104), cause the ML Training Function Server (100) to perform one or more ML models cluster operations in compliance with 3GPP management and analytics frameworks. The memory (102) may comprise one or more volatile and / or non-volatile computer-readable storage media, including RAM, ROM, flash memory, magnetic or optical storage, EPROM, EEPROM, or any combination thereof, and may be implemented as a non-transitory computer-readable storage medium. The memory (102) may further store ML model parameters, training round information, client capability information, and ML training request messages received from one or more MnS consumer devices (200).

[0062] The communicator (103) is configured to support communication between the ML Training Function Server (100), one or more MnS producers, and one or more MnS consumer devices (200) over standardized 3GPP management interfaces. The communicator (103) enables communication via service-based interfaces (SBI) and supports one or more communication protocols, including but not limited to Hypertext Transfer Protocol (HTTP / HTTPS), Transmission Control Protocol / Internet Protocol (TCP / IP), User Datagram Protocol (UDP), and other protocols defined or referenced in 3GPP specifications. In some embodiments, the communicator (103) further supports communication over non-3GPP access networks or satellite and broadcast systems, including Digital Video Broadcasting by Satellite (DVB-S2). The communicator (103) may include one or more transceivers, network interface controllers, protocol stacks, or virtualized communication functions implemented in hardware, software, or a combination thereof.

[0063] In an embodiment, the cluster-based training controller (104) is implemented as a dedicated integrated circuit or as a hardware logic block fabricated on a semiconductor substrate within the ML Training Function Server (100). The cluster-based training controller (104) includes, for example, one or more hardware processing units, control logic circuits, state machines, registers, and scheduling logic configured to autonomously group and train a set of selected ML models together and coordinate without reliance on general-purpose software execution. The cluster-based training controller (104) is operatively coupled to the processor (101), the memory (102), and the communicator (103) via one or more hardware interfaces, buses, or interconnects. The cluster-based training controller (104) manages the training constraints as described below using the MnS producer for enabling efficient execution within a 3GPP-compliant network management architecture. The MnS producer is a management service provider entity configured to expose 3GPP MnS―including management data and analytics―via standardized 3GPP interfaces to one or more MnS consumer entities.

[0064] The cluster-based training controller (104) receives an ML training request message for clustering and training the ML models from an MnS consumer device (200), where the ML training request message includes clustering criteria in order to decide which ML models with multiple contexts belonging to the MnS producer can be clustered together for training. The clustering criteria include one or more of, but not limited to, a learning type of the ML models, a performance metric of the ML models, a task type the ML models are trained for, an applicability domain of the ML models, an allowed cluster training time of the ML models, an allowed cluster inference time of the ML models, a preferred clustering technique of the ML models, and a preferred model diversity type of the ML models. Further, the cluster-based training controller (104) sends a response message to the MnS consumer device (200) indicating the creation of a MOI and selects a set of ML models from multiple ML models that can be clustered and trained together by the ML training function server (100) based on the clustering criteria. Furthermore, the cluster-based training controller (104) forms a cluster of the set of selected ML models and trains the cluster of the set of selected ML models together.

[0065] In an embodiment, the ML training request message is part of a create MOI request message and modify MOI attributes request message. Similarly, the response message is one of a create MOI response message and a modify MOI attributes response message.

[0066] Further, the cluster-based training controller (104) generates an ML training report after completion of training of the cluster of the set of selected ML models, and the cluster-based training controller (104) sends a notification message to the MnS consumer device (200), where the notification message indicates to the MnS consumer device (200) the availability of the ML training report at the ML training function server (100).

[0067] Furthermore, the cluster-based training controller (104) selects the set of ML models from the multiple ML models that can be clustered and trained together by the ML training function server (100) based on the clustering criteria. This includes selecting the set of ML models based on at least one of the learning types of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the learning type indicates the type of learning based on which the set of ML models with the same type of learning as indicated by the learning type is selected and clustered together for training. The learning type for deciding upon clustering of ML models includes one or more of, but not limited to, supervised learning, unsupervised learning, or reinforcement learning.

[0068] The cluster-based training controller (104) selects the performance metric of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the performance metric indicates performance characteristics for which the ML model is evaluated, and the set of ML models having the same performance characteristics as indicated by the performance metric is selected and clustered together for training. The performance metric for deciding upon clustering of ML models includes one or more of, but not limited to, accuracy, precision, and F1 score.

[0069] The cluster-based training controller (104) also selects the task type indicated in the clustering criteria by the MnS consumer device (200). The task type indicates the type of tasks that the ML models are trained for, and the set of ML models with the same type of task as indicated by the task type is selected and clustered together for training. The task type for deciding upon clustering of ML models includes one or more of, but not limited to, aIMLInferenceName and capabilityName. The aIMLInferenceName is a type of inference that the ML model supports, wherein aIMLInferenceName comprises values of at least one of, but not limited to, MDA type (3GPP TS 28.104), Analytics ID(s) of NWDAF (3GPP TS 23.288), NG-RAN inference type (TS 38.300 and TS 38.401) or vendor's specific extensions type and wherein the capabilityName comprises name of a capability for which an ML model can generate inference , wherein the capabilityName comprises at least one of, but not limited to, traffic analysis capability, coverage analysis capability, mobility analysis capability or vendor specific extensions.

[0070] Further, the cluster-based training controller (104) selects the applicability domain of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the applicability domain indicates the set of ML models that solve the same domain problem as indicated by the applicability domain is selected and clustered together for training. Domain problems indicated in the applicability domain for deciding upon clustering of ML models are one or more of, but not limited to, text analysis or image processing.

[0071] The cluster-based training controller (104) also selects the allowed cluster training time of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the allowed cluster training time indicates the combined time limit within which training of the cluster shall be completed. The ML models are selected and clustered together for training whose combined training time of the ML models does not exceed a required cluster training time limit as indicated by the allowed cluster training time.

[0072] Furthermore, the cluster-based training controller (104) selects the allowed cluster inference time of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the allowed cluster inference time indicates complying with a required inference time of the trained ML models. The ML models are selected and clustered together for training only if they result in an ML model whose inference time does not exceed a required cluster inference time limit as indicated by the allowed cluster inference time.

[0073] The cluster-based training controller (104) then selects the preferred clustering technique of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the preferred clustering technique indicates a clustering technique for selecting and clustering the set of ML models for training, and the set of ML models complying with the same clustering technique as indicated by the preferred clustering technique is selected and clustered together. The preferred clustering technique for deciding upon clustering of ML models is one or more of, but not limited to, bagging, boosting, or stacking.

[0074] The cluster-based training controller (104) then after selects based on the consumer preferred model diversity type of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the consumer preferred model diversity type indicates the preferred type of model diversity for selecting and clustering the set of ML models for training, and the set of ML models complying with the same model diversity type as indicated by the preferred model diversity is selected and clustered together. The preferred model diversity types for deciding upon clustering of ML models include one or more of, but not limited to, decision trees, neural networks, or linear regression.

[0075] In an embodiment, the clustering criteria are included in an ML Training Request Information Object Class (IOC) with an attribute clusteringInfo in the ML training request message or a new IOC name contained in MLTrainingFunction IOC.

[0076] FIG. 2 is a block diagram that illustrates hardware components associated with Management Services (MnS) consumer device according to embodiments as disclosed herein. As illustrated, the MnS consumer device (200) includes at least one processor (201), a memory (202), a communicator (203), and a clustering criteria controller (204). The components of the MnS consumer device (200) are operatively coupled to one another via one or more internal buses, interfaces, or interconnects.

[0077] Examples of the MnS consumer device (200) include, but are not limited to, Non-Real-Time Radio Access Network Intelligent Controller (Non-RT RIC) and an Operations Administration and Maintenance (OAM) system. The processor (201) is configured to manage local participation in ML training, including execution of local training operations and interaction with the ML training function server (100).

[0078] The processor (201) executes instructions stored in the memory (202) and controls interactions among the memory (202), the communicator (203), and the clustering criteria controller (204). The processor (201) may comprise one or more processing units, including but not limited to a Central Processing Unit (CPU), an Application Processor (AP), a Graphics Processing Unit (GPU), a Visual Processing Unit (VPU), a Neural Processing Unit (NPU), or any combination thereof to support efficient on-device ML processing. The processor (101) and the clustering criteria controller (204) can be integrally referred to as at least one processor.

[0079] The memory (202) is configured to store an operating system, application programs, configuration parameters, and temporary or persistent data used by the processor (201) during the execution of ML model clustering. The memory (202) stores instructions that, when executed by the processor (201), cause the MnS consumer device (200) to perform one or more local ML training, inference, aggregation-preparation, or reporting operations in support of clustering criteria. The memory (202) may include one or more volatile and / or non-volatile computer-readable storage media, including RAM, ROM, flash memory, magnetic storage devices, optical storage devices, EPROM, EEPROM, or any combination thereof, and may be implemented as a non-transitory computer-readable storage medium.

[0080] The communicator (203) is configured to facilitate communication between the MnS consumer device (200) and the ML training function server (100). The communicator (203) supports communication over standardized 3GPP management interfaces, including service-based interfaces (SBI), and one or more communication protocols such as Hypertext Transfer Protocol (HTTP / HTTPS), Transmission Control Protocol / Internet Protocol (TCP / IP), User Datagram Protocol (UDP), and other protocols defined or referenced in 3GPP specifications. In some embodiments, the communicator (203) further supports communication over non-3GPP access networks or satellite and broadcast systems, including Digital Video Broadcasting by Satellite (DVB-S2). The communicator (203) may include one or more transceivers, network interface controllers, protocol stacks, or virtualized communication functions implemented in hardware, software, or a combination thereof.

[0081] In an embodiment, the clustering criteria controller (204) is implemented as a dedicated integrated circuit or as a hardware logic block fabricated on a semiconductor substrate within the MnS consumer device (200). The clustering criteria controller (204) includes one or more hardware processing units, control logic circuits, state machines, registers, and hardware schedulers configured to autonomously manage local participation without reliance on general-purpose software execution. The clustering criteria controller (204) is operatively coupled to the processor (201), the memory (202), and the communicator (203) via one or more hardware interfaces, buses, or interconnects.

[0082] In an embodiment, the clustering criteria controller (204) generates a context based clustering criteria in order to decide which ML models with multiple contexts belonging to a MnS producer can be clustered together for training, where the clustering criteria include one or more of, but not limited to, a learning type of the ML models, a performance metric of the ML models, a task type the ML models are trained for, an applicability domain of the ML models, an allowed cluster training time of the ML models, an allowed cluster inference time of the ML models, a preferred clustering technique of the ML models, and a preferred model diversity type of the ML models.

[0083] Further, the clustering criteria controller (204) sends an ML training request message for clustering and training the ML models to an ML training function server (100), where the request message includes the clustering criteria. Then the clustering criteria controller (204) receives a response message from the MnS producer in the ML training function server (100) indicating the creation of a model object instance (MOI).

[0084] In an embodiment, the ML training request message is one of a create MOI request message and modify MOI attributes request message, and where the response message is one of a create MOI response message and a modify MOI attributes response message.

[0085] Furthermore, the clustering criteria controller (204) receives a notification message from the MnS producer in the ML training function server (100), where the notification message indicates to the MnS consumer device (200) the availability of an ML training report in the ML training function server (100).

[0086] In an embodiment, the clustering criteria controller (204) generates the clustering criteria. This includes setting in the clustering criteria at least one of the learning type indicating clustering based on the type of learning of the ML models, where the ML models with the same type of learning as indicated by the learning type are selected and clustered together for training by the ML training function server (100). The learning type for deciding upon clustering of ML models includes one or more of, but not limited to, supervised learning, unsupervised learning, or reinforcement learning.

[0087] The clustering criteria controller (204) generates the performance metric indicating performance characteristics based on which the ML models that intend to achieve the same performance characteristics as indicated by the performance metric are selected and clustered together for training by the ML training function server (100). The performance metric for deciding upon clustering of ML models includes one or more of, but not limited to, accuracy values, precision values, and F1 score.

[0088] The task type indicates a type of task that the ML model is trained for, where the ML models with the same type of task as indicated by the task type are selected and clustered together for training by the ML training function server (100). The task type for deciding upon clustering of ML models includes one or more of, but not limited to, aIMLInferenceName and capabilityName. The aIMLInferenceName is a type of inference that the ML model supports, wherein aIMLInferenceName comprises values of at least one of, but not limited to, MDA type (3GPP TS 28.104), Analytics ID(s) of NWDAF (3GPP TS 23.288), NG-RAN inference type (TS 38.300 and TS 38.401) or vendor's specific extensions type and wherein the capabilityName comprises name of a capability for which an ML model can generate inference , wherein the capabilityName comprises at least one of, but not limited to traffic analysis capability, coverage analysis capability, mobility analysis capability or vendor specific extensions.

[0089] The clustering criteria controller (204) generates the applicability domain indicating the domain problem that the ML models can solve. The ML models that solve the same domain problem as indicated by the applicability domain are selected and clustered together for training by the ML training function server (100). The domain problems indicated in the applicability domain for deciding upon clustering of ML models are one or more of, but not limited to, text analysis or image processing.

[0090] Further, the clustering criteria controller (204) generates the allowed cluster training time indicating a combined time limit within which training of the cluster is to be completed. The ML models are selected and clustered together for training by the ML training function server (100) whose combined training time of the ML models does not exceed a required cluster training time limit as indicated by the allowed cluster training time.

[0091] The clustering criteria controller (204) generates the allowed cluster inference time, which indicates complying with a required inference time of the trained ML models. The ML models are selected and clustered together for training by the ML training function server (100) only if they result in an ML model whose inference time does not exceed a required cluster inference time limit as indicated by the allowed cluster inference time.

[0092] The clustering criteria controller (204) generates the preferred clustering technique indicating a clustering technique preferred by the MnS consumer device (200) for selecting and clustering the set of ML models for training, and the set of ML models complying with the same clustering technique as indicated by the preferred clustering technique are selected and clustered together for training by the ML training function server (100). The preferred clustering technique for deciding upon clustering of ML models includes one or more of, but not limited to, bagging, boosting, or stacking.

[0093] The clustering criteria controller (204) generates the preferred model diversity indicating preferred model diversity types by the MnS consumer device (200) for selecting and clustering the set of ML models for training, where the set of ML models complying with the same model diversity type as indicated by the preferred model diversity are selected and clustered together by the ML training function server (100). The preferred model diversity types for deciding upon clustering of ML models include one or more of, but not limited to, decision trees, neural networks, or linear regression.

[0094] In an embodiment, the clustering criteria are included in an ML Training Request IOC with an attribute clusteringInfo in the ML training request message or a new IOC name contained in MLTrainingFunction IOC.

[0095] FIG. 3 is a flow diagram that illustrates a proposed method implemented by a Management Services (MnS) producer in an ML training function server for training ML models cluster network according to embodiments as disclosed herein. At step 301, the method includes receiving by a Management Services (MnS) producer in an ML training function server (100) an ML training request message for clustering and training the ML models from an MnS consumer device (200), where the ML training request message includes a clustering criteria in order to decide which all ML models with multiple contexts belonging to the MnS producer can be clustered together for training. The clustering criteria include one or more of, but not limited to, a learning type of the ML models, a performance metric of the ML models, a task type the ML models are trained for, an applicability domain of the ML models, an allowed cluster training time of the ML models, an allowed cluster inference time of the ML models, a preferred clustering technique of the ML models, and a preferred model diversity type of the ML models.

[0096] At step 302, the method includes sending by the MnS producer in the ML training function server (100) a response message to the MnS consumer device (200) indicating the creation of a model object instance (MOI). At step 303, the method includes selecting by the MnS producer in the ML training function server (100) a set of ML models from multiple ML models that can be clustered and trained together by the ML training function server (100) based on the clustering criteria.

[0097] At step 304, the method includes forming by the MnS producer in the ML training function server (100) a cluster of the set of selected ML models. At step 305, the method includes training by the MnS producer in the ML training function server (100) the cluster of the set of selected ML models together.

[0098] In an embodiment, the ML training request message is one of a create MOI request message and modify MOI attributes request message, and the response message is one of a create MOI response message and a modify MOI attributes response message.

[0099] In an embodiment, the method includes generating by the MnS producer in the ML training function server (100) an ML training report after completion of training of the cluster of the set of selected ML models and sending by the MnS producer in the ML training function server (100) a notification message to the MnS consumer device (200), where the notification message indicates the MnS consumer device (200) about the availability of the ML training report at the ML training function server (100).

[0100] In an embodiment, the method includes selecting by the MnS producer in the ML training function server (100) the set of ML models from the multiple ML models that can be clustered and trained together by the ML training function server (100) based on the clustering criteria. This includes selecting by the MnS producer in the ML training function server (100) the set of ML models based on, but not limited to, the learning type of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the learning type indicates the type of learning based on which the set of ML models with the same type of learning as indicated by the learning type is selected and clustered together for training. The learning type for deciding upon clustering of ML models includes one or more of, but not limited to, supervised learning, unsupervised learning, or reinforcement learning.

[0101] The performance metric of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the performance metric indicates performance characteristics for which the ML model is evaluated, and the set of ML models having the same performance characteristics as indicated by the performance metric is selected and clustered together for training. The performance metric for deciding upon clustering of ML models includes one or more, but not limited to, accuracy, precision, and F1 score.

[0102] The task type indicated in the clustering criteria by the MnS consumer device (200), where the task type indicates the type of tasks that the ML models are trained for, and the set of ML models with the same type of task as indicated by the task type is selected and clustered together for training. The task type for deciding upon clustering of ML models includes one or more of, but not limited to, aIMLInferenceName and capabilityName. The aIMLInferenceName is a type of inference that the ML model supports, wherein aIMLInferenceName comprises values of at least one of, but not limited to, MDA type (3GPP TS 28.104), Analytics ID(s) of NWDAF (3GPP TS 23.288), NG-RAN inference type (TS 38.300 and TS 38.401) or vendor's specific extensions type and wherein the capabilityName comprises name of a capability for which an ML model can generate inference , wherein the capabilityName comprises at least one of, but not limited to, traffic analysis capability, coverage analysis capability, mobility analysis capability or vendor specific extensions.

[0103] The applicability domain of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the applicability domain indicates the set of ML models that solve the same domain problem as indicated by the applicability domain, is selected and clustered together for training, where the domain problems indicated in the applicability domain for deciding upon clustering of ML models is one or more, but not limited to, text analysis or image processing.

[0104] The allowed cluster training time of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the allowed cluster training time indicates the combined time limit within which training of the cluster shall be completed, where the ML models are selected and clustered together for training whose combined training time of the ML models does not exceed a required cluster training time limit as indicated by the allowed cluster training time.

[0105] The allowed cluster inference time of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the allowed cluster inference time indicates complying with a required inference time of the trained ML models. The ML models are selected and clustered together for training only if they result in an ML model whose inference time does not exceed a required cluster inference time limit as indicated by the allowed cluster inference time.

[0106] The preferred clustering technique of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the preferred clustering technique indicates a clustering technique for selecting and clustering the set of ML models for training, and the set of ML models complying with the same clustering technique as indicated by the preferred clustering technique is selected and clustered together. The preferred clustering technique for deciding upon clustering of ML models is one or more of, but not limited to, bagging, boosting, or stacking.

[0107] The consumer preferred model diversity type of the ML models indicated in the clustering criteria by the MnS consumer device (200), where the consumer preferred model diversity type indicates the preferred type of model diversity for selecting and clustering the set of ML models for training, and the set of ML models complying with the same model diversity type as indicated by the preferred model diversity is selected and clustered together. The preferred model diversity types for deciding upon clustering of ML models include one or more of, but not limited to, decision trees, neural networks, or linear regression.

[0108] In an embodiment, the method clustering criteria are included in an ML Training Request IOC with an attribute clusteringInfo in the ML training request message or a new IOC name contained in MLTrainingFunction IOC.

[0109] FIG 4 is a flow diagram that illustrates a proposed method implemented by the MnS consumer device for training ML models cluster according to embodiments as disclosed herein. At step 401, the method includes generating by a MnS consumer device a clustering criteria in order to decide which all ML models with multiple contexts belonging to a MnS producer can be clustered together for training, where the clustering criteria includes one or more of, but not limited to, a learning type of the ML models, a performance metric of the ML models, a task type the ML models are trained for, an applicability domain of the ML models, an allowed cluster training time of the ML models, an allowed cluster inference time of the ML models, a preferred clustering technique of the ML models, and a preferred model diversity type of the ML models.

[0110] At step 402, the method includes sending by the MnS consumer device (200) an ML training request message for clustering and training the ML models to an ML training function server (100), where the request message includes the clustering criteria. In an embodiment, the ML training request message is one of a create MOI request message and modify MOI attributes request message, and where the response message is one of a create MOI response message and a modify MOI attributes response message.

[0111] At step 403, the method includes receiving by the MnS consumer device (200) a response message from the MnS producer in the ML training function server (100) indicating creation of a model object instance (MOI). At step 404, the method includes receiving by the MnS consumer device (200) a notification message from the MnS producer in the ML training function server (100), where the notification message indicates the MnS consumer device (200) about the availability of an ML training report in the ML training function server (100).

[0112] In an embodiment, the method includes generating by the MnS consumer device (200) the clustering criteria, which includes setting in the clustering criteria at least one of the learning type indicating clustering based on the type of learning of the ML models, where the ML models with the same type of learning as indicated by the learning type are selected and clustered together for training by the ML training function server (100). The learning type for deciding upon clustering of ML models includes one or more of, but not limited to, supervised learning, unsupervised learning, or reinforcement learning. The performance metric indicating performance characteristics based on which the ML models that intend to achieve the same performance characteristics as indicated by the performance metric are selected and clustered together for training by the ML training function server (100), where the performance metric for deciding upon clustering of ML models includes one or more, but not limited to, accuracy values, precision values, and F1 score. The task type indicating a type of task that the ML model is trained for, where the ML models with the same type of task as indicated by the task type are selected and clustered together for training by the ML training function server (100). The task type for deciding upon clustering of ML models includes one or more of, but not limited to, aIMLInferenceName and capabilityName. The aIMLInferenceName is a type of inference that the ML model supports, wherein aIMLInferenceName comprises values of at least one of, but not limited to, MDA type (3GPP TS 28.104), Analytics ID(s) of NWDAF (3GPP TS 23.288), NG-RAN inference type (TS 38.300 and TS 38.401) or vendor's specific extensions type and wherein the capabilityName comprises name of a capability for which an ML model can generate inference , wherein the capabilityName comprises at least one of, but not limited to, traffic analysis capability, coverage analysis capability, mobility analysis capability or vendor specific extensions. The applicability domain indicating the domain problem that the ML models can solve, where the ML models that solve the same domain problem as indicated by the applicability domain are selected and clustered together for training by the ML training function server (100), where domain problems indicated in the applicability domain for deciding upon clustering of ML models are one or more, but not limited to, text analysis or image processing. The allowed cluster training time indicating a combined time limit within which training of the cluster shall be completed, where the ML models are selected and clustered together for training by the ML training function server (100) whose combined training time of the ML models does not exceed a required cluster training time limit as indicated by the allowed cluster training time. The allowed cluster inference time, which indicates complying with a required inference time of the trained ML models, where the ML models are selected and clustered together for training by the ML training function server (100) only if they result in an ML model whose inference time does not exceed a required cluster inference time limit as indicated by the allowed cluster inference time. The preferred clustering technique indicating a clustering technique preferred by the MnS consumer device (200) for selecting and clustering the set of ML models for training, and the set of ML models complying with the same clustering technique as indicated by the preferred clustering technique are selected and clustered together for training by the ML training function server (100), where the preferred clustering technique for deciding upon clustering of ML models is one or more, but not limited to, one of bagging, boosting, or stacking. The preferred model diversity indicating preferred model diversity types by the MnS consumer device (200) for selecting and clustering the set of ML models for training, where the set of ML models complying with the same model diversity type as indicated by the preferred model diversity shall be selected and clustered together by the ML training function server (100). The preferred model diversity types for deciding upon clustering of ML models include one or more, but not limited to, decision trees, neural networks, or linear regression.

[0113] In an embodiment, the method clustering criteria are included in an ML Training Request IOC with an attribute clusteringInfo in the ML training request message or a new IOC name contained in MLTrainingFunction IOC.

[0114] FIG 5 is a sequence diagram that illustrates an example scenario for training ML models cluster according to embodiments as disclosed herein. In an embodiment, at step 501, the MnS Consumer device (200) sends a request to MnS Producer of ML training Function for clustering and training of ML models by clustering or grouping them together as per desired context criteria. These clustering requirements include one of learningType, performanceMetric, taskType, applicabilityDomain, allowedClusterTrainingTime, allowedClusterInferenceTime, preferredClusteringTechnique, preferredModelDiversity. At step 502, the MnS Producer in ML training function sends the response of createMOI request to MnS consumer device (200). At step 503, the Producer in ML training function selects the ML models and clusters them for training together as per the requested criteria / requirements. At step 504, the MnS Producer in ML training function then performs the training of models cluster. At step 505, after completion of models training, the producer in ML training function creates an ML training report. At step 506, assuming that the MnS consumer device (200) has subscribed for the creation notification, the producer in ML training function sends a notifyMOIcreation notification to the consumer notifying about the availability of the ML Training report.

[0115] In an embodiment, the proposed disclosure introduces a new attribute for ensuring clustering of ML models for training together based on their learning type . An embodiment introduces a new attribute for ensuring clustering of ML models which intend to achieve the same performance characteristics. In an embodiment, the proposed disclosure introduces a new attribute for ensuring clustering of ML models having the same type of task, such as inference type or capability for which an ML model can generate inference. An embodiment introduces a new attribute to ensure clustering of ML models that are designed to solve similar kinds of domain problems like text analysis and image processing. In an embodiment, the proposed disclosure introduces a new attribute ensuring clustering of ML models whose combined training time does not exceed the set time limit for cluster training. An embodiment introduces a new attribute ensuring clustering of ML models based on the required inference time of the final trained model. In an embodiment, the proposed disclosure introduces a new attribute based on the preferred technique by the consumer for clustering the models for training. An embodiment introduces a new attribute based on the preferred model diversity types that can be considered for models clustering.

[0116] In an embodiment, the proposed disclosure clusters the ML models based on their learning type paradigm, performance characteristics, type of task, application domain, combined training time, required inference time, preferred clustering technique, and preferred model diversity, providing several advantages such as more efficient, robust, optimized, and high-quality ML model generation. The model becomes more robust in handling variations within the data, leading to better performance across diverse scenarios and efficient parameter sharing or gradient aggregation across the cluster. Better model selection for specific tasks with improved understanding of data patterns and efficient resource allocation. Ability to leverage strengths across different learning paradigms to create hybrid models and more focused analysis and optimization based on the problem at hand.

[0117] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and / or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modificationsshouldand are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the scope of the embodiments as described herein.

Claims

1.A method for training Machine Learning (ML) model cluster by a Management Services (MnS) producer in a ML training function server in a telecommunication network, comprising:receiving a ML training request message from a MnS consumer device, wherein the ML training request message comprises a clustering criteria comprising at least one of a learning type of ML models, a performance metric of the ML models, a task type for the ML models, an applicability domain of the ML models, an allowed cluster training time of the ML models, an allowed cluster inference time of the ML models, a preferred clustering technique of the ML models, or a preferred model diversity type of the ML models;sending a response message to the MnS consumer device indicating creation of a model object instance (MOI);selecting a set of ML models from a plurality of ML models that can be trained together by the ML training function server based on the clustering criteria;forming a cluster of the set of selected ML models; andtraining the cluster of the set of selected ML models together.2.The method of claim 1, comprising:generating a ML training report after training the cluster of the set of selected ML models; andsending a notification message to the MnS consumer device, wherein the notification message indicates the MnS consumer device about availability of the ML training report at the ML training function server.3.The method of claim 1, wherein the ML training request message is one of a create MOI request message and modify MOI attributes request message, and wherein the response message is one of a create MOI response message and a modify MOI attributes response message.4.The method of claim 1, wherein the learning type of the ML models indicates type of learning based on which the set of ML models with same type of learning as indicated by the learning type shall be selected and clustered together for training,the performance metric of the ML models indicated in the clustering criteria indicates performance characteristics for which ML model is evaluated and the set of ML models having same performance characteristics as indicated by the performance metric shall be selected and clustered together for training,the task type indicates type of tasks that the ML models are trained for and the set of ML models with same type of task as indicated by the task type shall be selected and clustered together for training,the applicability domain of the ML models indicates the set of ML models that solve the same domain problem as indicated by the applicability domain shall be selected and clustered together for training,the allowed cluster training time of the ML models indicates combined time limit within which training of the cluster shall be completed, wherein the ML models are selected and clustered together for training whose combined training time of the ML models does not exceed a required cluster training time limit as indicated by the allowed cluster training time,the allowed cluster inference time of the ML models indicates complying to a required inference time of the trained ML models, wherein the ML models are selected and clustered together for training only if they result in a ML model whose inference time does not exceed a required cluster inference time limit as indicated by the allowed cluster inference time,the preferred clustering technique of the ML models indicates a clustering technique for selecting and clustering the set of ML models for training and the set of ML models complying to same clustering technique as indicated by the preferred clustering technique shall be selected and clustered together, andthe consumer preferred model diversity type of the ML models indicates preferred type of model diversity for selecting and clustering the set of ML models for training and the set of ML models complying to same model diversity type as indicated by the preferred model diversity shall be selected and clustered together.5.The method claim of claim 1, wherein the clustering criteria is included in an ML Training Request Information Object Class (IOC) with an attribute clusteringInfo in the ML training request message or a new IOC name contained in MLTrainingFunction IOC.6.A method for training Machine Learning (ML) model cluster by a Management Services (MnS) consumer device in a telecommunication network, comprising:generating a clustering criteria for clustering ML models to train together, wherein the clustering criteria comprises at least one of a learning type of the ML models, a performance metric of the ML models, a task type for the ML models, an applicability domain of the ML models, an allowed cluster training time of the ML models, an allowed cluster inference time of the ML models, a preferred clustering technique of the ML models, and a preferred model diversity type of the ML models;sending a ML training request message comprising the clustering criteria to a ML training function server; andreceiving a response message from the MnS producer in the ML training function server (100) indicating creation of a model object instance (MOI).7.The method of claim 6, further comprising:receiving a notification message from the MnS producer in the ML training function server (100), wherein the notification message indicates the MnS consumer device (200) about availability of a ML training report in the ML training function server (100).8.The method of claim 6, wherein the ML training request message is one of a create MOI request message and modify MOI attributes request message, and wherein the response message is one of a create MOI response message and a modify MOI attributes response message.9.The method of claim 6, wherein the learning type indicates clustering based on type of learning of the ML models, wherein the ML models with same type of learning as indicated by the learning type shall be selected and clustered together for training by the ML training function server;the performance metric indicates performance characteristics based on which the ML models that intend to achieve same performance characteristics as indicated by the performance metric shall be selected and clustered together for training by the ML training function server;the task type indicates a type of task that the ML model is trained for, wherein the ML models with same type of task as indicated by the task type shall be selected and clustered together for training by the ML training function server;the applicability domain indicates domain problem that the ML models can solve, wherein the ML models that solve the same domain problem as indicated by the applicability domain shall be selected and clustered together for training by the ML training function server;the allowed cluster training time indicates a combined time limit within which training of the cluster shall be completed, wherein the ML models are selected and clustered together for training by the ML training function server whose combined training time of the ML models does not exceed a required cluster training time limit as indicated by the allowed cluster training time;the allowed cluster inference time indicates complying to a required inference time of the trained ML models, wherein the ML models are selected and clustered together for training by the ML training function server only if they result in a ML model whose inference time does not exceed a required cluster inference time limit as indicated by the allowed cluster inference time;the preferred clustering technique indicates a clustering technique preferred by the MnS consumer device for selecting and clustering the set of ML models for training, and the set of ML models complying to same clustering technique as indicated by the preferred clustering technique shall be selected and clustered together for training by the ML training function server; andthe preferred model diversity indicates preferred model diversity types by the MnS consumer device for selecting and clustering the set of ML models for training, wherein the set off ML models complying to same model diversity type as indicated by the preferred model diversity shall be selected and clustered together by the ML training function server.10.The method claim of claim 6, wherein the clustering criteria is included in an ML Training Request Information Object Class (IOC) with an attribute clusteringInfo in the ML training request message or a new IOC name contained in MLTrainingFunction IOC.11.A Machine Learning (ML) training function server including a Management Services (MnS) producer for training ML model cluster in a telecommunication network, the ML training function server comprising:memory; andat least one processor operably coupled to the memory, wherein the at least one processor is configured to:receive a ML training request message from a MnS consumer device, wherein the ML training request message comprises a clustering criteria comprising at least one of a learning type of ML models, a performance metric of the ML models, a task type for the ML models, an applicability domain of the ML models, an allowed cluster training time of the ML models, an allowed cluster inference time of the ML models, a preferred clustering technique of the ML models, or a preferred model diversity type of the ML models,send a response message to the MnS consumer device indicating creation of a model object instance (MOI),select a set of ML models from a plurality of ML models that can be trained together by the ML training function server based on the clustering criteria,form a cluster of the set of selected ML models, andtrain the cluster of the set of selected ML models together.12.The ML training function server of claim 11, wherein the at least one processor is further configured to:generate a ML training report after training the cluster of the set of selected ML models; andsend a notification message to the MnS consumer device, wherein the notification message indicates the MnS consumer device about availability of the ML training report at the ML training function server.13.The ML training function server of claim 11, wherein the ML training request message is one of a create MOI request message and modify MOI attributes request message, and wherein the response message is one of a create MOI response message and a modify MOI attributes response message.14.The ML training function server of claim 11, whereinthe learning type of the ML models indicates type of learning based on which the set of ML models with same type of learning as indicated by the learning type shall be selected and clustered together for training,the performance metric of the ML models indicates performance characteristics for which ML model is evaluated and the set of ML models having same performance characteristics as indicated by the performance metric shall be selected and clustered together for training,the task type indicates type of tasks that the ML models are trained for and the set of ML models with same type of task as indicated by the task type shall be selected and clustered together for training,the applicability domain of the ML models indicates the set of ML models that solve the same domain problem as indicated by the applicability domain shall be selected and clustered together for training,the allowed cluster training time of the ML models indicates combined time limit within which training of the cluster shall be completed, wherein the ML models are selected and clustered together for training whose combined training time of the ML models does not exceed a required cluster training time limit as indicated by the allowed cluster training time,the allowed cluster inference time of the ML models indicates complying to a required inference time of the trained ML models, wherein the ML models are selected and clustered together for training only if they result in a ML model whose inference time does not exceed a required cluster inference time limit as indicated by the allowed cluster inference time,the preferred clustering technique of the ML models indicates a clustering technique for selecting and clustering the set of ML models for training and the set of ML models complying to same clustering technique as indicated by the preferred clustering technique shall be selected and clustered together, andthe consumer preferred model diversity type of the ML models indicates preferred type of model diversity for selecting and clustering the set of ML models for training and the set of ML models complying to same model diversity type as indicated by the preferred model diversity shall be selected and clustered together.15.The ML training function server of claim 11, wherein the clustering criteria is included in an ML Training Request Information Object Class (IOC) with an attribute clusteringInfo in the ML training request message or a new IOC name contained in MLTrainingFunction IOC.

Images