Domain-aware cell similarity prediction framework for advanced transfer learning in dynamic and sparse networks

Abstract

Methods and systems for utilizing transfer learning to deal with sparse datasets in wireless networks, particularly those associated with system-level network modeling. The methods and systems are designed to overcome the limitations posed by the sparsity and dynamicity of real network data, which is often difficult to collect due to the substantial costs and potential performance degradation associated with conducting system-level experiments on large numbers of base stations.

Description

DOMAIN- AW ARE CELL SIMILARITY PREDICTION FRAMEWORK FORADVANCED TRANSFER LEARNING IN DYNAMIC AND SPARSE NETWORKSSTATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0001] Not applicableCROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 655,900, filed June 4, 2024, the disclosure of which are hereby incorporated by reference in their entirety.BACKGROUND

[0003] Al-based data-driven network models can be used for system level intelligent network planning. These models can also then be used for post-deployment optimization and zero touch automation in cellular networks by orchestrating a plethora of network parameters to maintain optimal multi-faceted network performance in terms of all-important top-level key performance indicators, without much human involvement. However, despite their immense potential, the training process of AI / ML-based network behavior models is limited by the sparsity and uneven distribution (unrepresentativeness) of real network data collected by the operators, especially for system-level network modeling. This is because operators only dedicate a few cells in the network for system-level experiments. Experimenting on a large number of base stations to gather data is very costly and can degrade the performance of the network, thus affecting the user experience. Using deep transfer learning, knowledge learned from a cell can be used to generate data in areas, where little or no data is available. However, in those extremely sparse data scenarios, the conventional deep transfer learning doesn’t perform well, resulting in inaccurate network models.

[0004] Furthermore, in real networks, the underlying network environment is continuously changing due to, e.g., user mobility, traffic, weather, vegetation, etc. This dynamicity causes a shift in the independent variable (covariate shift), a shift in the target / depen dent variable (prior probability shift), or a shift in the relationship between independent and dependent variables (concept drift) of the created network models. Due to these shifts in the distribution of network data with time, the model accuracy reduces over time. In these situations, transfer learning canhelp, as the existing model can be fine-tuned with little or no data, and the model’s performance is greatly improved in new or unforeseen situations. But the selection of similar (source cells) is challenging in a dynamic environment. The traditional similarity (distance) metrics such as KL, Jensen Shannon or Wasserstein metrics calculate the distance between the estimated data distribution of (data rich) source and (sparse) target cell (domain) (FIG. 1). However, in sparse and unrepresentative data scenarios, the data distribution can’t be accurately estimated, hence the estimated distance or similarity between different cells will also be inaccurate.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] FIG. 1 is a schematic of showing an example of using transfer learning in cellular networks for network behavior prediction to find the optimal network configuration in a target cell by transferring the learned network behavior from the source cell (with rich data) to the target cell (with scarce data).

[0006] FIG. 2 shows a schematic of the novel ML-based Domain Aware Cell SimilarityPrediction Framework to enable advanced transfer learning in a dynamic and sparse network.

[0007] FIG. 3A shows a conventional transfer learning (TL) framework.

[0008] FIG. 3B shows a disclosed Advanced TL Framework for handling extremely sparse datasets in accordance with the present disclosure.

[0009] FIG. 4 shows results of a performance comparison of the presently disclosed (‘'proposed”) TL framework with existing techniques, including source cell model (no transfer learning from data-rich source cell), XGBoost model (trained on sparse target cell data), and conventional transfer learning (source-cell DNN model fine-tuned on target cell data. Extensive transfer learning experiments show the superior performance of the presently disclosed approach against baseline approaches for predicting the mean RSRP, mean RSRQ, and mean SINR of the UEs in the footprint of the target cell, across various data sparsity levels (i.e., the available number of network experiments in target cell). The dotted line represents the mean value and the filled area / polygon around it represents the 95% confidence interval of the reported mean absolute error after 5-fold repeated cross-validation.

[0010] FIG. 5 shows the transfer learning performance (RMSE improvement) using the energy-based similarity metric.

[0011] FIG. 6 shows the transfer learning performance (RMSE improvement) using the Jenson-Shannon similarity metric.

[0012] FIG. 7 shows the transfer learning performance (RMSE improvement) using the Wasserstein similarity metric.

[0013] FIG. 8 shows the transfer learning performance (RMSE improvement) of the presently disclosed domain-aware cell similarity’ prediction model.

[0014] FIG. 9A shows the impact of using a low-similarity source cell on the transfer learning performance.

[0015] FIG. 9B shows the impact of using a high-similarity source cell on the transfer learning performance.

[0016] The following abbreviations may be used herein:Al: Artificial intelligence,COP: Configuration and optimization parameter,CQI: Channel Quality Indicator,DNN: Deep Neural Network, eNB: enhanced Node B,KPI: Key Performance Indicator,MAS: Multi-agent system,MIMO: multiple input, multiple output,ML: machine learning.ORAN: Open Radio Access Networks,RAN: Radio Access Network,RIC: RAN Intelligent Controller,RMSE: Root Mean Square Error,RSPR: Reference Signal Received Power,RSRQ: Reference Signal Received Quality.RT: Real-time,SINR: Signal to Interference plus Noise Ratio,SMO: Service Management and Orchestration,SMS: short message service,TL: transfer learning,UE: User Equipment,XGBoost: Extreme Gradient Boosting model, and5G: Fifth generation.DETAILED DESCRIPTION

[0017] The present disclosure is directed to methods for utilizing transfer learning to deal with sparse datasets in wireless networks, particularly those associated with system-levelnetwork modeling. This approach is designed to overcome the limitations posed by the sparsity and dynamicity of real network data, which is often difficult to collect due to the substantial costs and potential performance degradation associated with conducting system-level experiments on large numbers of base stations.

[0018] The technology' comprises a two-pronged approach. First, a novel method for transfer learning is disclosed that utilizes a cascade-ensemble of Deep Neural Networks and Extreme Gradient Boosting models, which significantly outperforms the conventional deep transfer learning techniques in scenarios with extremely sparse data. This method is capable of increasing models' prediction accuracy by 25% compared to baseline transfer learning if a similar amount of training data is used. The enhanced efficiency of this method also means that less data is required to achieve the same performance, leading to potential cost savings for mobile network operators.

[0019] Second, to better facilitate transfer learning in dynamic network environments, an Al-based domain-aware cell similarity' prediction model is developed. This model can accurately predict the potential gain of transfer learning between two cells before even collecting network experiment data in a new area, which can greatly assist in the planning and optimization of new macro and small cells. The model is robust to the spatio-temporal dynamics in the netyvork, taking into account various latent features such as geographical clutter pattern and cell load, user mobility, and traffic pattern. This approach can successfully measure similarity between cells deployed across different geographical environments and with different user traffic and mobility patterns, which can be particularly useful in situations yvhere the underlying network conditions change over time.

[0020] The combined impact of these tyvo novel methods is a significant enhancement of the capabilities of deep transfer learning in the domain of wireless network modeling, particularly in scenarios with sparse data. This technology represents a significant leap forward in the application of AI / ML techniques to wireless netyvorks, enabling neyv possibilities for efficient and cost-effective network management.

[0021] The presently disclosed method, a ‘"domain-aware similarity metric for TL in cellular networks," can be used, in particular, in the telecommunications industry, specifically in the context of Open Radio Access Networks (O-RAN). The O-RAN architecture is a crucial aspect of 5G and future mobile netyvork technologies, providing a frameyvork that supports multivendor interoperability, intelligence, agility, and programmability in radio access networks. In this context, the disclosed technology can be integrated into the O-RAN architecture as part of an Al-based xApp or rApp. These software plug-ins are designed toprovide functional extensibility to the Radio Access Network (RAN) and can be used to automate and optimize RAN operations at scale. The disclosed technology addresses the challenge of sparsity and dynamicity in real network data, which is crucial for the training of effective xApps and rApps. By designing an Al-based Domain-Aware Similarity Metric, the technology enables more advanced TL in Cellular Networks, improving the efficiency of training these applications and ultimately enhancing the performance of the network. In terms of specific applications within the O-RAN architecture, the technology can be utilized within the RIC and the SMO Framework. The RIC is divided into non-real-time (non-RT) and near- real-time components, both of which play a crucial role in controlling and optimizing RAN functions. The disclosed technology can be integrated into the non-RT RIC as part of an rApp, using network data, performance metrics, and subscriber data to provide Al-based recommendations for network optimization and policy guidance to xApps running on the near- RT RIC. The SMO, in turn, can use the technology to orchestrate and automate network operations at massive 5G scales, managing all network resources across different domains and clouds.

[0022] Particular users of the presently disclosed technology are mobile network operators, who can leverage it to lower their total cost of ownership and enhance their customers' quality of experience. Moreover, third-party7developers can use the technology7to develop more efficient and effective xApps and rApps. promoting innovation and competitiveness within the industry. In the O-RAN architecture, the RIC serves as a critical component, responsible for controlling and optimizing RAN functions, incorporating AI / ML into its decision-making functionalities. It can be adapted to specific latency or control loop requirements, providing an open hosting platform for applications developed by third-party7specialist software providers, known as "xApps" and "rApps." Overall, the presently disclosed technology presents a significant advancement in the field of Al-based network management, enabling the way cellular networks operate to be revolutionized, paving the way for more efficient, flexible, and high-performance 5G and beyond networks.

[0023] In particular, the presently disclosed technology7addresses several key challenges in the current state of wireless communication networks, particularly in the context of 5G and beyond.(1) Efficiency and Accuracy

[0024] Conventional DNN-based TL techniques have certain limitations, particularly in terms of prediction accuracy and data requirements for training. The present method, which isa unique blend of DNN and XGBoost models, significantly improves upon these aspects. Experimental results suggest that this novel method yield about a 25% increase in prediction accuracy over the baseline DNN-based TL techniques, given a similar amount of training data. This is a significant and substantial improvement, considering the high stakes of prediction accuracy in network operations and the subsequent impact on Quality of Service (QoS) and user experience.(2) Data Requirements

[0025] Furthermore, the present method offers a distinct advantage in terms of data economy. The same level of prediction performance can be achieved with less training data compared to the baseline method. This implies that mobile network operators can attain potential savings by reducing the need for extensive network experiments.(3) Robust Similarity Metric

[0026] Another key novelty of the present technology’ lies in the development of a robust domain-knowledge-based similarity metnc. This 44-dimensional ML-based metric is designed to be independent of the spatial and temporal changes or dynamics in the data to be transferred (such as eNB configuration and optimization parameters), enabling the identification of similarity among cells across different geographical, user traffic, and user mobility patterns. The presently’ disclosed similarity metric outperforms cunent distance and divergence metrics, and is capable of predicting the gain achievable with transfer learning between two eNBs with a normalized accuracy of ±6%.(4) Exploration across Data Sparsity Conditions

[0027] Extensive TL experiments were performed herein under various data sparsity conditions to draw insights into designing an optimal TL model based on the available data in the target domain. This strategic exploration is particularly useful in designing such TL systems in the absence of ground truth data.

[0028] Thus, compared to the existing commercial products or services, the presently disclosed technology brings a level of sophistication, accuracy, and efficiency heretofor unseen. It disrupts the status quo by not just improving the prediction performance, but also by drastically reducing the data requirements, thereby delivering substantial cost savings to mobile network operators. This combination of improved accuracy, reduced data requirement, robust similarity’ metric, and the ability to customize and optimize TL strategies makes the technologynovel, non-obvious, rare, and useful, thereby making it a significant leap over the current state of the art.

[0029] Before describing various embodiments of the present disclosure in more detail by way of exemplary description, examples, and results, it is to be understood that the present disclosure is not limited in application to the details of methods as set forth in the following description. The present disclosure is capable of other embodiments or of being practiced or carried out in various ways. As such, the language used herein is intended to be given the broadest possible scope and meaning; and the embodiments are meant to be exemplary, not exhaustive. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting unless otherwise indicated as so. Moreover, in the following detailed description, numerous specific details are set forth to provide a more thorough understanding of the disclosure. However, it will be apparent to a person having ordinary skill in the art that the embodiments of the present disclosure may be practiced without these specific details. In other instances, features which are well known to persons of ordinary skill in the art have not been described in detail to avoid unnecessary complication of the description.

[0030] Unless otherwise defined herein, scientific and technical terms used in connection with the present disclosure shall have the meanings that are commonly understood by those having ordinary skill in the art. Further, unless otherwise required by context, singular terms shall include pluralities and plural terms shall include the singular.

[0031] All patents, published patent applications, and non-patent publications mentioned in the specification, particularly U.S. Provisional Patent Application Serial No. 63 / 655,900, filed June 4, 2024, and FIGS. 1, 2, 3 and 9 and associated descriptions (e.g., ^|

[0018] -

[0056] , and ** 10084| -

[0089] ) from United States Publication No. 2024 / 0056842, having a publication date of February 15, 2024), are expressly incorporated herein by reference in their entireties and are indicative of the level of skill of those skilled in the art to which the present disclosure pertains. All patents, provisional applications, published patent applications, and non-patent publications referenced in any portion of this application, are herein expressly incorporated by reference in their entirety to the same extent as if each individual patent or publication was specifically and individually indicated to be incorporated by reference.

[0032] As utilized in accordance with the methods and compositions of the present disclosure, the following terms, unless otherwise indicated, shall be understood to have the following meanings:

[0033] The use of the word “a” or "an" when used in conjunction with the term “comprising’7in the claims and / or the specification may mean “one,” but it is also consistent with the meaning of “one or more,” “at least one,” and “one or more than one.” The use of the term “or” in the claims is used to mean “and / or” unless explicitly indicated to refer to alternatives only or when the alternatives are mutually exclusive, although the disclosure supports a definition that refers to only alternatives and “and / or.” The use of the term “at least one” will be understood to include one as well as any quantity more than one, including but not limited to, 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 20, 30, 40, 50, 100, or any integer inclusive therein. The term “at least one” may extend up to 100 or 1000 or more, depending on the term to which it is attached; in addition, the quantities of 100 / 1000 are not to be considered limiting, as higher limits may also produce satisfactory results. In addition, the use of the term “at least one of X, Y and Z” will be understood to include X alone, Y alone, and Z alone, as well as any combination of X, Y and Z. The term “plurality” gerally refers to two or more items. Unless otherwise required by context, singular terms shall include pluralities and plural terms shall include the singular. Where used herein, the specific term “single” is limited to only “one.” and a “pair” means two.

[0034] As used herein, all numerical values or ranges include fractions of the values and integers within such ranges and fractions of the integers within such ranges unless the context clearly indicates otherwise. Thus, to illustrate, reference to a numerical range, such as 1-10 includes 1, 2. 3, 4, 5. 6. 7, 8, 9. 10, as well as 1.1, 1.2, 1.3, 1.4. 1.5. etc., and so forth. Reference to a range of 1 -50 therefore includes 1 , 2, 3, 4, 5, 6, 7, 8, 9, 10, 1 1 , 12, 13, 14, 15, 16, 17, 18, 19, 20, etc., up to and including 50, as well as 1.1, 1.2, 1.3, 1.4, 1.5, etc., 2.1, 2.2, 2.3, 2.4, 2.5, etc., and so forth. Reference to a series of ranges includes ranges which combine the values of the boundaries of different ranges within the series. Thus, to illustrate reference to a series of ranges, for example, of 1-10, 10-20, 20-30, 30-40, 40-50, 50-60, 60-75, 75-100, 100-150, 150- 200, 200-250, 250-300, 300-400, 400-500, 500-750, 750-1,000, includes ranges of 1-20, 10- 50, 50-100, 100-500, and 500-1,000, for example. Thus a reference to degrees such as 1 to 90 is intended to explicitly include all degrees in the range.

[0035] As noted above, any numerical range listed or described herein is intended to include, implicitly or explicitly, any number or sub-range within the range, particularly all integers, including the end points, and is to be considered as having been so stated. For example, "a range from 1.0 to 10.0" is to be read as indicating each possible number, including integers and fractions, along the continuum between and including 1.0 and 10.0, that is. having a minimum value equal to or greater than 1.0 and a maximum value equal to or less than 10.0,such as, for example, 3.25 to 8.65. Any maximum numerical limitation recited herein is intended to include all lower numerical limitations subsumed therein, and any minimum numerical limitation recited in this specification is intended to include all higher numerical limitations subsumed therein. Accordingly, Applicant reserves the right to amend this specification, including the claims, to expressly recite any sub-range subsumed within the ranges expressly recited herein. Thus, even if a particular data point within the range is not explicitly identified or specifically referred to. it is to be understood that any data points within the range are to be considered to have been specified, and that the inventor(s) possessed knowledge of the entire range and the points within the range.

[0036] As used herein, the words “comprising” (and any form of comprising, such as “comprise” and “comprises”), “having” (and any form of having, such as “have” and “has”), “including” (and any form of including, such as “includes” and “include”) or “containing” (and any form of containing, such as “contains” and “contain”) are inclusive or open-ended and do not exclude additional, unrecited elements or method steps.

[0037] The term “or combinations thereof as used herein refers to all permutations and combinations of the listed items preceding the term. For example, “A. B, C. or combinations thereof’ is intended to include at least one of: A, B, C, AB, AC, BC, or ABC, and if order is important in a particular context, also BA, CA, CB, CBA, BCA, ACB, BAC, or CAB. Continuing with this example, expressly included are combinations that contain repeats of one or more item or term, such as BB, AAA. AAB, BBC, AAABCCCC, CBBAAA, CABABB, and so forth. The skilled artisan will understand that typically there is no limit on the number of items or terms in any combination, unless otherwise apparent from the context.

[0038] Throughout this application, the terms “about” or “approximately” are used to indicate that a value includes an inherent variation. As used herein the qualifiers “about” or “approximately” are intended to include not only the exact value, amount, degree, dimension, measurement, orientation, event, circumstance, parameter, or other qualified characteristic, but are intended to include some slight variations due to measuring error, manufacturing tolerances, observer error, and combinations thereof, for example. The term “about" or “approximately”, where used herein when referring to a measurable value such as an amount, a temporal duration, and the like, is meant to encompass, for example, variations of ± 20% or ± 10%, or ± 5%, or ± 1 %, or ± 0. 1 % from the specified value, as such variations are appropriate to perform the disclosed methods and as understood by persons having ordinary skill in the art. As used herein, the term “substantially” means that the subsequently described value, amount, degree, dimension, measurement, orientation, event, circumstance or parameter, or otherqualified characteristic completely occurs, or occurs to a great extent or degree. For example, the term ■■substantially” means that the subsequently described value, amount, degree, dimension, measurement, orientation, event, circumstance, or parameter or other qualified characteristic occurs at least 80% of the time, at least 90% of the time, at least 91% of the time, at least 92% of the time, at least 93% of the time, at least 94% of the time, at least 95% of the time, at least 96% of the time, at least 97% of the time, at least 98% of the time, or at least 99% of the time.

[0039] As used herein any reference to "one embodiment" or "an embodiment" means that a particular element, feature, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0040] As used herein any reference to "we" as a pronoun herein refers generally to assistants or other contributors who assisted in data collection and manipulation and is not intended to represent an inventorship role by said assistants or other contributors in any subject matter disclosed herein.

[0041] Returning now to the description of non-limiting embodiments of the disclosure, to address the challenge of selecting suitable transfer candidates (similar cells) in a dynamic and unrepresentative environment, the present disclosure sets forth a novel Al-based domain-aware cell similarity prediction model that is robust to the spatio-temporal dynamics in the network (FIG. 2). By leveraging latent features such as geographical clutter pattern which can act as a proxy for spatial dynamics, and cell load, user mobility and traffic pattern etc., which can act as a proxy for temporal dynamics in the network, the disclosed framework can measure similarity between cells deployed across different geographical environments and with different user traffic and mobility patterns. The disclosed novel set of latent features can be computed for each cell using already collected network data (e.g., cell trace data, geographical clutter data, cell performance management KPI / counter data, etc.). This data is readily available to the operators, without the need for further standardization. Furthermore, by doing extensive transfer learning experiments between different combinations of selected data rich cells, a 44-dimensional ML-based similarity metric is trained. This similarity metnc takes as an input the existing network environment data of the source and target cell, and accurately predicts the similarity (and thus potential gain that can be achieved with transfer learning) between two cells, before even collecting network experiment data in a new area. This can greatly help in Day 0 and Day 1 planning for deploying new macro and small cells and for optimizing existing sites. This is because by using the disclosed metric the operators canmeasure the similarity between the new area (target cell) and the existing pool of candidate source cells. It can help determine which cell can most accurately represent the target cell environment for creating network behavior models through transfer learning, thus handling the spatial dynamicity challenge. Similarly, even if the underlying network conditions change over time, operators can accurately measure the cell similarity at any given time and fine-tune the model (using a new source cell based on updated network conditions), thus handling the temporal dynamicity challenge.

[0042] The parameters that were considered for the domain-aware similarity metric are delineated in Table 1, and are categorized into five groups. These include: (1) the percentage of an eNB's coverage area belonging to a certain geographical clutter, (2) the percentage of User Equipments (UEs) displaying a specific traffic pattern, (3) UEs' MIMO adoption percentage, (4) ratio of mobile users, and proportions of UEs demonstrating a particular mobility pattern, and (5) the pattern of UEs' distribution within a cell. The percentages in the sub-columns of the last column represent the proportion of certain features within each subgroup for a given eNB, and thus the sum of all features in a group should equate to 100%.

[0043] In preparing the data for training the domain-aware cell similarity prediction model, TL experiments are executed using network parameter data, and the error of each source cell's performance on the target cell data is computed. The cell similarity prediction model is formed by training an XGBoost model utilizing the proposed 44-dimensional features as input (with 22 features each from source and target cells). The target variable (output) values of the prediction model are derived from testing the source cell model on a target cell. This Root Mean Square Error (RMSE) is scaled between 0 and 1, where 1 denotes the minimum prediction error between a source and target cell, thus indicating similar cells. Conversely, a 0 represents the maximum prediction error, indicative of entirely dissimilar cells.Table 1 - Description of 22 features for each eNB in the network and an example of data types collected for each of the 32 eNBs analyzed.

[0044] It's noteworthy to mention that the input features of this proposed model are derived from auxiliary contextual data readily available to operators for all cells in the network. However, the respective output data is procured from network experiment data (COP-KPI), which is not ubiquitously available and is sparse and unrepresentative. As such, operators can train this cell similarity prediction model utilizing data only from cells where network experiment data is available, and correlate it with the readily available auxiliary data (based on geographical, cell counter, and cell trace data).

[0045] Upon conducting TL experiments for the considered 32-cell system, a total of 1024 possible TL experiments were undertaken, and each experiment's result corresponds to the target variable of one data instance in the resultant database. A trained XGBoost model based on this input-output data can predict the similarity and potential transfer learning gain using only the auxiliary features of a new set of source and target cells not included in the training process.

[0046] Furthermore, a novel method for transfer learning (as compared to state-of-the-art DNN based) is also disclosed which uses a boosting cascade-ensemble of DNN s and XGBoost models (FIG. 3, lower panel). The disclosed transfer learning framework combines DNN and XGBoost models in a boosting cascade ensemble. This approach is distinct from state-of-the- art deep TL, which simply fine-tunes the source cell model from a data-rich cell. Instead, theerrors of the DNN-based source cell model are calculated using a small amount of ground truth data in the target cell and training an ML-based XGBoost model to learn the error behavior. This error model captures where the source cell model fails and augments it in those instances. At inference time, the output from both models is added to give the final prediction, which is helpful in sparse training data scenarios, as the error model does not need to learn the entire network behavior, requiring significantly less training data in the target cell.

[0047] A conventional TL framework (FIG. 3, upper panel) is partitioned into the training phase on the left and the inference phase on the right. In the training phase, a DNN model is trained on the copious data available from the source cell to learn the network model. The antenna configuration parameters - transmit power, antenna tilt, and antenna height - are utilized as input features, while the KPIs, including RSRP, RSRQ, and SINR, function as the output labels of the network model. Following the training, the model is fine-tuned with limited data from the target cell, involving retraining the final layers of the source model and updating the weights. This fine-tuned DNN model is then applied during the inference phase to predict the target cell's KPIs based on the input parameters of transmit power, antenna tilt, and antenna height.

[0048] DNNs are recognized for their high parameter count, necessitating substantial data for effective training. Even in TL applications, the fine-tuning phase requires significant data, albeit less than that required for the initial training of the source model. Therefore, the TL approach may not yield optimal results when dealing with extremely sparse datasets at the target cell, leading to insufficient parameter updates during fine-tuning and sub-optimal model performance.

[0049] To enhance the performance of TL models fine-tuned with sparse datasets, the presently disclosed system uses a hybrid boosted cascade ensemble TL framework (FIG. 3, lower panel). This approach amalgamates the strengths of DNNs and XGBoost to enhance the accuracy of network behavior predictions. Like the conventional TL framework, a pre-trained DNN from a source cell is transferred to the target cell and fine-tuned using an extremely sparse target dataset. However, the fine-tuned model may yield inaccurate predictions due to the paucity of data. To increase prediction accuracy, we supplement the fine-tuned model with an XGBoost model, trained to learn and correct the residual error present in predictions from the fine-tuned model.

[0050] During the training phase of the present framework, two models are generated: the target cell DNN and the XGBoost model. The XGBoost model is trained to discern the relationship between the COPs as input features and the error in KPI prediction as the modeloutput. In the inference phase, for each input combination of COPs, the XGBoost model produces a residual signal, which is added to the KPI predictions of the fine-tuned model to enhance prediction accuracy.

[0051] Training an ML model to learn the residual error and correct the output of the finetuned model has a two-fold rationale. First, ML models typically possess fewer parameters than DNNs, enabling them to be effectively trained using the extremely sparse data available, thus improving the overall performance of the framework. Second, the residual learning approach is particularly beneficial in extremely sparse training data scenarios, as the error model need not learn the entire network behavior, but rather the deficiencies of the original source model, thus requiring significantly less training data in the target cell.

[0052] Below, the presently disclosed advanced TL framework is compared to existing techniques, the TL strategy (namely, the number of re-trained layers) and its impact on overall system performance, and also the performance of the disclosed similarity prediction model to find suitable transfer candidates for TL.Simulation Setup:

[0053] A ray-tracing-based radio network planning and optimization platform, ’’Atoll”, is utilized to create a large network topology, consisting of 272 base stations and 65,000 Poisson distributed UEs in the city of Brussels, Belgium (see U.S. Provisional Patent Application Serial No. 63 / 655,900, FIG. 4). Atoll simulations are based on the Aster propagation model, which employs advanced ray-tracing techniques to calculate various phenomena that impact radio wave propagation, such as vertical diffraction over roof-tops, horizontal diffraction and reflections based on ray-launching, atmospheric absorption, rain attenuation and vegetation loss, among others. Additionally, the Aster propagation model parameters are pre-calibrated using more than 1.5 million channel measurements from the real environment. For accurate simulations and performance modeling, actual antenna heights and antenna radiation patterns are utilized. The simulation settings utilized in our analysis, which include system, geographic and eNB parameters, are detailed in Table 2. Based on the sophisticated propagation modeling and network setting considerations, data calculated by Atoll is taken as ground truth for designing network behavior (COP-KPI) models.Table 2 - Network Scenario SettingsResults & Discussion:

[0054] The comparison of the error performance of the presently disclosed scheme with three different schemes is shown for all KPIs in FIG. 4. The comparing scheme is the Source Cell Model representing the no transfer situation, Baseline XGBoost, Conventional TL using the DNN only, and the presently disclosed TL scheme using the DNN and residual error-based boosting model. The conventional and presently disclosed schemes are trained by training the only last layer of the model. Each sub-figure is dedicated to the respective KPI, as their titles show. These comparisons are made with respect to the increased density of the number of experiments in the target cell, as shown on the x-axis of the figures.

[0055] These results show that the MAS of all TL schemes decreases with respect to the increasing number of experiments, which highlights the better performance of TL with increased target data density. The non-varying performance of the non-transfer case is also observed in all the comparisons, and this case is also having the worst performance of all comparing schemes, justifying the use of transfer learning and proving the positive gainsachieved by the transfers. Comparing the TL schemes shows that the XGBoost performs worst in all cases than the conventional DNN and presently disclosed scheme because the TL inherently works by training some layers of deep learning models unavailable in XGBoost. The improved performance of the presently disclosed scheme is also observed in all KPIs cases, when compared with the conventional DNN-based scheme, because of the positive gain achieved by the residual error-based boosting.

[0056] The performance of the similarity metrics is evaluated by comparing the TL gain of each source to the target transfer with respect to the similarity of the corresponding metric (FIGS. 5-8). Each dot in these results shows the TL gain of a source to target transfer experiment, and overall 1089 experiments are carried out. The solid line in the center shows the regression fitting of all 1089 points, and the shaded region around this line shows the 95% confidence interval. Finally, the external dotted lines show the 95% prediction interval of the fitted regression curve.

[0057] Each subfigure depicts the abovementioned aspects of TL gain with respect to the corresponding similarity metric, i.e., Wasserstein, Jensen Shannon, Energy Based and Proposed ML -based metric. In all these results, we aimed to establish if there is any correlation between the TL gain and the respective similarity adopted to achieve that gain. Comparing all four results for this criteria, show that only the presently disclosed ML-based similarity metric exhibits the linear trend between similarity and TL gain. Furthermore, it is worth noting that as similarity between source and target cell becomes very high, transfer learning between them yields negative transfer, i.e., the error in the source cell model increases. This insight was further investigated. Results are shown in FIGS. 9A and 9B.

[0058] In summary, the presently disclosed system uses a novel, robust, and scalable domain-knowledge-based similarity metric. This 44-dimensional ML-based metric contains latent features independent of the spatial and temporal changes / dynamics in the data to be transferred. Compared to existing distance and divergence metrics, the proposed metric predicts potential gains from TL between two eNBs with an impressive normalized accuracy of ±6%. A novel method for TL by using a boosting cascade ensemble of DNN s and XGBoost models is described. This approach is estimated to increase the models' prediction accuracy by 25% compared to traditional DNN-based TL, given a similar amount of training data. This could also mean less data is required to achieve the same performance, potentially leading to significant cost savings for mobile network operators. Extensive TL experiments and analysis across various data sparsity conditions are presented. This in-depth examination resuled in the design of a TL model based on the available data in the target domain, providing a usefulstrategy for TL system design in the absence of ground truth data. These methods provide substantial improvement over conventional approaches, particularly in situations with sparse and unrepresentative data in the target environment.

[0059] In at least certain embodiments, the present disclosure is directed to a non- transitory computer readable medium storing processor executable logic that when executed by a processing unit of a system cause the system to identify target cell parameters for enhancing performance of a target cell in a wireless telecommunication network by executing a monitoring program configured to monitor target cell parameters and target cell key performance indicators (KPIs) of the target cell, and by executing multiple machine learning (ML) models based upon source cell configuration parameters and source cell KPIs of a source cell, wherein the multiple ML models are executed as at least one App associated with the wireless telecommunication network; wherein a first ML model among the multiple ML models is trained with the configuration parameters and KPIs of the source cell to generate predicted values of target cell KPIs upon receipt of target cell parameters; wherein the monitoring program calculates error residuals between the target cell KPIs and the predicted values of the target cell KPIs from the first ML model; and wherein a second ML model among the multiple ML models is trained with the target cell configuration parameters and the error residuals to generate predicted error residuals of target cell KPIs upon receipt of the target cell parameters; and perform an action based upon the predicted values of target cell KPIs and the predicted error residuals of target cell KPIs. The action may be configuring at least one target cell parameter of the target cell. The at least one target cell parameter is selected from a group comprising a target cell configuration parameter or a target cell optimization parameter. The action may be generating a report based upon the predicted values of target cell KPIs and the predicted error residuals of target cell KPIs. The processor executable logic that when executed by a processing unit of the system may cause the system to sum the predicted values of target cell KPIs generated by the first ML model with the predicted error residuals of target cell KPIs to generate a prediction. The source cell configuration parameters may be selected from the group comprising transmit power, antenna tilt, antenna height, antenna azimuth, handover parameters, and combinations thereof. The source cell KPIs of the source cell may be selected from the group comprising Reference Signal Received Power (RSRP), Reference Signal Received Quality (RSRQ), and Signal to Interference plus Noise Ratio (SINR) and combinations thereof. The first ML model may also trained with target cell parameters and target cell KPIs. The at least one App may comprise at least one xApp or rApp in an Open Radio Access Network. 1

[0060] In at least certain embodiments, the present disclosure is directed to a non- transitory computer readable medium storing processor executable logic that when executed by a processing unit of a system cause the system to identify a suitable source cell from a plurality of source cells for enhancing performance of a target cell in a wireless telecommunication network by identifying similarities between source cell contextual features and target cell contextual features, wherein the plurality of source cells are individually modeled resulting in a plurality of first ML models, with each of the first ML models being trained with configuration parameters and key performance indicators (KPIs) of a particular source cell of the plurality of source cells whereby after training each of the first ML models generates first predicted values of KPIs upon receipt of configuration parameters; for particular first ML models trained for particular source cells, generate a plurality of error values by providing configuration parameters of source cells other than the source cell upon which the first ML models were trained to generate second predicted values of KPIs and then calculating the error values between the second predicted values of KPIs and respective ground truths; wherein a second ML model among the multiple ML models is trained with cell level contextual features of the source cells and the error values to generate similarity scores upon receipt of source cell contextual features and target cell contextual features; provide source cell context features from various ones of the source cells and target cell contextual features of a target cell to identify similarities between particular ones of the source cells and the target cell using the second ML model; and perform an action based upon the identified similarities. The action may be configuring at least one target cell parameter of the target cell. The at least one target cell parameter may be selected from a group comprising a target cell configuration parameter or a target cell optimization parameter. The action may be generating a report based upon the identified similarities between various ones of the source cells and the target cell.

[0061] In at least certain embodiments, the present disclosure is directed to a method of identifying target cell parameters for enhancing performance of a target cell in a wireless telecommunication network by executing a monitoring program configured to monitor target cell parameters and target cell key performance indicators (KPIs) of the target cell, and by executing multiple machine learning (ML) models based upon source cell configuration parameters and source cell KPIs of a source cell, wherein the multiple ML models are executed as at least one App associated with the wireless telecommunication network; wherein a first ML model among the multiple ML models is trained with the configuration parameters and KPIs of the source cell to generate predicted values of target cell KPIs upon receipt of target cell parameters; wherein the monitoring program calculates error residuals between the targetcell KPIs and the predicted values of the target cell KPIs from the first ML model; and wherein a second ML model among the multiple ML models is trained with the target cell configuration parameters and the error residuals to generate predicted error residuals of target cell KPIs upon receipt of the target cell parameters; and performing an action based upon the predicted values of target cell KPIs and the predicted error residuals of target cell KPIs. Performing the action may be defined further as configuring at least one target cell parameter of the target cell. The at least one target cell parameter may be selected from a group comprising a target cell configuration parameter or a target cell optimization parameter. Performing the action may be defined further as generating a report based upon the predicted values of target cell KPIs and the predicted error residuals of target cell KPIs. Before the step of performing an action, the method may further comprise summing the predicted values of target cell KPIs generated by the first ML model with the predicted error residuals of target cell KPIs to generate a prediction. The source cell configuration parameters may be selected from the group comprising transmit power, antenna tilt, antenna height, antenna azimuth, handover parameters, and combinations thereof. The source cell KPIs of the source cell may be selected from the group comprising Reference Signal Received Power (RSRP), Reference Signal Received Quality (RSRQ), and Signal to Interference plus Noise Ratio (SINR) and combinations thereof. The first ML model may be trained with target cell parameters and target cell KPIs. The at least one App may comprise at least one xApp or rApp in an Open Radio Access Network.

[0062] In at least certain embodiments, the present disclosure is directed to a method of identifying a suitable source cell from a plurality of source cells for enhancing performance of a target cell in a wireless telecommunication network by identifying similarities between source cell contextual features and target cell contextual features, wherein the plurality of source cells are individually modeled resulting in a plurality of first ML models, with each of the first ML models being trained with configuration parameters and key performance indicators (KPIs) of a particular source cell of the plurality of source cells whereby after training each of the first ML models generates first predicted values of KPIs upon receipt of configuration parameters; for particular first ML models trained for particular source cells, generate a plurality of error values by providing configuration parameters of source cells other than the source cell upon which the first ML models were trained to generate second predicted values of KPIs and then calculating the error values between the second predicted values of KPIs and respective ground truths; wherein a second ML model among the multiple ML models is trained with cell level contextual features of the source cells and the error values to generate similarity scores upon receipt of source cell contextual features and target cellcontextual features; providing source cell context features from various ones of the source cells and target cell contextual features of a target cell to identify similarities between particular ones of the source cells and the target cell using the second ML model; and performing an action based upon the identified similarities. Performing an action may be defined further as configuring at least one target cell parameter of the target cell. The at least one target cell parameter may be selected from a group comprising a target cell configuration parameter or a target cell optimization parameter. Performing an action may be defined further as generating a report based upon the identified similarities between various ones of the source cells and the target cell.

[0063] While the present disclosure has been described in comiection with certain embodiments so that aspects thereof may be more fully understood and appreciated, it is not intended that the present disclosure be limited to these particular embodiments. On the contrary, it is intended that all alternatives, modifications and equivalents are included within the scope of the present disclosure. Thus the examples described above, which include particular embodiments, will serve to illustrate the practice of the present disclosure, it being understood that the particulars shown are by way of example and for purposes of illustrative discussion of particular embodiments only and are presented in the cause of providing what is believed to be the most useful and readily understood description of procedures as well as of the principles and conceptual aspects of the presently disclosed methods. Changes may be made in various features, aspects, and components of the frameworks and analytical tools described herein, or the methods described herein without departing from the spirit and scope of the present disclosure.

Claims

What is claimed is:

1. A non-transitory computer readable medium storing processor executable logic that when executed by a processing unit of a system cause the system to: identify target cell parameters for enhancing performance of a target cell in a wireless telecommunication network by executing a monitoring program configured to monitor target cell parameters and target cell key performance indicators (KPIs) of the target cell, and by executing multiple machine learning (ML) models based upon source cell configuration parameters and source cell KPIs of a source cell, wherein the multiple ML models are executed as at least one App associated with the wireless telecommunication network; wherein a first ML model among the multiple ML models is trained with the configuration parameters and KPIs of the source cell to generate predicted values of target cell KPIs upon receipt of target cell parameters; wherein the monitoring program calculates error residuals between the target cell KPIs and the predicted values of the target cell KPIs from the first ML model; and wherein a second ML model among the multiple ML models is trained with the target cell configuration parameters and the error residuals to generate predicted error residuals of target cell KPIs upon receipt of the target cell parameters; and perform an action based upon the predicted values of target cell KPIs and the predicted error residuals of target cell KPIs.

2. The non-transitory computer readable medium of claim 1, wherein the action is configuring at least one target cell parameter of the target cell.

3. The non-transitory computer readable medium of claim 1, wherein the at least one target cell parameter is selected from a group comprising a target cell configuration parameter or a target cell optimization parameter.

4. The non-transitory computer readable medium of claim 1, wherein the action is generating a report based upon the predicted values of target cell KPIs and the predicted error residuals of target cell KPIs.

5. The non-transitory computer readable medium of claim 1, wherein the processor executable logic that when executed by a processing unit of the system causes the system to sum the predicted values of target cell KPIs generated by the first ML model with the predicted error residuals of target cell KPIs to generate a prediction.

6. The non-transitory computer readable medium of claim 1, wherein the source cell configuration parameters are selected from the group comprising transmit power, antenna tilt, antenna height, antenna azimuth, handover parameters, and combinations thereof.

7. The non-transitory computer readable medium of claim 1 , wherein the source cell KPIs of the source cell are selected from the group comprising Reference Signal Received Power (RSRP), Reference Signal Received Quality (RSRQ), and Signal to Interference plus Noise Ratio (SINR) and combinations thereof.

8. The non-transitory computer readable medium of claim 1, wherein the first ML model is also trained with target cell parameters and target cell KPIs.

9. The non-transitory computer readable medium of claim 1, wherein the at least one App comprises at least one xApp or rApp in an Open Radio Access Network.

10. A non-transitory computer readable medium storing processor executable logic that when executed by a processing unit of a system cause the system to: identify a suitable source cell from a plurality of source cells for enhancing performance of a target cell in a wireless telecommunication network by identifying similarities between source cell contextual features and target cell contextual features. wherein the plurality of source cells are individually modeled resulting in a plurality of first ML models, with each of the first ML models being trained with configuration parameters and key performance indicators (KPIs) of a particular source cell of the plurality of source cells whereby after training each of the first ML models generates first predicted values of KPIs upon receipt of configuration parameters; for particular first ML models trained for particular source cells, generate a plurality of error values by providing configuration parameters of source cells other than the source cell upon which the first ML models were trained to generate second predicted values of KPIs and then calculating the error values between the second predicted values of KPIs and respective ground truths; wherein a second ML model among the multiple ML models is trained with cell level contextual features of the source cells and the error values to generate similarity scores upon receipt of source cell contextual features and target cell contextual features; provide source cell context features from various ones of the source cells and target cell contextual features of a target cell to identify similarities between particular ones of the source cells and the target cell using the second ML model; and perform an action based upon the identified similarities.

11. The non-transitory computer readable medium of claim 10, wherein the action is configuring at least one target cell parameter of the target cell.

12. The non-transitory computer readable medium of claim 10, wherein the at least one target cell parameter is selected from a group comprising a target cell configuration parameter or a target cell optimization parameter.

13. The non-transitory computer readable medium of claim 10, wherein the action is generating a report based upon the identified similarities between various ones of the source cells and the target cell.

14. A method, comprising: identifying target cell parameters for enhancing performance of a target cell in a wireless telecommunication network by executing a monitoring program configured to monitor target cell parameters and target cell key performance indicators (KPIs) of the target cell, and by executing multiple machine learning (ML) models based upon source cell configuration parameters and source cell KPIs of a source cell, wherein the multiple ML models are executed as at least one App associated with the wireless telecommunication network; wherein a first ML model among the multiple ML models is trained with the configuration parameters and KPIs of the source cell to generate predicted values of target cell KPIs upon receipt of target cell parameters; wherein the monitoring program calculates error residuals between the target cell KPIs and the predicted values of the target cell KPIs from the first ML model; and wherein a second ML model among the multiple ML models is trained with the target cell configuration parameters and the error residuals to generate predicted error residuals of target cell KPIs upon receipt of the target cell parameters; and performing an action based upon the predicted values of target cell KPIs and the predicted error residuals of target cell KPIs.

15. The method of claim 14, wherein performing the action is defined further as configuring at least one target cell parameter of the target cell.

16. The method of claim 14. wherein the at least one target cell parameter is selected from a group comprising a target cell configuration parameter or a target cell optimization parameter.

17. The method of claim 14, wherein performing the action is defined further as generating a report based upon the predicted values of target cell KPIs and the predicted error residuals of target cell KPIs.

18. The method of claim 14, wherein before the step of performing an action, the method further comprises summing the predicted values of target cell KPIs generated by the first ML model with the predicted error residuals of target cell KPIs to generate a prediction.

19. The method of claim 14, wherein the source cell configuration parameters are selected from the group comprising transmit power, antenna tilt, antenna height, antenna azimuth, handover parameters, and combinations thereof.

20. The method of claim 14, wherein the source cell KPIs of the source cell are selected from the group comprising Reference Signal Received Power (RSRP), Reference Signal Received Quality (RSRQ), and Signal to Interference plus Noise Ratio (SINR) and combinations thereof.

Images