First node, second node, computer system and methods performed thereby, for handling hyperparameters corresponding to a plurality of sets of data

Abstract

A first node (111) obtains (302) two or more first sets of data out of a plurality of sets collected by third nodes (113). Each third node has collected a set. A plurality of clusters have been determined. Each of the two or more first sets of data corresponds to a respective set of data in a center of a respective cluster. The plurality of clusters have been determined based on a similarity of respective statistical features of the sets of data. A number of the two or more first sets of data is smaller than a second number of the plurality of sets. The first node (111) determines (303) and tunes a respective hyperparameter for each obtained two or more first sets of data to train a respective machine learning model with a corresponding set of data of the plurality of sets of data and outputs (305) an indication indicating the hyperparameters.

Description

[0001] FIRST NODE, SECOND NODE, COMPUTER SYSTEM AND METHODS PERFORMED THEREBY, FOR HANDLING HYPERPARAMETERS CORRESPONDING TO A PLURALITY OF SETS OF DATA

[0002] TECHNICAL FIELD

[0003] The present disclosure relates generally to a first node and methods performed thereby for handling hyperparameters corresponding to a plurality of sets of data. The present disclosure further relates generally to a second node and methods performed thereby, for handling hyperparameters corresponding to a plurality of sets of data. The present disclosure also relates generally to a computer system and methods performed thereby, for handling hyperparameters corresponding to a plurality of sets of data.

[0004] BACKGROUND

[0005] Computer systems in a communications network or communications system may comprise one or more nodes. A node may comprise one or more processors which, together with computer program code may perform different functions and actions, a memory, a receiving port, and a sending port. A node may be, for example, a server. Nodes may perform their functions entirely on the cloud.

[0006] Computer systems may be comprised in a telecommunications network. The telecommunications network, communications network, or wireless communications network, may cover a geographical area which may be divided into cell areas, each cell area being served by a type of node, a network node in the Radio Access Network (RAN), radio network node or Transmission Point (TP), for example, an access node such as a Base Station (BS), e.g., a Radio Base Station (RBS), which sometimes may be referred to as e.g., gNB, evolved Node B (“eNB”), “eNodeB”, “NodeB”, “B node”, or Base Transceiver Station (BTS), depending on the technology and terminology used. The base stations may be of different classes such as e.g., Wide Area Base Stations, Medium Range Base Stations, Local Area Base Stations and Home Base Stations, based on transmission power and thereby also cell size. A cell may be understood to be the geographical area where radio coverage may be provided by the base station at a base station site. One base station, situated on the base station site, may serve one or several cells. Further, each base station may support one or several communication technologies. The telecommunications network may also comprise network nodes which may serve receiving nodes, such as user equipments or devices, with serving beams.

[0007] A radio access technology (RAT) may be understood as the connection method for a radio communications network. The RAT used may depend on the type of network nodes or devices involved in the communication. The standardization organization Third Generation Partnership Project (3GPP) is currently in the process of specifying a New Radio Interface called Next Generation Radio or New Radio (NR), as well as a Fifth Generation (5G) Packet Core Network, which may be referred to as 5G Core Network (5GC). The advantages of 5G NR may include higher bandwidth, more resources, low latency and network slicing. 5G may provide services to various applications, such as enhanced Mobile Broad Band (eMBB), machine to Machine type communication (mMTC), Ultra Reliable Low Latency Communication (URLLC), etc.

[0008] 5G may be understood to bring in sizeable flexibility with technological advancements along with innovations of cloud and Al. This may be understood to bring a whole new set of opportunities in the enterprise segment.

[0009] Artificial Intelligence (Al) and Machine Learning (ML)

[0010] Al is considered as an enabler of enhancements in the future generation network, e.g., 6G, and may be regarded as an enabler of a new level of adaptivity to customize radio systems to diverse radio environments. This may be understood to be especially important given the growing complexity in RANs with each new generation. The learning capabilities of Al may be understood to create advantageous policy or strategies directly based on data, instead of human logics and symbolic modelling and analysis.

[0011] Machine learning (ML) may be understood as the study of computer algorithms that may improve automatically through experience. It is seen as a part of Al. ML algorithms may build a model based on sample data, known as "training data", in order to make predictions or decisions without being explicitly programmed to do so. ML algorithms may be used in a wide variety of applications, such as email filtering and computer vision, where it may be difficult or unfeasible to develop conventional algorithms to perform the needed tasks.

[0012] AI / ML enabled methods may be understood to employ data-driven learning approaches where the models may learn the underlying data distribution and relationships between the inputs and outputs without the need for understanding the inherent complex processes. AI / ML enabled methods may be understood to mainly rely on statistical techniques.

[0013] In ML, there may be basically three types of ML Algorithms: Supervised Learning, Unsupervised Learning, and Reinforcement Learning (RL).

[0014] Supervised Learning algorithms may comprise a target / outcome variable, or dependent variable, which may have to be predicted from a given set of predictors, that is, independent variables. Using this set of variables, a function may be generated that may map inputs to desired outputs. The training process may continue until the model may achieve a desired level of accuracy on the training data. Once an ML model may have been trained, an inference process may begin, whereby new data may be run through the ML model to calculate an output. Examples of Supervised Learning may be Regression, Decision Tree, Random Forest, KNN, Logistic Regression etc. In Unsupervised Learning algorithms, there may be no target or outcome variable to predict / estimate. It may be used for clustering a population into different groups, which may be widely used for segmenting customers in different groups for specific intervention. Examples of Unsupervised Learning may be K-means, mean-shift clustering, Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Expectation-Maximization (EM) Clustering using Gaussian Mixture Models (GMM), Agglomerative Hierarchical Clustering, etc....

[0015] Cluster analysis or clustering may be understood as an ML technique which may comprise grouping a set of objects in such a way that objects in the same group, which may be called a cluster, may be understood to be more similar, in some sense, to each other than to those in other groups, that is, other clusters. It may be understood as a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including pattern recognition, image analysis, information retrieval, bioinformatics, data compression, computer graphics and ML.

[0016] Using an RL algorithm, a machine may be trained to make specific decisions. It may be understood to work as follows: the machine may be exposed to an environment where it may train itself continually using trial and error. This machine may learn from past experience and may try to capture the best possible knowledge to make accurate decisions. An example of RL may be a Markov Decision Process (MDP). The training using RL may comprise generating an ML model. To train such an ML model, an agent, given a state of the environment, may take an action in this environment and receive a reward. The action may result in a new state of the environment. This process may be repeated in a loop. Over time, the agent may learn to take actions that may result in larger immediate and future rewards, meaning that it may be understood to be in the best interest of the agent not to take the action that may only lead to the highest reward in the next state, but the action that may cumulatively lead to the highest reward in the next state and in a future number of states.

[0017] The agent may comprise a neural network which may input the state and may produce an action. There may be several ML algorithms that may be used for training the network of the agent, e.g., policy-learning based, such as actor-critic approaches, or value-based learning, such as deep-q networks.

[0018] Hyperparameter tuning (HPT) is one of the crucial parts of each machine learning process. A hyperparameter may be understood as a set of variables that may control a learning progress as well as a performance of a machine learning model. HPT may be understood as the process of choosing the right, that is, optimal, value for the hyperparameters of a model, based on the system objective. Although various methods have been developed to optimize the hyperparameters for each ML algorithm during HPT, the complexity and efforts limit their applicability and encourage engineers to use the default value that may be typically suggested by the tool developer. However, as the optimal hyperparameters of each machine learning algorithm may highly depend on the characteristics of each data set, setting a default value may reduce the algorithm performance. The inherent tradeoff between setting a default value or running an HPT to set the optimal value has been a subject of many studies such as [1-9], In [1], a simple optimization strategy has been developed to generate a new default value which may maintain the balance between tuning and a using a default value. In [2], a method is described to select a default value from a finite set of possibilities based on k dataset. The nearest neighbor sequential model-free optimization is used to assess the performance of the default value on the test dataset. In [3], meta learning is used to find the symbolic default hyperparameter setting that may be suitable across multiple datasets. The symbolic hyperparameter may depend on characteristics of the dataset rather than a static value, such as the number of features for each dataset. In [4], the space search was reduced to find the default value for hyperparameters on a new dataset. A set of complementary default values, learned from a large database of prior empirical results was proposed as an alternative. In [5], an iteratively based method to automatically find the hyperparameters for a large scale dataset was proposed. This method started with a small training set and a set of hyperparameters was randomly chosen and the best one was found within that set. The dataset was gradually increased until the hyperparameters converged to a particular value. In [6], a weight sharing algorithm in a federated learning architecture was proposed to find one global HP. Similar to another approach, the global HPs were optimized in [7] using a different search method. In [8], Hyperparameter Optimization (HPO) of a federated learning model (xgboost) was focused on in order to find one global HP for the model.

[0019] Existing methods for HPT may limit the performance and applicability of the algorithms.

[0020] SUMMARY

[0021] As part of the development of embodiments herein, one or more problems with the existing technology will first be identified and discussed.

[0022] Hyperparameter tuning is inherently an overwhelming task which becomes more complex when having multiple data sets. Methods have been proposed to find a default value among all datasets. Existing methods for HPT focus on different ways of finding “one HP” over distributed / centralized data sets. Some of these approaches are not able to provide more than one HP, and some of them need to revisit their architecture to be able to provide more that HPs. In fact, with the current form, they are not capable of providing a set of HPs. However, existing methods still cannot solve the following challenges.

[0023] One challenge is that existing approaches suggest only one default value for all datasets. As hyperparameters highly depend on the dataset characteristics, there is an inherent tradeoff between setting a default hyperparameter and optimizing it based on each dataset in terms of complexity and model performance. In many use cases in the telecommunications domain, datasets, for example cell coverage data and cell load data, may be similar to each other or vary a lot, depending on location of the node that is collected from. Therefore, assigning one default value for all datasets limits the performance and applicability of the algorithms.

[0024] Another challenge is that most of these studies propose centralized approaches that depend on having all datasets in one place. This becomes even more challenging when data sets are available in different locations and hence, it may not be possible to merge all of them. This issue becomes even more relevant when a telecommunications feature such as ML based secondary carrier prediction is implemented. This feature comes with a default hyperparameter which may limit the performance, depending on the deployment scenario. Moreover, it is very complicated to collect and merge multiple data sets due to practical limitations, such as capacity limitation, data collection, distance, etc. In existing approaches, due to resource limitations and multitude number of ML models on a single node, it may not always be possible to fine tune the parameters while the default parameter may not be the best parameter. This degrades the network performance.

[0025] An additional challenge is that existing approaches are not addressing adaptability of the algorithm. In many scenarios, the ML algorithm HPs may need to be adapted, including the following situations: a) when the network may experience a change, one of the examples of such a scenario, may be secondary carrier prediction, where the feature may be understood to be very dependent on the network deployment and neighbor cells, and b) when network performance criteria or ML performance criteria may change. For instance, it may happen that accuracy in one scenario and precision in another scenario may be of interest. Changing these KPIs, may change the optimal HPs for the solution.

[0026] Furthermore, in existing approaches, when a network feature is to be implemented in the network, it comes with only one HP as a default parameter.

[0027] In all of these situations, current approaches may need to redo HPT.

[0028] Embodiments herein may address the problems of the existing methods just described.

[0029] According to a first aspect of embodiments herein, the object is achieved by a computer- implemented method, performed by a first node. The method is for handling hyperparameters corresponding to a plurality of sets of data. The first node operates in a computer system. The first node obtains two or more first sets of data, out of the plurality of sets of data. The plurality of sets of data has been collected by a plurality of third nodes operating in the computer system. Each third node in the plurality of third nodes has collected a set of data of the plurality of sets of data. A plurality of clusters of sets of data have been determined, to be formed by the plurality of sets of data. Each of the two or more first sets of data corresponds to a respective set of data in a center of a respective cluster of the plurality of clusters. The plurality of clusters have been determined based on a similarity of respective statistical features of the sets of data in the plurality of sets of data. A first number of the two or more first sets of data is smaller than a second number of sets of data in the plurality of sets of data. The first node determines a respective hyperparameter for each of the obtained two or more first sets of data and tunes the determined respective hyperparameters. The respective tuned hyperparameters are to train a respective machine learning model with a corresponding set of data of the plurality of sets of data. The first node outputs an indication, indicating the determined two or more respective tuned hyperparameters.

[0030] According to a second aspect of embodiments herein, the object is achieved by a computer-implemented method, performed by a second node. The method is for handling hyperparameters corresponding to the plurality of sets of data. The second node operates in the computer system. The second node obtains one or more respective statistical features corresponding to a set of data having been collected by a fourth node operating in the computer system. The second node determines, using a first machine learning model, a first cluster of sets of data corresponding to the obtained one or more respective statistical features. The determining is based on a first similarity of the one or more respective statistical features to respective statistical features of the sets of data in the determined first cluster. The first cluster is one of the plurality of clusters of sets of data formed by the plurality of sets of data having been collected by the plurality of third nodes operating in the computer system. Each third node in the plurality of third nodes has collected the set of data of the plurality of sets of data. The plurality of clusters have been determined using the first machine learning model based on the similarity of the respective statistical features of the sets of data in the plurality of sets of data. The second node provides, to the fourth node, a respective tuned hyperparameter corresponding to a respective first set of data in a first center of the determined first cluster.

[0031] According to a third aspect of embodiments herein, the object is achieved by a computer-implemented method, performed by the computer system. The method is for handling the hyperparameters corresponding to the plurality of sets of data. The computer system comprises the first node and the second node. The computer system, obtains, by the first node, the two or more first sets of data, out of the plurality of sets of data. The plurality of sets of data have been collected by the plurality of third nodes operating in the computer system. Each third node in the plurality of third nodes have collected a set of data of the plurality of sets of data. The plurality of clusters of sets of data have been determined to be formed by the plurality of sets of data. Each of the two or more first sets of data corresponds to a respective set of data in the center of the respective cluster of the plurality of clusters. The plurality of clusters have been determined based on the similarity of respective statistical features of the sets of data in the plurality of sets of data. The first number of the two or more first sets of data is smaller than the second number of sets of data in the plurality of sets of data. The computer system determines, by the first node, the respective hyperparameter for each obtained two or more first sets of data and tunes the determined respective hyperparameters. The respective tuned hyperparameters are to train the respective second machine learning model with the corresponding set of data of the plurality of sets of data. The computer system outputs, by the first node the second indication indicating the determined two or more respective tuned hyperparameters to the second node. The computer system receives, by the second node, the second indication from the first node. The computer system obtains, by the second node, the one or more respective statistical features corresponding to the set of data having been collected by the fourth node operating in the computer system. The computer system determines, by the second node, using the first machine learning model, the first cluster of sets of data corresponding to the obtained one or more respective statistical features. The determining is based on the first similarity of the one or more respective statistical features to the respective statistical features of the sets of data in the determined first cluster. The first cluster is one of the plurality of clusters of sets of data formed by the plurality of sets of data. The computer system provides, by the second node and based on the received second indication, to the fourth node, the respective tuned hyperparameter corresponding to the respective first set of data in the first center of the determined first cluster.

[0032] According to a fourth aspect of embodiments herein, the object is achieved by the first node. The first node may be understood to be for handling the hyperparameters corresponding to the plurality of sets of data. The first node is configured to operate in the computer system. The first node is configured to obtain the two or more first sets of data, out of the plurality of sets of data. The plurality of sets of data is configured to have been collected by the plurality of third nodes configured to operate in the computer system. Each third node in the plurality of third nodes is configured to have collected a set of data of the plurality of sets of data. The plurality of clusters of sets of data is configured to have been determined to be formed by the plurality of sets of data- Each of the two or more first sets of data is configured to correspond to a respective set of data in the center of the respective cluster of the plurality of clusters. The plurality of clusters are configured to have been determined based on the similarity of respective statistical features of the sets of data in the plurality of sets of data. The first number of the two or more first sets of data is configured to be smaller than the second number of sets of data in the plurality of sets of data. The first node is also configured to determine the respective hyperparameter for each of the two or more first sets of data configured to be obtained and tune the determined respective hyperparameters. The respective tuned hyperparameters are configured be to train the respective machine learning model, that is, the respective second machine learning model, with the corresponding set of data of the plurality of sets of data. The first node is further configured to output the indication configured to indicate the two or more respective tuned hyperparameters configured to be determined.

[0033] According to a fifth aspect of embodiments herein, the object is achieved by the second node. The second node may be understood to be for handling the hyperparameters corresponding to the plurality of sets of data. The second node is configured to operate in the computer system. The second node is configured to obtain the one or more respective statistical features corresponding to the set of data configured to have been collected by the fourth node configured to operate in the computer system. The second node is also configured to determine, using the first machine learning model, the first cluster of sets of data corresponding to the one or more respective statistical features configured to be obtained, based on the first similarity of the one or more respective statistical features to the respective statistical features of the sets of data in the first cluster configured to be determined. The first cluster is one of the plurality of clusters of sets of data formed by the plurality of sets of data configured to have been collected by the plurality of third nodes configured to operate in the computer system. Each third node in the plurality of third nodes is configured to have collected the set of data of the plurality of sets of data. The plurality of clusters are configured to have been determined using the first machine learning model based on the similarity of the respective statistical features of the sets of data in the plurality of sets of data. The second node is further configured to provide, to the fourth node, the respective tuned hyperparameter corresponding to the respective first set of data in the first center of the first cluster configured to be determined.

[0034] According to a sixth aspect of embodiments herein, the object is achieved by the computer system. The computer system is for handling the hyperparameters corresponding to the plurality of sets of data. The computer system is configured to comprise the first node and the second node. The computer system is configured to obtain, by the first node, the two or more first sets of data, out of the plurality of sets of data. The plurality of sets of data is configured to have been collected by the plurality of third nodes configured to operate in the computer system. Each third node in the plurality of third nodes is configured to have collected the set of data of the plurality of sets of data. The plurality of clusters of sets of data is configured to have been determined to be formed by the plurality of sets of data. Each of the two or more first sets of data corresponds to the respective set of data in the center of the respective cluster of the plurality of clusters. The plurality of clusters are configured to have been determined based on the similarity of the respective statistical features of the sets of data in the plurality of sets of data. The first number of the two or more first sets of data is configured to be smaller than the second number of sets of data in the plurality of sets of data. The computer system is also configured to determine, by the first node, the respective hyperparameter for each of the two or more first sets of data configured to be obtained and tune the determined respective hyperparameters. The respective tuned hyperparameters are configured to train the respective second machine learning model with the corresponding set of data of the plurality of sets of data. The computer system is further configured to output, by the first node, the second indication configured to indicate the two or more respective tuned hyperparameters configured to be determined to the second node. The computer system is further configured to receive, by the second node, the second indication from the first node. The computer system is additionally configured to obtain, by the second node, the one or more respective statistical features corresponding to the set of data configured to have been collected by the fourth node configured to operate in the computer system. The computer system is also configured to determine, by the second node, using the first machine learning model, the first cluster of sets of data. The first cluster of sets of data corresponds to the one or more respective statistical features configured to have been obtained, based on the first similarity of the one or more respective statistical features to the respective statistical features of the sets of data in the first cluster configured to be determined. The first cluster is configured to be one of the plurality of clusters of sets of data formed by the plurality of sets of data. The computer system is further configured to provide, by the second node 12 and based on the second indication configured to be received, to the fourth node, the respective tuned hyperparameter corresponding to the respective first set of data in the first center of the first cluster configured to be determined.

[0035] By obtaining the two or more first sets of data, wherein each of the two or more first sets of data corresponds to a respective set of data in a center of a respective cluster of the plurality of clusters determined to be formed by the plurality of sets of data based on the similarity of the respective statistical features of the sets of data, the first node is then enabled to determine and tune the respective tuned hyperparameters only for each of the obtained two or more first sets of data. Since the first number of the two or more first sets of data is smaller than the second number of sets of data in the plurality of sets of data, the first node may be advantageously enabled to refrain from having to determine and tune hyperparameters for all the clusters, and especially for all the sets of data.

[0036] By then outputting the indication indicating the determined two or more respective tuned hyperparameters, the first node enables the tuned hyperparameters to be later distributed by the second node to all the third nodes having collected the sets of data in the same cluster.

[0037] The fine-tuned parameters determined by the first node may then be used as a default parameter for the second ML-model to be used either for each data set, or new data sets. When a new data set may be available, such as that obtained from the fourth node, instead of fine tuning for the new data set, the method of embodiments herein may enable to associate it to one of the pre tuned hyperparameters and hence the hyperparameter tuning may no longer be needed.

[0038] Embodiments herein may be understood to provide a framework wherein HPT may no longer need to be performed for each cluster. Rather, a set of HPs may be used by the different features. This may be understood to enable to save a lot of resources and time when activating a new feature on a third node, since the parameter tuning time may be enabled to be decreased due to the availability of the set of HP.

[0039] Embodiments herein may enable to avoid redoing unnecessary HPT for data sets that show similar characteristics to those of earlier data sets. This approach may be understood to increase the lifetime of the HPs and to save many resources due a smaller number of HPT processes, which may be time and resource consuming. This capability may be enabled due to providing multiple HPs, which may increase the chance of a new data set to be similar to at least one of the already determined HP clusters.

[0040] BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Examples of embodiments herein are described in more detail with reference to the accompanying drawings, according to the following description.

[0042] Figure 1 is a schematic diagram illustrating two non-limiting examples, in panels a) and b), of a computer system, according to embodiments herein.

[0043] Figure 2 is a flowchart depicting a method in a first node, according to embodiments herein. Figure 3 is a flowchart depicting a method in a second node, according to embodiments herein.

[0044] Figure 4 is a flowchart depicting a method in a computer system, according to embodiments herein.

[0045] Figure 5 is a schematic diagram depicting a non-limiting example of aspects of a method performed by the computer system, according to embodiments herein.

[0046] Figure 6 is a schematic diagram depicting another non-limiting example of aspects of a method performed by the computer system, according to embodiments herein.

[0047] Figure 7 is a schematic diagram depicting a further non-limiting example of aspects of a method performed by the computer system, according to embodiments.

[0048] Figure 8 is a flowchart depicting another non-limiting example of a method in a computer system, according to embodiments herein.

[0049] Figure 9 is a schematic block diagram illustrating an example of a computer system, according to embodiments herein.

[0050] Figure 10 is a schematic block diagram illustrating an embodiment of a first node, according to embodiments herein. Figure 11 is a schematic block diagram illustrating an embodiment of a second node, according to embodiments herein.

[0051] Figure 12 is a schematic block diagram illustrating an embodiment of a computer system, according to embodiments herein.

[0052] DETAILED DESCRIPTION

[0053] Certain aspects of the present disclosure and their embodiments address the challenges identified in the Background and Summary sections with the existing methods and provide solutions to the challenges discussed.

[0054] Embodiments herein may be understood to relate to a method for hyperparameter selection for ML models with multiple datasets.

[0055] According to embodiments herein, an ML algorithm may be used to find the best default values for hyperparameters of ML models when having more than one dataset.

[0056] Embodiments herein may be understood to consider the tradeoff between HPT and choosing multiple default values and allowing an engineer to decide on a number of default values based on the available resources. In a real network, dataset gathering from near cells may have similar characteristics leading to the same default value for their hyperparameters. On the other hand, cells in different regions may show different behavior requiring different hyperparameters sets. Embodiments herein attempt to find the best default parameter for a group of similar datasets based on the dataset characteristics. In this condition, the number of default values for hyperparameters may be understood to highly depend on a number of dataset categories. It is assumed that there are n number of separate datasets, which correspond to n sets of optimal hyperparameters, a method according to embodiments herein may be able to suggest another best m sets of hyperparameters where m«n, such that the performance of each model may not be degraded significantly. There is an inherent tradeoff between the performance of each model and the number of allowed sets for hyperparameter selection. The closer m is to n, the overall performance of models may be understood to be better. However, the price for deployment of a higher number of hyperparameters sets increases.

[0057] Some of the embodiments contemplated will now be described more fully hereinafter with reference to the accompanying drawings, in which examples are shown. In this section, the embodiments herein will be illustrated in more detail by a number of exemplary embodiments. Other embodiments, however, are contained within the scope of the subject matter disclosed herein. The disclosed subject matter should not be construed as limited to only the embodiments set forth herein; rather, these embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art. Several embodiments and examples are comprised herein. It should be noted that the exemplary embodiments herein are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments and / or examples.

[0058] Figure 1 depicts two non-limiting examples, in panels “a” and “b”, respectively, of a computer system 100, in which embodiments herein may be implemented. In some example implementations, such as that depicted in the non-limiting examples of Figure 1 a), the computer system 100 may be a computer network. In other example implementations, the computer system 100 may be implemented in a communications system 101, that is, a telecommunications system, sometimes also referred to as a cellular radio system, cellular network or wireless communications system. In some examples, the communications system 101 may comprise network nodes which may serve receiving nodes, such as wireless devices, with serving beams.

[0059] In some examples, the communications system 101 may be, for example, a communications network, such as 5G system, or Next Gen network. The communications system 101 may also, or alternatively, support other technologies, such as LTE, e.g., LTE Frequency Division Duplex (FDD), LTE Time Division Duplex (TDD), LTE Half-Duplex Frequency Division Duplex (HD-FDD), LTE operating in an unlicensed band, Wideband Code Division Multiple Access (WCDMA), Universal Terrestrial Radio Access (UTRA) TDD, Global System for Mobile communication (GSM) / Enhanced Data Rate for GSM Evolution (EDGE) Radio Access Network (GERAN) network, Ultra-Mobile Broadband (UMB), EDGE network, network comprising of any combination of Radio Access Technologies (RATs) such as e.g., Multi-Standard Radio (MSR) base stations, multi-RAT base stations etc., any 3rd Generation Partnership Project (3GPP) cellular network, Wireless Local Area Network / s (WLAN) or WiFi network / s, Worldwide Interoperability for Microwave Access (WiMax), Internet of Things (loT), Machine Type Communication (MTC), IEEE 802.15.4-based low-power short-range networks such as IPv6 over Low-Power Wireless Personal Area Networks (6LowPAN), Zigbee, Z-Wave, Bluetooth Low Energy (BLE), or any cellular network or system. In some examples, the communication system 101 may be an Open Radio Access Network (O-RAN)

[0060] The computer system 100 comprises nodes, whereof a first node 111, a second node

[0061] 112 and a plurality of third nodes 113 are depicted in Figure 1. The plurality of third nodes

[0062] 113 may comprise, in some embodiments, a fourth node 114. In other examples, the plurality of third nodes 113 may additionally comprise a fifth node 115, as depicted in the non-limiting examples of Figure 1. Yet in other examples, the plurality of third nodes 113 may comprise additional nodes. In some examples, not depicted in Figure 1 , the first node 111 and the second node 112 may be co-located or be the same node. The computer system 100 may comprise additional nodes.

[0063] Any of the first node 111 , the second node 112 and the plurality of third nodes 113, such as for example, the fourth node 114, may be understood, respectively, as a first computer system or server, a second computer system or server, and a plurality of third computer systems or servers. Any of the first node 111 , the second node 112 and the plurality of third nodes 113, such as for example, the fourth node 114 may be implemented as a standalone server in e.g., a host computer in the cloud 117, as depicted in the non-limiting example of Figure 1b). In some examples, any of the first node 111 , the second node 112 and the plurality of third nodes 113, such as for example, the fourth node 114 may be a distributed node or distributed server, such as a virtual node in the cloud 117, and may perform some of its respective functions locally, e.g., by a client manager, and some of its functions in the cloud 117, by e.g., a server manager. In other examples, any of the first node 111 , the second node 112 and the plurality of third nodes 113, such as for example, the fourth node 114 may perform its functions entirely on the cloud 117, or partially, in collaboration or collocated with a radio network node. Yet in other examples, any of the first node 111 , the second node 112 and the plurality of third nodes 113, such as for example, the fourth node 114 may also be implemented as processing resources in a server farm. In yet other examples, any of the first node 111, the second node 112 and the plurality of third nodes 113, such as for example, the fourth node 114 may be comprised the edge. Any of the first node 111 , the second node 112 and the plurality of third nodes 113, such as for example, the fourth node 114 may be under the ownership or control of a service provider or may be operated by the service provider, or on behalf of the service provider.

[0064] The first node 111 may be understood to be a node that may have a capability to perform HPT. The first node 111 may manage or run, for example, a tuning unit.

[0065] The second node 112 may be understood to be a node that may have a capability to process data a perform clustering of data sets. The second node 112 may be, for example, a data processing unit. In particular examples wherein the communications system 101 may be implemented in O-RAN, the second node 112 may be a System Management and Orchestration (SMO) platform.

[0066] Any of the third nodes in the plurality of third nodes 113 may be understood to have a capability of collecting sets of data from the computer system 100, e.g., from the communications system 101.

[0067] Any of the first node 111, the second node 112 and the one or more third nodes 113 may have a capability to perform machine-implemented learning procedures, which may be also referred to as “machine learning” (ML). Any of the first node 111 , the second node 112 and the one or more third nodes 113 may have a capability to manage an artificial neural network. The artificial neural network may be understood as a machine learning framework, which may comprise a collection of connected nodes, where in each node or perceptron, there may be an elementary decision unit. Each such node may have one or more inputs and an output. The input to a node may be from the output of another node or from a data source. Each of the nodes and connections may have certain weights or parameters associated with it. In order to solve a decision task, the weights may be learnt or optimized over a data set which may be representative of the decision task. The most commonly used node may have each input separately weighted, and the sum may be passed through a non-linear function which may be known as an activation function. The nature of the connections and the node may determine the type of the neural network, for example a feedforward network, recurrent neural network etc. That any of the first node 111 , the second node 112 and the one or more third nodes 113 may have the capability to manage the artificial neural network may be understood herein as having the capability to store the training data set and the models that may result from the machine learning, to train a new model, and once the model may have been trained, to use this model for prediction. In some embodiments, the system that may be used for training the model and the one used for prediction may be different.

[0068] Any of the first node 111 , the second node 112 and the one or more third nodes 113, used for training the artificial neural network, may require more computational resources than the first node 111 that may use the trained model to make predictions. Therefore, any of the first node 111 , the second node 112 and the one or more third nodes 113, used for training the artificial neural network may, for example, support running python / Java with Tensorflow or Pytorch, Theano etc... Any of the first node 111, the second node 112 and the one or more third nodes 113 may also have GPU capabilities.

[0069] In some embodiments, any of the first node 111 , the second node 112 and the one or more third nodes 113 may be a core network node, such as, e.g., a network data analytics function (NWDAF), a Service management and orchestration (SMO) node, a positioning node, a coordinating node, a Self-Optimizing / Organizing Network (SON) node, a Minimization of Drive Test (MDT) node, etc... . In 5G, for example, any of the first node 111 and the second node 112 may be located in the Operations Support Systems (OSS).

[0070] In other examples, any of the first node 111 , the second node 112 and the one or more third nodes 113 may be a radio network node, as depicted in Figure 1 for the fourth node 114 and the fifth node 115. That is, a transmission point such as a radio base station, for example a gNB, an eNB, or any other network node with similar features capable of serving a wireless device, such as a user equipment or a machine type communication device, in the communications system 101. In an O-RAN architecture, each radio network node may be an access node that may comprise an Open-Radio Unit (O-RU), Open-Distributed Unit (O-DU) and Open-Centralized Unit (O-CU). In typical examples, the radio network node may be a base station, such as a gNB or an eNB. In other examples, the radio network node may be a distributed node, such as a virtual node in the cloud 117, and may perform its functions entirely on the cloud 117, or partially, in collaboration with a radio network node.

[0071] The communications system 101 may cover a geographical area, which in some embodiments may be divided into cell areas, wherein each cell area may be served by a radio network node, although, one radio network node may serve one or several cells. The communications system 101 comprises one or more cells, whereof a first cell 121 and a second cell 122 are depicted in the non-limiting examples of Figure 1. In the non-limiting examples of Figure 1 , the fourth node 114 is a radio network node serving the first cell 121 and the fifth node 115 is a second radio network node serving the second cell 122. The radio network node may be of different classes, such as, e.g., macro eNodeB, home eNodeB or pico base station, based on transmission power and thereby also cell size. In some examples, the radio network node may serve receiving nodes with serving beams. The radio network node may be directly connected to one or more core networks.

[0072] Any of the first node 111 , the second node 112 and the one or more third nodes 113 comprised in the communications system 101 may support one or several communication technologies, and its name may depend on the technology and terminology used.

[0073] The communications system 101 may comprise one or more devices. In the non-limiting example depicted in Figure 1 , the one or more devices comprise a first device 131 and a second device 132. This may be understood to be for illustrative purposes only, and nonlimiting. Any of the one or more devices comprised in the communications system 101 may be a wireless communication device such as a 5G UE, or a UE, which may also be known as e.g., mobile terminal, wireless terminal and / or mobile station, a Customer Premises Equipment (CPE) a mobile telephone, cellular telephone, or laptop with wireless capability, just to mention some further examples. The one or more devices comprised in the communications system 101 may be, for example, portable, pocket-storable, hand-held, computer-comprised, or a vehicle-mounted mobile device, enabled to communicate voice and / or data, via the RAN, with another entity, such as a server, a laptop, a Personal Digital Assistant (PDA), or a tablet, Machine-to-Machine (M2M) device, device equipped with a wireless interface, such as a printer or a file storage device, modem, sensor, loT device, or any other radio network unit capable of communicating over a radio link in the communications system 101. The one or more devices comprised in the communications system 101 may be enabled to communicate wirelessly in the communications system 101. The communication may be performed e.g., via a RAN, and possibly the one or more core networks, which may be comprised within the communications system 101. It may be understood that the communications system 101 may comprise additional radio network nodes and / or additional devices.

[0074] The first node 111 may be configured to communicate within the computer system 100 with the second node 112 over a first link 161 , e.g., a radio link, or a wired link. The second node 112 may be configured to communicate within the communications system 101 with the plurality of third nodes 113 over a respective link. In the non-limiting examples of Figure 1 , the second node 112 may be configured to communicate within the communications system 101 with the fourth node 114 over a second link 162, e.g., a radio link, or a wired link. The second node 112 may be configured to communicate within the communications system 101 with the fifth node 115 over a third link 163, e.g., a radio link, or a wired link. Each third node of the plurality of third nodes 113 may be configured to communicate within the communications system 101 with respective one or more devices over a respective link. In the non-limiting examples of Figure 1 , the fourth node 114 may be configured to communicate within the communications system 101 with the first device 131 over a fourth link 164, e.g., a radio link. The fifth node 115 may be configured to communicate within the communications system 101 with the second device 132 over a fifth link 165, e.g., a radio link.

[0075] Any of the first link 161 , the second link 162, the third link 163, the fourth link 164 and the fifth link 165 may be a direct link or may be comprised of a plurality of individual links, wherein it may go via one or more computer systems or one or more core networks in the computer system 100, which are not depicted in Figure 1 , or it may go via an optional intermediate network. The intermediate network may be one of, or a combination of more than one of, a public, private or hosted network; the intermediate network, if any, may be a backbone network or the Internet; in particular, the intermediate network may comprise two or more sub-networks, which is not shown in Figure 1 .

[0076] In general, the usage of “first”, “second”, “third”, “fourth” and / or “fifth” herein may be understood to be an arbitrary way to denote different elements or entities, and may be understood to not confer a cumulative or chronological character to the nouns they modify.

[0077] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Other embodiments, however, are contained within the scope of the subject matter disclosed herein, the disclosed subject matter should not be construed as limited to only the embodiments set forth herein; rather, these embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art.

[0078] Embodiments of a computer-implemented method, performed by the second node 112, will now be described with reference to the flowchart depicted in Figure 2. The method is for handling hyperparameters corresponding to a plurality of sets of data. The second node 112 operates in the computer system 100.

[0079] The second node 112, in some examples, may be a data processing unit (DPU).

[0080] Several embodiments are comprised herein. In some embodiments all the actions may be performed. In some embodiments, some embodiments of the actions may be optional. In Figure 2, optional actions are indicated with dashed lines. It should be noted that the examples herein are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments. One or more embodiments may be combined, where applicable. All possible combinations are not described to simplify the description.

[0081] Action 201

[0082] Embodiments herein may be understood to be performed in a context wherein each third node in the plurality of third nodes 113 has collected a set of data of the plurality of sets of data. The data may be understood to be any data that may be of interest to analyze through machine learning. The data may be of different types, such as, for example, different Key Performance Indicators (KPIs) of the computer system 100 or the communications system 101. The plurality of third nodes 113 may be, for example, a plurality of radio network nodes, e.g., gNBs. The second node 112 may be understood to have access to all data from the plurality of third nodes 113, which may be understood as n resources. The plurality of sets of data may comprise, e.g., data set 1, data set 2, data set 3... data set n, one data set per third node.

[0083] Throughout the course of operations of the computer system 100, the plurality of third nodes 113 may collect a large number of data. The plurality of sets of data may be used for ML to analyze particular aspects of the computer system 100. For ML purposes, each set of data in the plurality of sets of data may require an HPT. Large amounts of data sets, where each of them may need an HPT may be understood to require many processing resources, and the process may be time consuming. Moreover, the cost may not provide as much performance improvement as may be expected. Embodiments herein may therefore be understood to aim, given a certain number, e.g., n, of data sets collected by the plurality of third nodes 113, to decrease the number of HPT that may need to be performed.

[0084] In this Action 201 , the second node 112 may obtain respective statistical features of the sets of data in the plurality of sets of data. Respective may be understood to mean that for each set of data of the plurality, corresponding statistical features, e.g., a set of statistical features, may be obtained. Obtaining in this Action 201 may be understood as receiving, e.g., via the second link 162 and / or the third link 163, determining, calculating, deriving, generating or similar. In some examples, the second node 112 may obtain the plurality of sets of data, and may calculate the respective statistical features itself in this Action 201 , whereas in other examples, the second node 112 may receive the respective statistical features from the plurality of third nodes 113, which may calculate themselves the respective statistical features based on the respective sets of data they may have collected.

[0085] The set of statistical features that may be or may have been generated from each dataset may reflect the statistical characteristics of the dataset, such as dataset size, number of features, maximum and minimum value, mean, variance value, bias, median, percentile, etc.

[0086] Action 202

[0087] After having obtained the respective statistical features of the sets of data in the plurality of sets of data, the second node 112 may pre-process, in Action 202, Action 203, or both, the obtained respective statistical features.

[0088] To pre-process may be understood as to process, e.g., manipulate or calculate, for further processing.

[0089] In some embodiments, the pre-processing of the received respective statistical features may comprise, in this Action 202, that the second node 112 may vectorize the received respective statistical features to create a matrix of the respective statistical features for each third node of the plurality of third nodes 113. The resulting matrix may have an N*K dimension, where N may be understood to be the number of the third nodes in the plurality of third nodes 113, and K may be understood to be the number of respective statistical features.

[0090] Vectorizing in this Action 202 may be understood as a weighted concatenation of different features received from different nodes.

[0091] By in this Action 202 vectorizing the received respective statistical features to create the matrix of the respective statistical features for each third node of the plurality of third nodes 113, the second node 112 may increase the importance of the sets of data in HPT due to different reasons, such as, e.g., better quality of the data or, e.g., higher priority.

[0092] Action 203

[0093] In some embodiments, the pre-processing of the received respective statistical features may comprise, in this Action 203, that the second node 112 may scale the vectorized respective statistical features to a real value between 0 and 1. Scaling in this Action 203 may be understood as normalizing the values of the vectors, which may involve subtracting the minimum value of each element from the vector, then dividing by the difference between the maximum and minimum values of the vector.

[0094] By in this Action 203 scaling the vectorized respective statistical features to a real value between 0 and 1 , the second node 112 may enable to obtain smaller standard deviations, which may be understood to suppress the effect of outliers.

[0095] Action 204

[0096] In this Action 204, the second node 112 may determine, using a first machine learning model, a plurality of clusters of sets of data formed by the plurality of sets of data having been collected by the plurality of third nodes 113 operating in the computer system 100. The plurality of clusters may comprise e.g., m, clusters.

[0097] A cluster may be understood as a group of sets of data that may be close to each other based on predefined similarity measures such as, e.g., Euclidean distance.

[0098] The sets of data may be located in different base stations, that is, in different third nodes of the plurality of third nodes 113. Therefore, gathering all the sets of data in one central unit to perform HPT may not be feasible.

[0099] In order to manage distributed HPT, the second node 112, e.g., via its clustering unit, may define, so called cluster centers. That is, for each cluster of data sets that may be determined in this Action 204, the second node 112 may select a cluster that may represent the whole cluster of data sets by having statistical features that are closest to all other data sets in its cluster.

[0100] The plurality of clusters may be determined in this Action 204 using the first machine learning model based on a similarity of the respective statistical features of the sets of data in the plurality of sets of data. The similarity may be measured, for example, by Euclidean distance.

[0101] The first machine learning model may be, for example, an unsupervised ML model, such as e.g., K-mean, Hierarchical Clustering, Gaussian Mixture Models and etc.. The clustering algorithm may be understood to be an iterative model that may require a predefined number of clusters, which may be denoted as K. The clustering algorithm may begin by randomly initializing the cluster centers. Then, it may compute similarity scores between these cluster centers and the other points in the sets of data. Each data point may be assigned to the cluster with the closest center. Once all data points may be assigned, the cluster centers may be recalculated based on the new cluster memberships. This process of recalculating centers and reassigning data points may be repeated until the cluster centers may stabilize, or a maximum number of iterations may be reached. The determining in this Action 204 of the plurality of clusters may be performed using the pre-processed respective statistical features. For example, the scaled matrix obtained in Action 203 may be fed to a clustering unit managed by the second node 112. The clustering unit may then create m clusters in this Action 204. After vectorizing the features and preprocessing them, the unsupervised ML model may be used to cluster the input data and find the similar datasets if they exist.

[0102] The first machine learning model may map the n vector datasets to m separate datasets, where m « n and datasets in one cluster may be understood to have similar characteristics.

[0103] Parameter m, that is, the number of clusters, may be understood to be a design parameter and may be decided based on a complexity and accuracy tradeoff. To clarify the trade-off between performance and number of clusters, the following two extreme cases may be considered. To obtain the best performance, the maximum number of clusters may be determined. In this case, the parameters for each data set and KPI may fine-tuned. A degraded performance may be obtained with a mininum number of clusters. In this case, there may be only one cluster and all data sets may be fine-tuned. This case may be similar to the existing methods.

[0104] Generally, the higher the number of clusters, the higher the resources, and the more time consuming the determining may be, although a better overall performance may be provided and vice versa.

[0105] The number of clusters may be determined by many methods, such as the elbow method. The elbow method may be understood as a method for choosing the best value of cluster numbers. It may involve plotting the performance measures of the clustering method versus the number of cluster and looking for an "elbow" point where the rate of decrease may sharply change. However, the method to find the optimal number of clusters may be understood to be out of scope of this disclosure.

[0106] According to examples of embodiments herein, third nodes of the plurality of third nodes 113 that may have similar statistical performance, and not necessary in proximity of each other so that they may be served by same edge, may be determined to be in a same cluster.

[0107] According to examples of embodiments herein, the second node 112 may build multiple clusters which may have HPs close to each other.

[0108] By determining the plurality of clusters of sets of data in this Action 204, the second node 112 may enable to group, that is, cluster, the similar data sets in order to ultimately decrease the number of HPT that may need to be performed. This may be understood to be advantageous since, as explained earlier, large amounts of sets of data, where each of them may need an HPT, may be understood to require huge processing resources, and it may be time consuming. Moreover, the cost may not provide as much performance improvement as may be expected. Action 205

[0109] In this Action 205, the second node 112 may determine two or more first sets of data out of the plurality of sets of data. Each of the two or more first sets of data may correspond to a respective set of data in a center of a respective cluster of the plurality of clusters.

[0110] The center of the respective cluster of the plurality of clusters may be understood as the cluster that may represent the whole respective cluster by having statistical features that may be understood to be closest to all other data sets in the respective cluster.

[0111] The center of the respective cluster of the plurality of clusters may be determined by taking the mean of all the data points that may be assigned to that cluster. Each datapoint may be a vector, and the average point of the vector, also called centroid, may be calculated by taking the average of each component, that is, dimension, of the vectors in a set.

[0112] Determining may be understood as calculating, deriving, mapping or similar. The determining in this Action 205 may comprise, for each cluster, mapping the center of the cluster to the corresponding input dataset.

[0113] A first number of the two or more first sets of data may be smaller than a second number, e.g., n, of sets of data in the plurality. In other words, the number of sets of data in the centers of the clusters may be understood to be smaller than then total number of sets of data.

[0114] Action 206

[0115] In this Action 206, the second node 112 may provide a first indication to the first node 111 operating in the computer system 100. The first indication may indicate which may be the two or more first sets of data corresponding to the centers of the respective clusters of the plurality of clusters.

[0116] In some examples, the first indication may be a message comprising the two or more first sets of data. In other words, in this Action 206, the data corresponding to the cluster centers that may have been determined in Action 205, may be sent to the first node 111. The first node 111 may manage or run e.g., a tuning unit. That is, a hyperparameter tuning unit.

[0117] In other examples, the second node 112 may not send the two or more first sets of data to the first node 111 , and instead the second node 112 may just send an indication of the cluster centers The indication of the cluster centers may be, for example, an ID of the center node 113 of each cluster, with which the first node 111 may reach the corresponding third node 113 and fetch data to perform HPT for that cluster. In such embodiments, the first node 111 may then need to obtain, e.g., fetch, the two or more first sets of data.

[0118] Providing may be understood as sending, e.g., via the first link 161. By providing the first indication to the first node 111 in this Action 206, the first node 111 may then be enabled to, once it may have obtained the those two or more first sets of data, perform HPT on those two or more first sets of data to find the hyperparameters for those two or more first sets of data. The first node 111 may be placed at the node where the data set may exist, e.g., co-located with the corresponding third node 113, or in the cloud 117, where the first node 111 may need to receive the data set from the corresponding third node 113, and then perform HPT.

[0119] Action 207

[0120] In this Action 207, the second node 112 may receive a second indication from the first node 111 responsive to the provided first indication. As stated earlier, the first node 111 may manage or run a tuning unit. The second indication may indicate a respective tuned hyperparameter for each of the determined two or more first sets of data. In other words, in this Action 207, the first node 111, e.g., the tuning unit, may send back the HPs for each of the two or more first sets of data, that is, each of the cluster centers, to the second node 112, e.g., to the DPU. The respective tuned hyperparameter may be understood to be to train a respective second machine learning model with a corresponding set of data of the plurality of sets of data. Each third node 113 may use the respective tuned hyperparameter to train its own second machine learning model, that is, its respective second machine learning model, with its own set of data. That is, every third node of the plurality of third nodes 113 may train its own, e.g., respective second machine learning model, with the respective set of data it may have collected.

[0121] The receiving may be performed, e.g., via the first link 161.

[0122] That the second indication may indicate respective tuned hyperparameters may comprise, in some examples, that the second indication may comprise the respective tuned hyperparameters. Once received, the second node 112 may then store the second indication, e.g., may store the indicated respective hyperparameters in a memory of the second node 112.

[0123] The indicated respective tuned hyperparameter for each first set of data may be to be used by each set of data in the same cluster.

[0124] Hyperparameter Selection

[0125] The received second indication may indicate respective tuned hyperparameters that may have been selected based on one or more criteria. The selection may have been performed, e.g., by the first node 111 in Action 304, prior to the tuning. It may be understood that ML models, such as the respective second ML models, may have tens of parameters to be set and tuned. However, not all parameters may have big impact on the model. Moreover, HPT may be understood to be a time and resource consuming task. Therefore, it may neither be practical nor logical to fine-tune all the parameters. Only relevant, or the most relevant, parameters may need to be selected. This may be performed before the first node 111 may send back the respective tuned hyperparameters. In order to select the most important subset of HPs to be tuned, different methods may be used. Some models may introduce their most important parameters, and those parameters may be selected for parameter tuning. One ML model may have multiple hyperparameters. The sensitivity or dependency of the performance of the respective second machine learning model on tunning each hyperparameter may be different. For example, for a random forest model, the HPs may be number of trees, depth of each tree, number of sample for training each tree, minimum number of samples on each leaf node in each tree, maximum number of leaf nodes etc. Number of trees may have more significant impact on model performance than the maximum number of nodes, and thus may be considered to be more important to tune.

[0126] However, these pieces of information, that is, which HPs may be more relevant to tune may not be available for all models. There may be other methods that may be used such as the following method. First, one or a few available data sets may be picked. Second, the set of HPs for the model, e.g., the respective second ML model, may be listed. Third, all the HPs may be kept fixed but one. Fourth, the model may be evaluated on the training set, and the statistical information, e.g., min, mean, max, std, etc, of KPI, e.g., accuracy, precision, etc, may be derived. Fifth, the third to fourth step may be repeated for all HPs. Sixth, the HPs may be sorted based on their performance. Seventh, the top P HPs may be picked. Last, HPT may be performed. Although this may be an exhaustive task, it is a one time task that may be performed offline in the laboratory, or on the node, e.g., the third node 113, before deployment. The number of HPs to be tuned, P, may be set based on how much complexity the operator of the computer system 100 may bear.

[0127] Action 208

[0128] The indicated respective tuned hyperparameter for each first set of data, that is, by each cluster center, may be to be used by each set of data in the same cluster. As stated earlier, the sets of data in the plurality of sets of data may have been collected by the plurality of third nodes 113 operating in the computer system 100. Each third node in the plurality of third nodes 113 may have collected a set of data of the plurality of sets of data. In some embodiments, in this Action 208, the second node 112 may provide a respective third indication to the respective third nodes in the plurality of third nodes 113 having collected the sets of data in the same cluster. The respective third indication may indicate the respective tuned hyperparameter to be used by each set of data in the same cluster. The second node 112 may send the third indication by broadcasting the HPs to the corresponding third nodes of the plurality of third nodes 113, e.g., via the second link 162 and the third link 163. By the second node 112 sending the respective third indication to the respective third nodes in the plurality of third nodes 113 in this Action 208, each member of a cluster may be able to get the fined tuned parameters. With this approach, it may not be necessary to perform HPT for all the third nodes in the same cluster. Moreover, since the third nodes may be understood to be distributed, they may be understood to not need to send their data sets, and consequently, only minimal information may need to be exchanged.

[0129] Action 209

[0130] Sometime after the different Actions 201-208 of the method performed by the second node 112 may have been performed, a new dataset may arrive. Action 209 may be performed once the first machine learning model may have been trained to a satisfactory accuracy level.

[0131] Within such a context, in this Action 209, the second node 112 obtains one or more respective statistical features corresponding to a set of data having been collected by the fourth node 114 operating in the computer system 100. The fourth node 114 may be understood to be one of the plurality of third nodes 113, which may have been comprised in the plurality of third nodes 113 during the performance of Actions 201-208, or which may have been deployed, or added to the plurality of third nodes 113 subsequently.

[0132] The set of data having been collected by the fourth node 114 may be understood to be a new set of data.

[0133] Obtaining may be understood as calculating or deriving or as receiving. For example, the fourth node 114 itself may have calculated the respective statistical features, and the second node 112 may have received or fetched them from the fourth node 114.

[0134] In this Action 209, the second node 112 may further generate, e.g., calculate or determine, the dataset features vector from the new dataset. That is, the second node 112 may in this Action 209, preprocess, e.g., vectorize and scale, as described in Actions 202 and 203, respectively, the obtained one or more respective statistical features.

[0135] Action 210

[0136] In this Action 210, the second node 112 determines, using the first machine learning model, a first cluster of sets of data corresponding to the obtained one or more respective statistical features obtained in Action 209. The determining in this Action 210 is based on a first similarity of the one or more respective statistical features to respective statistical features of the sets of data in the determined first cluster.

[0137] The first cluster is one of the plurality of clusters of sets of data formed by the plurality of sets of data having been collected by the plurality of third nodes 113 operating in the computer system 100. In other words, the first cluster may be understood to be a selected cluster from the plurality of clusters previously determined by the second node 112, e.g., in Action 204. As stated earlier, each third node in the plurality of third nodes 113 has collected a set of data of the plurality of sets of data. The plurality of clusters have been determined using the first machine learning model based on the similarity of the respective statistical features of the sets of data in the plurality of sets of data, as described in Action 204.

[0138] According to the foregoing, in this Action 210, the second node 112 may use the trained first machine learning model, that is, the clustering model used in Action 204, to, in an inference phase, infer the first cluster and its corresponding tuned hyperparameter. Put differently, once the second node 112 may have completed Actions 201-207 a first time, it may, in this Action 210, reuse the calculated, and tuned, hyperparameters for future data sets by determining which of the already calculated clusters may be most similar to the new data. If a satisfactory similarity is found, it may be understood that the processing may be reduced by reusing the already determined, and tuned, hyperparameter for the first cluster.

[0139] In some examples, the performance of the first machine learning model may be also evaluated in this Action 210. Clustering and HPT, corresponding to Actions 201-208, may be understood to be performed only once until a performance drift of the clustering model may be found. If the performance of the first machine learning model gets degraded, the second node 112 may need to repeat Actions 201-208 to retrain the first machine learning model and redo HPT correspondingly. When a new dataset may arrive, the second node 112 may perform Action 209 and 210. In Action 210, the second node 112 may need to check the performance of the existing clustering model to decide to retrain the clustering model and redo HPT or not.

[0140] Action 211

[0141] In this Action 211, the second node 112 may send a fourth indication to the fourth node 114, e.g., via the second link 162. The third indication may indicate the determined first cluster.

[0142] Action 212

[0143] After inference of the clustering for the new data set, the performance of the clustering may need to be evaluated by the second node 112. There may be many different metrics in order to evaluate the clustering performance, including Silhouette Coefficient, Dunn’s Index, etc. Most of them may be based on computing a similarity or dissimilarity measure, such as the distance between cluster points. In some examples, the algorithm may compute the Silhouette coefficient which may be understood to be bound between -1 for incorrect clustering and +1 for highly dense clustering. If the computed score is below a predefined threshold, the clustering may be rejected by the fourth node 114, and the second node 112 may need to run the HPT on the new data set. The threshold to decide whether to accept and / or to reject the clustering may be configured, e.g., any s operator-set value. For example, the current maximum distance of cluster which may have been stored in the fourth node 114 earlier.

[0144] In this Action 212, the second node 112 may receive a fifth indication from the fourth node 114 responsive to the sent third indication. The fourth indication may indicate whether or not the fourth node 114 may have accepted the determined first cluster.

[0145] Action 213

[0146] In this Action 213, the second node 112 provides, to the fourth node 114, a respective tuned hyperparameter corresponding to a respective first set of data in a first center of the determined first cluster. The providing in this Action 213 of the respective tuned hyperparameter may be understood to be performed with the proviso the fourth node 114 may have accepted the determined first cluster.

[0147] The respective tuned hyperparameter provided in this Action 213 may be understood to one of the respective tuned hyperparameters that may have been indicated to the second node 112 in the second indication that the second node 112 may have received in Action 207, and e.g., subsequently stored.

[0148] The providing, e.g., sending, in this Action 213 may be performed via the second link 162.

[0149] With the proviso that the fourth node 114 may have accepted the determined first cluster, by providing the respective tuned hyperparameter corresponding to the respective first set of data in the first center of the determined first cluster in this Action 213, the second node 112 may enable to maintain network KPIs, with lower number sets of hyperparameters, which may be more feasible to be deployed in a real network. By the second node 112 enabling to associate the tuned hyperparameters to a new data set or deployment scenario, such as the new set of data having been collected by the fourth node 114, the second node 112 may enable to improve the performance of an ML-model, such as the second ML model of the fourth node 114, when it may be deployed in a new environment. The improvement may be enabled by reducing the set up time to deploy the ML model in a new environment. The second node 112 may be further understood to enable an adaptable solution to different network situations.

[0150] Embodiments of a computer-implemented method, performed by the first node 111 , will now be described with reference to the flowchart depicted in Figure 3. The method is for handling hyperparameters corresponding to the plurality of sets of data. The first node 111 operates in the computer system 100.

[0151] Several embodiments are comprised herein. In some embodiments all the actions may be performed. In some embodiments, some actions may be optional. In Figure 3, optional actions are indicated with dashed lines. It should be noted that the examples herein are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments. One or more embodiments may be combined, where applicable. All possible combinations are not described to simplify the description. The detailed description of some of the following corresponds to the same references provided above, in relation to the actions described for the first node 111 , and will thus not be repeated here. For example, in some examples, the first node 111 may manage or run a tuning unit. The second node 112 may manage or run a data processing unit.

[0152] Action 301

[0153] In this Action 301 , the first node 111 may obtain the first indication from the second node 112 operating in the computer system 100. The first indication may indicate which may be the two or more first sets of data corresponding to the centers of the respective clusters of the plurality of clusters.

[0154] In some examples, this Action 301 may be performed in some examples wherein the second node 112 may not send the two or more first sets of data to the first node 111, and instead, the first node 111 may just receive an indication of the cluster centers. In such embodiments, the first node 111 may then need to fetch the two or more first sets of data in the next Action 302.

[0155] In other examples, the two or more first sets of data may be comprised in the first indication.

[0156] Obtaining in this Action 302 may comprise e.g., receiving, e.g., from the second node 112 via the first link 161.

[0157] Action 302

[0158] In this Action 302, the first node 111 obtains the two or more first sets of data, out of the plurality of sets of data. As stated earlier, the plurality of sets of data has been collected by the plurality of third nodes 113 operating in the computer system 100, e.g., a plurality of radio network nodes. Each third node in the plurality of third nodes 113 has collected a set of data of the plurality of sets of data. The plurality of clusters of sets of data have been determined, to be formed by the plurality of sets of data e.g., by the second node 112 as described in Figure 2. Each of the two or more first sets of data corresponds to a respective set of data in the center of the respective cluster of the plurality of clusters. The plurality of clusters have been determined based on the similarity of the respective statistical features of the sets of data in the plurality of sets of data. The first number of the two or more first sets of data is smaller than the second number of sets of data in the plurality of sets of data. That is, there may be understood to be fewer cluster centers than sets of data.

[0159] The obtaining in this Action 302 of the two or more first sets of data may be responsive to the obtained first indication. That is, the receiving of the first indication in Action 301 may prompt the first node 111 to fetch the two or more first sets of data, as indicated by the first indication.

[0160] In some examples, the two or more first sets of data may be comprised in the first indication.

[0161] Obtaining in this Action 302 may comprise receiving, e.g., from the second node 112 via the first link 161 , or fetching, e.g., from the second node 112 via the first link 161. In the examples wherein the two or more first sets of data may be comprised in the first indication, the obtaining in this Action 302 may comprise extracting the data from the first indication.

[0162] Action 303

[0163] In this Action 303, the first node 111 determines the respective hyperparameter for each obtained two or more first sets of data and tunes the determined respective hyperparameters. The respective tuned hyperparameters are to train a respective machine learning model, that is, the respective second machine learning model, with the corresponding set of data of the plurality of sets of data. As stated earlier, every third node of the plurality of third nodes 113 may train its own, e.g., respective second machine learning model, with the respective set of data it may have collected.

[0164] Determining in this Action 303 may be understood as calculating, deriving, or similar. The determining method may be chosen based on the implementation of the respective second machine learning model. Different machine learning models may have different hyperparameters. For example, for a neural network model, the HPs may be number of layers and number of nodes per layer etc, for a random forest model, the HPs may be number of trees, maximum depth of each tree, etc. Determining may be understood, in some examples, as identifying the HPs corresponding to the type of the given second machine learning model. As a non-limiting example, the first node 111 may use, for example, a table to map model type to the corresponding HPs..

[0165] The determined respective tuned hyperparameter for each first set of data is to be used by each set of data in the same cluster.

[0166] Action 304

[0167] In this Action 304, the first node 111 may select, prior to the tuning, one or more of the determined two or more respective hyperparameters based on the one or more criteria. The selection may be performed as described earlier. The one or more criteria may comprise, for example, algorithm robustness with respect to the hyperparameter.

[0168] Action 305

[0169] In this Action 305, the first node 111 outputs an indication, that is, the second indication, indicating the determined two or more respective tuned hyperparameters.

[0170] To output may be understood as sending, e.g., to the second node 112 via the first link 161 , providing, etc.

[0171] In some embodiments, the outputting in this Action 305 of the indication may comprise sending the indication to the second node 112 as the second indication, responsive to the obtained first indication.

[0172] In some embodiments, the output indication, that is, the output second indication, may indicate the selected respective tuned hyperparameters. In some examples, the output indication may comprise the respective parameters, e.g., values of the respective parameters.

[0173] Embodiments of a computer-implemented method, performed by the computer system 100, will now be described with reference to the flowchart depicted in Figure 4. The method may be understood to be for handling the hyperparameters corresponding to the plurality of sets of data. The computer system 100 comprises the first node 111 and the second node 112.

[0174] The method may comprise the actions described below. In some embodiments all the actions may be performed. In some embodiments, some of the actions may be performed. In Figure 4, optional actions are indicated with a dashed box. One or more embodiments may be combined, where applicable. All possible combinations are not described to simplify the description. It should be noted that the examples herein are not mutually exclusive. Components from one example or embodiment may be tacitly assumed to be present in another example or embodiment and it will be obvious to a person skilled in the art how those components may be used in the other examples or embodiments.

[0175] The detailed description of some of the following corresponds to the same references provided above, in relation to the actions described for the first node 111 and second node 112 and will thus not be repeated here to simplify the description. For example, in some examples, the first node 111 may manage or run a tuning unit. The second node 112 may manage or run a data processing unit.

[0176] Action 401

[0177] In in this Action 401 , the computer system 100 may obtain, by the second node 112, the respective statistical features of the sets of data in the plurality of sets of data. This Action 401 may be understood to correspond to Action 201.

[0178] Action 402

[0179] In Action 402, in Action 403, or both, the computer system 100 may pre-process, by the second node 112, the obtained respective statistical features.

[0180] The pre-processing of the obtained respective statistical features may comprise Action 402, Action 403, or both.

[0181] In this Action 402, the computer system 100 may vectorize, by the second node 112, the received respective statistical features to create the matrix of the respective statistical features for each third node of the plurality of third nodes 113.

[0182] This Action 402 may be understood to correspond to Action 202.

[0183] Action 403

[0184] In this Action 403, the computer system 100 may scale, by the second node 112, the vectorized respective statistical features to a real value between 0 and 1.

[0185] This Action 403 may be understood to correspond to Action 203.

[0186] Action 404

[0187] In this Action 404, the computer system 100 may determine, by the second node 112, using the first machine learning model, the plurality of clusters of sets of data.

[0188] The determining in this Action 404 of the plurality of clusters may be performed using the pre-processed respective statistical features.

[0189] This Action 404 may be understood to correspond to Action 204.

[0190] Action 405

[0191] In this Action 405, the computer system 100 may determine, by the second node 112, the two or more first sets of data.

[0192] This Action 405 may be understood to correspond to Action 205.

[0193] Action 406

[0194] In this Action 406, the computer system 100 may provide, by the second node 112, the first indication to the first node 111 operating in the computer system 100. The first indication may indicate which may be the two or more first sets of data corresponding to the centers of the respective clusters of the plurality of clusters.

[0195] This Action 406 may be understood to correspond to Action 206. Action 407

[0196] In this Action 407, the computer system 100 may obtain, by the first node 111, the first indication from the second node 112.

[0197] This Action 407 may be understood to correspond to Action 301.

[0198] Action 408

[0199] In in this Action 408, the computer system 100, obtains, by the first node 111, the two or more first sets of data, out of the plurality of sets of data. As stated earlier, the plurality of sets of data have been collected by the plurality of third nodes 113 operating in the computer system 100. Each third node in the plurality of third nodes 113 have collected the set of data of the plurality of sets of data. The plurality of clusters of sets of data have been determined to be formed by the plurality of sets of data. Each of the two or more first sets of data corresponds to a respective set of data in the center of the respective cluster of the plurality of clusters. The plurality of clusters have been determined based on the similarity of the respective statistical features of the sets of data in the plurality of sets of data. The first number of the two or more first sets of data is smaller than the second number of sets of data in the plurality of sets of data.

[0200] The obtaining in this Action 408 of the two or more first sets of data may be responsive to the obtained first indication.

[0201] This Action 408 may be understood to correspond to Action 302.

[0202] Action 409

[0203] In this Action 409, the computer system 100, determines, by the first node 111, the respective hyperparameter for each obtained two or more first sets of data and tunes the determined respective hyperparameters. The respective tuned hyperparameters are to train the respective second machine learning model with the corresponding set of data of the plurality of sets of data. The training of the respective second machine learning model may be performed by the respective third node of the plurality of third nodes 113 with the set of data it may have collected.

[0204] This Action 409 may be understood to correspond to Action 303.

[0205] Action 410

[0206] In this Action 410, the computer system 100 may select, by the first node 111, prior to the tuning, the one or more of the determined two or more respective hyperparameters based on the one or more criteria.

[0207] This Action 410 may be understood to correspond to Action 304. Action 411

[0208] In this Action 411 , the computer system 100 outputs, by the first node 111 , the second indication indicating the determined two or more respective tuned hyperparameters to the second node 112.

[0209] The outputting in this Action 411 of the second indication may comprise sending the second indication responsive to the obtained first indication.

[0210] The output second indication may indicate the selected respective tuned hyperparameters.

[0211] This Action 411 may be understood to correspond to Action 305.

[0212] Action 412

[0213] In this Action 412, the computer system 100 may receive, by the second node 112, the second indication from the first node 111.

[0214] The receiving in this Action 412 of the second indication may be responsive to the provided first indication.

[0215] This Action 412 may be understood to correspond to Action 207.

[0216] Action 413

[0217] In some embodiments, the indicated respective tuned hyperparameter for each first set of data may be to be used by each set of data in the same cluster. In some of such embodiments, in this Action 413, the computer system 100 may provide, by the second node 112, the respective third indication to the respective third nodes in the plurality of third nodes 113 having collected the sets of data in the same cluster. The respective third indication may indicate the respective tuned hyperparameter to be used by each set of data in the same cluster.

[0218] This Action 413 may be understood to correspond to Action 208.

[0219] In some examples, the method performed by the computer system 100 may comprise that the respective tuned hyperparameter may then be used by the respective third node to train its respective second machine learning model.

[0220] Action 414

[0221] In in this Action 414, the computer system 100, obtains, by the second node 112, the one or more respective statistical features corresponding to the set of data having been collected by the fourth node 114 operating in the computer system 100. That is, the new data set. Action 414 may be performed once the first machine learning model may have been trained to a satisfactory accuracy level.

[0222] This Action 414 may be understood to correspond to Action 209. Action 415

[0223] In this Action 415, the computer system 100 determines, by the second node 112, using the first machine learning model, the first cluster of sets of data corresponding to the obtained one or more respective statistical features. The determining in this Action 415 is based on the first similarity of the one or more respective statistical features to the respective statistical features of the sets of data in the determined first cluster. The first cluster is one of the plurality of clusters of sets of data formed by the plurality of sets of data.

[0224] This Action 415 may be understood to correspond to Action 210.

[0225] Action 416

[0226] In this Action 416, the computer system 100 may send, by the second node 112, the fourth indication to the fourth node 114. The fourth indication may indicate the determined first cluster.

[0227] This Action 416 may be understood to correspond to Action 211.

[0228] Action 417

[0229] In this Action 417, the computer system 100 may receive, by the second node 112, the fifth indication from the fourth node 114 responsive to the sent fourth indication. The fifth indication may indicate whether or not the fourth node 114 may have accepted the determined first cluster.

[0230] This Action 417 may be understood to correspond to Action 212.

[0231] Action 418

[0232] In this Action 418, the computer system 100, provides, by the second node 112 and based on the received second indication, to the fourth node 114, the respective tuned hyperparameter corresponding to the respective first set of data in the first center of the determined first cluster.

[0233] The providing in this Action 418 of the respective tuned hyperparameter may be understood to be performed with the proviso the fourth node 114 may have accepted the determined first cluster.

[0234] This Action 418 may be understood to correspond to Action 213.

[0235] In some examples, the method performed by the computer system 100 may comprise that the fourth node 114 may train its respective second machine learning model with the respective tuned hyperparameter provided by the second node 112. Figure 5 is a schematic diagram depicting a non-limiting example of a method performed by computer system 100, for model training according to embodiments herein. In examples of embodiments herein, such as that depicted in Figure 5, it may be assumed that the computer system 100 may have the access to all data from n resources. Large amount of data sets where each of them may need an HPT may be understood to need a large amount of processing resources and it may be time consuming. On the other hand, the cost may not provide as much as performance improvement as may be expected. Therefore, according to embodiments herein, the second node 112 of the computer system may cluster the similar data set in order to decrease the number of HPT that may need to be performed. In this method, in accordance with Action 401, a set of statistical features (SF) may be generated from each dataset that may reflect the statistical characteristics of the datasets, such as dataset size, number of features, maximum and minimum value, mean, variance value, etc. After vectorization of the features and preprocessing them according to Action 402 and Action 403, an unsupervised ML model may be used, according to Action 404, to cluster the input data and find the similar datasets, if they exist. This function may be understood to map the n vector datasets to m separate datasets, where m « n and the datasets in one cluster may be understood to have similar characteristics. Then for each cluster, the second node 112 may, according to Action 405, map the center of cluster features to the input dataset and HPT may then be used, according to Action 409 and 410, to find the tuned hyperparameter for that dataset. The first node 111 may then output the second indication according to Action 411 , indicating for each cluster center, the respective tuned hyperparameter. In the non-limiting example of Figure 5, this is, for Cluster 1 center data set, HPT 1, for Cluster 2 center data set, HPT 2, for Cluster m center data set, HPT m.

[0236] Figure 6 is another schematic diagram depicting distributed hyperparameter tuning during a training phase of the first machine learning model, according to a non-limiting example of embodiments herein. In some examples of embodiments herein, the data sets may be located in different third nodes, e.g., base stations. Therefore, gathering all the datasets in one central unit may not be feasible. Figure 6 shows a non-limiting example of a method for the management of distributed HPT, according to embodiments herein to address this issue. As illustrated in Figure 6, each node may send their SF, e.g., mean, variance, bias, median, percentile, etc. to the second node 112, managing a data processing unit (DPU) in this example. The DPU may be understood to be responsible for collecting information from the plurality of third nodes 113, according to Action 401, and provide them with necessary information, such as center clusters, tuned HPs, etc. The DPU may first obtain the SFs from the nodes, according to Action 401. In the preprocessing step, according to Action 402, it may vectorize the inputs to create a Matrix with NxK dimension where N may be the number of the nodes and K may be the number of SFs. Then, according to Action 403, the values may be scaled to be a real value between 0 and 1. The scaled matrix may be fed to the clustering unit. The clustering unit , according to Action 404, may create m clusters and define the cluster centers, according to Action 405. The cluster centers , according to Action 406, may be sent to the first node 111 , which may manage a T uning unit which may be responsible for fetching the data set from the cluster center, according to Action 408, and the HPT, according to Action 409. It may be worth noting that the Tuning unit may be placed at the node where the data set may exist, or at the cloud 117, where it may need to receive the data set from the corresponding third node 113, according to Action 408, and then perform HPT, according to Action 409. The Tuning unit may then, according to Action 411 and Action 412, sends back the HPs for each cluster to the DPU. The DPU may then, according to Action 413, broadcast the HPs to the corresponding third nodes 113.

[0237] Figure 7 is another schematic diagram depicting distributed hyperparameter tuning, but this time during an inference phase of machine learning, after having trained the first machine learning model, according to a non-limiting example of embodiments herein. Figure 7 illustrates a non-limiting example of the inference process of HPT, where the second node 112 may manage a data processing unit. The DPU, according to Action 407, may ask the first node 111 , which in this example may be co-localized with the third node 113 who may have the center data set, to fine tune the predefined set of hyperparameters and may then, according to Action 413, send back the tuned parameters to the rest of datasets in other clusters. Then, each member of that cluster may get the fined tuned parameters. With this approach, the computer system 100 may enable to avoid having to perform HPT for all the third nodes 113. Furthermore, since the third nodes 113 may be distributed, they may be enabled to avoid having to send their respective data sets. Hence, only minimal information may need to be exchanged.

[0238] Figure 8 is a flowchart depicting a non-limiting example of inference of the first machine learning model for a new dataset, according to embodiments herein. The depicted example may be used to associate a default HPT to a dataset, new or old. Figure 8 represents non-limiting example of how embodiments herein may enable to associate a hyperparameter to a new dataset obtained at 801. In this example, the second node 112 may need to, according to Action 414, at 802 preprocess the statistical features obtained in Action 414 to generate the dataset features vector from the new dataset. Then, at 803, the second node 112 may use the first machine learning mode, that is, the clustering model, to, according to Action 415, inference the cluster, k, in this example, and its corresponding tuned hyperparameter. The second node 112 may then, according to Action 416 and Action 417, check if the clustering is accepted with the fourth node 114. If the fourth node 114 rejects the clustering, the second node 112 may need to initiate performance of hyperparameter tuning at 804. If the fourth node 114 accepts the clustering, the second node 112 may, according to Action 418, associate the HPT k of cluster k to the new data set, thereby refraining from having to perform additional hyperparameter tuning.

[0239] Figure 9 is a schematic diagram depicting a non-limiting example of an Open Radio Access Network (O-RAN) implementation. Embodiments herein may be implemented in O- RAN. As a reference, a non-limiting example of the architecture of O-RAN is illustrated in Figure 9. In this architecture, each access node, e.g., each third node 113, may comprise an Open Radio Unit (O-RU) 901 , Open Distributed Unit (O-DU) 902 and Open Centralized Unit (O-CU), which may comprise an O-CU Control Plane (CP) 903 and an O-CU User Plane (UP) 904. Each third node 113 may collect a dataset and generate statistical features of the data set. The generated statistical features may be sent , according to Action 401, to the second node 112, implemented in a System Management and Orchestration(SMO) platform 905 via 01 interface 906. The SMO 905 may, according to Action 402 and Action 403, pre-process the received SFs from all third nodes 113 and, according to Action 404, cluster the third nodes 113 into certain number of clusters based on the processed SFs. Each cluster may contain a subset of all the third nodes 113. For each cluster, a center node may be selected, according to Action 405. The first node 111, also implemented in this example in the SMO 905 may then, according to Action 408, request a raw model training dataset from the selected center node via A1 interface 907. The selected center node may then send the raw model training data to the SMO via the A1 interface 907. The SMO 905 may the, according to Action 409, perform HPT with the raw model training dataset received from the center node. The tuned hyperparameter may then be distributed, according to Action 413, to all the nodes within that cluster via A1 interface 907. Then each third node 113 may use the received tuned hyperparameter to train its respective second machine learning model. As depicted in Figure 9, a NearReal Time RAN Intelligent Controller 908, may be located between the SMO 905 and the O-CU-CP 903, to which it may connect via an E2 interface 909. The O-CU-CP 903 and the O-CU-UP 904 may connect via an E1 interface 910. Each of the O-CU-CP 903 and the O-CU- UP 904 may connect to an O-DU 902 via an F1 interface 911.

[0240] As a summarized overview of the foregoing, embodiments herein may be understood to relate to a novel method, using ML and optimization theory, to fine tune the hyperparameters of an ML-model, the second ML model, when there may be many hyperparameters and multiple data sets. The method may provide one or multiple hyperparameters depending on the requirements, deployments, etc. In case of multiple tuned hyperparameters, the method may provide a framework to associate each data set to a set of hyper parameters. The fine-tuned parameters may be used as a default parameter for the second ML-model to be used either for each data set, or new data sets. When a new data set may be available, instead of fine tuning for the new data set, the method of embodiments herein may enable to associate it to one of the pre tuned hyperparameters and hence the hyperparameter tuning may no longer be needed.

[0241] Certain embodiments herein may provide one or more of the following technical advantage(s). As one advantage, embodiments herein may be understood to provide a new general framework for hyperparameter tuning when systems may have multiple datasets. As another advantage, embodiments herein, by avoiding usage of too many processing resources, may enable to maintain network KPIs, with lower number sets of hyperparameters that may be feasible to be deployed in real network. As yet another advantage, embodiments herein may provide a new framework to select hyperparameters for all data sets while maintaining good network KPIs. As a further advantage, embodiments herein may enable to associate the tuned hyperparameters to a new data set or deployment scenario. As yet another advantage, embodiments herein may enable to improve the second ML-model performance when it may be deployed in a new environment by reducing the set up time to deploy the second ML-model in a new environment. An additional advantage of embodiments herein may be providing an adaptable approach to different network situations, and a performance metric by providing a pareto optimal set of HPs, rather than one HP based on the number of clusters.

[0242] Figure 10 depicts an example of the arrangement that the first node 111 may comprise to perform the method described in Figure 3, Figures 4-7 and / or Figure 9. The first node 111 may be understood to be for handling the hyperparameters corresponding to the plurality of sets of data. The first node 111 is configured to operate in the computer system 100.

[0243] Several embodiments are comprised herein. It should be noted that the examples herein are not mutually exclusive. One or more embodiments may be combined, where applicable. All possible combinations are not described to simplify the description. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments. The detailed description of some of the following corresponds to the same references provided above, in relation to the actions described for the second node 112, and will thus not be repeated here. For example, in some examples, the first node 111 may be configured to manage or run a tuning unit. The second node 112 may be configured to manage or run a data processing unit. The first node 111 is configured to obtain the two or more first sets of data, out of the plurality of sets of data. The plurality of sets of data is configured to have been collected by the plurality of third nodes 113 configured to operate in the computer system 100. Each third node in the plurality of third nodes 113 is configured to have collected the set of data of the plurality of sets of data. The plurality of clusters of sets of data is configured to have been determined to be formed by the plurality of sets of data. Each of the two or more first sets of data is configured to correspond to a respective set of data in the center of the respective cluster of the plurality of clusters. The plurality of clusters are configured to have been determined based on the similarity of respective statistical features of the sets of data in the plurality of sets of data. The first number of the two or more first sets of data is configured to be smaller than the second number of sets of data in the plurality of sets of data.

[0244] The first node 111 is also configured to determine the respective hyperparameter for each of the two or more first sets of data configured to be obtained and tune the determined respective hyperparameters,. The respective tuned hyperparameters are configured be to train the respective machine learning model, that is, the second machine learning model, with the corresponding set of data of the plurality of sets of data.

[0245] The first node 111 is further configured to output the indication configured to indicate the two or more respective tuned hyperparameters configured to be determined.

[0246] In some embodiments, the respective tuned hyperparameter configured to be determined for each first set of data may be to be used by each set of data in the same cluster.

[0247] In some embodiments, the first node 111 may be further configured to obtain the first indication from the second node 112 configured to operate in the computer system 100. The first indication may be configured to indicate which may be the two or more first sets of data corresponding to the centers of the respective clusters of the plurality of clusters. The obtaining of the two or more first sets of data may be configured to be responsive to the first indication configured to be obtained. The outputting of the indication may be configured to comprise sending the indication to the second node 112 as the second indication, responsive to the first indication configured to be obtained.

[0248] In some embodiments, the first node 111 may be further configured to select, prior to the tuning, the one or more of the two or more respective hyperparameters configured to be determined based on the one or more criteria. The output indication may be configured to indicate the respective tuned hyperparameters configured to be selected.

[0249] The embodiments herein in the first node 111 may be implemented through one or more processors, such as a processing circuitry 1001 in the first node 111 depicted in Figure 10, together with computer program code for performing the functions and actions of the embodiments herein. A processor, as used herein, may be understood to be a hardware component. The program code mentioned above may also be provided as a computer program product, for instance in the form of a data carrier carrying computer program code for performing the embodiments herein when being loaded into the first node 111. One such carrier may be in the form of a CD ROM disc. It is however feasible with other data carriers such as a memory stick. The computer program code may furthermore be provided as pure program code on a server and downloaded to the first node 111.

[0250] The first node 111 may further comprise a memory 1002 comprising one or more memory units. The memory 1002 is arranged to be used to store obtained information, store data, configurations, schedulings, and applications etc. to perform the methods herein when being executed in the first node 111.

[0251] In some embodiments, the first node 111 may receive information from, e.g., the second node 112, any of the third nodes in the plurality of third nodes 113, such as the fourth node 114 and / or the fifth node 115, any of the radio network nodes, any of the one or more devices, such as the first device 131 , and / or the second device 132, and / or another structure in the communications network 101 and / or the computer system 100, through a receiving port 1003. In some embodiments, the receiving port 1003 may be, for example, connected to one or more antennas in first node 111. In other embodiments, the first node 111 may receive information from another structure in the communications network 100 through the receiving port 1003. Since the receiving port 1003 may be in communication with the processing circuitry 1001, the receiving port 1003 may then send the received information to the processing circuitry 1001. The receiving port 1003 may also be configured to receive other information.

[0252] The processing circuitry 1001 in the first node 111 may be further configured to transmit or send information to e.g., the second node 112, any of the third nodes in the plurality of third nodes 113, such as the fourth node 114 and / or the fifth node 115, any of the radio network nodes, any of the one or more devices, such as the first device 131, and / or the second device 132, and / or another structure in the communications network 101 and / or the computer system 100, through a sending port 1004, which may be in communication with the processing circuitry 1001, and the memory 1002.

[0253] Those skilled in the art will also appreciate that the units comprised within the first node 111 described above as being configured to perform different actions, may refer to a combination of analog and digital circuits, and / or one or more processors configured with software and / or firmware, e.g., stored in memory, that, when executed by the one or more processors such as the processing circuitry 1001 , perform as described above. One or more of these processors, as well as the other digital hardware, may be included in a single Application-Specific Integrated Circuit (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a System-on-a-Chip (SoC).

[0254] Also, in some embodiments, the different units comprised within the first node 111 described above as being configured to perform different actions described above may be implemented as one or more applications running on one or more processors such as the processing circuitry 1001.

[0255] Thus, the methods according to the embodiments described herein for the first node 111 may be respectively implemented by means of a computer program 1005 product, comprising instructions, i.e., software code portions, which, when executed on at least one processing circuitry 1001, cause the at least one processing circuitry 1001 to carry out the actions described herein, as performed by the first node 111. The computer program 1005 product may be stored on a computer-readable storage medium 1006. The computer- readable storage medium 1006, having stored thereon the computer program 1005, may comprise instructions which, when executed on at least one processing circuitry 1001, cause the at least one processing circuitry 1001 to carry out the actions described herein, as performed by the first node 111. In some embodiments, the computer-readable storage medium 1006 may be a non-transitory computer-readable storage medium, such as a CD ROM disc, or a memory stick. In other embodiments, the computer program 1005 product may be stored on a carrier containing the computer program 1005 just described, wherein the carrier is one of an electronic signal, optical signal, radio signal, or the computer-readable storage medium 1006, as described above.

[0256] The first node 111 may comprise a communication interface configured to facilitate, or an interface unit to facilitate, communications between the first node 111 and other nodes or devices, e.g. the second node 112, any of the third nodes in the plurality of third nodes 113, such as the fourth node 114 and / or the fifth node 115, any of the radio network nodes, any of the one or more devices, such as the first device 131 , and / or the second device 132, and / or another structure in the communications network 101 and / or the computer system 100. The interface may, for example, include a transceiver configured to transmit and receive radio signals over an air interface in accordance with a suitable standard.

[0257] In other embodiments, the first node 111 may comprise a radio circuitry 1007, which may comprise e.g., the receiving port 1003 and the sending port 1004.

[0258] The radio circuitry 1007 may be configured to set up and maintain at least a wireless connection with the second node 112, any of the third nodes in the plurality of third nodes 113, such as the fourth node 114 and / or the fifth node 115, any of the radio network nodes, any of the one or more devices, such as the first device 131 , and / or the second device 132, and / or another structure in the communications network 101 and / or the computer system 100. Circuitry may be understood herein as a hardware component. Hence, embodiments herein also relate to the first node 111 operative to operate in the communications network 100. The first node 111 may comprise the processing circuitry 1001 and the memory 1002, said memory 1002 containing instructions executable by said processing circuitry 1001, whereby the first node 111 is further operative to perform the actions described herein in relation to the first node 111 , e.g., in Figure 3, Figures 4-7 and / or Figure 9, e.g., with an architecture as depicted with the non-limiting example of Figure 0.

[0259] Figure 11 depicts an example of the arrangement that the second node 112 may comprise to perform the method described in Figure 4 and / or Figures 5-9. The second node 112 may be understood to be for handling the hyperparameters corresponding to the plurality of sets of data. The second node 112 is configured to operate in the computer system 100.

[0260] Several embodiments are comprised herein. It should be noted that the examples herein are not mutually exclusive. One or more embodiments may be combined, where applicable. All possible combinations are not described to simplify the description. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments. The detailed description of some of the following corresponds to the same references provided above, in relation to the actions described for the second node 112, and will thus not be repeated here. For example, in some examples, the first node 111 may be configured to manage or run a tuning unit. The second node 112 may be configured to manage or run a data processing unit.

[0261] The second node 112 is configured to obtain the one or more respective statistical features corresponding to the set of data configured to have been collected by the fourth node 114 configured to operate in the computer system 100.

[0262] The second node 112 is also configured to determine, using the first machine learning model, the first cluster of sets of data corresponding to the one or more respective statistical features configured to be obtained, based on the first similarity of the one or more respective statistical features to the respective statistical features of the sets of data in the first cluster configured to be determined. The first cluster is one of the plurality of clusters of sets of data formed by the plurality of sets of data configured to have been collected by the plurality of third nodes 113 configured to operate in the computer system 100. Each third node in the plurality of third nodes 113 is configured to have collected the set of data of the plurality of sets of data. The plurality of clusters are configured to have been determined using the first machine learning model based on the similarity of the respective statistical features of the sets of data in the plurality of sets of data. The second node 112 is further configured to provide, to the fourth node 114, the respective tuned hyperparameter corresponding to the respective first set of data in the first center of the first cluster configured to be determined.

[0263] In some embodiments, the second node 112 may be configured with the following four configurations.

[0264] In some embodiments, the second node 112 may be further configured to determine, using the first machine learning model, the plurality of clusters of sets of data.

[0265] In some embodiments, the second node 112 may be further configured to determine the two or more first sets of data out of the plurality of sets of data. Each of the two or more first sets of data may be configured to correspond to the respective set of data in the center of the respective cluster of the plurality of clusters. The first number of the two or more first sets of data may be configured to be smaller than the second number of sets of data in the plurality.

[0266] In some embodiments, the second node 112 may be further configured to provide the first indication to the first node 111 configured to operate in the computer system 100. The first indication may be configured to indicate which may be the two or more first sets of data corresponding to the centers of the respective clusters of the plurality of clusters.

[0267] In some embodiments, the second node 112 may be further configured to receive the second indication from the first node 111 responsive to the first indication configured to be provided. The second indication may be configured to indicate the respective tuned hyperparameter for each of the two or more first sets of data configured to be determined. The respective tuned hyperparameter may be configured to be to train the respective second machine learning model with the corresponding set of data of the plurality of sets of data.

[0268] The respective tuned hyperparameter configured to be indicated for each first set of data may be to be used by each set of data in the same cluster. The sets of data in the plurality of sets of data may be configured to have been collected by the plurality of third nodes 113 configured to operate in the computer system 100. Each third node in the plurality of third nodes 113 may be configured to have collected the set of data of the plurality of sets of data. In some embodiments, the second node 112 may be further configured to provide the respective third indication to the respective third nodes in the plurality of third nodes 113 configured to have collected the sets of data in the same cluster. The respective third indication may be configured to indicate the respective tuned hyperparameter to be used by each set of data in the same cluster.

[0269] In some embodiments, the second indication configured to be received may be configured to indicate the respective tuned hyperparameters that may be configured to have been selected based on the one or more criteria.

[0270] In some embodiments, the second node 112 may be configured with the following two configurations. In some embodiments, the second node 112 may be further configured to obtain the respective statistical features of the sets of data in the plurality of sets of data.

[0271] In some embodiments, the second node 112 may be further configured to pre-process the respective statistical features configured to be obtained, and the determining of the plurality of clusters may be configured to be performed using the respective statistical features configured to be pre-processed.

[0272] In some embodiments, the pre-processing of the respective statistical features configured to be obtained may be configured to comprise vectorizing the received respective statistical features to create the matrix of the respective statistical features for each third node of the plurality of third nodes 113.

[0273] In some embodiments, the pre-processing of the respective statistical features configured to be obtained may be configured to comprise scaling the vectorized respective statistical features to the real value between 0 and 1.

[0274] In some embodiments, the second node 112 may be configured with the following two configurations.

[0275] In some embodiments, the second node 112 may be further configured to send the fourth indication to the fourth node 114. The fourth indication may be configured to indicate the first cluster configured to be determined.

[0276] In some embodiments, the second node 112 may be further configured to receive the fifth indication from the fourth node 114 responsive to the fourth indication configured to be sent. The fifth indication may be configured to indicate whether or not the fourth node 114 may have accepted the first cluster configured to be determined. The providing of the respective tuned hyperparameter may be configured to be performed with the proviso the fourth node 114 may have accepted the first cluster configured to be determined.

[0277] The embodiments herein in the second node 112 may be implemented through one or more processors, such as a processing circuitry 1101 in the second node 112 depicted in Figure 11, together with computer program code for performing the functions and actions of the embodiments herein. A processor, as used herein, may be understood to be a hardware component. The program code mentioned above may also be provided as a computer program product, for instance in the form of a data carrier carrying computer program code for performing the embodiments herein when being loaded into the second node 112. One such carrier may be in the form of a CD ROM disc. It is however feasible with other data carriers such as a memory stick. The computer program code may furthermore be provided as pure program code on a server and downloaded to the second node 112.

[0278] The second node 112 may further comprise a memory 1102 comprising one or more memory units. The memory 1102 is arranged to be used to store obtained information, store data, configurations, schedulings, and applications etc. to perform the methods herein when being executed in the second node 112.

[0279] In some embodiments, the second node 112 may receive information from, e.g., the first node 111 , any of the third nodes in the plurality of third nodes 113, such as the fourth node 114 and / or the fifth node 115, any of the radio network nodes, any of the one or more devices, such as the first device 131, and / or the second device 132, and / or another structure in the communications network 101 and / or the computer system 100, through a receiving port 1103. In some embodiments, the receiving port 1103 may be, for example, connected to one or more antennas in second node 112. In other embodiments, the second node 112 may receive information from another structure in the communications network 100 through the receiving port 1103. Since the receiving port 1103 may be in communication with the processing circuitry 1101 , the receiving port 1103 may then send the received information to the processing circuitry 1101. The receiving port 1103 may also be configured to receive other information.

[0280] The processing circuitry 1101 in the second node 112 may be further configured to transmit or send information to e.g., the first node 111, any of the third nodes in the plurality of third nodes 113, such as the fourth node 114 and / or the fifth node 115, any of the radio network nodes, any of the one or more devices, such as the first device 131, and / or the second device 132, and / or another structure in the communications network 101 and / or the computer system 100, through a sending port 1104, which may be in communication with the processing circuitry 1101, and the memory 1102.

[0281] Those skilled in the art will also appreciate that the units comprised within the second node 112 described above as being configured to perform different actions, may refer to a combination of analog and digital circuits, and / or one or more processors configured with software and / or firmware, e.g., stored in memory, that, when executed by the one or more processors such as the processing circuitry 1101, perform as described above. One or more of these processors, as well as the other digital hardware, may be included in a single Application-Specific Integrated Circuit (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a System-on-a-Chip (SoC).

[0282] Also, in some embodiments, the different units comprised within the second node 112 described above as being configured to perform different actions described above may be implemented as one or more applications running on one or more processors such as the processing circuitry 1101.

[0283] Thus, the methods according to the embodiments described herein for the second node 112 may be respectively implemented by means of a computer program 1105 product, comprising instructions, i.e., software code portions, which, when executed on at least one processing circuitry 1101 , cause the at least one processing circuitry 1101 to carry out the actions described herein, as performed by the second node 112. The computer program 1105 product may be stored on a computer-readable storage medium 1106. The computer- readable storage medium 1106, having stored thereon the computer program 1105, may comprise instructions which, when executed on at least one processing circuitry 1101, cause the at least one processing circuitry 1101 to carry out the actions described herein, as performed by the second node 112. In some embodiments, the computer-readable storage medium 1106 may be a non-transitory computer-readable storage medium, such as a CD ROM disc, or a memory stick. In other embodiments, the computer program 1105 product may be stored on a carrier containing the computer program 1105 just described, wherein the carrier is one of an electronic signal, optical signal, radio signal, or the computer-readable storage medium 1106, as described above.

[0284] The second node 112 may comprise a communication interface configured to facilitate, or an interface unit to facilitate, communications between the second node 112 and other nodes or devices, e.g. the first node 111, any of the third nodes in the plurality of third nodes 113, such as the fourth node 114 and / or the fifth node 115, any of the radio network nodes, any of the one or more devices, such as the first device 131 , and / or the second device 132, and / or another structure in the communications network 101 and / or the computer system 100. The interface may, for example, include a transceiver configured to transmit and receive radio signals over an air interface in accordance with a suitable standard.

[0285] In other embodiments, the second node 112 may comprise a radio circuitry 1107, which may comprise e.g., the receiving port 1103 and the sending port 1104.

[0286] The radio circuitry 1107 may be configured to set up and maintain at least a wireless connection with the first node 111, any of the third nodes in the plurality of third nodes 113, such as the fourth node 114 and / or the fifth node 115, any of the radio network nodes, any of the one or more devices, such as the first device 131 , and / or the second device 132, and / or another structure in the communications network 101 and / or the computer system 100. Circuitry may be understood herein as a hardware component.

[0287] Hence, embodiments herein also relate to the second node 112 operative to operate in the communications network 100. The second node 112 may comprise the processing circuitry 1101 and the memory 1102, said memory 1102 containing instructions executable by said processing circuitry 1101 , whereby the second node 112 is further operative to perform the actions described herein in relation to the second node 112, e.g., in Figure 4 and / or Figures 5-9.

[0288] Figure 12 depicts an example of the arrangement that the computer system 100 may comprise to perform the method described in Figure 4 and / or any of Figures 5-9. The computer system 100 may be understood to be for handling hyperparameters corresponding to the plurality of sets of data. The computer system 100 is configured to comprise the first node 111 and the second node 112.

[0289] Several embodiments are comprised herein. It should be noted that the examples herein are not mutually exclusive. One or more embodiments may be combined, where applicable. All possible combinations are not described to simplify the description. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments. The detailed description of some of the following corresponds to the same references provided above, in relation to the actions described for the second node 112 and will thus not be repeated here. For example, in some examples, the first node 111 may be configured to manage or run a tuning unit. The second node 112 may be configured to manage or run a data processing unit.

[0290] The computer system 100 is configured to obtain, by the first node 111 , the two or more first sets of data, out of the plurality of sets of data. The plurality of sets of data is configured to have been collected by the plurality of third nodes 113 configured to operate in the computer system 100. Each third node in the plurality of third nodes 113 is configured to have collected the set of data of the plurality of sets of data. The plurality of clusters of sets of data is configured to have been determined to be formed by the plurality of sets of data. Each of the two or more first sets of data corresponds to the respective set of data in the center of the respective cluster of the plurality of clusters. The plurality of clusters are configured to have been determined based on the similarity of the respective statistical features of the sets of data in the plurality of sets of data. The first number of the two or more first sets of data is configured to be smaller than the second number of sets of data in the plurality of sets of data.

[0291] The computer system 100 is also configured to determine, by the first node 111, the respective hyperparameter for each of the two or more first sets of data configured to be obtained and tune the determined respective hyperparameters,. The respective tuned hyperparameters are configured to be to train the respective second machine learning model with the corresponding set of data of the plurality of sets of data.

[0292] The computer system 100 is further configured to output, by the first node 111 , the second indication configured to indicate the two or more respective tuned hyperparameters configured to be determined to the second node 112.

[0293] The computer system 100 is further configured to receive, by the second node 112, the second indication from the first node 111.

[0294] The computer system 100 is additionally configured to obtain, by the second node 112, the one or more respective statistical features corresponding to the set of data configured to have been collected by the fourth node 114 configured to operate in the computer system 100. The computer system 100 is also configured to determine, by the second node 112, using the first machine learning model, the first cluster of sets of data. The first cluster of sets of data corresponds to the one or more respective statistical features configured to have been obtained, based on the first similarity of the one or more respective statistical features to the respective statistical features of the sets of data in the first cluster configured to be determined. The first cluster is configured to be one of the plurality of clusters of sets of data formed by the plurality of sets of data.

[0295] The computer system 100 is further configured to provide, by the second node 112 and based on the second indication configured to be received, to the fourth node 114, the respective tuned hyperparameter corresponding to the respective first set of data in the first center of the first cluster configured to be determined.

[0296] In some embodiments, the computer system 100 may be further configured with the four following configurations.

[0297] In some embodiments, the computer system 100 may be further configured to determine, by the second node 112, using the first machine learning model, the plurality of clusters of sets of data.

[0298] In some embodiments, the computer system 100 may be further configured to determine, by the second node 112, the two or more first sets of data.

[0299] In some embodiments, the computer system 100 may be further configured to provide, by the second node 112, the first indication to the first node 111 configured to operate in the computer system 100. The first indication may be configured to indicate which may be the two or more first sets of data corresponding to the centers of the respective clusters of the plurality of clusters. The receiving of the second indication may be configured to be responsive to the first indication configured to be provided.

[0300] In some embodiments, the computer system 100 may be further configured to obtain, by the first node 111, the first indication from the second node 112. The obtaining of the two or more first sets of data may be configured to be responsive to the first indication configured to be obtained. The outputting of the second indication may be configured to comprise sending the second indication responsive to the first indication configured to be obtained.

[0301] The respective tuned hyperparameter configured to be indicated for each first set of data may be to be used by each set of data in the same cluster. In some embodiments, the computer system 100 may be further configured to provide, by the second node 112, the respective third indication to the respective third nodes in the plurality of third nodes 113 configured to have collected the sets of data in the same cluster. The respective third indication may be configured to indicate the respective tuned hyperparameter to be used by each set of data in the same cluster. In some embodiments, the computer system 100 may be further configured to select, by the first node 111, prior to the tuning, the one or more of the two or more respective hyperparameters configured to be determined based on the one or more criteria. The output second indication may be configured to indicate the respective tuned hyperparameters configured to be selected.

[0302] In some embodiments, the computer system 100 may be further configured with the two following configurations.

[0303] In some embodiments, the computer system 100 may be further configured to obtain, by the second node 112, the respective statistical features of the sets of data in the plurality of sets of data.

[0304] In some embodiments, the computer system 100 may be further configured to pre- process, by the second node 112, the respective statistical features configured to be obtained. The determining of the plurality of clusters may be configured to be performed using the respective statistical features configured to be pre-processed.

[0305] In some embodiments, the pre-processing of the respective statistical features configured to be obtained may be configured to comprise vectorizing, by the second node 112, the respective statistical features configured to be received to create the matrix of the respective statistical features for each third node of the plurality of third nodes 113 .

[0306] In some embodiments, the pre-processing of the respective statistical features configured to be obtained may be configured to comprise scaling, by the second node 112, the vectorized respective statistical features to the real value between 0 and 1.

[0307] In some embodiments, the computer system 100 may be further configured with the two following configurations.

[0308] In some embodiments, the computer system 100 may be further configured to send, by the second node 112, the fourth indication to the fourth node 114. The fourth indication may be configured to indicate the first cluster configured to be determined.

[0309] In some embodiments, the computer system 100 may be further configured to receive, by the second node 112, the fifth indication from the fourth node 114 responsive to the sent fourth indication. The fifth indication may be configured to indicate whether or not the fourth node 114 may have accepted the first cluster configured to be determined. The providing of the respective tuned hyperparameter may be configured to be performed with the proviso the fourth node 114 may have accepted the first cluster configured to be determined.

[0310] Each of the components of the first node 111 and the second node 112 have already been described in relation to, and as depicted in, Figure 9 and Figure 10, respectively, and will therefore not be repeated.

[0311] Embodiments herein also relate to the computer system 100 comprising the first node 111 and the second node 112, whereby the first node 111 and the second node 112 are further operative to perform the actions described herein in relation to the first node 111 and the second node 112, e.g., in Figure 4 and / or any of Figures 5-9.

[0312] REFERENCES

[0313] 1. Gomes Mantovani R, Debiaso Rossi AL, Alcobaga E, Castro Gertrudes J, Barbon Junior S, Ferreira de Carvalho AC. Rethinking Default Values: a Low Cost and Efficient Strategy to Define Hyperparameters. arXiv e-prints. 2020 Jul:arXiv-2008.

[0314] 2. Wistuba M, Schilling N, Schmidt-Thieme L (2015) Sequential model-free hyperparameter tuning. In: 2015 IEEE International Conference on Data Mining, pp 1033{1038, DOI 10.1109 / ICDM.2015.20

[0315] 3. Gijsbers P, Pfisterer F, van Rijn JN, Bischl B, Vanschoren J. Meta-learning for symbolic hyperparameter defaults. InProceedings of the Genetic and Evolutionary Computation Conference Companion 2021 Jul 7 (pp. 151-152).

[0316] 4. Pfisterer, F., van Rijn, J. N., Probst, P., Muller, A. C., & Bischl, B. (2021, July). Learning multiple defaults for machine learning algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (pp. 241-242).

[0317] 5. Basu, K., Jiang, C., Ouyang, Y. and Fleming, J., Microsoft Technology Licensing LLC, 2022. Large-scale automated hyperparameter tuning. U.S. Patent 11 ,392,859.

[0318] 6. Khodak, Mikhail, et al. "Federated hyperparameter tuning: Challenges, baselines, and connections to weight-sharing." Advances in Neural Information Processing Systems 34 (2021): 19184-19197.

[0319] 7. Khodak, Mikhail, et al. "Weight sharing for hyperparameter optimization in federated learning." Int. Workshop on Federated Learning for User Privacy and Data Confidentiality in Conjunction with ICML. Vol. 2020. 2020.

[0320] 8. Zhou, Yi, et al. "Flora: Single-shot hyper-parameter optimization for federated learning." arXiv preprint arXiv:2112.08524 (2021).

[0321] 9. Farooq, Hasan, et al. "Coordinated hyper-parameter search for edge machine learning in beyond-5g networks." 2021 IEEE International Conference on Communications Workshops (ICC Workshops). IEEE, 2021.

Claims

CLAIMS:

1. A computer-implemented method performed by a first node (111), the method being for handling hyperparameters corresponding to a plurality of sets of data, the first node (111) operating in a computer system (100), the method comprising:- obtaining (302) two or more first sets of data, out of the plurality of sets of data, the plurality of sets of data having been collected by a plurality of third nodes (113) operating in the computer system (100), each third node in the plurality of third nodes (113) having collected a set of data of the plurality of sets of data, wherein a plurality of clusters of sets of data have been determined to be formed by the plurality of sets of data, wherein each of the two or more first sets of data corresponds to a respective set of data in a center of a respective cluster of the plurality of clusters, wherein the plurality of clusters have been determined based on a similarity of respective statistical features of the sets of data in the plurality of sets of data, wherein a first number of the two or more first sets of data is smaller than a second number of sets of data in the plurality of sets of data,- determining (303) a respective hyperparameter for each obtained two or more first sets of data, and tuning the determined respective hyperparameters, the respective tuned hyperparameters being to train a respective machine learning model with a corresponding set of data of the plurality of sets of data, and- outputting (305) an indication indicating the determined two or more respective tuned hyperparameters.

2. The method according to claim 1 , wherein the determined respective tuned hyperparameter for each first set of data is to be used by each set of data in a same cluster.

3. The method according to any of claims 1-2, further comprising:- obtaining (301) a first indication from a second node (112) operating in the computer system (100), the first indication indicating which are the two or more first sets of data corresponding to the centers of the respective clusters of the plurality of clusters, wherein the obtaining (302) of the two or more first sets of data is responsive to the obtained first indication, and wherein the outputting (305) of the indication comprises sending the indication to the second node(112) as a second indication, responsive to the obtained first indication.

4. The method according to any of claims 1-3, further comprising:- selecting (304), prior to the tuning, one or more of the determined two or more respective hyperparameters based on one or more criteria, and wherein the output indication indicates the selected respective tuned hyperparameters.

5. A computer-implemented method performed by a second node (112), the method being for handling hyperparameters corresponding to a plurality of sets of data, the second node (112) operating in a computer system (100), the method comprising:- obtaining (209) one or more respective statistical features corresponding to a set of data having been collected by a fourth node (114) operating in the computer system (100),- determining (210), using a first machine learning model, a first cluster of sets of data corresponding to the obtained one or more respective statistical features, based on a first similarity of the one or more respective statistical features to respective statistical features of the sets of data in the determined first cluster, wherein the first cluster is one of a plurality of clusters of sets of data formed by the plurality of sets of data having been collected by a plurality of third nodes (113) operating in the computer system (100), each third node in the plurality of third nodes (113) having collected a set of data of the plurality of sets of data, wherein the plurality of clusters have been determined using the first machine learning model based on the similarity of the respective statistical features of the sets of data in the plurality of sets of data, and- providing (213), to the fourth node (114), a respective tuned hyperparameter corresponding to a respective first set of data in a first center of the determined first cluster.

6. The method according to claim 5, further comprising:- determining (204), using the first machine learning model, the plurality of clusters of sets of data,- determining (205) two or more first sets of data out of the plurality of sets of data, wherein each of the two or more first sets of data corresponds to a respective set of data in a center of a respective cluster of the plurality of clusters, wherein a first number of the two or more first sets of data is smaller than a second number of sets of data in the plurality,- providing (206) a first indication to a first node (111) operating in the computer system (100), the first indication indicating which are the two or more first setsof data corresponding to the centers of the respective clusters of the plurality of clusters, and- receiving (207) a second indication from the first node (111) responsive to the provided first indication, wherein the second indication indicates a respective tuned hyperparameter for each of the determined two or more first sets of data, the respective tuned hyperparameter being to train a respective second machine learning model with a corresponding set of data of the plurality of sets of data.

7. The method according to claim 6, wherein the indicated respective tuned hyperparameter for each first set of data is to be used by each set of data in the same cluster, wherein the sets of data in the plurality of sets of data have been collected by a plurality of third nodes (113) operating in the computer system (100), each third node in the plurality of third nodes (113) having collected a set of data of the plurality of sets of data, and wherein the method further comprises:- providing (208) a respective third indication to the respective third nodes in the plurality of third nodes (113) having collected the sets of data in the same cluster, the respective third indication indicating the respective tuned hyperparameter to be used by each set of data in the same cluster.

8. The method according to any of claims 5-7, wherein the received second indication indicates respective tuned hyperparameters that have been selected based on one or more criteria.

9. The method according to any of claims 5-8, further comprising:- obtaining (201) the respective statistical features of the sets of data in the plurality of sets of data, and- pre-processing (202, 303) the obtained respective statistical features, and wherein the determining (204) of the plurality of clusters is performed using the pre-processed respective statistical features.

10. The method according to any of claims 5-9, wherein the pre-processing (202, 303) of the obtained respective statistical features comprises:- vectorizing (202) the received respective statistical features to create a matrix of the respective statistical features for each third node of the plurality of third nodes (113), andscaling (203) the vectorized respective statistical features to a real value between 0 and 1.

11. The method according to any of claims 5-10, further comprising:- sending (211) a fourth indication to the fourth node (114), the fourth indication indicating the determined first cluster, and- receiving (212) a fifth indication from the fourth node (114) responsive to the sent fourth indication, the fifth indication indicating whether or not the fourth node (114) has accepted the determined first cluster, and wherein the providing (213) of the respective tuned hyperparameter is performed with the proviso the fourth node (114) has accepted the determined first cluster.

12. A computer-implemented method performed by a computer system (100), the method being for handling hyperparameters corresponding to a plurality of sets of data, the computer system (100) comprising a first node (111) and a second node (112), the method comprising:- obtaining (408), by the first node (111), two or more first sets of data, out of the plurality of sets of data, the plurality of sets of data having been collected by a plurality of third nodes (113) operating in the computer system (100), each third node in the plurality of third nodes (113) having collected a set of data of the plurality of sets of data, wherein a plurality of clusters of sets of data have been determined to be formed by the plurality of sets of data, wherein each of the two or more first sets of data corresponds to a respective set of data in a center of a respective cluster of the plurality of clusters, wherein the plurality of clusters have been determined based on a similarity of respective statistical features of the sets of data in the plurality of sets of data, wherein a first number of the two or more first sets of data is smaller than a second number of sets of data in the plurality of sets of data,- determining (409), by the first node (111), a respective hyperparameter for each obtained two or more first sets of data, and tuning the determined respective hyperparameters, the respective tuned hyperparameters being to train a respective second machine learning model with a corresponding set of data of the plurality of sets of data,- outputting (411), by the first node (111), a second indication indicating the determined two or more respective tuned hyperparameters to the second node (112),- receiving (412), by the second node (112), the second indication from the first node (111),- obtaining (414), by the second node (112), one or more respective statistical features corresponding to a set of data having been collected by a fourth node (114) operating in the computer system (100),- determining (415), by the second node (112), using a first machine learning model, a first cluster of sets of data corresponding to the obtained one or more respective statistical features, based on a first similarity of the one or more respective statistical features to respective statistical features of the sets of data in the determined first cluster, wherein the first cluster is one of the plurality of clusters of sets of data formed by the plurality of sets of data, and- providing (418), by the second node (112) and based on the received second indication, to the fourth node (114), the respective tuned hyperparameter corresponding to a respective first set of data in a first center of the determined first cluster.

13. The method according to claim 12, further comprising:- determining (404), by the second node (112), using the first machine learning model, the plurality of clusters of sets of data,- determining (405), by the second node (112), the two or more first sets of data,- providing (406), by the second node (112), a first indication to a first node (111) operating in the computer system (100), the first indication indicating which are the two or more first sets of data corresponding to the centers of the respective clusters of the plurality of clusters, wherein the receiving (412) of the second indication is responsive to the provided first indication, and- obtaining (407), by the first node (111), the first indication from the second node (112), wherein the obtaining (408) of the two or more first sets of data is responsive to the obtained first indication, and wherein the outputting (411) of the second indication comprises sending the second indication responsive to the obtained first indication.

14. The method according to claim 13, wherein the indicated respective tuned hyperparameter for each first set of data is to be used by each set of data in the same cluster, and wherein the method further comprises:- providing (413), by the second node (112), a respective third indication to the respective third nodes in the plurality of third nodes (113) having collected the sets of data in the same cluster, the respective third indication indicating therespective tuned hyperparameter to be used by each set of data in the same cluster.

15. The method according to any of claims 12-14, further comprising:- selecting (410), by the first node (111), prior to the tuning, one or more of the determined two or more respective hyperparameters based on one or more criteria, and wherein the output second indication indicates the selected respective tuned hyperparameters.

16. The method according to any of claims 12-15, further comprising:- obtaining (401), by the second node (112), the respective statistical features of the sets of data in the plurality of sets of data, and- pre-processing (402, 403), by the second node (112), the obtained respective statistical features, and wherein the determining (404) of the plurality of clusters is performed using the pre-processed respective statistical features.

17. The method according to any of claims 12-16, wherein the pre-processing (402, 403) of the obtained respective statistical features comprises:- vectorizing (402), by the second node (112), the received respective statistical features to create a matrix of the respective statistical features for each third node of the plurality of third nodes (113), and- scaling (403), by the second node (112), the vectorized respective statistical features to a real value between 0 and 1 .

18. The method according to any of claims 12-17, further comprising:- sending (416), by the second node (112), a fourth indication to the fourth node (114), the fourth indication indicating the determined first cluster, and- receiving (417), by the second node (112), a fifth indication from the fourth node (114) responsive to the sent fourth indication, the fifth indication indicating whether or not the fourth node (114) has accepted the determined first cluster, and wherein the providing (418) of the respective tuned hyperparameter is performed with the proviso the fourth node (114) has accepted the determined first cluster.

19. A first node (111 for handling hyperparameters corresponding to a plurality of sets of data, the first node (111) being configured to operate in a computer system (100), the first node (111) being further configured to:- obtain two or more first sets of data, out of the plurality of sets of data, the plurality of sets of data being configured to have been collected by a plurality of third nodes (113) configured to operate in the computer system (100), each third node in the plurality of third nodes (113) being configured to have collected a set of data of the plurality of sets of data, wherein a plurality of clusters of sets of data is configured to have been determined to be formed by the plurality of sets of data, wherein each of the two or more first sets of data is configured to correspond to a respective set of data in a center of a respective cluster of the plurality of clusters, wherein the plurality of clusters are configured to have been determined based on a similarity of respective statistical features of the sets of data in the plurality of sets of data, wherein a first number of the two or more first sets of data is configured to be smaller than a second number of sets of data in the plurality of sets of data,- determine a respective hyperparameter for each of the two or more first sets of data configured to be obtained, and tune the determined respective hyperparameters, the respective tuned hyperparameters being configured be to train a respective machine learning model with a corresponding set of data of the plurality of sets of data, and- output an indication configured to indicate the two or more respective tuned hyperparameters configured to be determined.

20. The first node (111) according to claim 19, wherein the respective tuned hyperparameter configured to be determined for each first set of data is to be used by each set of data in a same cluster.

21. The first node (111) according to any of claims 19-20, being further configured to:- obtain a first indication from a second node (112) configured to operate in the computer system (100), the first indication being configured to indicate which are the two or more first sets of data corresponding to the centers of the respective clusters of the plurality of clusters, wherein the obtaining of the two or more first sets of data is configured to be responsive to the first indication configured to be obtained, and wherein the outputting of the indication is configured to comprise sending the indication to the second node (112) as a second indication, responsive to the first indication configured to be obtained.

22. The first node (111) according to any of claims 10-21, being further configured to:- select, prior to the tuning, one or more of the two or more respective hyperparameters configured to be determined based on one or more criteria, and wherein the output indication is configured to indicate the respective tuned hyperparameters configured to be selected.

23. A second node (112), for handling hyperparameters corresponding to a plurality of sets of data, the second node (112) being configured to operate in a computer system (100), the second node (112) being further configured to:- obtain one or more respective statistical features corresponding to a set of data configured to have been collected by a fourth node (114) configured to operate in the computer system (100),- determine, using a first machine learning model, a first cluster of sets of data corresponding to the one or more respective statistical features configured to be obtained, based on a first similarity of the one or more respective statistical features to respective statistical features of the sets of data in the first cluster configured to be determined, wherein the first cluster is one of a plurality of clusters of sets of data formed by the plurality of sets of data configured to have been collected by a plurality of third nodes (113) configured to operate in the computer system (100), each third node in the plurality of third nodes (113) being configured to have collected a set of data of the plurality of sets of data, wherein the plurality of clusters are configured to have been determined using the first machine learning model based on the similarity of the respective statistical features of the sets of data in the plurality of sets of data, and- provide, to the fourth node (114), a respective tuned hyperparameter corresponding to a respective first set of data in a first center of the first cluster configured to be determined.

24. The second node (112) according to claim 23, being further configured to:- determine, using the first machine learning model, the plurality of clusters of sets of data,- determine two or more first sets of data out of the plurality of sets of data, wherein each of the two or more first sets of data is configured to correspond to a respective set of data in a center of a respective cluster of the plurality of clusters, wherein a first number of the two or more first sets of data is configured to be smaller than a second number of sets of data in the plurality,- provide a first indication to a first node (111) configured to operate in the computer system (100), the first indication being configured to indicate which are the two or more first sets of data corresponding to the centers of the respective clusters of the plurality of clusters, and- receive a second indication from the first node (111) responsive to the first indication configured to be provided, wherein the second indication is configured to indicate a respective tuned hyperparameter for each of the two or more first sets of data configured to be determined, the respective tuned hyperparameter being configured to be to train a respective second machine learning model with a corresponding set of data of the plurality of sets of data.

25. The second node (112) according to claim 24, wherein the respective tuned hyperparameter configured to be indicated for each first set of data is to be used by each set of data in the same cluster, wherein the sets of data in the plurality of sets of data are configured to have been collected by a plurality of third nodes (113) configured to operate in the computer system (100), each third node in the plurality of third nodes (113) being configured to have collected a set of data of the plurality of sets of data, and wherein the second node (112) is further configured to:- provide a respective third indication to the respective third nodes in the plurality of third nodes (113) configured to have collected the sets of data in the same cluster, the respective third indication being configured to indicate the respective tuned hyperparameter to be used by each set of data in the same cluster.

26. The second node (112) according to any of claims 23-25, wherein the second indication configured to be received is configured to indicate respective tuned hyperparameters that are configured to have been selected based on one or more criteria.

27. The second node (112) according to any of claims 23-26, being further configured to:- obtain the respective statistical features of the sets of data in the plurality of sets of data, and- pre-process (202, 303) the respective statistical features configured to be obtained, and wherein the determining of the plurality of clusters is configured to be performed using the respective statistical features configured to be pre- processed.

28. The second node (112) according to any of claims 23-27, wherein the pre-processing (202, 303) of the respective statistical features configured to be obtained is configured to comprise:- vectorizing the received respective statistical features to create a matrix of the respective statistical features for each third node of the plurality of third nodes(113), and- scaling the vectorized respective statistical features to a real value between 0 and 1.

29. The second node (112) according to any of claims 23-28, being further configured to:- send a fourth indication to the fourth node (114), the fourth indication being configured to indicate the first cluster configured to be determined, and- receive a fifth indication from the fourth node (114) responsive to the fourth indication configured to be sent, the fifth indication being configured to indicate whether or not the fourth node (114) has accepted the first cluster configured to be determined, and wherein the providing of the respective tuned hyperparameter is configured to be performed with the proviso the fourth node(114) has accepted the first cluster configured to be determined.

30. A computer system (100), for handling hyperparameters corresponding to a plurality of sets of data, the computer system (100) comprising a first node (111) and a second node (112), the computer system (100) being configured to:- obtain, by the first node (111), two or more first sets of data, out of the plurality of sets of data, the plurality of sets of data being configured to have been collected by a plurality of third nodes (113) configured to operate in the computer system (100), each third node in the plurality of third nodes (113) being configured to have collected a set of data of the plurality of sets of data, wherein a plurality of clusters of sets of data is configured to have been determined to be formed by the plurality of sets of data, wherein each of the two or more first sets of data corresponds to a respective set of data in a center of a respective cluster of the plurality of clusters, wherein the plurality of clusters are configured to have been determined based on a similarity of respective statistical features of the sets of data in the plurality of sets of data, wherein a first number of the two or more first sets of data is configured to be smaller than a second number of sets of data in the plurality of sets of data,- determine, by the first node (111), a respective hyperparameter for each of the two or more first sets of data configured to be obtained, and tune thedetermined respective hyperparameters, the respective tuned hyperparameters being configured to be to train a respective second machine learning model with a corresponding set of data of the plurality of sets of data,- output, by the first node (111), a second indication configured to indicate the two or more respective tuned hyperparameters configured to be determined to the second node (112),- receive, by the second node (112), the second indication from the first node (111),- obtain, by the second node (112), one or more respective statistical features corresponding to a set of data configured to have been collected by a fourth node (114) configured to operate in the computer system (100),- determine, by the second node (112), using a first machine learning model, a first cluster of sets of data corresponding to the one or more respective statistical features configured to have been obtained, based on a first similarity of the one or more respective statistical features to respective statistical features of the sets of data in the first cluster configured to be determined, wherein the first cluster is configured to be one of the plurality of clusters of sets of data formed by the plurality of sets of data, and- provide, by the second node (112) and based on the second indication configured to be received, to the fourth node (114), the respective tuned hyperparameter corresponding to a respective first set of data in a first center of the first cluster configured to be determined.

31. The computer system (100) according to claim 30, being further configured to:- determine, by the second node (112), using the first machine learning model, the plurality of clusters of sets of data,- determine, by the second node (112), the two or more first sets of data,- provide, by the second node (112), a first indication to a first node (111) configured to operate in the computer system (100), the first indication being configured to indicate which are the two or more first sets of data corresponding to the centers of the respective clusters of the plurality of clusters, wherein the receiving of the second indication is configured to be responsive to the first indication configured to be provided, and- obtain, by the first node (111), the first indication from the second node (112), wherein the obtaining of the two or more first sets of data is configured to be responsive to the first indication configured to be obtained, and wherein theoutputting of the second indication is configured to comprise sending the second indication responsive to the first indication configured to be obtained.

32. The computer system (100) according to claim 31 , wherein the respective tuned hyperparameter configured to be indicated for each first set of data is to be used by each set of data in the same cluster, and wherein the computer system (100) is further configured to:- provide, by the second node (112), a respective third indication to the respective third nodes in the plurality of third nodes (113) configured to have collected the sets of data in the same cluster, the respective third indication being configured to indicate the respective tuned hyperparameter to be used by each set of data in the same cluster.

33. The computer system (100) according to any of claims 30-32, being further configured to:- select, by the first node (111), prior to the tuning, one or more of the two or more respective hyperparameters configured to be determined based on one or more criteria, and wherein the output second indication is configured to indicate the respective tuned hyperparameters configured to be selected.

34. The computer system (100) according to any of claims 30-33, being further configured to:- obtain, by the second node (112), the respective statistical features of the sets of data in the plurality of sets of data, and- pre-process, by the second node (112), the respective statistical features configured to be obtained, and wherein the determining of the plurality of clusters is configured to be performed using the respective statistical features configured to be pre-processed.

35. The computer system (100) according to any of claims 30-34, wherein the preprocessing of the respective statistical features configured to be obtained is configured to comprise:- vectorizing, by the second node (112), the respective statistical features configured to be received to create a matrix of the respective statistical features for each third node of the plurality of third nodes (113), and- scaling, by the second node (112), the vectorized respective statistical features to a real value between 0 and 1.

36. The computer system (100) according to any of claims 30-35, being further configured to:- send, by the second node (112), a fourth indication to the fourth node (114), the fourth indication being configured to indicate the first cluster configured to be determined, and- receive, by the second node (112), a fifth indication from the fourth node (114) responsive to the sent fourth indication, the fifth indication being configured to indicate whether or not the fourth node (114) has accepted the first cluster configured to be determined, and wherein the providing of the respective tuned hyperparameter is configured to be performed with the proviso the fourth node (114) has accepted the first cluster configured to be determined.

Images