Anomaly detection using digital twin of personalized federated learning with embedded driver

Abstract

The invention relates to anomaly detection using digital twinning of personalized federated learning with embedded drivers. A client device configured to provide a user service and establish communication with a server device is described, the client device comprising: at least one processor; at least one memory storing instructions that, when executed by the at least one processor, cause the client device to: receive, from the server device, a plurality of models for estimating at least one parameter associated with the user service; for each received model, apply user service related data at the client device to a model evaluation function and determine, for each model, a corresponding model evaluation value indicative of a performance of each model in providing an estimate for the at least one parameter associated with the user service; based on all model evaluation values, obtain, from the server device, an indicator of a selected model; train and / or update the selected model based on the user service related data; and send, to the server device, trained and / or updated model related information for the server device to update the selected model.

Description

Technical Field

[0001] This disclosure relates to machine learning techniques in distributed environments, and more particularly to digital twin techniques using machine learning, and more specifically to federated learning techniques applied to digital twins. Background Technology

[0002] Any discussion of the background art in this specification should not be regarded as an admission that such art is widely known or forms part of general common sense in the field.

[0003] Data used in machine learning (ML) typically comes from distributed sources. Furthermore, due to privacy, bandwidth, legal, or policy restrictions, the data may not be movable, for example: Data generated by IoT devices, mobile phones, or digital twins that cannot leave the device or a specific domain / network; Data generated by network components (e.g., routers) that cannot move across national / jurisdictional boundaries; and The data is generated by the hospital and cannot leave the hospital system.

[0004] Therefore, ML algorithms need to be trained on distributed data without moving the data to a single central point. Thus, the goal is to train models on distributed data while still gaining the advantages of centralized training systems.

[0005] The unique challenges that ML faces in distributed environments include: Privacy: Data owners do not want their data to leave their devices / networks; that is, data is not allowed to be moved to the servers on which it is used for training. Data scarcity: Training a model on a small dataset from a single device is insufficient; in other words, training a model solely on the client side cannot provide sufficiently satisfactory results.

[0006] One example of applying machine learning to a distributed environment is digital twins (DT). The increasing popularity of DTs is due to their unique ability to replicate the behavior, characteristics, and performance of their physical counterparts in real time, enabling continuous monitoring, in-depth analysis, and optimization of complex systems.

[0007] DT is typically provided with machine learning models for corresponding client devices to compute loss based on output and input and perform desired tasks (e.g., determining whether the input is anomalous in an unsupervised setting, or predicting class labels in a supervised setting).

[0008] A key application of Data Transmission (DT) is anomaly detection. By leveraging real-time data from sensors, network components, and IoT devices, DTs can compare this data to predefined baselines or expected behavior, identifying anomalies when deviations occur. Furthermore, they can recognize patterns in data collected over time and detect anomalies when these patterns are broken, triggering automated alarms that ensure timely attention from owners, operators, or maintenance personnel.

[0009] However, building effective machine learning models for anomaly detection using data points (DTs) typically requires large amounts of efficient training data. Relying solely on data from a single DT from a single machine or smart home may be insufficient, as it may fail to capture the full range of potential anomalies or edge cases. This underscores the importance of diverse and comprehensive datasets, along with advanced analytics and machine learning techniques, to unlock the full potential of DTs in anomaly detection.

[0010] To address the aforementioned challenges, data from distributed sources (e.g., multiple data points (DTs) from similar physical entities in different settings) can be combined to form a more comprehensive dataset. For example, a dataset from all distributed sources (e.g., DTs from all smart homes in a region or town) can provide a more robust and representative sample. However, this approach also raises two significant issues: 1. Privacy: Much of the data collected by distributed sources (e.g., DT) can be sensitive and private, making it difficult to share or aggregate without revealing personal privacy. 2. Operational Variability: Even if distributed sources (e.g., DTs) are similar physical entities, differences in operational settings, usage patterns, or environmental conditions can cause variations in baselines, such as anomaly definitions. For example, behavior considered normal for one entity might be anomalous for another, making the development of a unified anomaly detection model challenging.

[0011] To overcome these challenges, solutions must be developed that balance data sharing and privacy issues while taking into account operational variability.

[0012] Specifically, there is a need to provide a machine learning approach in a distributed environment that offers improved privacy protection for local data at the client site, as well as improved training for ML models, taking into account the differences between client devices and / or between local data from those client devices. Summary of the Invention

[0013] According to a first aspect of the present invention, a client device is provided, configured to provide user services and establish communication with a server device, the server device being configured to provide digital twin machine learning (ML) model training services to the client device, the client device comprising: At least one processor, and At least one memory storing instructions that, when executed by at least one processor, cause the client device to at least: Receive multiple models from the server device, which are used to provide an estimate of at least one parameter associated with the user service; For each of the received multiple models, data associated with the user service at the client device is applied to the model evaluation function, and a corresponding model evaluation value is determined for each model, which is used to indicate the performance of each model in providing an estimate of the at least one parameter associated with the user service. Obtain an indicator from the server device to indicate the model selected by the server device from among the received multiple models based on all model evaluation values ​​determined for the multiple received models; The selected model is trained and / or updated based on the data at the client device associated with the user service; and Send information about the selected model that has been trained and / or updated to the server device so that the server device can further update the selected model.

[0014] In some embodiments, the client device is also configured to acquire the selected model via: Send a model evaluation vector to the server device, the model evaluation vector including multiple determined model evaluation values ​​for the received multiple models; and Receive an indicator from the server device to indicate the model selected by the server device from among multiple models received for the client device based on the model evaluation vector.

[0015] In some embodiments, the client device is also configured to acquire the selected model via: For each model, a corresponding model selection preference probability is determined, wherein the corresponding model selection preference probability is determined based on applying a pre-configured probability density function to the determined model evaluation value for each model; The model evaluation vector also includes model selection preference probabilities determined for multiple models, allowing the server device to select one model from among them; and Receive the selected model from the server device.

[0016] In some embodiments, the client device is also configured to train and / or update the selected model by: Update the value of at least one model parameter of the selected model based on the data at the client device; and Send the updated value of at least one model parameter of the selected model to the server device.

[0017] In some embodiments, the client device is also configured to train the selected model using a gradient descent optimization method.

[0018] In some embodiments, the client device is also configured to train and / or update the selected model by: For the selected model, determine the updated gradient values ​​of the model evaluation function; and Send updated gradient values ​​of the model evaluation function for the selected model to the server device.

[0019] In some embodiments, the client device is configured to perform, in the current iteration, the steps according to the first aspect and any of the foregoing related examples. The multiple models received are updated by the server device in the previous iteration immediately preceding the current iteration. The client device is also configured to receive multiple models from the server device that have been further updated by the server device in the current iteration in the next iteration immediately following the current iteration. The client device is also configured to repeat the steps according to any one of the first aspects and the related examples above in the next iteration for multiple models.

[0020] In some embodiments, the model evaluation function is a loss function that indicates the loss when the model is applied to provide an estimate of at least one parameter associated with a user service.

[0021] In some embodiments, for a model, a smaller loss value given by the loss function indicates a higher probability of the model choosing a preference, given by a pre-configured probability density function.

[0022] According to a second aspect of this disclosure, a server device is provided, configured to establish communication with a plurality of client devices for providing digital twin machine learning (ML) model training services to the plurality of client devices, including: At least one processor, and At least one memory, the memory storing instructions that, when executed by at least one processor, cause the server device to at least: Multiple models are sent to a subset of multiple client devices. These models are used to train and / or update to provide estimates of at least one parameter associated with the user service. Receive a corresponding model evaluation vector from each client device included in the subset, wherein the corresponding model evaluation vector includes a corresponding model evaluation value for each of a plurality of models, the corresponding model evaluation value indicating the performance of each model in providing an estimate of at least one parameter associated with the user service; Based on all evaluation vectors received from the subset of client devices, for each client device included in the subset, a corresponding model is selected from multiple models; Send an indicator to each client device included in the subset, the indicator being used to indicate which of the multiple models sent is selected by the server device for at least one client device accordingly; Receive information from each client device included in the subset related to the correspondingly selected model, which is locally trained and / or updated by each client device included in the subset; and The selected model for client devices included in the subset is updated centrally based on the information.

[0023] In some embodiments, the server device is also configured to select a corresponding model for each client device included in the subset by means of the following: Based on clustering, all model evaluation vectors received from the subset of client devices are clustered into multiple groups by the client devices included in that subset, wherein each client device included in the subset is clustered into a group based on a vector clustering method; and Select the corresponding model for each group.

[0024] In some examples, the server device is also configured to select the appropriate model for each group by matching each group with the corresponding model one-to-one based on a matching method.

[0025] In some examples, the information includes the value of at least one model parameter and / or the gradient value of the model evaluation function.

[0026] In some examples, the server device is also configured to centrally update the selected model for client devices included in the subset via the following: For each group, the values ​​included in the information are averaged, wherein preferably the weight of the client device in each group in the average corresponds to the amount of data at the client device in each group.

[0027] In some examples, the server device is also configured as follows: Based on the updated gradient value of the model evaluation function used for the correspondingly selected model, the updated value of at least one model parameter of the correspondingly selected model is determined.

[0028] In some embodiments, the server device is also configured to randomly initialize multiple models.

[0029] In some embodiments, the server device is configured to perform, in the current iteration, the steps according to the second aspect and any of the foregoing related examples. The multiple models received are updated by the server device in the previous iteration immediately preceding the current iteration. The server device is also configured to send multiple models updated by the server device in the current iteration to client devices included in another subset in the next iteration immediately following the current iteration. The server device is also configured to repeat the steps according to the second aspect and any of the related examples above for another subset of client devices in the next iteration.

[0030] In some examples, the server device is configured to stop the iteration when the clustering result has not been modified after a predetermined number of iterations, wherein in each of the predetermined number of iterations, the loss of the selected model for the corresponding client device is lower than a pre-configured threshold.

[0031] According to a third aspect of this disclosure, a system is provided for providing digital twin machine learning (ML) model training services to a plurality of client devices configured to provide user services, the system comprising a plurality of client devices according to any one of the first aspect and related examples, and at least one server device according to any one of the second aspect and related examples.

[0032] According to a fourth aspect of this disclosure, a method is provided for a client device configured to provide user services and establish communication with a server device, the server device being used to provide a digital twin machine learning (ML) model training service to the client device, the method comprising: Receive multiple models from the server device, which are used to provide an estimate of at least one parameter associated with the user service; For each of the received multiple models, data associated with the user service at the client device is applied to the model evaluation function, and a corresponding model evaluation value is determined for each model, which is used to indicate the performance of each model in providing an estimate of the at least one parameter associated with the user service. Obtain an indicator from the server device to indicate the model selected by the server device from among the received multiple models based on all model evaluation values ​​determined for the multiple received models; The selected model is trained and / or updated based on the data at the client device associated with the user service; and Send information about the selected model that has been trained and / or updated to the server device so that the server device can further update the selected model.

[0033] According to a fifth aspect of this disclosure, a method for a server device is provided, the server device being configured to establish communication with a plurality of client devices for providing a digital twin machine learning (ML) model training service to the plurality of client devices, the method comprising: Multiple models are sent to a subset of multiple client devices. These models are used to train and / or update to provide estimates of at least one parameter associated with the user service. Receive a corresponding model evaluation vector from each client device included in the subset, wherein the corresponding model evaluation vector includes a corresponding model evaluation value for each of a plurality of models, the corresponding model evaluation value indicating the performance of each model in providing an estimate of at least one parameter associated with the user service; Based on all evaluation vectors received from the subset of client devices, for each client device included in the subset, a corresponding model is selected from multiple models; Send an indicator to each client device included in the subset, the indicator being used to indicate which of the multiple models sent is selected by the server device for at least one client device accordingly; Receive information from each client device included in the subset related to the correspondingly selected model, which is locally trained and / or updated by each client device included in the subset; and The selected model for client devices included in the subset is updated based on the information.

[0034] According to a sixth aspect of this disclosure, a system method is provided for providing a digital twin machine learning (ML) model training service to a plurality of client devices configured to provide user services. The system includes a plurality of client devices and at least one server device. The method includes steps according to a fourth aspect performed by the plurality of client devices and / or steps according to a fifth aspect performed by the at least one server device.

[0035] According to a seventh aspect of the invention, a computer program is provided, the computer program including instructions for causing a device to perform the method according to the fourth aspect or for causing the device to perform the method according to the fifth aspect.

[0036] According to a seventh aspect of the invention, a memory is provided that stores computer-readable instructions for causing a device to perform the method according to the fourth aspect or for causing the device to perform the method according to the fifth aspect.

[0037] Additionally, according to some other example embodiments, for instance, a computer program product for a wireless communication device is provided, the wireless communication device including at least one processor, the at least one processor including software code portions for performing corresponding steps disclosed herein when the product is run on the device. The computer program product may include a computer-readable medium on which the software code portions are stored. Furthermore, the computer program product may be directly loadable into a computer's internal memory and / or transmittable via a network through at least one of the processes of uploading, downloading, and pushing.

[0038] While some exemplary embodiments will be described herein with reference to the above application, it should be understood that this disclosure is not limited to such areas of use and is applicable to a broader context.

[0039] It is worth noting that the methods according to this disclosure relate to methods operating the apparatus according to the above-described exemplary embodiments and variations thereof, and the corresponding statements made for the apparatus also apply to the corresponding methods, and vice versa, so that similar descriptions may be omitted for brevity. Furthermore, even if not explicitly disclosed, the above aspects can be combined in various ways. Those skilled in the art will understand that these combinations of aspects and features / steps are possible unless an explicit contradiction arises.

[0040] Implementations of the disclosed device may include, but are not limited to, the use of one or more processors, one or more application-specific integrated circuits (ASICs), and / or one or more field-programmable gate arrays (FPGAs). Implementations of the device may also include the use of other conventional and / or custom hardware, such as software-programmable processors, such as graphics processing unit (GPU) processors.

[0041] Other and additional exemplary embodiments of this disclosure will become apparent during the process discussed below and by reference to the accompanying drawings. Attached Figure Description

[0042] Exemplary embodiments of this disclosure will now be described by way of example only with reference to the accompanying drawings, in which:

[0043] Figure 1An example of a typical federated learning architecture is illustrated schematically;

[0044] Figure 2 An example of a typical personalized federated learning architecture is illustrated schematically.

[0045] Figure 3 An example of an ML method applied to a client device in a distributed environment according to an exemplary embodiment of the present disclosure is illustrated;

[0046] Figure 4 An example of an ML method applied to a server device in a distributed environment according to an exemplary embodiment of the present disclosure is illustrated.

[0047] Figure 5 An example of an ML method applied to a distributed environment according to an exemplary embodiment of the present disclosure is illustrated schematically;

[0048] Figure 6 An example of a matching method for assigning client clusters to corresponding models according to an exemplary embodiment of this disclosure is illustrated;

[0049] Figure 7 The illustration schematically depicts an example of an ML method applied to a distributed environment according to exemplary embodiments of the present disclosure; and

[0050] Figure 8 An example of a reasoning method according to an exemplary embodiment of this disclosure is illustrated schematically. Detailed Implementation

[0051] In the following description, various exemplary embodiments will be used as examples of communication network architectures based on 3GPP standards for communication networks (such as 5G / NR (New Radio)) as examples of communication networks to which such embodiments may be applied, but the embodiments are not limited to such architectures. It will be apparent to those skilled in the art that the embodiments can also be applied to other types of communication networks in which mobile communication principles are integrated with D2D (device-to-device) or V2X (vehicle-to-everything) configurations (such as SL (sidelinks), e.g., Wi-Fi, Global Microwave Access Interoperability (WiMAX), Bluetooth®, Personal Communication Services (PCS), ZigBee®, Wideband Code Division Multiple Access (WCDMA), systems using Ultra Wideband (UWB) technology, Mobile Ad Hoc Networks (MANET), wired access, etc.). Furthermore, without loss of generality, while the description of some exemplary embodiments relates to mobile communication networks, the principles of this disclosure can be extended and applied to any other type of communication network, such as wired communication networks.

[0052] The following examples and embodiments should be understood as illustrative examples only. Although the specification may refer to "a," "an," or "some" examples or embodiments in several places, this does not necessarily mean that every such reference relates to the same example(s) or the same embodiment(s), or that the feature applies only to a single example or embodiment. Individual features of different embodiments may also be combined to provide other embodiments. Furthermore, terms such as "comprising" and "including" should not be construed as limiting the embodiments to include only those features that have been mentioned; such examples and embodiments may also include features, structures, units, modules, etc., that have not yet been specifically mentioned.

[0053] The basic system architecture of a (telecommunications) communication network (including mobile communication systems, some examples of which are applicable) may include the architecture of one or more communication networks, which includes (multiple) radio access network subsystems and (multiple) core networks. Such an architecture may include one or more communication network control elements or functions, access network elements, radio access network elements, access service network gateways, or base transceiver stations, such as base stations (BS), access points (AP), NodeBs (NBs), eNBs or gNBs, distributed units (DUs) or centralized / central units (CUs), which control corresponding coverage areas or (multiple) cells, and one or more communication stations (such as communication elements or functions, such as user equipment or terminal equipment, such as user equipment (UE), or another device with similar functionality, such as modem chipsets, chips, modules, etc.), which may also be part of a communication-enabled station, element, function, or application, such as a UE, an element or function usable in a machine-to-machine communication architecture, or attached as a separate element to such a communication-enabled element, function, or application, capable of communicating with it via one or more channels via one or more communication beams for transmitting several types of data in multiple access domains. In addition, core network elements or network functions (such as gateway network elements / functions, mobility management entities, mobile switching centers, servers, databases, etc.) may be included.

[0054] The following description provides further details on alternatives, modifications, and differences: gNB includes, for example, a node that provides NR user plane and control plane protocol termination to the UE and is connected to 5GC via the NG interface, for example, according to Part 3.2 of 3GPP TS 38.300 V16.6.0 (2021-06) incorporated by reference.

[0055] The gNB Central Unit (gNB-CU) includes logical nodes that host, for example, the RRC, SDAP, and PDCP protocols of the gNB, or the RRC and PDCP protocols of the en-gNB, for example, to control the operation of one or more gNB-DUs. The gNB-CU terminates the F1 interface connected to the gNB-DU.

[0056] A gNB Distributed Unit (gNB-DU) includes, for example, a logical node that hosts the RLC, MAC, and PHY layers of a gNB or en-gNB, and its operation is partially controlled by the gNB-CU. One gNB-DU supports one or more cells. A cell is supported by only one gNB-DU. The gNB-DU terminates the F1 interface connected to the gNB-CU.

[0057] The gNB-CU control plane (gNB-CU-CP) includes, for example, logical nodes that host the control plane portions of the gNB-CU's RRC and PDCP protocols, for example, for en-gNB or gNB. The gNB-CU-CP terminates the E1 interface connected to the gNB-CU-UP and the F1-C interface connected to the gNB-DU.

[0058] The gNB-CU-User Plane (gNB-CU-UP) includes, for example, a logical node that hosts, for example, the user plane portion of the PDCP protocol for the gNB-CU of the en-gNB, and the user plane portions of the PDCP and SDAP protocols for the gNB-CU of the gNB. The gNB-CU-UP terminates the E1 interface connected to the gNB-CU-CP and the F1-U interface connected to the gNB-DU, for example, according to section 3.1 of 3GPP TS 38.401 V16.6.0 (2021-07) incorporated by reference.

[0059] There can be different functional divisions between the central unit and the distributed units, for example, referred to as options: Option 1 (Class 1A split): This option's functional division is similar to the 1A architecture in a DC. RRC is in the central unit. PDCP, RLC, MAC, physical layer, and RF are in the distributed units. Option 2 (3C category split): This option's functional division is similar to the 3C architecture in a DC (Distributed Control) system. RRC and PDCP are in the central unit. RLC, MAC, physical layer, and RF are in the distributed units. Option 3 (Split within RLC): Low RLC (part of the RLC functionality), MAC, physical layer, and RF are located in the distributed unit. PDCP and high RLC (another part of the RLC functionality) are located in the central unit. Option 4 (RLC-MAC partitioning): MAC, physical layer, and RF are located in the distributed unit. PDCP and RLC are located in the central unit. Or otherwise, for example, according to section 11 of 3GPP TR 38.801 V14.0.0 (2017-03) incorporated by reference.

[0060] gNB supports different protocol layers, such as Layer 1 (L1) – the physical layer.

[0061] NR's Layer 2 (L2) is divided into the following sublayers: Media Access Control (MAC), Radio Link Control (RLC), Packet Data Convergence Protocol (PDCP), and Service Data Adaptation Protocol (SDAP), among which: The physical layer provides the transmission channel to the MAC sublayer; The MAC sublayer supplies logical channels to the RLC sublayer; The RLC sublayer supplies RLC channels to the PDCP sublayer; The PDCP sublayer supplies radio bearers to the SDAP sublayer; The SDAP sublayer supplies QoS flows to 5GC; "Comp." refers to header compression, and "Segm." refers to segmentation; The control channels include (BCCH, PCCH).

[0062] Layer 3 (L3) includes, for example, Radio Resource Control (RRC), as per section 6 of 3GPP TS38.300 V16.6.0 (2021-06) incorporated by reference.

[0063] RAN (Radio Access Network) nodes or network nodes, such as gNBs, base stations, gNB CUs or gNB DUs or parts thereof, may be implemented using means, for example, having at least one processor and / or at least one memory (with computer-readable instructions (computer program)) configured to support and / or provide and / or process CU and / or DU related functions and / or features, and / or at least one protocol (sub) layer of the RAN (Radio Access Network), such as layer 2 and / or layer 3.

[0064] The gNB CU and gNB DU portions may, for example, be co-located or physically separate. The gNB DU can even be further divided, for example, split into two parts, such as one part including processing equipment and the other part including antennas. The Central Unit (CU) may also be referred to as BBU / REC / RCC / C-RAN / V-RAN, O-RAN, or a portion thereof. The Distributed Unit (DU) may also be referred to as RRH / RRU / RE / RU, or a portion thereof. In the various exemplary embodiments of this disclosure below, the CU-CP (or more generally, the CU) may also be referred to as a (first) network node supporting Central Unit control plane functions or a Layer 3 protocol of the Radio Access Network; similarly, the DU may be referred to as a (second) network node supporting at least one Layer 2 protocol of the Distributed Unit or the Radio Access Network.

[0065] gNB-DU supports one or more cells and can therefore be used as a serving cell, for example, for user equipment (UE).

[0066] User equipment (UE) may include wireless or mobile devices, devices with a wireless interface for interacting with a RAN (Radio Access Network), smartphones, in-vehicle devices, IoT devices, M2M devices, etc. Such a UE or device may include: at least one processor; and at least one memory, including computer program code; wherein the at least one memory and the computer program code are configured, together with the at least one processor, to cause the device to perform at least certain operations, such as, for example, an RRC connection to the RAN. The UE may be configured, for example, to generate messages (e.g., including a cell ID) to be transmitted to the RAN via radio (to reach and communicate with the serving cell). The UE may generate, transmit, and receive RRC messages containing one or more RRC PDUs (Packet Data Units).

[0067] The UE may have different states (e.g., according to 3GPP TS 38.331 V16.5.0 (2021-06) Parts 42.1 and 4.4, which are incorporated herein by reference).

[0068] For example, when an RRC connection has been established, the UE is in the RRC_CONNECTED state or the RRC_INACTIVE state.

[0069] In the RRC_CONNECTED state, the UE can: Store the AS context; Transmit / transmit unicast data to / from the UE; Monitor the control channel associated with the shared data channel to determine whether data has been scheduled for that data channel; Provides channel quality and feedback information; Perform neighboring cell measurements and report the measurements.

[0070] The RRC protocol includes the following main functions: RRC connection control; Measurement configuration and reporting; Establishing / modifying / releasing measurement configurations (e.g., intra-frequency, inter-frequency, and RAT-based measurements). Establishing and releasing the measurement gap; Measurements and reports.

[0071] The general functions and interconnections of the components and functions (which also depend on the actual network type) are known to those skilled in the art and are described in the corresponding specifications, so that detailed descriptions may be omitted herein for brevity. However, it should be noted that, in addition to those described in detail below, several additional network components and signaling links may be employed for communication to or from components, functions, or applications, such as communication endpoints, communication network control elements (such as servers, gateways, wireless network controllers), and other components in the same or other communication networks.

[0072] The communication network architecture considered in the examples of the embodiments can also communicate with other networks, such as the public switched telephone network or the Internet. This communication network can also support the use of cloud services for virtual network elements or their functions. It should be noted that the virtual network portion of a telecommunications network can also be provided by non-cloud resources (e.g., internal networks). It should be understood that network elements and / or corresponding functions of access systems, core networks, etc., can be implemented using any node, host, server, access node, or entity suitable for this purpose. Typically, network functions can be implemented as network elements on dedicated hardware, as software instances running on dedicated hardware, or as virtualized functions instantiated on a suitable platform (e.g., cloud infrastructure).

[0073] Furthermore, network elements (such as communication elements, like UEs, terminal devices), control elements or functions (such as access network elements, like base stations / BSs, gNBs, radio network controllers), core network control elements or functions (such as gateway elements) or other network elements or functions (as described herein), and any other elements, functions, or applications, may be implemented by software (e.g., by a computer program product for a computer) and / or by hardware. To perform their respective processing, the corresponding devices, nodes, functions, or network elements used may include several parts, modules, units, components, etc. (not shown) required for control, processing, and / or communication / signaling functions. Such components, modules, units, and assemblies may include, for example, one or more processors or processor units (including one or more processing sections for executing instructions and / or programs and / or processing data), storage units or memory units or components for storing instructions, programs, and / or data, working areas for use as processors or processing sections, etc. (e.g., ROM, RAM, EEPROM, etc.); input or interface components for inputting data and instructions via software (e.g., floppy disk, CD-ROM, EEPROM, etc.); user interfaces for providing monitoring and operation functions to users (e.g., screen, keyboard, etc.); other interfaces or components for establishing links and / or connections under the control of processor units or sections (e.g., wired and wireless interface components, radio interface components including, for example, antenna units, components for forming radio communication sections, etc.), wherein the corresponding components forming the interface (such as radio communication sections) may also be located at remote sites (e.g., wireless heads or wireless stations, etc.). It should be noted that, in this specification, "processing section" should not only be considered as representing a corresponding physical part of one or more processors, but also as a logical division of the processing tasks performed by one or more processors. It should be understood that, according to some examples, the so-called "liquid" or flexible network concept can be adopted, in which the operation and function of network elements, network functions, or another entity in the network can be performed in a flexible manner across different entities or functions (such as nodes, hosts, or servers). In other words, the "division of labor" between the network elements, functions, or entities involved may vary depending on the specific circumstances.

[0074] As shown above, this disclosure is generally intended to provide improved ML methods for distributed environments where the privacy of local data at client devices needs to be better protected and the advantages of centralized training at a central server based on diverse local data at different client devices are still needed for improved training of ML models to be applied at client devices.

[0075] In view of the foregoing, a method for personalized model training on diverse distributed data, such as a FL-based ML method, is proposed according to the disclosure. Specifically, this disclosure proposes a joint approach of client-side clustering and personalized model construction based on their embeddings, wherein clustering informs model construction, and vice versa. Specifically, clustering can be performed at a central server based on embeddings (e.g., loss vectors) provided by client devices, and information related to the updated model can be provided between the server and client devices to improve model training while respecting the privacy of local data.

[0076] As is well known to those skilled in the art, in ML, embedding refers to a transformation that reduces the complexity of raw data to a lower-dimensional representation while preserving meaningful relationships and structures within the data.

[0077] The proposed solution offers the following advantages: Privacy: Data does not need to leave the client's device. Personalization: We leverage data diversity, where training and customization of models are separate, tailored to the data and client clustering needs of each data set. Fast convergence: The algorithm achieves correct clustering in just a few rounds and trains the model with low loss. Efficiency: As demonstrated by experiments on real data, the significantly improved model efficiency and accuracy outperform existing techniques. Robustness: The correctness of the algorithm output is not sensitive to initialization, unlike error-prone existing techniques (i.e., outputting incorrect results if no specific initial model parameters are given). Applicable to both supervised and unsupervised settings: Accessing labeled data (supervised settings) is actually more difficult, especially in network and anomaly detection applications. The method proposed according to this disclosure has the advantage of being able to be trained without labels, thus making it applicable to both settings.

[0078] Reference will now be made to the accompanying drawings. It should be noted in particular that, unless otherwise stated, the same or similar reference numerals used in the drawings of this disclosure may indicate the same or similar elements, such that repeated descriptions thereof may be omitted for the sake of brevity.

[0079] Detailed background information relating to federated learning and personalized federated learning is provided below, but this is only for the purpose of better understanding the prior art and its shortcomings, and therefore should not be construed as a limitation in any way on the solutions provided under this disclosure. Federated Learning: What it is, why it is needed, how it works, and its applications.

[0080] Figure 1An example of a typical federated learning architecture is illustrated schematically.

[0081] Machine learning (ML) has made tremendous progress in the past few years. Building and training the largest centralized ML systems today typically requires a lot of money and computing equipment, such as GPUs. At the same time, training requires a lot of data, which is not always available, limiting the applicability of the system. However, data is now mostly generated and / or stored in a distributed manner. Therefore, privacy plays an increasingly important role, as data owners do not want to share their private data to train centralized models for others, and regulations such as the General Data Protection Regulation (GDPR) stipulate the consequences for violations. On the other hand, each individual owner's data may not be sufficient to train a good enough model locally. These challenges are addressed by the relatively new paradigm of federated learning (FL), which was originally introduced by [1].

[0082] In the FL paradigm, instead of storing or sending data to a single location for training, the local data of the data holders—the clients—is used to train models that will be applicable to all of them simultaneously. The standard FL architecture consists of a server and multiple clients. Its goal is to train models using distributed data held by the clients. To this end, the server initializes a model and sends it to a subset (or all) of the clients, where it is trained on the local data of each client. The clients then send the parameters of their locally trained models back to the server. The server collects updated parameters from all clients and aggregates them according to some aggregation method (the simplest of which is to average or weighted average based on how much data each client has trained on their model—this algorithm is called federated averaging[1]). The resulting parameters are sent back to the clients for another round of local training, the new parameters are aggregated again on the server, and so on.

[0083] This process has several advantages: First, it preserves client privacy because only model parameters, not client data, are sent to the server. Second, it is also more communication-efficient because model parameters are typically only a fraction of the client data that would be sent to the server in a centralized architecture. Third, it allows the server to train models based on sources that are otherwise inaccessible (especially when an equivalent centralized dataset does not exist) and leverages the diversity of data from each client to improve the trained model. Finally, FL training does not require a large amount of computation because clients hold only a small amount of data, while the overall new model comes from a centralized combination of client models.

[0084] These characteristics make FL particularly suitable for applications where data is distributed by nature (as in, for example, edge computing) or isolated in silos (e.g., patient data residing in hospital settings). Perhaps the most well-known application of FL is for predicting the next word from user input on mobile phones: local training occurs on each user's phone with a small sample of words they generate (but these should not leave the device), and the models from different users are aggregated to produce a global model that is trained on sufficient data while respecting privacy, and then the global model is sent to the user's phone. Another application with huge potential impact is using data from isolated databases maintained by hospital systems to train ML models for diagnosing various types of diseases. FL would allow joint training and aggregation without compromising sensitivity to the nature of the data.

[0085] Typical FL architecture, such as Figure 1 As shown in the image. One of the servers... S With multiple clients Communication (where for example, i = 1…… For example, ).server S Sending models to multiple clients m The client trains a model on its private data and sends the updated model to the server, which then averages these updates.

[0086] It has the typical advantage of FL: it can train on data that would otherwise be inaccessible. Data privacy: Data does not leave the client; only model updates are exchanged. Communication efficiency: Only the model is transmitted, and its size is much smaller than the original data; and Each device has a small computational load: the device trains the model only on its small private dataset.

[0087] Typical examples of non-exhaustive applications of FL: Next word prediction for smartphone keyboards Training diagnostic models based on hospital / patient data Anomaly detection in distributed networks / digital twin data Personalized Federated Learning: Why It's Needed, Existing Technologies, and Limitations

[0088] Figure 2 This illustration shows an example of a typical personalized federated learning architecture.

[0089] Challenges arise in federated learning (FL) when the client’s data is not independent and identically distributed (non-iid). A single model built by the server through aggregation performs well on the data of the “average client”, and therefore will not generalize well to clients with local data that are significantly different from the global data distribution [2]. Therefore, it is necessary to customize the trained model for each client’s local data, or in other words, personalized federated learning (PFL).

[0090] In order to better understand PFL, it is necessary to place PFL within the spectrum of ML architectures that can be implemented in the scenario of interest, namely a setup with multiple clients (each client having its own data) and a server acting as a coordinator. One extreme case is a centralized global model: the client sends its data to the server, and the server centrally trains a single model and then distributes it to the clients. However, such a model will perform poorly on client data that differs greatly from the global training data distribution. Training a single model based on data from all clients via traditional FL will face the same problem. Another extreme case is training a local model only at each client, only for the data of each client. These models will perform well on the data of their respective clients, but will not generalize to data from other clients or data that has not been seen before. PFL aims to have the best of both worlds, achieving generalization of the global model along with alignment of the local model with the client data distribution[3].

[0091] One approach for PFL is through personalization of clustering. In settings where clients can be grouped based on their properties (e.g., similar types of devices or users) or their data, clients can be clustered, and a personalized model can be trained for each cluster, such as... Figure 2 As shown in the image.

[0092] For example, in Figure 2 In the middle, a server S With multiple clients Communication (e.g., with 4 clients), where these clients are clustered into different groups, such as clients 1 and 2 in group or cluster 1, and clients 3 and 4 in group or cluster 2. Server S Send multiple models to multiple clients Specifically, the server S Send the model to clients 1 and 2 in group or cluster 1 And send the model to clients 3 and 4 in group or cluster 2. The client trains the received model on its private data accordingly and sends the updated model to the server accordingly, and the server averages the updates for each model updated by the corresponding cluster / group.

[0093] The aforementioned general approach to personalization through clustering has been adopted by some work. Ghosh et al. [4] assumed that clients are divided into different clusters (e.g., based on each user’s interests, as happens in recommendation system applications), and a model needs to be trained per cluster. The cluster identifier for each client is unknown, so the problem is to identify the cluster for each client while training the model corresponding to each cluster in a distributed manner. Their algorithm (called IFCA—Iterative Federated Clustering Algorithm) alternates between training (minimizing the loss function of each client on its local data) and aggregating model updates at the server, and estimates the cluster identifier of the client based on the minimum loss of the client’s data on each of the cluster models. The algorithm is supervised (based on the labels of the data points) and accompanied by some theoretical guarantees for convergence under some assumptions about good initialization and (non-iid) data distribution.

[0094] Similarly, Mansour et al. [3] investigated whether personalization could be achieved through an intermediate model between the global and local models, and proposed clustering FL as a way for personalization, which is very similar to IFCA, that is, assigning each client to the model with the lowest loss. Werner et al. [5] did not use model parameters to cluster clients, but instead used threshold clustering on the model gradients on the data of each client. Dennis et al. [6] designed a federated version of the k-means algorithm for clustering, which has a hierarchical structure and focuses on reducing communication costs. Clustering is based on the data points themselves. Unlike the above works, the solution proposed according to this disclosure considers the whole picture and clusters on all loss / gradient vectors from all clients at the same time.

[0095] Sattler et al. [7] used the cosine similarity between gradients of different clients to recursively divide them into subgroups with the greatest differences, thus generating a hierarchical structure of an increasingly specific model. In contrast, the method of this disclosure runs a vector clustering algorithm (e.g., k-means) and clusters all clients at once, rather than recursively dividing the client set into two groups. In addition, [7] assumes that all clients participate in each round of FL, while this disclosure does not impose such an assumption.

[0096] Most work on PFL considers both labeled data and supervised learning settings. A notable exception is the work of Chung et al. [8], which proposed UIFCA, an IFCA-like algorithm tailored for unsupervised settings where the data is unlabeled. Cluster assignment is performed locally for each client, just like IFCA, and clusters are selected based on the lowest loss criterion. The model used in this paper is a generative model known as the normalized flow model. However, the cluster accuracy achieved by the UIFCA algorithm on experiments with the MNIST handwritten digit dataset [9] is mediocre. In contrast, the method in this disclosure works for both supervised and unsupervised settings and shows excellent performance on MNIST.

[0097] Anomaly detection (AD) in the context of digital twins (DT) is beginning to receive considerable attention

[10] ,

[11] . One of the main challenges of AD in DT is the lack of labeled data. As a result, many traditional anomaly detection algorithms are not effective for AD in DT

[10] , while unsupervised ML methods such as Bayesian online change point methods and generative adversarial networks (GANs) are being considered

[10] ,

[11] . Another challenge for AD in DT is the processing of large amounts of data. In view of this, an edge computing-based DT framework for AD was proposed in

[12] . However, in its context, there is no previous work that utilizes federated learning (FL) as this disclosure has done. By utilizing FL, this disclosure is able not only to utilize edge computing, but also to make full use of computing resources in the cloud in the most efficient way, while minimizing communication costs and maintaining the privacy of local data.

[0098] The prior art has the following disadvantages, such as those disclosed in

[13] : It is only applicable to recommendation systems. For example, it cannot be applied to anomaly detection in web data. This disclosure is more generally applicable to any ML use case and domain, including anomaly detection. The embeddings they use are specifically tailored for recommendation systems (user and item embeddings), and there are no equivalents in other ML use cases (such as anomaly detection). The embeddings proposed according to this disclosure are model loss vectors and are applicable to all ML use cases. Their method offers weak privacy protection because it requires sharing both user and item embeddings with the server. Furthermore, these embeddings can also be shared with other users (although the user's ID is encrypted / anonymized when their embedding is sent by the server to other users). Using this information, the server can estimate each user's preference for each item (by evaluating the similarity between the user's embedding and the item embedding), which is part of what needs to be kept private. Additionally, each user can also learn about such information used by other users (although they don't know which specific user this information is used for). Furthermore, it is assumed that the server is trusted; that is, the server does not collude with users or tell users which information comes from which user. According to the method of this disclosure, no additional information is shared between the client and the server beyond the information shared in traditional federated learning (i.e., model parameters / gradients and loss), thus ensuring complete privacy preservation.

[0099] In summary, there is a need and motivation to modify FL to PFL. Specifically, FL is insufficient to capture the non-iid distribution of data, i.e., situations where data at different clients may have different properties / distributions, for example: Network traffic data may belong to different types of digital twin / IoT devices (cameras, sensors, etc.); and Temperature sensors in one area may generate different data than temperature sensors in another area.

[0100] Therefore, it is not enough to train a single model for all clients, and thus multiple personalized models are needed, for example, by grouping similar clients / data streams simultaneously and training a model for each group (cluster).

[0101] Advantages of PFL: Data sets from multiple clients can be utilized without moving data to a central location; and Each cluster's individual model will be customized for the data of that cluster and will perform well on it.

[0102] Example non-exhaustive applications of PFL: For example, in network security and anomaly detection, and network optimization issues.

[0103] Therefore, there is a particular need for a novel machine learning approach based on federated learning that leverages embeddings and clustering to enable the development of robust and effective anomaly detection models for DT while respecting individual privacy and adapting to operational differences.

[0104] Specifically, the following issues were resolved: Scalable, personalized model training based on federated learning (e.g., for anomaly detection) leverages data from a large number of clients (e.g., digital twins) while protecting privacy. This method represents a significant improvement over existing methods, most of which fail to find ideal solutions and are prone to getting trapped in poor local minima. Training requires no special initialization and works even when starting in random states, unlike existing personalized federated learning model training algorithms, which only work in a limited number of specific starting states and therefore exhibit poor performance on common datasets such as MNIST. Design of algorithms applicable to both supervised and unsupervised settings in any personalized federated learning setup. It converges quickly to a high-quality solution.

[0105] This disclosure presents a novel approach for building personalized machine learning models (e.g., for anomaly detection) for individual clients in distributed environments, such as digital twins. By combining joint clustering, federated learning, and embedding generation, the proposed solution effectively addresses operational differences between different client datasets while maintaining data privacy.

[0106] Anomaly detection is an example of its application in unsupervised settings. More generally, in unsupervised settings, the proposed method can be used for data clustering applications. In supervised settings, the proposed method can be used for class label prediction, covering all classification applications. Additionally, the proposed method can also be used for regression applications (i.e., where available data is used to predict future outcomes).

[0107] In general, the proposed method assigns a unique embedding to each client based on the data characteristics of each client, ensuring that clients with similar data are located close together in the embedding space, while clients with different data are separated. These embeddings are generated locally by each client in a privacy-preserving manner by measuring the loss or gradients on multiple machine learning models without sharing data. As training progresses, the embeddings are improved with respect to the client's data, thus enabling a highly novel and efficient mechanism for embedding generation and improvement during the federated learning process.

[0108] Furthermore, client embeddings are used to identify clusters of clients with similar data-sharing characteristics. This enables efficient grouping of similar clients and the construction of individual personalized models for each group, while minimizing communication and computational costs. The novelty of the proposed method lies particularly in the joint output of client clustering based on their embeddings and the construction of personalized models, where clustering informs model construction, and vice versa. Notably, in the proposed solution, the server possesses global knowledge of all client embeddings, thus allowing for globally optimal clustering of clients. This contrasts with existing solutions where clients must rely on local knowledge to select their groups, potentially leading to suboptimal assignments.

[0109] Furthermore, the proposed solution constructs multiple personalized machine learning models simultaneously through a novel combination of federated learning and client-embedded clustering, with one personalized machine learning model for each client data type. Specifically, for each round of federated learning, the server selects which clients to assign to the machine learning model based on the clustering of client embeddings. This ensures that the machine learning model is trained only on clients whose data have similar characteristics (which is particularly important for anomaly detection). By avoiding the mixing of training data from different clients, the method according to this disclosure prevents the development of invalid machine learning models, which may be normal for one set of clients but anomalous for another.

[0110] Figure 3 An example of an ML method according to an exemplary embodiment of this disclosure being applied to a client device in a distributed environment is illustrated. This distributed environment preferably includes a server device and the client device. Therefore, a client device for performing such a method is also provided. Specifically, the client device is configured to provide user services and establish communication with the server device for providing a digital twin machine learning ML model training service to the client device. The digital twin ML model training service refers to a service used to train an ML model in a digital twin scenario.

[0111] Figure 3 The method includes the following steps performed by the client device. In other words, the client device is configured to perform the following steps.

[0112] At step S31: The client device receives multiple models from the server device to provide estimates of at least one parameter associated with the user service. Preferably, the step of receiving the models includes receiving at least one model parameter of the model.

[0113] At step S32: For each of the received multiple models, the client device applies the data associated with the user service at the client device to the model evaluation function, and determines a corresponding model evaluation value for each model, indicating the performance of each model in providing estimates for the at least one parameter associated with the user service. The data at the client device refers to local data stored at the client device. The model evaluation function is preferably a loss function, where applying the loss function to the ML model by calculating a loss value presented by comparing the output of the ML model with the expected output value is a very basic skill in the field of ML. Therefore, the client device provides an evaluation of each model's performance in providing estimates for the user service.

[0114] At step S33: Based on all the model evaluation values ​​determined for the multiple received models, the client device obtains from the server device an indicator indicating the model selected by the server device from the multiple received models. Since the client device has already received the model in step S31 or has previously stored the model (i.e., its model parameters) locally, it only needs to be notified of the indicator for the selected model.

[0115] In step S33, the model selection can be provided by the server device and / or by the client device.

[0116] For example, a client device sends a model evaluation vector to a server device, the model evaluation vector including multiple determined model evaluation values ​​for multiple received models; and receives from the server device an indicator indicating the model selected by the server device from among the multiple models received for the client device based on the model evaluation vector.

[0117] Alternatively, the client device determines a corresponding model selection preference probability for each model, where the corresponding model selection preference probability is determined based on applying a pre-configured probability density function to a determined model evaluation value for each model. The model evaluation vector also includes model selection preference probabilities determined for multiple models, for the server device to determine which model to select; and the client device receives the selected model from the server device. If the model evaluation function is a loss function, then for a model, the smaller loss value given by the loss function indicates a higher model selection preference probability given by the pre-configured probability density function.

[0118] For the probability distribution applied to the loss, and for each client to select the model to update in the PFC method / algorithm based on that probability distribution, a softmax function can be applied, or the probability can be any function inversely proportional to the loss (absolute value) (the higher the absolute loss, the lower the probability), such as proportional to -|loss| or 1 / |loss|, where the values ​​are normalized to sum to 1. A strictly inversely proportional function is one possibility. Other possibilities, such as the softmax function, are not mathematically inversely proportional, but satisfy what is required according to this disclosure: "the higher the absolute loss, the lower the probability." The latter part is used to select a suitable function.

[0119] Using the aforementioned candidate models with preference probabilities, the client selects a preferred model. It sends its preference to the server, which then selects a model for the client to use in the next round. Specifically, the model evaluation vector sent by the client to the server contains its preference probability for each model.

[0120] Note that in the softmax / probabilistic case, where each client is able to choose a model locally, the final model selection for the client is made by the server. This allows the server to potentially override the client's preferences based on some global objective.

[0121] At step S34: The client device trains and / or updates the selected model based on the data associated with the user service at the client device. The training and / or updating of the selected model preferably includes training and / or updating at least one model parameter of the selected model based on the (local) data at the client device. Possible training methods (e.g., gradient descent optimization methods) are fundamental knowledge to those skilled in the art. Alternatively or additionally, the training and / or updating of the selected model includes: determining an updated gradient value for the model evaluation function for the selected model; and sending the updated gradient value for the model evaluation function for the selected model to the server device, such that the server device determines an updated at least one model parameter of the selected model based on the updated gradient value.

[0122] At step S35: The client device sends information to the server device related to the trained and / or updated selected model, so that the server device can further update the selected model. The information includes (as explained above with respect to step S34) updated values ​​of at least one model parameter of the selected model and / or updated gradient values ​​of the model evaluation function for the selected model.

[0123] Furthermore, the proposed method is preferably executed in multiple iterations or rounds. For example, the client device is configured to execute steps S31 to S35 in the current iteration, wherein the received multiple models are updated by the server device in a previous iteration immediately preceding the current iteration; wherein the client device is also configured to receive multiple models from the server device that were further updated by the server device in the current iteration in the next iteration immediately following the current iteration; wherein the client device is also configured to repeat steps S31 to S35 for the multiple models in the next iteration. Thus, at the end of each round of iteration, updated models are provided for further updates in the next round.

[0124] Figure 4 An example of an ML method applied to a server device in a distributed environment according to an exemplary embodiment of this disclosure is illustrated. This distributed environment preferably includes multiple client devices and the server device. Therefore, a server device for performing such a method is also provided. Specifically, the server device is configured to provide user services, and communication is established with the client devices to provide digital twin machine learning ML model training services. Digital twin ML model training services refer to services used to train ML models in a digital twin scenario. (The last sentence appears to be incomplete and possibly refers to a different context.) Figure 3 Similar aspects related to the explained ML methods will not be elaborated here, especially since Figure 4 Server equipment and Figure 3 Client devices interact in a distributed environment.

[0125] At step S41: The server device sends multiple models to a subset of multiple client devices. These models are used to train and / or update to provide estimates of at least one parameter associated with the user service. This subset includes at least one client device from the multiple client devices. The selection of such a subset should not be specifically limited, depending on the application scenario of the distributed environment, the type of user service, the model to be trained, etc.

[0126] At step S42: The server device receives a corresponding model evaluation vector from each client device included in the subset, wherein the corresponding model evaluation vector includes a corresponding model evaluation value for each of a plurality of models, the corresponding model evaluation value indicating the performance of each model in providing an estimate of at least one parameter associated with the user service. In other words, each client device in the subset (e.g., according to...) Figure 3 The method described in the article provides the server device with a corresponding model evaluation vector, which includes model evaluation values ​​for multiple models.

[0127] At step S43: The server device selects a corresponding model from multiple models for each client device included in the subset, based on all evaluation vectors received from the subset of client devices. In other words, the server device selects a model for each client device in the subset from multiple models based on the evaluation values ​​provided by all client devices in the subset.

[0128] For example, the server device selects a corresponding model for each client device included in a subset by: clustering all model evaluation vectors received from the subset of client devices based on clustering; wherein each client device included in the subset is clustered into a group based on a vector clustering method; and selecting a corresponding model for each group. Preferably, the selection is performed by: for each group, based on a matching method (which will... Figure 6 (As explained in detail in the text) Each group is matched one-to-one with the corresponding model.

[0129] In reality, client devices belong to groups, but this information is unknown or inaccessible to the server. Through the proposed joint clustering and training method, the grouping of client devices can be computed, ideally matching the true groupings after a certain number of iterations.

[0130] At step S44: The server device sends an indicator to each client device included in the subset, the indicator indicating which of the multiple models is selected by the server device for at least one client device.

[0131] At step S45: The server device receives information from each client device included in the subset related to a correspondingly selected model that has been locally trained and / or updated by each client device included in the subset. The client device trains and / or updates the selected model locally based on its corresponding local data and sends information related to the trained and / or updated model to the server.

[0132] At step S46: The server device centrally updates the selected models for the client devices included in the subset based on the information. In this disclosure, a "centralized" step refers to a step performed by the server device, and a "local" step refers to a step performed by the client device(s). Because the server device communicates with all the multiple client devices, it has a comprehensive understanding of the information from the multiple client devices; while each client device only has access to its own local data and additional information locally provided by it related to the updated and / or trained models(s).

[0133] The centralized updates at the server are preferably performed by averaging the values ​​included in the information for each group, wherein preferably the weight of the client devices in each group in the averaging corresponds to the amount of data at the client devices in each group. Since the server device has selected a corresponding model for each client device cluster / group, the averaging can be performed for the corresponding model for each client device cluster / group, for example, wherein the updated values ​​of at least one model parameter of the corresponding selected model provided by the client devices included in each group are averaged. Alternatively, the averaging is performed with respect to the updated gradient values ​​of the model evaluation function used for the corresponding selected model. More preferably, the averaging can be a weighted average. In the implementation of the weighted average, preferably, each client has sent the size of its local dataset to the server before the start of the entire learning process, so that the server is informed of the amount of data (points) at each client for determining the corresponding weights in the averaging process. The differences and similarities between the gradient values ​​of the applied model parameters or model evaluation function will be explained in more detail in “Example 2” below.

[0134] Furthermore, the method is preferably repeated multiple times in multiple iterations or rounds. More specifically, the server device is configured to perform steps S41 to S46 in the current iteration, wherein the received plurality of models were updated by the server device in a previous iteration immediately preceding the current iteration; wherein the server device is also configured to send the plurality of models updated by the server device in the current iteration to client devices included in another set in the next iteration immediately following the current iteration; wherein the server device is also configured to repeat steps S41 to S46 in the next iteration for the plurality of models updated in the current iteration for client devices in another subset.

[0135] There are two convergence criteria used to stop the iteration: the convergence of the grouping of client devices calculated in each round to the (stable) group (ideally the correct one); and the lower loss of the chosen model (depending on the grouping) on ​​the client's data for each client device.

[0136] More specifically, iteration is configured to stop when the loss of the selected model used for the corresponding client device is lower than a pre-configured threshold, and the clusters or groups determined for the client device have not been modified within a certain number of iterations and can therefore be considered stable. The true groupings are initially unknown and may (ideally) or may not be reached. However, as long as the method outputs stable groupings within a certain number of rounds, these can be considered the final groupings.

[0137] Figure 5An example of an ML method applied to a distributed system according to an exemplary embodiment of this disclosure is illustrated. The distributed system includes server devices (e.g., ...). Figure 4 (provided) and multiple client devices (each such as) Figure 3 (As provided above). Figure 3 and Figure 4 Similar aspects related to the ML method described herein will not be elaborated upon here.

[0138] An example application could be anomaly detection for digital twins or smart homes. The client devices here could be edge routers / firewalls in many factories or home routers in many homes, while the server could reside in an edge cloud or a public cloud.

[0139] Specifically, Figure 5 Examples of local model training and global clustering (at the server) on a loss vector, for example, using clustering algorithms (such as k-means), are shown.

[0140] like Figure 5 As shown, the client uses its personal data to build its embeddings locally in a privacy-preserving manner. These embeddings are used as data for each client and K The server measures how closely each model in the dataset is close to the others, and can use this information to identify clients with similar types of data. Clients also update the models assigned to them by the server using their local data and send their updated models (and optionally their gradients) to the server. The server clusters the client embeddings (e.g., using the k-means algorithm) to help identify groups of similar clients. It then uniquely selects a model for all these similar clients belonging to a group. For this, the server computes a bipartite graph match using a matching algorithm (e.g., minimum-cost matching, where the cost is the total loss for each cluster / group). The server also applies updates received from clients to different models and assigns new client-to-model assignments and updates. K Each model is sent back to all clients. This process is repeated until convergence of client-to-model matching is achieved.

[0141] Figure 5 The method includes the following steps:

[0142] At step S51: The server device sends the updated model to the client. As described above, this method is preferably executed iteratively. Therefore, from the perspective of the current iteration, the model sent to the client has already been updated in the previous iteration immediately preceding the current iteration.

[0143] At step S52: The client device receives from the server device: KThere are 10 models. The number of models is not limited (i.e., it can be set randomly), and it can be any integer starting from 1.

[0144] Step S53: The client device performs local computation. K The loss vector (embedding) is generated and sent to the server device. Specifically, each client device has received the updated model, and each client device correspondingly sends a corresponding loss vector to the server device. K dimensional loss vector.

[0145] At step S54: The server device uses a vector clustering algorithm to cluster the client's loss vector (embedding). Those skilled in the art should be aware of possible implementations of the clustering method used by the server to cluster the clients, including one or more of the following non-limiting examples: k-means, DBSCAN, K-Medoid, spectral clustering, Gaussian mixture model, etc.

[0146] At step S55: The server device calculates the new client-to-model assignment by: first, assigning clusters to models using a matching algorithm; and then assigning clients within the cluster to their selected models. The cluster-to-model assignment or mapping will be targeted at… Figure 6 This will be explained in more detail. A "new" assignment refers to the assignment provided in the current iteration, replacing a previous assignment provided in the iteration immediately preceding the current one.

[0147] At step S56: The server device sends a new model assignment to each client (e.g., in the form of an indicator indicating which of a plurality of models is assigned to each client).

[0148] At step S57: The client device receives from the server device the corresponding model assigned to the client device. Because each client device is clustered into a group, and each group is matched with a model, each client device is assigned the matching model for the group to which it belongs.

[0149] At step S58: The client device computes locally and sends to the server device: the assigned model (and optionally the gradient) updated with local data.

[0150] At step S59: The server device calculates the new first... j A model (for all 1≤ j ≤ K ): and preferably for the model received from the client j All updates (model parameters or gradients) are averaged.

[0151] Figure 6An example of a matching method for assigning a cluster of clients to the corresponding model is illustrated.

[0152] Matching refers to assigning each model to one and only one cluster, and each cluster to one and only one model. In practice, a match is, for example, the set of edges from a model to a cluster.

[0153] For example, one option is to match based on the minimum cost of the total cluster loss: the total loss (cost) of a match can be defined as the sum of the losses of all edges including that match. Then, among all possible matches, the one with the minimum loss ( Figure 6 The solid arrow in the example is selected.

[0154] For matching clusters with models, alternatives include: sorting clusters by the smallest client ID, matching by the minimum cost based on the total cluster loss, matching by the minimum cost based on the overlap between the cluster and previous clusters, etc.

[0155] The proposed method improves upon the prior art in the following ways: Many personalized federated learning solutions require very good initialization of the model they begin with. Otherwise, they will be unable to find a model with low loss on the client's local data. This can be extremely disadvantageous for anomaly detection with digital twins, as these models may fail to distinguish between anomalies and normal operating states for the client. On the other hand, the proposed solution works even with randomized model initialization. This is because the loss-based embeddings represent the client's data well, even with randomized model initialization. Many existing models (such as IFCA) tend to map clients to models based on only a simple criterion (e.g., the model with the minimum loss on the client's data). This can lead to suboptimal selection because, with such a simple criterion, many models may be equally qualified (e.g., all with nearly equal losses), so even clients belonging to the same cluster may be assigned to different models, even if the data differences are small. Over time, such a problem can become more complex, leading to convergence to a suboptimal solution. The proposed solution overcomes this problem by grouping clients based on their embeddings (using clustering), thus ensuring that clients with similar data end up in the same group and are therefore assigned to the same model. The probability of finding a good solution is high in the proposed solutions. This is because, even in the first round of our algorithm, there is a high probability that the client's embeddings represent its local data well, and therefore the client will be correctly mapped to the correct model in the next round. Consequently, in subsequent rounds, the embeddings will only improve, thus increasing the chances of finding the optimal solution. Compared to existing techniques, the proposed solution converges to the final solution much faster. This is because, even after only a few rounds, most clients are correctly assigned to the final model, leading to rapid convergence thereafter. Currently, there is no existing technology for efficient, federated learning-based anomaly detection solutions for digital twins that can operate in the unsupervised settings common in digital twins, where data labels are scarce and can be costly.

[0156] The proposed invention is particularly applicable to anomaly detection, such as in digital twins, where unsupervised machine learning is necessary to compare real-time data with a predefined baseline or expected behavior. Unlike existing federated learning methods (which rely on supervised learning and require pre-labeled training data for both anomalous and normal behavior), the proposed solution according to this disclosure provides a more flexible and efficient approach for anomaly detection in complex systems. This is particularly valuable in environments where anomalies are constantly evolving, and historical data of previously observed anomalies are insufficient to detect new, unknown anomalies. Furthermore, since the proposed solution is based on federated learning, it effectively addresses the additional challenges of anomaly detection in digital twins, including the scale and cost of handling large amounts of data, and the preservation of privacy for local data.

[0157] In summary, the proposed joint clustering, federated learning, and embedding generation approach provides an efficient and novel solution for building personalized anomaly detection machine learning models. Its unique features include: minimal assumptions about data distribution; embedding generation based on data characteristics; joint clustering and personalized model building; globally optimal client-side clustering in a privacy-preserving manner based on complete client-side embeddings (loss / gradient vectors, in contrast to the more short-sighted local versions of previous work that only focused on individual loss / gradient values); and improved anomaly detection performance in unsupervised settings. These features make it an ideal, flexible, and scalable solution for complex systems, especially digital twins. An exemplary embodiment of the method described above, based on this disclosure, is as follows:

[0158] A typical FL architecture consists of one server and multiple clients. M The number of clients, these clients belong to K A cluster. K Clients in each of the clusters receive data from different distributions. ( Extract data from (where the symbol [) K ] represents the set {1,2,……, K}). Each client i ∈[M ]have Data points.

[0159] make To be with data points z The associated loss function, where It is the parameter space. The goal of the FL algorithm is to minimize the population loss function. For all This is for client use. i Data Loss of experience is defined as .

[0160] Our PFL algorithm aims to simultaneously cluster clients into... K Clusters, and training K There are 3 models, each best suited to the data of a single client cluster. Note that this assumes the number of data categories (the actual cluster) is known (equal to...). K The number of models selected is chosen to match the number of data categories. However, the proposed method (e.g., the method implemented by the following algorithm) does not necessarily require prior knowledge of the number of data categories: if the number of selected models is lower than the number of data categories, multiple similar data categories will be mapped to the same model; if a larger number is selected, some data categories may be split into two. Therefore, the proposed method remains effective in a completely unsupervised setting where there is no prior information about the number of real clusters. Example 1: EFC, First Variant - Model Average The pseudocode for the proposed Embedded Federation Cluster (EFC) algorithm is as follows:

[0161] The method in Example 1 (implemented by the algorithm described above) operates as follows: Initially, the server creates... K There are 10 models (row 1), each of which will eventually be trained to fit the model. K Data for each cluster in each cluster, and to M A subset (or all) of the clients send all of them (lines 3-4). Each of these clients runs its data through all K There are several models, and the loss is obtained, which forms a loss of length for each client. K The loss vectors are then sent to the server (lines 5-7). The server treats the loss vectors as data points and runs vector clustering algorithms (e.g., k-means, DBSCAN, K-Medoid, spectral clustering, Gaussian mixture model, etc.) on them to cluster them using a minimum cost matching algorithm. KEach group (line 9): A bipartite graph is created, with the top nodes on the left representing groups and the top nodes on the right representing models, and the weight of each edge is the sum of the losses of clients in the top groups on the left on the models on the right. On this graph, minimum-cost bipartite matching can be found using standard methods (e.g., based on maximum flow). Alternatives could be: sorting clusters by minimum client ID; performing minimum-cost matching based on total cluster loss; performing minimum-cost matching based on clusters overlapping with previous clusters, etc. Therefore, after this step, each client (represented by its corresponding loss vector) is assigned to a cluster in this iteration. (Line 10). It should be noted that there is a one-to-one mapping between the cluster and the model, so each client is effectively assigned to a specific model in this iteration. After this assignment, each client runs the model locally. The function LocalUpdate(), which starts with the parameter, represents... The algorithm iterates through gradient descent (e.g., stochastic gradient descent - SGD, adaptive moment estimation - Adam, etc.) and sends the obtained model parameters to the server to train the assigned model on its private data (lines 11-12). Then, the averaging step follows: the server aggregates all the data from the clients that have updated it. The received model parameters are used as a weighted average of their model parameters, where each client... i The contribution is based on the number of samples. Normalized (lines 13-15). In the next iteration, the server again sends the computed updated parameters to the client (line 4), and this process continues until some parameters for each model are finally reached that cause the client to perform “good” clustering (ideally matching the real clusters).

[0162] EFC (e.g., Example 1 implemented by Algorithm 1) differs from IFCA (see [4]) in the following ways. First, in IFCA, clustering / model selection for each client is performed locally at the client by selecting the model with the lowest loss for that client, whereas in Example 1, client-to-cluster (client-to-model) assignment is performed on a server using the k-means algorithm on the set of loss vectors for all clients. Second, in the averaging phase, Example 1 explicitly uses the number of data points per client to ensure fair treatment in averaging, similar to [8]; whereas IFCA assumes from the outset that all clients have the same amount of data and treats them equally, which may not be the case in practice.

[0163] In IFCA, the client selects the model with the lowest loss. In Example 1, the client evaluates all models, sends a loss vector to the server, the server clusters the loss vectors (embeddings), and assigns the client to a model based on that clustering.

[0164] It should be noted that the method proposed in Example 1 is a generalization of the original FL algorithm [1]. In the original FL algorithm, a single model is trained by all clients. If we set in the algorithm K =1, then lines 5-10 become very simple, all clients in lines 11-12 train the same model, and the server in lines 13-15 aggregates the updated parameters from all clients. Example 2: EFC, Second Variant - Gradient Averaging

[0165] When gradient averaging is used instead of model averaging, another variant of the EFC algorithm emerges. The corresponding pseudocode is shown in Algorithm 2.

[0166] In Example 2, the difference from the model averaging method in Example 1 is that the order of the gradient steps and the order of averaging have been reversed. Now, in Example 2, each client computes about its assigned model (by the server). The client calculates the gradient of the loss function (line 12) and sends this gradient to the server. The server then performs a weighted average of the gradients, where each client... i The gradient is determined by the number of samples. The parameters are normalized, and the aggregated gradient multiplied by the learning rate is subtracted from the previous parameters to obtain new parameters (line 15). In experiments, both the model-averaged and gradient-averaged versions were tested, and both worked equally well.

[0167] Examples 1 and 2 are able to cluster clients better than IFCA in this unsupervised setting. The reason is that in each round, IFCA assigns each client to the model with the lowest loss for it among all models. However, in some iteration, when clients have similar losses across models (such as in the initial iteration when they haven't been trained), this assignment can be problematic because many, or even all, clients may be assigned to the wrong single model that presents the lowest loss (because the model is untrained and the client's specific data doesn't play a significant role). This would be good if the correct clustering were recovered in subsequent iterations. However, when many clients from different clusters are assigned to a single model, that model will be trained on all their data, so in the next iteration, the same model will likely again present the lowest loss for the same clients. This creates a vicious cycle where these clients are trapped in the same k-means clusters and cannot be separated. However, this does not occur when using the methods proposed in Examples 1 and 2 (e.g., implemented via EFC algorithms 1 and 2): each round of clustering is based on the complete loss vector of the client, rather than on a single minimum value of the loss vector. The vector clustering algorithm detects the similarity between the loss vectors of different clients and clusters them into the same cluster. It should be noted that clusters belonging to the same true cluster are likely to end up belonging to the same client cluster (as output by the vector clustering algorithm) because their fingerprints (their losses across all models) will be similar. This is the intuitive reason why the proposed Examples 1 and 2 are able to cluster correctly where IFCA cannot. Example 3: PFC, First Variant - Model Average

[0168] By modifying the proposed method to use client-to-cluster probability assignment when the number of clients is very large, the potential scalability problem is addressed. Clustering is an NP-hard problem, therefore there is no multinomial-time algorithm. This can lead to high complexity for our algorithm when applied to a large number of clients. Of course, sampling steps (e.g., line 3 in Algorithms 1 and 2) are helpful because not all clients should have to participate in every round. However, in cases where centralized clustering needs to be avoided for reasons of complexity or decentralization, the following Example 3 constitutes an improvement to IFCA, where local cluster assignment is still determined solely by each client rather than by the server.

[0169] The above explanation of Example 3 (as shown in Algorithm 3) provides the (first) model-averaged version of the proposed Probabilistic Federated Clustering (PFC) method. A second variant using gradient averaging can be easily derived by replacing rows 12 and 15 with rows 12 and 15 of Algorithm 2. Therefore, the second variant of Example PFC will not be explained in detail here. Specifically, Figure 7 This illustrates local model training and local clustering (at each client) using a chosen loss distribution (e.g., softmax).

[0170] Example 3 (e.g., implemented by the PFC algorithm in Algorithm 3) works as follows. Similarly, server initialization... K For each model, a subset of clients is selected and their model parameters are sent (lines 1-4). Each client then runs its model again on all the data to obtain a loss vector (lines 5-6). Then, a local clustering step follows (lines 7-10): each client, based on its model loss, uses a predefined distribution (through its probability density function). q - Line 9) defines the preference probabilities used to select each model, and randomly selects a cluster and its corresponding model to be updated in that round based on those probabilities (Line 10). Training occurs as before (Lines 11-12), the model parameters of each client are sent to the server, and the server aggregates them (Lines 13-15).

[0171] probability density function q It can be, for example, the softmax function, or any function that is inversely proportional to the loss (the higher the absolute value), such as being proportional to -|loss| or 1 / |loss|, where the values ​​are normalized to sum to 1.

[0172] The core idea of ​​Example 3 is that, instead of deterministically selecting the model with the lowest loss, the randomness in the selection of the model that each client trains can, to some extent, help clients from different clusters avoid getting stuck using the same model in multiple rounds, thus making it easier to separate them.

[0173] In IFCA, the client selects the model with the lowest loss. In Example 3, the client randomly selects a model from all models based on the probability distribution defined by the loss.

[0174] like Figure 7 As shown, the method more specifically includes the following steps:

[0175] At step S71: The server sends the updated model to the client.

[0176] At step S72: The client receives from the server: K One model.

[0177] At step S73: The client performs the following calculations locally: K- Dimensional loss vector (embedding).

[0178] At step S74: Clustering steps at the client: A. Cluster assignment probabilities based on the selected distribution; B. Assign clients to the cluster based on the calculated probabilities.

[0179] At step S75: The client computes locally and sends to the server: the assigned model (and optionally the gradient) updated with local data.

[0180] At step S76: The server calculates the new first... j A model (for all 1≤ j ≤ K Preferably, the model received from the client... j All updates (model parameters or gradients) are averaged.

[0181] In Embodiment 3 above, a local cluster is provided by the client based on the model selection preference probability, wherein the client selects the model based on the probability. Therefore, this embodiment involves a local variant of the probabilistic version. That is, the model selection preference probability is determined locally by the client and then applied locally by the client to select the corresponding model.

[0182] Furthermore, according to this disclosure, alternatively or otherwise, a centralized variant of the probabilistic version is provided, wherein the server makes a decision on which model to select for the corresponding client. In this centralized variant, in order for the server to determine the selected model, the local model selection preference probability determined by the client is sent to the server so that the server selects the model based on those probabilities calculated by the client (as explained above for possible implementations of step S33).

[0183] For both EFC and PFC processes, once training is completed according to the above procedures, inference is very simple, and as... Figure 8 The following occurs as shown.

[0184] according to Figure 8 The process at each client device includes the following steps:

[0185] At step S81: The client has stored (since the training phase) the model corresponding to the client's cluster; and

[0186] At step S82: The client feeds input data to the model locally and calculates the output; and calculates the loss based on the output and input, and performs the required task (e.g., determining whether the input is anomalous in an unsupervised setting, or predicting the class label in a supervised setting).

[0187] It should be noted that, although in the above example embodiments (refer to the accompanying drawings), the messages communicated / exchanged between network components / elements may have specific / explicit names, depending on various implementations (e.g., underlying technologies), and these messages may have different names and / or be communicated and exchanged in different forms / formats, as can be understood and recognized by those skilled in the art.

[0188] According to some example embodiments, corresponding methods applicable to devices (network elements / components) as described above, such as UE, CU, DU, etc., are also provided.

[0189] However, it should be noted that the aforementioned device (equipment) features correspond to corresponding method features, but for the sake of brevity, these method features may not be explicitly described. This disclosure is also intended to be extended to these method features. Specifically, this disclosure is understood to relate to methods of operating the aforementioned equipment, and / or providing and / or arranging corresponding elements of such equipment.

[0190] Furthermore, according to some other example embodiments, a corresponding apparatus (e.g., implementing the UE, CU, DU, etc. as described above) is also provided, the apparatus including at least one processing circuitry system and at least one memory for storing instructions which will be executed by the processing circuitry system, wherein the at least one memory and the instructions are configured together with the at least one processing circuitry system such that the corresponding apparatus performs at least the corresponding steps as described above.

[0191] However, in some other example embodiments, a corresponding apparatus (e.g., implementing the UE, CU, DU, etc. as described above) is provided, which includes corresponding components configured to perform at least the corresponding steps as described above.

[0192] It should be noted that the examples of embodiments of this disclosure are applicable to a variety of different network configurations. In other words, the examples shown in the above figures (which serve as the basis for the examples discussed above) are merely exemplary and do not limit this disclosure in any way. That is, additional existing and proposed new functionalities available in the corresponding operating environment can also be used in conjunction with the examples of embodiments of this disclosure based on the defined principles.

[0193] It should also be noted that the disclosed example embodiments can be implemented in many ways using hardware and / or software configurations. For example, the disclosed embodiments can be implemented using dedicated hardware and / or hardware associated with the software executable thereon. The components and / or elements in the drawings are merely examples and do not limit the scope or functionality of any hardware, software combined with hardware, firmware, embedded logic components, or combinations of two or more such components that implement a particular embodiment of this disclosure.

[0194] It should also be noted that the specification and drawings are for illustrative purposes only. Those skilled in the art will be able to implement various arrangements, although these arrangements are not explicitly described or shown herein, but they embody the principles of this disclosure and are included within the spirit and scope of this disclosure. Furthermore, all examples and embodiments outlined in this disclosure are primarily intended for illustrative purposes only to aid the reader in understanding the principles of the proposed methods. In addition, all statements herein providing the principles, aspects, and embodiments of this disclosure, as well as specific examples thereof, are intended to cover their equivalents. List of abbreviations: DT: Digital Twin ML: Machine Learning FL: Federated Learning PFL: Personalized Federated Learning IFCA: Iterative Federated Clustering Algorithm iid: Independent and identically distributed EFC: Embedded-Driven Federated Clustering PFC: Probabilistic Federated Clustering References [1] B. McMahan, E. Moore, D. Ramage, S. Hampson and BA y Arcas, "Communication-Efficient Learning of Deep Networks from Decentralized Data", Proceedings of the 20th International Conference on Artificial Intelligence and Statistics A. Singh and J. Zhu, *Collected Papers on Machine Learning Research*, Vol. 54, PMLR, April 2017, pp. 1273–1282. [Online]. Available at: https: / / proceedings.mlr.press / v54 / mcmahan17a.html [2] AZ Tan, H. Yu, L. Cui and Q. Yang, “Towards Personalized FederatedLearning”, IEEE Transactions on Neural Networks and Learning SystemsVolume 34, Issue 12, pp. 9587-9603, December 2023, doi: 10.1109 / TNNLS.2022.3160699. [3] Y. Mansour, M. Mohri, J. Ro, and AT Suresh, “Three Approaches for Personalization with Applications to Federated Learning,” July 19, 2020. arXiv: arXiv:2002.10619 Accessed: March 7, 2023. [Online]. Available: http: / / arxiv.org / abs / 2002.10619 [4] A. Ghosh, J. Chung, D. Yin and K. Ramchandran, “An EfficientFramework for Clustered Federated Learning”, IEEE Transactions on Information Theory Volume 68, Issue 12, pp. 8076-8091, December 2022, doi: 10.1109 / it.2022.3192506. [5] M. Werner, L. He, M. Jordan, M. Jaggi and SP Karimireddy, "Provably Personalized and Robust Federated Learning", Journal of Machine Learning Research 》, August 2023, Accessed: March 29, 2024. [Online]. Available at: https: / / openreview.net / forum?id=B0uBSSUy0G [6] DK Dennis, T. Li and V. Smith, “Heterogeneity for the Win: One-Shot Federated Clustering”, Proceedings of the 38th International Conference on Machine Learning PMLR, July 2021, pp. 2611–2620. Accessed: May 7, 2024. [Online]. Available at: https: / / proceedings.mlr.press / v139 / dennis21a.html [7] F. Sattler, K.-R. Müller and W. Samek, “Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints”, IEEE Transactions on Neural Networks and Learning Systems Volume 32, Issue 8, pp. 3710–3722, August 2021, doi: 10.1109 / TNNLS.2020.3015958. [8] J. Chung, K. Lee and K. Ramchandran, “Federated Unsupervised Clustering with Generative Models”, presented at the 2022 AAAI International Workshop on Trustworthy, Verifiable and Auditable Federated Learning. [9] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based LearningApplied to Document Recognition”, IEEE Journal Volume 86, Issue 11, pp. 2278–2324, November 1998, doi: 10.1109 / 5.726791.

[10] Q. Lu, X. Xie, AK Parlikad and JM Schooling, “Digital Twin-enabled Anomaly Detection for Built Asset Monitoring in Operation andMaintenance”, Building Automation Volume 118, page 103277, October 2020, doi: 10.1016 / jc autton.2020.103277.

[11] Q. Xu, S. Ali and T. Yue, “Digital Twin-based Anomaly Detection in Cyber-physical Systems”, The 14th IEEE Software Testing, Verification and Qualification Conference (ICST) in 2021 , April 2021, pp. 205-216. doi: 10.1109 / ICST49551.2021.00031.

[12] H. Huang, L. Yang, Y. Wang, X. Xu and Y. Lu, “Digital Twin-drivenOnline Anomaly Detection for an Automation System based on EdgeIntelligence”, Manufacturing Systems Magazine , Volume 59, pp. 138–150, April 2021, doi: 10.1016 / j.jmsy.2021.02.010.

[13] US 2023 / 0419123 A1.

Claims

1. A client device configured to provide user services and establish communication with a server device, the server device being configured to provide digital twin machine learning (ML) model training services to the client device, the client device comprising: At least one processor, and At least one memory storing instructions that, when executed by the at least one processor, cause the client device to at least: Receives multiple models from the server device, the multiple models being used to provide an estimate of at least one parameter associated with the user service; For each of the received plurality of models, data associated with the user service at the client device is applied to a model evaluation function, and a corresponding model evaluation value is determined for each model, the corresponding model evaluation value being used to indicate the performance of each model in providing an estimate of at least one parameter associated with the user service; Obtain from the server device an indicator for indicating the model selected by the server device from among the received plurality of models based on all model evaluation values ​​determined for the received plurality of models; The selected model is trained and / or updated based on the data at the client device associated with the user service; as well as Information relating to the trained and / or updated selected model is sent to the server device so that the server device can further update the selected model.

2. The client device of claim 1, wherein the client device is further configured to acquire the selected model by: Sending model evaluation vectors to the server device, the model evaluation vectors including determined model evaluation values ​​for the received plurality of models; and The server device receives an indicator for selecting a model from among the plurality of models received by the server device based on the model evaluation vector.

3. The client device of claim 2, wherein the client device is further configured to acquire the selected model by: For each model, a corresponding model selection preference probability is determined, wherein the corresponding model selection preference probability is determined based on applying a pre-configured probability density function to the determined model evaluation value for each model; The model evaluation vector further includes model selection preference probabilities determined for the plurality of models, for the server device to select one of the plurality of models; and Receive the selected model from the server device.

4. The client device according to any one of claims 1 to 3, wherein the client device is further configured to train and / or update the selected model by: Update the value of at least one model parameter of the selected model based on the data at the client device; and The updated value of at least one model parameter of the selected model is sent to the server device.

5. The client device according to any one of claims 1 to 4, wherein the client device is further configured to train the selected model using a gradient descent optimization method.

6. The client device according to any one of claims 1 to 5, wherein the client device is further configured to train and / or update the selected model by: For the selected model, determine the updated gradient value of the model evaluation function; and The updated gradient value of the model evaluation function for the selected model is sent to the server device.

7. The client device according to any one of claims 1 to 6, wherein the client device is configured to perform the steps of any one of claims 1 to 6 in the current iteration. The received multiple models are updated by the server device in the previous iteration immediately preceding the current iteration. The client device is further configured to receive, in the next iteration immediately following the current iteration, the plurality of models that have been further updated by the server device in the current iteration. The client device is further configured to repeat the steps of any one of claims 1 to 6 in the next iteration for the plurality of models.

8. The client device according to any one of claims 1 to 7, wherein the model evaluation function is a loss function that indicates the loss when the model is applied to provide an estimate of the at least one parameter associated with the user service.

9. The client device according to claim 8, which is dependent on claim 3, wherein, for the model, a smaller loss value given by the loss function indicates a higher probability of model selection preference given by the pre-configured probability density function.

10. A server device configured to establish communication with a plurality of client devices for providing digital twin machine learning (ML) model training services to the plurality of client devices, comprising: At least one processor, and At least one memory, the at least one memory storing instructions, the instructions, when executed by the at least one processor, causing the server device to at least: Multiple models are sent to a subset of the plurality of client devices. These models are used to be trained and / or updated to provide estimates of at least one parameter associated with the user service. Receive a corresponding model evaluation vector from each client device included in the subset, wherein the corresponding model evaluation vector includes a corresponding model evaluation value for each of the plurality of models, the corresponding model evaluation value indicating the performance of each model in providing an estimate of the at least one parameter associated with the user service; Based on all the evaluation vectors received from the subset of client devices, for each client device included in the subset, a corresponding model is selected from the plurality of models; Send an indicator to each of the client devices included in the subset, the indicator indicating which of the plurality of models sent is selected by the server device for the at least one client device accordingly; Receive information from each of the client devices included in the subset related to the correspondingly selected model, which is locally trained and / or updated by each of the client devices included in the subset; as well as The selected model for client devices included in the subset is updated based on the information.

11. The server device of claim 10, wherein the server device is further configured to select a corresponding model for each client device included in the subset by means of: Based on clustering, all model evaluation vectors received from the subset of client devices are clustered into multiple groups by the client devices included in the subset, wherein each client device included in the subset is clustered into a group based on a vector clustering method; and Select the corresponding model for each group.

12. The server device of claim 11, wherein the server device is further configured to select a corresponding model for each group by matching each group with a corresponding model one-to-one based on a matching method.

13. The server device according to any one of claims 10 to 12, wherein the information includes the value of at least one model parameter and / or the gradient value of the model evaluation function.

14. The server device of claim 13, wherein the server device is further configured to centrally update the model selected for the client devices included in the subset by: For each group, the values ​​included in the information are averaged, wherein preferably the weight of the client device in each group in the average corresponds to the amount of data at the client device in each group.

15. The server device according to claim 13 or 14, wherein the server device is further configured to: Based on the updated gradient value of the model evaluation function used for the correspondingly selected model, the updated value of at least one model parameter of the correspondingly selected model is determined.

Images