SAFE FEDERATION OF DISTRIBUTED STOCHASTIC GRADIENT DESCENTS

Abstract

System that features the following: a processing unit that is functionally connected to a memory; an AI (Artificial Intelligence) platform that exchanges data with the processing unit, wherein the AI ​​platform is designed to train a machine learning (ML) model, wherein the AI ​​platform has the following features: a registration manager to register participating entities in a cooperative relationship, to arrange the registered entities in a topology, and to establish a topological data transmission protocol; an encryption manager to generate a public additive homomorphic encryption key, AHE key, and distribute it to each registered entity; an entity manager to control direct local encryption of machine learning model weights of local entities on registered entities using a distributed public AHE key, to selectively aggregate the encrypted local machine learning model weights including distributing the aggregated encrypted weights to one or more other participating entities in the topology, or to support local aggregation on the registered entities according to the topological data transmission protocol; where the encryption manager decrypts an aggregated sum of the encrypted local machine learning model weights with a corresponding private AHE key, and distributes the decrypted aggregated sum to each entity in the topology.

Description

TECHNICAL AREA

[0001] The present invention relates generally to training a machine learning model based on gradient descent, including deep neural networks. More specifically, the embodiments relate to collaborative training of a machine learning model based on an iterative algorithm in a distributed, federated, private, and secure manner.

[0002] A document already exists in this context. Document US 2018 / 0359084A1 describes encryption based on homomorphic factorization. For this, an encryption key and a decryption key are generated. Then, a computation key is generated based on the encryption key. Based on this, the data is transferred to a remote system for analysis.

[0003] Despite these known aspects, there remains a need to better protect artificial intelligence systems in distributed environments. BACKGROUND

[0004] This task is solved by the subject matter of the independent claims. Further details arise from the respective dependent claims.

[0005] Artificial intelligence (AI) refers to the field of computer science that deals with computers and their behavior in relation to humans. AI refers to the intelligence that enables machines to make decisions based on information, maximizing their chances of success in a given area. More specifically, AI is able to learn from a dataset to solve problems and provide relevant recommendations. For example, in the field of AI computer systems, natural language systems (such as the IBM Watson AI computer system) process... ®or other natural language question-answering systems) natural language based on knowledge acquired by the system. To process natural language, the system can be trained with data from a database or a knowledge corpus; however, the result can be incorrect or inaccurate for a variety of reasons.

[0006] Machine learning (ML), a subfield of artificial intelligence (AI), uses algorithms to learn from data and make predictions based on that data. ML is the application of AI through the creation of models, including neural networks, that can exhibit learning behavior by performing tasks that are not explicitly programmed. Deep learning is a type of ML where systems can handle complex tasks by using multiple layers of decision-making based on the outputs of a previous layer, thus drawing increasingly intelligent and abstract conclusions. Deep learning uses neural networks, referred to here as artificial neural networks, to model complex relationships between input and output and to identify patterns within them.

[0007] At the heart of AI and its associated inferences lies the principle of similarity. The process of understanding natural language and objects requires inferences from a relational perspective, which can be challenging. Structures, including static and dynamic structures, prescribe a specific output or action for a given input. More precisely, the specific output or action is based on an explicit or implicit relationship within the structure. This regulation can be satisfactory under certain circumstances and conditions. However, it is understood that dynamic structures are inherently subject to change, and the output or action may change accordingly. SUMMARY

[0008] According to one aspect of the invention, a system is provided for use with an artificial intelligence (AI) platform to train a machine learning model. The processing unit is functionally connected to the memory and exchanges data with the AI ​​platform, which incorporates tools in the form of a registration manager, an encryption manager, and an entity manager. The registration manager serves to register participating entities in a cooperative relationship, arrange the registered entities in a topology, and establish a topological data transmission direction. The encryption manager serves to generate a public additive homomorphic encryption (AHE) key and distribute it to each registered entity.The entity manager is used to locally manage the encryption of local machine learning (ML) model weights of entities using a corresponding distributed AHE key. The entity manager also selectively aggregates the encrypted local ML weights and distributes the aggregated weights to one or more entities in the topology, depending on the topological data transmission direction. The encryption manager decrypts an aggregated sum of the encrypted local ML model weights using a corresponding private AHE key and distributes the aggregated sum to each entity in the topology. Finally, the encryption manager shares the decrypted aggregated sum of the encrypted local ML model weights with the registered participating entities.

[0009] According to another aspect, a computer program product is provided for training a machine learning model. The computer program product includes a computer-readable storage medium containing program code. This program code is executable by a processor to register participating entities in a cooperative relationship, arrange the registered entities in a topology, and establish a topological data transmission direction. Program code is provided to generate a public additive homomorphic encryption (AHE) key and distribute it to each registered entity. The program code then locally performs encryption of local ML model weights of entities using a corresponding distributed AHE key.The local ML model weights are selectively aggregated, and the aggregated weights are distributed to one or more entities in the topology depending on the topological data transmission direction. Furthermore, program code is provided to decrypt an aggregated sum of the encrypted local ML model weights using a corresponding private AHE key. The decrypted aggregated sum is distributed to each entity in the topology, with the decrypted aggregated sum of the encrypted local ML model weights being shared with the registered participating entities.

[0010] According to another aspect, a procedure for training a machine learning model is provided. Participating entities are registered in a cooperative relationship. The registered participating entities are arranged in a topology, and a topological data transfer direction is established. Each registered participating entity receives a public additive homomorphic encryption (AHE) key, and local machine learning model weights are encrypted with the received key. The encrypted local machine learning model weights are selectively aggregated, and the selectively aggregated, encrypted weights are distributed to one or more participating entities in the topology, depending on the topological data transfer direction. The aggregated sum of the encrypted local machine learning model weights is then decrypted using a corresponding private AHE key.The decrypted aggregated sum of the encrypted local ML model weights is shared with the registered participating entities.

[0011] These and other features and advantages will become apparent from the detailed description below of the preferred embodiment(s) in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings referenced herein form part of the specification. Features shown in the drawings serve only to illustrate some embodiments and not all embodiments, unless expressly stated otherwise. Fig. Figure 1 shows a flowchart illustrating a system connected in a network environment that supports a secure federation of distributed stochastic gradient descents. Fig. Figure 2 shows a block diagram illustrating an AI platform and tools, as described in Fig. 1. presented and described, and their associated application programming interfaces illustrated. Fig. Figure 3 shows a block diagram illustrating an administrative domain and domain-internal aggregation. Fig. Figure 4 shows a flowchart illustrating a process for performing an intra-domain aggregation for an administrative domain. Fig. Figure 5 shows a flowchart illustrating a process for domain-internal collaboration and training of ML programs. Fig. Figure 6 shows a block diagram illustrating an exemplary ring topology to support the in Fig. 5 processes shown and described. Fig. Figure 7 shows a flowchart illustrating a process for arranging the entities in a fully connected topology and deploying a broadcast data transmission protocol across the topology. Fig. Figure 8 shows a flowchart illustrating a process for supporting and enabling weight encryption and aggregation over a channel or broadcast group whose membership changes dynamically. Fig. Figure 9 shows a flowchart illustrating a process for encrypting local weight fields and synchronously aggregating blocks of the fields in parallel. Fig. Figure 10 shows a block diagram illustrating an example of a computer system / server of a cloud-based support system to implement the system and processes described above in relation to Fig. 1 to 9 were described. Fig. Figure 11 shows a block diagram illustrating a cloud computing environment. Fig. Figure 12 shows a block diagram illustrating a series of functional abstraction model layers provided by the cloud computing environment. DETAILED DESCRIPTION

[0013] It is understood that the components of the present embodiments, as generally described and illustrated in the figures, can be arranged and designed in a multitude of different configurations. Therefore, the following detailed description of the embodiments of the device, system, method, and computer program product of the present embodiments, as illustrated in the figures, is not intended to limit the scope of the claimed embodiments, but is merely representative of selected embodiments.

[0014] When this specification refers to "a selected embodiment," "(exactly) one embodiment," or "(any) embodiment," it means that a particular feature, structure, or property described in connection with the embodiment is included in at least one embodiment. Therefore, the expressions "a selected embodiment," "in (exactly) one embodiment," or "in (any) embodiment" appearing at different points in this specification do not necessarily refer to the same embodiment.

[0015] The embodiments shown are best understood by reference to the figures, in which identical parts are consistently designated by the same reference numerals. The following description is intended only as an example and merely illustrates certain selected embodiments of units, systems, and processes that are consistent with the embodiments claimed herein.

[0016] Deep learning is a machine learning technique that uses neural networks in successive layers to learn iteratively from data. Neural networks are models of how the human brain processes information. Basic units of neural networks are called neurons, which are typically organized in layers. The neural network operates by simulating a large number of interconnected processing units that resemble abstract versions of neurons. A neural network typically consists of three parts: an input layer with units representing input fields, one or more hidden layers, and an output layer with one or more units representing one or more target fields. The units are connected with varying connection strengths or weights.Input data is presented to the first layer, and values ​​are passed from each neuron to each neuron in the next layer. Finally, a result is delivered by the output layers. Complex deep learning neural networks are designed to emulate the workings of the human brain, allowing computers to be trained to support vaguely defined abstractions and problems. Neural networks and deep learning are widely used in image recognition, natural language processing, and computer vision applications.

[0017] Neural networks contain interconnected layers, corresponding algorithms, and adjustable weights. An optimization function that adjusts the weights is called gradient descent. More precisely, gradient descent is an optimization algorithm used to minimize a function by iteratively moving towards a steepest descent, defined by a negative gradient. In machine learning, gradient descent is used to update parameters of the neural network and its corresponding neural model. This is straightforward when training takes place on a single physical machine or between computers within a single entity. However, when multiple entities are involved, sharing data may be impossible due to data transfer restrictions or legal requirements (regulations such as HIPAA, etc.).One solution is to share the weights and insights of the individual participating entities. It is known in the field that sharing insights derived from data can lead to the creation of a desirable or improved neural model. However, data sharing also introduces other problems, such as confidentiality and data privacy breaches, by other participating entities who can reverse engineer data from the shared insights. Therefore, as shown and described here, a system, a computer program product, and a method are provided to merge encrypted weights by sharing encrypted model parameters without sharing data or weights in the form of unencrypted text, e.g., plaintext.

[0018] As shown and described here, an encryption key and a corresponding encryption platform are used to encrypt the shared weights, and an algorithm or process is used to support and enable aggregation of the encrypted weights. The encryption platform uses additive homomorphic encryption (AHE), such as Paillier encryption, a type of key-pair-based cryptography that uses a public key and a corresponding private key. Each entity uses the same public key to support and enable homomorphism for each training exercise.AHE provides an additive homomorphism that allows messages or corresponding data to be added while they are in encrypted form, and furthermore supports correct decryption of the additively encrypted form with the corresponding private key. As shown and described here, AHE is applied to ML to encrypt weights of a corresponding neural network and to share the encrypted weights with registered participating entities in a cooperative environment without encrypting or sharing the corresponding data.

[0019] Fig. Figure 1 shows a schematic diagram (100) illustrating a secure federation of a distributed stochastic gradient descent. As shown, a server (110) exchanges data with a plurality of data processing units (180), (182), (184), (186), (188), and (190) via a network connection (105). The server (110) is configured with a processing unit (112) that exchanges data with a memory (116) via a bus (114). The server (110) is shown with an AI platform (150) to support collaboration in training a machine learning model based on an iterative optimization algorithm in a distributed, federated, private, and secure environment. The server (110) exchanges data via the network (105) with one or more of the data processing units (180), (182), (184), (186), (188) and (190).More precisely, the data processing units (180), (182), (184), (186), (188), and (190) exchange data with each other and with other units or components via one or more wired and / or wireless data transmission links, each data transmission link potentially comprising lines and / or routers and / or switches and / or transmitters and / or receivers and / or the like. In this networked arrangement, the server (110) and the network connection (105) enable the acquisition, detection, and resolution of data transmissions. Other embodiments of the server (110) can be used with components, systems, subsystems, and / or units other than those described here.

[0020] The AI ​​platform (150) is configured to receive inputs (102) from various sources. For example, the AI ​​platform (150) can receive inputs from the network (105) and use a data source (160), also referred to here as a corpus or knowledge base, to generate output or response content. As shown, the data source (160) is configured with a library (162), or in one embodiment with a plurality of libraries, wherein the library (162) contains one or more deep neural networks, referred to here as neural models, including a model A (164 A ), a model (164 B ), a model C (164 C ) and a model D (164 DIn one embodiment, the library (162) may contain a reduced number of models or an expanded number of models. Likewise, in one embodiment, the libraries in the data source (160) may be organized according to common areas or topics, although this is not a requirement. Models included in the library may originate from similar or different sources.

[0021] The AI ​​platform (150) is equipped with tools to support and enable collaboration in machine learning. The various data processing units (180), (182), (184), (186), (188), and (190) that exchange data with the network (105) can contain access points for the models of the data source (160). The AI ​​platform (150) acts as a platform to enable and support collaboration without sharing insights or data. As shown and described here, the collaboration uses a public key infrastructure (PKI) that isolates AHE key generation from weighted encryption and aggregation.As described in detail herein, additive homomorphic encryption is used to enable identified or selected entities to share weights of neural models in encrypted form without exposing the data. A response output (132) in the form of a neural model with the desired accuracy is received and passed on to the entities participating in the collaboration. In one embodiment, the AI ​​platform (150) transmits the response output (132) to members of a cooperative topology, as described in [reference to relevant document]. Fig. 6 and Fig. 7 is shown and described, which is functionally connected via the network (105) to the server (110) or to one or more of the data processing units (180) - (190).

[0022] In various embodiments, the network (105) can include local network connections and remote connections such that the AI ​​platform (150) can operate in environments of any size, including local and global, e.g., the internet. The AI ​​platform (150) serves as a back-end system to support collaboration. In this way, several processes populate the AI ​​platform (150), which also includes input interfaces to receive and respond to requests.

[0023] The AI ​​platform (150) is shown here with various tools to support the collaboration of neural models, including a registration manager (152), an encryption manager (154), and an entity manager (156). The registration manager (152) serves to register participating entities in a cooperative relationship, including arranging the registered entities in a topology and establishing a data transmission direction and a data transmission protocol between the entities in the topology. In one embodiment, for example, the registered entities are arranged in a ring topology, as shown and described below. However, the data transmission protocols can vary. Examples of the protocol include, but are not limited to, a linear-direction protocol, a broadcast protocol, and an all-reduce protocol.As further shown and described here, an additive homomorphic PKl encryption platform is used for sharing and cooperating the weights of the neural models. The encryption manager (154), shown here as functionally connected to the registration manager (152), serves to generate a public additive homomorphic encryption (AHE) key for each training job and distribute it to the registered entities. Distribution typically occurs per machine learning training job, but can also occur per iteration. A corresponding private AHE key is generated but not distributed. The public key is stored by a corresponding recipient entity. The private AHE key, hereinafter referred to as the "private key," which is associated with each of the distributed public AHE keys, is not shared with any of the recipient entities, e.g.,the participating entities. Accordingly, the registration manager (152) and the encryption manager (154) serve to register participating entities, establish data transmission protocols, and generate and selectively distribute public AHE encryption keys.

[0024] As shown, the entity manager (156) is functionally connected to the registration and encryption managers (152) and (154), respectively. The entity manager (156) serves to locally manage the encryption of local ML model weights of entities with a corresponding distributed AHE key, followed by aggregation. In one embodiment, for example, each of the models, which are referred to here as models, is A (164 A ), models (164 B ), Model C (164 C ) and model D (164 D) are assigned to a respective set of entities. In one embodiment, an entity can be any of the data processing machines (180) - (190) that are functionally connected to the server (110). Each model has one or more corresponding weights that are the subject of cooperation. For example, in one embodiment, model A (164 A ) corresponding weightings (166 A ), model A (164 B ) has corresponding weightings (166 B ), model C (164 C ) has corresponding weightings (166 C ) and model D (164 D ) has corresponding weightings (166 DThe entity manager (156) selectively aggregates the encrypted local ML model weights using a corresponding public key. Various aggregation and cooperation protocols can be used, including, but not limited to, linear transfer, broadcast, and all-reduce. Regardless of the cooperation protocol, the model weights of each entity are encrypted with a corresponding public AHE key at a certain point in the cooperation and aggregation process. As shown here, the weights (166 A ) with a corresponding public AHE key (168 A ) encrypted, the weightings (166 B ) are used with a corresponding public AHE key (168 A ) encrypted, the weightings (166 C ) are used with a corresponding public AHE key (168 A ) encrypted and the weightings (166 D) are used with a corresponding public AHE key (168 A ) encrypted. Each of the weights is therefore encrypted separately with the same public AHE key (168 A ) encrypted.

[0025] It is known in the field that AHE supports additive properties. This allows the weights of the corresponding models to be aggregated in encrypted form. Depending on the communication and cooperation protocol, the encrypted weights undergo aggregation in different phases. In a linear ring topology, for example, the registration manager (152) assigns a rank to each participating entity in the topology. Based on its corresponding rank and the established data transmission direction, each of the model weights is encrypted and aggregated step by step. The entity manager (156) encrypts the weights with a locally provided public AHE key, e.g., the public key (168). A), and transmits the encrypted weights to a neighboring entity for aggregation. More precisely, the entity manager (156) aggregates the AHE-encrypted weights along the topology without facilitating or enabling decryption. The registration manager (152) establishes the data transmission direction and, in one embodiment, also modifies it. In a ring topology, for example, the registration manager can establish a clockwise or counterclockwise data transmission direction and can also change the direction. In one embodiment, the registration manager (152) can, for example, change the direction based on available bandwidth. In a broadcast protocol, the registration manager (152) establishes the local encryption of the weights and the transmission of the encrypted weights from each entity to the others and to the AI ​​platform.Accordingly, the entity manager (156) supports and enables the aggregation and distribution of the encrypted weights based on or in response to the topological direction and the data transmission protocol(s).

[0026] The public AHE key has a corresponding private key that is not shared with the participating entities. In one embodiment, the private key, e.g., key p (168) P ), stored locally in the encryption manager (154) of the AI ​​platform (150). It is understood that the aggregated and encrypted weights are subject to decryption based on the data transmission protocols. At a time suitable for decryption, the encryption manager (154) subjects an aggregated and encrypted sum of the encrypted weights (166) to decryption. P,E ) a decryption using the private key, e.g. the key P (168P ), resulting in an aggregated sum of decoded weights (166 P,UE ) is created. The encryption manager (154) distributes or divides the aggregated and decrypted sum (166) P,UE ) the local weights with each of the participating and contributing entities. Accordingly, each entity that contributed to the aggregation receives the aggregated and decoded sum.

[0027] It is understood that a participating entity can contain a single sub-entity or, in one embodiment, a plurality of internal sub-entities. In one embodiment, each entity has a single set of security and configuration policies for a network domain. Fig. Figure 3 shows an example of an entity containing multiple internal sub-entities. The entity manager (156) is configured to support and enable cooperative aggregation of weights based on a single sub-entity or multiple sub-entities. More specifically, the entity manager (156) performs an intra-entity aggregation of weights representing a homogeneous data type from each internal sub-entity and encrypts the intra-entity aggregation using the entity's public AHE key. Thus, the intra-entity aggregation occurs before the aggregation is subjected to AHE encryption.

[0028] The entity manager (156) encrypts the entity-internal aggregation using a local public AHE encryption key. The encrypted aggregation is then distributed topology-wide among the entities. As described above, this distribution among the entities involves an aggregation of encrypted weights. After cross-entity aggregation of the weights and decryption using the corresponding private key, the entity manager (156) distributes the aggregated sum to each of the internal sub-entities. Thus, each participating entity and its associated internal sub-entity benefits from and participates in this collaboration.

[0029] The registration manager (152) is responsible for setting up the topology and the data transmission protocols. In one embodiment, the registration manager (152) creates a fully connected topology, also known as a mesh topology, and a corresponding broadcast protocol, in which each participating entity sends its encrypted local weights across the topology and directly to every other participating entity in the topology, e.g., via broadcast. The entity manager (156) further supports and enables selective aggregation, which in this embodiment includes each participating entity locally aggregating all received encrypted weights. The encryption manager (154) subjects each local aggregation to a participation check. The goal of the aggregation is for each participating entity to receive and benefit from the encrypted weights of the other participating entities.However, it is difficult to determine whether one or more entities in the topology have not contributed to, or are not contributing to, the weight aggregation. In the mesh topology, each participating member can directly exchange data with the encryption manager (154), and thus the encryption manager (154) is configured to assess whether it is receiving different aggregated weight values ​​from different members of the topology. For example, if there are four participating entities, and three of the entities have the same aggregated weight values, while one entity has a different aggregated weight value, the encryption manager (154) can identify the non-contributing entity.In one embodiment, the encryption manager (154) can restrict the sharing of the decrypted aggregated weight sum with participating entities or require the identified non-contributing entity to send its encrypted local weights to each of the participating members of the topology. Accordingly, the mesh topology, as shown and described herein, uses a broadcast protocol and, in one embodiment, an entity participation check to support federated machine learning.

[0030] As in Fig. As shown and described in Figure 1, the registration manager (152) can implement an All-Reduce algorithm or All-Reduce protocol for cooperation. In this embodiment, the entity manager (156) represents the weights of each entity as an array of weights. The entity manager (156) encrypts the array with the entity's corresponding public AHE key, splits the encrypted array into two or more blocks, and synchronously aggregates the blocks in parallel and depending on the topology. The entity manager (156) completes the synchronous aggregation when each participating entity has received a single aggregated block. Each aggregated block is decrypted by the encryption manager (154) with the corresponding private key, after which the decrypted blocks are concatenated and the concatenated decrypted blocks are distributed to the registered participating entities.Accordingly, the All-Reduce protocol is an algorithm that is used efficiently in a parallel and collective manner.

[0031] In some illustrative embodiments, the server (110) may be the IBM Watson™ system available from International Business Machines Corporation in Armonk, New York, enhanced with the mechanisms of the illustrative embodiments described below. The IBM Watson shown and described herein ® The system includes tools for implementing federated machine learning based on iterative optimization algorithms. These tools enable selective aggregation of encrypted model weights without exposing the underlying data, thus ensuring data confidentiality and privacy.

[0032] The registry manager (152), the encryption manager (154), and the entity manager (156), hereinafter collectively referred to as AI tools or AI platform tools, are shown to be contained in or integrated within the AI ​​platform (150) of the server (110). The AI ​​tools can be implemented in a separate data processing system (e.g., 190) connected to the server (110) via the network (105). The AI ​​tools support and enable iterative federated machine learning, including the encryption of local model weights and the transmission of the encrypted local model weights among participating entities without sharing or disclosing underlying data. The output content (132) can be in the form of a decrypted format of the aggregated weights, which is subject to transmission among the entities.

[0033] Types of information processing systems that can utilize the AI ​​platform (150) range from small handheld units such as handheld computers / mobile phones (180) to large mainframe systems such as the mainframe computer (182). Examples of handheld computers (180) include personal digital assistants (PDAs), personal entertainment devices such as MP4 players, portable televisions, and compact disc players. Other examples of information processing systems include a pen or tablet computer (184), a laptop or notebook computer (186), a personal computer system (188), and a server (190). As shown, the various information processing systems can be networked together via a computer network (105).Types of computer networks (105) that can be used to connect the various information processing systems include local area networks (LANs), wireless local area networks (WLANs), the Internet, the public switched telephone network (PSTN), other wireless networks, and any other network topology that can be used to connect the information processing systems. Many of the information processing systems contain non-volatile data storage, such as hard disks and / or non-volatile memory. Some of the information processing systems may use separate non-volatile data storage (e.g., the server (190) uses a non-volatile data storage device (190). A ) and the mainframe computer (182) a non-volatile data storage device (182 A ). In the case of the non-volatile data storage (182 A) it can be a component located outside of the various information processing systems, or an internal component of one of the information processing systems.

[0034] The information processing system used to support the AI ​​platform (150) can take many forms, some of which are in Fig. Figure 1 illustrates an information processing system. For example, an information processing system can take the form of a desktop computer, a server, a portable computer, a laptop, a notebook, or a computer or data processing system with a different form factor. Furthermore, an information processing system can also take other forms, such as a personal digital assistant (PDA), a gaming unit, an ATM, a portable telephone unit, a data transmission unit, or other units that contain a processor and memory. Additionally, an information processing system does not necessarily have to have the Northbridge / Southbridge controller architecture, as other architectures can, of course, be used.

[0035] In this field, an application programming interface (API) is understood to be a software intermediary between two or more applications. Regarding the in Fig. 1 AI platform shown and described (150) can use one or more APIs to support one or more of the tools (152) - (156) and their associated functionality. Fig. Figure 2 shows a block diagram (200) depicting the tools (152) - (156) and their associated APIs. As shown, a plurality of tools are embedded in the KL platform (205), including the Registry Manager (252), which is connected to API0 (212), the Encryption Manager (254), which is connected to API1 (222), and the Entity Manager (256), which is connected to API2 (232). Each of the APIs can be implemented in one or more languages ​​and interface specifications.API0 (212) provides functional support for registering participating entities, arranging the topology, and setting up data transmission protocols; API1 (222) provides functional support for generating and distributing public AHE keys for each of the registered entities, managing the decryption of aggregated weights with a corresponding private key, and managing the distribution of the decrypted weights; and API2 (232) provides functional support for managing intra-entity and cross-entity aggregation depending on the topology. As shown, each of the APIs (212), (222), and (232) is functionally connected to an API orchestrator (260), also known as the orchestration layer, which in the field is understood as an abstraction layer for the transparent linking of the separate APIs.In one embodiment, the functionality of the individual APIs can be interconnected or combined. Therefore, the API configuration shown here should not be considered restrictive. Accordingly, as demonstrated here, the functionality of the tools can be embodied or supported by their respective APIs.

[0036] Fig. Figure 3 shows a block diagram (300) illustrating an administrative domain and domain-internal aggregation. A registered participating entity (310) is referred to here as a local aggregator (LA), which is functionally connected to one or more local data processing entities. In the example shown here, there are four local data processing entities, including Entity0 (320), Entity1 (330), Entity2 (340), and Entity3 (350). Each data processing entity contains or uses one or more machine learning programs, referred to here as learners, supported by functionally connected data.As shown, entity0 (320) is shown with learner0 (322) and functionally connected data0 (324), entity1 (330) is shown with learner1 (332) and functionally connected data1 (334), entity1 (340) is shown with learner2 (342) and functionally connected data2 (344), and entity3 (350) is shown with learner3 (352) and functionally connected data3 (354). Each machine learning program, e.g., learner, extracts and processes the local data into a corresponding local neural model.

[0037] Data originating from the same classification can be applied to different neural models created with or using the same data classification. In the example shown here, each of the learners (322), (332), (342), and (352) represents the same machine learning program for the same data type, e.g., classification of homogeneous data, but with different data. The LA (310) supports and enables the learners to share the weights with or without sharing the underlying data. The LA (310) performs an aggregation of the received weights and, in one embodiment, averages the received weights without performing AHE encryption. Accordingly, the management domain shown and described here represents an entity, which in one embodiment can be a business entity or domain, to perform internal aggregation of weights, e.g.,To support entity-internal aggregation of processes within the domain.

[0038] Fig. Figure 4 shows a flowchart (400) illustrating a process for performing an intra-domain aggregation for an administrative domain. The variable X Total represents the number of data processing entities within the domain (402). The domain can consist of one or more data processing entities. As in Fig. As shown in Figure 3, each data processing entity has a machine learning program and locally connected data, with each machine learning program representing a homogeneous class of data. The variable Y Totalrepresents the number of data types that can be present in the locally linked data (404). In one embodiment, the value of the data types is aligned with the number of machine learning programs. The data type counter variable Y is initialized (406). For each data processing entity X, the weights in the ML program are Y , which corresponds to the data type Y corresponds to, for example, weightings Y, identified and aggregated (408). The process of aggregating weights can be applied to different ML programs for different data types. As shown, after step (408), the data type counter variable Y is incremented (410) to include the next ML program, and it is determined whether each of the data types has been processed for weight aggregation (412). If the determination is negative, the process returns to step (408), and if the determination is positive, the aggregation is completed. In one embodiment, the data type can be specified, and the aggregation can be restricted to the specified data type. Accordingly, intra-entity aggregation of weights can be performed across two or more data processing entities located in a specific or defined domain without performing or using AHE encryption.

[0039] Multiple domains can be arranged in a defined topology. Each domain has a corresponding LA (Layout Array) that is functionally connected to one or more entities and their associated ML (machine learning) programs. Weights assigned by the ML programs can be shared across domains without exchanging the data. More precisely, the weights are encrypted in such a way that aggregation is supported while preserving the encryption. This cross-domain sharing of weights supports and enables collaboration and improved training of ML programs. Fig. Figure 5 shows a flowchart (500) to illustrate a process for cross-domain collaboration and training of ML programs. The variable N Totalis assigned to the number of LAs that are subject to cooperation (502). It is understood that each LA is addressable and has a corresponding address identifier. Each of the LAs is arranged in a topology and assigned a rank that depends on its respective position in the topology (504). In addition, a data transmission protocol is established for cross-domain data exchange within the topology. For descriptive purposes, a linear ring topology is used here, in which the LAs are connected in a ring and pass information to or from each other according to their proximity in the ring structure and a specific direction, e.g., clockwise or counterclockwise. A server, such as the one in Fig. The central server (620), also referred to as the third coordinator, shown and described below, and in one embodiment the AI ​​platform (150) locally assigned to the central server (110), exchanges data with the topology and the LAs assigned to the topology and serves to generate and assign encryption keys. Each LA in the topology is assigned an encryption key. As shown, the AI ​​platform (150) generates the public encryption key and sends it to each LA in the topology (506). The public key has a corresponding private key, which is stored by the central server. The encryption platform used by the central server employs additive homomorphic encryption (AHE), e.g., Paillier encryption. Accordingly, the topology and the communication protocols are configured with three or more LAs inserted into the topology.

[0040] As in Fig. 3 and Fig. As shown and described in Figure 4, each machine learning program is representative of a specific data type. Each machine learning environment (LA) can have one or more machine learning programs, with each program associated with or mapped to a different data type. The variable Y Total is assigned to represent the number of data types (508), and the data type counter variable and the LA counter variable are initialized individually at (510) and (512), respectively. The weight aggregation process is then initiated. As shown, LA N identified, and the weightings for the ML programs that are local to the LA N for the data type Y The data located there are aggregated and encrypted with the public encryption key (514). In one embodiment, the LA N on a single ML program for the data type Ylimited. After step (514), the LA counter variable is incremented (516), followed by determining whether there are other LAs in the topology that have not yet been subjected to weight aggregation (518). If the answer to the determination in step (518) is negative, the LA sends N-1 the weightings for the ML program Y,N-1 to the LA N (520). After receiving the weights, the weights for the ML programs that are local to the LA are N for the data type Y are located, locally aggregated, and encrypted with the public encryption key (522). The LA N-1 The received encrypted weights are combined with the encrypted weights for the ML program. Y,N aggregated (524). Once the aggregation is at LA N Once completed, the process returns to step (516). Accordingly, the weights are aggregated both within and across domains.

[0041] A positive response to the determination in step (518) indicates that each of the LAs in the topology has completed one traverse of the ring. As shown here, the weights of the individual LAs were completed in encrypted form, with the weights of each contributing LA sharing the same public encryption key. The aggregated and encrypted weights are provided by the LA N Total transferred to the central server (526). The only entity with full aggregation is the LA. NTotal The central server uses the private key associated with the public key distributed across the topology and decrypts the aggregation of the encrypted weights for the data type. Y (528) The central server distributes the decrypted aggregation for data type Yto each LA in the topology (530). After receiving the decrypted aggregation from the central server, the respective LA passes the weights to internal learning processes (532). Then the data type counter variable is incremented (534), and it is determined whether each of the data types, e.g., ML programs as in Fig. As shown and described in Figure 4, the weighting aggregation process (536) was processed. If the response to the determination is negative, the process returns to step (514), and if the response is positive, the aggregation process is completed. Accordingly, the aggregation shown and described here is limited to the weights in the corresponding ML programs and does not extend to the associated data.

[0042] Fig. Figure 6 shows a block diagram (600) to illustrate an exemplary ring topology to support the in Fig. The process shown and described in Figure 5. As shown, a central server (620), also referred to here as the third coordinator, is configured or equipped with a key generator (622) to generate the public key for distribution and a private key (680) to be stored locally. In this example, four Learning Agents (LAs) are represented in the topology (610), including LA0 (630), LA1 (640), LA2 (650), and LA3 (660). However, the number of LAs is for descriptive purposes only and should not be considered a limitation. Each individual LA can consist of a single learner or multiple learners, as shown in Figure 5. Fig. Figure 3 shows an internal domain. The central server (620) is functionally connected to each LA in the topological structure. More precisely, the central server (620) generates a public key for each LA (630), (640), (650), and (660) and transmits the public key via a respective data transmission channel. As shown, the server (620) transmits the public key (632) to LA0 (630) via data transmission channel 0 (634). Similarly, the server (620) transmits the public key (642) to LA1 (640) via data transmission channel 1 (644), the public key (652) to LA2 (650) via data transmission channel 2 (654), and the public key (662) to LA3 (660) via data transmission channel 3 (664). The public key (632), (642), (652) and (662) is the same public key for each LA and supports AHE encryption.

[0043] As shown here, the encryption of the weights in this example is performed from LA0 (630). The weights of the local model at LA0 (630) for a specific data type or data classification are calculated, encrypted with key 0 (632), and transmitted over the data transmission channel. 0,1(670) is transmitted to LA1 (640). The encrypted weights for LA0 (630) are referred to here as Weights0 (636). After receiving Weights0 (636) from LA0 (630), the weights of the local model at LA1 (640) are calculated for the same specific data type or data classification and encrypted with Key1 (642). The encrypted weights for LA1 (640) are referred to here as Weights1 (646). The encrypted weights of the local model LA1 (640), Weights1 (646), are aggregated with the encrypted weights, Weights0 (636), of the local model LA0 (630). This aggregation is also referred to here as the first aggregation, e.g., Aggregation0 (648). The encryption and aggregation process continues via the ring topology in the established direction. As shown, aggregation0 (648) is transmitted via the data transmission channel 1,2(672) is transmitted to LA2 (650). After receiving Aggregation0 (648) from LA1 (640), the weights of the local model are calculated at LA2 (650) for the same specific data type or data classification and encrypted with Key2 (652). The encrypted weights for LA2 (650) are referred to here as Weights2 (656). The encrypted weights of the local model LA2 (650), Weights1 (656), are aggregated with Aggregation0 (648) received from LA1 (640). This aggregation is also referred to here as the second aggregation, e.g., Aggregation1 (658). As shown, Aggregation1 (658) is transmitted via the data transmission channel. 2,3(674) is transmitted to LA3 (660). After receiving Aggregation1 (658) from LA2 (650), the weights of the local model at LA3 (660) are calculated for the same specific data type or data classification and encrypted with Key3 (662). The encrypted weights for LA3 (660) are referred to here as Weights3 (666). The encrypted weights of the local model LA3 (660), Weights3 (666), are aggregated with the Aggregation1 (658) received from LA2 (650). This aggregation is also referred to here as the third aggregation, e.g., Aggregation2 (668). Accordingly, weights are encrypted and aggregated across the topology in a defined direction.

[0044] After aggregation is complete at LA3 (660), the aggregation2 (668) is transmitted via the data transmission channel (664) to the central server (620), e.g., the third coordinator. The central server (620) does not possess the underlying data associated with the aggregated weights or the individual weights that comprise the aggregation. The central server (620) possesses a private key (680) that is associated with the public key. The central server (620) decrypts the aggregation, e.g., aggregation2 (668), using the private key (680) and sends the decrypted aggregation to each LA that is a member of the topology.As shown here, the decrypted aggregation is transmitted via data transmission channel 0 (634) to LA0 (630), and further via data transmission channel 1 (644) to LA1 (640), via data transmission channel 2 (654) to LA2 (650), and via data transmission channel 3 (664) to LA3 (660). Accordingly, the homomorphic encryption platform shown and described here, with respect to the ring topology, supports additive encryption of weights associated with each neural model, while maintaining data privacy and confidentiality.

[0045] The in Fig. The encryption platform shown and described in Figure 6 is designed for a ring topology for a homogeneous data type, e.g., a single data type. In one embodiment, the aggregation and encryption supported by the platform can be used for a second or other data type, with the encryption and aggregation for each data type being performed serially or in parallel.

[0046] As in Fig. As shown and described in Figure 1, the topology and the corresponding data transmission protocol are not limited to a ring topology. Fig. Figure 7 shows a flowchart (700) illustrating a process for arranging the entities in a fully connected topology and implementing a broadcast data transmission protocol across the topology. The variable N Totalrepresents the number of entities in the domain (702). The entities are arranged in a fully connected topology, also referred to herein as a mesh topology (704). In one embodiment, each participating entity contains a locally encrypted weight or exists in the form of a locally encrypted weight. Each participating entity has locally encrypted weights and sends its locally encrypted weights, e.g., AHE-encrypted weights, directly to every participating entity in the topology (706). The aggregation of the AHE-encrypted weights is performed locally. More precisely, each participating entity aggregates all received encrypted weights. Each participating entity is functionally connected to the decryptor, e.g., the third-party coordinator, and sends its aggregated weights to the decryptor for decryption using the corresponding private key (708).

[0047] Based on the topology and the established data transmission protocol, the decryptor is configured to share the decryption with each participating entity and, in one embodiment, can verify participation. After step (708), it is determined whether a verification protocol should be performed (710). If the response to the verification is negative, the decrypted aggregation is sent back to the participating entities so that each participating entity receives the decrypted aggregation (712). It is understood in the field that bandwidth limitations may exist. In one embodiment, a single participating entity can be designated for data exchange with the decryptor to transmit the encrypted aggregated sum.Similarly, in one embodiment, each participating entity can separately exchange data with the decryptor to transmit the encrypted aggregated sum and receive the decrypted aggregated sum. In another embodiment, the participating entities do not have knowledge or details regarding the other participating entities, so the decryptor is responsible for transmitting the decrypted aggregation of the weights.

[0048] Theoretically, each participating entity should have an identical encrypted aggregation. A positive response to the query in step (710) is followed by the execution of a verification protocol. The received decrypted aggregated weights of each participating entity are compared to identify a non-participating entity (714). In one embodiment, step (714) compares the number of received encrypted weight aggregations with the number of requested decryptions. Similarly, in another embodiment, step (714) compares the values ​​of the received encrypted weight aggregations to determine if there is an outlier. If a non-participating entity is identified in step (716), the return of the decrypted aggregation can be restricted to the participating entities (716).If no entity is identified as non-participating in step (718), the decrypted aggregation is transmitted to each of the registered participating entities (720). Accordingly, the topology shown and described here supports and enables the identification of non-participating entities.

[0049] The aggregation protocol can be extended or modified to support dynamic changes in membership within the topology, e.g., the membership of local aggregators. Fig. Figure 8 shows a flowchart (800) illustrating a process for supporting and enabling weighted encryption and aggregation over a channel or broadcast group whose membership changes dynamically. A server or third-party coordinator generates a public mailing key and a corresponding private key and prepares the distribution of the public key to LAs in the topology (802). The variable N Totalis assigned to the number, or in one embodiment an initial number, of LAs in the topology (804). The generated public paillier key is shared with each LA in the topology (806). When an LA joins the topology, which is also referred to herein as a group of interconnected LAs, the server or third-party coordinator either generates the public paillier key and the corresponding private key and shares the public key with each joining or joined LA, or it shares a previously generated public paillier key with the LA joining the topology. Accordingly, each LA that is a member of the topology exchanges data with the central server and receives the public paillier key for weighted encryption.

[0050] The LAs that have received the encryption key(s) form a group. However, the individual LAs within this group do not need to be aware of the other LAs. As shown here, one LA in the group, referred to as LA [insert LA here], encrypts N The LA specifies its weights using its public key and then broadcasts the encrypted weights to all other LAs in the group (808). After the broadcast of the LA's encrypted weights... N In step (808) the LA receives N Encrypted weights from all other LAs that are members of the group (810). The LA NIt adds its encrypted weight to each of the received encrypted weights (812), hereinafter referred to as encrypted aggregation weights, and sends the aggregated encrypted weights to the central server, e.g., the third-party coordinator (814). The central server uses the private key to decrypt the aggregated encrypted weights (816) and distributes the decrypted aggregated weights to the individual LAs (818). Accordingly, the process shown here uses the encryption keys in a broadcast scenario.

[0051] In the field of AI and ML, it is understood that one or more LAs, the members of the in Fig. The 6 topologies shown and described, e.g. the ring topology, can have a large field of weights that correspond to the results of a local aggregation. Fig. Figure 9 shows a flowchart (900) illustrating a process for encrypting local weight fields and synchronously aggregating blocks of the fields in parallel. A plurality of LAs are arranged in a ring topology and a data transmission direction is established (902), as shown in Fig. 6 shown and described. The variable N Total is assigned to the number of LAs that are members of the topology (904). Each LA, e.g., the LA N , uses the public Paillier key to encrypt its field of local weights (906). Instead of sending the field of weights in its entirety across the topology, either along the ring or as a broadcast message, each LA divides the encrypted field into sections (908), here called chunks, where the number of chunks in each LA array is equal to the number of LAs that are members of the topology, i.e., N TotalA ring all-reduce algorithm is invoked by initializing the LA and block counter variables N (910). LA N sends block N to the next LA in the ring, e.g. LA N+1 , while he, e.g. LA N , simultaneously the block N-1 from the preceding LA in the topology according to the data transmission direction (912). Each LA in the topology then aggregates its received block. N-1 and its own corresponding block N-1 and sends the aggregated block N-1 to the next LA in the ring, e.g. LA N+1 , (914). Then the counter variable N is incremented (916), followed by determining whether N is greater or less than N Totalis (918). A negative response to the query in step (918) leads to a return to step (912), and a positive response indicates that each LA has an aggregated block of weights. The blocks are aggregated synchronously and in parallel across the ring topology. Accordingly, each LA adds its local block to a received block and sends it to the next LA in the direction of data transmission.

[0052] Following the positive response to the determination in step (918), each LA in the topology has an aggregated block of weights, which is Paillier-encrypted. In an example with four LAs, LA1 has aggregated block 2, LA2 has aggregated block 3, LA3 has aggregated block 4, and LA4 has aggregated block 1. Each LA sends its aggregated block to the third coordinator (920), which decrypts the aggregated encrypted weights received from each LA (922). The third coordinator concatenates the decrypted weights and distributes them to all LAs in the topology (924). Accordingly, the procedure shown and described here has adapted the all-reduce algorithm for efficient and secure aggregation of weights between LAs arranged in a topology.

[0053] Aspects of the functional tools (152) - (156) and their associated functionality may be embodied in a computer system / server at a single location, or in one embodiment they may be configured in a cloud-based system that shares data processing resources. Fig. Figure 10 shows a block diagram (1000) illustrating an example of a computer system / server (1002), hereinafter referred to as the host (1002), exchanging data with a cloud-based support system to illustrate the above-mentioned aspects. Fig. to implement the processes described in sections 1 to 9. The host (1002) is compatible with numerous other general-purpose or specialized data processing system environments or configurations. Examples of known data processing systems, environments, and / or configurations suitable for use with the host (1002) include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, portable or laptop units, multiprocessor systems, microprocessor systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe systems, and file systems (e.g., distributed storage environments and distributed cloud computing environments) that incorporate any of the aforementioned systems, units, and their equivalents.

[0054] The host (1002) can, in a general context, be described by a computer system of executable instructions, for example, program modules that are executed by a computer system. Generally, program modules contain routines, programs, objects, components, logic, data structures, and so on, that perform specific tasks or implement certain abstract data types. The host (1002) can operate in distributed cloud computing environments (1080) where tasks are performed by remotely located processing units connected by a data transmission network. In a distributed cloud computing environment, program modules can reside in both local and remote storage media of the computer system, including short-term storage units.

[0055] As in Fig. As shown in Figure 10, the host (1001) is represented as a universal data processing device. The components of the host (1002) may, but are not limited to, one or more processors or processing units (1004), e.g., hardware processors, a system memory (1006), and a bus (1008) that connects various system components, including the system memory (1006), to a processor (1004). The bus (1008) represents one or more of different types of bus structures, including a memory bus or memory control unit, a peripheral bus, an AGP bus, and a processor or local bus, which uses any of a variety of bus architectures.These architectures include, but are not limited to, the ISA (Industry Standard Architecture) bus, the MCA (Micro Channel Architecture) bus, the EISA (Enhanced ISA) bus, the local VESA (Video Electronics Standards Association) bus, and the PCI (Peripheral Component Interconnects) bus. The host (1002) typically contains a variety of media readable by the computer system. Such media can be any available medium accessible to the host (1002), including both volatile and non-volatile media, removable and non-removable media.

[0056] The memory (1006) can contain computer system-readable media in the form of volatile memory, such as random access memory (RAM) (1030) and / or cache memory (1032). By way of example only, the memory system (1034) can be provided for reading from and writing to a non-removable, non-volatile magnetic medium (not shown and commonly referred to as "hard disk storage"). Although not shown, a magnetic disk drive can be provided for reading from and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive can be provided for reading from or writing to a removable, non-volatile optical disk, such as a CD-ROM, DVD-ROM, or other optical media. In such cases, each of the aforementioned can be connected to the bus (1008) via one or more data media interfaces.

[0057] As a non-restrictive example, the memory (1006) can contain a program / utility (1040) with a group (at least one) of program modules (1042), as well as an operating system, one or more application programs, other program modules, and program data. The operating systems, one or more application programs, other program modules, and program data, or a combination thereof, can each comprise an implementation of a network environment. The program modules (1042) generally perform the functions and / or methods of embodiments for dynamically identifying and processing data transmission evaluation queries. The set of program modules (1042) can, for example, include the following: Fig. 1 described tools (152) - (156) included.

[0058] The host (1002) can also exchange data with one or more external units (1014), such as a keyboard, pointing device, etc., a display (1024), one or more units that allow a user to interact with the host (1002), and / or any units (e.g., network card, modem, etc.) that allow the host (1002) to exchange data with one or more other data processing units. Such data exchange can take place via one or more input / output (I / O) interfaces (1022). Furthermore, the host (1002) can exchange data with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or a public network (e.g., the Internet), via a network adapter (1020). As shown, the network adapter (1020) exchanges data with the other components of the host (1002) via the bus (1008).In one embodiment, a plurality of nodes of a distributed file system (not shown) exchange data with the host (1002) via the I / O interface (1022) or via the network adapter (1020). It should be noted that, although not shown, other hardware and / or software components could be used in conjunction with the host (1002). Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk arrays, RAID systems, tape drives, and storage systems for data archiving, etc.

[0059] In this document, the terms “computer program medium”, “computer-usable medium” and “computer-readable medium” generally refer to media such as main memory (1006), including RAM (1030), cache (1032) and the storage system (1034), such as a removable storage drive and a hard disk installed in a hard disk drive.

[0060] Computer programs (also called computer control logic) are stored in memory (1006). Computer programs can also be received via a data transmission interface, for example, a network adapter (1020). When executed, such computer programs enable the computer system to perform the features of the present embodiments described herein. In particular, when executed, the computer programs enable the processing unit (1004) to perform the features of the computer system. Accordingly, such computer programs constitute controls of the computer system.

[0061] A computer-readable storage medium can be a physical unit capable of retaining and storing instructions for use by a unit to execute instructions. For example, a computer-readable storage medium can be an electronic storage unit, a magnetic storage unit, an optical storage unit, an electromagnetic storage unit, a semiconductor storage unit, or any suitable combination thereof, without limitation. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, dynamic or static random-access memory (RAM), read-only memory (ROM), and erasable programmable read-only memory (EPROM).Flash memory), a magnetic storage unit, a portable compact storage disk-read-only storage device (CD-ROM), a DVD (digital versatile disc), a memory stick, a floppy disk, a mechanically coded unit such as punched cards or raised structures in a groove on which instructions are stored, and any suitable combination of these. A computer-readable storage medium shall not, in its use herein, be understood as volatile signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses traveling through an optical fiber cable), or electrical signals transmitted by a wire.

[0062] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to individual data processing units or, via a network such as the internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission lines, wireless transmission, routing computers, firewalls, switching units, gateway computers, and / or edge servers. A network adapter card or network interface in each data processing unit receives computer-readable program instructions from the network and forwards them for storage on a computer-readable storage medium within the respective data processing unit.

[0063] Computer-readable program instructions for executing work steps of the present embodiments may be assembly instructions, ISA (Instruction Set Architecture) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., as well as conventional procedural programming languages ​​such as the programming language "C" or similar programming languages.The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer, server, or server cluster. In the latter case, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be established with an external computer (for example, via the internet using an internet service provider).In some embodiments, electronic circuits, including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can execute computer-readable program instructions by using state information from the computer-readable program instructions to personalize the electronic circuits to perform aspects of the present embodiments.

[0064] In one embodiment, the host (1002) is a node in a cloud computing environment. As is known in the field, cloud computing is a service delivery model for enabling seamless, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, main memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management overhead or interaction with a service provider. This cloud model can include at least five properties, at least three service models, and at least four implementation models. Examples of such properties are as follows: On-Demand Self-Service: A cloud user can unilaterally and automatically provide data processing functions such as server time and network storage as needed, without requiring human interaction with the service provider. Broad Network Access: Functions are available over a network, accessed through standard mechanisms that support use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs). Resource pooling: The provider's data processing resources are pooled to serve multiple users using a multi-tenant model, with various physical and virtual resources being dynamically allocated and reassigned as needed. There is a perceived location independence, as the user generally has no control over or knowledge of the exact location of the provided resources, but may be able to specify a location at a higher level of abstraction (e.g., country, state, or data center). Rapid Elasticity: Features can be deployed quickly and elastically for rapid horizontal scaling (scale out), in some cases automatically, and released quickly for rapid scale-in. To the user, the available features often appear unlimited and can be purchased in any quantity at any time. Measured Service: Cloud systems automatically control and optimize resource usage by employing a measurement function at a specific abstraction layer appropriate for the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource utilization can be monitored, controlled, and reported, thereby creating transparency for both the service provider and the user.

[0065] The service models are as follows: Software as a Service (SaaS): The functionality provided to the user consists of using the provider's applications running in a cloud infrastructure. These applications are accessible from various client devices via a thin-client interface such as a web browser (e.g., web-based email). The user does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, with the possible exception of limited user-specific application configuration settings. Platform as a Service (PaaS): The function provided to the user is to deploy applications created or obtained by the user, using programming languages ​​and tools supported by the provider, within the cloud infrastructure. The user does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but has control over the deployed applications and potentially over configurations of the application hosting environment. Infrastructure as a Service (IaaS): The functionality provided to the user consists of supplying processing, storage, networking, and other basic data processing resources, enabling the user to deploy and run any software, including operating systems and applications. The user does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and potentially limited control over selected network components (e.g., host firewalls).

[0066] The following are the deployment models: Private Cloud: The cloud infrastructure is operated solely for one organization. It can be managed by the organization or a third party and can be located on the organization's own premises or on external premises. Community Cloud: This cloud infrastructure is shared by multiple organizations and supports a specific user community with shared concerns (e.g., mission, security requirements, policies, and regulatory compliance considerations). It can be managed by the organizations themselves or a third party and can be located on-premises or external premises. Public Cloud: The cloud infrastructure is made available to the general public or a large industry group and is owned by an organization that sells cloud services. Hybrid Cloud: The cloud infrastructure is a composition of two or more clouds (private, community or public) that remain separate entities but are connected by a standardized or proprietary technology that enables data and application portability (e.g. cloud audience distribution for load balancing between clouds).

[0067] A cloud computing environment is service-oriented, focusing on state independence, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that comprises a network of interconnected nodes.

[0068] Fig. Figure 11 shows an illustrative cloud computing network (1100). As shown, the cloud computing network (1100) contains a cloud computing environment (1150) with one or more cloud computing nodes (1110) with which local data processing units used by cloud customers can exchange data. Examples of these local data processing units include, but are not limited to, a personal digital assistant (PDA) or mobile phone (1154A), a desktop computer (1154B), a laptop computer (1154C), and / or an automotive computer system (1154N). Individual nodes within the nodes (1110) can also exchange data with each other. They can be grouped physically or virtually into one or more networks, such as private, community, public, or hybrid clouds (not shown), as described above, or into a combination thereof.This enables the cloud computing environment (1100) to offer infrastructure, platforms, and / or software as services, for which a cloud user does not need to maintain resources on a local data processing unit. It is understood that the types of in . Fig. The data processing units shown (1154AN) are for illustrative purposes only and the cloud computing environment (1150) can exchange data with any type of computer unit via any type of network and / or any type of network-accessible connection (e.g., using a web browser).

[0069] In Fig. Figure 12 shows a set of functional abstraction layers (1200) that are provided by the cloud computing network. Fig. 11 will be provided. It should be clear beforehand that the in Fig.The components, layers, and functions shown in Figure 12 are intended to be illustrative only, and embodiments are not limited to these. As shown, the following layers and corresponding functions are provided: hardware and software layer (1210), virtualization layer (1220), management layer (1230), and workload layer (1240).

[0070] The hardware and software layer (1210) contains hardware and software components. Examples of hardware components include mainframe computers, such as the zSeries. ® -Systems from IBM ® , Servers with RISC (Reduced Instruction Set Computer) architecture, in one example pSeries ® -Systems from IBM ® , xSeries ® -Systems from IBM ® , BladeCenter ®IBM systems, storage units, networks, and networking components. Examples of software components include network application server software, such as the WebSphere application server software. ® from IBM, and database software, in one example the database software DB2 ® from IBM. (IBM, zSeries, pSeries, xSeries, BladeCenter, WebSphere and DB2 are trademarks of International Business Machines Corporation, registered in many legal systems worldwide.)

[0071] The virtualization layer (1220) provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers, virtual storage, virtual networks, including virtual private networks, virtual applications and operating systems, and virtual clients.

[0072] In one example, the administration layer (1230) can provide the following functions: resource provisioning, metering and pricing, a user portal, service layer management, and SLA planning and fulfillment. Resource provisioning provides dynamic procurement of compute resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing provide cost monitoring for resources used within the cloud computing environment, as well as billing and invoicing for the use of these resources. In one example, these resources could include application software licenses. Security provides identity verification for cloud customers and tasks, as well as protection for data and other resources. A user portal provides access to the cloud computing environment for customers and system administrators.Shift management provides the allocation and management of cloud computing resources to ensure that required shifts are met. Shift agreement (SLA) planning and implementation provide the preparation and procurement of cloud computing resources for anticipated future demand, in accordance with the SLA.

[0073] A workload layer (1240) provides examples of the functionality for which the cloud computing environment can be used. Examples of workloads and functions that can be provided by this layer include, but are not limited to: mapping and navigation; software development and lifecycle management; delivery of training in virtual classrooms; data analytics processing; transaction processing; and federated machine learning.

[0074] It is clear that the present disclosure is of a system, a method, a device and a computer program product for evaluating inputs in natural language, for recognizing a question in a corresponding communication and for resolving the recognized question with an answer and / or supporting content.

[0075] While certain embodiments of the present embodiments have been shown and described, it is obvious to a person skilled in the art that changes and modifications can be made based on the teachings contained herein without departing from the embodiments and their broader aspects. Therefore, the scope of the accompanying claims is intended to encompass all changes and modifications that fall within the actual scope of application of the embodiments. Furthermore, it should be noted that the embodiments are defined exclusively by the accompanying claims. A person skilled in the art understands that if a specific number of an introduced claim element is intended, this intention will be expressly stated in the claim, and that if no such statement is made, no such limitation exists.As a non-restrictive example and for better understanding, the introductory phrases "at least one" and "one or more" are used in the following claims to introduce claim elements. However, the use of such phrases should not be interpreted as limiting a particular claim containing such an inserted claim element to embodiments containing only one such element, even if the same claim contains the introductory phrases "one or more" or "at least one" and indefinite articles such as "one / e"; the same applies to the use of definite articles in the claims.

[0076] The present embodiments may be a system, a method, and / or a computer program product. Furthermore, selected aspects of the present embodiments may take the form of an exclusively hardware implementation, an exclusively software implementation (including firmware, native software, microcode, etc.), or an embodiment that combines software and / or hardware aspects and which may be referred to hereinafter, more generally, as a "circuit," a "module," or a "system." In addition, aspects of the present embodiments may take the form of a computer program product contained in a computer-readable storage medium (or media) on which computer-readable program instructions are located to instruct a processor to execute aspects of the present embodiments.In such a form, the disclosed system, method and / or computer program product can improve the functionality and operation of an AI platform to resolve queries with the identification of intents and a corresponding response in relation to the identified intents.

[0077] A computer-readable storage medium can be a physical unit capable of retaining and storing instructions for use by a unit to execute instructions. For example, a computer-readable storage medium can be an electronic storage unit, a magnetic storage unit, an optical storage unit, an electromagnetic storage unit, a semiconductor storage unit, or any suitable combination thereof, without limitation. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, dynamic or static random-access memory (RAM), read-only memory (ROM), and erasable programmable read-only memory (EPROM).Flash memory), a magnetic storage unit, a portable compact storage disk-read-only storage device (CD-ROM), a DVD (digital versatile disc), a memory stick, a floppy disk, a mechanically coded unit such as punched cards or raised structures in a groove on which instructions are stored, and any suitable combination of these. A computer-readable storage medium shall not, in its use herein, be understood as volatile signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses traveling through an optical fiber cable), or electrical signals transmitted by a wire.

[0078] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to individual data processing units or, via a network such as the internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission lines, wireless transmission, routing computers, firewalls, switching units, gateway computers, and / or edge servers. A network adapter card or network interface in each data processing unit receives computer-readable program instructions from the network and forwards them for storage on a computer-readable storage medium within the respective data processing unit.

[0079] Computer-readable program instructions for executing work steps of the present embodiments may be assembly instructions, ISA (Instruction Set Architecture) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., as well as conventional procedural programming languages ​​such as the programming language "C" or similar programming languages.The computer-readable program instructions can be executed entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer, server, or server cluster. In the latter case, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be established with an external computer (for example, via the internet using an internet service provider).In some embodiments, electronic circuits, including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can execute computer-readable program instructions by using state information from the computer-readable program instructions to personalize the electronic circuits to perform aspects of the present embodiments.

[0080] Aspects of the present embodiments are described herein with reference to flowcharts and / or block diagrams of processes, devices (systems), and computer program products according to the embodiments. It is understood that each block of the flowcharts and / or block diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be executed by means of computer-readable program instructions.

[0081] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or any other programmable data processing device to create a machine, such that the instructions executed by the processor of the computer or other programmable data processing device will generate a means of implementing the functions / steps specified in the block(s) of the flowcharts and / or block diagrams.These computer-readable program instructions may also be stored on a computer-readable storage medium capable of controlling a computer, programmable data processing device and / or other units to function in a particular manner, such that the computer-readable storage medium on which instructions are stored has a manufactured product, including instructions that implement aspects of the function / step specified in the block(s) of the flowchart and / or block diagrams.

[0082] The computer-readable program instructions can also be loaded onto a computer, other programmable data processing device or other unit to cause the execution of a series of process steps on the computer or other programmable device or other unit in order to generate a process executed on a computer, such that the instructions executed on the computer, other programmable device or other unit implement the functions / steps specified in the block(s) of the flowcharts and / or block diagrams.

[0083] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, processes, and computer program products according to various embodiments of the present embodiments. In this context, each block in the flowcharts or block diagrams can represent a module, segment, or part of instructions that includes one or more executable instructions for performing the specific logical function(s). In some alternative implementations, the functions specified in the block may occur in a different order than shown in the figures. For example, two blocks shown consecutively may in reality be executed essentially simultaneously, or the blocks may sometimes be executed in reverse order, depending on the corresponding functionality.It should also be noted that each block of the block diagrams and / or flowcharts, as well as combinations of blocks in the block diagrams and / or flowcharts, can be implemented by special hardware-based systems that perform the specified functions or steps, or execute combinations of special hardware and computer instructions.

[0084] It is understood that, although specific embodiments have been described herein for illustrative purposes, various modifications can be made without deviating from the scope of the invention. Accordingly, the scope of protection of the embodiments is limited only by the following claims and their equivalents.

Claims

[1] System which has the following features: a processing unit that is functionally connected to a memory; an AI (Artificial Intelligence) platform that exchanges data with the processing unit, wherein the AI ​​platform is designed to train a machine learning (ML) model, wherein the AI ​​platform has the following features: a registration manager to register participating entities in a cooperative relationship, to arrange the registered entities in a topology, and to establish a topological data transmission protocol; an encryption manager to generate a public additive homomorphic encryption key, AHE key, and distribute it to each registered entity; an entity manager to control direct local encryption of machine learning model weights of local entities on registered entities using a distributed public AHE key, to selectively aggregate the encrypted local machine learning model weights including distributing the aggregated encrypted weights to one or more other participating entities in the topology, or to support local aggregation on the registered entities according to the topological data transmission protocol; where the encryption manager decrypts an aggregated sum of the encrypted local machine learning model weights with a corresponding private AHE key, and distributes the decrypted aggregated sum to each entity in the topology. [2] System according to claim 1, wherein a single participating entity has two or more internal entities, and further comprising that the entity manager is designed to: To aggregate weights from one or more machine learning models that are locally connected to the two or more internal units; and to encrypt the aggregated weights locally with the public AHE key, where the aggregated weights represent a homogeneous data type. [3] System according to claim 2, further comprising that the entity manager is designed to receive the decrypted aggregated sum from the encryption manager and to pass the aggregated sum to the two or more locally connected ML models. [4] System according to claim 1, wherein the topology is a ring topology, and further comprising that the registration manager is designed to assign a rank to each participating entity in the topology and to incrementally encode ML model weights and aggregate them in a first topological direction which depends on the assigned rank in the topology. [5] System according to claim 4, further comprising that the registration manager is designed to modify the first topological direction depending on an available data transmission bandwidth. [6] System according to claim 1, further comprising that the registration manager is designed to arrange the participating units in a fully connected topology, and further comprising: that the entity manager is designed to use a broadcast protocol, wherein each participating entity sends the encrypted local ML model weights topology-wide via broadcast message, and wherein the selective aggregation further features that each participating entity is designed to locally aggregate received, broadcast-sent, encrypted weights; and The encryption manager is designed to subject each local aggregation to a participation check. [7] System according to claim 1, further comprising that the entity manager is designed to represent the local ML model weights as a field of weights, to divide the encrypted field into a plurality of two or more blocks, wherein a number of blocks is an integer representing a number of registered participants, to encrypt each block locally with the public AHE key, and to aggregate the blocks in parallel and synchronously depending on the topology. [8] Computer program product for training a machine learning model, wherein the computer program product comprises a computer-readable storage medium containing program code, wherein the program code is executable by a processor, and comprises the following: Registering participating entities in a cooperative relationship, arranging the registered entities in a topology, and setting up a topological data transmission protocol; Generating and distributing a public additive homomorphic encryption key (AHE key) to each registered participating entity; a direct local encryption of registered entities using machine learning model weightings of the entities with the distributed public AHE key, Direct aggregation of the encrypted local machine learning model weights by the registered participating entities, including distribution of the encrypted weights or aggregated encrypted weights to one or more entities in the topology according to the topological data transmission protocol; and Decrypting an aggregated sum of the encrypted local machine learning model weights with a corresponding private AHE key and distributing the decrypted aggregated sum to each entity in the topology. [9] Computer program product according to claim 8, wherein a single participating entity contains two or more internal entities, and further comprising program code designed to: To aggregate weights from one or more machine learning models that are locally connected to the two or more internal units; and to encrypt the aggregated weights locally with the public AHE key, where the aggregated weights represent a homogeneous data type. [10] Computer program product according to claim 9, further comprising program code for receiving the decrypted aggregated sum and for passing the aggregated sum to the two or more internal units. [11] Computer program product according to claim 8, wherein the topology is a ring topology, and further comprising program code designed to assign a rank to each participating entity in the topology and to incrementally encode ML model weights and aggregate them in a first topological direction which depends on the assigned rank in the topology. [12] Computer program product according to claim 11, further comprising that the program code is designed to modify the first topological direction depending on an available data transmission bandwidth. [13] Computer program product according to claim 8, further comprising program code designed to represent the local ML model weights as a field of weights, to divide the encrypted field into a plurality of two or more blocks, wherein a number of blocks is an integer representing a number of registered participants, to encrypt each block locally with the public AHE key, and to aggregate the blocks in parallel and synchronously depending on the topology. [14] Computer program product according to claim 8, wherein the topology is fully interconnected, and further comprising program code designed to: to send the encrypted local ML model weights topology-wide via broadcast message; to locally aggregate encrypted weights received via broadcast message; and to subject each local aggregation to an entity participation review. [15] Methods comprising: Register participating entities in a cooperative relationship to train a machine learning model; Arranging the registered participating entities in a topology and setting up a topological data transmission protocol; Receiving, by each registered participating entity, a public additive homomorphic encryption key (AHE key) from an encryption manager and encrypting, by each registered participating entity, local machine learning model weights with the received key; Aggregating the encrypted local machine learning model weights, including distributing the encrypted weights or aggregated encrypted weights to one or more other participating entities in the topology, depending on the topological data transmission protocol; and Decrypting an aggregated sum of the encrypted local machine learning model weights of the registered participating entities with a corresponding private AHE key at the encryption manager and distributing the decrypted aggregated sum of the local weights to the registered participating entities. [16] The method of claim 15, wherein a single participating entity has two or more internal entities, and further comprising: Aggregating weights from one or more ML models that are locally connected to the two or more internal units; Local encryption of the aggregated weights with the public AHE key, where the aggregated weights represent a homogeneous data type; and Receiving the decrypted aggregated sum by the individual participating entity and forwarding the aggregated sum to the two or more internal entities. [17] Method according to claim 15, wherein the topology is a ring topology, and further comprising assigning a rank to each participating entity in the topology and incrementally encoding and aggregating ML model weights in a first topological direction which depends on the assigned rank in the topology. [18] Method according to claim 15, further comprising representing the local ML model weights as an array of weights, subdividing the encrypted array into a plurality of two or more blocks, wherein a number of blocks is an integer representing a number of registered participants, locally encrypting each block with the public AHE key and synchronously aggregating the blocks in parallel and depending on the topology. [19] Method according to claim 18, further comprising completing the synchronous aggregation when each participating entity has received a single aggregated block, transferring the single aggregated block to a decrypting entity, performing a decryption with the corresponding private AHE key on the transferred block, concatenating the decrypted blocks, and distributing the concatenated decrypted blocks to the registered participating entities. [20] The method of claim 15, wherein the topology is fully connected, and further comprising: Topology-wide transmission of the encrypted local ML model weights via broadcast message by each participating entity; wherein the selective aggregation further involves local aggregation by each participating entity of encrypted weights received via broadcast message; and Performing an entity participation check on each local aggregation.

Images