Learning a dictionary of patterns from noisy numerical data in distributed networks

By using a collaborative learning framework in joint learning of machine learning, using consensus between network nodes to build a consensus collection of pattern dictionaries and local statistics, the problem of traditional security concepts being difficult to verify data consistency and protect IoT data security is solved, and the problem of traditional security concepts is realized. Efficient data privacy protection and computing resource conservation, suitable for a variety of application scenarios.

CN114503505BActive Publication Date: 2024-11-19INTERNATIONAL BUSINESS MACHINE CORPORATION
View PDF 14 Cites 0 Cited by

Patent Information

Application Number
CN202080066780.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-10-16
Filing Date
2020-10-12
Publication Date
2024-11-19
Estimated Expiration
2040-10-12

AI Technical Summary

Technical Problem

In the field of joint learning of machine learning, traditional security concepts are difficult to effectively verify data consistency, especially when processing noise multivariate real-valued data generated by IoT devices, existing encryption technologies cannot effectively protect data security, and the security requirements are: The security of financial transactions is different. Traditional methods have problems such as high consumption of computing resources and difficulty in protecting data privacy.

Method used

A collaborative learning framework is proposed to build a pattern dictionary based on consensus among multiple client systems through multiple network nodes interconnected through networks, calculate the consensus set of local statistics, and update the pattern dictionary to realize collaborative learning, avoiding Encrypted multi-party computing overhead, transactions are suitable for large amounts of low-level sensor data for IoT devices, allowing each client system to learn based on secure collaborative learning without sharing raw data.

Benefits of technology

Achieve security and data privacy protection for collaborative learning in a decentralized multi-task environment, reducing the use of computing resources, suitable for anomaly detection of industrial assets, biomarker disease analysis, anti-money laundering models of financial transaction data and battery life Applications such as prediction have improved data security and computing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114503505B_ABST
    Figure CN114503505B_ABST
Patent Text Reader

Abstract

A collaborative learning framework is presented. The collaborative framework is implemented by a plurality of network nodes interconnected by a network. The network nodes belong to a plurality of client systems of the framework. A network node belonging to a first client system builds a prediction model for the first client system by using a pattern dictionary built based on a consensus among the plurality of client systems. The network node computes a set of local statistics for the first client system based on raw data of the first client system. The network node computes a consensus set of local statistics by aggregating a set of local statistics from the plurality of client systems. The network node updates a pattern dictionary based on a current value of the pattern dictionary and the consensus set of local statistics.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure generally relates to computing over distributed networks. Background Art

[0002] The design principles of decentralized, secure and transparent transaction management provide new insights into machine learning algorithms, especially in the field of federated learning. In addition, traditional security concepts are partially replaced by random methods when verifying data consistency. Summary of the invention

[0003] Some embodiments of the present disclosure provide a collaborative learning framework implemented by multiple network nodes interconnected by a network. The network nodes belong to multiple client systems of the framework. The network node belonging to a first client system builds a prediction model of the first client system by using a pattern dictionary established based on a consensus between multiple client systems. The network node calculates a set of local statistics of the first client system based on raw data of the first client system. The network node calculates a consensus set of local statistics by aggregating a set of local statistics from multiple client systems. The network node updates the pattern dictionary based on the current value of the pattern dictionary and the consensus set of local statistics.

[0004] In some embodiments, the network node computes a consensus set of local statistics by distributing different subsets of the set of local statistics of the first client system to different network nodes belonging to different client systems. The different subsets of the set of local statistics of the first client system are distributed to network nodes identified based on a randomly generated incident matrix.

[0005] According to one aspect, a computer-implemented method for collaborative learning by multiple network nodes interconnected by a network, the multiple network nodes belonging to multiple client systems, the method comprising: at a first network node belonging to the first client system, building a prediction model for the first client system by using a pattern dictionary built based on a consensus among the multiple client systems; calculating a set of local statistics for the first client system based on original data of the first client system; calculating a consensus set of local statistics by aggregating a set of local statistics from the multiple client systems; and updating the pattern dictionary based on a current value of the pattern dictionary and the consensus set of local statistics.

[0006] According to another aspect, a computer program product is provided for implementing a network node among a plurality of network nodes interconnected by a network, the plurality of network nodes belonging to a plurality of client systems in a collaborative learning framework, the computer program product comprising: one or more non-transitory computer-readable storage devices and program instructions stored on at least one of the one or more non-transitory storage devices, the program instructions being executable by a processor, the program instructions comprising an instruction set for the following operations: at a first network node belonging to a first client system, building a prediction model for the first client system by using a pattern dictionary built based on a consensus among a plurality of client systems; calculating a set of local statistics for the first client system based on raw data of the first client system; calculating a consensus set of local statistics by aggregating a set of local statistics from a plurality of client systems; and updating a pattern dictionary based on a current value of the pattern dictionary and the consensus set of local statistics.

[0007] According to another aspect, a computing device is provided for implementing a network node among a plurality of network nodes interconnected by a network, the plurality of network nodes belonging to a plurality of client systems in a collaborative learning framework, the computing device comprising: a processor; and a storage device storing an instruction set, wherein execution of the instruction set by the processor configures the computing device to perform actions, the actions comprising: at a first network node belonging to a first client system, building a prediction model for the first client system by using a pattern dictionary built based on a consensus among the plurality of client systems; calculating a set of local statistics of the first client system based on original data of the first client system; calculating a consensus set of local statistics by aggregating a set of local statistics from the plurality of client systems; and updating the pattern dictionary based on a current value of the pattern dictionary and the consensus set of local statistics.

[0008] The foregoing summary is intended to serve as a brief introduction to some embodiments of the present disclosure. It is not meant to be an introduction or overview of all inventive subject matter disclosed herein. The following detailed description and the drawings referenced in the detailed description will further describe the embodiments described in the summary as well as other embodiments. Therefore, an overview, detailed description, and drawings are provided in order to understand all embodiments described by this document. In addition, the claimed subject matter is not limited by the illustrative details in the summary, specific embodiments, and drawings, but is defined by the appended claims, because the claimed subject matter may be embodied in other specific forms without departing from the spirit of the present subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings are illustrative embodiments. They do not show all embodiments. In addition or alternatively, other embodiments may be used. Details that may be obvious or unnecessary may be omitted to save space or for more effective description. Some embodiments may be implemented with additional components or steps and / or without all components or steps shown. When the same number appears in different drawings, it refers to the same or similar components or steps.

[0010] Figure 1 A decentralized system 100 implementing a collaborative learning framework is shown consistent with an exemplary embodiment.

[0011] Figure 2 A consensus node of different client systems performing collaborative dictionary learning is shown, consistent with an exemplary embodiment.

[0012] Figure 3a -c shows the consensus node of the client system performing collaborative dictionary learning according to the preferred embodiment.

[0013] Figure 4a -h shows an example of a cyclic graph used as an incidence matrix for consensus nodes.

[0014] Figure 5 A process 500 for performing collaborative dictionary learning is conceptually illustrated, consistent with an exemplary embodiment.

[0015] Figure 6 A block diagram of components of a data processing system is shown in accordance with an illustrative embodiment. DETAILED DESCRIPTION

[0016] In the following detailed description, numerous specific details are set forth by way of example in order to provide a thorough understanding of the relevant teachings. However, it should be appreciated that the present teachings may be practiced without these details. In other instances, well-known methods, procedures, components, and / or circuits have been described at a relatively high level without detail in order to avoid unnecessarily obscuring aspects of the present teachings.

[0017] Collaborative learning using IoT (Internet of Things) devices is one of the applications of distributed learning. The statistical nature of IoT data is different from transaction data between financial entities, which is mainly noisy multivariate real-valued data. Real-valued data may be too low-level to be protected by existing encryption techniques. In addition, the security requirements of IoT data are different from those of currency transfers. For many IoT applications, high-level statistics such as yields are often more interesting than the exact values ​​of individual data samples.

[0018] Some embodiments of the present disclosure provide a framework for collaborative learning in a decentralized multi-tasking environment. The framework performs collaborative learning in three steps: local update, consensus building, and optimization. The computing device of the client system implementing the collaborative learning framework can achieve consensus building and data privacy without the computational overhead of encrypted multi-party computation, especially for transactions involving large amounts of low-level sensor data from IoT devices. In this way, the computing device implementing the client system discussed in this article uses fewer computing resources, including computing cycles and memory.

[0019] The collaborative learning framework can be used for collaborative anomaly detection of industrial assets owned by different companies. The collaborative learning framework allows each client system to learn S anomaly detection models customized for each asset based on a secure collaborative effort. Other examples of using the collaborative learning framework include: collaborative disease or symptom analysis based on biomarkers collected by various medical institutions; collaborative anti-money laundering models based on transaction data from various financial entities; and battery life prediction based on battery sensor data from different electric vehicles.

[0020] For some embodiments, Figure 1 A decentralized system 100 implementing a collaborative learning framework consistent with an embodiment of the present disclosure is shown. As shown, the decentralized system 100 is implemented on a network 105 of S client systems interconnected including client systems 111-119 (client systems 113-118 are not shown). Each client system may belong to a company, a financial institution, a government agency, a private residence, or other types of entities that may be interested in utilizing useful information from other client systems. Each client system or company wishes to improve its own predictive model by utilizing data owned by other client systems, but does not wish to share its own data with its competitors or external entities.

[0021] Each client system uses one or more computing devices, including computing devices connected to network 105. It is expected that the administrator of network 105 performs bookkeeping work, such as managing communication paths, clock synchronization, etc. The administrator of network 105 is not allowed to see the raw data of the client. Each client system includes one or more computing devices connected to network 105 as a network node. The network nodes of the client system may include client nodes and consensus nodes as partners. As shown, client system 111 has client node 121 and consensus node 131, client system 112 has client node 122 and consensus node 132, and client system 119 has client node 129 and consensus node 139. The consensus node is a network node of network 105. In some embodiments, some or all client nodes are not network nodes of network work 105 and are not directly connected to network 105.

[0022] A client node is the owner of the local data of a client system, for example by connecting to a database of the client system. A client node of a client system may act as a manager of a network node (e.g., a consensus node) of the client system. In some embodiments, multiple client nodes may be connected to a consensus node. A client node communicates only with pre-selected, trusted partner consensus nodes (e.g., consensus nodes of the same client system). Consensus nodes communicate with other consensus nodes to establish consensus (in the form of a pattern dictionary). For example, client node 121 is the owner of the local data of client system 111, and sends and receives data belonging to client system 111. Consensus node 131 is a trusted partner consensus node of client node 121, and receives statistics from client node 121. Consensus node 131 communicates with consensus nodes belonging to other client systems, including consensus node 132 of client system 112 and consensus node 139 of client system 119.

[0023] The client node and its partner consensus node may be implemented on the same computing device. For example, the client node 122 and the consensus node 132 of the client system 112 are implemented by the same computing device 102. The client node and its partner consensus node may also be implemented by different computing devices. The two computing devices implementing the client node and the consensus node may be connected via a dedicated connection within the client system. The two computing devices may also be in different physical locations and interconnected via the network 105. In some embodiments, the network traffic between the client node of the client system and its partner consensus node may be encrypted according to the privacy requirements of the client, while the network traffic between the consensus nodes of different client systems is not encrypted. In some embodiments, in order to avoid leakage of client original data from the client system, the client node is not connected to the network.

[0024] Figure 2 Consensus nodes of different client systems performing collaborative dictionary learning according to an embodiment of the present disclosure are shown. The consensus nodes of different client systems communicate with each other to collaboratively build a pattern dictionary, which can be used by each client system to build a predictive model. The figure shows the operation of the consensus node 139 of the client system 119 (client S). The consensus node 139 receives client-specific data from the partner client node 129 (not shown) of the client system 119. The consensus node 139 communicates with other consensus nodes of other client systems (e.g., the consensus node 131 of client 1 and the consensus node 132 of client 2) through the network 105 to collaboratively build and update the pattern dictionary, which is stored in the blockchain 239.

[0025] In some embodiments, the computing device implementing client system 119 implements consensus node 139 and blockchain 239. In some embodiments, consensus node 139 and blockchain 239 are modules of software instructions executed by one or more processing units (e.g., processors) of computing device 105. In some embodiments, consensus node 139 and blockchain 239 are modules of hardware circuits implemented by one or more integrated circuits (ICs) of an electronic device. Figure 6 An example computing device 600 is depicted upon which the client system 119 may be implemented.

[0026] Consensus node 139 is based on the original data 219 (x S ), mixed weight 229(π S ) and the current content of the pattern dictionary 239, as well as the consensus information 250 exchanged with other consensus nodes through the network 105, generate an update of the pattern dictionary Θ (stored in the blockchain 239). The original data 219 and the mixing weights 229 are information dedicated or proprietary to the client system 119 (client S). The pattern dictionary 239 stores the collaborative learning results of different consensus nodes of different client systems. The content of the blockchain 239 (pattern dictionary Θ) is the consensus between different client systems. The pattern dictionary includes K model parameters or words {θ1,...θ K}, where each word corresponds to a mixed component used to calculate the prediction model 249. The client system 119 mixes the corresponding component pattern dictionary 239, the original data 219, and the mixing weights 229 to generate a prediction model 249 for the client system 119. The prediction model 249 is used by an application of the client system, such as to provide incentives to the control module 259. In some embodiments, the application of the client system 119 including the prediction model 249 and the control module 259 is not connected to the network 105 to avoid leakage or client data.

[0027] Original data 219(x S ) is data, statistics, or measurements that are specific or proprietary to a client system 119 (client S) and not shared with other client systems. In some embodiments, each client maintains its own raw data privately in a database. For some embodiments, the raw data 219 may include noisy multivariate real-valued data, such as measurements or sensor data generated by IoT devices owned by the client system 119. In some embodiments, the raw data 219 is provided to the consensus node 139 by its partner client nodes over the network 105.

[0028] Mixing weight 229(π S ) is used to determine how much weight to give to a word or mixture component of the pattern dictionary 239. When compared with the private raw data 219 (x S) When mixing, the mixing weight Applied to the pattern dictionary {θ1,...θ K} to determine the prediction model P of the client system S (client system 119) s .

[0029] The pattern dictionary 239 stores the words or mixed components {θ1, . . . θ K} blockchain structure. The blockchain is an immutable record that can be used for auditing, pricing, evaluation, etc. The results of secure collaborative learning are stored in the blockchain at each client system. Based on the consensus building algorithm, the blockchains of different client systems (blockchains 231-239 of client systems 111-119) have the same content. In some embodiments, when an iteration or round of the consensus building algorithm is completed, a new block is appended to the blockchain. Each block contains an updated agreed-on pattern dictionary. The learned dictionary contains a set of model parameter vectors as words {θ1,...θ K}, each word is a multivariate real value. When the convergence criteria are met, the pattern dictionary stops updating and the blockchain is "locked". The final dictionary word is stored in the last block of the blockchain.

[0030] The client systems s=1,...S privately have their own raw data such that:

[0031]

[0032] in:

[0033] N s is the number of samples for the sth client, and

[0034] M is the dimensionality of the data, which is assumed to be common across all client systems.

[0035] The prediction model of the client system S is the probability distribution p s , which is determined according to the following formula:

[0036]

[0037] in:

[0038] {θ1,...θ K} is the mixed component of the pattern dictionary, and is the corresponding mixing weight of the mixing component of client system s;

[0039] The capital K is the number of patterns in the pattern dictionary;

[0040] Lowercase k is the index of the pattern in the pattern dictionary;

[0041] Capital S is the number of entities in the network; and

[0042] The lowercase s is the index of the client in the network.

[0043] The computing device at the client system s may determine the predicted distribution p based on the collaborative determination according to (2). s To make a determination and / or trigger a corresponding action. For example, the collaborative learning system can be used to detect anomalies based on sensor data from industrial assets of various companies. In some embodiments, the anomaly score can be calculated as a negative log likelihood ratio according to the following formula:

[0044]

[0045] in is a new measurement made at the client system s.

[0046] For some embodiments, the problem of collaborative dictionary learning can be formalized as collaborative density estimation of a mixture of exponential families. Based on the exponential family, the collaborative learning framework performs collaborative dictionary learning in three steps: local update, consensus building, and optimization.

[0047] The exponential family of probability densities is expressed as:

[0048]

[0049] in:

[0050] is the vector (or matrix) transpose;

[0051] H(.), G(.) are scalar functions; and

[0052] η(.), T(.) are (column) vector functions that satisfy the required mathematical properties as probability distribution functions.

[0053] T(x s ) In particular, based on the original data x of client s s A statistic is sufficient for a family of probability distributions (e.g., the exponential family) if the sample from which the statistic is computed gives no additional information beyond the statistic, where the additional information is information about the probability distribution of the population from which the sample was taken. Based on equation (4), the dictionary word {θ k}:

[0054]

[0055] In Equation 5, all terms that are independent of Θ are discarded. is a set of local sufficient statistics of the raw data of client system s based on the mixture component k. is the sample size of the client system for mixture component k. N s is the total sample size of all mixed components at client system s. For example, the collaborative learning system can be used to detect violations in different financial institutions (as different client systems), N s represents the total number of accounts at a particular bank, and Represents the number of accounts with specific characteristics (e.g., accounts with funds over one million) at a specific bank.

[0056] According to the following formula, different client systems and Aggregate into a consensus set of sample size N k and the consensus set T of sufficient statistics k :

[0057]

[0058] Therefore, the dictionary word {θ can be found by maximizing expression (5) according to the following formula k}:

[0059]

[0060] Since all terms in Eq. (7) are already available to the client systems, Eq. (7) can be evaluated without any communication between the client systems. Formalizing collaborative learning as an exponential family allows performing collaborative learning in three distinct steps: (i) local update, (ii) consensus, and (iii) optimization. During the local update step, the client system determines and This can be determined entirely at client system s without communicating with other client systems. During the consensus step, client system s communicates with the consensus nodes of other client systems to compute the aggregated consensus N based on equation (6) k and T k During the optimization step, the client system calculates θ based on equation (7) k This can be done entirely within the client system based on the aggregated N k and T k to execute without communicating with other consensus nodes.

[0061] Figure 3a-c shows a consensus node of a client system performing collaborative dictionary learning. Specifically, the figure shows that the consensus node 139 of the client system 119 performs the steps of local update, consensus establishment and optimization to determine the parameters of the exponential family (e.g., the aggregate sufficient statistics T k and the aggregated sample size N k ) and updates the contents of the pattern dictionary 239 by communicating with other consensus nodes via network 105.

[0062] Figure 3a 2 shows the local update step of collaborative dictionary learning. As shown, the consensus node 139 receives the mixing weights 229 of the client system 119 and the original data 219 (e.g., from a partner client node). The consensus node 139 then calculates the sufficient statistics of the client system 119 (client s) based on the original data 219. and sample size Local updates of the consensus node. In this operation, the consensus node does not use information from other client systems. Sufficient statistics for calculation / update and sample size Stored locally in the client system 119 as collaboration data 319 .

[0063] In some embodiments, the consensus node is based on the following formula (Mixed Weight 229) Calculate sample weight and

[0064]

[0065]

[0066] in is the sample weight of the nth sample of the kth mixture component of client s.

[0067] In some embodiments, sufficient statistics for the mixture component k at the client system s Can be defined as:

[0068]

[0069] Figure 3b 1 shows the steps of establishing consensus for collaborative dictionary learning. As shown in the figure, consensus node 139 communicates with consensus nodes of other client systems through network 105 to obtain consensus nodes of other client systems. and value, and the local and The value (from collaboration data 319) is sent to the consensus nodes of other client systems. The consensus node 139 then and The values ​​are aggregated to the consensus sufficient statistics T for different mixture components. k and consensus sample size N k (e.g., by using Equation (6)). The consensus sufficient statistic T k and consensus sample size N k Also stored as collaboration data 319 .

[0070] Figure 3c 1 shows the optimization steps of collaborative dictionary learning. In this operation, consensus node 139 uses consensus sufficient statistics T k and consensus sample size N k To determine the mode dictionary θ for different mixture components k The updated content of is stored in the blockchain 239. k and N k Aggregated at the consensus node 139 (stored as collaboration data 319), so they are already available to the consensus node 139 without communicating with other client systems.

[0071] In some embodiments, for T k and N k The summation of equation (6) can be performed element-wise and can be generalized as the sum of scalars:

[0072]

[0073] in:

[0074] ξ s is the set of local statistics of client system s (e.g., sufficient statistics or sample size );as well as

[0075] It is a consensus set of local statistics aggregated from different client systems.

[0076] In some embodiments, the communication paths between consensus nodes of different client systems are restricted based on a connected graph A. The connected graph A is also called an association matrix, where "1" indicates the presence of an edge or path and "0" indicates the absence of an edge or path. In each consensus node of the client system, the set of local statistics ξ is iteratively updated according to the following formula s

[0077]

[0078] where ∈ is a parameter that controls convergence.

[0079] In some embodiments, ∈ is selected based on the incidence matrix A as ∈=1 / dmax , where d max is the maximum node degree or number of associated edges of the consensus node. In some embodiments, in order to protect data privacy, the set of local statistics of the client system s is s Can be divided into N c Chunks So that:

[0080]

[0081] The client system can determine how to s Split into N c In some embodiments, Figure 3b During each iteration of the consensus building step, the consensus node 119 randomly generates an association matrix and distributes chunks or subsets of the set of local statistics to the consensus nodes of other client systems based on the randomly generated association matrix. Randomly distributing subsets of local statistics to other consensus nodes during consensus building enhances the security of the client system because other client systems cannot reconstruct the private original data.

[0082] If the association matrix A is non-overlapping and the routers always choose a different A from the previous iteration of consensus building and chunking local statistics, then even if N c = 2 also fully guarantees privacy. Even when it is impossible to generate multiple non-overlapping association matrices (which may happen when A is dense or when the router cannot keep a memory of previously used A), the probability of privacy leakage P b Upper limit is

[0083]

[0084] where d max is the maximum degree of the incidence matrix A. As long as A is sparse, then d max Much smaller than S, P b By choosing N to be sufficiently large c , which can make P b Arbitrarily small.

[0085] In some embodiments, to ensure equality between different consensus nodes, a regular and sparse cyclic graph is used as the incidence matrix. Figure 4a -h shows an example of a cyclic graph used as an incidence matrix for consensus nodes. In the graph, each vertex represents a consensus node, and each edge represents a communication path that a consensus node can use to communicate a subset of its local statistics to establish consensus.

[0086] In some embodiments, to improve convergence speed, the extended graph is used as an association matrix for distributing subsets of local statistics. Specifically, for each consensus node s=1,...S, the extended graph-based association matrix provides edges for consensus nodes s-1, s+1, and j, where j is a solution to (s-1)(j-1) mod S=1. Using the extended graph suggests ~log(S) iterations to achieve a given accuracy.

[0087] Figure 5 A process 500 for performing collaborative dictionary learning according to an exemplary embodiment is conceptually illustrated. Collaborative learning is performed by a plurality of network nodes interconnected by a network of a plurality of different client systems belonging to a collaborative learning framework. In some embodiments, one or more processing units (e.g., processors) of a computing device implementing a network node of a client system (e.g., consensus node 139 of client system 119) perform process 500 by executing instructions stored in a computer-readable medium.

[0088] The network node constructs (at 510) a prediction model for the first client system by using a pattern dictionary established based on a consensus among the plurality of client systems. The pattern dictionary may be stored in a hash chain data structure. The prediction model for the first client system may be based on the raw data (x S ), the set of mixing weights of the first client system (π S ) and the probability distribution (p s ). The raw data may include noisy real values ​​generated by one or more Internet of Things (IoT) devices. In some embodiments, the first network node is a consensus node of a first client system, and the raw data of the first client system is received encrypted from a client node of the first client system.

[0089] The network node calculates (at 520) a set of local statistics (ξ ) of the first client system based on the raw data of the first client system. s ). The set of local statistics may include the set of sufficient statistics (T S ) and the set of sample sizes (N S ). The network node may update the set of sample sizes for the first client system based on the set of sample weights for the first client system. In some embodiments, the network node also updates the sample weights for the first client system based on the raw data for the first client system and the pattern dictionary (r s(n) ) and the set of sample sizes (N S) is determined based on the sample weights. In some embodiments, the pattern dictionary, the set of local statistics, the set of sample sizes, the set of sample weights, and the set of mixing weights each include components corresponding to the set of mixing components (for k=1...K θ k , ).

[0090] Network nodes aggregate local statistics from multiple client systems (ξ s ) to calculate (at 530) a consensus set of local statistics For example, by aggregating the set of sufficient statistics To calculate the sufficient statistical consensus set (T k ) and / or by aggregating sample sizes from multiple client systems To calculate the consensus set of sample sizes (N k In some embodiments, the network node computes a consensus set of local statistics by distributing different subsets of the set of local statistics of the first client system to different network nodes belonging to different client systems. In some embodiments, different subsets of the set of sufficient statistics of the first client system are distributed to network nodes identified based on a randomly generated association matrix.

[0091] A consensus set of network nodes based on the current value of the pattern dictionary and local statistics To update (at 540) the pattern dictionary (θ). In some embodiments, the network nodes may return to 520 to compute the next iteration of local updates, consensus building, and optimization. When convergence or consensus is reached, for example, when local statistics of different client systems have the same value or are within a threshold of each other, process 500 may end.

[0092] The present application may be a system, method and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon, the computer-readable program instructions being used to cause a processor to perform aspects of the present disclosure.

[0093] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card or a raised structure in a groove on which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be interpreted as a temporary signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0094] The computer-readable program instructions described herein can be downloaded to the corresponding computing / processing equipment from the computer-readable storage medium, or via a network, such as the Internet, a local area network, a wide area network and / or a wireless network, downloaded to an external computer or an external storage device. The network can include copper transmission cables, optical transmission optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers and / or edge servers. The network adapter card or the network interface in each computing / processing equipment receives the computer-readable program instructions from the network, and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in the corresponding computing / processing equipment. The computer-readable program instructions for performing the operation disclosed herein can be assembly instructions, instruction set architecture (ISA) instructions, machine-related instructions, microcodes, firmware instructions, state setting data, configuration data of integrated circuits, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages, such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as "C" programming languages ​​or similar programming languages). Computer readable program instructions can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider through the Internet). In some embodiments, electronic circuits including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) can execute computer readable program instructions to personalize the electronic circuits by utilizing the state information of computer readable program instructions, so as to perform aspects of the present disclosure.

[0095] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present disclosure. It will be understood that each frame of the flowchart and / or block diagram and the combination of frames in the flowchart and / or block diagram can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a computer or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the function / action specified in one or more frames of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can guide a computer, a programmable data processing device and / or other equipment to work in a particular manner, so that the computer-readable storage medium having instructions stored therein includes a product, and the product includes instructions for implementing various aspects of the function / action specified in one or more frames of the flowchart and / or block diagram.

[0096] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operation steps are performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more blocks of the flowchart and / or block diagram. Figure 5 ) illustrate the possible implementation architecture, functions and operations of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, segment or portion of an instruction, which includes one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions noted in the box may not occur in the order noted in the figure. For example, two boxes shown in succession may actually be executed substantially simultaneously, or the boxes may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each box illustrated in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented by a dedicated hardware-based system that performs a specified function or action or performs a combination of dedicated hardware and computer instructions.

[0097] Figure 6 6 is a block diagram showing components of data processing systems 600 and 650 that may be used to implement a client system or consensus node in a collaborative learning framework according to an illustrative embodiment of the present disclosure. It should be understood that Figure 6 This merely provides an illustration of one implementation and does not imply any limitation on the environments in which different embodiments may be implemented. Many modifications to the described environment may be made based on design and implementation requirements.

[0098] Data processing systems 600 and 650 represent any electronic device capable of executing machine-readable program instructions. Data processing systems 600 and 650 may represent a smart phone, a computer system, a PDA, or other electronic device. Examples of computing systems, environments, and / or configurations that may be represented by data processing systems 600 and 650 include, but are not limited to, personal computer systems, server computer systems, thin clients, fat clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, network PCs, minicomputer systems, and distributed cloud computing environments that include any of the above systems or devices.

[0099] Data processing systems 600 and 650 may include Figure 6 , and a set of internal components 605 and a set of external components 655, the set of internal components 605 includes one or more processors 620, one or more computer-readable RAMs 622, and one or more computer-readable ROMs 624 on one or more buses 626, as well as one or more operating systems 628 and one or more computer-readable tangible storage devices 630. One or more operating systems 628 and programs such as those used to perform process 500 are stored on one or more computer-readable tangible storage devices 630 for execution by one or more processors 620 via one or more RAMs 622 (which typically include cache memory). Figure 6 In the embodiment shown in , each of the computer-readable tangible storage devices 630 is a disk storage device of an internal hard drive. Alternatively, each computer-readable tangible storage device 630 is a semiconductor memory device such as ROM 624, EPROM, flash memory or any other computer-readable tangible storage device capable of storing computer programs and digital information.

[0100] The collection of internal components 605 also includes an R / W drive or interface 632 to read from and write to one or more portable computer-readable tangible storage devices 686, such as a CD-ROM, DVD, memory stick, tape, disk, optical disk, or semiconductor storage device. Instructions for performing process 500 may be stored on one or more of the respective portable computer-readable tangible storage devices 686, read via the respective R / W drive or interface 632, and loaded into the respective hard disk drive 630.

[0101] The collection of internal components 605 may also include a network adapter (or switch port card) or interface 636, such as a TCP / IP adapter card, a wireless Wi-Fi interface card, or a 3G or 4G wireless interface card or other wired or wireless communication link. The instructions of the above-mentioned process or program can be downloaded from an external computer (e.g., a server) via a network (e.g., the Internet, a local area network or other wide area network) and a corresponding network adapter or interface 636. From the network adapter (or switch port adapter) or interface 636, the instructions and data of the described program or process are loaded into the corresponding hard disk drive 630. The network may include copper wire, optical fiber, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers.

[0102] The set of external components 655 may include a computer display monitor 670, a keyboard 680, and a computer mouse 684. The set of external components 655 may also include a touch screen, a virtual keyboard, a touch pad, a pointing device, and other human interface devices. The collection of internal components 605 also includes a device driver 640 to interface with the computer display monitor 670, the keyboard 680, and the computer mouse 684. The device driver 640, the R / W driver or interface 632, and the network adapter or interface 636 include hardware and software (stored in the storage device 630 and / or the ROM 624).

[0103] One or more of the foregoing embodiments implement a client system or network node of a collaborative learning framework within a computer infrastructure by causing one or more computing devices to update local statistics. The computing device also exchanges updated local statistics with other client systems or network nodes over the network to establish consensus in the form of a pattern dictionary stored in a blockchain. The computing device also divides the updated local statistics into subsets and randomly distributes the subsets to other computing devices.

[0104] The description of various embodiments of the present disclosure has been presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements existing in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A computer-implemented method for collaborative learning by a plurality of network nodes interconnected by a network, the plurality of network nodes belonging to a plurality of client systems, the method comprising: At a first network node belonging to a first client system, constructing a prediction model of the first client system by using a pattern dictionary constructed based on consensus building among the plurality of client systems; enhancing the security of the first client system by splitting the set of local statistics of the first client system into subsets and randomly distributing the subsets of local statistics to other consensus nodes during the consensus establishment; calculating a set of local statistics of the first client system based on the raw data of the first client system; computing a consensus set of local statistics by aggregating the sets of local statistics from the plurality of client systems; as well as updating the pattern dictionary based on the current value of the pattern dictionary and the consensus set of local statistics, The set of local statistics includes a set of sufficient statistics of the raw data of the first client system and a set of sample sizes of the raw data of the first client system.

2. The computer-implemented method of claim 1 , wherein the predictive model for the first client system is based on the raw data for the first client system, a set of mixing weights for the first client system, and a probability distribution determined by the pattern dictionary.

3. The computer-implemented method of claim 2, wherein each of the pattern dictionary, the set of local statistics, and the set of mixing weights respectively comprises components corresponding to a set of mixing components.

4. The computer-implemented method of any one of claims 1 to 3, wherein the pattern dictionary is stored in a hash chain data structure.

5. The computer-implemented method of any one of claims 1 to 3, wherein the raw data comprises real-valued values ​​generated by one or more Internet of Things (IoT) devices.

6. The computer-implemented method of any one of claims 1 to 3, wherein aggregating the set of local statistics from the plurality of client systems comprises exchanging the set of local statistics between the plurality of network nodes over the network without encryption.

7. A computer-implemented method according to any one of claims 1 to 3, wherein the first network node is a consensus node of the first client system, and wherein the original data of the first client system is received from a client node of the first client system in an encrypted manner.

8. The computer-implemented method according to any one of claims 1 to 3, further comprising updating a set of sample weights for the first client system based on the original data of the first client system and the pattern dictionary, wherein the set of sample sizes is determined based on the sample weights.

9. A computer-implemented method according to any one of claims 1 to 3, wherein computing the consensus set of local statistics comprises distributing different subsets of the set of local statistics for the first client system to different network nodes belonging to different client systems.

10. The computer-implemented method of claim 9, wherein the different subsets of the set of local statistics of the first client system are distributed to network nodes identified based on a randomly generated affinity matrix.

11. A computing device for implementing a network node among a plurality of network nodes interconnected by a network, the plurality of network nodes belonging to a plurality of client systems in a collaborative learning framework, the computing device comprising: processor; as well as A storage device storing an instruction set, wherein execution of the instruction set by the processor configures the computing device to perform actions, the actions comprising: At a first network node belonging to a first client system, constructing a prediction model of the first client system by using a pattern dictionary constructed based on consensus building among the plurality of client systems; enhancing the security of the first client system by splitting the set of local statistics of the first client system into subsets and randomly distributing the subsets of local statistics to other consensus nodes during the consensus establishment; calculating a set of local statistics of the first client system based on the raw data of the first client system; computing a consensus set of local statistics by aggregating the sets of local statistics from the plurality of client systems; and updating the pattern dictionary based on the current value of the pattern dictionary and the consensus set of local statistics, The set of local statistics includes a set of sufficient statistics of the raw data of the first client system and a set of sample sizes of the raw data of the first client system.

12. The computing device of claim 11, wherein the prediction model of the first client system is a probability distribution determined based on the raw data of the first client system, a set of mixing weights of the first client system, and the pattern dictionary.

13. The computing device of claim 12, wherein each of the pattern dictionary, the set of local statistics, and the set of mixing weights respectively comprises a component corresponding to a set of mixing components.

14. The computing device of any one of claims 11 to 13, wherein the pattern dictionary is stored in a hash chain data structure.

15. The computing device of any one of claims 11 to 13, wherein the raw data comprises real value values ​​generated by one or more Internet of Things (IoT) devices.

16. The computing device of any one of claims 11 to 13, wherein aggregating the set of local statistics from the plurality of client systems comprises exchanging the set of local statistics between the plurality of network nodes over the network without encryption.

17. A computing device according to any one of claims 11 to 13, wherein the first network node is a consensus node of the first client system, and wherein the original data of the first client system is received from a client node of the first client system in an encrypted manner.

18. A computing device according to any one of claims 11 to 13, wherein the device is operable to perform steps including the following: updating a set of sample weights for the first client system based on the original data of the first client system and the pattern dictionary, wherein the set of sample sizes is determined based on the sample weights.

19. A computing device according to any one of claims 11 to 13, wherein calculating the consensus set of local statistics includes distributing different subsets of the set of local statistics of the first client system to different network nodes belonging to different client systems, wherein the different subsets of the set of local statistics of the first client system are distributed to network nodes identified based on a randomly generated association matrix.

20. A computer program product for implementing a network node of a plurality of network nodes interconnected by a network, the plurality of network nodes belonging to a plurality of client systems in a collaborative learning framework, the computer program product comprising: One or more non-transitory computer-readable storage devices and program instructions stored on at least one of the one or more non-transitory storage devices, the program instructions being executable by a processor, the program instructions comprising an instruction set for: At a first network node belonging to a first client system, constructing a prediction model of the first client system by using a pattern dictionary constructed based on consensus building among the plurality of client systems; enhancing the security of the first client system by splitting the set of local statistics of the first client system into subsets and randomly distributing the subsets of local statistics to other consensus nodes during the consensus establishment; calculating a set of local statistics of the first client system based on the raw data of the first client system; computing a consensus set of local statistics by aggregating the sets of local statistics from the plurality of client systems; as well as updating the pattern dictionary based on the current value of the pattern dictionary and the consensus set of local statistics, The set of local statistics includes a set of sufficient statistics of the raw data of the first client system and a set of sample sizes of the raw data of the first client system.

21. The computer program product of claim 20, wherein aggregating the set of local statistics from the plurality of client systems comprises exchanging the set of local statistics between the plurality of network nodes over the network without encryption.

22. A computer program product according to claim 20 or 21, wherein calculating the consensus set of local statistics includes distributing different subsets of the set of local statistics of the first client system to different network nodes belonging to different client systems, wherein the different subsets of the set of local statistics of the first client system are distributed to network nodes identified based on a randomly generated association matrix.

Citation Information

Patent Citations

  • Data dictionary-based distributed data unified access method

    CN107147683A

  • Distributed cloud storage system construction method based on block chain

    CN107249046A

  • Communication efficient federated learning

    CN107871160A

  • A data safety isolation method and device based on block chain storage

    CN109284624A

  • A data security access method and device based on a block chain

    CN109284625A