Cross-domain intelligent monitoring method and system based on Byzantine robustness privacy protection federated learning

Abstract

The invention relates to the technical field of intelligent monitoring, in particular to a cross-domain intelligent monitoring method and system based on Byzantine robustness privacy protection federated learning, a cloud server divides a base station into a plurality of clusters with similar data distribution through an overlapping community detection algorithm COPRA, and selects a leader node in each cluster; in each round of training, each base station obtains a gradient through training based on local monitoring data, the encrypted gradient is uploaded to a leader node of a cluster where the base station is located, the leader node identifies and filters a poisoning gradient, and then synchronous aggregation is executed to obtain an intra-cluster encrypted aggregation gradient; the cloud server firstly decrypts the intra-cluster encryption aggregation gradients, and then performs weighted aggregation on all intra-cluster aggregation gradients through availability weights to obtain global model parameters of the training of this round; and finally, each base station obtains a convergent intelligent monitoring model. According to the method, the influence of non-independent identically distributed data can be resisted, and poisoning attacks and privacy attacks can be effectively resisted.

Description

Technical Field

[0001] The present invention relates to the field of intelligent monitoring technology, and in particular to a cross-domain intelligent monitoring method and system based on Byzantine robustness privacy-preserving federated learning. Background Art

[0002] From 2015 to 2023, the market size of video surveillance equipment in my country has doubled from nearly US$60 billion to US$120 billion, with the growth rate remaining at around 8% in the near future, and the market has stabilized. Urban security, transportation, government projects, finance and commercial use account for more than 50%, showing great development potential. As a comprehensive technology, video surveillance not only realizes the functions of video acquisition, processing, control and emergency command of the monitoring site, but also has been widely used in security, missing persons and social empowerment. In the field of security, video surveillance systems effectively monitor and prevent criminal activities, improving the safety of communities and public places; in terms of missing persons, video surveillance systems can be used for search and rescue of missing persons, improving the efficiency and accuracy of search and rescue. At the same time, video surveillance technology provides data support and intelligent decision-making for various fields of society, and provides important support for urban management, traffic management, environmental monitoring, etc., thereby empowering society.

[0003] However, traditional intelligent monitoring systems usually rely on centralized artificial intelligence functions located in the cloud or data center to learn and analyze the identity and behavior of citizens or specific people. With the rapid growth of the amount of video stream data from modern cameras, this centralized solution is inefficient in terms of communication delay, and cannot achieve high availability and low cost, resulting in high-value monitoring data being discretely distributed across various edge devices, forming data island barriers. In addition, the monitoring data collected by each edge device usually involves privacy issues such as citizens' living habits and personal information. Once leaked, it will pose a great threat to the personal and property safety of citizens. In the future, with the changes in the nature of the institutions to which the cameras belong and the distribution of video data, centralized machine learning architectures will no longer be applicable. In this context, federated learning, as a distributed machine learning framework, allows multiple devices to collaboratively train machine learning models without sharing their original data, and can be effectively applied to intelligent monitoring systems. However, in the face of threats from malicious users and diverse attack methods, although traditional federated learning can solve the "data island" problem in intelligent monitoring to obtain sufficient data sets to support model accuracy, it cannot ensure data security throughout the joint modeling process. Based on the above requirements of the federated learning framework and the actual problems currently faced by intelligent monitoring systems in practical applications, this paper aims to conduct a demand analysis, mainly covering the following aspects:

[0004] (1) The surveillance data collected by cameras deployed in different areas are not independent and identically distributed.

[0005] For traditional federated learning, it is assumed that client data is independent and identically distributed. However, in monitoring scenarios, different cameras are deployed in different areas, and the monitoring data collected often presents non-independent and identically distributed (Non-IID) characteristics in data distribution. In this case, there are large differences in the intelligent models trained based on different monitoring data, which makes it difficult for the global model to converge. Therefore, there is an urgent need for a federated learning algorithm that can handle data heterogeneity to solve the problem of joint modeling of cross-domain intelligent monitoring systems.

[0006] (2) There are threats of advanced privacy attacks and poisoning attacks.

[0007] The use of federated learning architecture in intelligent monitoring systems may face the threat of privacy attacks and poisoning attacks, which may have a serious impact on the security and performance of the system. Specifically:

[0008] ① Existing attack methods such as model reversal and model extraction can infer the privacy information of the original training data based on the model parameters transmitted by federated learning, which means that intelligent monitoring based on federated learning still has the risk of privacy leakage.

[0009] ② In a distributed framework such as federated learning, any base station participating in federated learning may be a malicious node, uploading samples with misleading data to the federated learning system, such as manipulating image or video data to guide the model to make incorrect identification or prediction. This may cause the monitoring system to misidentify specific objects, events or behaviors, reducing the credibility and accuracy of the system. Summary of the invention

[0010] The present invention aims to solve the problem that federated learning is difficult to converge normally under non-independent and identically distributed data, and there are threats of privacy attacks and poisoning attacks. A cross-domain intelligent monitoring method and system based on Byzantine robust privacy-preserving federated learning are proposed, a clustered federated learning architecture is designed to resist the influence of non-independent and identically distributed data, and a robust aggregation strategy based on linear homomorphic encryption is designed to effectively resist poisoning attacks and privacy attacks, thereby ensuring data security and confidentiality.

[0011] To achieve the above purpose, the technical solution adopted is:

[0012] A cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning includes the following five stages:

[0013] Initialization phase: The key generation center generates an asymmetric key pair for the cloud server and each base station. The cloud server generates encrypted initialization global model parameters and sends them to all base stations.

[0014] Clustering stage: The cloud server divides the base stations with non-independent and identically distributed data into multiple clusters with similar data distribution through the overlapping community detection algorithm COPRA, and selects the leader node in each cluster;

[0015] In-cluster security training phase: In each round of training, each base station obtains gradients based on local monitoring data training, and uploads the encrypted gradients to the leader node of the cluster. The leader node identifies and filters poisoned gradients, and then performs synchronous aggregation to obtain encrypted aggregated gradients within the cluster.

[0016] Global robust aggregation stage: The cloud server first decrypts the encrypted aggregate gradients in the cluster, and then performs weighted aggregation on all aggregate gradients in the cluster using the availability weights to obtain the global model parameters for this round of training.

[0017] Real-time monitoring stage: When model training is completed, each base station obtains a converged intelligent monitoring model and sends the model to all cameras within the coverage area for real-time intelligent monitoring.

[0018] According to the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning of the present invention, further, in the initialization phase, a key generation center generates an asymmetric key pair for linear homomorphic encryption; the base station decrypts the encrypted initialization global model parameters after receiving them from the cloud server, and performs several rounds of training based on local monitoring data.

[0019] According to the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning of the present invention, further, in the clustering stage, the cloud server first constructs a weighted undirected graph based on the base station similarity adjacency matrix, in which the nodes represent base stations, and every two nodes have a weighted edge, and the weight is the similarity of the data distribution of the two base stations; then the above weighted undirected graph is used as input, and the COPRA algorithm is used to discover the hidden community structure in the graph; finally, the base stations are divided into different clusters, and the base stations in each cluster have similar data distribution.

[0020] According to the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning of the present invention, further, the method for calculating the similarity between base stations is: each base station uploads its own label vector X to the cloud server k =(x 1 ,x 2 ,…,x l )∈{0,1} l , where x i =1 means that there is label i in the local data set of base station k; the cloud server calculates the similarity of two base stations according to the Jaccard algorithm.

[0021] According to the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning of the present invention, further, selecting a leader node in each cluster includes: each base station uploads its resource parameters to the cloud server, and the cloud server selects the base station with the strongest computing and communication capabilities from each cluster as the leader node.

[0022] According to the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning of the present invention, further, identifying and filtering poisoned gradients by the leader node includes:

[0023] Given a set of gradients of cluster K [G k ]]=g 1 ,g 2 ,...,g n , first the leader node passes through the gradient [[g i ]] adds a random number [[r]] to mask it, and [[g i ]]·[[r]] is sent to the cloud server; then after receiving the ciphertext sent by all the leader nodes, the cloud server uses its own private key to decrypt it, calculates the median value, and encrypts it back to all the leader nodes. The leader node subtracts [[r]] from the ciphertext to finally obtain the ciphertext of the gradient median value;

[0024] For each encrypted gradient in the cluster, the leader node selects a random number noise as the power exponent of the encrypted gradient and calculates D i At the same time, the above random number noise is used as the power exponent of the ciphertext of the gradient median value to obtain D med ; Then D i and D med At the same time, it is sent to the cloud server; the cloud server uses the private key to decrypt and obtain d i and d med , then calculate d i and d med The cloud server sends the cosine similarity to the leader, and the leader discards the gradients with similarity below the threshold as poisoned gradients.

[0025] According to the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning of the present invention, further, performing synchronous aggregation to obtain an encrypted aggregate gradient within a cluster includes:

[0026] Each base station maintains a cluster list, in which cluster labels and memberships are stored in the form of tuples (c, b). The probability that a node belongs to cluster c is represented by the membership b. When calculating the encrypted aggregate gradient of cluster k, first, the weight of the local gradient of each base station in the cluster is the membership of the base station to cluster k divided by the sum of the memberships of all base stations in the cluster to cluster k. Then, the leader node weightedly aggregates the encrypted gradients of all base stations in cluster k with the weights obtained above to obtain the encrypted aggregate gradient corresponding to cluster k.

[0027] According to the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning of the present invention, further, the cloud server performs weighted aggregation on all intra-cluster aggregated gradients by availability weights to obtain global model parameters for this round of training, including:

[0028] The cloud server selects the clustered aggregate gradient with the lowest cosine similarity with the global gradient in the previous round as the comparison benchmark for evaluating poisoned gradients, and then calculates the cosine similarity between each clustered aggregate gradient and the benchmark, and calculates the availability weight; the cloud server weightedly aggregates all clustered aggregate gradients with the availability weight to obtain the global model parameters for this round of training, and sends them to all base stations. The base stations use the global model parameters to update the local model and enter the next round of training.

[0029] Furthermore, the present invention also proposes a cross-domain intelligent monitoring system based on Byzantine robust privacy-preserving federated learning, which is used to implement the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning as described above, including:

[0030] Key generation center: generates asymmetric key pairs for the cloud server and each base station;

[0031] Camera: collects monitoring data in real time and uploads it to nearby base stations; the camera receives the intelligent monitoring model sent by the base station;

[0032] Base station: responsible for collecting monitoring data from cameras within the coverage area, and working with other base stations to train intelligent monitoring models under the coordination of the cloud server.

[0033] Cloud server: assists base stations in identifying poisoned gradients, is responsible for receiving legitimate gradients submitted by base stations, and performs the final global aggregation.

[0034] The beneficial effects achieved by adopting the above technical solution are:

[0035] 1. The present invention designs a clustering federated learning architecture to effectively solve the problem that federated learning is difficult to converge normally under non-independent and identically distributed data.

[0036] The present invention abstracts the similarity between base station data distribution in federated learning into a network structure, where the node is the base station participating in federated learning, which is used to collect camera monitoring data within the coverage area; the edge is the similar link between base stations, and the hidden community structure between nodes is the cluster structure in clustered federated learning. The present invention introduces the classic overlapping community detection algorithm COPRA, which can discover the potential clustering structure between base stations and group base stations with similar data distribution into one category, thereby effectively solving the problem of difficult convergence of federated learning and facilitating the detection of poisoning attacks.

[0037] 2. The present invention designs a robust aggregation strategy based on linear homomorphic encryption, which can effectively resist poisoning attacks and privacy attacks and ensure data security and confidentiality.

[0038] To enhance the security and stability of the intelligent monitoring system, a trusted key generation center generates different key pairs for the cloud server and base station to ensure the security of communication. Each base station performs training locally to obtain gradients and converts the gradients into ciphertext using the Paillier homomorphic encryption algorithm. After receiving the encrypted gradients, the leader cooperates with the cloud server to execute the secure median algorithm, secure cosine similarity algorithm, and intra-cluster weighted aggregation algorithm to detect possible malicious gradients and complete secure aggregation within the cluster. This process can effectively resist poisoning attacks and ensure data privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings of the embodiments of the present invention, wherein the drawings are only used to illustrate some embodiments of the present invention, but not to limit all embodiments of the present invention thereto.

[0040] Figure 1 It is one of the flow charts of the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning according to an embodiment of the present invention;

[0041] Figure 2 This is the second flow chart of the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning according to an embodiment of the present invention;

[0042] Figure 3 It is a weighted undirected graph of the embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings of specific embodiments of the present invention to clearly and completely describe the exemplary scheme of the embodiment of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should be the common meanings understood by people with ordinary skills in the field.

[0044] like Figure 1 and Figure 2 As shown, this embodiment discloses a cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning, which includes five stages: initialization stage, clustering stage, intra-cluster security training stage, global robust aggregation stage, and real-time monitoring stage. Among them, only the intra-cluster security training stage and the global robust aggregation stage need to be repeatedly executed in each round of training until a trained intelligent monitoring model is obtained.

[0045] Step S101: Initialization phase.

[0046] During the initialization phase, a trusted key generation center generates a linear homomorphic encrypted asymmetric key pair (pk s ,sk s ), and for each base station U i Generate an asymmetric key pair (pk i ,sk i ). The cloud server generates encrypted initialization global model parameters [[W (0) ]] and sent to all base stations, which decrypt and perform several rounds of training based on local monitoring data.

[0047] Step S102: clustering stage.

[0048] In the clustering stage, the cloud server divides the base stations with non-independent and identically distributed data into multiple clusters with similar data distribution through the overlapping community detection algorithm COPRA, and selects the leader node in each cluster.

[0049] Through observation, it is found that the cluster federated learning architecture is similar to the community structure in complex networks. Complex networks are usually composed of various community structures. Nodes in the same community are closely connected, and nodes in different communities are sparsely connected. If the similarity of data distribution between base stations participating in federated learning is abstracted as a network structure, then the nodes are the base stations participating in federated learning, the edges are the similar links between base stations, and the hidden community structure between nodes is the cluster structure in cluster federated learning. This scheme introduces the classic overlapping community detection algorithm COPRA, which can discover the potential clustering structure between base stations and realize dynamic and flexible clustering division without pre-defining the number of clusters.

[0050] Based on these ideas, this scheme proposes a similarity-based soft clustering federated learning algorithm. U is the set of U base stations, L is the set of l tags, then in the case of Non-IID data distribution, the tags owned by each base station are a subset of the set L. Define base station k∈S U The label vector is X k =(x 1 ,x 2 ,…,x l)∈{0,1} l , where x i =1 means that there is label i in the local data set of base station k. In order to effectively calculate the similarity of data distribution between base stations and meet the real-time requirements, we focus on whether there is a certain type of label in the local monitoring data set of the base station, and then obtain the label vector of the base station. The similarity between base stations is measured by measuring the similarity of the label vector. First, each base station uploads its own label vector X to the server. k The server calculates the similarity between each two base stations and obtains the similarity matrix Since the label vector is a binary vector, the Jaccard algorithm is selected to calculate the similarity between vectors, because the Jaccard algorithm only considers elements with a value of 1 when calculating similarity, and does not consider elements with a value of 0. The formula for calculating similarity using the Jaccard algorithm is: J(A,B) = |A∩B| / |A∪B|, where A and B represent two vectors, |A∩B| represents the number of matching labels in vectors A and B, and |A∪B| represents the total number of labels with a value of 1 in the two label distribution vectors. Therefore, the similarity between the two base stations can be calculated as:

[0051] The cloud server constructs a weighted undirected graph based on the base station similarity adjacency matrix. The nodes in the graph represent base stations, and every two nodes have a weighted edge. The weight is the similarity of the data distribution of the two base stations. Then the above weighted undirected graph is used as input, and the COPRA algorithm is used to discover the hidden community structure in the graph, which is the clustering structure between base stations.

[0052] However, in practice, when the similarity is below a certain threshold, the probability that two nodes belong to the same community is very low. When the COPRA algorithm updates the node label, it must traverse the label list of all neighboring nodes. Therefore, in order to reduce the computational overhead of label propagation, some edges with low similarity are deleted to improve the efficiency of community detection. For example, suppose that there are 10 base stations participating in this round of federated learning training. The edge weight is visualized by the length of the edge: the shorter the edge, the higher the similarity between the data distributions of different base stations. The weighted undirected graph before deletion is as follows: Figure 3 (a) shows that there is a similar link between every two base station nodes. The weighted undirected graph after deletion is as follows Figure 3 As shown in (b), only some similar links with large similarity are retained. Figure 3There are three obvious cluster structures in both (a) and 3(b), so deleting links with similarity below a certain threshold will not have much impact on the final community detection results. And because of the reduction of similar links between nodes, the community structure becomes clearer. In practical application scenarios, deleting redundant similar links can not only reduce the computational overhead of constructing weighted undirected graphs, but also reduce the computational overhead of cluster label propagation while ensuring the accuracy of cluster detection results, thereby improving the efficiency of cluster label propagation.

[0053] Through the above clustering process, the base stations are divided into different clusters, and the base stations in each cluster have similar data distribution, thereby resisting the impact of non-independent and identically distributed data. At the same time, the base station must upload its resource parameters V (such as bandwidth and computing power) to the cloud server, and the cloud server selects the base station with the strongest computing and communication capabilities from each cluster k as the leader node LN k , responsible for the next stage of intra-cluster aggregation and poisoning attack detection.

[0054] Step S103: intra-cluster security training phase.

[0055] In each round of training, each base station U i Gradients obtained by training based on local monitoring data After Paillier encryption, it is uploaded to the leader node, which identifies and filters poisoned gradients, and then performs synchronous aggregation to obtain encrypted aggregated gradients within the cluster. This stage specifically includes the following three steps:

[0056] Step ①: Design the Smed function to calculate the median value of the gradient without leaking any privacy. The pseudo code of the Smed function is as follows. Specifically, given a set of gradients [[G k ]]=g 1 ,g 2 ,...,g n , first the leader node LN k By using the gradient [[g i ]] adds a random number [[r]] to mask it, and [[g i ]]·[[r]] is sent to the cloud server; then after the cloud server receives the ciphertext sent by all the leader nodes, it uses its own private key to decrypt it, calculates the median value, and encrypts it and sends it back to all the leader nodes. The leader node subtracts the ciphertext of the random number r from the ciphertext, and finally obtains the ciphertext of the gradient median value.

[0057]

[0058]

[0059] Step ②: Design the SCos function to calculate the cosine similarity of encrypted gradients under the premise of privacy protection. The pseudo code of the SCos function is as follows. Specifically, for each encrypted gradient in a cluster Leader node selection random number noise r i∈[1,n] , as The power of D is calculated i , and r i∈[1,n] Ciphertext as the median value of the gradient The power of D is calculated med ; Then D i and D med At the same time, it is sent to the cloud server. The cloud server uses the private key to decrypt and obtain d i and d med , then calculate d i and d med The cosine similarity of , according to the property of additive homomorphic encryption, when calculating the cosine similarity, the random number noise r i∈[1,n] is eliminated, so the result is equal to the intra-cluster gradient With median The cloud server sends the cosine similarity to the leader, which discards the gradients with similarity below the threshold as poisoned gradients.

[0060]

[0061]

[0062] Step ③, the above COPRA algorithm can obtain the clustering structure between base stations and the labels of each class. However, some base stations may belong to multiple classes, and the membership degree to each class is different. Therefore, each base station maintains a cluster list, and the cluster label and membership degree are stored in the cluster list in the form of tuples (c, b). The probability that a node belongs to cluster c is represented by the membership degree b. The membership degree indicates the preference of the base station for different clusters. From the perspective of cluster federated learning, base stations with different membership degrees have different contributions to the update of the model within the cluster. The larger the membership degree, the greater the contribution of the base station weight to the update of the model within the cluster. Therefore, this scheme adopts a weighted average strategy for intra-cluster aggregation, and the weight is related to the membership degree of the base station to the cluster to which it belongs. For example, in a round of training, the cluster list of the base station was updated. At this time, cluster k contains base stations 1, 2, and 3, and their cluster label lists are respectively When calculating the encrypted aggregate gradient of cluster k, the weights of the local gradients of base stations 1, 2, and 3 are calculated as:

[0063]

[0064] Then, the leader node LN kBased on the additive homomorphism of the Paillier algorithm, the encrypted gradients of the three base stations are weighted and aggregated according to the weights obtained above to obtain the encrypted aggregate gradient corresponding to cluster k:

[0065]

[0066] Step S104: global robust aggregation stage.

[0067] Considering more complex attack scenarios, the attacker may destroy the leader node to interfere with the training process in the cluster, or there are more than 50% malicious nodes in the system. In this case, the contribution of each cluster aggregate gradient to the global model will be weakened. For this reason, this scheme designs a global robust aggregation algorithm. The basic idea is: first, the cloud server decrypts the encrypted aggregate gradient in the cluster, and the cloud server selects the same global gradient g as the last round. w The clustered gradient with the lowest cosine similarity (note that this gradient is the decrypted gradient) is used as the comparison benchmark g for evaluating the poisoned gradient 0 , and then calculate the aggregate gradient g within each cluster k The cosine similarity with this benchmark is calculated and the availability weights are set as follows:

[0068]

[0069] It can be seen that the closer the cluster aggregate gradient is to the poisoning benchmark, the lower its availability. Finally, the cloud server weights and aggregates all cluster aggregate gradients with the availability weight to obtain the global model parameters for this round of training (as shown below), and sends them to all leader nodes. The leader node sends the global model parameters to all base stations in the cluster, and the base stations use the global model parameters to update the local model, thus entering the next round of training.

[0070]

[0071] w t ←w t-1 -ηg w

[0072] Step S105: real-time monitoring stage.

[0073] After the above-mentioned federated learning of the initialization phase, clustering phase, intra-cluster security training phase and global robust aggregation phase, each base station finally obtains a converged intelligent monitoring model, and then sends the model to all intelligent monitoring cameras in the jurisdiction. The camera can use the model for real-time intelligent monitoring to achieve tasks such as intelligent security and pedestrian recognition.

[0074] In order to realize the above-mentioned cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning, this embodiment also discloses a cross-domain intelligent monitoring system based on Byzantine robust privacy-preserving federated learning, including several cameras and base stations, a key generation center and a cloud server.

[0075] Key Generation Center: An independent and trusted entity that generates and distributes public and private key pairs for cloud servers and base stations. The keys are used to encrypt the local gradients of the base stations, thus preventing any sensitive information of the local data from being leaked.

[0076] Camera: collects monitoring data in real time and uploads it to nearby base stations; the camera has AI capabilities and can receive intelligent monitoring models sent by base stations to achieve tasks such as intelligent security and pedestrian recognition.

[0077] Base station: Responsible for collecting surveillance data from cameras within the coverage area, and coordinating with other base stations to train intelligent surveillance models under the coordination of the cloud server. Base stations can be benign or malicious, and malicious base stations will submit encrypted toxic gradients to destroy the global model.

[0078] Cloud server: assists the base station in identifying poisoned gradients, is responsible for receiving legitimate gradients submitted by the base station, and performs the final global aggregation. The cloud server is considered semi-honest, that is, it will honestly perform all operations according to the protocol while striving to obtain more sensitive information about the monitoring data.

[0079] In summary, the present invention has excellent defense accuracy. The present invention can defend against poisoning attacks in both IID data and Non-IID data environments, and ensure that the global accuracy of the intelligent monitoring model is within a reasonable range. The present invention also has strong privacy protection. The present invention can ensure the confidentiality of gradient parameters and will not be stolen by any attacker. Monitoring data privacy.

[0080] Unless otherwise specifically stated, the components, steps, numerical expressions and values ​​set forth in these embodiments do not limit the scope of the present invention.

[0081] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0082] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.

[0083] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.

[0084] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning, characterized in that: It includes the following five stages: Initialization phase: The key generation center generates an asymmetric key pair for the cloud server and each base station. The cloud server generates encrypted initialization global model parameters and sends them to all base stations. Clustering stage: The cloud server divides the base stations with non-independent and identically distributed data into multiple clusters with similar data distribution through the overlapping community detection algorithm COPRA, and selects the leader node in each cluster; In-cluster security training phase: In each round of training, each base station obtains gradients based on local monitoring data training, and uploads the encrypted gradients to the leader node of the cluster. The leader node identifies and filters poisoned gradients, and then performs synchronous aggregation to obtain encrypted aggregated gradients within the cluster. Global robust aggregation stage: The cloud server first decrypts the encrypted aggregate gradients in the cluster, and then performs weighted aggregation on all aggregate gradients in the cluster using the availability weights to obtain the global model parameters for this round of training. Real-time monitoring stage: When model training is completed, each base station obtains a converged intelligent monitoring model and sends the model to all cameras within the coverage area for real-time intelligent monitoring.

2. According to claim 1, the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning is characterized in that: In the initialization phase, the key generation center generates a linearly homomorphically encrypted asymmetric key pair; the base station decrypts the encrypted initialization global model parameters sent by the cloud server and performs several rounds of training based on local monitoring data.

3. According to claim 1, the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning is characterized in that: In the clustering stage, the cloud server first constructs a weighted undirected graph based on the base station similarity adjacency matrix. The nodes in the graph represent base stations, and every two nodes have a weighted edge, and the weight is the similarity of the data distribution of the two base stations. Then the above weighted undirected graph is used as input, and the COPRA algorithm is used to discover the hidden community structure in the graph. Finally, the base stations are divided into different clusters, and the base stations in each cluster have similar data distribution.

4. According to claim 3, the cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning is characterized in that: The method for calculating the similarity between base stations is: each base station uploads its own label vector X to the cloud server k =(x1,x2,…,x l )∈{0,1} l , where x i =1 means that there is label i in the local data set of base station k; the cloud server calculates the similarity of two base stations according to the Jaccard algorithm.

5. The cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning according to claim 1 is characterized in that: Selecting the leader node in each cluster includes: each base station uploads its resource parameters to the cloud server, and the cloud server selects the base station with the strongest computing and communication capabilities from each cluster as the leader node.

6. The cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning according to claim 1, characterized in that: Poisoned gradients identified and filtered by the leader node include: A set of gradients for a given cluster K First, the leader node passes through the gradient Add a random number [[r]] to mask it and Send it to the cloud server; then after the cloud server receives the ciphertext sent by all the leader nodes, it uses its own private key to decrypt it, calculates the median value, and encrypts it and sends it back to all the leader nodes. The leader node subtracts [[r]] from the ciphertext and finally obtains the ciphertext of the gradient median value; For each encrypted gradient in the cluster, the leader node selects a random number noise as the power exponent of the encrypted gradient and calculates D i At the same time, the above random number noise is used as the power exponent of the ciphertext of the gradient median value to obtain D med ; Then D i and D med At the same time, it is sent to the cloud server; the cloud server uses the private key to decrypt and obtain d i and d med , then calculate d i and d med The cloud server sends the cosine similarity to the leader, and the leader discards the gradients with similarity below the threshold as poisoned gradients.

7. The cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning according to claim 1, characterized in that: Performing synchronous aggregation to obtain the encrypted aggregate gradient within the cluster includes: Each base station maintains a cluster list, in which cluster labels and memberships are stored in the form of tuples (c, b). The probability that a node belongs to cluster c is represented by the membership b. When calculating the encrypted aggregate gradient of cluster k, first, the weight of the local gradient of each base station in the cluster is the membership of the base station to cluster k divided by the sum of the memberships of all base stations in the cluster to cluster k. Then, the leader node weightedly aggregates the encrypted gradients of all base stations in cluster k with the weights obtained above to obtain the encrypted aggregate gradient corresponding to cluster k.

8. The cross-domain intelligent monitoring method based on Byzantine robust privacy-preserving federated learning according to claim 1, characterized in that: The cloud server performs weighted aggregation of all cluster aggregate gradients using the availability weights to obtain the global model parameters for this round of training, including: The cloud server selects the clustered aggregate gradient with the lowest cosine similarity with the global gradient in the previous round as the comparison benchmark for evaluating poisoned gradients, and then calculates the cosine similarity between each clustered aggregate gradient and the benchmark, and calculates the availability weight; the cloud server weightedly aggregates all clustered aggregate gradients with the availability weight to obtain the global model parameters for this round of training, and sends them to all base stations. The base stations use the global model parameters to update the local model and enter the next round of training.

9. A cross-domain intelligent monitoring system based on Byzantine robust privacy-preserving federated learning, characterized in that: A cross-domain intelligent monitoring method for implementing Byzantine robust privacy-preserving federated learning as described in any one of claims 1 to 8, comprising: Key generation center: generates asymmetric key pairs for the cloud server and each base station; Camera: collects monitoring data in real time and uploads it to nearby base stations; the camera receives the intelligent monitoring model sent by the base station; Base station: responsible for collecting monitoring data from cameras within the coverage area, and working with other base stations to train intelligent monitoring models under the coordination of the cloud server. Cloud server: assists base stations in identifying poisoned gradients, is responsible for receiving legitimate gradients submitted by base stations, and performs the final global aggregation.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images