An object detection method based on horizontal federated learning framework

Abstract

The present invention discloses a target detection method based on a horizontal federated learning framework, including constructing a horizontal federated learning framework, the framework including a client detection model, a central service network and a data set; allocating an independent data set to each client detection model, and training the model through the data set; optimizing the gradient descent algorithm used in the client detection model training process to accelerate the convergence speed of the model; extracting the parameters of the trained model, and transmitting these parameters to the central service network through a differential privacy strategy; training the central service network, and returning the model parameters after parameter aggregation to each client detection model, and the client detection model then updates the model according to the returned parameters; and using the updated model for detection. The present invention adopts a target detection method based on a horizontal federated learning framework, which can solve the problems of low accuracy and efficiency of existing target detection methods and the lack of data privacy protection.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a target detection method based on a horizontal federated learning framework. Background Art

[0002] With the advent of the information society, computer vision technology has gradually changed the way people live. Common application scenarios include electronic payment, medical image detection, and the Sky Eye system. A large amount of complex visual information data provides a rich sample for the computer's processing and analysis capabilities, but also brings severe challenges. Object detection is one of the important topics in the field of computer vision. Its task is to classify the objects of interest in the image and mark the corresponding positions. In recent years, the development of deep learning has provided a strong driving force for the field of computer vision. It has the characteristics of efficient learning and wide coverage. Therefore, people gradually apply deep learning to object detection tasks and achieve good results.

[0003] In traditional target detection algorithms, candidate box positioning, feature extraction, and classifier judgment are the research focuses. The detection steps can be summarized as: first, the target of the image is positioned by the sliding window method, then the feature extraction algorithm is used to extract features, and finally the classifier is used to make a judgment. Common classifiers include support vector machines, Adaboost, etc. Although the detection process of the traditional method is simple and easy to understand, there are two obvious problems to be solved: 1) The region selection module does not adopt a targeted strategy, resulting in the generation of repeated invalid windows and too long region selection time. 2) The features extracted by the feature extraction module are artificially designed. Such features have poor robustness performance and directly affect the classification accuracy.

[0004] The emergence of deep learning has brought about a qualitative improvement in the field of object detection. The object detection algorithm based on deep learning combines the advantages of classifiers and convolutional neural networks, which can enhance the detection model's ability to extract features and efficiently classify. Faster R-CNN is currently a widely used object detection model. It can obtain candidate boxes more efficiently and accurately through a region generation network. In addition, a feature map pyramid network can be embedded in the model to solve the multi-scale problem in object detection. Without basically increasing the computational complexity of the original model, the detection effect is greatly improved by changing the network connection method.

[0005] In real life, target detection is widely used in many fields such as regional monitoring and traffic management. In these applications, data exchange and collaboration between multiple models are often required to form an enhanced regional joint detection network. For example, in traffic sign detection, each detection point will collect pictures for local model training, but the detection effect cannot be guaranteed to be ideal due to the small amount of collected picture data. In addition, during the target detection process, the privacy of the data cannot be protected. Summary of the invention

[0006] The purpose of the present invention is to provide a target detection method based on a horizontal federated learning framework to solve the problems of low accuracy and efficiency of existing target detection methods and the lack of data privacy protection.

[0007] To achieve the above object, the present invention provides a target detection method based on a horizontal federated learning framework, comprising the following steps:

[0008] S1. Build a horizontal federated learning framework, which includes a client detection model, a central service network, and a dataset.

[0009] S2. Allocate an independent homologous dataset to each client detection model and train the model using the dataset;

[0010] S3. Use the gradient descent algorithm to optimize the client detection model training process to speed up the convergence of the model;

[0011] S4, extract the parameters of the trained client detection model and transmit these parameters to the central service network through the differential privacy strategy;

[0012] S5. Train the central service network, and return the client detection model parameters after parameter aggregation to each client detection model. The client detection model then updates the client detection model according to the returned parameters, and uses the updated client detection model for detection.

[0013] Preferably, in step S1, Faster R-CNN is selected as the basic client detection model, and the FPN network is embedded into the model to generate feature maps of different scales.

[0014] Preferably, in step S3, the Adam algorithm is used for optimization during the training process. The Adam algorithm uses the historical gradient mean m of the parameter t and the mean value of the squared sum of historical gradients of the parameters v t Adaptively adjust the learning rate and keep learning information at all times; the calculation formulas for the two means are as follows:

[0015] m t =β1 m t-1 +(1-β 1 ) t

[0016] v t =β 2 v t-1 +(1-β 2 ) t 2

[0017] Among them, β 1 , β 2 represents the attenuation factor, g t is time t t The gradient value of

[0018] Take β 1 =0.9,β 2 =0.999,ε=10 -8 Modify the above formula;

[0019]

[0020]

[0021]

[0022] Among them, α is the learning rate.

[0023] Preferably, in step S4, the data is homomorphically encrypted using a differential privacy strategy to prevent gradient information leakage, and [[x]] is used to represent the encryption operation on x. The homomorphic encryption formula is as follows:

[0024]

[0025] Among them, u and v represent any two elements in the plaintext space, and their encryption results are [[u]] and [[v]], represents addition or multiplication operation, Dec sk Represents a decryption operation.

[0026] Preferably, in step S5, parameter aggregation is implemented based on the average influence algorithm, and a weight is assigned to each client detection model according to its influence; the number of data sets is m, the number of client detection models is n, and the influence factor matrix X and the output Y are respectively expressed as:

[0027]

[0028] Y=[y 1 y 2 …y m ] T

[0029] First, fine-tune the i-th influencing factor by δ, where δ means increasing or decreasing the value by 10%. The two new sets of inputs and outputs are expressed as and

[0030]

[0031]

[0032] Secondly, calculate the MIV value of the client, which represents the relative influence of the current client. The calculation formula is as follows:

[0033]

[0034] Finally, calculate the contribution C corresponding to each client detection model i and the average impact value C,

[0035]

[0036]

[0037] The advantages and positive effects of the target detection method based on the horizontal federated learning framework described in the present invention are:

[0038] 1. The present invention applies the horizontal federated learning framework to the target detection task, achieves the purpose of regional joint detection, and updates parameters by sharing data between regional models. Compared with the traditional centralized training method, it can reach model convergence faster.

[0039] 2. The present invention utilizes the advantage of cross-layer connection of FPN network, and can achieve better detection effect on multi-scale images. FPN network includes three connection modes: bottom-up connection relies on forward propagation, and the size of feature map is usually reduced after convolution operation in this process, so the global information contained in deep features is richer. Top-down connection will upsample the deep features, thereby expanding the size of feature map. Lateral connection combines the feature maps obtained by the first two at the same size level, which can enhance both high-resolution information and high-semantic information in the feature map.

[0040] 3. The present invention introduces an attention mechanism during parameter aggregation. The attention mechanism is implemented by using the average influence (MIV) weighted algorithm. The algorithm is used to evaluate the influence of each regional model on the joint detection result, and improves the aggregation effect by strengthening local models with large influence and weakening local models with small influence.

[0041] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flow chart of an embodiment of a target detection method based on a horizontal federated learning framework of the present invention;

[0043] Figure 2 This is a FPN network structure diagram of an embodiment of a target detection method based on a horizontal federated learning framework of the present invention;

[0044] Figure 3 It is a model of a target detection method embodiment based on a horizontal federated learning framework of the present invention, in which Faster R-CNN and FPN network are combined;

[0045] Figure 4 A federated learning framework of an embodiment of a target detection method based on a horizontal federated learning framework of the present invention;

[0046] Figure 5 This is a comparison chart of the accuracy of horizontal federated learning training and centralized training in an embodiment of a target detection method based on a horizontal federated learning framework of the present invention;

[0047] Figure 6 This is a comparison chart of loss values ​​between horizontal federated learning training and centralized training in an embodiment of a target detection method based on a horizontal federated learning framework of the present invention. DETAILED DESCRIPTION

[0048] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0049] Example

[0050] Figure 1 This is a flow chart of an embodiment of a target detection method based on a horizontal federated learning framework of the present invention. Figure 2 This is a FPN network structure diagram of an embodiment of a target detection method based on a horizontal federated learning framework of the present invention. Figure 3 This is a model of a target detection method embodiment based on a horizontal federated learning framework in the present invention, which is a combination of Faster R-CNN and FPN network. Figure 4 The present invention is a federated learning framework of an object detection method embodiment based on a horizontal federated learning framework. As shown in the figure, a method for object detection based on a horizontal federated learning framework includes the following steps:

[0051] S1. Build a horizontal federated learning framework, which mainly includes a client detection model, a central service network, and a dataset. The dataset uses the PASCAL VOC public dataset, which is suitable for tasks such as target detection and image segmentation, and contains a total of twenty categories. Select Faster R-CNN as the client detection model, and embed the FPN network into the model to generate feature maps of different scales. By fusing these feature maps, the location information of the shallow network and the semantic information of the deep network are fully combined, alleviating the multi-scale problem in the target detection task.

[0052] S2. Allocate an independent homologous dataset to each client detection model and train the model using the dataset.

[0053] S3. Use the gradient descent algorithm to optimize the client detection model training process to speed up the convergence of the model. The Adam algorithm is used for optimization during the training process. Adam is a first-order optimization algorithm that can replace the traditional stochastic gradient descent process. It can iteratively update the weights of the neural network based on the training data. The basic mechanism of the Adam optimization algorithm is different from the traditional stochastic gradient descent algorithm. The latter maintains a single learning rate to update all weights, which does not change during the process; while the former independently designs different adaptive learning rates for different parameters by calculating the first-order moment estimate and the second-order moment estimate of the gradient.

[0054] The Adam algorithm not only realizes adaptive adjustment of the learning rate, but also keeps learning information at all times. This is achieved by using the historical gradient mean m of the parameter. t and the mean value of the squared sum of historical gradients of the parameters v t achieved.

[0055] m t =β 1 m t-1 +(1-β 1 ) t

[0056] v t =β 2 v t-1 +(1-β 2 ) t 2

[0057] Among them, β 1 , β 2 represents the attenuation factor, g t is time t t The gradient value of . When the model is initialized, when β 1 and β 2 When the values ​​are all 1, the decay rate is very small and the learning rate is 0. To solve this problem, we need tot and v t The following deviation correction is performed, usually taking β 1 =0.9,β 2 =0.999,ε=10 -8 The effect is better.

[0058]

[0059]

[0060] At this time, the parameter update equation is:

[0061]

[0062] Among them, α is the learning rate.

[0063] S4. Extract the parameters of the trained client detection model and transmit these parameters to the central service network through the differential privacy strategy. In order to ensure the security of the data, the central server will introduce Gaussian noise to perform differential privacy protection on important information when updating the global model parameters. Differential privacy provides a guarantee for information theoretical security. It uses a random mechanism so that when a sample in the input sample changes, the output distribution will not change significantly. The outside world cannot judge which party provided the input data through the output data distribution. Therefore, it is a reliable security mechanism. The data is homomorphically encrypted through the differential privacy strategy to prevent gradient information leakage. [[x]] is used to represent the encryption operation of x. The homomorphic encryption formula is as follows:

[0064]

[0065] Among them, u and v represent any two elements in the plaintext space, and their encryption results are [[u]] and [[v]], represents addition or multiplication operation, Dec sk Represents a decryption operation.

[0066] S5. Train the central service network, and return the client detection model parameters after parameter aggregation to each client detection model. The client detection model then updates the client detection model according to the returned parameters, and uses the updated client detection model for detection.

[0067] Parameter aggregation is achieved based on the average influence algorithm, and weights are assigned to each client detection model according to its influence. The number of data sets is m, the number of client detection models is n, and the influencing factor matrix X and output Y are expressed as:

[0068]

[0069] Y=[y 1 y 2 …y m ] T

[0070] First, fine-tune the i-th influencing factor by δ, where δ means increasing or decreasing the value by 10%. The two new sets of inputs and outputs are expressed as and

[0071]

[0072]

[0073] Secondly, calculate the MIV value of the client detection model, which represents the relative influence of the current client detection model. The calculation formula is as follows:

[0074]

[0075] Finally, calculate the contribution C corresponding to each client detection model i and the average impact value C,

[0076]

[0077]

[0078] The central server training steps are as follows:

[0079] 1) Set the communication round t to 0 and start federated learning training.

[0080] 2) Use the server’s MA (moment account) mechanism to calculate the privacy loss δ of the current round. The calculation formula is:

[0081] δ←MA(σ t ,ε,K)

[0082] Among them, σ t represents the Gaussian noise of the tth round, ε represents the privacy tolerance value, K represents the number of client detection models, and if δ>Q, the current round of training is stopped and returned Otherwise proceed to the next step, where Q refers to the loss threshold.

[0083] 3) Set the global model parameters of this round and adaptive training period τ t Broadcast to K client detection models. τ t The calculation formula is:

[0084]

[0085] Among them, τt represents the local training period of the tth round of communication, τ 0 represents the initial local training cycle, η 0 , η t They represent the initial learning rate and the learning rate of the tth round of communication, l represents the objective function of the model, represents the global model parameters of the tth round of communication.

[0086] 4) The central service network receives all client detection model parameters to obtain parameter update values The client detects the median M of the model parameter update norm and calculates the Gaussian noise parameter S for this round.

[0087]

[0088]

[0089] 5) Calculate the mean X(i,j) and variance Y(i,j) of the (i,j)th parameter in the updated parameters of the client detection model in this round. 2 , and the variance σ of all updated parameters of the client detection model 2 .

[0090]

[0091]

[0092]

[0093] Among them, the Gaussian noise formula is:

[0094]

[0095] 6) Calculate the global model parameters and update the communication round t=t+1. If t is less than the total number of communication rounds T, jump to step 2, otherwise return to the current value. The global model parameter calculation formula is:

[0096]

[0097] Figure 5 This is a comparison chart of the accuracy of horizontal federated learning training and centralized training in an embodiment of a target detection method based on a horizontal federated learning framework of the present invention. Figure 6This is a comparison chart of the loss values ​​of horizontal federated learning training and centralized training in an embodiment of a target detection method based on a horizontal federated learning framework of the present invention. As shown in the figure, the updated model is finally used for the experiment. The experiment includes a total of 10 clients, and 5 of them are randomly selected to participate in the training in each round. Each client detection model needs to be iterated for 3 times, and the number of global communication rounds is set to 25 times. Centralized training is equivalent to only one client participating in the training in each round. In order to obtain a high-performance configuration, the number of local iterations is also set to 1, 2, and 3 times for comparison. According to the attached manual Figure 5 and attached Figure 6 It can be seen that the target detection method using the horizontal federated learning framework described in the present invention has higher accuracy and lower loss value.

[0098] Therefore, the present invention adopts a target detection method based on a horizontal federated learning framework, which can solve the problems of low accuracy and efficiency of existing target detection methods and the lack of data privacy protection.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A target detection method based on a horizontal federated learning framework, characterized in that: The following steps are involved: S1. Build a horizontal federated learning framework, which includes a client detection model, a central service network, and a dataset. S2. Allocate an independent homologous dataset to each client detection model and train the model using the dataset; S3. Use the gradient descent algorithm to optimize the client detection model training process to speed up the convergence of the model; S4, extract the parameters of the trained client detection model and transmit these parameters to the central service network through the differential privacy strategy; S5. Train the central service network, obtain the client detection model parameters after parameter aggregation, and return them to each client detection model. The client detection model then updates the client detection model according to the returned parameters, and uses the updated client detection model for detection. In step S5, parameter aggregation is implemented based on the average influence algorithm, and a weight is assigned to each client detection model according to its influence; The number of datasets is , the number of client detection models is , influencing factor matrix With output Respectively expressed as: First, for Fine-tune the influencing factors , the two new sets of inputs and outputs are expressed as and ; ； Secondly, calculate the MIV value of the client, which represents the relative influence of the current client. The calculation formula is as follows: Finally, calculate the contribution of each client detection model and the average impact value , ； The above-mentioned object detection method based on the horizontal federated learning framework is used for image processing.

2. The target detection method based on the horizontal federated learning framework according to claim 1 is characterized in that: In step S1, Faster R-CNN is selected as the basic client detection model, and the FPN network is embedded into the model to generate feature maps of different scales.

3. The target detection method based on the horizontal federated learning framework according to claim 1 is characterized in that: In step S3, the Adam algorithm is used for optimization during the training process. The Adam algorithm uses the historical gradient mean of the parameters. and the mean of the squared sum of historical gradients of the parameters Adaptively adjust the learning rate and keep learning information at all times; the calculation formulas for the two means are as follows: in, represents the attenuation factor, for time The gradient value of Pick , , Modify the above formula; in, is the learning rate.

4. The target detection method based on the horizontal federated learning framework according to claim 1 is characterized in that: In step S4, the data is homomorphically encrypted using a differential privacy strategy to prevent gradient information leakage. Express Perform encryption operation, the homomorphic encryption formula is as follows: in, and Represents any two elements in the plaintext space, and the encryption result is , represents addition or multiplication operations, Represents a decryption operation.

Images