Low-altitude unmanned aerial vehicle real-time traffic monitoring method and system based on federated transfer learning and YOLO-SG

Abstract

The invention relates to the technical field of unmanned aerial vehicle remote sensing, in particular to a low-altitude unmanned aerial vehicle real-time traffic monitoring method and system based on federated transfer learning and YOLO-SG. Communication transmission is performed between a central server and an unmanned aerial vehicle client by applying a federated transfer learning technology. A YOLO-SG local model in the federated learning process is constructed by taking YOLOv8 as a basic network, all key discrimination feature information of the image is reserved to the maximum extent by utilizing an SPD convolution block from space to depth in the YOLO-SG model, and global context information in the image is captured through a global context network attention mechanism GCNet. According to the method, the communication cost between the server and the client can be reduced, the risk of data leakage is prevented, the model detection performance and generalization ability are improved, and the method can be suitable for diversified ground traffic real-time monitoring application scenes with small target size, non-uniform distribution, complex positioning, easy shielding and the like.

Claims

1. A low-altitude UAV real-time traffic monitoring method based on federated transfer learning and YOLO-SG, characterized in that: Include: The central server pre-trains the global model using parallel data sets, and distributes the pre-trained model parameters to each drone client after encryption using a scrambled matrix. The parallel data set is a remote sensing data set consisting of a public data set and a synthetic data set. The synthetic data set is remote sensing image data obtained by pre-processing the public data set using data enhancement methods. The global model is a YOLO-SG model built on the YOLOv8 basic network. In the YOLO-SG model, the space-to-depth SPD convolution block is used to maximize the retention of all key discriminant feature information of the image, and the global context information in the image is captured through the global context network attention mechanism GCNet. Each drone client uses the received model parameters as the initial parameters of the local model, and locally trains the local model using the local data set. The locally trained model parameters are encrypted and uploaded to the central server. The central server decrypts the received model parameters, aggregates them, and uses the aggregated model parameters to update the global model. The updated global model parameters are encrypted with a scrambled matrix and sent to each drone client for the next round of federated learning iteration until the federated learning iteration termination condition is met. The global model parameters of the most recent iteration round are used as the target model parameters and encrypted with a scrambled matrix and distributed to each drone client. The local model adopts a model structure consistent with the global model. Each drone client updates and deploys the local model after decrypting the received target model parameters, so as to use the updated and deployed local model to monitor the traffic status in the remote sensing image of the drone area in real time.

2. According to claim 1, the low-altitude UAV real-time traffic monitoring method based on federated transfer learning and YOLO-SG is characterized in that: The data enhancement means include but are not limited to: image flipping, image scaling and image cropping.

3. The low-altitude UAV real-time traffic monitoring method based on federated transfer learning and YOLO-SG according to claim 1 is characterized in that: The YOLO-SG model includes: an input end for preprocessing the input image, a backbone network for extracting features from the preprocessed image, a head network for fusing the extracted image features, and a prediction output end for classifying and predicting the fused output image features. Both the backbone network and the head network use space-to-depth SPD convolution blocks to maximize the retention of all key discriminative feature information of the image, and the global context information in the image is captured through the global context network attention mechanism GCNet in the head network.

4. The low-altitude UAV real-time traffic monitoring method based on federated transfer learning and YOLO-SG according to claim 3 is characterized in that: The backbone network includes: a plurality of convolutional layers consisting of space-to-depth SPD convolutional blocks and Conv convolutional blocks, a plurality of C2F layers stacked with the convolutional layers and used for feature conversion, and a fast spatial pyramid pooling layer SPPF connected to the terminal C2F layer for capturing supplementary scale image features.

5. The low-altitude UAV real-time traffic monitoring method based on federated transfer learning and YOLO-SG according to claim 4 is characterized in that: The head network includes: two C2F components that are stacked and used for feature conversion fusion, each C2F component includes an upsampling layer, a splicing layer and a feature conversion C2F layer, a global context network attention mechanism GCNet is set at the output end of each C2F component, a convolution layer is connected to the output end of the second global context network attention mechanism GCNet, the output end of the convolution layer and an output end of the feature conversion C2F layer in the first C2F component are sent to a C2F layer in the head network through a splicing layer, the C2F layer in the head network is connected to a convolution layer and sent to another C2F component of the head network through a splicing layer connected to the convolution layer and a fast spatial pyramid pooling layer SPPF, and the outputs of the feature conversion C2F layer in the second C2F component, the C2F layer in the head network and the another C2F component are sent to the prediction output end.

6. The low-altitude UAV real-time traffic monitoring method based on federated transfer learning and YOLO-SG according to claim 1 is characterized in that: Local training of local models using local datasets also includes: Collect remote sensing datasets from the drone’s perspective, label the target categories and bounding boxes in the remote sensing datasets, and use the labeled remote sensing datasets as local datasets to train local models locally.

7. The low-altitude UAV real-time traffic monitoring method based on federated transfer learning and YOLO-SG according to claim 1 is characterized in that: The encrypted transmission process of model parameters between the central server and the drone client includes: Generate a network information data sequence using the pre-trained diffusion model and GRU, and use the network information data sequence to construct a scrambled matrix represented by a two-dimensional array, wherein each element value in the scrambled matrix represents a mapping relationship between the original position of the model parameter and the encrypted position of the model parameter; The model parameters to be transmitted are encrypted and transmitted using a scrambled matrix; The inverse matrix of the scrambled matrix is ​​used to restore the original information of the model parameters to be transmitted.

8. A low-altitude UAV real-time traffic monitoring system based on federated transfer learning and YOLO-SG, characterized in that: Contains: federated learning module and real-time detection module, among which, In the federated learning module, the central server pre-trains the global model using parallel data sets, and distributes the pre-trained model parameters to each drone client after encryption using a scrambled matrix. The parallel data set is a remote sensing data set consisting of a public data set and a synthetic data set. The synthetic data set is remote sensing image data obtained by pre-processing the public data set using data enhancement methods. The global model is a YOLO-SG model built on the YOLOv8 basic network. In the YOLO-SG model, the space-to-depth SPD convolution block is used to maximize the retention of all key discriminant feature information of the image, and the global context information in the image is captured through the global context network attention mechanism GCNet. Each drone client uses the received model parameters as the initial parameters of the local model, and locally trains the local model using the local data set. The locally trained model parameters are encrypted and uploaded to the central server. The central server decrypts the received model parameters, aggregates them, and uses the aggregated model parameters to update the global model. The updated global model parameters are encrypted with a scrambled matrix and sent to each drone client for the next round of federated learning iteration until the federated learning iteration termination condition is met. The global model parameters of the most recent iteration round are used as the target model parameters and encrypted with a scrambled matrix and distributed to each drone client. The local model adopts a model structure consistent with the global model. In the real-time detection module, each drone client updates and deploys the local model according to the received target model parameters, so as to use the updated and deployed local model to monitor the traffic status in the remote sensing image of the drone area in real time.

9. An electronic device, characterized in that: include: at least one processor, and a memory coupled to the at least one processor; The memory stores a computer program, and the computer program can be executed by the at least one processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 can be implemented.

Images