Power transmission line damper defect detection system and method based on federated learning

Abstract

The embodiment of the invention provides a power transmission line damper defect detection system and method based on federated learning, and relates to the technical field of target detection, the detection system comprises a student model and a teacher model which are pre-constructed through a client based on an improved YOLOv10 model, and model updating is realized by using index moving average EMA; under a federated learning (FL) framework, introducing a fully homomorphic encryption (FHE) technology, and after training of a local model of each client is completed, encrypting model parameters of each client by using the fully homomorphic encryption technology and uploading the encrypted model parameters to a server side; according to the embodiment of the invention, model training is independently carried out at each client, accurate detection of the defects of the vibration damper of the power transmission line is realized by using data resources distributed at different places under the condition of not sharing data, and the defects of the vibration damper of the power transmission line are accurately detected according to a federated learning theory method. A fully homomorphic encryption algorithm is introduced to encrypt model parameters transmitted in a model training process, so that the privacy security of data is effectively protected.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a system and method for detecting anti-vibration hammer defects in power transmission lines based on federated learning. Background Art

[0002] With the continuous development of modern power systems, the safe operation of transmission lines is becoming increasingly important. As a key component in the transmission system, the health status of the transmission line anti-vibration hammer directly affects the stability and service life of the transmission line. Due to long-term exposure to complex outdoor environments, the anti-vibration hammer is easily affected by various factors such as corrosion and mechanical damage, which in turn causes failures. Therefore, it is of great significance to perform efficient and accurate defect detection on the anti-vibration hammer. However, traditional anti-vibration hammer defect detection methods mostly rely on manual inspections, which is not only time-consuming and labor-intensive, but also easily restricted by natural conditions such as weather and terrain, resulting in low detection efficiency and difficulty in ensuring comprehensive coverage.

[0003] In recent years, deep learning technology has made significant progress in the fields of image processing and target detection, providing new ideas for the detection of anti-vibration hammer defects in power transmission lines. By using sensors such as high-definition cameras on unmanned inspection terminals, image data of transmission lines and anti-vibration hammers can be collected efficiently and accurately. By training deep learning models, automatic recognition and classification of defects in anti-vibration hammer images can be achieved, greatly improving the accuracy and efficiency of detection.

[0004] However, the training of deep learning models requires a large amount of labeled data. The image data of transmission line anti-vibration hammers are often distributed in different power companies and regions, forming data islands that are difficult to use directly for model training. The centralized model training method requires the data to be centralized to a central server, which may bring the risk of privacy leakage. In transmission line detection, the monitoring data of various regions often contains information such as geographic location and video or image data collected by high-definition camera equipment. This information not only shows the layout of the transmission line and the status of the equipment, but may also involve the privacy of surrounding residents.

[0005] The method for detecting defects of anti-vibration hammers on power transmission lines based on federated learning mainly needs to consider two aspects:

[0006] (1) How to introduce and improve the YOLOv10 (You Only Look Once Version 10) model, train and build a vibration hammer defect detection model when there are few labeled samples, and realize efficient and accurate detection of vibration hammer defects in transmission lines.

[0007] (2) How to simultaneously introduce Federated Learning (FL) and Fully Homomorphic Encryption (FHE) and apply them to transmission line anti-vibration hammer defect detection to ensure data privacy and security. Summary of the invention

[0008] The purpose of the embodiments of the present application is to provide a system and method for detecting defects of anti-vibration hammers for power transmission lines based on federated learning, so as to solve the problem of detecting defects of anti-vibration hammers for power transmission lines.

[0009] To achieve the above-mentioned purpose, on the one hand, an embodiment of the present application provides a transmission line anti-vibration hammer defect detection system based on federated learning, including multiple clients and a server, the client including: a data preprocessing module, a target detection training module and a first privacy protection module, the server including: a second privacy protection module;

[0010] The data preprocessing module is used to obtain the transmission line inspection images, preprocess the obtained anti-vibration hammer images, and construct a data set;

[0011] The target detection training module is used to update the model using the exponential moving average EMA through the student model and teacher model pre-built by this client based on the improved YOLOv10 model;

[0012] The first privacy protection module is used to introduce the fully homomorphic encryption (FHE) technology under the federated learning (FL) framework. After the local model training of each client is completed, the model parameters of each client are encrypted using the fully homomorphic encryption technology and uploaded to the server.

[0013] The second privacy protection module is used by the server to obtain the parameters of the global model through aggregate calculation, and send them to each client for decryption to update its own local model.

[0014] According to an embodiment of the present application, the data preprocessing module is used to determine the effective anti-vibration hammer image according to the obtained anti-vibration hammer image and the anti-vibration hammer image quality; it is also used to perform data expansion according to the obtained anti-vibration hammer image and the anti-vibration hammer image diversity; it is also used to label the data collected by each client according to a preset ratio through the LabelImg labeling tool, and divide the data set into a labeled data set D l And the unlabeled dataset D u .

[0015] According to one embodiment of the present application, the target detection training module is used to use the improved YOLOv10 model to establish two YOLOv10 network models and initialize the model parameters, which are divided into a student model and a teacher model; it is also used to convert the original input image I into a multi-layer feature map T through a backbone network; it is also used to convert the multi-layer feature map T into a feature map set T' after multi-scale feature fusion and enhancement; it is also used to input the feature map set T' into the CBAM attention mechanism to weight the feature map T' and output F, and perform the final prediction through the decoding prediction end.

[0016] According to one embodiment of the present application, the first privacy protection module is used in each iterative training round; it is also used to input unlabeled data and labeled data into the student model, and the unlabeled data into the teacher model; it is also used to calculate the supervised learning loss of labeled data and the consistency loss of unlabeled data respectively; it is also used to calculate the total loss function by combining the supervised learning loss of labeled data and the consistency loss of unlabeled data; the parameters of the student model are updated by minimizing the total loss function L; it is also used to update the parameters of the teacher model by the exponential moving average of the student model parameters, and when a preset iterative round or preset condition is reached, the training of the current iterative round is completed.

[0017] According to one embodiment of the present application, the first privacy protection module is also used for each client to select a key length and generate a key pair through a key generation function; the key pair includes: a public key and a private key; the public key is used to encrypt model parameters, and the private key is used for decryption; it is also used for each client to obtain model parameters after the model training is completed, and to generate corresponding encrypted model parameters together with the public key.

[0018] According to an embodiment of the present application, the second privacy protection module is used by the server to aggregate the encryption model parameters of each client; and is also used by the server to send the aggregated encrypted global parameters to each client.

[0019] According to one embodiment of the present application, the first privacy protection module is also used for each client to decrypt the ciphertext global parameters through a key pair and update the local model; it is also used for each client to determine whether the global error meets the preset conditions; the global parameters include the global error; if the global error meets the preset conditions, the global optimal model is output; if the global error does not meet the preset conditions, it returns to the step of inputting the unlabeled data and the labeled data into the student model, and the unlabeled data into the teacher model.

[0020] On the other hand, an embodiment of the present application provides a method for detecting anti-vibration hammer defects in a power transmission line based on federated learning, which is applied to the anti-vibration hammer defect detection system for a power transmission line based on federated learning as described in any one of the aforementioned embodiments, including an anti-vibration hammer defect detection sub-method applied to a client and a fully homomorphic encryption sub-method applied to a server;

[0021] The anti-vibration hammer defect detection sub-methods applied to the client include:

[0022] Acquire the inspection images of the transmission lines, pre-process the obtained anti-vibration hammer images, and construct a data set;

[0023] Through the pre-built student model and teacher model based on the improved YOLOv10 model of this client, the exponential moving average EMA is used to update the model;

[0024] Under the federated learning FL framework, fully homomorphic encryption FHE technology is introduced. After the local model training of each client is completed, the model parameters of each client are encrypted using fully homomorphic encryption technology and uploaded to the server.

[0025] The fully homomorphic encryption sub-method applied to the server includes:

[0026] The server obtains the parameters of the global model through aggregate calculation and sends them to each client for decryption to update its own local model.

[0027] According to an embodiment of the present application, obtaining a transmission line inspection image, preprocessing the obtained anti-vibration hammer image, and constructing a data set include: data screening, data expansion, and constructing a data set;

[0028] The data screening includes: determining a valid anti-vibration hammer image according to the obtained anti-vibration hammer image and the anti-vibration hammer image quality;

[0029] The data expansion includes: performing data expansion according to the obtained anti-vibration hammer image and the diversity of the anti-vibration hammer image;

[0030] The data set construction includes: labeling the data collected by each client according to a preset ratio using the LabelImg labeling tool, and dividing the data set into a labeled data set D l And the unlabeled dataset D u .

[0031] According to one embodiment of the present application, the student model and the teacher model pre-built by the client based on the improved YOLOv10 model, and the exponential moving average EMA are used to update the model. The student model and the teacher model pre-built by the client based on the improved YOLOv10 model, and the exponential moving average EMA are used to update the model, which specifically includes:

[0032] Using the improved YOLOv10 model, two YOLOv10 network models are established and the model parameters are initialized, which are divided into a student model and a teacher model;

[0033] The original input image I is converted into a multi-layer feature map T through the backbone network;

[0034] The multi-layer feature map T is transformed into a feature map set T' after multi-scale feature fusion and enhancement;

[0035] The feature map set T' is input into the CBAM attention mechanism to weight the feature map T' and output F, and the final prediction is made through the decoding prediction end.

[0036] According to an embodiment of the present application, the fully homomorphic encryption FHE technology is introduced under the federated learning FL framework. After the local model training of each client is completed, the model parameters of each client are encrypted using the fully homomorphic encryption technology and uploaded to the server. Specifically, it includes: in each iterative training round;

[0037] Input unlabeled data and labeled data into the student model, and unlabeled data into the teacher model;

[0038] Calculate the supervised learning loss for labeled data and the consistency loss for unlabeled data respectively;

[0039] The total loss function is calculated by combining the supervised learning loss of labeled data and the consistency loss of unlabeled data;

[0040] Update the parameters of the student model by minimizing the total loss function L;

[0041] The parameters of the teacher model are updated through the exponential moving average of the student model parameters. When the preset iteration round or preset condition is reached, the training of the current iteration round is completed.

[0042] According to an embodiment of the present application, the fully homomorphic encryption FHE technology is introduced under the federated learning FL framework. After the local model training of each client is completed, the model parameters of each client are encrypted using the fully homomorphic encryption technology and uploaded to the server, which also includes:

[0043] Each client selects a key length and generates a key pair through a key generation function; the key pair includes: a public key and a private key; the public key is used to encrypt model parameters, and the private key is used to decrypt;

[0044] After the training of each client model is completed, the model parameters are obtained, and the corresponding encrypted model parameters are generated together with the public key.

[0045] According to an embodiment of the present application, the server obtains the parameters of the global model through aggregate calculation and sends them to each client for decryption to update its own local model, including:

[0046] The server aggregates the encryption model parameters of each client;

[0047] The server sends the aggregated encrypted global parameters to each client.

[0048] According to an embodiment of the present application, the server obtains the parameters of the global model through aggregate calculation and sends them to each client for decryption to update its own local model, and then further includes:

[0049] Each client decrypts the ciphertext global parameters using the key pair and updates the local model;

[0050] Each client determines whether the global error meets the preset conditions; the global parameters include the global error;

[0051] If the global error meets the preset conditions, the global optimal model is output; if the global error does not meet the preset conditions, the process returns to the step of inputting unlabeled data and labeled data into the student model, and unlabeled data into the teacher model.

[0052] The beneficial effects of the embodiments of the present application include, for example:

[0053] The transmission line anti-vibration hammer defect detection system and method based on federated learning provided in the embodiment of the present application adopts the YOLOv10 network model, and independently trains the model on each client. Without sharing data, it makes full use of data resources distributed in different locations, and realizes accurate detection of transmission line anti-vibration hammer defects. In addition, according to the theoretical method of federated learning, the model parameters updated during model training are encrypted by a fully homomorphic encryption algorithm to ensure that the model parameters cannot be easily decrypted by attackers even if they are intercepted during transmission, thereby achieving optimization of the model while protecting the privacy and security of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only used for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the embodiments of the present application. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0055] Figure 1 It is a block diagram of a transmission line anti-vibration hammer defect detection system based on federated learning provided in an embodiment of the present application;

[0056] Figure 2It is a flow chart of a method for detecting defects of a power transmission line anti-vibration hammer based on federated learning provided in an embodiment of the present application;

[0057] Figure 3 is another flow chart of the method provided in the embodiment of the present application;

[0058] Figure 4 is another flow chart of the method provided in the embodiment of the present application;

[0059] Figure 5 is another flow chart of the method provided in the embodiment of the present application;

[0060] Figure 6 is another flow chart of the method provided in the embodiment of the present application;

[0061] Figure 7 is another flow chart of the method provided in the embodiment of the present application;

[0062] Figure 8 is another flow chart of the method provided in the embodiment of the present application;

[0063] Fig. 9 It is another flowchart of the method provided in an embodiment of the present application.

[0064] Icon: 110 - data preprocessing module; 120 - target detection training module; 130 - first privacy protection module; 140 - second privacy protection module. DETAILED DESCRIPTION

[0065] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0066] like Figure 1 As shown, an embodiment of the present application provides a transmission line anti-vibration hammer defect detection system based on federated learning, including multiple clients and server ends, the client end includes: a data preprocessing module 110, a target detection training module 120 and a first privacy protection module 130, and the server end includes: a second privacy protection module 140.

[0067] The data preprocessing module is used to obtain the inspection images of the transmission lines, preprocess the obtained anti-vibration hammer images, and construct a data set; the target detection training module is used to update the model by using the exponential moving average EMA through the student model and teacher model pre-built by this client based on the improved YOLOv10 model; the first privacy protection module is used to introduce the fully homomorphic encryption FHE technology under the federated learning FL framework. After the local model training of each client is completed, the model parameters of each client are encrypted using the fully homomorphic encryption technology and uploaded to the server; the second privacy protection module is used by the server to obtain the parameters of the global model through aggregate calculation, and send them to each client for decryption to update its own local model.

[0068] The embodiment of the present application adopts the YOLOv10 network model and independently trains the model on each client. Without sharing data, it utilizes data resources distributed in different locations to achieve accurate detection of transmission line anti-vibration hammer defects. Based on the theoretical method of federated learning, a fully homomorphic encryption algorithm is introduced to encrypt the model parameters transmitted during the model training process, thereby effectively protecting the privacy and security of the data.

[0069] The specific implementation plan is:

[0070] (1) The data preprocessing module is used to determine the effective anti-vibration hammer image based on the obtained anti-vibration hammer image and the anti-vibration hammer image quality; it is also used to expand the data based on the obtained anti-vibration hammer image and the anti-vibration hammer image diversity; it is also used to use the LabelImg annotation tool to annotate the data collected by each client according to a preset ratio, and divide the data set into annotated data sets and unannotated data sets. Each client selects high-quality images and removes those blurred, noisy or irrelevant images. The data set is expanded by using data enhancement techniques such as rotation and scaling to improve the generalization ability of the model. For the construction of annotated data sets, each client uses the LabelImg tool to annotate the anti-vibration hammer position in the image.

[0071] (2) The target detection training module is used to use the improved YOLOv10 model to establish two YOLOv10 network models and initialize the model parameters, which are divided into a student model and a teacher model; it is also used to convert the original input image into a multi-layer feature map through the backbone network; it is also used to convert the multi-layer feature map into a feature map set after multi-scale feature fusion and enhancement; it is also used to input the feature map set into the CBAM attention mechanism to weight the feature map output, and the final prediction is made through the decoding prediction end. After the data preprocessing is completed, the three clients train the YOLOv10 network model on their own data sets respectively. Since each client only trains the model locally, the data is always saved on their respective devices and no data exchange is performed. After multiple rounds of training and optimization, the model learns the characteristics of the anti-vibration hammer defect. Through the above process, each client can improve and optimize the detection capability of the model while protecting its own data privacy.

[0072] (3) To further improve the performance of the model, each client shares the parameters of the model under the federated learning framework. The first privacy protection module is used in each iterative training round; it is also used to input unlabeled data and labeled data into the student model, and unlabeled data into the teacher model; it is also used to calculate the supervised learning loss of labeled data and the consistency loss of unlabeled data respectively; it is also used to calculate the total loss function by combining the supervised learning loss of labeled data and the consistency loss of unlabeled data; it is used to update the parameters of the student model by minimizing the total loss function; it is also used to update the parameters of the teacher model by the exponential moving average of the student model parameters, and when the preset iterative rounds or preset conditions are reached, the training of the current iterative round is completed. After completing the local training, each client uses the fully homomorphic encryption technology to encrypt the trained model parameters. The encrypted model parameters are sent to the central server, and the server aggregates and calculates these encrypted parameters to form a global model. Since the data is always encrypted throughout the process, the privacy of the data is fully protected. Finally, the central server sends the aggregated global model parameters back to each client, and each client decrypts them and uses them to update its own local model. This method can protect the privacy of the data while optimizing the model.

[0073] 1. Architecture

[0074] Figure 1The structural diagram of the transmission line anti-vibration hammer defect detection system based on federated learning is given, which mainly includes three parts: data preprocessing module, target detection training module and privacy protection module. The data preprocessing module in the figure is mainly used to perform preprocessing operations such as data screening, partial data annotation, and data set construction on the image data collected by the original unmanned inspection terminal. This process aims to provide a data basis for the training of subsequent models; the target detection training module uses the improved YOLOv10 model to initialize two models with the same network structure on each client, and updates and iterates the model through exponential moving average to solve the problem of a large amount of unlabeled data in actual scenarios. At the same time, each client performs separate training, which realizes the full utilization of data resources scattered in various places without sharing data; the privacy protection module encrypts the model parameters of each client after training and uploads them to the server based on the fully homomorphic encryption theory method, obtains the parameters of the global model through aggregate calculation, and sends them to each client for decryption, so as to update the local model without leaking individual data.

[0075] The following is a detailed introduction:

[0076] Data preprocessing module: The core task of this module is to preprocess the image data collected by the original unmanned inspection terminal and construct a data set to provide a data basis for the subsequent training of the transmission line anti-vibration hammer defect detection model. First, the data is screened to filter out data that is conducive to the learning of the deep learning model. Secondly, the data set is expanded to prevent problems such as poor model detection effect and overfitting due to too few data sets or uneven sample distribution during training. Finally, the labeling tool LabelImg is used to label some data and divide the data set into labeled data sets and unlabeled data sets.

[0077] Target detection training module: This module uses the improved YOLOv10 model. Each client independently trains the local model without directly sharing local data, effectively protecting data privacy and security. At the same time, considering the problem of under-annotated data, two network models are constructed during the training process, and the exponential moving average is used for update and iteration, so as to make full use of data resources scattered in various places.

[0078] Privacy protection module: This module introduces fully homomorphic encryption technology under the federated learning framework. After each client completes local training, the encrypted model parameters are uploaded to the server. These encrypted data are aggregated and calculated on the server to obtain the optimization parameters of the global model. Finally, the global model parameters are sent back to each client for decryption and update of the local model, thereby achieving collaborative optimization and performance improvement of the model while protecting data privacy.

[0079] Equation Section 2

[0080] 1. Data preprocessing module

[0081] This module aims to provide a data basis for the construction of the transmission line anti-vibration hammer defect detection model. It uses the image data collected by the unmanned inspection terminal during actual operation, and performs data preprocessing operations to ensure that the subsequent transmission line anti-vibration hammer defect detection model can be trained on a comprehensive and diverse data set, thereby improving the accuracy of model recognition. The specific process is as follows:

[0082] (1) Data screening

[0083] Since the transmission lines are located in a complex outdoor environment, the quality of the anti-vibration hammer image data collected by the unmanned inspection terminal is uncertain. In order to adapt to the subsequent YOLOv10 model training, it is necessary to screen the image data obtained during the actual operation of the unmanned inspection terminal to eliminate data with low clarity and incomplete shooting content.

[0084] (2) Data expansion

[0085] In the field of deep learning, the quality of model training depends on the size and diversity of the data set. When faced with the specific task of anti-vibration hammer defect detection, due to the scarcity and uneven distribution of data set samples, direct model training will lead to overfitting problems, resulting in poor generalization ability and practical application effect of the model. This step aims to enhance the robustness and adaptability of the model by expanding the data set. The specific method is as follows:

[0086] 1) Image flip: When the unmanned inspection terminal actually shoots the transmission line, the shooting angle is not fixed. Therefore, in addition to the traditional horizontal image flip, random angle image flip can also be introduced, such as mirroring along any axis. This can not only increase the number of samples, but also enable the model to learn defect features at different angles.

[0087] 2) Image rotation: When implementing rotation enhancement, we are not limited to fixed angles, but use random rotation angles to ensure that the rotated image remains natural and diverse.

[0088] 3) Noise addition: When adding noise, different types of noise can be introduced (such as Gaussian noise, salt and pepper noise, Poisson noise, etc.), and the intensity and distribution of the noise can be dynamically adjusted according to the different defect characteristics to simulate various interference conditions that may be encountered in the actual detection environment.

[0089] Through the above method, the overfitting problem caused by the small number of data set samples and uneven distribution can be effectively solved, and the accuracy and robustness of the anti-vibration hammer defect detection model can be improved.

[0090] (3) Constructing a dataset

[0091] The data collected by each client is partially labeled using the LabelImg labeling tool, and the data set is divided into labeled data set D l And the unlabeled dataset D u ,This process aims to provide a data basis for the subsequent introduction of the average teacher model.

[0092] 2. Object detection training module

[0093] As an efficient real-time target detection framework, the YOLO algorithm can realize the rapid recognition and positioning of multiple targets in an image. At the same time, as a single-stage detection algorithm, the core idea of ​​the YOLO series is to convert the detection task into a regression problem. The YOLOv10 used in the embodiment of the present application inherits and optimizes the network structure of the previous version, which reduces the processing delay and improves the detection accuracy. This module introduces the YOLOv10 model, uses local data, considers the problem of insufficient sample annotation in practice, constructs two models for collaborative training, and trains the transmission line anti-vibration hammer defect detection model through exponential moving average. The specific process is as follows:

[0094] (1) Model initialization

[0095] Each client builds two YOLOv10 network models and initializes the model parameters. The student model parameters are W s , the teacher model parameter is W t Initially:

[0096]

[0097] In the formula, represents the initial parameters of the teacher model, represents the initial parameters of the student model.

[0098] (2) Extracting image features

[0099] The YOLOv10 network model converts the original input image I into a multi-layer feature map T through the backbone network for use in subsequent object detection tasks. T represents a series of features extracted from the image through the backbone network.

[0100] (3) Multi-scale feature fusion

[0101] The extracted series of features T are further processed through the neck layer to enhance the feature expression capability so as to provide the final prediction for the decoding prediction end.

[0102] T'←Neck(T)(2)

[0103] Where Neck(T) represents the neck layer, and T' represents the set of feature maps after multi-scale feature fusion and enhancement.

[0104] (4) Attention Mechanism

[0105] In order to further improve the effectiveness of feature expression, the CBAM (Convolutional Block Attention Module) attention mechanism is introduced in the output part of Neck.

[0106] Channel Attention Module (CAM): used to enhance important channel features and suppress irrelevant channels.

[0107] M c (T')=σ(MLP(Ap(T'))+MLP(Mp(T'))) (3)

[0108] Where M c (F) represents channel attention weight, MLP represents multi-layer perceptron, σ represents Sigmoid function, Ap represents average pooling, and Mp represents maximum pooling.

[0109] Spatial Attention Module (SAM): used to highlight important spatial locations in the image.

[0110] M s (T')=σ(Conv(Concat(Ap(T'),Mp(T')))) (4)

[0111] Where M s (T') represents the spatial attention weight, Conv represents the convolution operation, and Concat represents the concatenation operation.

[0112] Finally, the feature map T' is weighted and output.

[0113] F=M c (T')×M s (T')×T' (5)

[0114] Where F represents the feature map after being processed by the attention mechanism.

[0115] (5) Prediction output

[0116] The decoding prediction end is responsible for receiving the feature maps output by the neck layer and making the final prediction O on these feature maps. This includes the location of the predicted bounding box, the confidence level, and the probability of belonging to each category.

[0117] (6) Student model parameter update

[0118] The student model inputs both labeled data and unlabeled data, where the supervised learning loss L for labeled data is s By the classification loss L cls and regression loss L reg composition.

[0119]

[0120] In the formula, the supervised learning loss with labeled data is expressed as L s , N l Indicates the number of labeled data. represents the output of the student model with labeled data, is the corresponding label.

[0121] Unlabeled data is used to calculate the prediction difference between the student model and the teacher model, and this difference is reduced to improve the performance of the model on unlabeled data.

[0122]

[0123] Where, L u represents the consistency loss of unlabeled data, N u represents the number of unlabeled data, L cls_u represents the classification loss of unlabeled data, L reg_u represents the regression loss of unlabeled data, represents the output of the student model for unlabeled data, Represents the output of the teacher model on unlabeled data.

[0124] The total loss function L combines the supervised learning loss for labeled data and the consistency loss for unlabeled data.

[0125] L=L s +λL u (8)

[0126] Where λ represents the weight coefficient, which is used to balance the supervised learning loss and consistency loss.

[0127] Through back propagation, the parameters of the student model are updated by minimizing the total loss function L.

[0128]

[0129] In the formula, η represents the learning rate.

[0130] (7) Teacher model parameter update

[0131] The parameters of the teacher model are updated by taking the exponential moving average of the parameters of the student model.

[0132] W t ←αW t +(1-α)W s (10)

[0133] In the formula, α represents the smoothing coefficient.

[0134] (8) Training ends

[0135] Iterate continuously until the model converges or reaches the predetermined number of training rounds. After training is completed, the teacher model is used for reasoning and practical application.

[0136] 3. Privacy protection module

[0137] This module introduces federated learning technology, which aims to protect data privacy while making full use of data resources distributed on various clients. Each client can retain its original data without central uploading, thus effectively avoiding the risk of data leakage. Each client independently trains the YOLOv10 network model on the local dataset. After the training is completed, only the model parameters are transmitted to the central server to reduce the privacy risks brought by centralized data processing.

[0138] To further enhance the security during data transmission, this module introduces fully homomorphic encryption technology to ensure the security of model parameters even during transmission.

[0139] On the server side, the homomorphic properties of fully homomorphic encryption are used to aggregate the encrypted model parameters from different clients to generate global model parameters. Finally, the server distributes the aggregated global model parameters to each client in encrypted form. Each client decrypts the parameters locally and updates the local model. The specific process is as follows:

[0140] (1) Generate a key pair

[0141] Select the key length to generate a key pair (pk, sk), including the public key pk and the private key sk. The public key is used to encrypt the model parameters, and the private key is used for decryption. This step provides the basis for encrypted communication between the client and the server, ensuring security.

[0142] (pk,sk)←key(bits) (11)

[0143] In the formula, key(bits) represents the key generation function, bits is the key length, the public key is represented by pk, and the private key is represented by sk.

[0144] (2) Encryption parameters

[0145] The encryption process encrypts the plaintext with the public key to generate ciphertext, thereby protecting the confidentiality of the data. After each client K completes the model training, the model parameters W K After encryption, we get W K ' and sent to the server, there is no need to worry about the data being stolen or leaked during transmission, thus achieving secure communication.

[0146] W K '=Enc(pk,W K )(12)

[0147] In the formula, Enc(pk,W K ) represents the encryption function, W K is the plain text parameter, W K ' is the ciphertext parameter.

[0148] (3) Model aggregation

[0149] The server receives the encryption parameters W of all clients K 'Afterwards, the encrypted model parameters are aggregated using the homomorphic properties of fully homomorphic encryption.

[0150]

[0151] Where n K is the size of the local dataset of client K, and n is the total size of all client datasets.

[0152]

[0153] Wherein, the ciphertext global parameter is represented as W'.

[0154] (4) Decryption parameters

[0155] The server sends the aggregated encrypted global parameters to each client. Each client uses its own private key to decrypt the received ciphertext global parameters to restore the original plaintext.

[0156] W=Dec(sk,W')(15)

[0157] Where Dec(sk,W') represents the decryption function, and the global parameter after decryption is represented as W.

[0158] (5) Iteration

[0159] Continue to train the model and update and upload it, and iterate until the model performance meets the actual application requirements. This step ensures that the model can continuously update and improve its own model, thereby improving its generalization ability and the accuracy of model detection.

[0160] The embodiment of the present application also provides a method for detecting anti-vibration hammer defects in power transmission lines based on federated learning. The method of the embodiment of the present application is a strategic method. First, the image data transmitted by the unmanned inspection terminal is preprocessed and divided into labeled data sets and unlabeled data sets to adapt to the subsequent training of the anti-vibration hammer defect detection model for power transmission lines; secondly, the client establishes a student model and a teacher model based on the improved YOLOv10 model, and uses the exponential moving average (EMA) to update the model; finally, after the local model training is completed, the parameters of each client are encrypted using fully homomorphic encryption technology and uploaded to the server, the server aggregates the encrypted parameters of each client and sends the aggregated model to each client, and continuously iterates to reduce the global error until the global error meets the set error value or reaches the maximum number of communications, and outputs the global optimal model and sends it to each client.

[0161] like Figures 2 to 8 As shown, an embodiment of the present application provides a method for detecting anti-vibration hammer defects in a power transmission line based on federated learning, including an anti-vibration hammer defect detection sub-method applied to a client and a fully homomorphic encryption sub-method applied to a server;

[0162] The anti-vibration hammer defect detection sub-methods applied to the client include:

[0163] Step S210: Obtain transmission line inspection images, pre-process the obtained anti-vibration hammer images, and construct a data set; specifically, assume that there are three clients in different locations, each maintaining the transmission lines in their respective areas and taking a large number of anti-vibration hammer images. Figure 2 As shown, the transmission line inspection image is obtained, the obtained anti-vibration hammer image is preprocessed, and the data set is constructed, including: data screening, data expansion and data set construction;

[0164] Step S310, the data screening includes: determining a valid anti-vibration hammer image according to the obtained anti-vibration hammer image and the anti-vibration hammer image quality;

[0165] Step S320, the data expansion includes: performing data expansion according to the obtained anti-vibration hammer image and the anti-vibration hammer image diversity;

[0166] Step S330: constructing a data set includes: labeling the data collected by each client according to a preset ratio using the LabelImg labeling tool, and dividing the data set into a labeled data set D l And the unlabeled dataset D u .

[0167] Step S220, using the client based on the improved YOLOv10 model and the pre-built student model and teacher model, using the exponential moving average EMA to implement model update; specifically, the client based on the improved YOLOv10 model and the pre-built student model and teacher model, using the exponential moving average EMA to implement model update, using the client based on the improved YOLOv10 model and the pre-built student model and teacher model, using the exponential moving average EMA to implement model update specifically includes:

[0168] Step S410: Using the improved YOLOv10 model, two YOLOv10 network models are established and the model parameters are initialized, which are divided into a student model and a teacher model;

[0169] Step S420, converting the original input image I into a multi-layer feature map T through the backbone network;

[0170] Step S430, converting the multi-layer feature map T into a feature map set T' after multi-scale feature fusion and enhancement;

[0171] Step S440: The feature map set T' is input into the CBAM attention mechanism to perform weighted output F on the feature map T', and the final prediction is performed through the decoding prediction end.

[0172] Step S230, under the framework of federated learning FL, introducing the fully homomorphic encryption FHE technology, after each client local model training is completed, the model parameters of each client are encrypted by using the fully homomorphic encryption technology and uploaded to the server side; the introduction of the fully homomorphic encryption FHE technology under the framework of federated learning FL, after each client local model training is completed, the model parameters of each client are encrypted by using the fully homomorphic encryption technology and uploaded to the server side specifically includes: in each iterative training round;

[0173] Step S510: input the unlabeled data and the labeled data into the student model, and input the unlabeled data into the teacher model;

[0174] Step S520, respectively calculating the supervised learning loss of the labeled data and the consistency loss of the unlabeled data;

[0175] Step S530, calculating the total loss function by combining the supervised learning loss of labeled data and the consistency loss of unlabeled data;

[0176] Step S540, updating the parameters of the student model by minimizing the total loss function;

[0177] Step S550, update the parameters of the teacher model through the exponential moving average of the student model parameters, and when the preset iteration round or preset condition is reached, complete the training of the current iteration round.

[0178] In the federated learning FL framework, the fully homomorphic encryption FHE technology is introduced. After the local model training of each client is completed, the model parameters of each client are encrypted using the fully homomorphic encryption technology and uploaded to the server, which also includes:

[0179] Step S710: Each client selects a key length and generates a key pair through a key generation function; the key pair includes: a public key and a private key; the public key is used to encrypt model parameters, and the private key is used to decrypt;

[0180] Step S720: After the training of each client model is completed, the model parameters are obtained, and the corresponding encrypted model parameters are generated together with the public key.

[0181] The fully homomorphic encryption sub-method applied to the server includes:

[0182] Step S240: The server obtains the parameters of the global model through aggregate calculation and sends them to each client for decryption to update its local model; the server obtains the parameters of the global model through aggregate calculation and sends them to each client for decryption to update its local model, including:

[0183] Step S710: The server aggregates the encryption model parameters of each client;

[0184] Step S720: The server sends the aggregated encrypted global parameters to each client.

[0185] The server obtains the parameters of the global model through aggregate calculation and sends them to each client for decryption to update its own local model, and then further includes:

[0186] Step S810: Each client decrypts the encrypted global parameters using the key pair and updates the local model;

[0187] Step S820: Each client determines whether the global error meets a preset condition; the global parameters include the global error;

[0188] Step S830: If the global error meets the preset conditions, output the global optimal model; if the global error does not meet the preset conditions, return to the step of inputting the unlabeled data and the labeled data into the student model, and the unlabeled data into the teacher model.

[0189] The embodiment of the present application proposes a method for detecting anti-vibration hammer defects in power transmission lines based on federated learning. Through federated learning technology, it is possible to share models between multiple distributed nodes without sharing original data, thereby ensuring that data privacy is effectively protected. At the same time, the detection performance of the model is further improved by improving the YOLOv10 network model. This method can not only solve data privacy and security issues, but also provide strong technical support for anti-vibration hammer defect detection in practical applications. Under the premise of protecting data privacy, data resources distributed in different locations are used to improve the performance of the model.

[0190] like Fig. 9 As shown, the embodiment of the present application provides a method for detecting defects of a power transmission line anti-vibration hammer based on federated learning, and the entire execution process mainly includes the following steps:

[0191] Step 1: Filter the image data obtained during the actual operation of the unmanned inspection terminal, and remove the data with low definition and incomplete shooting content. Go to step 2;

[0192] Step 2: Expand the obtained data by mirror flipping, image rotation, and noise addition. Go to step 3;

[0193] Step 3: Use LabelImg as a dataset annotation tool to annotate some data and create annotated and unannotated datasets. Go to step 4;

[0194] Step 4: Each client builds two YOLOv10 network models and initializes the model parameters, which are divided into student model and teacher model. Go to step 5;

[0195] Step 5: Convert the original input image I into a multi-layer feature map T through the backbone network. Go to step 6;

[0196] Step 6: Further processing is performed through the neck layer, T'←Neck(T) to achieve multi-scale feature fusion, T' is represented as a feature map set after multi-scale feature fusion and enhancement. Go to step 7;

[0197] Step 7: The feature map set T' is input into the CBAM attention mechanism to perform weighted output on the feature map T' to obtain the feature map F = M c (T')×M s (T') × T', where M c (F) represents the channel attention weight, M s (T') represents the spatial attention weight, and F represents the feature map after being processed by the attention mechanism. Go to step 8;

[0198] Step 8: The final prediction is made through the decoding prediction end. The model output includes the location of the bounding box, the confidence level, and the probability of belonging to each category, thereby realizing the detection of transmission line anti-vibration hammer defects. Go to step 9;

[0199] Step 9: Input the labeled data to the student model and calculate the supervised learning loss of the labeled data

[0200] Step 10: Input the unlabeled data into the student model and the teacher model and calculate the consistency loss of the unlabeled data Reduce this difference to improve the performance of the model on unlabeled data. u represents the number of unlabeled data, L cls_u represents the classification loss of unlabeled data, L reg_u represents the regression loss of unlabeled data, represents the output of the student model for unlabeled data, Represents the output of the teacher model without label data. Go to step 11;

[0201] Step 11: Combine the supervised learning loss of labeled data and the consistency loss of unlabeled data to get the total loss function L = L s +λL u . Go to step 12;

[0202] Step 12: Back propagation, update the parameters of the student model by minimizing the total loss function L Go to step 13;

[0203] Step 13: The parameters of the teacher model are updated by the exponential moving average of the student model parameters W t ←αW t +(1-α)W s . Go to step 14;

[0204] Step 14: Repeat steps 5 to 13 multiple times until the training error meets the set conditions or the number of iterations reaches the maximum value. Go to step 15;

[0205] Step 15: Select the key length bits, and generate the key pair (pk, sk) through the key generation function key(bits). The public key pk is used to encrypt the model parameters, and the private key sk is used for decryption. Go to step 16;

[0206] Step 16: After each client K completes model training, the model parameters W can be obtained K , and the public key pk together to generate the corresponding ciphertext W K '=Enc(pk,W K ), where W K is the plain text parameter, WK ' is the ciphertext parameter. Go to step 17;

[0207] Step 17: The server encrypts the model parameters W of each client K 'Perform aggregation Where n K is the size of the local data set of client K, and n is the total size of all client data sets. Go to step 18;

[0208] Step 18: The server sends the aggregated encrypted global parameters to each client. Each client decrypts the encrypted global parameters using the private key sk, restores the original plaintext data W = Dec (sk, W'), and updates the local model. Go to step 19;

[0209] Step 19: Repeat steps 5 to 13 multiple times until the global error is less than a given value or the maximum number of communications is reached. Go to step 20;

[0210] Step 20: Output the global optimal model and transmit it to each client.

[0211] In this embodiment, the implementation principle of the power transmission line anti-vibration hammer defect detection method based on federated learning can refer to the implementation principle of the aforementioned power transmission line anti-vibration hammer defect detection system based on federated learning, which will not be repeated here.

[0212] In summary, the embodiment of the present application provides a transmission line anti-vibration hammer defect detection system and method based on federated learning. Through the federated learning technology, the model is shared among multiple distributed nodes without sharing the original data, thereby ensuring that data privacy is effectively protected. At the same time, the detection performance of the model is further improved by improving the YOLOv10 network model.

[0213] It should be noted that the above embodiments illustrate rather than limit the embodiments of the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The embodiments of the present application may be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

[0214] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if these modifications and variations to the embodiments of the present application fall within the scope of the claims of the embodiments of the present application and their equivalents, the embodiments of the present application are also intended to include these modifications and variations.

Claims

1. A transmission line anti-vibration hammer defect detection system based on federated learning, characterized in that: It includes multiple clients and servers, the client includes: a data preprocessing module, a target detection training module and a first privacy protection module, and the server includes: a second privacy protection module; The data preprocessing module is used to obtain the transmission line inspection images, preprocess the obtained anti-vibration hammer images, and construct a data set; The target detection training module is used to update the model using the exponential moving average EMA through the student model and teacher model pre-built by this client based on the improved YOLOv10 model; The first privacy protection module is used to introduce the fully homomorphic encryption (FHE) technology under the federated learning (FL) framework. After the local model training of each client is completed, the model parameters of each client are encrypted using the fully homomorphic encryption technology and uploaded to the server. The second privacy protection module is used by the server to obtain the parameters of the global model through aggregate calculation, and send them to each client for decryption to update its own local model.

2. A transmission line anti-vibration hammer defect detection system based on federated learning as claimed in claim 1, characterized in that: The data preprocessing module is used to determine the effective anti-vibration hammer image according to the obtained anti-vibration hammer image and the anti-vibration hammer image quality; it is also used to perform data expansion according to the obtained anti-vibration hammer image and the anti-vibration hammer image diversity; it is also used to label the data collected by each client according to a preset ratio through the LabelImg labeling tool, and divide the data set into a labeled data set D l And the unlabeled dataset D u .

3. A transmission line anti-vibration hammer defect detection system based on federated learning as claimed in claim 1, characterized in that: The target detection training module is used to use the improved YOLOv10 model to establish two YOLOv10 network models and initialize the model parameters, which are divided into a student model and a teacher model; it is also used to convert the original input image I into a multi-layer feature map T through a backbone network; it is also used to convert the multi-layer feature map T into a feature map set T' after multi-scale feature fusion and enhancement; it is also used to input the feature map set T' into the CBAM attention mechanism to weight the feature map T' and output F, and make a final prediction through the decoding prediction end.

4. A transmission line anti-vibration hammer defect detection system based on federated learning as claimed in claim 1, characterized in that: The first privacy protection module is used in each iterative training round; it is also used to input unlabeled data and labeled data into the student model, and unlabeled data into the teacher model; it is also used to calculate the supervised learning loss of labeled data and the consistency loss of unlabeled data respectively; it is also used to calculate the total loss function by combining the supervised learning loss of labeled data and the consistency loss of unlabeled data; the parameters of the student model are updated by minimizing the total loss function L; it is also used to update the parameters of the teacher model by the exponential moving average of the student model parameters, and when the preset iterative round or preset condition is reached, the training of the current iterative round is completed.

5. A transmission line anti-vibration hammer defect detection system based on federated learning as claimed in claim 1, characterized in that: The first privacy protection module is also used for each client to select a key length and generate a key pair through a key generation function; the key pair includes: a public key and a private key; the public key is used to encrypt model parameters, and the private key is used for decryption; it is also used for each client to obtain model parameters after the model training is completed, and to generate corresponding encrypted model parameters together with the public key.

6. A transmission line anti-vibration hammer defect detection system based on federated learning as claimed in claim 1, characterized in that: The second privacy protection module is used by the server to aggregate the encryption model parameters of each client; and is also used by the server to send the aggregated encrypted global parameters to each client.

7. A transmission line anti-vibration hammer defect detection system based on federated learning as claimed in claim 1, characterized in that: The first privacy protection module is also used for each client to decrypt the encrypted global parameter through the key pair and update the local model; it is also used for each client to determine whether the global error meets the preset conditions; the global parameters include the global error; if the global error meets the preset conditions, the global optimal model is output; If the global error does not meet the preset conditions, return to the step of inputting unlabeled data and labeled data into the student model, and unlabeled data into the teacher model.

8. A method for detecting defects of anti-vibration hammers in power transmission lines based on federated learning, characterized in that: The transmission line anti-vibration hammer defect detection system based on federated learning as described in claim 1 comprises an anti-vibration hammer defect detection sub-method applied to the client and a fully homomorphic encryption sub-method applied to the server; The anti-vibration hammer defect detection sub-methods applied to the client include: Acquire the inspection images of the transmission lines, pre-process the obtained anti-vibration hammer images, and construct a data set; Through the pre-built student model and teacher model based on the improved YOLOv10 model of this client, the exponential moving average EMA is used to update the model; Under the federated learning FL framework, fully homomorphic encryption FHE technology is introduced. After the local model training of each client is completed, the model parameters of each client are encrypted using fully homomorphic encryption technology and uploaded to the server. The fully homomorphic encryption sub-method applied to the server includes: The server obtains the parameters of the global model through aggregate calculation and sends them to each client for decryption to update its own local model.

9. A method for detecting defects of transmission line anti-vibration hammers based on federated learning according to claim 1, characterized in that: Acquire the inspection images of the transmission line, pre-process the obtained anti-vibration hammer images, and construct the data set including: data screening, data expansion and data set construction; The data screening includes: determining a valid anti-vibration hammer image according to the obtained anti-vibration hammer image and the anti-vibration hammer image quality; The data expansion includes: performing data expansion according to the obtained anti-vibration hammer image and the diversity of the anti-vibration hammer image; The data set construction includes: labeling the data collected by each client according to a preset ratio using the LabelImg labeling tool, and dividing the data set into a labeled data set D l And the unlabeled dataset D u .

10. A method for detecting defects of transmission line anti-vibration hammers based on federated learning according to claim 1, characterized in that: The method of updating the model by using the exponential moving average EMA to implement the student model and teacher model pre-built based on the improved YOLOv10 model by the client specifically includes: Using the improved YOLOv10 model, two YOLOv10 network models are established and the model parameters are initialized, which are divided into a student model and a teacher model; The original input image I is converted into a multi-layer feature map T through the backbone network; The multi-layer feature map T is transformed into a feature map set T' after multi-scale feature fusion and enhancement; The feature map set T' is input into the CBAM attention mechanism to weight the feature map T' and output F, and the final prediction is made through the decoding prediction end.

Images