Federal learning-based multi-station power transmission line foreign matter identification method, device and system

Abstract

The invention discloses a federated learning-based multi-station power transmission line foreign matter identification method, device and system. A plurality of foreign matter recognition sites are arranged along a power transmission line to perform foreign matter recognition on the power transmission line, each foreign matter recognition site is provided with a client in communication connection with a server, and the method is executed by the client and comprises the steps of collecting a power transmission line image within a monitoring range of the current foreign matter recognition site; and inputting the acquired power transmission line image into a foreign matter recognition model trained based on a federated learning method for foreign matter recognition, and outputting a recognition result of the power transmission line image. The method can effectively solve the problem of low foreign matter recognition precision caused by non-uniform foreign matter distribution of multiple stations of the power transmission line.

Description

Technical Field

[0001] The present invention belongs to the technical field of foreign object detection in power transmission lines, and in particular relates to a method, device and system for identifying foreign objects in multi-site power transmission lines based on federated learning. Background Art

[0002] Transmission lines are an important part of the power system, and their safe and stable operation is crucial to the overall power grid. Since transmission lines are located outdoors, they are susceptible to interference and intrusion from the external environment. Common foreign objects such as kite strings, plastic bags, bird nests, etc. may attach to transmission lines, causing short circuits, tripping, and even more serious power failures. Traditional transmission line inspections rely on regular manual inspections, but this method is not only inefficient and costly, but also difficult to implement in areas with complex environments. In recent years, with the development of drone inspection technology and computer vision technology, automated foreign object detection methods have been widely used.

[0003] However, most existing automated detection systems use centralized learning methods, that is, all data needs to be transmitted to the cloud for model training. Centralized learning methods not only have risks in terms of data privacy, but also face the problem of uneven distribution of foreign objects, resulting in poor performance of the model when dealing with data heterogeneity. Therefore, there is an urgent need for a transmission line foreign object detection method that can effectively handle the problem of multi-site data heterogeneity while ensuring data privacy. Summary of the invention

[0004] In order to solve the deficiencies in the prior art, the present invention provides a method, device and system for identifying foreign objects in multi-site transmission lines based on federated learning, which can effectively solve the problem of low foreign object identification accuracy caused by uneven distribution of foreign objects at multiple sites of the transmission line.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] In a first aspect, a multi-site transmission line foreign object identification method based on federated learning is provided, wherein a plurality of foreign object identification sites are set along the transmission line to identify foreign objects on the transmission line, each of the foreign object identification sites is configured with a client that is communicatively connected to a server, and the method is executed by the client, including: collecting images of the transmission line within the monitoring range of the current foreign object identification site; inputting the collected transmission line images into a foreign object identification model trained based on the federated learning method to identify foreign objects, and outputting the identification results of the transmission line images.

[0007] Furthermore, the foreign object recognition model is a foreign object recognition model based on Vision Transformer, which divides the input power transmission line image into multiple image blocks, and flattens each image block into a vector through a flattening operation, and then projects the vector into the embedding dimension of the Transformer encoder through a linear projection operation. At the same time, an additional learnable vector is added to the vector, and its output at the Transformer encoder layer is used as an output to obtain an output sequence; the output sequence is then input into the Transformer encoder to obtain a feature vector, and finally, the feature vector is fed into the foreign object classifier to produce the final foreign object classification.

[0008] Furthermore, after projecting the vector into the embedding dimension of the Transformer encoder via a linear projection operation, a one-dimensional positional encoding is applied to the vector to preserve the position information.

[0009] Furthermore, the foreign object recognition model extracts global features of the power transmission line image through a self-attention mechanism to improve the accuracy of foreign object recognition.

[0010] Furthermore, the process of training the foreign object recognition model based on the federated learning method includes: receiving global model parameters distributed by the server; using the data locally stored on the client and the received global model parameters to independently train the local model of the client to obtain updated gradient information; sending the updated gradient information to the server; receiving the aggregated gradient distributed by the server, which is used to update the local model of the client to keep it consistent with the global model; wherein the aggregated gradient is obtained by the server based on the updated gradient information of each client; after multiple iterative training, a foreign object recognition model trained based on the federated learning method is obtained.

[0011] Furthermore, the gradient information is calculated based on client local data and a loss function.

[0012] Furthermore, the aggregated gradient is obtained using a client gradient dynamic aggregation strategy FedCDG, which dynamically adjusts the weight coefficient of each local model according to the client loss. The contribution of each local model to the global model is determined by its own weight coefficient, which is adjusted according to the local training loss and model performance.

[0013] The specific formula is as follows:

[0014]

[0015]

[0016] in, , N is the total number of clients, is the loss function of the i-th client in the e-th training cycle, is the loss function of the i-th client in the e-1-th training cycle, , E is the total training cycle, d is the hyperparameter, is the absolute value function, is the maximum value function, is the unnormalized weight coefficient of the nth client in the eth training cycle, is the weight coefficient of the nth client in the e-1th training cycle, is the weight coefficient of the nth client in the eth training cycle, is the unnormalized weight coefficient of the ith client in the eth training cycle, and the global model parameter Aggregated from the local model parameters of each client:

[0017]

[0018] in, are the local model parameters of the nth client in the eth training cycle.

[0019] Furthermore, the foreign objects identified by the foreign object recognition model include balloons, kites, bird nests and garbage.

[0020] In a second aspect, a multi-site transmission line foreign object identification device based on federated learning is provided, wherein a plurality of foreign object identification sites are arranged along the transmission line to identify foreign objects on the transmission line, and each of the foreign object identification sites is configured with a client that is communicatively connected to a server, and the client comprises: a data acquisition module for collecting images of the transmission line within the monitoring range of the current foreign object identification site; a foreign object identification module for inputting the collected transmission line images into a foreign object identification model trained based on the federated learning method for foreign object identification, and outputting the identification result of the transmission line image.

[0021] In a third aspect, a multi-site transmission line foreign object identification system based on federated learning is provided, comprising: a memory for storing instructions; a processor for executing the instructions, so that the device performs operations to implement the multi-site transmission line foreign object identification method based on federated learning as described in Example 1.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] (1) Compared with the existing centralized learning method, the present invention uses federated learning technology to share knowledge among multiple users, effectively overcoming the problem of insufficient data. At the same time, it solves the problems of low efficiency of data transmission and data privacy leakage caused by the centralized method. It can effectively solve the problem of low accuracy of foreign object recognition caused by uneven distribution of foreign objects at multiple sites of the transmission line.

[0024] (2) The client gradient dynamic aggregation strategy FedCDG proposed in this paper does not take the average value of all local model updates during the aggregation process, but assigns different weight coefficients to local models and dynamically adjusts them according to the client loss during the training process, thereby achieving higher recognition accuracy;

[0025] (3) Compared with the traditional foreign body recognition model, the foreign body recognition model based on Vision Transformer used in the present invention has better recognition performance. The foreign body recognition model based on Vision Transformer can effectively utilize its global attention mechanism and powerful feature learning ability, and performs well in foreign body detection tasks, especially suitable for fine detection tasks in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of a method for identifying foreign objects in multi-site power transmission lines based on federated learning provided by an embodiment of the present invention;

[0027] Figure 2 It is a structural block diagram of a multi-site transmission line foreign object identification system based on federated learning provided by an embodiment of the present invention;

[0028] Figure 3 is a structural diagram of the Vision Transformer model in an embodiment of the present invention;

[0029] Figure 4 It is a schematic diagram of the federated learning training process in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0031] Embodiment 1

[0032] like Figure 1As shown, a multi-site transmission line foreign object recognition method based on federated learning is provided, wherein multiple foreign object recognition sites are set along the transmission line to perform foreign object recognition on the transmission line, and each foreign object recognition site is configured with a client that is communicatively connected to a server; the method is executed by the client, and includes: collecting images of the transmission line within the monitoring range of the current foreign object recognition site; inputting the collected transmission line images into a foreign object recognition model trained based on the federated learning method to perform foreign object recognition, and outputting the recognition result of the transmission line image.

[0033] like Figure 2 As shown in the figure, in the task of foreign body detection on power transmission lines, the system as a whole adopts a cloud-edge-end architecture. During the training of the system, the cloud server is responsible for initializing the global model and distributing it to the local edge client. The model training is performed on the local edge client. The local client uses local data for model training to avoid sending local data to the cloud server, which protects the privacy of local data. After the local client trains the model, it sends the model parameters to the cloud server, which aggregates the model parameters and further updates the global parameters.

[0034] like Figure 3 As shown in FIG. 1 , the foreign body recognition model is a foreign body recognition model based on Vision Transformer. The main structure of the Vision Transformer model is described as follows.

[0035] For the input Q image (transmission line image), it is first divided into m image blocks of the same size , i=1,2,···,m,

[0036]

[0037] For each image block , first by flattening It is flattened into a vector; the vector is then projected into the embedding dimension of the Transformer encoder through a linear projection operation 𝓛, which is defined as follows:

[0038]

[0039] At the same time, an additional learnable vector α is added to the vector and its output at the Transformer encoder layer is used as the output; in order to preserve the position information, a one-dimensional position encoding is applied to the vector to obtain the output sequence x v as follows:

[0040]

[0041] in, is a learnable weight matrix.

[0042] Finally, the output sequence x v Used as input to the Transformer encoder. The feature vector F v is obtained after the Transformer encoder extracts feature information from the input sequence. This feature vector is then fed into the foreign object classifier to generate the final foreign object classification y i The process is represented as follows:

[0043]

[0044]

[0045] Among them, 𝒯 is the Transformer encoder and 𝒞 is the foreign object classifier.

[0046] The foreign object recognition model extracts the global features of the transmission line image through the self-attention mechanism without relying on local convolution operations, thereby improving the accuracy of foreign object recognition.

[0047] like Figure 4 As shown in the figure, the foreign object recognition model is trained based on the federated learning method. The entire training and testing process can be divided into six steps:

[0048] (1) Server distribution: The server distributes the parameters of the global model to each participating client.

[0049] (2) Local training: Each local client uses its locally stored data and received global model parameters to independently perform the training process. For each local client, the gradient information is calculated based on its local data and loss function.

[0050] (3) Gradient submission: The updated gradient information of each local model is sent to the server, all without exchanging original data.

[0051] (4) Gradient aggregation: The server receives and aggregates gradient updates from local models to derive new gradients. This aggregated gradient is then used to update a new global model with the same architecture as the local model.

[0052] (5) Broadcast: The aggregated gradient information of the global model is propagated to all local clients via broadcast. Importantly, this step ensures that the aggregated gradients are provided to local clients without compromising their individual privacy. The broadcasted aggregated gradient information is used to further update the local model to keep it consistent with the global model.

[0053] The next round of local training starts with the parameters of the global model of the current round. This process is repeated in all rounds through steps (1) to (5).

[0054] (6) Server model: Finally, after multiple iterations, the server will update and summarize to form the final foreign object recognition model.

[0055] In gradient aggregation, FedAvg is a commonly used method, but its average method of parameter aggregation ignores the heterogeneity of data and local updates, which may affect the effectiveness and robustness of the global model. This paper proposes a new parameter aggregation method: client gradient dynamic aggregation strategy, namely FedCDG. FedCDG dynamically adjusts the weight coefficient of each local model according to the client loss. The contribution of each local model to the global model is determined by its own weight coefficient, which is adjusted according to the local training loss and model performance; the specific formula is as follows:

[0056]

[0057]

[0058] in, , N is the total number of clients, is the loss function of the i-th client in the e-th training cycle, is the loss function of the i-th client in the e-1-th training cycle, , E is the total training cycle, d is the hyperparameter, is the absolute value function, is the maximum value function, is the unnormalized weight coefficient of the nth client in the eth training cycle, is the weight coefficient of the nth client in the e-1th training cycle, is the weight coefficient of the nth client in the eth training cycle, is the unnormalized weight coefficient of the ith client in the eth training cycle, and the global model parameter Aggregated from the local model parameters of each client:

[0059]

[0060] in, are the local model parameters of the nth client in the eth training cycle.

[0061] The present invention aims at the situation that foreign objects are unevenly distributed at foreign object detection sites on power transmission lines, which causes heterogeneous problems in federated learning training. A multi-site transmission line foreign object recognition method based on federated learning is proposed. The method includes a foreign object recognition model based on Vision Transformer and a federated learning method using a client gradient dynamic aggregation strategy. The proposed model uses a self-attention mechanism to process the global features of the image without relying on local convolution operations, which enables it to have better performance in foreign object recognition tasks. Using the client gradient dynamic aggregation strategy, the aggregation weight coefficient of the client is dynamically adjusted, which effectively solves the problem of heterogeneous distribution of different clients, improves the efficiency and accuracy of foreign object detection on transmission lines, and improves the data security of terminal devices. Foreign objects recognized by the foreign object recognition model of the present invention include but are not limited to balloons, kites, bird nests and garbage.

[0062] Embodiment 2

[0063] Based on the method for identifying foreign objects in a multi-site power transmission line based on federated learning described in Example 1, this embodiment provides a device for identifying foreign objects in a multi-site power transmission line based on federated learning, wherein a plurality of foreign object identification sites are arranged along the power transmission line to identify foreign objects in the power transmission line, and each of the foreign object identification sites is configured with a client connected to a server for communication, and the client comprises:

[0064] A data acquisition module is used to collect images of power transmission lines within the monitoring range of the current foreign object recognition station;

[0065] The foreign object recognition module is used to input the collected transmission line image into the foreign object recognition model trained based on the federated learning method to perform foreign object recognition and output the recognition result of the transmission line image.

[0066] Embodiment 3

[0067] Based on the method for identifying foreign objects in a multi-site transmission line based on federated learning described in Embodiment 1, this embodiment provides a system for identifying foreign objects in a multi-site transmission line based on federated learning, including:

[0068] A memory for storing instructions;

[0069] The processor is used to execute the instruction so that the device performs the operation of implementing the multi-site transmission line foreign object identification method based on federated learning as described in Example 1.

[0070] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0072] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0074] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A method for identifying foreign objects in multi-site transmission lines based on federated learning, characterized in that: A plurality of foreign object identification sites are arranged along the power transmission line to identify foreign objects on the power transmission line, each of the foreign object identification sites is configured with a client connected to a server for communication, and the method is executed by the client, comprising: Collect images of power transmission lines within the monitoring range of the current foreign object recognition site; The collected transmission line images are input into a foreign object recognition model trained based on the federated learning method for foreign object recognition, and the recognition results of the transmission line images are output.

2. The method for identifying foreign objects in multi-site power transmission lines based on federated learning according to claim 1, characterized in that: The foreign object recognition model is a foreign object recognition model based on Vision Transformer. The foreign object recognition model divides the input power transmission line image into multiple image blocks, and flattens each image block into a vector through a flattening operation, and then projects the vector into the embedding dimension of the Transformer encoder through a linear projection operation. At the same time, an additional learnable vector is added to the vector, and its output at the Transformer encoder layer is used as the output to obtain an output sequence; the output sequence is then input into the Transformer encoder to obtain a feature vector, and finally, the feature vector is fed into the foreign object classifier to produce the final foreign object classification.

3. The method for identifying foreign objects in multi-site power transmission lines based on federated learning according to claim 2 is characterized in that: After the vector is projected into the embedding dimension of the Transformer encoder via a linear projection operation, a one-dimensional positional encoding is applied to the vector to preserve the position information.

4. The method for identifying foreign objects in multi-site power transmission lines based on federated learning according to claim 2 is characterized in that: The foreign object recognition model extracts global features of the transmission line image through a self-attention mechanism to improve the accuracy of foreign object recognition.

5. The method for identifying foreign objects in multi-site power transmission lines based on federated learning according to claim 1, characterized in that: The process of training the foreign body recognition model based on the federated learning method includes: Receive global model parameters distributed by the server; Use the data stored locally on the client and the received global model parameters to independently train the client's local model to obtain updated gradient information; Send the updated gradient information to the server; Receive the aggregated gradient distributed by the server, and use it to update the local model of the client to make it consistent with the global model; wherein the aggregated gradient is obtained by the server based on the updated gradient information of each client; After multiple iterations of training, a foreign object recognition model trained based on the federated learning method was obtained.

6. The method for identifying foreign objects in multi-site power transmission lines based on federated learning according to claim 5, characterized in that: The gradient information is calculated based on the client local data and the loss function.

7. The method for identifying foreign objects in multi-site power transmission lines based on federated learning according to claim 5, characterized in that: The aggregated gradient is obtained using the client gradient dynamic aggregation strategy FedCDG. FedCDG dynamically adjusts the weight coefficient of each local model according to the client loss. The contribution of each local model to the global model is determined by its own weight coefficient, which is adjusted according to the local training loss and model performance. The specific formula is as follows: ， ， in, , N is the total number of clients, is the loss function of the i-th client in the e-th training cycle, is the loss function of the i-th client in the e-1-th training cycle, , E is the total training cycle, d is the hyperparameter, is the absolute value function, is the maximum value function, is the unnormalized weight coefficient of the nth client in the eth training cycle, is the weight coefficient of the nth client in the e-1th training cycle, is the weight coefficient of the nth client in the eth training cycle, is the unnormalized weight coefficient of the ith client in the eth training cycle, and the global model parameter Aggregated from the local model parameters of each client: ， in, are the local model parameters of the nth client in the eth training cycle.

8. The method for identifying foreign objects in multi-site power transmission lines based on federated learning according to claim 1, characterized in that: The foreign objects identified by the foreign object identification model include balloons, kites, bird nests and garbage.

9. A multi-site transmission line foreign object identification device based on federated learning, characterized in that: A plurality of foreign object identification stations are arranged along the power transmission line to identify foreign objects on the power transmission line, each of the foreign object identification stations is configured with a client connected to the server for communication, and the client includes: A data acquisition module is used to collect images of power transmission lines within the monitoring range of the current foreign object recognition station; The foreign object recognition module is used to input the collected transmission line image into the foreign object recognition model trained based on the federated learning method to perform foreign object recognition and output the recognition result of the transmission line image.

10. A multi-site transmission line foreign object identification system based on federated learning, characterized in that: include: A memory for storing instructions; A processor is used to execute the instruction so that the device performs the operation of implementing the multi-site transmission line foreign object identification method based on federated learning as described in any one of claims 1 to 8.

Images