Method and apparatus for detecting power lines

Abstract

The present disclosure provides a method and device for detecting power lines, wherein the method comprises: obtaining a target blurred image and a target event pulse vector corresponding to the target blurred image; wherein, within a preset time, the target blurred image is a motion blurred image about a power line to be detected in a target region collected based on a plenoptic camera, and the target event pulse vector is an event pulse vector determined based on an event stream about the power line to be detected in the target region collected based on an event camera; inputting the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset power line detection model, and outputting a preliminary detection image representing a power line detection result through the preset power line detection model; and based on a preset optimization rule, performing structural optimization on the preliminary detection image to obtain a final detection image. By using the method of the present disclosure, when facing a complex detection scene, the power line edge in the obtained final detection image is clear, and the detection precision and accuracy are high.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more particularly to a method, apparatus, electronic device, non-transitory computer-readable storage medium, and computer program product for detecting power lines. Background Technology

[0002] Power transmission plays a vital role in all sectors of modern society. Regular inspections of power transmission lines can promptly detect faults and potential hazards that could endanger line safety. This helps power companies to carry out targeted maintenance and upkeep of power transmission lines, thereby ensuring their safety and extending their service life.

[0003] Currently, power line inspection mainly relies on manual inspection and surveying of the lines. However, this method has significant drawbacks. Manual inspectors need to work at high altitudes and in complex environments, posing certain safety risks. Furthermore, the inspection range is limited, making it impossible to comprehensively inspect large areas of power lines. To address this issue, technicians in related fields have begun using drones to perform power line inspection tasks. Specifically, drones equipped with optical or infrared cameras acquire images of the power lines in the area to be inspected, and then the images are analyzed to determine the actual condition of the power lines.

[0004] However, current solutions for "UAVs performing power line detection tasks" suffer from problems such as blurred edges of power lines and low detection accuracy when facing complex detection scenarios. Summary of the Invention

[0005] This disclosure provides a method, apparatus, electronic device, non-transitory computer-readable storage medium, and computer program product for detecting power lines to address the deficiencies in the prior art.

[0006] This disclosure provides a method for detecting power lines, including: Obtain the target blurred image and the target event pulse vector corresponding to the target blurred image; Wherein, the target blurred image is a motion blurred image of the electric field lines to be detected in the target area, acquired by a general-purpose camera within a preset time period, and the target event pulse vector is an event pulse vector determined based on the event flow of the electric field lines to be detected in the target area, acquired by an event camera within the preset time period. The target blurred image and the target event pulse vector corresponding to the target blurred image are input into a preset electric line detection model, and the preset electric line detection model outputs a preliminary detection map representing the electric line detection result. The preset power line detection model is trained based on at least one set of training data. Each set of training data includes sample input data and sample ground truth. The sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image. The sample ground truth includes annotation data that matches the blurred sample image. The annotation data is used to mark real power line segments. Based on preset optimization rules, the preliminary detection image is structurally optimized to obtain the final detection image.

[0007] This disclosure also provides an apparatus for detecting power lines, comprising: The data acquisition module is configured to acquire a target blurred image and a target event pulse vector corresponding to the target blurred image; Wherein, the target blurred image is a motion blurred image of the electric field lines to be detected in the target area, acquired by a general-purpose camera within a preset time period, and the target event pulse vector is an event pulse vector determined based on the event flow of the electric field lines to be detected in the target area, acquired by an event camera within the preset time period. The detection execution module is configured to: input the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset electric line detection model, and output a preliminary detection map representing the electric line detection result through the preset electric line detection model; The preset power line detection model is trained based on at least one set of training data. Each set of training data includes sample input data and sample ground truth. The sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image. The sample ground truth includes annotation data that matches the blurred sample image. The annotation data is used to mark real power line segments. The result optimization module is configured to: perform structural optimization on the preliminary detection image based on preset optimization rules to obtain the final detection image.

[0008] This disclosure also provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method for detecting power lines as described above.

[0009] This disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for detecting power lines as described in any of the preceding embodiments.

[0010] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the method for detecting power lines as described in any of the above.

[0011] As described above, the method for detecting power lines provided in this disclosure addresses the problems of the prior art by introducing auxiliary information from event camera data. Specifically, firstly, the preprocessing module in the feature fusion network adaptively weights image and event features based on channel attention. Then, the fusion module in the feature fusion network enhances deep multi-scale features intramodally and interacts with them intermodally based on self-attention, resulting in features fused from the event and image. Next, the feature map is input into the edge detection network (i.e., a network improved based on boundary detection convolutional network FE-BDCN) to obtain relatively clear power line edges, i.e., the preliminary detection map. Finally, the preliminary detection map can be optimized using preset optimization rules to obtain the final detection map.

[0012] Furthermore, as verified by the verification embodiments described below, the method for detecting power lines (by fusing an event camera and a general-purpose camera) provided by this disclosure produces clear power line edges and high detection accuracy in the final detection image when facing complex detection scenarios, thus demonstrating the effectiveness of the method disclosed herein. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is one of the flowcharts illustrating a method for detecting power lines provided in this embodiment of the disclosure; Figure 2 This is a schematic diagram of the structure of the preset power line detection model provided in the embodiments of this disclosure; Figure 3 This is a second schematic flowchart of a method for detecting power lines provided in this embodiment of the present disclosure; Figure 4 This is a schematic diagram of the process for obtaining the at least one set of training data provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram comparing the power line detection results obtained by the preset power line detection model provided in this embodiment with other detection models based on the same test data. Figure 6This is a schematic diagram of the structure of a device for detecting power lines provided in an embodiment of this disclosure; Figure 7 This is a schematic diagram of the structure of the electronic device provided in this disclosure. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0016] Overview of the inventive concept: The inventors of this disclosure have discovered that the problems with the prior art stem from the following: Although drones possess efficient and comprehensive detection capabilities, enabling them to cover a wide area of ​​power lines during aerial flight, the frame rate and dynamic range of their mounted general-purpose cameras are limited. In challenging scenarios such as low light, rapid movement, and background clutter, these cameras struggle to function properly due to overexposure, underexposure, and motion blur. While drones equipped with infrared cameras can detect the temperature distribution of electrical equipment, their temperature measurement accuracy is affected by ambient temperature and atmospheric conditions, thus limiting their precision. Furthermore, both ordinary optical and infrared cameras cannot penetrate obstacles for detection, potentially leading to the overlooking of hidden defects in power lines in complex environments.

[0017] Based on the above reasons, the inventors further discovered that event cameras have high temporal resolution and high dynamic range, thus exhibiting good resistance to motion blur. Furthermore, event cameras have low power consumption and low data transmission requirements, making them ideal for use with power line inspection drones that are limited in size and require long-term operation. However, the inventors also found that if power line detection is based solely on event cameras, the solution suffers from limitations or problems in resolution, color information, and sensitivity to static scenes. Moreover, in scenarios with frequently changing ambient light, event cameras may generate a large number of "noise" events, affecting the clarity and usability of the visual data. In other words, while event cameras can capture information with extremely high temporal resolution (microseconds), making them ideal for monitoring fast-moving objects, they cannot capture color information and static image information.

[0018] In view of this, after comprehensively analyzing the advantages and disadvantages of general-purpose cameras and event cameras, the inventors of this disclosure considered that the complementary characteristics of general-purpose cameras and event cameras could be utilized to achieve accurate and clear power line detection by fusing the image obtained from the general-purpose camera and the event stream obtained from the event camera. Based on this inventive concept, the following embodiments of a scheme for detecting power lines are proposed.

[0019] Example: The following description, in conjunction with the accompanying drawings, illustrates the scheme for detecting power lines disclosed herein.

[0020] Figure 1 This is a schematic flowchart of a method for detecting power lines provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices (e.g., servers or cloud computing platforms), such as... Figure 1 As shown, the method for detecting power lines includes the following steps: S110. Obtain the target blurred image and the target event pulse vector corresponding to the target blurred image.

[0021] The target blurred image is a motion-blurred image of the electric field lines to be detected within a target area, acquired by a general-purpose camera within a preset time period. The target event pulse vector is an event pulse vector determined based on the event flow of the electric field lines to be detected within the target area, acquired by an event camera within the preset time period.

[0022] The “target area” is the area where power line detection is required, and the power lines to be detected exist within this area.

[0023] This disclosure does not limit the method of acquisition. For example, an electronic device (e.g., a server or cloud computing platform) acting as the execution subject can communicate with a server storing the "target blurred image and target event pulse vector" to obtain the "target blurred image and target event pulse vector".

[0024] S120. Input the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset power line detection model, and output a preliminary detection map representing the power line detection result through the preset power line detection model.

[0025] The preset power line detection model is trained based on at least one set of training data. Each set of training data includes sample input data and sample ground truth. The sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image. The sample ground truth includes annotation data that matches the blurred sample image. The annotation data is used to mark real power line segments.

[0026] Optionally, the labeled data can be represented by labels with values ​​of 0 or 1. 0 indicates that the labeled data is not a power line; 1 indicates that the labeled data is a power line.

[0027] Optionally, the preset power line detection model may include a cascaded feature fusion network and an edge detection network. Specifically, the feature fusion network may include a VGG16 network and a channel attention network; the edge detection network may include a network based on a boundary detection convolutional network (BDCN), which can be denoted as FE-BDCN.

[0028] It should be noted that the details of the preset power line detection model will be described below, and will not be repeated here.

[0029] S130. Based on preset optimization rules, the preliminary detection map is structurally optimized to obtain the final detection map.

[0030] Since the preliminary detection map output by the preset power line detection model may contain noise interference, further optimization can effectively filter out the noise in the preliminary detection map, thereby obtaining a clear and structured power line sketch as the final detection map.

[0031] As described above, the method for detecting power lines provided in this disclosure addresses the problems of the prior art by introducing auxiliary information from event camera data. Specifically, firstly, the preprocessing module in the feature fusion network adaptively weights image and event features based on channel attention. Then, the fusion module in the feature fusion network enhances deep multi-scale features intramodally and interacts with them intermodally based on self-attention, resulting in features fused from the event and image. Next, the feature map is input into the edge detection network (i.e., a network improved based on boundary detection convolutional network FE-BDCN) to obtain relatively clear power line edges, i.e., the preliminary detection map. Finally, the preliminary detection map can be optimized using preset optimization rules to obtain the final detection map.

[0032] Furthermore, as verified by the verification embodiments described below, the method for detecting power lines (by fusing an event camera and a general-purpose camera) provided by this disclosure produces clear power line edges and high detection accuracy in the final detection image when facing complex detection scenarios, thus demonstrating the effectiveness of the method disclosed herein.

[0033] exist Figure 1 Based on the embodiments, as an optional implementation, the preset power line detection model may include a cascaded feature fusion network and an edge detection network.

[0034] Figure 1 Based on the embodiments and the above implementation methods, as an optional implementation method, the feature fusion network includes a cascaded feature extraction subnetwork, a channel splicing unit, and a channel attention fusion subnetwork.

[0035] As an optional example, see [reference] Figure 2 The feature extraction subnetwork includes a parallel image feature extraction unit VGG16_C (for side-outputting the image feature sequence List_rgb) and an event feature extraction unit VGG16_10C (for side-outputting the time feature sequence List_eve). The output of the image feature extraction unit VGG16_C is connected to the input of the channel stitching unit (for "stitching by channel"), and the output of the event feature extraction unit VGG16_10C is connected to the input of the channel stitching unit.

[0036] As an optional example, see [reference] Figure 2 The channel attention fusion subnetwork ( Figure 2 The “CAFusionModule” shown includes cascaded channel splitting units (i.e. Figure 2 The unit where X is located), the attention unit, and the fusion unit (i.e., Figure 2 The unit where "fusion" is located is shown. The attention unit includes a first attention subunit connected in parallel (i.e., Figure 2 The unit corresponding to the "attention" point connected to channel X1 (shown in the diagram) and the second attention subunit are also shown. Figure 2 The unit corresponding to the "attention" point connected to the X2 channel is shown; the first output of the channel splitting unit (i.e., Figure 2 The X1 channel shown is connected to the input of the first attention subunit, and the second output of the channel splitting unit (i.e., Figure 2 The X2 channel shown is connected to the input of the second attention subunit; the output of the first attention subunit is connected to the input of the fusion unit, and the output of the second attention subunit is connected to the input of the fusion unit. The output of the fusion unit is connected to the input of the edge detection network.

[0037] Figure 1 Based on the embodiments and the above implementation methods, as an optional example, the image feature extraction unit VGG16_C includes a cascaded first convolution module, a second convolution module, and three third convolution modules.

[0038] The first convolutional module comprises two cascaded convolutional layers and one pooling layer. The convolutional kernel of the first convolutional layer has a 3×3 kernel, 64 channels, and a stride of 1. The convolutional kernel of the pooling layer has a 2×2 kernel and a stride of 2. The first convolutional layer in the first convolutional module has 3 receiving channels. The second convolutional module comprises two cascaded convolutional layers and one pooling layer. The convolutional kernel of the second convolutional layer has a 3×3 kernel, 128 channels, and the convolutional kernel of the pooling layer has a 2×2 kernel and a stride of 2. The third convolutional module comprises a first convolutional layer, a second convolutional layer, a third convolutional layer, and one pooling layer. The convolutional kernel of the first convolutional layer has a 3×3 kernel and 256 channels. The convolutional kernel of the second convolutional layer has a 3×3 kernel and 512 channels. The convolutional kernel of the third convolutional layer has a 3×3 kernel and 512 channels. The convolutional kernel of the pooling layer has a 2×2 kernel and a stride of 2.

[0039] The structure of the event feature extraction unit VGG16_10C is the same as that of the image feature extraction unit VGG16_C; wherein, the number of receiving channels of the first convolutional layer in the first convolutional module of the event feature extraction unit VGG16_10C is 10.

[0040] Optionally, refer to Figure 2 The size of the target blurred image input to the image feature extraction unit VGG16_C is 512×512×3; the size of the target event pulse vector input to the event feature extraction unit VGG16_10C is 512×512×10.

[0041] Since the structure of the event feature extraction unit VGG16_10C is the same as that of the image feature extraction unit VGG16_C, the following example illustrates its processing of the input target blurred image: First, the target blurred image passes through the first convolution module for downsampling to reduce the spatial size of image features; second, these features are input into the second convolution module for further refinement of image features; third, the image features pass through three third convolution modules to continue downsampling and enhance the depth and complexity of the image feature representation. Finally, after this series of convolution and pooling operations, the resulting feature map will be used for downstream tasks, such as classification or detection, providing rich visual information content.

[0042] Figure 1 Based on the embodiments and the above implementation methods, as an optional example, the first attention subunit and the second attention subunit have the same structure.

[0043] Reference Figure 2The first attention subunit includes a cascaded input layer (for receiving and processing "input features F"), a pooling module, a connection layer, a convolutional layer, a sigmoid activation function layer, a multiplication module, a summation module, and an output layer.

[0044] The pooling module includes a parallel average pooling layer and a max pooling layer; the output of the input layer is connected to the input of the average pooling layer and the max pooling layer, respectively, and the output of the average pooling layer and the max pooling layer is connected to the input of the connection layer, respectively.

[0045] The output of the input layer is connected in a jump connection to the input of the multiplication module and the input of the summation module.

[0046] The following explanation details the method for optimizing the weights of the "target event impulse vector and target blurred image" input to the feature fusion network.

[0047] After extracting deep features from the input target event impulse vector using the event feature extraction unit VGG16_10C, and extracting deep features from the input target blurred image using the image feature extraction unit VGG16_C, these features are fed into the channel attention fusion subnetwork to further enhance its expressive power. Specifically, the channel attention fusion subnetwork, namely the first attention subunit and the second attention subunit, first evaluates the global features of each channel by calculating the average and maximum values ​​of each channel to capture channel statistics. This information is then fed into two independent fully connected layers. The first fully connected layer reduces dimensionality to decrease the number of parameters and computational complexity, while the second fully connected layer restores the original channel dimensions. During this process, a ReLU activation function is used to add non-linearity to extract richer feature representations. The resulting channel weights are then passed through a Sigmoid activation function to normalize the weights to between 0 and 1, allowing them to be directly applied as scaling factors to the original features. Finally, these weighted features are combined with the original features through element-wise multiplication, thus completing the feature recalibration. This process not only highlights more informative features but also suppresses less important features, thereby improving the model's sensitivity to key visual information and its final recognition performance.

[0048] Optionally, the entire shallow fusion process can be described by the following calculation formulas (1) to (4); (1) (2) (3) (4) Among them, CA F Indicates the first attention subunit; CA E Indicates the second attention subunit; Indicates element-wise multiplication; XF represents the features of the blurred target image; XE represents the features of the target event impulse vector; Y F Y represents the output feature after the target blurred image features have been processed by the first attention subunit and combined with the target event impulse vector features; E This represents the output feature after the event impulse vector features have been processed by the second attention subunit and combined with the target blurred image features; Attn F Attn represents the attention weights of the blurred image features after processing by the first attention subunit; E This represents the attention weights after the target event impulse vector features have been processed by the first attention subunit.

[0049] Continue to refer to Figure 2 The following explanation concerns the dynamic adjustment of the feature fusion method between the event branch and the image branch in the channel attention fusion subnetwork.

[0050] Channel attention mechanism is used to enhance and fuse features from different data sources. The channel attention fusion subnetwork first initializes two attention sub-modules (i.e., the first attention sub-unit and the second attention sub-unit), which are applied to different feature streams respectively. During forward propagation, the input feature vector X is first divided into two parts, namely X1 and X2. This split is performed along the channel dimension (i.e., split by channel). Each part of the feature passes through its corresponding channel attention sub-module, which independently adjusts its own features, strengthening key information and suppressing less important information by learning the importance of each channel. The calculation process of the attention sub-module AFF can be expressed by the following calculation formulas (5) to (8): (5) (6) (7) (8) in, This represents the first attention subunit; This represents the second attention subunit; This represents the input feature vector during the forward propagation process; express After segmentation, the feature vector of the first attention subunit is input accordingly; express After segmentation, the feature vector of the second attention subunit is input accordingly; express right Processed attention weights; express right The attention weights after processing.

[0051] Optionally, the first choice can be to adopt The function equally distributes the input feature vectors Divide into two sub-vectors and Then channel attention is applied, and finally the plus sign indicates that the two processed feature vectors are added element-wise to obtain the output. Fusion unit (i.e., Figure 2 The unit where "fusion" is located uses channel splicing and 1×1 convolution to fuse the input image features and event features into a single feature, which is then fed into the edge detection network.

[0052] Figure 1 Based on the embodiments and the above-described implementation methods, as an optional example, please continue to refer to... Figure 2 The edge detection network includes a first detection module (ID Block 1), a second detection module (ID Block 2), and a first upsampling layer (i.e., ...) cascaded together. Figure 2 As shown, the "up" connected to ID Block2), the third detection module (ID Block3), and the second upsampling layer (i.e., Figure 2 As shown, "up" is connected to ID Block3. The output of the second detection module is also connected to the input of the first detection module, and the output of the third detection module is also connected to the input of the second detection module.

[0053] Optionally, the first detection module includes three first detection sub-modules connected in parallel; wherein, the first detection sub-module (ID Block 1) includes cascaded first detection convolutional layers ( Figure 2 As shown in "3×3-32conv"), the first ReLU activation function layer, and the first detection convolutional block ( Figure 2 The “SEM” shown) and channel-reduced convolutional layers ( Figure 2 (as shown in "1×1-1conv"), the first detection convolutional layers in adjacent first detection submodules are cascaded; the outputs of the first detection convolutional blocks in the three first detection submodules are interconnected; the first detection convolutional block includes multiple cascaded convolutional layers with different dilation rates; the kernel parameters of the first detection convolutional layer are 3×3 and the number of channels is 32, and the kernel parameters of the channel reduction convolutional layer are 1×1 and the number of channels is 1.

[0054] Optionally, the second detection module includes three parallel second detection sub-modules; wherein, each second detection sub-module includes a cascaded second detection convolutional layer, a second activation function ReLU layer, a second detection convolutional block, and a channel-reduced convolutional layer; the second detection convolutional layers in adjacent second detection sub-modules are cascaded; the outputs of the second detection convolutional blocks in the three second detection sub-modules are interconnected; the second detection convolutional block includes multiple cascaded convolutional layers with different dilation rates; the number of channels in the second detection convolutional layer is 64, and the kernel parameter of the channel-reduced convolutional layer is 1×1 with 1 channel.

[0055] Optionally, the third detection module includes three parallel third detection sub-modules; wherein, each third detection sub-module includes a cascaded third detection convolutional layer, a third activation function ReLU layer, a third detection convolutional block, and a channel-reduced convolutional layer; the third detection convolutional layers in adjacent third detection sub-modules are cascaded; the outputs of the third detection convolutional blocks in the three third detection sub-modules are interconnected; each third detection convolutional block includes multiple cascaded convolutional layers with different dilation rates; the number of channels in the third detection convolutional layer is 128, and the kernel parameter of the channel-reduced convolutional layer is 1×1 with 1 channel.

[0056] It should be noted that the channel reduction convolutional layers in the first detection module (ID Block 1), second detection module (ID Block 2), and third detection module (ID Block 3) are mainly used to reduce the number of output channels to 21. The first upsampling layer (i.e., Figure 2 As shown, the "up" connected to ID Block2 is used to double the size of the feature map; the second upsampling layer (i.e., Figure 2 As shown, the “up” connected to ID Block3 is used to increase the size of the feature map by four times.

[0057] It should be noted that the first detection convolutional block ( Figure 2 The structures of the "SEM" (second detection convolutional block) and the third detection convolutional block shown can be consistent, i.e. (MSBLock class); however, their network layer parameters are different. These different parameters are used to transmit different numbers of input channels (c_in), convolutional layers (k), and dilation rates (rate), thereby enabling different feature extraction functions.

[0058] As an optional example, see [reference] Figure 2 The parameters for "SEM" can be described in detail as follows: r represents the dilation rate, r0 represents the basic dilation rate, and K represents a coefficient used to control the multiple of the dilation rate. r=r0 represents the dilation rate of the first convolutional layer; This represents the inflation rate of the second convolutional layer; This represents the dilation rate of the third convolutional layer; and so on, until... .

[0059] In the above Figure 1 , Figure 2 Based on the embodiments, as an optional implementation method, refer to Figure 3 The training steps of the preset power line detection model include: S310. Obtain the at least one set of training data.

[0060] The training data includes positive and negative sample input data.

[0061] As an alternative example, a training dataset can be constructed first, and then the at least one set of training data can be obtained based on the training dataset.

[0062] Furthermore, referring to Figure 4 The training dataset can be constructed in the following manner.

[0063] 1) By analyzing the general-purpose camera within a preset time T (based on the camera trajectory t0,...,t...), k ,...,t n Multiple clear images captured (i.e., Figure 4 The simulated time series (shown) is subjected to weighted averaging to obtain blurred sample images with motion blur. (Right now, Figure 4 (The blurred image shown). Optionally, the blurred sample image. It has been scaled down to a size of 512×512×3.

[0064] 2) Select the last clear image within the preset time T (e.g., Figure 4 The image shown is the "line segment annotation" image below the "original image". The annotations are then used to accurately identify the real line segments (i.e., to obtain the "line segment annotation" image below the blurred image). These annotations will be used in the subsequent training process.

[0065] Optionally, the labeled data can be represented by labels with values ​​of 0 or 1. 0 indicates that the labeled data is not a power line; 1 indicates that the labeled data is a power line.

[0066] It should be noted that the annotation data can be stored in .mat file format. In addition to header, version, and globals, the special variables of the metadata in the .mat file format also include the groundTruth variable, which contains a field called Boundaries, which can store the actual annotation data.

[0067] 3) As for the event stream time within the preset time T, we generate five frames by dividing the event stream data captured by the event camera within this time into five equal parts and performing linear interpolation. The event frame. The number 2 indicates the positive or negative polarity of the event.

[0068] The calculation formulas (9) to (11) for the linear interpolation are as follows: (9) (10) (11) in, Indicates the earliest timestamp in the event stream; H represents the duration of the event stream; W represents the height of the event frame; x represents the x-coordinate of a pixel in the event frame; y represents the y-coordinate of a pixel in the event frame; e represents the duration of the event stream. k x represents the k-th event in the event stream; k Indicates event e k The horizontal coordinate position in the event frame; y k Indicates event e k The vertical coordinate position in the event frame; B represents the size of the time dimension. Indicates event e k Abrupt changes in position within the event stream, Indicates event e k The weight of event e reflects the weight of event e. k The magnitude of its contribution to the interpolation result.

[0069] The polarity refers to the direction of change in light intensity captured by the event camera; an increase in light intensity is positive polarity, and a decrease in light intensity is negative polarity.

[0070] It should be noted that, to accommodate the three-channel input of the event feature extraction unit VGG16_10C, the polarity dimension is compressed into the time segmentation dimension. The vector is then scaled to obtain an event input of 512×512×10.

[0071] As an alternative example, the phrase "compressing the polarity dimension into the time segmentation dimension" can be performed as follows: First, event polarity mapping is performed. Specifically, the polarity information of the event is mapped to a time segment. Taking the original event vector of H×W×B×2 as an example, the polarity dimension is compressed to the time segment dimension, making it a vector of H×W×B', where B' represents the number of new time segments. Second, polarity merging is performed, that is, positive and negative polarity events within the same time segment are merged together. Specifically, positive and negative polarity events can be superimposed or weighted averaged. Finally, vector transformation is performed. Specifically, after polarity merging, the size is adjusted as needed. We adjust the number of time segments B to 10 to adapt to the input of the event feature extraction unit VGG16_10C.

[0072] In summary, by processing the sample images or data acquired by the general-purpose camera and the event camera through steps 1) to 3) above, paired blurred sample images can be obtained. The training dataset can be constructed by combining the event frames represented by event stream sample vectors and the corresponding labeled data.

[0073] S320. Input the sample input data from each group of training data into the power line detection model to be trained, and output the corresponding power line detection estimation result through the power line detection model to be trained.

[0074] S330. Using a preset comprehensive loss function, based on the true values ​​of the samples in each group of training data and the corresponding power line detection estimation results, obtain the function value of the comprehensive loss function.

[0075] As an optional example, S330 can be implemented as follows: I1) Based on the true values ​​of the samples in each set of training data and the corresponding power line detection estimation results, calculate the function values ​​of the loss function of the first detection module, the loss function of the second detection module, the loss function of the third detection module, and the fusion loss function, respectively.

[0076] Optionally, calculating the function value of the loss function of the first detection module based on the true values ​​of the samples in each set of training data and the corresponding power line detection estimation results includes: The function value of the loss function of the first detection module is calculated using the following formula (12); (12) in, denoted as the binary cross-entropy function; s1 represents the positive probability of the electric field detection estimate output by the first detection module for each set of sample input data; s11 represents the negative probability of the electric field detection estimate output by the first detection module for each set of sample input data; y represents the true value of the sample corresponding to each set of sample input data.

[0077] Optionally, calculating the function value of the loss function of the second detection module based on the true values ​​of the samples in each set of training data and the corresponding power line detection estimation results includes: The function value of the loss function of the second detection module is calculated using the following formula (13); (13) Where s2 represents the positive probability of the power line detection estimate output by the second detection module for each set of sample input data; s21 represents the negative probability of the power line detection estimate output by the second detection module for each set of sample input data; and y represents the true value of the sample corresponding to each set of sample input data.

[0078] Optionally, calculating the function value of the loss function of the third detection module based on the true values ​​of the samples in each set of training data and the corresponding power line detection estimation results includes: The function value of the loss function of the third detection module is calculated using the following formula (14); (14) Wherein, s3 represents the positive probability of the electric line detection estimate output by the third detection module for each set of sample input data; s31 represents the negative probability of the electric line detection estimate output by the third detection module for each set of sample input data; and y represents the true value of the sample corresponding to each set of sample input data.

[0079] Optionally, calculating the value of the fusion loss function based on the ground truth values ​​of the samples in each set of training data and the corresponding power line detection estimation results includes: The function value of the fusion loss function is calculated using the following formula (15); (15) Where fuse represents the fusion probability of the power line detection estimate output by the power line detection model to be trained for each set of sample input data; y represents the true value of the sample corresponding to each set of sample input data.

[0080] It should be noted that the BCELoss function involved in the calculation formulas (12) to (15) in the above I1 example can be adaptively calculated based on the following calculation formulas (16) to (19).

[0081] (16) (17) (18) The meanings of the symbols in (16) to (18) are explained in principle only and are not limited to the aforementioned embodiments of this disclosure.

[0082] Where N represents the total number of samples; Indicates the first i The true label (0 or 1) of each sample; Indicates the first i The predicted probability of each sample (the value after passing through the sigmoid function); Indicates the first i The weights of each sample are used to balance positive and negative samples; Represents the sigmoid function; Indicates the first i Input for each sample; When the value is 1, Indicates the weight of positive samples; When the value is 0, Indicates the weight of negative samples; Indicates the number of negative samples; Indicates the number of positive samples; This represents a hyperparameter used to balance positive and negative samples.

[0083] I2) Using preset weights, the function values ​​of the loss function of the first detection module, the loss function of the second detection module, the loss function of the third detection module, and the fusion loss function are weighted and fused to obtain the function value of the comprehensive loss function.

[0084] As an optional example, the function value of the comprehensive loss function can be calculated using the following formula (5); (5) in, This represents the weight of the loss function of the first detection module; This represents the weights of the loss function of the second detection module; The weights represent the weights of the loss function of the third detection module; This represents the weights of the fusion loss function.

[0085] S340. Based on the function value of the comprehensive loss function, train the power line detection model to be trained until the preset training completion condition is met, and obtain the neural network from the power line detection model to be trained.

[0086] As an alternative example, S340 can be implemented in the following way: First, the value of the comprehensive loss function is backpropagated to the electric line detection model to be trained. Second, the electric line detection model to be trained adjusts the network parameters of each network layer based on the value of the comprehensive loss function. Finally, the steps of obtaining the value of the comprehensive loss function and adjusting the network parameters of each network layer are iteratively executed until the preset training completion conditions are met.

[0087] Among them, the preset training completion condition can be that the number of training iterations reaches a preset threshold, or that the function value of the above comprehensive loss function is less than the preset loss threshold, i.e., convergence.

[0088] Based on the above embodiments, as an optional implementation, step S130, "optimizing the structure of the preliminary detection image based on preset optimization rules to obtain the final detection image," can be achieved as follows: using pre-configured structured prior knowledge of power lines to filter edge noise in the preliminary detection image to obtain the final detection image. The structured prior knowledge includes three dimensions: length, size, and direction.

[0089] Specifically, the preliminary detection map can be binarized, and its connected regions can be calculated. The direction of each region is approximated by the major axis, and the length is estimated by the diagonal of the bounding box. It is assumed that the longest segment represents the electric field line, while noise segments are filtered out through specific measurements of area and direction. Specifically, a segment whose area is much smaller than the area of ​​the longest segment or whose direction is significantly different is considered noise. This method effectively identifies electric field lines from noise, ultimately yielding a clear and structured sketch of the electric field lines, which can then be used as the final detection map.

[0090] Verification Example: This verification embodiment is mainly used to evaluate the effectiveness of the method for detecting power lines provided in the embodiments of this disclosure.

[0091] Specifically, a given set of test data is used, wherein the set of test data includes sample input data and sample ground truth; further, the sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image, and the sample ground truth includes labeled data matching the blurred sample image.

[0092] Models used for comparison with the preset power line detection model (i.e., FE-BDCN) provided in the embodiments of this disclosure include Gestaltic Grouping, DexiNed, LSD, HED, and BDCN.

[0093] The sample input data from the given set of test data is input into FE-BDCN, GestalticGrouping, DexiNed, LSD, HED, and BDCN respectively, to obtain the power line detection results output by each model, as shown below. Figure 5 As shown.

[0094] Reference Figure 5 The figure contains nine sets of data. The first row shows the motion-blurred RGB images obtained from the general-purpose camera, the second row shows the ground truth images, and the third row shows the event stream. Based on the existing ground truth images, the power line edge detection images obtained by different methods are arranged to visually compare the detection effects of different methods. The fourth row shows the detection results based on RGB images using Gestaltic Grouping. Rows 5 to 8 show the results based on blurred images using DexiNed, LSD, HED, and BDCN methods, respectively. The ninth row shows the detection results based on the improved FE-BDCN method, using a method that fuses event frames and motion-blurred RGB images. It can be seen that a large amount of false detection noise appears in LSD and Gestaltic Grouping. The images obtained by the DexiNed, LSD, and BDCN methods show obvious missed detections and false detections. The power line edge detection model using fused event frames and RGB images produces the cleanest and clearest power line edge map, verifying the effectiveness of the proposed power line detection method that fuses event cameras and general-purpose cameras.

[0095] Furthermore, based on the power line detection results and labeled data output by the above multiple models, each model was quantitatively evaluated, and the evaluation results are shown in Table 1.

[0096]

[0097] Table 1. Quantitative evaluation results of different edge detection models on the same test data. Referring to Table 1, where ER, Prec, Recon, and F1 represent the average error rate, average precision, average recall, and average F1 score of different models when the threshold is 5, respectively.

[0098] Based on the data in Table 1, it can be seen that the average F1 score of traditional models or methods (including LSD, HED, DexiNed, and GestalticGrouping) for processing blurred images is only 0.963 at most. The FE-BDCN proposed in this invention, however, can reduce the average error rate and improve the accuracy of power line edge capture compared to the original BDCN. Furthermore, the FE-BDCN model introduces an event flow and employs an attention mechanism for feature fusion, building upon the previous deep learning based solely on blurred RGB images. This demonstrates excellent robustness and accuracy in power line edge detection (extraction) scenarios with complex backgrounds.

[0099] As described above, the method for detecting power lines provided in this disclosure addresses the problems of the prior art by introducing auxiliary information from event camera data. Specifically, firstly, the preprocessing module in the feature fusion network adaptively weights image and event features based on channel attention. Then, the fusion module in the feature fusion network enhances deep multi-scale features intramodally and interacts with them intermodally based on self-attention, resulting in features fused from the event and image. Next, the feature map is input into the edge detection network (i.e., a network improved based on boundary detection convolutional network FE-BDCN) to obtain relatively clear power line edges, i.e., the preliminary detection map. Finally, the preliminary detection map can be optimized using preset optimization rules to obtain the final detection map.

[0100] Furthermore, based on the verification of the above-described embodiments, the method for detecting power lines (by fusing event cameras and general-purpose cameras) provided by this disclosure produces clear power line edges and high detection accuracy in the final detection image when facing complex detection scenarios, thus demonstrating the effectiveness of the method disclosed herein.

[0101] The apparatus for detecting power lines provided in this disclosure is described below. The apparatus for detecting power lines described below can be referred to in correspondence with the method for detecting power lines described above.

[0102] Figure 6 This is a schematic diagram of a device for detecting power lines provided in an exemplary embodiment of this disclosure. Figure 6 As shown, the device for detecting power lines includes: Data acquisition module 610 is configured to: acquire a target blurred image and a target event pulse vector corresponding to the target blurred image; wherein, the target blurred image is a motion-blurred image of a power line to be detected within a target area acquired by a general-purpose camera within a preset time period, and the target event pulse vector is an event pulse vector determined based on an event stream of the power line to be detected within the target area acquired by an event camera within the preset time period; detection execution module 620 is configured to: input the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset power line detection model, and perform the detection... Assume a power line detection model outputs a preliminary detection map representing the power line detection results; wherein the preset power line detection model is trained based on at least one set of training data, each set of training data includes sample input data and sample ground truth, the sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image, the sample ground truth includes annotation data matching the blurred sample image, the annotation data being used to identify real power line segments; the result optimization module 630 is configured to: perform structural optimization on the preliminary detection map based on preset optimization rules to obtain a final detection map.

[0103] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other through the communications bus 740. The processor 710 can call logic instructions in the memory 730 to execute a method for detecting power lines, the method comprising: acquiring a target blurred image and a target event pulse vector corresponding to the target blurred image; wherein the target blurred image is a motion blurred image of a power line to be detected within a target area acquired by a general-purpose camera within a preset time period, and the target event pulse vector is an event pulse vector determined based on an event stream of the power line to be detected within the target area acquired by an event camera within the preset time period; inputting the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset power line detection model, and outputting a preliminary detection map characterizing the power line detection result through the preset power line detection model; wherein the preset power line detection model is trained based on at least one set of training data, each set of training data including sample input data and sample ground truth, the sample input data including a blurred sample image and an event stream sample vector corresponding to the blurred sample image, the sample ground truth including annotation data matching the blurred sample image, the annotation data being used to mark real power line segments; and performing structural optimization on the preliminary detection map based on preset optimization rules to obtain a final detection map.

[0104] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] On the other hand, this disclosure also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the methods for detecting power lines provided by the above methods. The method includes: acquiring a target blurred image and a target event pulse vector corresponding to the target blurred image; wherein the target blurred image is a motion-blurred image of a power line to be detected within a target area acquired by a general-purpose camera within a preset time period, and the target event pulse vector is an event pulse vector determined based on the event flow of the power line to be detected within the target area acquired by an event camera within the preset time period. The target blurred image and the target event pulse vector corresponding to the target blurred image are input into a preset power line detection model. The preset power line detection model outputs a preliminary detection map representing the power line detection result. The preset power line detection model is trained based on at least one set of training data. Each set of training data includes sample input data and sample ground truth. The sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image. The sample ground truth includes annotation data matching the blurred sample image. The annotation data is used to mark real power line segments. Based on preset optimization rules, the preliminary detection map is structurally optimized to obtain the final detection map.

[0106] Furthermore, this disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods for detecting power lines provided by the above methods. The method includes: acquiring a target blurred image and a target event pulse vector corresponding to the target blurred image; wherein the target blurred image is a motion-blurred image of a power line to be detected within a target area, acquired by a general-purpose camera within a preset time period, and the target event pulse vector is an event pulse vector determined based on an event stream of the power line to be detected within the target area acquired by an event camera within the preset time period; and converting the target blurred image and the target event pulse vector into a target event pulse vector. The target event pulse vector corresponding to the blurred target image is input into a preset power line detection model, which outputs a preliminary detection map representing the power line detection result. The preset power line detection model is trained based on at least one set of training data. Each set of training data includes sample input data and sample ground truth. The sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image. The sample ground truth includes annotation data matching the blurred sample image, which is used to identify real power line segments. Based on preset optimization rules, the preliminary detection map is structurally optimized to obtain the final detection map.

[0107] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A method for detecting power lines, characterized in that, include: Obtain the target blurred image and the target event pulse vector corresponding to the target blurred image; Wherein, the target blurred image is a motion blurred image of the electric field lines to be detected in the target area, acquired by a general-purpose camera within a preset time period, and the target event pulse vector is an event pulse vector determined based on the event flow of the electric field lines to be detected in the target area, acquired by an event camera within the preset time period. The target blurred image and the target event pulse vector corresponding to the target blurred image are input into a preset electric line detection model, and the preset electric line detection model outputs a preliminary detection map representing the electric line detection result. The preset power line detection model is trained based on at least one set of training data. Each set of training data includes sample input data and sample ground truth. The sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image. The sample ground truth includes annotation data that matches the blurred sample image. The annotation data is used to mark real power line segments. Based on preset optimization rules, the preliminary detection image is structurally optimized to obtain the final detection image; The preset power line detection model includes a cascaded feature fusion network and an edge detection network; The feature fusion network includes a cascaded feature extraction subnetwork, a channel splicing unit, and a channel attention fusion subnetwork; The feature extraction subnetwork includes a parallel image feature extraction unit VGG16_C and an event feature extraction unit VGG16_10C. The output of the image feature extraction unit VGG16_C is connected to the input of the channel stitching unit, and the output of the event feature extraction unit VGG16_10C is connected to the input of the channel stitching unit. The event feature extraction unit VGG16_C has 3 receiving channels, and the event feature extraction unit VGG16_10C has 10 receiving channels. The channel attention fusion subnetwork includes cascaded channel splitting units, attention units, and fusion units; The attention unit includes a first attention subunit and a second attention subunit connected in parallel. The first output terminal of the channel splitting unit is connected to the input terminal of the first attention subunit, and the second output terminal of the channel splitting unit is connected to the input terminal of the second attention subunit. The output of the first attention subunit is connected to the input of the fusion unit, and the output of the second attention subunit is connected to the input of the fusion unit. The output of the fusion unit is connected to the input of the edge detection network; The edge detection network includes a first detection module, a second detection module, a first upsampling layer, a third detection module, and a second upsampling layer, which are cascaded in sequence. The output of the second detection module is also connected to the input of the first detection module, and the output of the third detection module is also connected to the input of the second detection module. The step of optimizing the structure of the preliminary detection image based on preset optimization rules to obtain the final detection image includes: The preliminary detection image is binarized, and its connected regions are calculated. The direction of each connected region is approximated by the major axis, and the length of the connected region is estimated by the diagonal of the bounding box. Each connected region is used as a candidate segment, and the longest segment is assumed to represent a power line. Noise segments are filtered out based on the area and direction of each candidate segment to obtain a clear and structured power line sketch as the final detection image. If the area of ​​a candidate segment is less than a preset proportion of the area of ​​the longest segment or the deviation of its direction from the longest segment is greater than a preset angle threshold, it is considered a noise segment.

2. The method according to claim 1, characterized in that, The first attention subunit has the same structure as the second attention subunit; The first attention subunit includes a cascaded input layer, a pooling module, a connection layer, a convolutional layer, a sigmoid activation function layer, a multiplication module, a summation module, and an output layer; The pooling module includes a parallel average pooling layer and a max pooling layer; the output of the input layer is connected to the input of the average pooling layer and the max pooling layer, respectively, and the output of the average pooling layer and the max pooling layer is connected to the input of the connection layer, respectively. The output of the input layer is connected in a jump connection to the input of the multiplication module and the input of the summation module.

3. The method according to any one of claims 1-2, characterized in that, The training steps of the preset power line detection model include: Obtain the at least one set of training data; wherein the sample input data in the training data includes positive sample input data and negative sample input data; The sample input data from each group of training data is input into the power line detection model to be trained, and the power line detection model to be trained outputs the corresponding power line detection estimation result. Using a preset comprehensive loss function, the function value of the comprehensive loss function is obtained based on the true values ​​of the samples in each group of training data and the corresponding power line detection estimation results; Based on the function value of the comprehensive loss function, the power line detection model to be trained is trained until the preset training completion conditions are met, and the preset power line detection model is obtained.

4. The method according to claim 3, characterized in that, The step of using a preset comprehensive loss function to obtain the function value of the comprehensive loss function based on the true values ​​of the samples in each group of training data and the corresponding power line detection estimation results includes: Based on the true values ​​of the samples in each set of training data and the corresponding power line detection estimation results, the function values ​​of the loss function of the first detection module, the second detection module, the third detection module, and the fusion loss function are calculated respectively. Using preset weights, the function values ​​of the loss functions of the first detection module, the second detection module, the third detection module, and the fusion loss function are weighted and fused to obtain the function value of the comprehensive loss function.

5. The method according to claim 4, characterized in that, The step of calculating the function value of the loss function of the first detection module based on the true values ​​of the samples in each set of training data and the corresponding power line detection estimation results includes: The function value of the loss function of the first detection module is calculated using the following formula; in, denoted as the binary cross-entropy function; s1 represents the positive probability of the electric field detection estimate output by the first detection module for each set of sample input data; s11 represents the negative probability of the electric field detection estimate output by the first detection module for each set of sample input data; y represents the true value of the sample corresponding to each set of sample input data. The step of calculating the function value of the loss function of the second detection module based on the true values ​​of the samples in each set of training data and the corresponding power line detection estimation results includes: The function value of the loss function of the second detection module is calculated using the following formula; Where s2 represents the positive probability of the electric line detection estimate output by the second detection module for each set of sample input data; s21 represents the negative probability of the electric line detection estimate output by the second detection module for each set of sample input data; and y represents the true value of the sample corresponding to each set of sample input data. The calculation of the loss function value of the third detection module based on the true values ​​of the samples in each set of training data and the corresponding power line detection estimation results includes: The function value of the loss function of the third detection module is calculated using the following formula; Where s3 represents the positive probability of the electric line detection estimate output by the third detection module for each set of sample input data; s31 represents the negative probability of the electric line detection estimate output by the third detection module for each set of sample input data; and y represents the true value of the sample corresponding to each set of sample input data. The step of calculating the value of the fusion loss function based on the true values ​​of the samples in each set of training data and the corresponding power line detection estimation results includes: The function value of the fusion loss function is calculated using the following formula; Where, fuse represents the fusion probability of the power line detection estimate output by the power line detection model to be trained for each set of sample input data; y represents the true value of the sample corresponding to each set of sample input data; The step of using preset weights to perform weighted fusion of the function values ​​of the loss functions of the first detection module, the second detection module, the third detection module, and the fusion loss function includes: The function value of the comprehensive loss function is calculated using the following formula; in, This represents the weights of the loss function of the first detection module; The weights represent the weights of the loss function of the second detection module; The weights represent the weights of the loss function of the third detection module; This represents the weights of the fusion loss function.

6. The method according to claim 3, characterized in that, Based on the function value of the comprehensive loss function, the power line detection model to be trained is trained until the preset training completion conditions are met, including: The value of the comprehensive loss function is then passed back to the power line detection model to be trained. The power line detection model to be trained adjusts the network parameters of each network layer according to the function value of the comprehensive loss function; The process iteratively executes the steps from obtaining the function value of the comprehensive loss function to adjusting the network parameters of each network layer until the preset training completion conditions are met.

7. A device for detecting power lines, characterized in that, include: The data acquisition module is configured to acquire a target blurred image and a target event pulse vector corresponding to the target blurred image; Wherein, the target blurred image is a motion blurred image of the electric field lines to be detected in the target area, acquired by a general-purpose camera within a preset time period, and the target event pulse vector is an event pulse vector determined based on the event flow of the electric field lines to be detected in the target area, acquired by an event camera within the preset time period. The detection execution module is configured to: input the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset electric line detection model, and output a preliminary detection map representing the electric line detection result through the preset electric line detection model; The preset power line detection model is trained based on at least one set of training data. Each set of training data includes sample input data and sample ground truth. The sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image. The sample ground truth includes annotation data that matches the blurred sample image. The annotation data is used to mark real power line segments. The result optimization module is configured to: perform structural optimization on the preliminary detection image based on preset optimization rules to obtain the final detection image; The preset power line detection model includes a cascaded feature fusion network and an edge detection network. The feature fusion network includes a cascaded feature extraction subnetwork, a channel splicing unit, and a channel attention fusion subnetwork; The feature extraction subnetwork includes a parallel image feature extraction unit VGG16_C and an event feature extraction unit VGG16_10C. The output of the image feature extraction unit VGG16_C is connected to the input of the channel stitching unit, and the output of the event feature extraction unit VGG16_10C is connected to the input of the channel stitching unit. The event feature extraction unit VGG16_C has 3 receiving channels, and the event feature extraction unit VGG16_10C has 10 receiving channels. The channel attention fusion subnetwork includes cascaded channel splitting units, attention units, and fusion units; The attention unit includes a first attention subunit and a second attention subunit connected in parallel. The first output terminal of the channel splitting unit is connected to the input terminal of the first attention subunit, and the second output terminal of the channel splitting unit is connected to the input terminal of the second attention subunit. The output of the first attention subunit is connected to the input of the fusion unit, and the output of the second attention subunit is connected to the input of the fusion unit. The output of the fusion unit is connected to the input of the edge detection network; The edge detection network includes a first detection module, a second detection module, a first upsampling layer, a third detection module, and a second upsampling layer, which are cascaded in sequence. The output of the second detection module is also connected to the input of the first detection module, and the output of the third detection module is also connected to the input of the second detection module. In the result optimization module, based on preset optimization rules, the preliminary detection image is structurally optimized to obtain the final detection image, including: The preliminary detection image is binarized, and its connected regions are calculated. The direction of each connected region is approximated by the major axis, and the length of the connected region is estimated by the diagonal of the bounding box. Each connected region is used as a candidate segment, and the longest segment is assumed to represent a power line. Noise segments are filtered out based on the area and direction of each candidate segment to obtain a clear and structured power line sketch as the final detection image. If the area of ​​a candidate segment is less than a preset proportion of the area of ​​the longest segment or the deviation of its direction from the longest segment is greater than a preset angle threshold, it is considered a noise segment.

Images