Method and device for detecting power line

By combining the data of the P-light camera and event camera, using the feature fusion network and edge detection network, the problems of blurring and low accuracy under complex scenes in drone power line detection are solved, and clear detection and high-precision recognition of the edge of the power line are achieved.

CN118941509AActive Publication Date: 2024-11-12WUHAN UNIV

Patent Information

Application Number
CN202410941691.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-12
Estimated Expiration
2044-07-15

AI Technical Summary

Technical Problem

The existing drone power line detection technology has problems such as blurring of the edges of power lines, low detection accuracy and accuracy in complex scenarios. The frame rate and dynamic range of the general camera are limited, and the infrared camera temperature measurement accuracy is affected by the environment and cannot penetrate obstacles. The object is tested.

Method used

Combining the characteristics of the P-light camera and event camera, the image and event features are adaptively weighted and modal internal and external interactions of deep multi-scale features are input to the edge detection network to obtain clear power line edges, and noise filtering through optimization rules , get the final detection map.

Benefits of technology

In complex detection scenarios, the edges of the power line are clear and the detection accuracy and accuracy are high, effectively solving the problem of insufficient detection accuracy and accuracy in the prior art.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941509A_ABST
    Figure CN118941509A_ABST
Patent Text Reader

Abstract

The invention provides a method and device for detecting a power line, and the method comprises the steps: obtaining a target blurred image and a target event pulse vector corresponding to the target blurred image; wherein in the preset time, the target blurred image is a motion blurred image, which is acquired based on a common light camera and is related to the to-be-detected power line in the target area, and the target event pulse vector is an event pulse vector which is determined based on an event flow, which is acquired based on an event camera and is related to the to-be-detected power line in the target area; inputting the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset power line detection model, and outputting a preliminary detection image representing a power line detection result through the preset power line detection model; and based on a preset optimization rule, carrying out structure optimization on the preliminary detection graph to obtain a final detection graph. By using the method disclosed by the invention, in a complex detection scene, the power line edge in the obtained final detection image is clear, and the detection precision and accuracy are relatively high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, electronic device, non-transitory computer-readable storage medium, and computer program product for detecting power lines. Background Art

[0002] Power transmission plays a vital role in all walks of life in modern society. Regular inspections of transmission lines can timely discover faults in the lines and hidden dangers that endanger the safety of the lines, which helps the power department to carry out targeted maintenance and upkeep on the transmission lines, thereby ensuring the safety of power transmission lines and extending their service life.

[0003] At present, power line detection mainly relies on manual inspection and survey of the lines, but this model has obvious defects. Manual line inspectors need to work at high altitudes and in complex environments, which poses certain safety risks. At the same time, the detection range is limited and it is impossible to conduct comprehensive inspections of power lines over a large area. To solve this problem, technicians in related fields have begun to use drones to perform power line detection tasks; specifically, drones are equipped with optical cameras or infrared cameras to collect images of power lines in the area to be detected, and then the power line images are analyzed to determine the actual status of the power lines.

[0004] However, the existing technology for "drone power line detection tasks" at least has problems such as blurred power line edges and low detection precision and accuracy in the detection results when faced with complex detection scenarios. Summary of the invention

[0005] The present disclosure provides a method, an apparatus, an electronic device, a non-transitory computer-readable storage medium, and a computer program product for detecting a power line, so as to solve the defects in the prior art.

[0006] The present disclosure provides a method for detecting a power line, comprising: Acquire a target blurred image and a target event pulse vector corresponding to the target blurred image; The target blurred image is a motion blurred image of the power lines to be detected in the target area acquired by the general light camera within a preset time, and the target event pulse vector is an event pulse vector determined based on the event stream of the power lines to be detected in the target area acquired by the event camera within the preset time; Inputting the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset power line detection model, and outputting a preliminary detection image representing the power line detection result through the preset power line detection model; The preset power line detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes sample input data and sample true values, the sample input data includes a fuzzy sample image and an event stream sample vector corresponding to the fuzzy sample image, the sample true value includes annotation data matching the fuzzy sample image, and the annotation data is used to mark the real power line segment; Based on preset optimization rules, the preliminary detection image is structurally optimized to obtain a final detection image.

[0007] The present disclosure also provides a device for detecting a power line, comprising: A data acquisition module is configured to: acquire a target blurred image and a target event pulse vector corresponding to the target blurred image; The target blurred image is a motion blurred image of the power lines to be detected in the target area acquired by the general light camera within a preset time, and the target event pulse vector is an event pulse vector determined based on the event stream of the power lines to be detected in the target area acquired by the event camera within the preset time; A detection execution module is configured to: input the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset power line detection model, and output a preliminary detection image representing the power line detection result through the preset power line detection model; The preset power line detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes sample input data and sample true values, the sample input data includes a fuzzy sample image and an event stream sample vector corresponding to the fuzzy sample image, the sample true value includes annotation data matching the fuzzy sample image, and the annotation data is used to mark the real power line segment; The result optimization module is configured to: perform structural optimization on the preliminary detection image based on preset optimization rules to obtain a final detection image.

[0008] The present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for detecting power lines as described in any one of the above is implemented.

[0009] The present disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for detecting a power line as described in any one of the above is implemented.

[0010] The present disclosure further provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above methods for detecting a power line.

[0011] As described above, the method for detecting power lines provided by the embodiment of the present disclosure introduces event camera data to provide auxiliary information to solve the problems of the prior art; specifically, first, the pre-processing module in the feature fusion network is used to adaptively weight the image and event features based on channel attention, and then the fusion module in the feature fusion network is used to enhance the deep multi-scale features within the modality and interact between the modalities based on self-attention to obtain the features of event and image fusion; then the feature map is input into the edge detection network (i.e., the network FE-BDCN improved based on the boundary detection convolutional network) to obtain a clearer power line edge, i.e., the preliminary detection map; finally, the preset optimization rules can be used to optimize the preliminary detection map to obtain the final detection map.

[0012] Furthermore, through verification of the verification embodiment to be described below, using the method for detecting power lines (fusion of event camera and general light camera) provided by the embodiment of the present disclosure, when faced with complex detection scenes, the edges of the power lines in the final detection image obtained are clear, and the detection precision and accuracy are high, thereby demonstrating the effectiveness of the method of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the present disclosure or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0014] Figure 1 It is one of the flowcharts of the method for detecting a power line provided by an embodiment of the present disclosure; Figure 2 is a structural schematic diagram of the preset power line detection model provided in an embodiment of the present disclosure; Figure 3 This is a second flow chart of a method for detecting a power line provided by an embodiment of the present disclosure; Figure 4 is a schematic diagram of a process for obtaining the at least one set of training data provided by an embodiment of the present disclosure; Figure 5 is a schematic diagram comparing power line detection results obtained based on the same test data by the preset power line detection model provided in an embodiment of the present disclosure and other multiple detection models; Figure 6is a structural schematic diagram of a device for detecting power lines provided by an embodiment of the present disclosure; Figure 7 It is a schematic diagram of the structure of an electronic device provided by the present disclosure. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the technical solutions in the present disclosure will be clearly and completely described below in conjunction with the drawings in the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0016] Summary of the invention concept: After research, the inventors of the present disclosure have found that the reasons for the problems of the prior art are: although drones have efficient and comprehensive detection capabilities and can fly in the air to cover a wide range of power lines, the frame rate and dynamic range of the general light camera they carry are limited. When faced with challenging scenes such as low light, fast motion, and background clutter, the general light camera is difficult to work normally due to problems such as overexposure, underexposure, and motion blur. In the case of drones equipped with infrared cameras, although the infrared camera can detect the temperature distribution of power equipment, its temperature measurement accuracy is affected by the ambient temperature and atmosphere, which leads to certain limitations in the temperature measurement accuracy. At the same time, ordinary optical cameras and infrared cameras cannot penetrate obstacles for detection, which can lead to the omission of hidden defects in power lines in complex scenes.

[0017] Based on the above reasons, the inventors of the present disclosure have further studied and found that event cameras have high temporal resolution and high dynamic range, and therefore have good resistance to motion blur; in addition, event cameras have low energy consumption and low data transmission requirements, and are therefore very suitable for power line inspection drones that are limited in size and need to work for a long time. However, the inventors also found that if power line detection is performed only based on event cameras, the solution has limitations or problems in terms of resolution, color information, and poor sensitivity to static scenes; and in scenes where ambient light changes frequently, event cameras may generate a large number of "noise" events, thereby affecting the clarity and availability of visual data. That is, although event cameras can capture information with extremely high temporal resolution (microseconds), which is very suitable for monitoring fast-moving objects, they cannot capture color information and static image information.

[0018] In view of this, the inventors of the present disclosure have comprehensively analyzed the advantages and disadvantages of general light cameras and event cameras, and considered that the complementary characteristics of general light cameras and event cameras can be utilized to fuse the images obtained by general light cameras with the event streams obtained by event cameras, in order to achieve accurate and clear power line detection. Based on this inventive concept, a scheme for detecting power lines as described in the following embodiment is proposed.

[0019] Example: The scheme for detecting power lines disclosed in the present invention is described below in conjunction with the accompanying drawings.

[0020] Figure 1 1 is a flow chart of a method for detecting a power line provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices (for example, servers or cloud computing platforms), such as Figure 1 As shown, the method for detecting a power line includes the following steps: S110, acquiring a target blurred image and a target event pulse vector corresponding to the target blurred image.

[0021] The target blurred image is a motion blurred image of the power lines to be detected in the target area acquired by the general light camera within a preset time. The target event pulse vector is an event pulse vector determined based on the event stream of the power lines to be detected in the target area acquired by the event camera within the preset time.

[0022] The “target area” is an area where power line detection is required, and there are power lines to be detected in the area.

[0023] The present disclosure does not limit the acquisition method. For example, an electronic device (e.g., a server or a cloud computing platform) as an execution subject may communicate with a server storing the “target blurred image, target event pulse vector” to acquire the “target blurred image, target event pulse vector”.

[0024] S120, inputting the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset power line detection model, and outputting a preliminary detection image representing the power line detection result through the preset power line detection model.

[0025] Wherein, the preset power line detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes sample input data and sample true value, the sample input data includes a fuzzy sample image and an event stream sample vector corresponding to the fuzzy sample image, the sample true value includes annotation data matching the fuzzy sample image, and the annotation data is used to mark the real power line segment.

[0026] Optionally, the marked data may be represented by a label, and the value may be 0 or 1, wherein 0 indicates that the marked data is not a power line, and 1 indicates that the marked data is a power line.

[0027] Optionally, the preset power line detection model may include a cascaded feature fusion network and an edge detection network. Specifically, the feature fusion network may include a VGG16 network and a channel attention network; the edge detection network may include an improved network based on a boundary detection convolutional network (BDCN), which may be recorded as FE-BDCN.

[0028] It should be noted that the details of the preset power line detection model will be described below and will not be repeated here.

[0029] S130: Based on preset optimization rules, structural optimization is performed on the preliminary detection image to obtain a final detection image.

[0030] Since noisy interference may exist in the preliminary detection image output by the preset power line detection model, further optimization can effectively filter out the noise in the preliminary detection image, thereby finally obtaining a clear and structured power line sketch as the final detection image.

[0031] As described above, the method for detecting power lines provided by the embodiment of the present disclosure introduces event camera data to provide auxiliary information to solve the problems of the prior art; specifically, first, the pre-processing module in the feature fusion network is used to adaptively weight the image and event features based on channel attention, and then the fusion module in the feature fusion network is used to enhance the deep multi-scale features within the modality and interact between the modalities based on self-attention to obtain the features of event and image fusion; then the feature map is input into the edge detection network (i.e., the network FE-BDCN improved based on the boundary detection convolutional network) to obtain a clearer power line edge, i.e., the preliminary detection map; finally, the preset optimization rules can be used to optimize the preliminary detection map to obtain the final detection map.

[0032] Furthermore, through verification of the verification embodiment to be described below, using the method for detecting power lines (fusion of event camera and general light camera) provided by the embodiment of the present disclosure, when faced with complex detection scenes, the edges of the power lines in the final detection image obtained are clear, and the detection precision and accuracy are high, thereby demonstrating the effectiveness of the method of the present disclosure.

[0033] exist Figure 1 Based on the embodiment, as an optional implementation, the preset power line detection model may include a cascaded feature fusion network and an edge detection network.

[0034] Figure 1 Based on the embodiment and the above implementation manner, as an optional implementation manner, the feature fusion network includes a cascaded feature extraction subnetwork, a channel splicing unit and a channel attention fusion subnetwork.

[0035] As an alternative example, refer to Figure 2 The feature extraction subnetwork includes a parallel image feature extraction unit VGG16_C (for side output of an image feature sequence List_rgb) and an event feature extraction unit VGG16_10C (for side output of a time feature sequence List_eve), the output end of the image feature extraction unit VGG16_C is connected to the input end of the channel splicing unit (for "splicing by channel"), and the output end of the event feature extraction unit VGG16_10C is connected to the input end of the channel splicing unit.

[0036] As an alternative example, refer to Figure 2 , the channel attention fusion sub-network ( Figure 2 The "CAFusionModule" shown in the figure includes cascaded channel splitting units (i.e. Figure 2 The unit where X is located), the attention unit, and the fusion unit (i.e., Figure 2 The attention unit includes a first attention sub-unit connected in parallel (i.e., Figure 2 The unit corresponding to the "attention" connected to the X1 channel) and the second attention sub-unit, that is, Figure 2 The unit corresponding to the "attention" connected to the X2 channel is shown); the first output end of the channel splitting unit (ie, Figure 2 The X1 channel shown in FIG. 1 is connected to the input end of the first attention sub-unit, and the second output end of the channel splitting unit (ie, Figure 2 The X2 channel shown in the figure is connected to the input end of the second attention sub-unit; the output end of the first attention sub-unit is connected to the input end of the fusion unit, and the output end of the second attention sub-unit is connected to the input end of the fusion unit. The output end of the fusion unit is connected to the input end of the edge detection network.

[0037] Figure 1 Based on the embodiment and the above implementation manner, as an optional example, the image feature extraction unit VGG16_C includes a cascaded first convolution module, a second convolution module and three third convolution modules.

[0038] Among them, the first convolution module includes two cascaded convolution layers and one pooling layer, the convolution kernel parameters of the convolution layer are 3×3, the number of channels is 64, and the step size is 1, and the convolution kernel parameters of the pooling layer are 2×2 and the step size is 2; the number of receiving channels of the first convolution layer in the first convolution module is 3; the second convolution module includes two cascaded convolution layers and one pooling layer, the convolution kernel parameters of the convolution layer are 3×3, the number of channels is 128, and the convolution kernel parameters of the pooling layer are 2×2 and the step size is 2; the third convolution module includes a cascaded first convolution layer, a second convolution layer, a third convolution layer and a pooling layer, the convolution kernel parameters of the first convolution layer are 3×3, the number of channels is 256, the convolution kernel parameters of the second convolution layer are 3×3, the number of channels is 512, the convolution kernel parameters of the third convolution layer are 3×3, the number of channels is 512, and the convolution kernel parameters of the pooling layer are 2×2 and the step size is 2.

[0039] The structure of the event feature extraction unit VGG16_10C is the same as that of the image feature extraction unit VGG16_C; wherein, the number of receiving channels of the first convolution layer in the first convolution module of the event feature extraction unit VGG16_10C is 10.

[0040] Optionally, refer to Figure 2 The size of the target blurred image input to the image feature extraction unit VGG16_C is 512×512×3; the size of the target event pulse vector input to the event feature extraction unit VGG16_10C is 512×512×10.

[0041] Since the structure of the event feature extraction unit VGG16_10C is the same as that of the image feature extraction unit VGG16_C, VGG16_10C is taken as an example to illustrate the processing process of the input target blur map as follows: First, the target blur map passes through the first convolution module for downsampling to reduce the spatial size of the image features; secondly, these features are input to the second convolution module to further refine the image features; thirdly, the image features pass through three third convolution modules to continue downsampling and enhance the depth and complexity of the image feature representation. Finally, after this series of convolution and pooling operations, the obtained feature map will be used for downstream tasks such as classification or detection to provide rich visual information content.

[0042] Figure 1 Based on the embodiment and the above-mentioned implementation manner, as an optional example, the first attention sub-unit and the second attention sub-unit have the same structure.

[0043] Reference Figure 2The first attention sub-unit includes a cascaded input layer (for receiving and processing the "input feature F"), a pooling module, a connection layer, a convolution layer, an activation function Sigmoid layer, a multiplication module, a summation module and an output layer.

[0044] Among them, the pooling module includes an average pooling layer and a maximum pooling layer in parallel; the output end of the input layer is connected to the input ends of the average pooling layer and the maximum pooling layer respectively, and the output ends of the average pooling layer and the maximum pooling layer are connected to the input end of the connection layer respectively.

[0045] The output end of the input layer is jump-connected to the input end of the multiplication module and the input end of the summation module respectively.

[0046] Next, the method of optimizing the weights of the "target event pulse vector and target blurred image" input into the feature fusion network is described as follows.

[0047] After the event feature extraction unit VGG16_10C extracts deep features from the input target event pulse vector, and the image feature extraction unit VGG16_C extracts deep features from the input target blurred image; these features are sent to the channel attention fusion sub-network to further enhance its expressiveness. The channel attention fusion sub-network, specifically, the first attention sub-unit and the second attention sub-unit, first evaluates the global features of each channel and captures the statistical information of the channel by calculating the average and maximum values ​​of each channel. This information is then sent to two independent fully connected layers, where the first fully connected layer is used to reduce the dimension to reduce the number of parameters and computational complexity, and the second fully connected layer is used to restore the original channel dimension. In this process, the ReLU activation function is used to intervene to increase nonlinearity to extract richer feature representations. The obtained channel weights are passed through a Sigmoid activation function to normalize the weights to between 0 and 1 so that they can be directly applied to the original features as scaling factors. Finally, these weighted features are combined with the original features through element-wise multiplication to complete the recalibration of the features. This process not only highlights more informative features but also suppresses less important features, thereby improving the model's sensitivity to key visual information and the final recognition performance.

[0048] Optionally, the entire shallow fusion process can be described by the following calculation formulas (1) to (4): (1) (2) (3) (4) Among them, CA F represents the first attention subunit; CA E represents the second attention subunit; represents element-by-element multiplication; XF represents the characteristics of the target blurred image; XE: represents the characteristics of the target event pulse vector; Y F Y represents the output feature of the target blurred image after being processed by the first attention sub-unit and combined with the target event impulse vector feature; E Attn represents the output feature of the event pulse vector after being processed by the second attention subunit and combined with the target blurred image feature; F Attn represents the attention weight of the target blurred image feature after being processed by the first attention subunit; E Represents the attention weight of the target event pulse vector feature after being processed by the first attention sub-unit.

[0049] Continue to refer to Figure 2 , the following description is made about the method of dynamically adjusting the feature fusion of the event branch and the image branch in the channel attention fusion subnetwork.

[0050] The channel attention mechanism is used to enhance and fuse features from different data sources. The channel attention fusion sub-network first initializes two attention sub-modules (i.e., the first attention sub-unit and the second attention sub-unit), which are applied to different feature streams respectively. During the forward propagation process, the input feature vector X is first divided into two parts, namely X1 and X2. This division is performed along the channel dimension (i.e., split by channel). Each part of the feature passes through its corresponding channel attention sub-module, which independently adjusts its respective features, strengthens key information and suppresses less important information by learning the importance of each channel. The calculation process of the attention sub-module AFF can be expressed as follows: (5) (6) (7) (8) in, represents the first attention subunit; represents the second attention subunit; Represents the input feature vector during the forward propagation process; express After segmentation, the corresponding feature vector of the first attention sub-unit is input; express After segmentation, the corresponding feature vector is input into the second attention sub-unit; express right The processed attention weights; express right The processed attention weights.

[0051] Optionally, first use The function equally divides the input feature vector Split into two sub-vectors and ; Then, channel attention is applied, and the final plus sign indicates that the two processed feature vectors are added element-wise to obtain the output. The fusion unit (i.e., Figure 2 The “fusion” unit shown in the figure uses channel splicing and 1×1 convolution to fuse the input image features and event features into one feature, and then sends it to the edge detection network.

[0052] Figure 1 Based on the embodiment and the above implementation mode, as an optional example, continue to refer to Figure 2 The edge detection network includes a first detection module (ID Block1), a second detection module (ID Block2), a first upsampling layer (ie, Figure 2 As shown, "up" connected to ID Block2), the third detection module (ID Block3) and the second upsampling layer (ie, Figure 2 As shown, "up" connected to ID Block3). The output end of the second detection module is also connected to the input end of the first detection module, and the output end of the third detection module is also connected to the input end of the second detection module.

[0053] Optionally, the first detection module includes three first detection submodules connected in parallel; wherein the first detection submodule (ID Block1) includes a cascaded first detection convolution layer ( Figure 2 "3×3-32conv"), the first activation function ReLU layer, the first detection convolution block ( Figure 2 "SEM") and channel reduction convolutional layers ( Figure 2 The first detection convolution layers in adjacent first detection submodules are cascaded; the output ends of the first detection convolution blocks in three first detection submodules are connected to each other; the first detection convolution block includes a plurality of cascaded convolution layers with different expansion rates; the convolution kernel parameters of the first detection convolution layer are 3×3 and the number of channels is 32, and the convolution kernel parameters of the channel reduction convolution layer are 1×1 and the number of channels is 1.

[0054] Optionally, the second detection module includes three second detection sub-modules in parallel; wherein, the second detection sub-module includes a cascaded second detection convolution layer, a second activation function ReLU layer, a second detection convolution block and a channel reduction convolution layer; the second detection convolution layers in adjacent second detection sub-modules are cascaded; the output ends of the second detection convolution blocks in the three second detection sub-modules are connected to each other; the second detection convolution block includes multiple cascaded convolution layers with different expansion rates; the number of channels of the second detection convolution layer is 64, and the convolution kernel parameters of the channel reduction convolution layer are 1×1 and the number of channels is 1.

[0055] Optionally, the third detection module includes three third detection sub-modules in parallel; wherein the third detection sub-module includes a cascaded third detection convolution layer, a third activation function ReLU layer, a third detection convolution block and a channel reduction convolution layer; the third detection convolution layers in adjacent third detection sub-modules are cascaded; the output ends of the third detection convolution blocks in the three third detection sub-modules are connected to each other; the third detection convolution block includes multiple cascaded convolution layers with different expansion rates; the number of channels of the third detection convolution layer is 128, and the convolution kernel parameters of the channel reduction convolution layer are 1×1 and the number of channels is 1.

[0056] It should be noted that, in the above-mentioned first detection module (ID Block1), the second detection module (ID Block2) and the third detection module (ID Block3), the channel reduction convolution layer involved is mainly used to reduce the number of output channels to 21. The first upsampling layer (i.e., Figure 2 As shown, the “up” connected to ID Block2 is used to increase the size of the feature map by two times; the second upsampling layer (i.e., Figure 2 As shown, “up” connected with ID Block3 is used to increase the size of the feature map by four times.

[0057] It should be noted that the first detection convolution block ( Figure 2 The structures of the second detection convolution block and the third detection convolution block can remain the same, i.e., (MSBLock class); however, their network level parameters are different. The different parameters are used to pass different input channel numbers (c_in), number of convolution layers (k), and expansion rates (rate), thereby achieving different feature extraction functions.

[0058] As an alternative example, refer to Figure 2 , the parameters of “SEM” can be described as follows: r represents the expansion rate, r0 represents the basic expansion rate, and K represents a coefficient used to control the multiple of the expansion rate. r=r0 represents the expansion rate of the first convolutional layer; represents the dilation rate of the second convolutional layer; represents the expansion rate of the third convolutional layer; and so on until .

[0059] In the above Figure 1 , Figure 2 Based on the embodiment, as an optional implementation method, refer to Figure 3 , the training steps of the preset power line detection model include: S310: Obtain the at least one set of training data.

[0060] The sample input data in the training data includes positive sample input data and negative sample input data.

[0061] As an optional example, a training data set may be constructed first, and then based on the training data set, the at least one set of training data may be acquired.

[0062] Further, refer to Figure 4 , the training data set can be constructed as follows.

[0063] 1) By using the normal light camera within the preset time T (based on the camera trajectory t0,...,t k ,...,t n ) captured multiple clear images (i.e., Figure 4 The “simulation time series” shown in the figure is weighted averaged to obtain a blurred sample image with motion blur. (Right now, Figure 4 Optionally, blur the sample image It is scaled to 512×512×3 size.

[0064] 2) Select the last clear image within the preset time T (for example, Figure 4 The corresponding "line segment annotation" image below the "original image" shown in the figure is obtained, and it is annotated to accurately mark the real line segments (that is, the corresponding "line segment annotation" image below the blurred image is obtained). These annotations will be used in the subsequent training process.

[0065] Optionally, the marked data may be represented by a label, and the value may be 0 or 1, wherein 0 indicates that the marked data is not a power line, and 1 indicates that the marked data is a power line.

[0066] It should be noted that the annotation data can be stored in a .mat file format. The special variables of the metadata of the .mat file format include not only header, version, and globals, but also a groundTruth variable, which contains a field called Boundaries, which can store the actual annotation data.

[0067] 3) As for the event stream time within the preset time T, we divide the event stream data collected by the event camera during this period into five equal parts and perform linear interpolation to generate five frames The number 2 indicates the positive or negative polarity of the event.

[0068] The linear interpolation calculation formulas (9) to (11) are as follows: (9) (10) (11) in, Indicates the earliest timestamp of the event stream; represents the duration of the event stream; H represents the height of the event frame; W represents the height of the event frame; x represents the horizontal coordinate of the pixel in the event frame; y represents the vertical coordinate of the pixel in the event frame; e k represents the kth event in the event stream; x k Indicates event e k The horizontal coordinate position in the event frame; y k Indicates event e k The vertical coordinate position in the event frame; B represents the size of the time dimension; Indicates event e k a sudden change in position in the event stream, Indicates event e k The weight of event e k The contribution to the interpolation result.

[0069] The polarity indicates the direction of change of light intensity captured by the event camera, where an increase in light intensity is positive polarity and a decrease in light intensity is negative polarity.

[0070] It should be noted that in order to adapt to the three-channel input of the event feature extraction unit VGG16_10C, the polarity dimension is compressed into the time segment dimension to obtain The vector is then scaled to get a 512×512×10 event input.

[0071] As an optional example, the “compressing the polarity dimension into the time segment dimension” may be performed in the following manner: First, event polarity mapping is performed. Specifically, the polarity information of the event is first mapped to a time segment. Taking the original event vector of H×W×B×2 as an example, the polarity dimension is compressed to the time segment dimension to make it a vector of H×W×B', where B' represents the new number of time segments. Secondly, polarity merging is performed, that is, the positive and negative polarity events in the same time segment are merged together. Specifically, positive polarity events and negative polarity events can be superimposed or weighted averaged. Finally, vector transformation is performed. Specifically, after polarity merging, the size is adjusted as needed. We adjust the number of time segments of B to 10 to adapt to the input of the event feature extraction unit VGG16_10C.

[0072] In summary, by processing the sample images or data collected by the general light camera and the event camera through the above steps 1) to 3), a pair of blurred sample images can be obtained. , the corresponding event frames represented by event stream sample vectors, and the corresponding labeled data, so as to construct a training data set.

[0073] S320, respectively inputting the sample input data in each group of training data into the power line detection model to be trained, and the power line detection model to be trained outputs the corresponding power line detection estimation result.

[0074] S330. Using a preset comprehensive loss function, based on the true value of the samples in each set of training data and the corresponding power line detection estimation result, obtain the function value of the comprehensive loss function.

[0075] As an optional example, S330 may be implemented in the following manner: I1) Based on the sample true value in each set of training data and the corresponding power line detection estimation result, respectively calculate the function value of the first detection module loss function, the function value of the second detection module loss function, the function value of the third detection module loss function and the function value of the fusion loss function.

[0076] Optionally, the calculating the function value of the loss function of the first detection module based on the sample true value in each set of training data and the corresponding power line detection estimation result includes: Using the following calculation formula (12), calculate the function value of the loss function of the first detection module; (12) in, Represents a binary cross entropy function; s1 represents the forward probability of the power line detection estimate output by the first detection module for each set of sample input data; s11 represents the reverse probability of the power line detection estimate output by the first detection module for each set of sample input data; y represents the sample true value corresponding to each set of sample input data.

[0077] Optionally, the calculating the function value of the loss function of the second detection module based on the sample true value in each set of training data and the corresponding power line detection estimation result includes: Using the following calculation formula (13), calculate the function value of the loss function of the second detection module; (13) Among them, s2 represents the forward probability of the power line detection estimate output by the second detection module for each set of sample input data; s21 represents the reverse probability of the power line detection estimate output by the second detection module for each set of sample input data; y represents the sample true value corresponding to each set of sample input data.

[0078] Optionally, the calculating the function value of the loss function of the third detection module based on the sample true value in each set of training data and the corresponding power line detection estimation result includes: The function value of the loss function of the third detection module is calculated using the following calculation formula (14); (14) Among them, s3 represents the forward probability of the power line detection estimate output by the third detection module for each set of sample input data; s31 represents the reverse probability of the power line detection estimate output by the third detection module for each set of sample input data; y represents the sample true value corresponding to each set of sample input data.

[0079] Optionally, the calculating a function value of a fusion loss function based on the sample true value in each set of training data and the corresponding power line detection estimation result includes: The function value of the fusion loss function is calculated using the following calculation formula (15); (15) Wherein, fuse represents the fusion probability of the power line detection estimation output by the power line detection model to be trained for each group of sample input data; y represents the sample true value corresponding to each group of sample input data.

[0080] It should be noted that the BCELoss function involved in the calculation formulas (12) to (15) in the above example I1) can be adaptively calculated based on the following calculation formulas (16) to (19).

[0081] (16) (17) (18) The meanings of the symbols in (16) to (18) are only provided for the purpose of explanation and are not intended to limit the scope of the present disclosure to the aforementioned embodiments.

[0082] Where N represents the total number of samples; Indicates i The true label of each sample (0 or 1); Indicates i The predicted probability of a sample (the value after the sigmoid function); Indicates i The weight of each sample is used to balance positive and negative samples; Represents the sigmoid function; Indicates i The input of samples; When the value is 1, represents the weight of positive samples; When the value is 0, Represents the weight of negative samples; Indicates the number of negative samples; Represents the number of positive samples; Represents a hyperparameter for balancing positive and negative samples.

[0083] I2) Using preset weights, weighted fusion is performed on the function value of the loss function of the first detection module, the function value of the loss function of the second detection module, the function value of the loss function of the third detection module, and the function value of the fusion loss function to obtain the function value of the comprehensive loss function.

[0084] As an optional example, the function value of the comprehensive loss function may be calculated using the following calculation formula (5); (5) in, represents the weight of the loss function of the first detection module; represents the weight of the loss function of the second detection module; Represents the weight of the loss function of the third detection module; Represents the weight of the fusion loss function.

[0085] S340. Based on the function value of the comprehensive loss function, the power line detection model to be trained is trained until a preset training completion condition is met, and the neural network is obtained from the power line detection model to be trained.

[0086] As an optional example, S340 may be implemented in the following manner: First, the function value of the comprehensive loss function is fed back to the power line detection model to be trained. Secondly, the power line detection model to be trained adjusts the network parameters of each network layer according to the function value of the comprehensive loss function. Finally, iteratively executes the step of obtaining the function value of the comprehensive loss function to the step of adjusting the network parameters of each network layer until the preset training completion condition is met.

[0087] Among them, the preset training completion condition can be that the number of training iterations reaches a preset number threshold, or the function value of the above-mentioned comprehensive loss function is less than a preset loss threshold, that is, convergence.

[0088] Based on the above embodiment, as an optional implementation, step S130 "the structural optimization of the preliminary detection image based on the preset optimization rule to obtain the final detection image" can be implemented as follows: using the pre-configured structural prior knowledge of the power line, the edge noise of the preliminary detection image is filtered to obtain the final detection image. The structure shown includes three dimensions: length, size, and direction.

[0089] Specifically, for the preliminary detection map, binarization can be performed, and its connected areas can be calculated, the direction of each area is approximated by the major axis, and the length is estimated by the diagonal of the bounding box. It is assumed that the longest fragment represents the power line, and the noise fragments are filtered out by specific measurements of area and direction. Specifically, if the area of ​​a fragment is much smaller than the area of ​​the longest fragment or the direction is significantly different, it is considered to be noise. This method can effectively identify the power lines from the noise, and finally obtain a clear and structured power line sketch, which can be used as the final detection map.

[0090] Verification example: This verification embodiment is mainly used to evaluate the effect of the method for detecting power lines provided by the embodiment of the present disclosure.

[0091] Specifically, a given set of test data is used, wherein the set of test data includes sample input data and sample true values; further, the sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image, and the sample true value includes labeled data matching the blurred sample image.

[0092] Models used for comparison with the preset power line detection model (ie, FE-BDCN) provided in the embodiment of the present disclosure include Gestaltic Grouping, DexiNed, LSD, HED, and BDCN.

[0093] The sample input data in the given set of test data are respectively input into FE-BDCN, GestalticGrouping, DexiNed, LSD, HED and BDCN to obtain the power line detection results output by each model as shown in Figure 5 shown.

[0094] Reference Figure 5 , which contains 9 groups of data. The first line in the figure is the motion blurred RGB image (BlurredImage) obtained by the general light camera, the second line is the annotated image (Ground Truth), and the third line is the event stream (Events). Relying on the existing annotated image (Ground Truth), the power line edge detection images obtained by different methods are arranged, and the detection effects of different methods can be intuitively compared. The fourth line is the detection result based on Gestaltic Grouping based on RGB images, and the fifth to eighth lines are the results of the comparison methods based on DexiNed, LSD, HED and BDCN based on blurred images, and the ninth line is the detection result based on the improved FE-BDCN, using the method of fusion event frame and motion blurred RGB image. It can be seen that a large amount of false detection noise appears in LSD and Gestaltic Grouping. The images obtained by DexiNed, LSD, and BDCN methods have obvious missed detection and wrong detection. The power line edge detection model using the fusion event frame and RGB image obtains the cleanest and clearest power line edge map, which verifies the effectiveness of the power line detection method of the fusion event camera and general light camera proposed in the present invention.

[0095] Furthermore, based on the power line detection results and the annotation data output by the above-mentioned multiple models, each model is quantitatively evaluated, and the obtained evaluation results are shown in Table 1.

[0096]

[0097] Table 1 Quantitative evaluation results of different edge detection models on the same test data Refer to Table 1, where ER, Prec, Recon, and F1 represent the average error rate, average precision, average recall, and average F1 score of different models when the threshold is 5, respectively.

[0098] Based on the data in Table 1, the average F1 score of the fuzzy image processing based on the traditional models or methods (including LSD, HED, DexiNed, GestalticGrouping) is only 0.963 at most, while the FE-BDCN proposed in this invention can reduce the average error rate and improve the accuracy of power line edge capture based on the original BDCN. In addition, the FE-BDCN model introduces event streams based on the original deep learning based only on fuzzy RGB images, and uses the attention mechanism for feature fusion, showing excellent robustness and accuracy in the scenario of power line edge detection (extraction) under complex backgrounds.

[0099] As described above, the method for detecting power lines provided by the embodiment of the present disclosure introduces event camera data to provide auxiliary information to solve the problems of the prior art; specifically, first, the pre-processing module in the feature fusion network is used to adaptively weight the image and event features based on channel attention, and then the fusion module in the feature fusion network is used to enhance the deep multi-scale features within the modality and interact between the modalities based on self-attention to obtain the features of event and image fusion; then the feature map is input into the edge detection network (i.e., the network FE-BDCN improved based on the boundary detection convolutional network) to obtain a clearer power line edge, i.e., the preliminary detection map; finally, the preset optimization rules can be used to optimize the preliminary detection map to obtain the final detection map.

[0100] Furthermore, after verification by the above-mentioned verification embodiments, using the method for detecting power lines (fusion of event camera and general light camera) provided by the embodiments of the present disclosure, when faced with complex detection scenes, the edges of the power lines in the final detection image obtained are clear, and the detection precision and accuracy are high, thereby demonstrating the effectiveness of the method of the present disclosure.

[0101] The following is a description of an apparatus for detecting a power line provided by the present disclosure. The apparatus for detecting a power line described below and the method for detecting a power line described above can be referred to each other in correspondence.

[0102] Figure 6 FIG. 1 is a schematic diagram of a structure of an apparatus for detecting power lines provided by an exemplary embodiment of the present disclosure. Figure 6 As shown, the device for detecting a power line comprises: The data acquisition module 610 is configured to: acquire a target blurred image and a target event pulse vector corresponding to the target blurred image; wherein the target blurred image is a motion blurred image of the power line to be detected in the target area acquired by the general light camera within a preset time, and the target event pulse vector is an event pulse vector determined based on the event stream of the power line to be detected in the target area acquired by the event camera within the preset time; the detection execution module 620 is configured to: input the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset power line detection model, and after the preset Assume that the power line detection model outputs a preliminary detection image representing the power line detection result; wherein the preset power line detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes sample input data and sample true value, the sample input data includes a fuzzy sample image and an event stream sample vector corresponding to the fuzzy sample image, the sample true value includes annotation data matching the fuzzy sample image, and the annotation data is used to mark the real power line segment; the result optimization module 630 is configured to: based on preset optimization rules, perform structural optimization on the preliminary detection image to obtain a final detection image.

[0103] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor (processor) 710, a communication interface (Communications Interface) 720, a memory (memory) 730 and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other through the communication bus 740. The processor 710 can call the logic instructions in the memory 730 to execute a method for detecting power lines, the method comprising: obtaining a target blurred image and a target event pulse vector corresponding to the target blurred image; wherein the target blurred image is a motion blurred image of the power lines to be detected in the target area collected by the general light camera within a preset time, and the target event pulse vector is an event pulse vector determined based on the event stream of the power lines to be detected in the target area collected by the event camera within the preset time; the target blurred image and the target event pulse vector corresponding to the target blurred image are input into a preset power line detection model, and the preset power line detection model outputs a preliminary detection map representing the power line detection result; wherein the preset power line detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes sample input data and sample true value, the sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image, and the sample true value includes annotation data matching the blurred sample image, and the annotation data is used to mark the real power line segment; based on a preset optimization rule, the preliminary detection map is structurally optimized to obtain a final detection map.

[0104] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0105] On the other hand, the present disclosure also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for detecting power lines provided by the above methods, the method including: acquiring a target blurred image and a target event pulse vector corresponding to the target blurred image; wherein the target blurred image is a motion blurred image of the power lines to be detected in the target area acquired based on a general light camera within a preset time, and the target event pulse vector is an event pulse vector determined within the preset time based on an event stream of the power lines to be detected in the target area acquired based on an event camera. The target blurred image and the target event pulse vector corresponding to the target blurred image are input into a preset power line detection model, and the preset power line detection model outputs a preliminary detection map representing the power line detection result; wherein the preset power line detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes sample input data and sample true value, the sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image, the sample true value includes annotation data matching the blurred sample image, and the annotation data is used to mark the real power line segment; based on the preset optimization rules, the preliminary detection map is structurally optimized to obtain the final detection map.

[0106] On the other hand, the present disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for detecting power lines provided by the above-mentioned methods, the method comprising: acquiring a target blurred image and a target event pulse vector corresponding to the target blurred image; wherein the target blurred image is a motion blurred image of the power lines to be detected in the target area acquired by a general light camera within a preset time, and the target event pulse vector is an event pulse vector determined within the preset time based on an event stream of the power lines to be detected in the target area acquired by an event camera; and The target event pulse vector corresponding to the target blurred image is input into a preset power line detection model, and the preset power line detection model outputs a preliminary detection map representing the power line detection result; wherein, the preset power line detection model is trained based on at least one set of training data, and each set of training data in the at least one set of training data includes sample input data and sample true value, and the sample input data includes a blurred sample image and an event stream sample vector corresponding to the blurred sample image, and the sample true value includes annotation data matching the blurred sample image, and the annotation data is used to mark the real power line segment; based on the preset optimization rules, the preliminary detection map is structurally optimized to obtain the final detection map.

[0107] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0108] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for detecting a power line, characterized in that: include: Acquire a target blurred image and a target event pulse vector corresponding to the target blurred image; The target blurred image is a motion blurred image of the power lines to be detected in the target area acquired by the general light camera within a preset time, and the target event pulse vector is an event pulse vector determined based on the event stream of the power lines to be detected in the target area acquired by the event camera within the preset time; Inputting the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset power line detection model, and outputting a preliminary detection image representing the power line detection result through the preset power line detection model; The preset power line detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes sample input data and sample true values, the sample input data includes a fuzzy sample image and an event stream sample vector corresponding to the fuzzy sample image, the sample true value includes annotation data matching the fuzzy sample image, and the annotation data is used to mark the real power line segment; Based on preset optimization rules, the preliminary detection image is structurally optimized to obtain a final detection image.

2. The method according to claim 1, characterized in that The preset power line detection model includes a cascaded feature fusion network and an edge detection network.

3. The method according to claim 2, characterized in that The feature fusion network includes a cascaded feature extraction subnetwork, a channel splicing unit, and a channel attention fusion subnetwork; The feature extraction subnetwork includes an image feature extraction unit VGG16_C and an event feature extraction unit VGG16_10C connected in parallel, the output end of the image feature extraction unit VGG16_C is connected to the input end of the channel splicing unit, and the output end of the event feature extraction unit VGG16_10C is connected to the input end of the channel splicing unit; wherein the number of receiving channels of the event feature extraction unit VGG16_C is 3, and the number of receiving channels of the event feature extraction unit VGG16_10C is 10; The channel attention fusion sub-network includes a cascaded channel splitting unit, an attention unit, and a fusion unit; The attention unit includes a first attention subunit and a second attention subunit connected in parallel, a first output end of the channel splitting unit is connected to an input end of the first attention subunit, and a second output end of the channel splitting unit is connected to an input end of the second attention subunit; The output end of the first attention sub-unit is connected to the input end of the fusion unit, and the output end of the second attention sub-unit is connected to the input end of the fusion unit; The output end of the fusion unit is connected to the input end of the edge detection network.

4. The method according to claim 3, characterized in that The first attention subunit has the same structure as the second attention subunit; The first attention subunit includes a cascaded input layer, a pooling module, a connection layer, a convolution layer, an activation function Sigmoid layer, a multiplication module, a summation module and an output layer; The pooling module includes an average pooling layer and a maximum pooling layer connected in parallel; the output end of the input layer is connected to the input ends of the average pooling layer and the maximum pooling layer respectively, and the output ends of the average pooling layer and the maximum pooling layer are connected to the input end of the connection layer respectively; The output end of the input layer is jump-connected to the input end of the multiplication module and the input end of the summation module respectively.

5. The method according to claim 2, characterized in that: The edge detection network includes a first detection module, a second detection module, a first upsampling layer, a third detection module and a second upsampling layer which are cascaded in sequence; The output end of the second detection module is also connected to the input end of the first detection module, and the output end of the third detection module is also connected to the input end of the second detection module.

6. The method according to any one of claims 1 to 5, characterized in that The training steps of the preset power line detection model include: Acquire the at least one set of training data; wherein the sample input data in the training data includes positive sample input data and negative sample input data; Inputting the sample input data in each group of training data into the power line detection model to be trained respectively, and the power line detection model to be trained outputs the corresponding power line detection estimation result; Using a preset comprehensive loss function, based on the true value of the samples in each set of training data and the corresponding power line detection estimation result, a function value of the comprehensive loss function is obtained; Based on the function value of the comprehensive loss function, the power line detection model to be trained is trained until a preset training completion condition is met, and the neural network is obtained from the power line detection model to be trained.

7. The method according to claim 6, characterized in that The using of a preset comprehensive loss function, based on the sample true value in each set of training data and the corresponding power line detection estimation result, to obtain the function value of the comprehensive loss function, comprises: Based on the sample true value in each set of training data and the corresponding power line detection estimation result, respectively calculate the function value of the first detection module loss function, the function value of the second detection module loss function, the function value of the third detection module loss function and the function value of the fusion loss function; Using preset weights, the function value of the loss function of the first detection module, the function value of the loss function of the second detection module, the function value of the loss function of the third detection module, and the function value of the fusion loss function are weightedly fused to obtain the function value of the comprehensive loss function.

8. The method according to claim 7, characterized in that The calculating the function value of the loss function of the first detection module based on the sample true value in each set of training data and the corresponding power line detection estimation result includes: Calculate the function value of the loss function of the first detection module using the following calculation formula; in, represents a binary cross entropy function; s1 represents the forward probability of the power line detection estimate output by the first detection module for each set of sample input data; s11 represents the reverse probability of the power line detection estimate output by the first detection module for each set of sample input data; y represents the sample true value corresponding to each set of sample input data; The calculating the function value of the loss function of the second detection module based on the sample true value in each set of training data and the corresponding power line detection estimation result includes: Calculate the function value of the loss function of the second detection module using the following calculation formula; Wherein, s2 represents the forward probability of the power line detection estimate output by the second detection module for each set of sample input data; s21 represents the reverse probability of the power line detection estimate output by the second detection module for each set of sample input data; y represents the sample true value corresponding to each set of sample input data; The calculating the function value of the loss function of the third detection module based on the sample true value in each set of training data and the corresponding power line detection estimation result includes: Calculate the function value of the loss function of the third detection module using the following calculation formula; Wherein, s3 represents the forward probability of the power line detection estimation output by the third detection module for each set of sample input data; s31 represents the reverse probability of the power line detection estimation output by the third detection module for each set of sample input data; y represents the sample true value corresponding to each set of sample input data; The calculating the function value of the fusion loss function based on the sample true value in each set of training data and the corresponding power line detection estimation result includes: The function value of the fusion loss function is calculated using the following calculation formula; Wherein, fuse represents the fusion probability of the power line detection estimation output by the power line detection model to be trained for each set of sample input data; y represents the sample true value corresponding to each set of sample input data; The step of using preset weights to weightedly fuse the function value of the first detection module loss function, the function value of the second detection module loss function, the function value of the third detection module loss function, and the function value of the fusion loss function includes: The function value of the comprehensive loss function is calculated using the following calculation formula; in, represents the weight of the loss function of the first detection module; represents the weight of the loss function of the second detection module; Represents the weight of the loss function of the third detection module; Represents the weight of the fusion loss function.

9. The method according to claim 6, characterized in that Based on the function value of the comprehensive loss function, the power line detection model to be trained is trained until a preset training completion condition is met, including: Passing back the function value of the comprehensive loss function to the power line detection model to be trained; The power line detection model to be trained adjusts the network parameters of each network layer according to the function value of the comprehensive loss function; Iterate the steps of obtaining the function value of the comprehensive loss function to adjusting the network parameters of each network layer until a preset training completion condition is met.

10. A device for detecting a power line, characterized in that: include: A data acquisition module is configured to: acquire a target blurred image and a target event pulse vector corresponding to the target blurred image; The target blurred image is a motion blurred image of the power lines to be detected in the target area acquired by the general light camera within a preset time, and the target event pulse vector is an event pulse vector determined based on the event stream of the power lines to be detected in the target area acquired by the event camera within the preset time; A detection execution module is configured to: input the target blurred image and the target event pulse vector corresponding to the target blurred image into a preset power line detection model, and output a preliminary detection image representing the power line detection result through the preset power line detection model; The preset power line detection model is trained based on at least one set of training data, each set of training data in the at least one set of training data includes sample input data and sample true values, the sample input data includes a fuzzy sample image and an event stream sample vector corresponding to the fuzzy sample image, the sample true value includes annotation data matching the fuzzy sample image, and the annotation data is used to mark the real power line segment; The result optimization module is configured to: perform structural optimization on the preliminary detection image based on preset optimization rules to obtain a final detection image.

Citation Information

Patent Citations

  • Motion blurred image line segment detection method and system fusing event and image

    CN114913342A

  • Bionic vision fusion severe environment imaging device and method

    CN115631123A

  • Linear detection method based on dynamic visual sensor

    CN116188425A

  • Wire detection method, device and equipment

    CN116664517A

  • Feature fusion for input picture data preprocessing for learning model

    US20240221363A1

Cited By

  • Power monitoring system and method based on power internet of things

    CN121147172A