Power line fog penetration monitoring method, system and device based on polarization-infrared multi-mode fusion and medium

Abstract

The invention discloses a power line fog penetration monitoring method, system and device based on polarization-infrared multi-mode fusion and a medium. An atomic clock is tamed to trigger infrared pulse and polarization exposure, and pixel offset caused by jitter of an unmanned aerial vehicle is compensated through an image motion analysis method; constructing a polarization physical model, and establishing a relationship between the fogdrop particle size and the polarization degree; performing multi-modal feature fusion, constructing a double-branch deep learning network, and performing output judgment on the image according to a dynamic fusion strategy; and optimizing conductor features and insulator features in the power line through a special convolution kernel, and accurately extracting target features.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart grid monitoring, and in particular to a power line fog-penetrating monitoring method, system, device and medium based on polarization-infrared multi-modal fusion. BACKGROUND

[0002] In the field of power line monitoring, the existing technology has many defects and pain points.

[0003] Single-mode fog penetration has limitations. In terms of pure infrared imaging, the 1550nm band has a penetration rate attenuation of up to 60% in thick fog (actual measurement data), which directly leads to a wire identification missed detection rate of >40%; when using pure polarization detection, the Stokes parameter calculation is greatly affected by ambient light interference, which will fail at night or on cloudy days.

[0004] Multi-modal fusion also faces many problems. The traditional weighted fusion algorithm does not take into account the Mie scattering (Mie Scattering) difference between the wire and the fog droplets, resulting in a decrease in SNR (signal-to-noise ratio) after fusion; the time synchronization error of the polarization sensor and the infrared detector is >10ms, causing image misplacement of moving targets (such as dancing wires).

[0005] At the same time, the power scene is not well adapted. The existing optical filter is not optimized for the reflection characteristics (refractive index 1.62) of the insulator ceramic surface, resulting in a polarization degree (DoP) measurement error of ±15%. SUMMARY

[0006] In view of the above existing problems, the present application is proposed.

[0007] To solve the above technical problems, the present application provides the following technical solutions:

[0008] In a first aspect, the present application provides a power line fog-penetrating monitoring method based on polarization-infrared multi-modal fusion, comprising:

[0009] Domestic atomic clock triggers infrared pulse and polarization exposure, and compensates for pixel shift caused by unmanned aerial vehicle jitter through image motion analysis method;

[0010] Construct a polarization physical model to establish the relationship between fog droplet size and polarization degree;

[0011] Multi-modal feature fusion, construct a double-branch deep learning network, and use attention mechanism in the fusion layer for dynamic weighting;

[0012] Optimize the wire features and insulator features in the power line through a special convolution kernel to accurately extract the target features.

[0013] As a preferred scheme of the power line fog monitoring method based on polarization-infrared multi-modal fusion, the polarization physical modeling is used to establish the relationship between the fog droplet particle size and the polarization degree, and the polarization physical modeling comprises the following steps:

[0014] The fog droplet particle size parameter is input into the polarization physical model, and the polarization physical model comprehensively considers the influence strength of the particle size on the polarization degree and the probability weighting of deviating from the peak particle size.

[0015] The polarization physical model outputs the polarization degree index to quantize the polarization response level under the specific particle size distribution condition.

[0016] As a preferred scheme of the power line fog monitoring method based on polarization-infrared multi-modal fusion, the multi-modal feature fusion is used to construct a double-branch deep learning network, and the fusion layer adopts a dynamic weighting attention mechanism, and the multi-modal feature fusion comprises the following steps:

[0017] The polarization branch inputs the polarization state parameter and outputs the scattering characteristic diagram.

[0018] The infrared branch inputs the time sequence image feature and outputs the target intensity diagram.

[0019] The fusion layer adopts a dynamic attention mechanism to perform image restoration.

[0020] The preferred technical scheme in the embodiment has the following beneficial effects: through the double-branch network structure and the dynamic weighting fusion mode, the advantages of the polarization and the infrared two modes can be fully utilized, and the quality of the image and the accuracy of the target identification are improved.

[0021] As a preferred scheme of the power line fog monitoring method based on polarization-infrared multi-modal fusion, the dedicated convolution kernel is used to optimize the wire feature and the insulator feature in the power line, and the target feature is accurately extracted, and the preferred scheme comprises the following steps:

[0022] A dedicated wire identification kernel is designed.

[0023] An insulator identification kernel is designed, and the insulator identification kernel is used for feature extraction.

[0024] The convolution response features of the wire and the insulator are subjected to multi-dimensional screening, and an identification result is output.

[0025] As a preferred scheme of the power line fog monitoring method based on polarization-infrared multi-modal fusion, the polarization physical modeling is represented as:

[0026]

[0027] Wherein, d represents the fog droplet size, lambda represents the probe light wavelength, d0 represents the fog droplet size distribution peak, sigma represents the particle size distribution standard deviation, beta represents the transmission line scene calibration coefficient.

[0028] The preferred technical scheme in the embodiment of the application has the beneficial effects that the model deeply considers the characteristics of the transmission line scene, can more accurately describe the relationship between the fog droplet size and the polarization degree, and improves the accuracy of polarization detection.

[0029] As a preferred scheme of the power line fog monitoring method based on polarization-infrared multi-modal fusion, the output judgment of the image according to the dynamic fusion strategy comprises:

[0030] When the fog droplet size d < 5 mu m, alpha = 0.7-0.8, the scattering characteristic map of the polarization branch is reserved;

[0031] When the fog droplet size d > 15 mu m, alpha = 0.2-0.3, the target intensity map of the infrared branch is emphasized;

[0032] When 5 mu m <= d <= 15 mu m, alpha dynamically changes with the fog droplet size distribution standard deviation sigma;

[0033] Wherein, alpha represents the weight coefficient.

[0034] As a preferred scheme of the power line fog monitoring method based on polarization-infrared multi-modal fusion, the multi-dimensional screening of the convolution response features of the conductor and the insulator comprises:

[0035] In the spatial dimension, the morphological operation is used to remove noise regions with an area < 50 pixels;

[0036] In the time dimension, the feature positions of 10 consecutive images are tracked, and the amplitude of conductor galloping and the change rate of insulator temperature are calculated;

[0037] The output recognition result comprises a hidden danger type, a position coordinate and a confidence.

[0038] The preferred technical scheme in the embodiment of the application has the beneficial effects that the conductor miss detection rate is controlled to be below 3%, the accuracy of insulator flashover identification is above 92%, and compared with the traditional general detection algorithm, the feature matching accuracy for the power scene is improved by 40%.

[0039] In a second aspect, the application provides a power line fog monitoring system based on polarization-infrared multi-modal fusion, comprising an optical layer and an algorithm layer.

[0040] The optical layer comprises an active emission unit and a polarization detection unit.

[0041] The algorithm layer comprises a polarization-infrared space-time alignment model, a fusion network based on scattering physics and a power special multi-band filter.

[0042] The polarization-infrared space-time alignment model realizes nanosecond-level synchronization (less than 5 ns) through FPGA hardware.

[0043] A ScatterNet based on scattering physics is developed to quantify the influence of fog droplet size distribution on polarization parameters.

[0044] A power special multi-band filter is designed to synchronously suppress background noise in the 1064 nm / 1550 nm / visible light band.

[0045] The polarization-infrared space-time alignment model (FPGA nanosecond-level synchronization) solves the problem of time misalignment of multi-modal data, the fusion network based on scattering physics removes fog from the physical nature and improves the imaging robustness under complex fog conditions, the multi-band filter is designed to specifically suppress power scene-specific noise and improve the quality of raw data.

[0046] In a third aspect, the present application provides an electronic device, comprising:

[0047] a memory and a processor;

[0048] The memory is used for storing computer executable instructions, and the processor is used for executing the computer executable instructions, so as to realize the steps of the power line dehazing monitoring method based on polarization-infrared multi-modal fusion.

[0049] In a fourth aspect, the present application provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are executed by a processor to realize the steps of the power line dehazing monitoring method based on polarization-infrared multi-modal fusion.

[0050] Compared with the prior art, the present application has the beneficial effects that the monitoring accuracy in bad weather can be significantly improved. Through the polarization-infrared multi-modal fusion technology, combined with the physical modeling of fog droplet size and polarization degree, the problems of low penetration rate of traditional infrared imaging and large environmental light interference of pure polarization detection in thick fog environment are effectively overcome. The dynamic attention fusion strategy adaptively adjusts the modal weight according to the fog droplet size distribution, so that the line recognition missed detection rate is reduced to below 3%, which is improved by more than 40% compared with the traditional method. In addition, the deep collaborative optimization of hardware-algorithm is realized. The tame atomic clock and FPGA hardware are used to realize nanosecond-level synchronization (<5ns), which solves the multi-modal data time misalignment problem; the special multi-band filter is optimized for the spectral characteristics of power equipment, and this collaborative design improves the signal-to-noise ratio from the data source, providing high-quality input for subsequent algorithm processing. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 The algorithm flowchart of a power line fog penetration monitoring method based on polarization-infrared multi-modal fusion according to an embodiment of the present application.

[0053] Figure 2 The framework diagram of a power line fog penetration monitoring system based on polarization-infrared multi-modal fusion according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings in the specification. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0055] Embodiment 1, refer to Figure 1 According to an embodiment of the present application, a power line fog penetration monitoring method based on polarization-infrared multi-modal fusion is provided, comprising:

[0056] S1: tame atomic clock triggers infrared pulse and polarization exposure, and compensates for pixel shift caused by unmanned aerial vehicle jitter through image motion analysis method;

[0057] S2: construct a polarization physical model to establish the relationship between fog droplet size and polarization degree;

[0058] S3: Multi-modal feature fusion, construct a double-branch deep learning network, and use attention mechanism in the fusion layer for dynamic weighting;

[0059] S4: Optimize the conductor feature and insulator feature in the power line through a special convolution kernel to accurately extract the target feature.

[0060] It should be noted that, in view of the problems of single-mode fog penetration limitation, poor multi-modal fusion effect, and insufficient adaptation to power scenarios in the prior art in the field of power line monitoring, the four key technologies are used to solve the problems respectively. S1 uses a tame atomic clock to realize high-precision synchronization of infrared pulses and polarization exposure, and combines image motion analysis method to compensate for pixel shift caused by unmanned aerial vehicle jitter, which significantly improves imaging stability and reduces missed detection rate; S2 constructs a polarization physical model to establish the relationship between fog droplet size and polarization degree, effectively reduces environmental light interference, and enhances the robustness of polarization detection in complex environments such as night or cloudy days; S3 introduces a double-branch deep learning network, which dynamically weights and fuses multi-modal features through attention mechanism, solves the problem of signal-to-noise ratio decline caused by not considering Mie scattering difference in traditional weighted fusion, and relieves image misplacement caused by time synchronization error; S4 designs a special convolution kernel to optimize the extraction of conductor and insulator features, and adapts to the reflection characteristics of insulator ceramic surface, which significantly reduces the polarization degree measurement error. The overall scheme improves the accuracy and reliability of power line monitoring through high-precision synchronization, physical modeling, intelligent fusion, and feature optimization.

[0061] Embodiment 2, refer to Figure 1 For an embodiment of the present application, based on the above embodiment, a power line fog penetration monitoring method based on polarization-infrared multi-modal fusion is provided.

[0062] In the embodiment of the present application, the tame atomic clock in step S1 triggers infrared pulses and polarization exposure, and compensates for pixel shift caused by unmanned aerial vehicle jitter through image motion analysis method. The specific implementation manner is as follows:

[0063] In the embodiment of the present application, the image motion analysis method uses optical flow method compensation.

[0064] A GPS tame atomic clock (accuracy ±10 ns) is used to trigger infrared pulses and polarization exposure.

[0065] The optical flow method is used to compensate for pixel shift caused by jitter of the installation on the power transmission line (compensation accuracy 0.3 pixel).

[0066] It should be noted that step S1 ensures accurate alignment of infrared and polarization images in time and space, which lays a foundation for subsequent fusion processing.

[0067] In the embodiments of the present application, the construction of the polarization physical model in step S2 includes steps A1-A2:

[0068] A1: input the droplet size parameter into the polarization physical model, which comprehensively considers the influence intensity of particle size on polarization degree and the probability weighting of deviation from the peak particle size

[0069] A2: the polarization physical model outputs the polarization degree index, which quantifies the polarization response level under specific particle size distribution conditions.

[0070] In the embodiments of the present application, the specific calculation method of constructing the polarization physical model in step S2 is:

[0071] Further, the relationship between the droplet size d and the polarization degree is established:

[0072]

[0073] Where λ=1064nm represents the detection light wavelength, which matches the transmission wavelength band of the ceramic reinforced filter. d0=10μm represents the peak of the droplet size distribution (the actual statistical value under typical haze conditions). σ=2μm represents the standard deviation of the particle size distribution, which represents the uniformity of the droplets. β=0.28 represents the calibration coefficient of the power line scene.

[0074] In an optional embodiment, the construction of the polarization physical model in step S2 can also be a polarization degree modeling based on physical mechanism driving. This scheme is based on the physical action mechanism between the droplet size and the polarization degree, and combines the actual environmental characteristics in the power line scene to construct the polarization physical model. The model comprehensively considers the influence intensity of the droplet size distribution on the polarization degree, and introduces a dynamic weighting mechanism for deviation from the peak particle size, thereby realizing accurate quantification of the polarization response level. When the droplet size distribution is relatively regular and the environmental light interference is controllable in the power line inspection scene, the polarization degree modeling based on physical mechanism driving can be used.

[0075] In another optional embodiment, the construction of the polarization physical model in step S2 can also be a multispectral fusion modeling based on polarization reflection characteristics. This scheme constructs a polarization physical model by fusing multispectral polarization information and combining the relationship between the droplet size and the polarization reflection characteristics. The model uses the response difference of the droplets to polarized light under different spectral bands to improve the accuracy of polarization degree measurement through multispectral fusion technology. It is suitable for power line monitoring scenes where multispectral polarization sensors are available, such as power inspection systems where multispectral imaging equipment has been deployed.

[0076] In the embodiments of the present application, the multi-modal feature fusion in step S3 includes steps B1-B3:

[0077] B1: Polarization branch input polarization state parameter, output scattering characteristic map;

[0078] B2: Infrared branch input time sequence image feature, output target intensity map;

[0079] B3: The fusion layer adopts a dynamic attention mechanism for image repair.

[0080] Further, in the embodiment of the present application, the specific implementation of step B1 is:

[0081] In the embodiment of the present application, the input polarization state parameter in step B1 is the Stokes parameter;

[0082] The polarization branch input Q / U / V Stokes parameter, (0° / 45° / 90° / 135° image collected by four-way polarization CMOS → calculate Stokes parameter (Q / U / V), as 3-channel input. Q = I(0°)-I(90°) U = I(45°)-I(135°) V = 0 (linearly polarized scene ignores circularly polarized component)) output scattering characteristic map.

[0083] In an optional embodiment, the input polarization state parameter in step B1 can be two physical quantities of polarization degree (DoP) and polarization angle (AoP), which can be used in occasions where the calculation resources are limited and lightweight deployment is required in the polarization imaging system, such as embedded edge computing devices or unmanned aerial vehicle real-time processing systems.

[0084] In an optional embodiment, the input polarization state parameter in step B1 can also be a full Stokes vector, the model inputs a full Stokes image in a four-channel manner, learns more comprehensive polarization scattering features through a deep network, and outputs a high-precision scattering characteristic map. It is suitable for occasions where high-precision polarization imaging systems are deployed, such as ground fixed monitoring stations or laboratory level devices.

[0085] Further, in the embodiment of the present application, the specific implementation of step B2 is:

[0086] In the embodiment of the present application, the input time sequence image feature in step B2 is a time domain difference image;

[0087] The infrared branch inputs 3 frames of time domain difference images (input 3 consecutive infrared images (interval 1ms), calculate difference images ΔI12 = I2-I1, ΔI23 = I3-I2, and stack the original frame I2 as 3-channel input.), output target intensity map.

[0088] Further, in the embodiment of the present application, the specific implementation of step B3 is:

[0089] In the embodiments of the present application, the dynamic attention mechanism in step B3 adopts attention mechanism dynamic weighting.

[0090] The fusion layer adopts attention mechanism dynamic weighting (weighting factor a = 0.2-0.8) (when the fog droplet size d < 5 pm (by step 2 model backstepping), a = 0.7-0.8, preferentially preserving the scattering characteristic map of the polarization branch, and using the polarization difference of small particle size fog droplets to enhance the target edge; when the fog droplet size d > 15 pm, a = 0.2-0.3, focusing on the target intensity map of the infrared branch, and highlighting the conductor profile by means of the penetration advantage of the 1550 nm waveband; when 5 pm ≤ d ≤ 15 pm, a dynamically changes with the fog droplet size distribution standard deviation s (the larger s is, the closer a is to 0.5), and the noise and signal intensity are balanced by complementary features of the two channels.).

[0091] It should be noted that by means of the dual-branch network structure and the dynamic weighting fusion mode, the advantages of the polarization and infrared modalities can be fully utilized, and the image quality and target recognition accuracy can be improved.

[0092] In an optional embodiment, the dynamic attention mechanism in step B3 can also adopt a dynamic attention mechanism based on adaptive gating fusion. The fusion layer introduces a gating unit (Gating Unit). According to the fog droplet size information output by the fog droplet size distribution model, the feature fusion ratio of the polarization branch and the infrared branch is dynamically adjusted. The gating unit controls the fusion weight through a learnable parameter, and the weight range is between 0 and 1. When the fog droplet size is small, the gating value is biased towards the polarization branch; when the fog droplet size is large, the gating value is biased towards the infrared branch. It is suitable for power line monitoring environments with frequent changes in fog droplet size distribution and unstable environmental conditions, such as mountainous areas or coastal areas.

[0093] In another optional embodiment, the dynamic attention mechanism in step B3 can also adopt a dynamic attention mechanism based on multi-scale feature fusion. The fusion layer performs multi-scale feature extraction (such as through pyramid pooling or hollow convolution) on the scattering characteristic map output by the polarization branch and the target intensity map output by the infrared branch, respectively, then fuses the features of different scales, and finally outputs the fused image through a fully connected layer. Multi-scale feature fusion can capture both detailed information and global information in the image, improving the quality of image restoration. It is suitable for scenes with complex background and small target in power line monitoring, such as urban areas or areas with dense vegetation.

[0094] In the embodiments of the present application, the optimization of the conductor features and the insulator features in the power line through the special convolution kernel in step S4 accurately extracts the target features, including steps C1-C3:

[0095] C1: Design a special conductor identification kernel.

[0096] C2: Design an insulator recognition kernel and perform feature extraction on the insulator recognition kernel.

[0097] C3: Perform multi-dimensional screening on the convolution response features of the conductor and the insulator, and output the recognition result.

[0098] The specific implementation of step C1 is as follows:

[0099] Design a special convolution kernel: the conductor recognition kernel is a 5x5 cross-shaped template, which matches the cross-sectional features of the conductor; the insulator recognition kernel is a ring-shaped gradient template, which matches the periodic ring structure of the insulator umbrella skirt. These special convolution kernels are optimized for the features of conductors and insulators in power lines, which can more accurately extract target features and improve the accuracy of hidden danger recognition.

[0100] The conductor recognition kernel uses a 5x5 cross-shaped template, and the convolution weight distribution is as follows: the cross intersection (center pixel) weight is set to 0.4, the horizontal and vertical directions each extend 2 pixels (a total of 8 pixels) with a weight of 0.1, and the remaining pixel weights are 0. This design matches the linear features of the conductor cross-section: the diameter of the power line conductor is usually 10-20mm, which appears as a continuous line of 1-3 pixels wide in the image, and the cross-shaped template can strengthen the response value of the linear region through sliding convolution (step length 1).

[0101] During the convolution process, for the 128-channel scattering feature map in the fusion feature map, the feature channels consistent with the polarization direction of the conductor (such as the 0° polarization channel corresponding to the horizontal conductor) are preferentially activated; for the 64-channel target intensity map, the time-domain intensity changes caused by conductor galloping are captured (related to the motion features of the infrared branch). After the response map output by the convolution is segmented by an adaptive threshold (threshold = 0.6x maximum response value), a binary mask of the conductor region can be obtained, and the continuity detection of the mask edge is used to determine whether there is a broken strand hidden danger (continuous broken pixels > 3 are determined as hidden dangers).

[0102] The specific implementation of step C2 is as follows:

[0103] The insulator recognition kernel is a 7x7 ring-shaped gradient template, which is divided into 3 layers of ring-shaped structure from inside to outside: the inner layer (radius 1 pixel) weight -0.2, the middle layer (radius 2-3 pixels) weight 0.5, and the outer layer (radius 4-6 pixels) weight -0.1, which presents a "high-low-high" gradient distribution, matching the periodic ring structure of the insulator umbrella skirt (typical umbrella skirt spacing 20-30mm, corresponding to 3-5 pixels in the image).

[0104] When the template is applied, weighted convolution is performed on the high polarization region (DoP>0.3) of the insulator ceramic surface in the fused feature map, and whether the shed is damaged is judged by calculating the gradient response difference (threshold ±0.2) of the adjacent shed. The gradient response difference of the normal shed is less than 0.1, and the response difference of the glaze damage area caused by flashover increases to more than 0.3. At the same time, combined with the temperature feature channel output by the infrared branch, the connection part of the insulator steel cap and the porcelain shed is detected (when the temperature abnormally rises by more than 5°C, the flashover early warning is triggered).

[0105] The specific implementation of step C3 is:

[0106] The convolution response characteristics of the conductor and the insulator are screened in multiple dimensions: in the spatial dimension, morphological operations (erosion-expansion) are used to remove noise regions with an area less than 50 pixels (such as false features caused by fog droplets); in the time dimension, the feature positions of 10 consecutive images are tracked, and the amplitude of the conductor dancing (more than 0.5m is determined as severe dancing) and the temperature change rate of the insulator (more than 2°C / s is determined as a flashover precursor) are calculated.

[0107] The final output includes the identification result containing the hazard type, location coordinates (accuracy ±1 pixel), and confidence (based on feature matching degree calculation, range 0-1), wherein the conductor miss detection rate is controlled to be less than 3%, the accuracy of insulator flashover identification is more than 92%, and compared with the traditional general detection algorithm, the feature matching accuracy for the power scene is improved by 40%.

[0108] In the power line routine inspection in the embodiment of the application, the application can realize all-weather monitoring capability in adverse weather. The traditional inspection is seriously affected by weather such as fog and haze, and the inspection is often interrupted due to imaging blur in thick fog, while the application can still stably identify the conductor and the insulator in a thick fog environment with visibility less than 500 meters through the polarization-infrared multi-modal fusion and dynamic weight adjustment strategy. For unmanned aerial vehicle inspection operation, the hardware-algorithm collaborative design of the scheme significantly improves the operation and maintenance efficiency. The combination of the tame atomic clock and the optical flow method controls the unmanned aerial vehicle jitter compensation accuracy within 0.3 pixels, and cooperates with the special convolution kernel to accurately extract the target, so that the unmanned aerial vehicle can also realize the inspection in foggy weather. The high-precision identification capability of the application directly improves the reliability of the power grid. The performance of conductor miss detection rate less than 3% can accurately identify the potential hazards of the conductor in foggy weather.

[0109] Embodiment 3 is as follows: Figure 2As shown in the above, it is a schematic scheme of a power line fog monitoring method based on polarization-infrared multi-modal fusion. It should be noted that the technical scheme of the power line fog monitoring system based on polarization-infrared multi-modal fusion belongs to the same concept as the technical scheme of the power line fog monitoring method based on polarization-infrared multi-modal fusion described above. The details of the technical scheme of the power line fog monitoring system based on polarization-infrared multi-modal fusion in this embodiment are not described in detail, and can be referred to the description of the technical scheme of the power line fog monitoring method based on polarization-infrared multi-modal fusion.

[0110] The embodiment also provides a power line fog monitoring system based on polarization-infrared multi-modal fusion, which comprises an optical layer and an algorithm layer. The core is the deep coupling of the hardware customization design of the optical layer and the physical-intelligent collaborative optimization of the algorithm layer, which solves the core problems of power line fog monitoring such as "weak signal, strong interference, and multi-modal misplacement".

[0111] The optical layer comprises an active emission unit and a polarization detection unit.

[0112] The active emission unit is designed to design a 1550nm infrared laser (peak power 20W) and integrate a vibration diffraction wave modulator (frequency 1-5kHz adjustable).

[0113] Specifically, the active emission unit is a 1550nm infrared laser and a vibration diffraction wave modulator. The modulated infrared light emitted by the active emission unit can break through the "signal flooding" problem of fog scattering and provide a strong specific target signal for the infrared branch. The fog-penetrating advantage of the 1550nm wave band and the anti-interference ability of the modulation technology ensure the effectiveness of the infrared data in bad weather.

[0114] The polarization detection unit is designed to be a four-way polarization CMOS (0° / 45° / 90° / 135°) equipped with a ceramic enhanced filter (transmittance > 90% @ 1064nm). The four-way polarization acquisition realizes the full-dimensional capture of polarization characteristics and provides a physical basis for the polarization branch to distinguish fog and targets.

[0115] The algorithm layer comprises a polarization-infrared space-time alignment model, a scattering physics-based fusion network, and a power-specific multi-band filter.

[0116] The polarization-infrared space-time alignment model realizes nanosecond-level synchronization (less than 5 ns) through FPGA hardware; the ScatterNet based on scattering physics is used to quantify the influence of fog droplet size distribution on polarization parameters; and the power special multi-band filter can synchronously suppress background noise in the 1064 nm / 1550 nm / visible light band. The core advantage of this design is that through hardware-algorithm collaborative optimization, the core pain points of power line monitoring in bad weather, such as noise interference and multi-modal data misplacement, are solved. The polarization-infrared space-time alignment model (FPGA nanosecond-level synchronization) solves the problem of time misplacement of multi-modal data, the ScatterNet based on scattering physics removes fog from the physical nature to improve the imaging robustness in complex fog conditions, the multi-band filter is designed to specifically suppress the noise specific to the power scene, and the quality of the original data is improved.

[0117] The embodiment also provides an electronic device suitable for the case of the polarization-infrared multi-modal fusion based power line fog removal monitoring, which comprises a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the method for the polarization-infrared multi-modal fusion based power line fog removal monitoring as proposed in the above embodiment.

[0118] The embodiment also provides a storage medium having a computer program stored thereon, which is executed by a processor to realize the method for the polarization-infrared multi-modal fusion based power line fog removal monitoring as proposed in the above embodiment.

[0119] The storage medium proposed in the embodiment belongs to the same inventive concept as the method for the polarization-infrared multi-modal fusion based power line fog removal monitoring proposed in the above embodiment, and the technical details not described in detail in the embodiment can be referred to the above embodiment, and the embodiment has the same beneficial effects as the above embodiment.

[0120] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary general hardware, and of course can also be realized by hardware. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH, a hard disk or an optical disk, and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method of each embodiment of the present application.

[0121] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not to limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced, without departing from the spirit and scope of the technical solutions of the present application, and all of them should be covered in the scope of the claims of the present application.

Claims

1. A power line route haze-penetrating monitoring method based on polarization-infrared multi-modal fusion, characterized in that, The method comprises the following steps: Domestic atomic clock triggers infrared pulse and polarized exposure, and pixel offset caused by unmanned aerial vehicle jitter is compensated by image motion analysis method; A polarization physical model is constructed to establish the relationship between fog droplet size and polarization degree; Multi-modal feature fusion is used to construct a double-branch deep learning network, and dynamic fusion strategy is used to output and judge the image; The features of the conductors and insulators in the power line are optimized by using special convolution kernels to accurately extract the target features.

2. The power line route dehazing monitoring method based on polarization-infrared multi-modal fusion according to claim 1, characterized in that, The polarization physical modeling, which establishes the relationship between the fog droplet size and the polarization degree, comprises the following steps: The fog droplet size parameter is input into the polarization physical model, which comprehensively considers the influence intensity of the particle size on the polarization degree and the probability weighting of the deviation from the peak particle size; The polarization physical model outputs the polarization degree index to quantize the polarization response level under the condition of a specific particle size distribution.

3. The power line route dehazing monitoring method based on polarization-infrared multi-modal fusion according to claim 2, characterized in that, The multi-modal feature fusion, which constructs a double-branch deep learning network, uses an attention mechanism for dynamic weighting in the fusion layer, and comprises the following steps: The polarization branch inputs the polarization state parameter and outputs the scattering characteristic map; The infrared branch inputs the time series image features and outputs the target intensity map; The fusion layer uses a dynamic attention mechanism for image inpainting.

4. The power line route dehazing monitoring method based on polarization-infrared multi-modal fusion according to claim 3, characterized in that, The method for accurately extracting target features by optimizing the features of conductors and insulators in the power line using special convolution kernels comprises the following steps: A special conductor recognition kernel is designed; An insulator recognition kernel is designed and feature extraction is performed on the insulator recognition kernel; The convolution response features of the conductors and insulators are screened in multiple dimensions, and the recognition results are output.

5. The power line route dehazing monitoring method based on polarization-infrared multi-modal fusion according to claim 4, characterized in that, The polarization physical modeling is represented as: Wherein, d represents the fog droplet size, λ represents the detection light wavelength, d0 represents the peak value of the fog droplet size distribution, σ represents the standard deviation of the particle size distribution, and β represents the power line scene calibration coefficient.

6. The power line route dehazing monitoring method based on polarization-infrared multi-modal fusion according to claim 5, characterized in that, The output judgment of the image according to the dynamic fusion strategy comprises the following steps: When the fog droplet size d is less than 5 μm, α is 0.7-0.8, and the scattering characteristic map of the polarization branch is retained; When the fog droplet size d is greater than 15 μm, α is 0.2-0.3, and the target intensity map of the infrared branch is emphasized; When 5 μm≤d≤15 μm, α changes dynamically with the standard deviation σ of the fog droplet size distribution; Wherein, α represents the weight coefficient.

7. The power line route dehazing monitoring method based on polarization-infrared multi-modal fusion according to claim 6, characterized in that, The multi-dimensional screening of the convolution response features of the conductors and insulators comprises the following steps: In the spatial dimension, morphological operations are used to remove noise regions with an area less than 50 pixels; In the time dimension, the feature positions of 10 consecutive images are tracked to calculate the amplitude of conductor galloping and the change rate of insulator temperature; The output recognition result includes hazard type, position coordinates, and confidence.

8. A power line route defogging monitoring system based on polarization-infrared multi-modal fusion, applying the method of any one of claims 1-7, characterized in that, The method comprises the following steps: Optical layer and algorithm layer; The optical layer comprises an active emission unit and a polarization detection unit; The algorithm layer comprises a polarization-infrared space-time alignment model, a scattering physics-based fusion network, and a power-specific multi-band filter. 9.An electronic device, comprising: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions, which realize the steps of the power line de-hazing monitoring method based on polarization-infrared multi-modal fusion in any one of claims 1 to 7 when executed by the processor. 10.A computer readable storage medium storing computer executable instructions that, when executed by a processor, implement the steps of the power line route dehazing monitoring method based on polarization-infrared multi-modal fusion according to any one of claims 1 to 7.

Images