A method and device for fusion of satellite remote sensing images of power transmission lines

Abstract

The invention relates to a method and device for fusion of satellite remote sensing images of a power transmission line, comprising: obtaining a fusion image of a satellite remote sensing high-resolution panchromatic band image and a fusion image of a satellite remote sensing multispectral low-resolution image of the power transmission line; substituting the fusion image of the satellite remote sensing high-resolution panchromatic band image and the fusion image of the satellite remote sensing multispectral low-resolution image of the power transmission line into a pre-established sparse fusion model of satellite remote sensing images of the power transmission line to obtain different band images of characteristic graphs corresponding to each high-resolution filter or low-resolution filter; determining different band images of a high-resolution multispectral image based on different band images of characteristic graphs corresponding to each high-resolution filter or low-resolution filter; the scheme considers the correlation in the frequency band of the multispectral image to propose an appropriate regularizer capable of capturing structural correlation, accelerates the fusion speed, and provides reliable data guarantee for subsequent environmental hidden danger inspections of satellite remote sensing channels of power transmission lines.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing data fusion, and in particular to a method and device for fusion of satellite remote sensing images of power transmission lines. Background Art

[0002] In recent years, the spatial resolution and update frequency of satellite remote sensing have become increasingly higher, and it has been gradually and widely used in the inspection of power transmission lines and channels. In practical applications, power transmission lines are weak targets in satellite remote sensing images, and their information needs to be enhanced to achieve high-precision inspections.

[0003] Typically, high spatial resolution (HR) panchromatic satellite remote sensing images (PAN) with a single broadband channel contain accurate spatial details, while low altitude resolution (LR) multispectral satellite remote sensing images (MS) can provide more spectral characteristics due to the physical trade-off between spatial and spectral resolution. Therefore, it is difficult to obtain a complete scene interpretation of the observed area only through high spatial resolution panchromatic images or low altitude resolution multispectral images. In order to make full use of the complementary spatial and spectral information, the fusion of high spatial resolution panchromatic images and low altitude resolution multispectral images is proposed to produce high spatial resolution multispectral images, which is the basis for realizing transmission line information enhancement.

[0004] In the past two decades, panchromatic and multispectral satellite remote sensing image fusion technology has attracted widespread interest, and researchers have proposed many methods, which can be divided into three categories: component replacement (CS) methods, multiresolution analysis (MRA) methods, and model-based methods. For the first category, the classic methods are Gram-Schmidt (GS), principal component analysis, and "intensity-hue-saturation" methods. These methods use specific transformations to separate the spatial structure and spectral information in the interpolated low-resolution multispectral (LR MS) images. Then, the components containing spatial information are replaced with high spatial resolution panchromatic (HRPAN) images. Finally, high spatial resolution multispectral (HR MS) images are generated through the corresponding inverse transformation. These methods can be easily implemented and perform well in enhancing spatial details. However, the mismatch of the spectral ranges of panchromatic (PAN) and multispectral (MS) images leads to obvious spectral distortion.

[0005] In response to the above problems, researchers have developed multiresolution analysis methods (MRA), assuming that the lost spatial details in low-resolution multispectral (LR MS) images can be inferred from high spatial resolution panchromatic (HR PAN) images. Therefore, these methods extract spatial details from high spatial resolution panchromatic (HR PAN) images and then inject them into low-resolution multispectral (LR MS) images to obtain high spatial resolution multispectral (HR MS) images. In order to find more suitable high frequencies, some MRA tools such as wavelet transform, contourlet transform and curvelet transform are used in the framework. In addition, according to the same scheme, a high spatial resolution multispectral image method based on support value transform (SVT) is also proposed. MRA methods can effectively preserve the spectral information of multispectral (MS) images. However, the injection coefficient depends on the appropriate model, which may cause some scale distortion.

[0006] Another group of researchers proposed a model-based approach to establish the relationship between the source image and the desired high spatial resolution multispectral (HR MS) image. Among them, the high spatial resolution panchromatic (HR PAN) and low spatial resolution multispectral (LR MS) images are regarded as degraded versions of the high spatial resolution multispectral (HR MS) images in the spectral domain and spatial domain, respectively. Therefore, the fusion task is reformulated as a restoration problem. Then, reasonable priors are introduced to regulate the restoration model, such as total variation, Bayesian paradigm, and non-negativity. But in fact, model-based methods are often difficult to objectively and finely characterize the actual situation, resulting in errors.

[0007] In recent years, sparsity-induced panchromatic and multispectral satellite remote sensing image fusion methods have received extensive attention and development. For example, inspired by compressed sensing, Li and Yang used the basis pursuit algorithm to solve the restoration model with sparse regularization to achieve the fusion of high spatial resolution panchromatic (HR PAN) and low spatial resolution multispectral (LR MS) images. This method can well preserve spectral and spatial details, but its application on full-scale data is limited because the dictionary is constructed from small patches from the desired HR MS image. Li et al. constructed a HR MS image dictionary from the HR PAN and LR MS image dictionaries using the K-SVD algorithm. In addition, Jiang et al. proposed a two-step sparse coding method with patch normalization (PN-TSSC) method to find the sparse code and coding residual of the image.

[0008] Although sparse coding-based image fusion methods have been widely studied and achieved state-of-the-art performance, they still have some shortcomings that need to be improved.

[0009] 1) Most of these methods adopt a patch partitioning-based strategy to process the source image, processing each patch independently without considering the consistency constraint of overlapping pixels. When reconstructing the fused image, the weighted sum of overlapping pixels from different adjacent patches will bring aggregate ambiguity.

[0010] 2) The divided blocks destroy the entire spatial structure of the image. In addition, due to the redundancy of overlapping pixels between adjacent blocks, the atomic structure of the dictionary will be affected to a certain extent. In addition, the redundant features in the dictionary may cause distortion of spatial details.

[0011] 3) Most sparse coding based fusion methods estimate the sparse coefficients of each band in low-resolution multispectral (LR-MS) images continuously and independently. However, there are strong correlations and similarities between MS image bands. An appropriate regularizer that can capture structural correlations will improve the performance of the fused image. Summary of the invention

[0012] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method and device for fusion of satellite remote sensing images of power transmission lines, which can avoid partition processing, retain more spatial texture features of power transmission lines, and reduce information distortion. At the same time, considering the correlation in the multispectral image band, an appropriate regularizer that can capture structural correlation is proposed to speed up the fusion speed and provide reliable data guarantee for subsequent environmental hazard inspections of satellite remote sensing channels of power transmission lines.

[0013] The purpose of the present invention is achieved by adopting the following technical solutions:

[0014] A method for fusion of satellite remote sensing images of power transmission lines, the improvement of which is that the method comprises:

[0015] Obtain satellite remote sensing high-resolution panchromatic band images and satellite remote sensing multispectral low-resolution images of transmission lines;

[0016] Preprocessing the satellite remote sensing high-resolution panchromatic band image and the satellite remote sensing multispectral low-resolution image of the power transmission line respectively, and obtaining a fused image of the satellite remote sensing high-resolution panchromatic band image and a fused image of the satellite remote sensing multispectral low-resolution image of the power transmission line;

[0017] Substituting the fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line and the fused image of the satellite remote sensing multispectral low-resolution image into a pre-established sparse fusion model of the power transmission line satellite remote sensing image, solving the pre-established sparse fusion model of the power transmission line satellite remote sensing image, and obtaining different band images of the feature map corresponding to each high-resolution filter or low-resolution filter;

[0018] Different band images of the high-resolution multispectral image are determined based on different band images of the characteristic graphs corresponding to each high-resolution filter or low-resolution filter.

[0019] Preferably, the preprocessing of the satellite remote sensing high-resolution panchromatic band image and the satellite remote sensing multispectral low-resolution image of the power transmission line respectively to obtain a fused image of the satellite remote sensing high-resolution panchromatic band image and a fused image of the satellite remote sensing multispectral low-resolution image of the power transmission line comprises:

[0020] Clipping the first buffer area and the second buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the transmission line, and the first buffer area and the second buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line;

[0021] Based on the first buffer area and the second buffer area of ​​the cropped satellite remote sensing high-resolution panchromatic band image of the transmission line, a fused image of the satellite remote sensing high-resolution panchromatic band image of the transmission line is obtained; based on the first buffer area and the second buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line, a fused image of the satellite remote sensing multispectral low-resolution image of the transmission line is obtained.

[0022] Furthermore, the method of obtaining a fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line based on clipping the first buffer area and the second buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the power transmission line includes:

[0023] Performing edge information enhancement on a first buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the transmission line;

[0024] The first buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the power transmission line after edge information enhancement is superimposed with the second buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the power transmission line to obtain a fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line;

[0025] The method of obtaining a fused image of the satellite remote sensing multi-spectral low-resolution image of the power transmission line based on the first buffer area and the second buffer area of ​​the satellite remote sensing multi-spectral low-resolution image of the power transmission line comprises:

[0026] Perform edge information enhancement on the first buffer area of ​​the satellite remote sensing multispectral low-resolution image of the power transmission line;

[0027] The first buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line after edge information enhancement is superimposed with the second buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line to obtain a fused image of the satellite remote sensing multispectral low-resolution image of the transmission line.

[0028] Furthermore, the first buffer area is an area extending 500 m from the transmission line to both sides of the transmission line, and the second buffer area is an area extending 2 km from the transmission line to both sides of the transmission line.

[0029] Preferably, the process of acquiring the pre-established sparse fusion model of the satellite remote sensing image of the power transmission line includes:

[0030] Acquire historical satellite remote sensing high-resolution panchromatic band images and historical satellite remote sensing multispectral low-resolution images of power transmission lines from the historical satellite remote sensing image database;

[0031] Preprocess the historical satellite remote sensing high-resolution panchromatic band images and the historical satellite remote sensing multispectral low-resolution images of the power transmission line respectively to obtain the fused images of the historical satellite remote sensing high-resolution panchromatic band images and the fused images of the historical satellite remote sensing multispectral low-resolution images;

[0032] Substituting the fused image of the historical satellite remote sensing high-resolution panchromatic band image and the fused image of the historical satellite remote sensing multispectral low-resolution image into a pre-established multi-resolution filter joint calculation model, and solving the pre-established multi-resolution filter joint calculation model to obtain a high-resolution filter and a low-resolution filter, as well as feature maps corresponding to the high-resolution filter and the low-resolution filter;

[0033] The high-resolution filter and the low-resolution filter are used to establish the pre-established sparse fusion model of the satellite remote sensing image of the power transmission line.

[0034] Furthermore, the pre-established multi-resolution filter joint calculation model is determined as follows:

[0035]

[0036] In the above formula, N is the total number of historical satellite remote sensing high-resolution panchromatic band images or historical satellite remote sensing multispectral low-resolution images of the power transmission line, K is the total number of preset high-resolution filters or low-resolution filters, and f k is the kth high-resolution filter, d k is the kth low-resolution filter, Z k,n is the feature map corresponding to the k-th high-resolution filter and the k-th low-resolution filter, P n is the nth historical satellite remote sensing high-resolution panchromatic band image, L n is the nth historical satellite remote sensing multispectral low-resolution image, and β controls the sparsity of feature mapping.

[0037] Furthermore, K∈[60,80], and β∈[0.15,0.3].

[0038] Furthermore, the use of the high-resolution filter and the low-resolution filter to establish the pre-established sparse fusion model of the satellite remote sensing image of the power transmission line includes:

[0039] The pre-established transmission line satellite remote sensing image sparse fusion model is determined as follows:

[0040]

[0041] In the above formula, N is the total number of historical satellite remote sensing high-resolution panchromatic band images or satellite remote sensing multispectral low-resolution images of the transmission line, K is the total number of preset high-resolution filters or low-resolution filters, and f k is the kth high-resolution filter, d k is the kth low-resolution filter, C is the total number of bands of the satellite remote sensing high-resolution panchromatic band image or the satellite remote sensing multispectral low-resolution image of the transmission line, M is the satellite remote sensing multispectral low-resolution image, Z k,c is the c-th band image of the feature map corresponding to the k-th high-resolution filter or low-resolution filter, α, λ and γ are all regularization parameters, P is the historical satellite remote sensing high-resolution panchromatic band image of the transmission line, ω c is the weight coefficient of the cth band image of the historical satellite remote sensing high-resolution panchromatic band image or the cth band image of the satellite remote sensing multispectral low-resolution image of the transmission line, Z k+1,c It is the c-th band image of the feature map corresponding to the k+1-th high-resolution filter or low-resolution filter.

[0042] Preferably, the determining of different band images of the high-resolution multispectral image based on different band images of the characteristic graphs corresponding to the high-resolution filters or the low-resolution filters comprises:

[0043] The c-th band image H of the high-resolution multispectral image is determined as follows: c :

[0044]

[0045] In the above formula, Z k,c is the cth band image of the feature map corresponding to the kth high-resolution filter or low-resolution filter, K is the total number of preset high-resolution filters or low-resolution filters, C is the total number of frequency bands of the satellite remote sensing high-resolution panchromatic band image or satellite remote sensing multispectral low-resolution image of the transmission line, f k is the kth high-resolution filter.

[0046] Based on the same inventive concept, the present invention also provides a satellite remote sensing image fusion device for power transmission lines, the improvement of which is that the device comprises:

[0047] The first acquisition module is used to acquire satellite remote sensing high-resolution panchromatic band images and satellite remote sensing multispectral low-resolution images of the power transmission line;

[0048] The second acquisition module is used to pre-process the satellite remote sensing high-resolution panchromatic band image and the satellite remote sensing multispectral low-resolution image of the power transmission line, and obtain a fused image of the satellite remote sensing high-resolution panchromatic band image and a fused image of the satellite remote sensing multispectral low-resolution image of the power transmission line;

[0049] A calculation module is used to substitute the fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line and the fused image of the satellite remote sensing multispectral low-resolution image into a pre-established sparse fusion model of the power transmission line satellite remote sensing image, solve the pre-established sparse fusion model of the power transmission line satellite remote sensing image, and obtain different band images of the feature map corresponding to each high-resolution filter or low-resolution filter;

[0050] The determination module is used to determine different band images of the high-resolution multispectral image based on different band images of the characteristic graphs corresponding to each high-resolution filter or low-resolution filter.

[0051] Compared with the closest prior art, the present invention has the following beneficial effects:

[0052] The transmission line is a strip-like continuous terrain target, and the hollow structure features of the transmission tower and the texture features of hardware such as insulator strings are obvious. The existing image fusion method based on sparse coding will cause the loss of texture structure features in the process of image segmentation. At the edge of the segmentation, the same terrain features such as transmission lines and vegetation are likely to show different spectral characteristics, which is not conducive to the fidelity of information after fusion. To this end, the present invention provides a method and device for fusion of satellite remote sensing images of transmission lines, the method comprising: obtaining satellite remote sensing high-resolution panchromatic band images and satellite remote sensing multi-spectral low-resolution images of the transmission line; pre-processing the satellite remote sensing high-resolution panchromatic band images and satellite remote sensing multi-spectral low-resolution images of the transmission line respectively to obtain a fused image of the satellite remote sensing high-resolution panchromatic band images and a fused image of the satellite remote sensing multi-spectral low-resolution images of the transmission line; combining the satellite remote sensing high-resolution panchromatic band images of the transmission line with the satellite remote sensing multi-spectral low-resolution images; The fused image of the high-resolution panchromatic band image and the fused image of the satellite remote sensing multispectral low-resolution image are substituted into the pre-established sparse fusion model of the transmission line satellite remote sensing image, the pre-established sparse fusion model of the transmission line satellite remote sensing image is solved, and the different band images of the feature map corresponding to each high-resolution filter or low-resolution filter are obtained; the different band images of the high-resolution multispectral image are determined based on the different band images of the feature map corresponding to each high-resolution filter or low-resolution filter; this scheme avoids zoning processing, retains more spatial texture features of the transmission line, reduces information distortion, and considers the correlation in the multispectral image band, thereby accelerating the fusion speed, and providing reliable data guarantee for the subsequent satellite remote sensing channel environmental hidden danger inspection of the transmission line. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of a method for fusion of satellite remote sensing images of power transmission lines provided by the present invention;

[0054] Figure 2 The present invention is a schematic structural diagram of a satellite remote sensing image fusion device for a power transmission line. DETAILED DESCRIPTION

[0055] The specific implementation modes of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] Transmission lines are continuous strip-shaped ground objects, and the hollow structure of transmission towers and the texture features of hardware such as insulator strings are obvious. The existing image fusion methods based on sparse coding will cause the loss of texture structure features during the image block processing. At the edge of the block, the same ground objects such as transmission lines and vegetation are likely to show different spectral characteristics, which is not conducive to the fidelity of information after fusion. In view of this, the present invention provides a method for fusion of satellite remote sensing images of transmission lines, such as Figure 1 As shown, including:

[0058] 101 Obtain satellite remote sensing high-resolution panchromatic band images and satellite remote sensing multispectral low-resolution images of transmission lines;

[0059] 102 respectively pre-process the satellite remote sensing high-resolution panchromatic band image and the satellite remote sensing multispectral low-resolution image of the power transmission line to obtain a fused image of the satellite remote sensing high-resolution panchromatic band image and a fused image of the satellite remote sensing multispectral low-resolution image of the power transmission line;

[0060] 103. Substituting the fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line and the fused image of the satellite remote sensing multispectral low-resolution image into a pre-established sparse fusion model of the power transmission line satellite remote sensing image, solving the pre-established sparse fusion model of the power transmission line satellite remote sensing image, and obtaining different band images of the feature map corresponding to each high-resolution filter or low-resolution filter;

[0061] 104 determines different band images of the high-resolution multispectral image based on different band images of the characteristic graphs corresponding to the high-resolution filters or the low-resolution filters.

[0062] In the optimal embodiment provided by the present invention, after step 101, the satellite remote sensing high-resolution panchromatic band image and the satellite remote sensing multispectral low-resolution image of the transmission line can be pre-processed by atmospheric correction, radiation calibration, orthorectification, etc., to obtain a historical satellite remote sensing image library.

[0063] Specifically, the step 102 includes:

[0064] Clipping the first buffer area and the second buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the transmission line, and the first buffer area and the second buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line;

[0065] Based on the first buffer area and the second buffer area of ​​the cropped satellite remote sensing high-resolution panchromatic band image of the transmission line, a fused image of the satellite remote sensing high-resolution panchromatic band image of the transmission line is obtained; based on the first buffer area and the second buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line, a fused image of the satellite remote sensing multispectral low-resolution image of the transmission line is obtained.

[0066] The method of obtaining a fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line based on clipping the first buffer area and the second buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the power transmission line includes:

[0067] Performing edge information enhancement on a first buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the transmission line;

[0068] The first buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the power transmission line after edge information enhancement is superimposed with the second buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the power transmission line to obtain a fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line;

[0069] The method of obtaining a fused image of the satellite remote sensing multi-spectral low-resolution image of the power transmission line based on the first buffer area and the second buffer area of ​​the satellite remote sensing multi-spectral low-resolution image of the power transmission line comprises:

[0070] Perform edge information enhancement on the first buffer area of ​​the satellite remote sensing multispectral low-resolution image of the power transmission line;

[0071] The first buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line after edge information enhancement is superimposed with the second buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line to obtain a fused image of the satellite remote sensing multispectral low-resolution image of the transmission line.

[0072] The first buffer area is an area extending 500 m from the transmission line to both sides of the transmission line, and the second buffer area is an area extending 2 km from the transmission line to both sides of the transmission line.

[0073] In the above scheme, the process of acquiring the pre-established sparse fusion model of the satellite remote sensing image of the power transmission line includes:

[0074] Acquire historical satellite remote sensing high-resolution panchromatic band images and historical satellite remote sensing multispectral low-resolution images of power transmission lines from the historical satellite remote sensing image database;

[0075] Preprocess the historical satellite remote sensing high-resolution panchromatic band images and the historical satellite remote sensing multispectral low-resolution images of the power transmission line respectively to obtain the fused images of the historical satellite remote sensing high-resolution panchromatic band images and the fused images of the historical satellite remote sensing multispectral low-resolution images;

[0076] Substituting the fused image of the historical satellite remote sensing high-resolution panchromatic band image and the fused image of the historical satellite remote sensing multispectral low-resolution image into a pre-established multi-resolution filter joint calculation model, and solving the pre-established multi-resolution filter joint calculation model to obtain a high-resolution filter and a low-resolution filter, as well as feature maps corresponding to the high-resolution filter and the low-resolution filter;

[0077] The high-resolution filter and the low-resolution filter are used to establish the pre-established sparse fusion model of the satellite remote sensing image of the power transmission line.

[0078] Furthermore, the pre-established multi-resolution filter joint calculation model is determined as follows:

[0079]

[0080] In the above formula, N is the total number of historical satellite remote sensing high-resolution panchromatic band images or historical satellite remote sensing multispectral low-resolution images of the power transmission line, K is the total number of preset high-resolution filters or low-resolution filters, and f k is the kth high-resolution filter, d k is the kth low-resolution filter, Z k,n is the feature map corresponding to the k-th high-resolution filter and the k-th low-resolution filter, P n is the nth historical satellite remote sensing high-resolution panchromatic band image, L n is the nth historical satellite remote sensing multispectral low-resolution image, and β controls the sparsity of feature mapping.

[0081] Furthermore, K∈[60,80], and β∈[0.15,0.3].

[0082] Furthermore, the use of the high-resolution filter and the low-resolution filter to establish the pre-established sparse fusion model of the satellite remote sensing image of the power transmission line includes:

[0083] The pre-established transmission line satellite remote sensing image sparse fusion model is determined as follows:

[0084]

[0085] In the above formula, N is the total number of historical satellite remote sensing high-resolution panchromatic band images or satellite remote sensing multispectral low-resolution images of the transmission line, K is the total number of preset high-resolution filters or low-resolution filters, and f k is the kth high-resolution filter, d kis the kth low-resolution filter, C is the total number of bands of the satellite remote sensing high-resolution panchromatic band image or the satellite remote sensing multispectral low-resolution image of the transmission line, M is the satellite remote sensing multispectral low-resolution image, Z k,c is the c-th band image of the feature map corresponding to the k-th high-resolution filter or low-resolution filter, α, λ and γ are all regularization parameters, P is the historical satellite remote sensing high-resolution panchromatic band image of the transmission line, ω c is the weight coefficient of the cth band image of the historical satellite remote sensing high-resolution panchromatic band image or the cth band image of the satellite remote sensing multispectral low-resolution image of the transmission line, Z k+1,c It is the c-th band image of the feature map corresponding to the k+1-th high-resolution filter or low-resolution filter.

[0086] Among them, when the given satellite remote sensing panchromatic image calibration spatial resolution is less than or equal to 0.5m, the regularization parameters α, λ and γ are set to 17-32, 3 and 1 respectively. When the given satellite remote sensing panchromatic image calibration spatial resolution is greater than 0.5m, the regularization parameters α, λ and γ are set to 8-16, 4 and 4 respectively.

[0087] Finally, the step 104 includes:

[0088] The c-th band image H of the high-resolution multispectral image is determined as follows: c :

[0089]

[0090] In the above formula, Z k,c is the cth band image of the feature map corresponding to the kth high-resolution filter or low-resolution filter, K is the total number of preset high-resolution filters or low-resolution filters, C is the total number of frequency bands of the satellite remote sensing high-resolution panchromatic band image or satellite remote sensing multispectral low-resolution image of the transmission line, f k is the kth high-resolution filter.

[0091] The 4-step transmission line satellite remote sensing image fusion process in the content of the present invention combines transmission line information enhancement and fusion for the first time to improve the final effect. Taking a certain area as an example, the existing GIHS, GS, AWLP and SRLD algorithms are used for fusion, and Q4, spectral angle mapper (SAM), universal image quality index (UIQI) and Erreur RelativeGlobale Adimensionnelle de Synthèse (ERGAS) indicators are used to evaluate the quality of the fusion results. Generally, larger Q4 and UIQI mean better fusion results. For SAM and ERGAS, smaller values ​​indicate better results, and the optimal value is 0. The quantitative comparison results of the present invention and the existing methods are as follows. As can be seen from Table 1, the present invention is better than the existing methods in all indicators, which shows that the present invention effectively improves the transmission line image fusion results and lays the foundation for the subsequent transmission line inspection.

[0092] Table 1 Comparison of quantitative indicators of the corresponding results of different fusion methods

[0093]

[0094] Based on the same inventive concept, the present invention also provides a satellite remote sensing image fusion device for power transmission lines, such as Figure 2 As shown, the device comprises:

[0095] The first acquisition module is used to acquire satellite remote sensing high-resolution panchromatic band images and satellite remote sensing multispectral low-resolution images of the power transmission line;

[0096] The second acquisition module is used to pre-process the satellite remote sensing high-resolution panchromatic band image and the satellite remote sensing multispectral low-resolution image of the power transmission line, and obtain a fused image of the satellite remote sensing high-resolution panchromatic band image and a fused image of the satellite remote sensing multispectral low-resolution image of the power transmission line;

[0097] A calculation module is used to substitute the fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line and the fused image of the satellite remote sensing multispectral low-resolution image into a pre-established sparse fusion model of the power transmission line satellite remote sensing image, solve the pre-established sparse fusion model of the power transmission line satellite remote sensing image, and obtain different band images of the feature map corresponding to each high-resolution filter or low-resolution filter;

[0098] The determination module is used to determine different band images of the high-resolution multispectral image based on different band images of the characteristic graphs corresponding to each high-resolution filter or low-resolution filter.

[0099] Preferably, the second acquisition module is specifically used for:

[0100] Clipping the first buffer area and the second buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the transmission line, and the first buffer area and the second buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line;

[0101] Based on the first buffer area and the second buffer area of ​​the cropped satellite remote sensing high-resolution panchromatic band image of the transmission line, a fused image of the satellite remote sensing high-resolution panchromatic band image of the transmission line is obtained; based on the first buffer area and the second buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line, a fused image of the satellite remote sensing multispectral low-resolution image of the transmission line is obtained.

[0102] Furthermore, the method of obtaining a fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line based on clipping the first buffer area and the second buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the power transmission line includes:

[0103] Performing edge information enhancement on a first buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the transmission line;

[0104] The first buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the power transmission line after edge information enhancement is superimposed with the second buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the power transmission line to obtain a fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line;

[0105] The method of obtaining a fused image of the satellite remote sensing multi-spectral low-resolution image of the power transmission line based on the first buffer area and the second buffer area of ​​the satellite remote sensing multi-spectral low-resolution image of the power transmission line comprises:

[0106] Perform edge information enhancement on the first buffer area of ​​the satellite remote sensing multispectral low-resolution image of the power transmission line;

[0107] The first buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line after edge information enhancement is superimposed with the second buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line to obtain a fused image of the satellite remote sensing multispectral low-resolution image of the transmission line.

[0108] Furthermore, the first buffer area is an area extending 500 m from the transmission line to both sides of the transmission line, and the second buffer area is an area extending 2 km from the transmission line to both sides of the transmission line.

[0109] Preferably, the process of acquiring the pre-established sparse fusion model of the satellite remote sensing image of the power transmission line includes:

[0110] Acquire historical satellite remote sensing high-resolution panchromatic band images and historical satellite remote sensing multispectral low-resolution images of power transmission lines from the historical satellite remote sensing image database;

[0111] Preprocess the historical satellite remote sensing high-resolution panchromatic band images and the historical satellite remote sensing multispectral low-resolution images of the power transmission line respectively to obtain the fused images of the historical satellite remote sensing high-resolution panchromatic band images and the fused images of the historical satellite remote sensing multispectral low-resolution images;

[0112] Substituting the fused image of the historical satellite remote sensing high-resolution panchromatic band image and the fused image of the historical satellite remote sensing multispectral low-resolution image into a pre-established multi-resolution filter joint calculation model, and solving the pre-established multi-resolution filter joint calculation model to obtain a high-resolution filter and a low-resolution filter, as well as feature maps corresponding to the high-resolution filter and the low-resolution filter;

[0113] The high-resolution filter and the low-resolution filter are used to establish the pre-established sparse fusion model of the satellite remote sensing image of the power transmission line.

[0114] Furthermore, the pre-established multi-resolution filter joint calculation model is determined as follows:

[0115]

[0116] In the above formula, N is the total number of historical satellite remote sensing high-resolution panchromatic band images or historical satellite remote sensing multispectral low-resolution images of the power transmission line, K is the total number of preset high-resolution filters or low-resolution filters, and f k is the kth high-resolution filter, d k is the kth low-resolution filter, Z k,n is the feature map corresponding to the k-th high-resolution filter and the k-th low-resolution filter, P n is the nth historical satellite remote sensing high-resolution panchromatic band image, L n is the nth historical satellite remote sensing multispectral low-resolution image, and β controls the sparsity of feature mapping.

[0117] Furthermore, K∈[60,80], and β∈[0.15,0.3].

[0118] Furthermore, the use of the high-resolution filter and the low-resolution filter to establish the pre-established sparse fusion model of the satellite remote sensing image of the power transmission line includes:

[0119] The pre-established transmission line satellite remote sensing image sparse fusion model is determined as follows:

[0120]

[0121] In the above formula, N is the total number of historical satellite remote sensing high-resolution panchromatic band images or satellite remote sensing multispectral low-resolution images of the transmission line, K is the total number of preset high-resolution filters or low-resolution filters, and f k is the kth high-resolution filter, d k is the kth low-resolution filter, C is the total number of bands of the satellite remote sensing high-resolution panchromatic band image or the satellite remote sensing multispectral low-resolution image of the transmission line, M is the satellite remote sensing multispectral low-resolution image, Z k,c is the c-th band image of the feature map corresponding to the k-th high-resolution filter or low-resolution filter, α, λ and γ are all regularization parameters, P is the historical satellite remote sensing high-resolution panchromatic band image of the transmission line, ω c is the weight coefficient of the cth band image of the historical satellite remote sensing high-resolution panchromatic band image or the cth band image of the satellite remote sensing multispectral low-resolution image of the transmission line, Z k+1,c It is the c-th band image of the feature map corresponding to the k+1-th high-resolution filter or low-resolution filter.

[0122] Preferably, the determining module is specifically used for:

[0123] The c-th band image H of the high-resolution multispectral image is determined as follows: c :

[0124]

[0125] In the above formula, Z k,c is the cth band image of the feature map corresponding to the kth high-resolution filter or low-resolution filter, K is the total number of preset high-resolution filters or low-resolution filters, C is the total number of frequency bands of the satellite remote sensing high-resolution panchromatic band image or satellite remote sensing multispectral low-resolution image of the transmission line, f k is the kth high-resolution filter.

[0126] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0127] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0128] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for fusion of satellite remote sensing images of power transmission lines, characterized in that: The method comprises: Obtain satellite remote sensing high-resolution panchromatic band images and satellite remote sensing multispectral low-resolution images of transmission lines; Preprocessing the satellite remote sensing high-resolution panchromatic band image and the satellite remote sensing multispectral low-resolution image of the power transmission line respectively, and obtaining a fused image of the satellite remote sensing high-resolution panchromatic band image and a fused image of the satellite remote sensing multispectral low-resolution image of the power transmission line; Substituting the fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line and the fused image of the satellite remote sensing multispectral low-resolution image into a pre-established sparse fusion model of the power transmission line satellite remote sensing image, solving the pre-established sparse fusion model of the power transmission line satellite remote sensing image, and obtaining different band images of the feature map corresponding to each high-resolution filter or low-resolution filter; Determine different band images of the high-resolution multispectral image based on different band images of the characteristic graphs corresponding to each high-resolution filter or low-resolution filter; The acquisition process of the pre-established transmission line satellite remote sensing image sparse fusion model includes: Acquire historical satellite remote sensing high-resolution panchromatic band images and historical satellite remote sensing multispectral low-resolution images of power transmission lines from the historical satellite remote sensing image database; Preprocess the historical satellite remote sensing high-resolution panchromatic band images and the historical satellite remote sensing multispectral low-resolution images of the power transmission line respectively to obtain the fused images of the historical satellite remote sensing high-resolution panchromatic band images and the fused images of the historical satellite remote sensing multispectral low-resolution images; Substituting the fused image of the historical satellite remote sensing high-resolution panchromatic band image and the fused image of the historical satellite remote sensing multispectral low-resolution image into a pre-established multi-resolution filter joint calculation model, and solving the pre-established multi-resolution filter joint calculation model to obtain a high-resolution filter and a low-resolution filter, as well as feature maps corresponding to the high-resolution filter and the low-resolution filter; Using the high-resolution filter and the low-resolution filter to establish the pre-established sparse fusion model of the satellite remote sensing image of the power transmission line; The pre-established multi-resolution filter joint calculation model is determined as follows: In the above formula, N is the total number of historical satellite remote sensing high-resolution panchromatic band images or historical satellite remote sensing multispectral low-resolution images of the power transmission line, K is the total number of preset high-resolution filters or low-resolution filters, and f k is the kth high-resolution filter, d k is the kth low-resolution filter, Z k,n is the feature map corresponding to the k-th high-resolution filter and the k-th low-resolution filter, P n is the nth historical satellite remote sensing high-resolution panchromatic band image, L n is the nth historical satellite remote sensing multispectral low-resolution image, and β is the sparsity of the control feature map; The method of using the high-resolution filter and the low-resolution filter to establish the pre-established sparse fusion model of the satellite remote sensing image of the power transmission line comprises: The pre-established transmission line satellite remote sensing image sparse fusion model is determined as follows: In the above formula, N is the total number of historical satellite remote sensing high-resolution panchromatic band images or satellite remote sensing multispectral low-resolution images of the transmission line, K is the total number of preset high-resolution filters or low-resolution filters, and f k is the kth high-resolution filter, d k is the kth low-resolution filter, C is the total number of bands of the satellite remote sensing high-resolution panchromatic band image or the satellite remote sensing multispectral low-resolution image of the transmission line, M is the satellite remote sensing multispectral low-resolution image, Z k,c is the c-th band image of the feature map corresponding to the k-th high-resolution filter or low-resolution filter, α, λ and γ are all regularization parameters, P is the historical satellite remote sensing high-resolution panchromatic band image of the transmission line, ω c is the weight coefficient of the cth band image of the historical satellite remote sensing high-resolution panchromatic band image or the cth band image of the satellite remote sensing multispectral low-resolution image of the transmission line, Z k+1,c is the c-th band image of the feature map corresponding to the k+1-th high-resolution filter or low-resolution filter; The determining of different band images of the high-resolution multispectral image based on different band images of the characteristic graphs corresponding to the high-resolution filters or the low-resolution filters includes: The c-th band image H of the high-resolution multispectral image is determined as follows: c : In the above formula, Z k,c is the cth band image of the feature map corresponding to the kth high-resolution filter or low-resolution filter, K is the total number of preset high-resolution filters or low-resolution filters, C is the total number of frequency bands of the satellite remote sensing high-resolution panchromatic band image or satellite remote sensing multispectral low-resolution image of the transmission line, f k is the kth high-resolution filter.

2. The method according to claim 1, characterized in that The preprocessing of the satellite remote sensing high-resolution panchromatic band image and the satellite remote sensing multispectral low-resolution image of the power transmission line respectively to obtain the fused image of the satellite remote sensing high-resolution panchromatic band image and the fused image of the satellite remote sensing multispectral low-resolution image of the power transmission line includes: Clipping the first buffer area and the second buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the transmission line, and the first buffer area and the second buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line; Based on the first buffer area and the second buffer area of ​​the cropped satellite remote sensing high-resolution panchromatic band image of the transmission line, a fused image of the satellite remote sensing high-resolution panchromatic band image of the transmission line is obtained; based on the first buffer area and the second buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line, a fused image of the satellite remote sensing multispectral low-resolution image of the transmission line is obtained.

3. The method according to claim 2, characterized in that The method of obtaining a fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line based on clipping the first buffer area and the second buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the power transmission line comprises: Performing edge information enhancement on a first buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the transmission line; The first buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the power transmission line after edge information enhancement is superimposed with the second buffer area of ​​the satellite remote sensing high-resolution panchromatic band image of the power transmission line to obtain a fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line; The method of obtaining a fused image of the satellite remote sensing multi-spectral low-resolution image of the power transmission line based on the first buffer area and the second buffer area of ​​the satellite remote sensing multi-spectral low-resolution image of the power transmission line comprises: Perform edge information enhancement on the first buffer area of ​​the satellite remote sensing multispectral low-resolution image of the power transmission line; The first buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line after edge information enhancement is superimposed with the second buffer area of ​​the satellite remote sensing multispectral low-resolution image of the transmission line to obtain a fused image of the satellite remote sensing multispectral low-resolution image of the transmission line.

4. The method according to claim 2, characterized in that The first buffer area is an area extending 500 m from the transmission line to both sides of the transmission line, and the second buffer area is an area extending 2 km from the transmission line to both sides of the transmission line.

5. The method according to claim 1, characterized in that The K∈[60,80], the β∈[0.15,0.3].

6. A device based on the method for fusion of satellite remote sensing images of power transmission lines according to any one of claims 1 to 5, characterized in that: The device comprises: The first acquisition module is used to acquire satellite remote sensing high-resolution panchromatic band images and satellite remote sensing multispectral low-resolution images of the power transmission line; The second acquisition module is used to pre-process the satellite remote sensing high-resolution panchromatic band image and the satellite remote sensing multispectral low-resolution image of the power transmission line, and obtain a fused image of the satellite remote sensing high-resolution panchromatic band image and a fused image of the satellite remote sensing multispectral low-resolution image of the power transmission line; A calculation module is used to substitute the fused image of the satellite remote sensing high-resolution panchromatic band image of the power transmission line and the fused image of the satellite remote sensing multispectral low-resolution image into a pre-established sparse fusion model of the power transmission line satellite remote sensing image, solve the pre-established sparse fusion model of the power transmission line satellite remote sensing image, and obtain different band images of the feature map corresponding to each high-resolution filter or low-resolution filter; The determination module is used to determine different band images of the high-resolution multispectral image based on different band images of the characteristic graphs corresponding to each high-resolution filter or low-resolution filter.

Images