Ground feature unmixing method and system based on same-platform hyperspectral and high-score information fusion

Abstract

The invention aims to provide a ground feature unmixing method and system based on same-platform hyperspectral and high-score information fusion, belongs to the field of remote sensing information processing and application, and solves the problem that end member extraction and abundance coefficient calculation precision of an existing hyperspectral mixed pixel decomposition method are not ideal. Obtaining an end member spectrum from the hyperspectral image through an end member spectrum extraction algorithm based on adaptive dictionary learning by using the inherent sparse characteristic of the hyperspectral image; performing accurate registration on the hyperspectral image and the same-platform high-resolution panchromatic image, and realizing hyperspectral image super-resolution reconstruction on the basis; the hyperspectral image is reconstructed based on super-resolution, the extracted end member spectrum is utilized, spatial information is used as priori knowledge to be introduced into the solution of a hyperspectral image abundance coefficient matrix, and a more accurate abundance coefficient extraction result of the hyperspectral image after super-resolution reconstruction is obtained. The method can estimate the number of end members and extract the spectrum of the end members at the same time, and improves the precision of end member estimation and the generalization of the algorithm in different scenes.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing information processing and application, and specifically relates to a ground object unmixing method and system based on the fusion of hyperspectral and high-resolution information on the same platform. Background Art

[0002] Due to the complexity and diversity of natural object spectra and the low spatial resolution of spectral sensors, each pixel of a hyperspectral remote sensing image is usually a mixture of multiple object spectra. Mixed spectra pose great difficulties for the application of hyperspectral remote sensing images in target detection, object classification, and change detection. Therefore, in the processing of hyperspectral remote sensing images, unmixing of mixed spectra is one of the important links. Unmixing is to obtain the spectral response value of the typical object spectrum (endmember spectrum) in the hyperspectral remote sensing image, as well as the proportion of each endmember spectral response value in each pixel (abundance coefficient). Due to the diversity of object components in the natural environment and in most observation scenes, the composition relationship of different objects in the scene is not a priori. The current mainstream method is to perform mixed pixel decomposition in an unsupervised manner, including two steps: endmember spectrum extraction and abundance coefficient estimation. Some researchers introduced the constraint of minimizing the volume of a single shape into the NMF model and proposed an NMF method with minimum volume constraint; some studies introduced weighted sparse regularization terms to enhance the sparsity of abundance in the NMF model, and proposed a total variation regularized weighted sparse NMF unsupervised unmixing method; some studies proposed the maximum entropy method for unsupervised mixed pixel decomposition, which maximizes the entropy of abundance to obtain the optimal solution under the constraints, and its effect is better than the least squares method.

[0003] Most existing spectral unmixing methods only use hyperspectral data itself, or use hyperspectral data from different platforms and high-resolution panchromatic and multispectral data for super-resolution processing to extract endmember spectra and then realize mixed pixel decomposition. Among them, the algorithms that only use hyperspectral data itself still have room for improvement in the accuracy of endmember extraction and abundance coefficient calculation due to the limitation of low spatial resolution. However, the mixed pixel decomposition using hyperspectral and panchromatic (or multispectral) data from different platforms faces a series of problems such as image registration errors and pure pixel spectrum extraction errors due to the difficulty of completely consistent temporal and spatial synchronization of data. For example, different objects may have the same gray level in the panchromatic image, resulting in very limited improvement in the accuracy of endmember extraction and abundance coefficient calculation. Summary of the invention

[0004] (I) Purpose of the invention

[0005] The purpose of the present invention is to propose a method and system for unmixing ground objects based on the fusion of hyperspectral and high-resolution information on the same platform, so as to solve the problems of unsatisfactory accuracy of hyperspectral mixed pixel decomposition method, endmember extraction and abundance coefficient calculation.

[0006] (II) Technical solution

[0007] In order to achieve the above objectives and solve the above technical problems, the technical solutions of the present invention are as follows:

[0008] A ground object unmixing system based on the fusion of hyperspectral and high-resolution information on the same platform, including a data preprocessing module, a high-precision registration module, a hyperspectral image super-resolution reconstruction module, an end member spectrum extraction module, and a super-resolution abundance coefficient matrix extraction module;

[0009] When the ground object unmixing system performs fusion unmixing, first, the hyperspectral image and high-resolution panchromatic image acquired on the same platform enter the data preprocessing module to improve the image quality;

[0010] The quality-enhanced hyperspectral images and high-resolution panchromatic images will be sent to the high-precision registration module for optimal spectral segment selection and feature matching;

[0011] The hyperspectral data after the data preprocessing module will be sent to the endmember spectrum extraction module for unsupervised endmember spectrum extraction and output endmember spectrum matrix;

[0012] The data after the high-precision registration module will enter the hyperspectral super-resolution reconstruction module for transformation parameter estimation and hyperspectral image difference super-resolution to obtain a hyperspectral super-resolution image fused with the panchromatic image;

[0013] The endmember spectral data output by the hyperspectral super-resolution image and the endmember spectral extraction module simultaneously enter the super-resolution abundance coefficient matrix extraction module to obtain the overall abundance coefficient of the hyperspectral super-resolution image and output the abundance coefficient matrix.

[0014] Furthermore, the data preprocessing module inputs the hyperspectral image and high-resolution panchromatic image acquired on the same platform. The data preprocessing module completes the functions of missing row supplement, dark current elimination, radiation correction, detection and elimination of bad pixels for the hyperspectral and high-resolution panchromatic images to improve the image quality; the hyperspectral data output by the data preprocessing module is sent to the high-precision registration module, and is also sent to the hyperspectral image endmember spectrum extraction module.

[0015] Furthermore, the input of the high-precision registration module is the hyperspectral image and high-resolution panchromatic image data that have undergone the data preprocessing module; the high-precision registration module completes the functions of optimal spectral band selection, point feature extraction, and feature matching of the hyperspectral image to achieve high-precision registration of the hyperspectral image and the high-resolution panchromatic image; the same platform data uses the known geometric model of the camera to effectively assist feature extraction and feature matching, and the output data is sent to the hyperspectral super-resolution reconstruction module.

[0016] Furthermore, the specific working process of the high-precision registration module is as follows:

[0017] (1) Taking advantage of the inherent sparse characteristics of hyperspectral images, the sparse principal component analysis method is used to select the optimal spectrum segment;

[0018] (2) Considering that the spatial resolution of hyperspectral images is low and it is not easy to directly obtain point features, the Canny edge detection algorithm is used to perform edge detection, obtain cross lines for the contour of a specific object, and then extract point features;

[0019] (3) Using the acquired point features and camera geometry model, feature matching is performed based on the SIFT algorithm. The specific steps are: selecting a suitable similarity metric based on the image feature points and feature descriptors obtained in the feature extraction stage, traversing the feature descriptors of the optimal spectrum of the hyperspectral image and the panchromatic reference image, and then calculating the distance between the descriptors. The image feature points are sorted in descending order according to the distance value, and a certain threshold is selected. The first N features within the threshold range are selected as the result of feature matching to establish a corresponding matching relationship for the feature sets extracted from the hyperspectral image and the panchromatic reference image. The spatial position relationship of the features is usually used to eliminate false matches.

[0020] Furthermore, the hyperspectral image super-resolution reconstruction module inputs the data after the high-precision registration module; the output hyperspectral super-resolution image enters the super-resolution abundance coefficient matrix extraction module; the specific steps are as follows:

[0021] (1) Establish a change model based on the high-precision registration results of hyperspectral images and high-resolution panchromatic images, and estimate the transformation parameters;

[0022] (2) Using the estimated values ​​of the transformation parameters obtained in the above steps, the hyperspectral image is sampled and some pixels on the image are interpolated. The hyperspectral image is aligned with the panchromatic reference image to obtain a hyperspectral super-resolution image fused with the panchromatic image.

[0023] Furthermore, the input of the hyperspectral image endmember spectrum extraction module is the hyperspectral image after the data preprocessing module; the hyperspectral image endmember spectrum extraction module utilizes the sparsity characteristics of the hyperspectral image and simultaneously performs endmember extraction and endmember number estimation based on the adaptive dictionary learning method to obtain endmember spectrum information; the output endmember spectrum matrix enters the super-resolution abundance coefficient matrix extraction module.

[0024] Furthermore, the specific working process of the hyperspectral image endmember spectrum extraction module is as follows:

[0025] (1) Expand the hyperspectral image 3D cube according to the scan line to obtain the hyperspectral image pixel spectral matrix X;

[0026] (2) Using the adaptive dictionary learning framework, we solve the endmember extraction objective function and obtain the endmember spectrum matrix:

[0027]

[0028] in, is the end member spectrum indicator vector, where a i,j is the abundance coefficient vector a of the i-th pixel i The abundance coefficient value of the jth band in the k ]∈R L×k represents the endmember matrix, each column represents an endmember, and k is the number of all endmembers contained in the hyperspectral image; the endmember spectral indicator vector uses the zero element counting method to measure the importance of the spectral endmembers obtained in the learning, which is specifically expressed in the last item in formula (2), where is the dictionary scale penalty term, constraining the number of endmember spectra, μ is the balance parameter; the indicator function is defined as

[0029]

[0030] The non-zero vector output is 1, the sum of all end member spectral indicator functions, that is, the dictionary scale penalty term Indicates the number of atoms in the endmember spectrum dictionary actually used;

[0031] 1) Using the multivariate Morrow approximation index (MMPI) penalty term to optimize the solution of formula (1) multivariate indicator term The optimized objective function is obtained:

[0032]

[0033] 2) Fix D and a alternately, and minimize the other when one is fixed, that is, optimize D and a interactively;

[0034] ① When a is fixed, the objective function (2) is transformed into a dictionary update problem, expressed as

[0035]

[0036] The optimization problem (6) is iteratively updated and solved using the gradient descent algorithm, and finally D is obtained;

[0037] ② When D is fixed, the objective function (2) is transformed into a dictionary update problem about a. Substituting formula (4) into it, it is expressed as

[0038]

[0039] Similar to is the corresponding row vector of matrix T, t j represents a column vector, There are two variables a and t in the optimization problem. The problem is decomposed into two sub-problems that have closed-form solutions.

[0040] For a, since the abundance coefficients of each pixel are independent, the optimization problem is equivalent to

[0041]

[0042] Solved by iterative shrinkage threshold method;

[0043] For t, due to the index j, the optimization problem is equivalent to

[0044]

[0045] ③ Repeatedly iterate ①② until the dictionary update converges, and finally obtain the endmember spectral matrix D.

[0046] Furthermore, the super-resolution abundance coefficient matrix extraction module inputs the endmember spectrum extraction module, outputs the endmember spectrum matrix and the hyperspectral super-resolution image output by the hyperspectral super-resolution reconstruction module; the super-resolution abundance coefficient matrix extraction module uses spatial prior knowledge to solve the abundance coefficient matrix of the super-resolution hyperspectral image; the corresponding abundance coefficient vector of each mixed pixel is considered to be sparse, and the sparse coding method is used to solve the abundance coefficient matrix; the output is the abundance coefficient matrix.

[0047] Furthermore, the working process of the super-resolution abundance coefficient matrix extraction module includes the following steps:

[0048] (1) Approximately segment the hyperspectral super-resolution image into regions with similar features, and roughly divide the entire image into M regions based on the uniform distribution of features in the panchromatic image;

[0049] (2) In each region obtained by segmentation, the TV term is introduced in the unmixing process, and the abundance coefficient matrix of each region is solved by the following minimization problem:

[0050]

[0051] in,

[0052]

[0053] It is the vector extension of the anisotropic TV, which ensures that the abundance values ​​of the same end members in the neighborhood pixels change smoothly;

[0054] The minimization problem is regarded as a constrained basis pursuit denoising problem considering neighborhood information, and is solved using the total variation-variable separation augmented Lagrangian unmixing method;

[0055] 1) Define H h : R k×n→R k×n is a linear operator used to calculate A Mi The difference between the abundance of each pixel in the horizontal neighboring pixel and its corresponding abundance. Similarly, define H v : R k×n →R k×n is the abundance difference in the vertical direction;

[0056] According to the above definition, let

[0057]

[0058] Then formula (8) is expressed as

[0059]

[0060] In the formula, is the indicator function, if If it belongs to the non-negative quadrant, the function value is zero, otherwise it is positive infinity;

[0061] 2) For equation (11), in order to solve it using the ADMM method, equation (11) is equivalently expressed as

[0062]

[0063] Formula (12) can be expressed in a more concise form as follows:

[0064]

[0065] in,

[0066]

[0067] Rewrite equation (15) into augmented Lagrangian form, and we have

[0068]

[0069] Among them, μ is a positive constant, Q / μ is the Lagrange multiplier of the constraint condition GU+PV=0; Finally, by solving equation (16), the abundance coefficient matrix of the hyperspectral super-resolution image Mi region is obtained;

[0070] (3) Traverse all areas to obtain the overall abundance coefficient matrix of the hyperspectral super-resolution image.

[0071] (III) Effective income

[0072] 1. The ground object unmixing method based on the fusion of hyperspectral and high-resolution information on the same platform proposed in the present invention can simultaneously estimate the number of endmembers and extract endmember spectra, thereby improving the accuracy of endmember estimation and the generalization of the algorithm in different scenarios.

[0073] 2. The object unmixing method based on the fusion of hyperspectral and high-resolution information on the same platform proposed in the present invention utilizes the advantage that the high-resolution panchromatic image and the hyperspectral image are on the same platform, and the image preprocessing accuracy is higher than that of non-same platform, so as to perform high-precision alignment and improve the image fusion accuracy.

[0074] 3. The object unmixing method based on the fusion of hyperspectral and high-resolution information on the same platform proposed in the present invention is based on the fused hyperspectral super-resolution image. By introducing spatial prior knowledge and using the sparse coding method to solve the super-resolution abundance coefficient matrix, the decomposition accuracy of mixed pixels in the hyperspectral image is ultimately improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 Schematic diagram of the system composition and method flow of the present invention. DETAILED DESCRIPTION

[0076] The following is combined with Figure 1 , the specific implementation process of the present invention is explained and illustrated in detail.

[0077] The closest prior art to the present invention is a method and device for decomposing mixed pixels of a high-spectral image in collaboration with a high-resolution image (CN117437479A). The invention obtains a high-resolution image that accurately corresponds to the hyperspectral image through image registration, extracts endmember bundles of the hyperspectral image, and then determines the number of endmembers, endmember spectra, and ground object categories of the hyperspectral image in collaboration with the high-resolution image. The high-resolution image is then subjected to hierarchical classification and fusion to obtain an accurate ground object classification result of the high-resolution image. For each pixel in the hyperspectral image, its pixel abundance value is determined according to the classification result of the corresponding matching area in the registered high-resolution image, thereby obtaining a more reliable mixed pixel decomposition result. The invention uses traditional interactive methods such as PPI when extracting endmember spectra, and uses a non-negative matrix decomposition method to solve the abundance coefficient on the basis of ground object segmentation.

[0078] With the continuous development of satellite platforms and optical payload technology, the simultaneous acquisition of hyperspectral images and high-resolution panchromatic images based on the same platform has become a future development trend. The improvement of its configuration level will further improve the image fusion effect and increase the accuracy of hyperspectral mixed pixel decomposition. To this end, the ground object unmixing system based on the fusion of hyperspectral and high-resolution information on the same platform proposed in the present invention has better adaptability and automation level, and makes full use of the high-precision registration characteristics of hyperspectral data and high-resolution panchromatic image data on the same platform and the sparsity characteristics of hyperspectral images. Among them, in view of the disadvantage that the hyperspectral endmember extraction algorithm needs to estimate the number of image endmembers in advance, an endmember spectrum extraction method based on adaptive dictionary learning is proposed, which makes full use of the sparsity characteristics of hyperspectral images for endmember extraction, improves the accuracy of endmember estimation and the generalization of the algorithm under different scenarios. In addition, based on the fused hyperspectral super-resolution image, by introducing spatial prior knowledge, the super-resolution abundance coefficient matrix is ​​solved by using the sparse coding method, and finally the decomposition accuracy of mixed pixels of the hyperspectral image is improved without the need to be based on ground object segmentation. At the same time, CN114399688A is not aimed at the unmixing of objects fused with high-resolution information on the same platform, so it uses spatial information by classifying high-resolution images in layers, and the spatial information of the hyperspectral image itself is not taken into consideration, so the registration accuracy is limited, resulting in subsequent unmixing only being carried out in uniformly classified objects, and the accuracy of end member extraction and abundance coefficient estimation is limited at the algorithm level, so this method cannot be transplanted to the same platform data that can be assisted by the camera geometric model for high-precision registration. Therefore, the present invention uses the advantages of the known and stable camera geometric model for the scene of high-resolution panchromatic images and hyperspectral images on the same platform to perform high-precision full-image registration to improve the image fusion accuracy. In order to maximize the advantages of high-precision image fusion, the present invention takes into account the spatial information of the hyperspectral image after super-resolution, and uses the total variation method to improve the global unmixing accuracy, which can be better applied to the scene of unmixing objects fused with high-resolution information on the same platform.

[0079] The present invention provides a method for unmixing ground objects based on the fusion of hyperspectral and high-resolution information on the same platform, including data preprocessing, high-precision registration, hyperspectral image super-resolution reconstruction, endmember spectrum extraction, and super-resolution abundance coefficient matrix extraction. The above steps are the design ideas of the present invention: first, the inherent sparse characteristics of hyperspectral images are utilized to obtain endmember spectra from hyperspectral images through an endmember spectrum extraction algorithm based on adaptive dictionary learning; then, the hyperspectral image is accurately registered with the high-resolution panchromatic image on the same platform, and on this basis, super-resolution reconstruction of the hyperspectral image is achieved; based on the super-resolution reconstruction of the hyperspectral image, the extracted endmember spectra are used to introduce spatial information as prior knowledge into the solution of the abundance coefficient matrix of the hyperspectral image, thereby obtaining a more accurate abundance coefficient extraction result of the hyperspectral image after super-resolution reconstruction.

[0080] The invention discloses a ground object unmixing system based on the fusion of hyperspectral and high-resolution information on the same platform, comprising a data preprocessing module, a high-precision registration module, a hyperspectral image super-resolution reconstruction module, an end-member spectrum extraction module, and a super-resolution abundance coefficient matrix extraction module. When the system is fused and unmixed, first, the hyperspectral image and the high-resolution panchromatic image acquired on the same platform enter the data preprocessing module for image quality improvement; the hyperspectral image and the high-resolution panchromatic image after quality improvement will be sent to the high-precision registration module for optimal spectrum selection and feature matching, in addition, the hyperspectral data after the data preprocessing module will be sent to the end-member spectrum extraction module for unsupervised end-member spectrum extraction and output of the end-member spectrum matrix; the data after the high-precision registration module will enter the hyperspectral super-resolution reconstruction module for transformation parameter estimation and hyperspectral image difference super-resolution to obtain the hyperspectral super-resolution image fused with the panchromatic image; the hyperspectral super-resolution image and the end-member spectrum data output by the end-member spectrum extraction module will simultaneously enter the super-resolution abundance coefficient matrix extraction module to obtain the overall abundance coefficient of the hyperspectral super-resolution image, and output the abundance coefficient matrix.

[0081] 1. Data preprocessing module

[0082] The input is the hyperspectral image and high-resolution panchromatic image acquired on the same platform. The data preprocessing module mainly completes the functions of missing line supplementation, dark current elimination, radiation correction, detection and elimination of bad pixels of hyperspectral and high-resolution panchromatic images to improve the image quality. The output data is sent to the high-precision registration module, and the output hyperspectral data is also sent to the hyperspectral image end member spectrum extraction module.

[0083] 2. High-precision registration module

[0084] The input is the hyperspectral image and high-resolution panchromatic image data that have been processed by the data preprocessing module. The high-precision registration module mainly completes the functions of optimal spectral band selection, point feature extraction, feature matching, etc. of the hyperspectral image to achieve high-precision registration of the hyperspectral image and the high-resolution panchromatic image. Since the registration is performed on the hyperspectral data and high-resolution panchromatic data on the same platform, unlike the registration of data on different platforms, the data on the same platform can use the known geometric model of the camera to effectively assist feature extraction and feature matching. Compared with the data on different platforms whose features all rely on algorithm extraction, the feature extraction and matching using the camera geometric model can achieve higher accuracy. The output data is sent to the hyperspectral super-resolution reconstruction module.

[0085] (1) Taking advantage of the inherent sparse characteristics of hyperspectral images, the sparse principal component analysis method is used to select the optimal spectral segment.

[0086] (2) Considering that the spatial resolution of hyperspectral images is low and it is not easy to directly obtain point features, the Canny edge detection algorithm is used to perform edge detection, obtain the cross lines of the specific object contour, and then extract the point features.

[0087] (3) Using the acquired point features and camera geometry model, feature matching is performed based on the SIFT algorithm. The specific steps are: select a suitable similarity metric based on the image feature points and feature descriptors obtained in the feature extraction stage, traverse the feature descriptors of the optimal spectrum of the hyperspectral image and the panchromatic reference image, and then calculate the distance between the descriptors. According to the size of the distance value, the image feature points are sorted in descending order, and a certain threshold is selected. The first N features within the threshold range are selected as the result of feature matching to establish a corresponding matching relationship for the feature sets extracted from the hyperspectral image and the panchromatic reference image. The spatial position relationship of the features is usually used to eliminate erroneous matches, thereby establishing a globally consistent feature matching relationship in the two images, and achieving high-precision registration of the hyperspectral image and the high-resolution panchromatic image.

[0088] 3. Hyperspectral image super-resolution reconstruction module

[0089] The input is the data after the high-precision registration module. The hyperspectral image super-resolution reconstruction module mainly establishes a change model based on the registration results, estimates the transformation parameters, and interpolates the hyperspectral image to achieve super-resolution reconstruction of the hyperspectral image fused with the high-resolution panchromatic image. The output hyperspectral super-resolution image enters the super-resolution abundance coefficient matrix extraction module.

[0090] (1) Based on the high-precision registration results of hyperspectral images and high-resolution panchromatic images, a change model is established to estimate the transformation parameters.

[0091] (2) Using the estimated values ​​of the transformation parameters obtained in the above steps, the hyperspectral image is sampled and some pixels on the image are interpolated. The hyperspectral image is aligned with the panchromatic reference image to obtain a hyperspectral super-resolution image fused with the panchromatic image.

[0092] 4. Hyperspectral image endmember spectrum extraction module

[0093] The input is the hyperspectral image after the data preprocessing module. The hyperspectral image endmember spectrum extraction module mainly utilizes the sparsity characteristics of hyperspectral images and simultaneously extracts and estimates the number of endmembers based on the adaptive dictionary learning method to obtain the endmember spectrum information. The output endmember spectrum matrix enters the super-resolution abundance coefficient matrix extraction module.

[0094] (1) Expand the hyperspectral image 3D cube according to the scan line to obtain the hyperspectral image pixel spectral matrix X;

[0095] (2) Using the adaptive dictionary learning framework, we solve the endmember extraction objective function and obtain the endmember spectrum matrix:

[0096]

[0097] in, is the background endmember indicator vectors (BEIV), where a i,j is the abundance coefficient vector a of the i-th pixel i The abundance coefficient value of the jth band in the k ]∈R L×k represents the endmember matrix, each column represents an endmember, and k is the number of all endmembers contained in the hyperspectral image. The endmember spectral indicator vector can use the zero element counting method to measure the importance of the spectral endmembers obtained in the learning. The specific form is the last item in formula (2), where is the dictionary scale penalty term, constraining the number of endmember spectra, and μ is the balance parameter. The indicator function is defined as

[0098]

[0099] Non-zero vectors are output as 1. Therefore, the sum of all end-member spectral indicator functions, i.e., the dictionary scale penalty term It can indicate the number of atoms in the endmember spectrum dictionary actually used.

[0100] 1) Using the multivariate Morrow approximation index (MMPI) penalty term to optimize the solution of formula (1) multivariate indicator term The optimized objective function is obtained:

[0101]

[0102] 2) Fix D and a alternately, and minimize the other when one is fixed, that is, optimize D and a interactively.

[0103] ① When a is fixed, the objective function (2) is transformed into a dictionary update problem, which can be expressed as

[0104]

[0105] The optimization problem (6) is iteratively updated and solved using the gradient descent algorithm, and finally D is obtained.

[0106] ② When D is fixed, the objective function (2) is transformed into a dictionary update problem about a. After substituting formula (4), it can be expressed as

[0107]

[0108] Similar to is the corresponding row vector of matrix T, t j represents a column vector, The optimization problem (7) has two variables a and t. The problem can be decomposed into two sub-problems, both of which have closed-form solutions.

[0109] For a, since the abundance coefficients of each pixel are independent, the optimization problem (7) can be equivalent to

[0110]

[0111] The above problem can be solved by iterative shrinkage threshold method.

[0112] For t, since for index j, the optimization problem (7) can be equivalent to

[0113]

[0114] Formula (10) is a standard MMPI penalty term and can be solved directly.

[0115] ③ Repeatedly iterate ①② until the dictionary update converges, and finally obtain the endmember spectral matrix D.

[0116] 5. Super-resolution abundance coefficient matrix extraction module

[0117] The input is the endmember spectrum matrix output by the endmember spectrum extraction module and the hyperspectral super-resolution image output by the hyperspectral super-resolution reconstruction module. The super-resolution abundance coefficient matrix extraction module mainly uses spatial prior knowledge to solve the abundance coefficient matrix of the super-resolution hyperspectral image. Considering the distribution characteristics of ground objects in actual scenes, although the entire hyperspectral image scene contains more ground object endmembers, each mixed pixel is usually composed of a finite number (2 to 3) of endmembers. Therefore, the corresponding abundance coefficient vector of each mixed pixel can be considered to be sparse, and the sparse coding method can be used to solve the abundance coefficient matrix. The output is the abundance coefficient matrix.

[0118] (1) Considering that similar objects in hyperspectral images are often concentrated, in order to improve the calculation accuracy of the abundance coefficient, the hyperspectral super-resolution image can first be approximately segmented into similar object regions, and the entire image can be roughly divided into M regions based on the uniform distribution of objects in the panchromatic image.

[0119] (2) In each segmented region, considering the relationship between each mixed pixel and its neighboring pixels, the abundance values ​​of the same end members should be smoothly transitioned, so the TV term is introduced in the unmixing process. Then the abundance coefficient matrix of the region can be solved by the following minimization problem:

[0120]

[0121] in,

[0122]

[0123] It is the vector extension of the anisotropic TV, ensuring that the abundance values ​​of the same end members in the neighborhood pixels change smoothly.

[0124] The minimization problem can be viewed as a constrained basis pursuit denoising (CBPDN) problem that takes neighborhood information into account and can be solved using the total variation-variable separation augmented Lagrangian unmixing method.

[0125] 1) Define H h : R k×n →R k×n is a linear operator used to calculate A Mi The difference between the abundance of each pixel in the horizontal neighboring pixel and its corresponding abundance. Similarly, define H v : R k×n →R k×n is the abundance difference in the vertical direction.

[0126] According to the above definition, let

[0127]

[0128] Then problem (8) can be expressed as

[0129]

[0130] In the formula, is the indicator function, if If it belongs to the non-negative quadrant, the function value is zero, otherwise it is positive infinity.

[0131] 2) For problem (11), in order to solve it using the ADMM method, it can be equivalently expressed as

[0132]

[0133] Problem (12) can be expressed in a more concise form as follows:

[0134]

[0135] in,

[0136]

[0137] Rewriting equation (15) into its augmented Lagrangian form, we have

[0138]

[0139] Among them, μ is a positive constant, Q / μ is the Lagrange multiplier of the constraint condition GU+PV=0. Finally, the abundance coefficient matrix of the hyperspectral super-resolution image Mi region can be obtained by solving equation (16).

[0140] (3) By traversing all areas, the overall abundance coefficient matrix of the hyperspectral super-resolution image can be obtained.

[0141] The above contents are further detailed descriptions of the present invention in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A ground object unmixing system based on the fusion of hyperspectral and high-resolution information on the same platform, characterized in that: The system includes a data preprocessing module, a high-precision registration module, a hyperspectral image super-resolution reconstruction module, an end member spectrum extraction module, and a super-resolution abundance coefficient matrix extraction module; When the ground object unmixing system performs fusion unmixing, first, the hyperspectral image and high-resolution panchromatic image acquired on the same platform enter the data preprocessing module to improve the image quality; The quality-enhanced hyperspectral images and high-resolution panchromatic images will be sent to the high-precision registration module for optimal spectral segment selection and feature matching; The hyperspectral data after the data preprocessing module will be sent to the endmember spectrum extraction module for unsupervised endmember spectrum extraction and output endmember spectrum matrix; The data after the high-precision registration module will enter the hyperspectral super-resolution reconstruction module for transformation parameter estimation and hyperspectral image difference super-resolution to obtain a hyperspectral super-resolution image fused with the panchromatic image; The endmember spectral data output by the hyperspectral super-resolution image and the endmember spectral extraction module simultaneously enter the super-resolution abundance coefficient matrix extraction module to obtain the overall abundance coefficient of the hyperspectral super-resolution image and output the abundance coefficient matrix.

2. The ground object unmixing system based on the fusion of hyperspectral and high-resolution information on the same platform according to claim 1 is characterized in that: The data preprocessing module inputs the hyperspectral image and high-resolution panchromatic image acquired on the same platform. The data preprocessing module completes the functions of missing line supplementation, dark current elimination, radiation correction, detection and elimination of bad pixels of the hyperspectral and high-resolution panchromatic images to improve the image quality; the hyperspectral data output by the data preprocessing module is sent to the high-precision registration module, and is also sent to the hyperspectral image end member spectrum extraction module.

3. The ground object unmixing system based on the fusion of hyperspectral and high-resolution information on the same platform according to claim 1 is characterized in that: The input of the high-precision registration module is the hyperspectral image and high-resolution panchromatic image data that have passed the data preprocessing module; the high-precision registration module completes the functions of optimal spectral band selection, point feature extraction, and feature matching of the hyperspectral image to achieve high-precision registration of the hyperspectral image and the high-resolution panchromatic image; the same platform data uses the known geometric model of the camera to effectively assist feature extraction and feature matching, and the output data is sent to the hyperspectral super-resolution reconstruction module.

4. The ground object unmixing system based on the fusion of hyperspectral and high-resolution information on the same platform according to claim 3 is characterized in that: The specific working process of the high-precision registration module is as follows: (1) Taking advantage of the inherent sparse characteristics of hyperspectral images, the sparse principal component analysis method is used to select the optimal spectrum segment; (2) Considering that the spatial resolution of hyperspectral images is low and it is not easy to directly obtain point features, the Canny edge detection algorithm is used to perform edge detection, obtain cross lines for the contour of a specific object, and then extract point features; (3) Using the acquired point features and camera geometry model, feature matching is performed based on the SIFT algorithm. The specific steps are: selecting a suitable similarity metric based on the image feature points and feature descriptors obtained in the feature extraction stage, traversing the feature descriptors of the optimal spectrum of the hyperspectral image and the panchromatic reference image, and then calculating the distance between the descriptors. The image feature points are sorted in descending order according to the distance value, and a certain threshold is selected. The first N features within the threshold range are selected as the result of feature matching to establish a corresponding matching relationship for the feature sets extracted from the hyperspectral image and the panchromatic reference image. The spatial position relationship of the features is usually used to eliminate false matches.

5. The ground object unmixing system based on the fusion of hyperspectral and high-resolution information on the same platform according to claim 1 is characterized in that: The hyperspectral image super-resolution reconstruction module inputs the data after the high-precision registration module; the output hyperspectral super-resolution image enters the super-resolution abundance coefficient matrix extraction module; the specific steps are as follows: (1) Establish a change model based on the high-precision registration results of hyperspectral images and high-resolution panchromatic images, and estimate the transformation parameters; (2) Using the estimated values ​​of the transformation parameters obtained in the above steps, the hyperspectral image is sampled and some pixels on the image are interpolated. The hyperspectral image is aligned with the panchromatic reference image to obtain a hyperspectral super-resolution image fused with the panchromatic image.

6. The ground object unmixing system based on the fusion of hyperspectral and high-resolution information on the same platform according to claim 1 is characterized in that: The input of the hyperspectral image endmember spectrum extraction module is the hyperspectral image after the data preprocessing module; the hyperspectral image endmember spectrum extraction module utilizes the sparsity characteristics of the hyperspectral image and simultaneously performs endmember extraction and endmember number estimation based on an adaptive dictionary learning method to obtain endmember spectrum information; the output endmember spectrum matrix enters the super-resolution abundance coefficient matrix extraction module.

7. The ground object unmixing system based on the fusion of hyperspectral and high-resolution information on the same platform according to claim 6 is characterized in that: The specific working process of the hyperspectral image endmember spectrum extraction module is as follows: (1) Expand the hyperspectral image 3D cube according to the scan line to obtain the hyperspectral image pixel spectral matrix X; (2) Using the adaptive dictionary learning framework, we solve the endmember extraction objective function and obtain the endmember spectrum matrix: in, is the end member spectrum indicator vector, where a i,j is the abundance coefficient vector a of the i-th pixel i The abundance coefficient value of the jth band in the k ]∈R L×k represents the endmember matrix, each column represents an endmember, and k is the number of all endmembers contained in the hyperspectral image; the endmember spectral indicator vector uses the zero element counting method to measure the importance of the spectral endmembers obtained in the learning, which is specifically expressed in the last item in formula (2), where is the dictionary scale penalty term, constraining the number of endmember spectra, μ is the balance parameter; the indicator function is defined as The non-zero vector output is 1, the sum of all end member spectral indicator functions, that is, the dictionary scale penalty term Indicates the number of atoms in the endmember spectrum dictionary actually used; 1) Using the multivariate Morrow approximation index (MMPI) penalty term to optimize the solution of formula (1) multivariate indicator term The optimized objective function is obtained: 2) Fix D and a alternately, and minimize the other when one is fixed, that is, optimize D and a interactively; ① When a is fixed, the objective function (2) is transformed into a dictionary update problem, expressed as The optimization problem (6) is iteratively updated and solved using the gradient descent algorithm, and finally D is obtained; ② When D is fixed, the objective function (2) is transformed into a dictionary update problem about a. Substituting formula (4) into it, it is expressed as Similar to is the corresponding row vector of matrix T, t j represents a column vector, There are two variables a and t in the optimization problem. The problem is decomposed into two sub-problems that have closed-form solutions. For a, since the abundance coefficients of each pixel are independent, the optimization problem is equivalent to Solved by iterative shrinkage threshold method; For t, due to the index j, the optimization problem is equivalent to ③ Repeatedly iterate ①② until the dictionary update converges, and finally obtain the endmember spectral matrix D.

8. The ground object unmixing system based on the fusion of hyperspectral and high-resolution information on the same platform according to claim 1 is characterized in that: The super-resolution abundance coefficient matrix extraction module inputs the endmember spectrum extraction module, the endmember spectrum matrix outputted by the endmember spectrum extraction module, and the hyperspectral super-resolution image outputted by the hyperspectral super-resolution reconstruction module; the super-resolution abundance coefficient matrix extraction module uses spatial prior knowledge to solve the abundance coefficient matrix of the hyperspectral image after super-resolution; The corresponding abundance coefficient vector of each mixed pixel is considered to be sparse, and the abundance coefficient matrix is ​​solved using the sparse coding method; the output is the abundance coefficient matrix.

9. The ground object unmixing system based on the fusion of hyperspectral and high-resolution information on the same platform according to claim 1 is characterized in that: The working process of the super-resolution abundance coefficient matrix extraction module includes the following steps: (1) Approximately segment the hyperspectral super-resolution image into regions with similar features, and roughly divide the entire image into M regions based on the uniform distribution of features in the panchromatic image; (2) In each region obtained by segmentation, the TV term is introduced in the unmixing process, and the abundance coefficient matrix of each region is solved by the following minimization problem: in, It is the vector extension of the anisotropic TV, which ensures that the abundance values ​​of the same end members in the neighborhood pixels change smoothly; The minimization problem is regarded as a constrained basis pursuit denoising problem considering neighborhood information, and is solved using the total variation-variable separation augmented Lagrangian unmixing method; 1) Define H h : R k×n →R k×n is a linear operator used to calculate AMi The difference between the abundance of each pixel in the horizontal neighboring pixel and its corresponding abundance. Similarly, define H v : R k×n →R k×n is the abundance difference in the vertical direction; according to the above definition, let Then formula (8) is expressed as In the formula, is the indicator function, if If it belongs to the non-negative quadrant, the function value is zero, otherwise it is positive infinity; 2) For equation (11), in order to solve it using the ADMM method, equation (11) is equivalently expressed as Formula (12) can be expressed in a more concise form as follows: in, Rewrite equation (15) into augmented Lagrangian form, and we have Among them, μ is a positive constant, Q / μ is the Lagrange multiplier of the constraint condition GU+PV=0; Finally, by solving equation (16), the abundance coefficient matrix of the hyperspectral super-resolution image Mi region is obtained; (3) Traverse all areas to obtain the overall abundance coefficient matrix of the hyperspectral super-resolution image.

10. A method for unmixing ground objects based on the fusion of hyperspectral and high-resolution information on the same platform, characterized in that: It includes data preprocessing, high-precision registration, hyperspectral image super-resolution reconstruction, endmember spectrum extraction, and super-resolution abundance coefficient matrix extraction, specifically including the following steps: Firstly, the inherent sparse characteristics of hyperspectral images are utilized to obtain endmember spectra from hyperspectral images through an endmember spectrum extraction algorithm based on adaptive dictionary learning. Accurately align the hyperspectral image with the high-resolution panchromatic image on the same platform, and on this basis, achieve super-resolution reconstruction of the hyperspectral image; Based on super-resolution reconstruction of hyperspectral images, the extracted endmember spectra are used to introduce spatial information as prior knowledge into the solution of the abundance coefficient matrix of hyperspectral images. Then, more accurate abundance coefficient extraction results of the hyperspectral image after super-resolution reconstruction can be obtained.

Images