Hyperspectral image mixed pixel decomposition method and device with high-resolution image cooperation

Abstract

The application discloses a hyperspectral image mixed pixel decomposition method and device cooperating with high-resolution images. The application firstly registers a hyperspectral image and a high-resolution image to obtain a high-resolution image corresponding to the hyperspectral image accurately and corresponding registration parameters, extracts an end member beam of the hyperspectral image, determines the end member number, the end member spectrum and the ground object category of the hyperspectral image by cooperating with the high-resolution image, classifies the high-resolution image in layers, solves the abundance of the hyperspectral image by using the classification result of the high-resolution image and the registration parameters of the hyperspectral image and the high-resolution image, and realizes the hyperspectral image mixed pixel decomposition by cooperating with the high-resolution image. The application utilizes the rich detail information provided by the high-resolution image to assist in the hyperspectral image mixed pixel decomposition, and provides a new idea for the hyperspectral image mixed pixel decomposition.

Description

[0001] Technology Neighborhood

[0002] This invention belongs to the field of image processing and relates to a method and apparatus for high-resolution image collaborative hyperspectral image hybrid pixel decomposition. Background Technology

[0003] Due to spatial resolution limitations and the complexity and diversity of land cover, mixed pixels are prevalent in hyperspectral images. To improve the accuracy of hyperspectral data applications, spectral unmixing is essential, which involves determining the typical land cover spectra (endmembers) in the hyperspectral image and the proportion (abundance) of each land cover in each pixel. Mixed pixel decomposition in hyperspectral images has been a research hotspot in the field of hyperspectral image processing, and various spectral unmixing methods have been continuously proposed. Currently, accurately determining the number of typical land cover categories in hyperspectral images remains a challenge. Furthermore, most existing spectral unmixing methods only utilize the hyperspectral data itself, and there is still room for improvement in the accuracy of endmember extraction and abundance calculation.

[0004] With the implementation of my country's High-Resolution Earth Observation System (HREOS) major project, my country is now able to simultaneously acquire hyperspectral and high-resolution images of the same region. High-resolution images provide the possibility for constructing publicly available, batch datasets with accurate annotations. The key to collaboratively solving the problem of hyperspectral image mixed pixel decomposition using high-resolution images of the same region lies in determining the high-resolution image region that precisely corresponds to the hyperspectral image and accurately classifying ground features within that region. This involves sub-pixel-level matching of multi-source images with large-scale and gray-level differences, as well as accurate segmentation of the high-resolution image. Currently, no research results have been reported on using high-resolution images to collaboratively solve the problem of hyperspectral image mixed pixel decomposition. This invention proposes a high-resolution image collaborative hyperspectral image mixed pixel decomposition method, which is expected to improve the accuracy of spectral unmixing. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing spectral variable unmixing algorithms by proposing a high-resolution image-coordinated hyperspectral image hybrid pixel decomposition method and apparatus. This method involves obtaining a high-resolution image precisely corresponding to the hyperspectral image through image registration. After endmember bundle extraction from the hyperspectral image, the number of endmembers, endmember spectra, and land cover categories of the hyperspectral image are determined through high-resolution image coordination. Then, hierarchical classification fusion is performed on the high-resolution image to obtain accurate land cover classification results. For each pixel in the hyperspectral image, its pixel abundance value is determined based on the classification results of its corresponding matching region in the registered high-resolution image, thus obtaining a more reliable hybrid pixel decomposition result. This result can also be used as a reference to better evaluate the performance of other spectral variable unmixing algorithms.

[0006] The technical solution adopted in this invention is as follows:

[0007] Step 1: Register the hyperspectral image and the high-resolution image;

[0008] Step 2: Endmember beam extraction from hyperspectral images;

[0009] Step 3: Collaboratively determine the number of endmembers, endmember spectra, and land cover categories of hyperspectral images using high-resolution images;

[0010] Step 4: Use a hierarchical classification and fusion method to classify ground features in high-resolution images:

[0011] Step 5: For each pixel in the hyperspectral image, calculate the pixel abundance based on the classification result of the corresponding matching region in the registered high-resolution image.

[0012] Furthermore, the registration method described in step 1 includes feature-based matching algorithms and deep learning-based matching algorithms.

[0013] Furthermore, the hyperspectral image endmember bundle extraction method described in step 2 includes an automatic endmember bundle extraction algorithm (EBE), an endmember bundle extraction algorithm combining PPI with spectral spatial information, and an endmember bundle extraction method based on superpixel segmentation and pixel purity index.

[0014] Furthermore, the specific process of collaboratively determining the number of endmembers, endmember spectra, and land cover categories of the hyperspectral image using high-resolution images in step 3 is as follows:

[0015] 3-1. Based on the high-resolution image scene, determine the typical land cover categories; and select typical endmembers corresponding to the scene from the spectral library, match the selected typical endmembers with the hyperspectral image, and determine the initial endmember subset E0 and the corresponding land cover categories.

[0016] 3-2. By matching and unmixing the endmembers in the hyperspectral image endmember bundle using the initial endmember subset E0, redundant endmembers in the hyperspectral image endmember bundle are removed.

[0017] 3-3. Cluster the endmembers retained in the endmember bundles of the hyperspectral image to obtain the number of categories M and the endmember subsets of each category; calculate the average spectrum of each endmember subset, and take the calculated M average spectra as the endmember subset E1 of the hyperspectral image. Merge the initial endmember subset E0 and the endmember subset E1 to obtain the endmember matrix E, where the number of endmembers N in E is the number of endmembers in the hyperspectral image.

[0018] 3-4. Determine the land cover category of endmember subset E1 by spectral matching of the spectral library;

[0019] Furthermore, the clustering method employs a clustering method that automatically determines the number of categories; the spectral matching method employs a similarity measurement method, including one or more of the following: Euclidean distance, spectral angular distance, correlation coefficient, and similarity coefficient.

[0020] Furthermore, the specific process of hierarchical classification fusion described in step 4 is as follows:

[0021] 4-1. Based on the land cover categories determined by the hyperspectral images, determine three classification levels: roads, roofs, and other land cover. Each classification level is assigned a label, where label 1 indicates that the classification level is included.

[0022] 4-2. If the road classification hierarchy label is 1, then perform binary classification of roads and non-roads on the high-resolution image to identify road regions;

[0023] 4-3. If the roof classification hierarchy label is 1, then perform roof and non-roof binary classification on the non-road area of ​​the high-resolution image to identify the roof area;

[0024] 4-4. Perform other ground feature classification and identification on non-road and non-roof areas of high-resolution images;

[0025] 4-5. Merge the classification results for roads, rooftops, and other land features;

[0026] Furthermore, this invention employs an improved UNet network for multi-classification of grassland, trees, water bodies, and bare land, obtaining segmentation results for these four elements. The SGCN semantic segmentation algorithm, which has high segmentation accuracy for fine-grained features, is used for binary classification of rooftops and roads, yielding segmentation results for rooftops and roads respectively. A segmentation result fusion method is then used to fuse the segmentation results of various features into the final high-resolution image segmentation result, achieving accurate segmentation of the high-resolution image.

[0027] Furthermore, the specific process for calculating pixel abundance in step 5 is as follows:

[0028] 5-1. For each pixel x in the hyperspectral image, calculate its matching region Mx in the high-resolution image;

[0029] 5-2. If the matching region Mx does not contain the background category, then count the number of pixels of each category in the classification results of the matching region Mx in the high-resolution image, calculate the proportion of pixels of each category in the matching region Mx, and use it as the abundance vector h of pixel x. x ;

[0030] 5-3. For the matching region Mx, the pixel set x consisting of m hyperspectral pixels of the background category. m The abundance of each pixel is calculated by partial nonnegative matrix decomposition, and the endmember matrix is ​​updated.

[0031] Furthermore, the specific process of calculating cell abundance through partial nonnegative matrix decomposition and updating the endmember matrix in step 5-3 is as follows:

[0032] ① Let t = 1, then denote the unknown endmember as e. t ;set up Let E be the endmember matrix;

[0033] ② To construct the updated endmember matrix, we add the unknown endmember e. t End-to-end matrix Image set x m Perform partial nonnegative matrix decomposition to obtain the endmember e. t and pixel set x m Abundance of each pixel in the image;

[0034] ③ Calculate the pixel set x m The reconstruction error is greater than a set threshold, and the pixels with reconstruction errors greater than a set threshold constitute a new pixel set x. m Update the m value and endmember matrix.

[0035] ④ Repeat steps ②-④ until the pixel set x m Empty.

[0036] A high-resolution image collaborative hyperspectral image mixing pixel decomposition device, including

[0037] The image registration module is used to register hyperspectral images and high-resolution images;

[0038] Endmember beam extraction module, used to extract endmember beams from hyperspectral images;

[0039] The image segmentation module is used to segment high-resolution images;

[0040] The feature category determination module is used to combine the endmember bundle extraction results and the high-resolution image scene to determine the types of features in the image;

[0041] The high-resolution image land cover classification module is used to classify land covers in high-resolution images using a hierarchical classification and fusion method.

[0042] Abundance Calculation Module: Used to count the number of pixels of each type of land cover in the matched high-resolution image for each pixel in the hyperspectral image, and to determine the pixel abundance based on the proportion of pixels of each type.

[0043] Furthermore, the high-resolution image collaborative hyperspectral image mixing pixel decomposition device also includes an application module, which uses the mixed pixel decomposition results as a reference result to evaluate the performance of other hyperspectral image demixing algorithms.

[0044] The beneficial effects of this invention are:

[0045] (1) The hierarchical classification fusion method proposed in this invention fully considers the precision requirements of different land cover classifications, the impact of large differences within some land cover categories, and the segmentation difficulties caused by occlusion of some land cover, which can improve the overall segmentation accuracy of high-resolution images.

[0046] (2) The method proposed in this invention for collaboratively determining the number of endmembers and typical endmembers of hyperspectral images using high-resolution images solves the problems of subjectivity in traditional endmember number determination and the variability of endmember spectra, thereby improving the accuracy of determining the number of endmembers and typical endmembers of hyperspectral images.

[0047] (3) The method proposed in this invention for determining the pixel abundance of each pixel in a hyperspectral image based on the classification result of the corresponding matching region in the registered high-resolution image provides a new approach for hybrid pixel decomposition.

[0048] (4) The application module of the high-resolution image collaborative hyperspectral image mixing pixel decomposition device proposed in this invention solves the problem that the current spectral unmixing performance evaluation method is not accurate enough, and can provide a reliable abundance value reference for the performance evaluation of other hyperspectral image unmixing algorithms. Attached Figure Description

[0049] Figure 1 Algorithm flowchart.

[0050] Figure 2(a) Hyperspectral image of Zhuhai-1.

[0051] Figure 2(b) Hyperspectral image from Gaofen-5

[0052] Figure 2(c) High-resolution image from Gaofen-2.

[0053] Figure 3 Registration results of hyperspectral and high-resolution images from Zhuhai-1 satellite.

[0054] Figure 4 Registration results of Gaofen-5 hyperspectral images and high-resolution images

[0055] Figure 5 High-resolution image segmentation results from Gaofen-2.

[0056] Figure 6 Endmember spectrum of Zhuhai-1 hyperspectral image.

[0057] Figure 7 Abundance solution results from Zhuhai-1 hyperspectral image.

[0058] Figure 8 Endmember spectra of hyperspectral images from Gaofen-5 satellite.

[0059] Figure 9 Abundance solution results from Gaofen-5 hyperspectral images. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the invention is described in detail below with reference to specific embodiments. Specific embodiments are described below to simplify the invention. However, it should be understood that the invention is not limited to the described embodiments, and various modifications are possible without departing from the basic principles; these equivalent forms also fall within the scope defined by the appended claims.

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0062] like Figure 1 The diagram shown is a flowchart illustrating the basic steps of an embodiment of the high-resolution image collaborative hyperspectral image hybrid pixel decomposition method invented.

[0063] Example 1

[0064] Step 1: Register the hyperspectral image and the high-resolution image.

[0065] The SIFT algorithm is used to solve for the registration parameters of the hyperspectral image and the high-resolution image, achieving accurate matching between the hyperspectral image and the high-resolution image. The formula for solving the registration parameters is as follows:

[0066]

[0067] Where (x,y) and (x′,y′) are feature point pairs obtained from the hyperspectral image and the high-resolution image using the SITF algorithm, respectively. a, b, c, d, e, and f represent the registration parameters to be solved. All registration parameters can be obtained from three pairs of non-collinear feature point pairs, where c and f represent the translation amounts in the horizontal and vertical directions, and a and e represent the scaling ratios in the horizontal and vertical directions.

[0068] Step 2: Endmember beam extraction from hyperspectral images;

[0069] Endmember bundle extraction was performed on the hyperspectral image using a PPI-based endmember extraction method that combines PPI with spectral spatial information. The PPI index of each pixel was obtained through the PPI endmember extraction method. Pixels with PPI indices greater than a threshold were identified as endmembers. The hyperspectral image was uniformly divided into blocks, and the endmembers retained in each block were further filtered using the HI index to obtain the hyperspectral image endmember bundles.

[0070] Preferably, the registration method described in step 1 can be any registration method, including feature-based matching algorithms such as SIFT, ORB and SURF, and deep learning-based matching algorithms such as D2Net, SuperPoint and R2D2.

[0071] Preferably, the hyperspectral image endmember bundle extraction method described in step 2 can employ any hyperspectral image endmember bundle extraction method, including the automatic endmember bundle extraction algorithm EBE, the endmember bundle extraction algorithm combining PPI with spectral spatial information, and the endmember bundle extraction method based on superpixel segmentation and pixel purity index, etc.

[0072] Preferably, the spectral matching method can employ any similarity measurement method, including one or more of the following: self-Euclidean distance, spectral angular distance, correlation coefficient, and similarity coefficient.

[0073] Step 3: Determine the number of endmembers, endmember spectra, and land cover categories in the hyperspectral image;

[0074] The specific process of collaboratively determining the number of endmembers, endmember spectra, and land cover categories in hyperspectral images using high-resolution images is as follows:

[0075] 3-1. Based on the high-resolution image scene, determine the typical land cover categories, and select typical endmembers corresponding to the scene from the spectral library for matching with the hyperspectral image. Use spectral angular distance and Euclidean distance as similarity measures for spectral matching to determine the initial endmember subset E0 and the corresponding land cover categories. The formulas for calculating spectral angular distance SAD and Euclidean distance ED are as follows:

[0076]

[0077] Where a represents the spectrum of a typical endmember in the spectral library, and b represents the spectrum of a pixel in the hyperspectral image.

[0078] 3-2. By matching and unmixing the endmembers in the hyperspectral image endmember bundle using the initial endmember subset E0, redundant endmembers in the hyperspectral image endmember bundle are removed.

[0079] 3-3. The density-based spatial clustering method (DBSCAN) is used to cluster the endmembers retained in the endmember bundle to obtain the number of categories M and the endmember subset of each category. The average spectrum of each endmember subset is calculated, and the calculated M average spectra are used as the endmember subset E1 of the hyperspectral image. The initial endmember subset E0 and the endmember subset E1 are merged to obtain the endmember matrix E, where the number of endmembers N in E is the number of endmembers in the hyperspectral image.

[0080] 3-4. Spectral angular distance and Euclidean distance are selected as similarity measurement methods, and the land cover category of the endmember subset E1 is determined by spectral matching with the spectral library;

[0081] Step 4: Use a hierarchical classification and fusion method to classify ground features in high-resolution images:

[0082] 4-1. An improved UNet algorithm is used to perform multi-classification of grassland, trees, water bodies, and bare land to obtain the predicted segmentation result R1.

[0083] 4-2. The SGCN network is used to segment the roof and the road, and the segmentation results of the roof prediction R2 and the road prediction R3 are obtained.

[0084] 4-3. The prediction results are sorted according to the segmentation precision requirements, with roads requiring the highest precision, followed by roofs. The ranking is determined as R1. <R2<R3。

[0085] 4-4. According to the segmentation refinement requirements, the segmentation results of different land features are merged. The fusion formula is as follows:

[0086]

[0087] Among them, R i The final predicted category for pixel i. Let i be the predicted category of pixel i in the segmentation result R1. Let i be the predicted category of pixel i in the segmentation result R2. represents the predicted class of pixel i in the segmentation result R3. A predicted class of 0 indicates that this pixel is in the background class.

[0088] The improved UNet network encoder uses a ResNet network, adds a PPM pyramid pooling module at the encoder and decoder connection to obtain global semantic features, and adds a channel and spatial dual attention mechanism CBAM module at each skip connection for detailed optimization, thereby improving the segmentation accuracy of the UNet network.

[0089] The improved UNet algorithm is used to perform multi-class classification of grassland, farmland, trees, water bodies, and bare land. The specific implementation process is as follows:

[0090] 4-1-1. Label the features to be segmented in the high-resolution image, create a segmentation dataset, and perform data augmentation to increase the number of samples, including horizontally flipping, vertically flipping, and random cropping of the image.

[0091] 4-1-2. Input the samples into the improved UNet network in batches to extract and fuse features layer by layer to obtain the final feature fusion result.

[0092] 4-1-3. Perform a softmax operation on the final feature fusion result to obtain the image segmentation result predicted by the improved UNet network.

[0093] 4-1-4. The cross-entropy loss function is used as the loss between the predicted image segmentation result and the standard segmentation map. The calculation formula is as follows:

[0094]

[0095] Where N represents the total number of pixels in the image, M represents the total number of image categories, and y ic This represents the probability that pixel i belongs to class c in the real case. If pixel i belongs to class c, then y ic =1, otherwise y ic =0, p ic This represents the probability that pixel i is predicted to belong to category c.

[0096] 4-1-5. Update the model parameters using gradient descent.

[0097] 4-1-6. Obtain the optimal model parameters. Using the optimal model parameters, multiple classifications of grassland, trees, water bodies, and bare land can be achieved, and the segmentation results of grassland, trees, water bodies, and bare land can be obtained.

[0098] Step 5: Calculate the pixel abundance of the hyperspectral image;

[0099] 5-1. For each pixel x in the hyperspectral image, calculate its matching region Mx in the high-resolution image, as shown in the following formula;

[0100]

[0101] Where (u,v) represents the position of pixel x, (u l ,v l (u) represents the top-left pixel of the matching region Mx. r ,v r () represents the bottom right pixel of the matching region Mx.

[0102] 5-2. If the matching region does not contain a background category, count the number of pixels of each category in the classification results of the matching region Mx in the high-resolution image, calculate the proportion of pixels of each category in the matching region Mx, and use it as the abundance vector h of the pixels. x h x =[h1,h2,…,h i ], i represents the total number of ground features, h i This represents the proportion of the i-th type of land cover in the matching region Mx.

[0103] 5-3. For the pixel set x consisting of m hyperspectral pixels of the matching region containing the background category. m The abundance of each pixel is calculated through partial nonnegative matrix decomposition, and the endmember matrix is ​​updated. The specific process is as follows:

[0104] ① Let t = 1, then denote the unknown endmember as e. t ;set up Let E be the endmember matrix;

[0105] ② To construct the updated endmember matrix, we add the unknown endmember e. t End-to-end matrix Image set x m Perform partial nonnegative matrix decomposition to obtain the endmember e. t and pixel set x m Abundance of each pixel in the image;

[0106] ③ Calculate the pixel set x m The reconstruction error is greater than a set threshold, and the pixels with reconstruction errors greater than a set threshold constitute a new pixel set x. m Update the m value and endmember matrix.

[0107] ④ Repeat steps ②-④ until the pixel set x m Empty.

[0108] Example 2:

[0109] The following uses hyperspectral images taken by the Zhuhai-1 satellite and high-resolution images taken by the Gaofen-2 satellite as examples to illustrate the specific implementation method. The image acquisition parameters are shown in Table 1:

[0110] Table 1: Image Acquisition Parameter Table

[0111] Image source Zhuhai No. 1 Gaofen-2 Spatial resolution 10 meters 3.2 meters Number of bands 32 4

[0112] For the Zhuhai-1 hyperspectral image, radiometric calibration and atmospheric correction were first performed. Figures 2(a) and 2(c) show the initial matching of the Zhuhai-1 hyperspectral image and the Gaofen-2 high-resolution image, where Figure 2(a) is the Zhuhai-1 hyperspectral image and Figure 2(c) is the Gaofen-2 high-resolution image. The registration parameters obtained using the SIFT algorithm are as follows:

[0113]

[0114] Figure 3 This is the result of registering and fusing two images. Figure 5 To obtain land cover classification results using a hierarchical classification fusion method on high-resolution images, the high-resolution images were collaboratively determined to have 7 endmembers in the hyperspectral images, and the land cover categories were trees, grassland, water bodies, bare land, roads, asphalt roofs, and corrugated steel roofs. Figure 6 This is the endmember spectrum of the Zhuhai-1 hyperspectral image. Figure 7 The results of abundance calculation for the Zhuhai-1 hyperspectral image. Figure 5The high-resolution image land cover classification results show a segmentation accuracy exceeding 92% compared to the true land cover categories. Figure 6 The seven types of land features extracted were typical land features present in the image. Furthermore, compared to standard land feature spectra, the extracted spectra met the criteria of a spectral angular distance of less than 0.2 and a minimum Euclidean distance, closely resembling standard spectra. Figure 7 In the abundance calculation results, the reconstruction error of the pixel abundance calculation results including background categories is less than 0.1, and the abundance map can accurately reflect the proportion of typical land features. Figure 5 , Figure 6 and Figure 7 The results show that the patented solution is effective.

[0115] Example 3:

[0116] The following uses hyperspectral images taken by the Gaofen-5 satellite and high-resolution images taken by the Gaofen-2 satellite as examples to illustrate the specific implementation method. The image acquisition parameters are shown in Table 2:

[0117] Table 2: Image Acquisition Parameter Table

[0118] Image source Gaofen-5 Gaofen-2 Spatial resolution 30 meters 3.2 meters Number of bands 330 4

[0119] For the Gaofen-5 hyperspectral image, radiometric calibration and atmospheric correction were first performed. After removing anomalous bands and bands significantly affected by water vapor, 275 effective bands remained. Figures 2(b) and 2(c) show the initially matched Gaofen-5 hyperspectral image and Gaofen-2 high-resolution image, where Figure 2(a) is the Gaofen-5 hyperspectral image and Figure 2(c) is the Gaofen-2 high-resolution image. The registration parameters obtained using the D2Net algorithm are:

[0120]

[0121] Figure 4 This is the result of registering and fusing two images. Figure 5 To obtain land cover classification results using a hierarchical classification fusion method on high-resolution images, the high-resolution images collaboratively determined the number of endmembers in the hyperspectral images to be 7, and the land cover categories were trees, grassland, water bodies, bare land, roads, asphalt roofs, and corrugated steel roofs. Figure 8 This is the endmember spectrum of the Gaofen-5 hyperspectral image. Figure 9 The result is the abundance solution for the Gaofen-5 hyperspectral image. Figure 5 The high-resolution image land cover classification results show a segmentation accuracy exceeding 92% compared to the true land cover categories. Figure 8 The seven types of land features extracted were typical land features present in the image. Furthermore, compared to standard land feature spectra, the extracted spectra met the criteria of a spectral angular distance of less than 0.4 and a minimum Euclidean distance, closely resembling standard spectra. Figure 9In the abundance calculation results, the reconstruction error of the pixel abundance calculation results including background categories is less than 0.1, and the abundance map can accurately reflect the proportion of typical land features. Figure 5 , Figure 8 and Figure 9 The results also demonstrate that the patented solution is effective.

Claims

1. A high-resolution image-coordinated hyperspectral image fusion pixel decomposition method, characterized in that... Includes the following steps: Step 1: Register the hyperspectral image and the high-resolution image; Step 2: Endmember beam extraction from hyperspectral images; Step 3: Collaboratively determine the number of endmembers, endmember spectra, and land cover categories of hyperspectral images using high-resolution images; Step 4: Use a hierarchical classification and fusion method to classify ground features in high-resolution images: Step 5: For each pixel in the hyperspectral image, calculate the pixel abundance based on the classification result of the corresponding matching region in the registered high-resolution image; Step 3 is as follows: 3-1. Based on the high-resolution image scene, determine the typical land cover categories; Then, typical endmembers corresponding to the scene are selected from the spectral library, and the selected typical endmembers are matched with hyperspectral images to determine the initial endmember subset E0 and the corresponding land cover category. 3-2. By matching and unmixing the endmembers in the hyperspectral image endmember bundle using the initial endmember subset E0, redundant endmembers in the hyperspectral image endmember bundle are removed. 3-3. Cluster the endmembers retained in the endmember bundles of the hyperspectral image to obtain the number of categories M and the endmember subsets of each category; calculate the average spectrum of each endmember subset, and use the calculated M average spectra as the endmember subset E1 of the hyperspectral image. Merge the initial endmember subset E0 and the endmember subset E1 to obtain the endmember matrix E, where E contains the number of endmembers. The number of endmembers in the hyperspectral image; 3-4. Determine the land cover category of endmember subset E1 by spectral matching of the spectral library; Step 4 is as follows: 4-1. Based on the land cover categories determined by the hyperspectral images, determine three classification levels: roads, roofs, and other land cover. Each classification level is assigned a label, where label 1 indicates that the classification level is included. 4-2. If the road classification hierarchy label is 1, then perform binary classification of roads and non-roads on the high-resolution image to identify road regions; 4-3. If the roof classification hierarchy label is 1, then perform roof and non-roof binary classification on the non-road area of ​​the high-resolution image to identify the roof area; 4-4. Perform other ground feature classification and identification on non-road and non-roof areas of high-resolution images; 4-5. Merge the classification results for roads, rooftops, and other land features; Step 5 is as follows: 5-1. For each pixel x in the hyperspectral image, calculate its matching region Mx in the high-resolution image; 5-2. If the matching region Mx does not contain the background category, then count the number of pixels of each category in the classification results of the matching region Mx in the high-resolution image, calculate the proportion of pixels of each category in the matching region Mx, and use it as the abundance vector h of pixel x. x ; 5-3. For the matching region Mx containing the background category A pixel set consisting of hyperspectral pixels The abundance of each pixel is calculated by partial nonnegative matrix decomposition, and the endmember matrix is ​​updated.

2. The high-resolution image collaborative hyperspectral image fusion pixel decomposition method according to claim 1, characterized in that... The registration method described in step 1 includes feature-based matching algorithms and deep learning-based matching algorithms.

3. The high-resolution image collaborative hyperspectral image fusion pixel decomposition method according to claim 1, characterized in that... The hyperspectral image endmember bundle extraction method described in step 2 includes an automatic endmember bundle extraction algorithm (EBE), an endmember bundle extraction algorithm combining PPI with spectral spatial information, and an endmember bundle extraction method based on superpixel segmentation and pixel purity index.

4. The high-resolution image collaborative hyperspectral image fusion pixel decomposition method according to claim 1, characterized in that... The clustering method employs an automatic clustering method to determine the number of categories; the spectral matching method employs a similarity measurement method, including one or more of the following: Euclidean distance, spectral angular distance, correlation coefficient, and similarity coefficient.

5. The high-resolution image collaborative hyperspectral image fusion pixel decomposition method according to claim 1, characterized in that... An improved UNet network is used to perform multi-class classification of grassland, trees, water bodies, and bare land, resulting in segmentation results for grassland, farmland, trees, water bodies, and bare land. The SGCN semantic segmentation algorithm, which has high segmentation accuracy for fine-grained features, is used to perform binary classification of rooftops and roads, resulting in segmentation results for rooftops and roads. A segmentation result fusion method is used to fuse the segmentation results of various features as the final high-resolution image segmentation result, achieving accurate segmentation of high-resolution images.

6. The high-resolution image collaborative hyperspectral image fusion pixel decomposition method according to claim 1, characterized in that... The specific process of calculating pixel abundance through partial nonnegative matrix decomposition and updating the endmember matrix in step 5-3 is as follows: ① Let Then denote the unknown terminator as ;set up Let E be the endmember matrix; ② To construct the updated endmember matrix, i.e., to add unknown endmembers. End-to-end matrix ; set of pixels Perform partial nonnegative matrix decomposition to obtain endmembers. and Abundance of each pixel in the image; ③ Calculation The reconstruction error is defined as the number of pixels whose reconstruction error exceeds a set threshold, and these pixels constitute a new set of pixels. ,renew Value and endmember matrix = , ; ④ Repeat steps ②-④ until the pixel set is reached. Empty.

7. A high-resolution image collaborative hyperspectral image mixing pixel decomposition device, characterized in that, The device is used to implement the method as described in claim 1, including The image registration module is used to register hyperspectral images and high-resolution images; Endmember beam extraction module, used to extract endmember beams from hyperspectral images; The image segmentation module is used to segment high-resolution images; The feature category determination module is used to combine the endmember bundle extraction results and the high-resolution image scene to determine the types of features in the image; A high-resolution image ground feature classification module is used to classify ground features in high-resolution images using a hierarchical classification and fusion method. Abundance Calculation Module: Used to count the number of pixels of each type of land cover in the matched high-resolution image for each pixel in the hyperspectral image, and to determine the pixel abundance based on the proportion of pixels of each type.

Images