Hyperspectral data annotation method based on unsupervised clustering

Abstract

The present invention discloses a hyperspectral data labeling method, which mainly solves the problem that the prior art cannot label the hyperspectral data of the target that is not visually visible. The implementation scheme is: with the help of Google Maps, the possible location of the target of the hyperspectral data set that is not visually visible to the target is locked; the original hyperspectral data is coarsely cropped and finely cropped in sequence using ENVI; the image data after fine cropping is visually analyzed, and the bands with relatively obvious target features are selected as characteristic spectrum bands and spliced ​​into a data cube; the data cube is unsupervisedly clustered using the K-means clustering method to obtain a class with a basically unchanged number of pixels; the target confidence point set is determined from the class with a basically unchanged number of pixels by visual analysis, and the final labeling is performed. The present invention can label visually invisible targets without the need for field measurements, saving a lot of material and manpower resources, and can be used for hyperspectral target detection and recognition.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a hyperspectral data annotation method, which can be used for hyperspectral target detection and recognition. Background Art

[0002] In the target detection and recognition algorithm based on hyperspectral, on the one hand, data annotation plays a very important role in improving the accuracy of the algorithm by providing high-standard basic data for the algorithm; on the other hand, since the current mainstream target detection algorithms are all implemented using machine learning and deep learning, a large number of training sample databases need to be established for algorithm training, and these data need to be annotated according to the differences in target categories. After the data is annotated, it is also beneficial to store the corresponding categories when new data is generated, and continuously expand the amount of training data. Furthermore, to ensure the performance of the detection algorithm used, the data annotation results corresponding to the hyperspectral data are needed to measure and verify the detection algorithm in order to achieve the ideal detection effect, so the collected data set needs to be annotated. In addition, before the final target detection and recognition, the raw data is usually preprocessed by super-resolution, fusion, etc., and the performance of these algorithms requires the results of data annotation to assist in evaluation. Therefore, after obtaining the available hyperspectral data, data annotation becomes an indispensable part of the data processing process.

[0003] The current hyperspectral data annotation methods mainly include field measurement and software-based methods.

[0004] The field measurement and labeling method mainly labels according to the spatial resolution and the actual environment, and checks one by one whether each pixel belongs to a certain type of target or background. However, this method requires a lot of manpower, material resources and time, and the labeling efficiency is very low.

[0005] Based on the software annotation method, for the hyperspectral images to be annotated, some targets can be observed by human eyes, and the annotation of such images is called target visually visible annotation; some targets cannot be observed by human eyes, and the annotation of such images is called target visually invisible annotation, where:

[0006] The annotation of visually visible targets is divided into the annotation of targets with certain morphological information and the annotation of targets without morphological information. For the annotation of targets with certain morphological information, the image annotation method based on spatial information is adopted. This annotation method is mainly based on the existing image annotation tools. First, the band with the most spatial information is selected from the hyperspectral image, and then the selected band image is input into the annotation tool. The commonly used annotation tool is ByLabel, which automatically extracts the image edge, manually selects the edge and groups it, realizes the accurate calibration of complex targets with holes, and finally outputs the edge map composed of smooth edges and the segmentation mask generated thereby. For the annotation of visually visible targets without morphological information, the spectral information is mainly used for annotation. First, the target detection algorithm is used for rough annotation. For different targets, the image can be preliminarily processed before rough annotation in combination with the target's own attributes to improve the accuracy of rough annotation; then the full-color image of the corresponding area obtained from Google Maps is combined with the detection results to achieve further accurate annotation.

[0007] For the labeling of targets that are not visually visible, on the one hand, due to the low resolution of hyperspectral sensors, for example, the spatial resolution of the data captured by the first satellite-borne civilian imaging spectrometer Hyperion is 30m, so the size of the target may be less than one pixel or sub-pixel. On the other hand, because the background and the target are similar in color, even with the assistance of high-resolution full-color images, the human eye cannot accurately identify the location of the target. In this case, the existing target labeling technology is no longer suitable for labeling visually invisible targets. Summary of the invention

[0008] The purpose of the present invention is to overcome the problem that the above-mentioned prior art cannot label visually invisible targets, and propose a software-based hyperspectral data labeling method to achieve labeling of target visually invisible hyperspectral data.

[0009] To achieve the above object, the technical solution of the present invention includes the following:

[0010] 1) For hyperspectral datasets where the target is not visually visible, use Google Maps to locate the possible location of the target;

[0011] 2) According to the target position locked in 1), the original hyperspectral data is coarsely cropped and finely cropped in sequence using the remote sensing image processing platform ENVI;

[0012] 3) Perform visual analysis on the finely cropped image data, select the bands with relatively obvious target features as characteristic spectrum segments to further increase the separability of the target and background, and then splice the selected characteristic spectrum segments into a data cube;

[0013] 4) Set the number of categories to 2-20, and use the K-means clustering method to perform unsupervised clustering of the data cube with different numbers of categories to obtain the classes {g1, g2, ...} with basically unchanged number of pixels;

[0014] 5) According to the fuzzy positioning in 1), the target class {g} is determined from the class with a substantially unchanged number of pixels obtained in 4), and the positions that are not the target are eliminated to obtain the target confidence point set {Tar};

[0015] 6) The target confidence point set {Tar} is the position that can be marked as the target. The corresponding pixel points are set to 1, and the remaining pixel points are set to 0 to obtain a relatively accurate data labeling result.

[0016] The beneficial effects of the present invention are:

[0017] (1) The present invention performs coarse cropping and fine cropping and characteristic spectral band selection on the hyperspectral data set of the target invisible to vision, thereby increasing the difference between the target and the non-target, avoiding unnecessary interference and influence of the background on the clustering, and solving the problem of the lack of hyperspectral data labeling method for the target invisible to vision.

[0018] (2) The present invention uses K-means clustering to roughly annotate the processed hyperspectral data, and only needs to adjust the number of category parameters, which reduces the complexity of the algorithm and realizes the rapid annotation of hyperspectral data.

[0019] The simulation results show that the present invention can better solve the problem of labeling visually invisible targets. For hyperspectral data with low resolution, there is no need to conduct field measurements. Rough labeling results with good effects can be obtained based on software, which saves a lot of material and manpower and lays a good foundation for subsequent fine calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a general flow chart for realizing the present invention;

[0021] Figure 2 A schematic diagram of the hyperspectral data clipping in the present invention;

[0022] Figure 3 This is a comparison chart of the results of labeling the hyperspectral data with annotations using the present invention. DETAILED DESCRIPTION

[0023] The specific implementation and effects of the present invention are further described below in conjunction with the accompanying drawings:

[0024] This example takes the Hyprion Hong Kong International Airport hyperspectral data as an example and labels it.

[0025] Reference Figure 1, the specific implementation of this example is as follows:

[0026] Step 1: Use Google Maps to locate the possible location of the target.

[0027] Google Maps is an electronic map service provided by Google to the world. The map contains information such as landmarks, lines, shapes, etc., and provides three views: vector maps, satellite photos, and topographic maps.

[0028] In this step, with the help of Google Maps, the specific implementation of locking the possible location of the target is as follows:

[0029] 1.1) Search for Hong Kong International Airport using Google Maps and find the scene corresponding to the Hyprion Hong Kong International Airport hyperspectral data, such as Figure 2 (a)

[0030] 1.2) Figure 2 (a) Observe the target and its surrounding area and lock on the possible location of the target aircraft.

[0031] For the airport data in this example, we need to observe the location of the apron to determine the approximate location of the aircraft, and then match its physical location to the hyperspectral data to lock the location of the aircraft in the Hyprion data, such as Figure 2 (b) as shown.

[0032] Step 2: Crop the hyperspectral data according to the approximate location of the target.

[0033] The cropping of hyperspectral data is usually done with the help of remote sensing image processing platforms. The existing common remote sensing image processing platforms mainly include ERDAS IMAGINE, ENVI and PCI Geomatica. In this example, the remote sensing image processing platform ENVI is used but not limited to it.

[0034] ENVI is a complete remote sensing image processing platform. The software processing technologies in its application collection cover image data input / output, regular and irregular cropping of image data, image calibration, image enhancement, image classification, knowledge-based decision tree classification, integration with GIS, DEM and terrain information extraction, radar data processing, 3D display analysis, etc. Regular and irregular cropping of image data is performed through regular graphics and irregular graphics.

[0035] This step is done by using the remote sensing image processing platform ENVI Figure 2 The upper right corner of the Hyprion airport data shown in (b) is subjected to coarse cropping and fine cropping in turn, which is specifically implemented as follows:

[0036] 2.1) Use regular graphics such as rectangles in the remote sensing image processing platform ENVI to crop the target area, i.e. the airport, and obtain the rough cropped data to reduce the data size and facilitate the observation of the next fine cropping, such as Figure 2 (c)

[0037] 2.2) Using the irregular graphics in the remote sensing image processing platform ENVI, on the basis of the coarsely cropped data, only the location where the aircraft is confirmed and its surrounding area are retained, and the values ​​of other areas are set to 0 to avoid unnecessary interference and influence of irrelevant background on clustering. In this way, the finely cropped data is obtained, such as Figure 2 (d) as shown.

[0038] Step 3: Perform visual analysis on the finely cropped image data and select the characteristic spectrum of the target.

[0039] Hyprion hyperspectral images have a total of 220 bands, but not every band has the characteristics of the target, so it is necessary to analyze visually and select the characteristic spectral bands. This step is to perform visual analysis on the data after fine cropping and select the bands with relatively obvious target characteristics as the characteristic spectral bands. The specific implementation is as follows:

[0040] 3.1) Combined with the fuzzy positioning in step 1, select the pixel point suspected to be the target in the finely cropped data, and draw the spectral curve a corresponding to the pixel point; then select the pixel point whose color is close to the target but can be determined not to be the target based on the position in the finely cropped data, and draw the spectral curve b corresponding to the pixel point;

[0041] 3.2) Compare curve a and curve b, note the band range where the difference between the two is relatively small, and then remove the data in this band range from the entire data, so as to increase the difference between the target and the non-target;

[0042] 3.3) Observe each remaining band in the image data one by one, and select the bands with relatively obvious target features as characteristic spectrum bands to further increase the separability of the target and the background; then splice these selected bands into a data cube for subsequent unsupervised clustering.

[0043] Step 4: Use the K-means clustering method to perform unsupervised clustering of the data cube with different numbers of categories.

[0044] According to experience, visually invisible targets belong to a small class with similar spectral characteristics in the entire data. If the spectral vector angle SAM is selected as the distance and different classification categories are set, when the number of categories is appropriate, after K-means clustering, the number of pixels of the target class will remain at a relatively stable value, and the target class can be separated from the background class. The specific implementation is as follows:

[0045] 4.1) Select the spectral vector angle SAM as the distance x;

[0046] 4.2) Let the category be (C 1 ,C 2 ,…C i …C k ), k is the number of categories, which is 2-20 according to experience;

[0047] 4.3) Use an iterative method to calculate the minimum value of the clustering objective function to minimize the square error E, and output the divided classes and record the clustering results, where:

[0048]

[0049] Where μ i It is the i-th type C i The mean vector of is expressed as:

[0050] 4.4) Execute 4.3) 19 times in total. In this example, clustering results in 5 categories with basically unchanged number of pixels, denoted as {g1, g2, g3, g4, g5}.

[0051] Step 5: Final annotation of the data

[0052] Since the classes {g1, g2, g3, g4, g5} obtained in step 4 with basically unchanged number of pixels have separated the target class from the background class, we need to find the target class and then make the final annotation. The specific implementation is as follows:

[0053] 5.1) According to the possible locations of the target locked with the help of Google Maps in step 1, the non-target pixel point classes included in the clusters {g1, g2, g3, g4, g5} with basically unchanged pixel numbers are deleted to obtain the determined target class {g}, and the locations that are not the target are eliminated to obtain the target confidence point set {Tar}, which is the location that can be calibrated as the target;

[0054] 5.2) Set the corresponding coordinates of the target confidence point set {Tar} to 1, and the corresponding coordinates of the remaining pixels to 0, to obtain a relatively accurate annotation result, such as Figure 2 As shown in (e), the labeling of the target visually invisible hyperspectral data is completed.

[0055] The effect of the present invention is further illustrated by the following simulation:

[0056] 1. Test data: The ABU dataset with annotations taken by the airborne visible infrared imaging spectrometer AVIRIS. Since the ABU data has a high spatial resolution, it is first downsampled by 2 times to meet the requirements of visually invisible targets;

[0057] 2. Simulation test content: Use the method of the present invention to mark the target aircraft in the downsampled ABU data. The results are as follows: Figure 3 As shown, where:

[0058] Figure 3 (a) is the pseudo-color image after AUB data sampling.

[0059] Figure 3 (b) is the corresponding annotation after AUB data sampling.

[0060] Figure 3 (c) is the result of labeling the ABU dataset using the method of the present invention;

[0061] from Figure 3 (c) It can be seen that 1 / 3 of the target positions in the labeling results are accurately labeled in full agreement with the actual distribution of the targets, thus verifying the accuracy of the method. Without the need for field measurements, a good rough labeling result can be obtained based on the software, saving a lot of manpower and material resources, and laying a good foundation for subsequent fine calibration.

Claims

1. A hyperspectral data annotation method based on unsupervised clustering, characterized in that: include: 1) For hyperspectral datasets where the target is not visually visible, use Google Maps to locate the possible location of the target; 2) According to the target position locked in 1), the original hyperspectral data is coarsely cropped and finely cropped in sequence using the remote sensing image processing platform ENVI; 3) Perform visual analysis on the finely cropped image data, select the bands with relatively obvious target features as characteristic spectrum segments to further increase the separability of the target and the background, and then splice the selected characteristic spectrum segments into a data cube; this is achieved as follows: 3a) Combined with the fuzzy positioning in 1), select the pixel point suspected to be the target in the finely cropped data, and draw the spectral curve a corresponding to the pixel point; then select the pixel point whose color is close to the target but can be determined not to be the target based on the position in the finely cropped data, and draw the spectral curve b corresponding to the pixel point; 3b) Compare the two curves a and b, and remove the image data in the band range with the smallest difference between the two in the entire data; 3c) Observe each remaining band in the image data one by one, select the band with relatively obvious target features as the characteristic spectrum band, and splice them into a data cube; 4) Set the number of categories to 2-20, and use the K-means clustering method to perform unsupervised clustering of the data cube with different numbers of categories to obtain the classes {g1, g2, ...} with basically unchanged number of pixels; 5) According to the fuzzy positioning in 1), the target class {g} is determined from the class with a substantially unchanged number of pixels obtained in 4), and the positions that are not the target are eliminated to obtain the target confidence point set {Tar}; 6) The target confidence point set {Tar} is the position that can be marked as the target. The corresponding pixel points are set to 1, and the remaining pixel points are set to 0 to obtain a relatively accurate data labeling result.

2. The method according to claim 1, characterized in that: In the above 1), the possible location of the target is locked by using the hyperspectral dataset where the target is not visually visible to the user using Google Maps, and the implementation is as follows: 1a) Observe different targets and their surrounding areas on Google Maps to determine the location of the target; 1b) Match the determined physical locations of different targets to the hyperspectral data.

3. The method according to claim 2, characterized in that: In step 1a), different targets and their surrounding areas are observed in Google Maps. For airport data, the location of the apron needs to be observed to determine the approximate location of the aircraft; for ship data, the location of the port needs to be observed to determine the location of the ship.

4. The method according to claim 1, characterized in that: In the above 2), the original hyperspectral data is subjected to coarse cropping and fine cropping in sequence by using the remote sensing image processing platform ENVI, which is implemented as follows: 2a) Use the regular graphics in the remote sensing image processing platform ENVI to crop the target area to reduce the data size and obtain the roughly cropped data; 2b) Using irregular graphics in the remote sensing image processing platform ENVI, on the basis of the roughly cropped data, the target and its surrounding areas are retained, and the values ​​of other areas are set to 0 to obtain the finely cropped data.

5. The method according to claim 1, characterized in that: In 4), the K-means clustering method is used to perform unsupervised clustering of data cubes with different numbers of categories, which is implemented as follows: 4a) Selecting the spectral vector angle SAM as the distance x; 4b) Let the categories be (C1, C2, …C i …C l ), k is the number of categories, ranging from 2 to 20; 4c) Use an iterative method to calculate the minimum value of the clustering objective function to minimize the square error E, and output the divided classes and record the clustering results, where: Where μ i It is the i-th type C i The mean vector of is expressed as: 4d) Execute 4c) 19 times in total, and cluster to obtain clusters with a basically unchanged number of pixels, denoted as {g1, g2, …}.

6. The method according to claim 1, characterized in that: The target class {g} is determined in 5) by locking the possible location of the target with the help of Google Maps in 1), deleting the cluster to obtain the non-target pixel point class included in the class {g1, g2, ...} with basically unchanged pixel number, and obtaining the determined target class {g}.

Images