Hyperspectral spectrum data fusion method and system for broadening spectral band

Abstract

The invention provides a hyperspectral spectrum data fusion method and a hyperspectral spectrum data fusion system capable of widening a spectral band, and the method comprises the steps: carrying out the wavelength domain matching of Vis-NIR hyperspectral data obtained by a ground object spectrometer based on the visible light band of a hyperspectral image, and storing the resampled Vis-NIR hyperspectral data in a spectrum library. Performing similarity measurement on the spectral vector of each pixel in the hyperspectral image and all standard spectrums in the spectrum library to generate a spectral feature fingerprint map and a spectral response coefficient map; and performing spectrum reconstruction on each pixel in the hyperspectral image data in combination with the spectral feature fingerprint map and the spectral response coefficient map. And finally, obtaining a hyperspectral image cube after broadened spectrum fusion. According to the scheme, on the basis of wavelength domain matching, the spectral feature fingerprint map, the spectral response coefficient map and the like, the hyperspectral image cube obtained through final fusion covers a wider spectral range, and the original high spatial resolution is reserved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of spectral data processing, and in particular, to a hyperspectral spectral data fusion method and system for widening spectral bands. BACKGROUND

[0002] Hyperspectral imaging can capture spectral information in a wide wavelength range, providing detailed spectral characteristics for various materials and ground objects. Visible (Vis) hyperspectral imaging data and visible-near infrared (Vis-NIR) hyperspectral data each provide unique advantages for different applications. Visible hyperspectral imaging data usually provides high spatial resolution, while Vis-NIR hyperspectral data provides a wider spectral range.

[0003] However, the spectral range covered by visible hyperspectral imaging data is limited (for example, 400-1000 nm), which limits its ability to distinguish materials with spectral characteristics beyond this range. Vis-NIR hyperspectral data is usually obtained as a point measurement, lacking the spatial context provided by imaging data. There are also ways in the prior art to fuse these data types, but these ways mostly have problems such as inaccuracy, loss of spectral information, and computational complexity. Therefore, the problem of how to effectively fuse visible hyperspectral imaging data and Vis-NIR hyperspectral data to generate comprehensive hyperspectral data with high spatial resolution and wide spectral range is urgent to be solved. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a hyperspectral spectral data fusion method and system for widening spectral bands, so as to obtain a hyperspectral image cube with a wider spectral range and preserving the original high spatial resolution.

[0005] In a first aspect, the present application provides a hyperspectral spectral data fusion method for widening spectral bands, the method comprising: acquiring visible hyperspectral image data and Vis-NIR visible near-infrared hyperspectral data of a target area; performing wavelength domain matching on the Vis-NIR hyperspectral data based on the wavelength range of the visible hyperspectral image data; saving the Vis-NIR hyperspectral data after wavelength domain matching to a spectral library; performing spectral similarity measurement based on the visible hyperspectral image data and the Vis-NIR hyperspectral data in the spectral library to generate a spectral feature fingerprint map and a spectral response coefficient map; performing spectral reconstruction on each pixel in the visible hyperspectral image data in combination with the spectral feature fingerprint map and the spectral response coefficient map; Based on the visible hyperspectral image data after spectrum reconstruction, a hyperspectral image cube with a widened spectrum is obtained.

[0006] In an optional embodiment, the step of performing wavelength domain matching on the Vis-NIR hyperspectral data based on the visible hyperspectral image data comprises: determining a plurality of wavebands corresponding to the visible hyperspectral image data, obtaining the center wavelength and waveband width of each waveband; based on the center wavelength and waveband width of each waveband, interpolating at the corresponding wavelength point of the Vis-NIR hyperspectral data to achieve wavelength domain matching between the Vis-NIR hyperspectral data and the visible hyperspectral image data.

[0007] In an optional embodiment, the step of saving the Vis-NIR hyperspectral data after wavelength domain matching to the spectral library comprises: setting a unique identifier for each spectral sample in the Vis-NIR hyperspectral data after wavelength domain matching; associating the unique identifier of each spectral sample and the spectral value of each waveband, and saving to the spectral library.

[0008] In an optional embodiment, the step of performing spectral similarity measurement based on the visible hyperspectral image data and the Vis-NIR hyperspectral data in the spectral library to generate a spectral feature fingerprint map and a spectral response coefficient map comprises: using spectral angle mapping to obtain the similarity between each pixel in the visible hyperspectral image data and each spectral sample of the Vis-NIR hyperspectral data in the spectral library to determine the matching spectral sample of each pixel; combining the matching spectral sample of each pixel to generate a spectral feature fingerprint map and a spectral response coefficient map.

[0009] In an optional embodiment, the step of using spectral angle mapping to obtain the similarity between each pixel in the visible hyperspectral image data and each spectral sample of the Vis-NIR hyperspectral data in the spectral library to determine the matching spectral sample of each pixel comprises: for each pixel in the visible hyperspectral image data, using spectral angle mapping to calculate the cosine value of the included angle between the spectral vector corresponding to the pixel and the spectral vector corresponding to each spectral sample of the Vis-NIR hyperspectral data in the spectral library; based on the cosine value of the included angle between the pixel and each spectral sample, determining the matching spectral sample from a plurality of spectral samples.

[0010] In an optional embodiment, the step of generating the spectral feature fingerprint map in combination with the matching spectral samples of each pixel comprises: obtaining a unique identifier of each matching spectral sample; mapping the unique identifiers of all matching spectral samples into the image space to generate the spectral feature fingerprint map.

[0011] In an optional embodiment, the step of generating the spectral response coefficient map in combination with the matching spectral samples of each pixel comprises: extracting, for each pixel, spectral values of all wavebands of the pixel in the visible light waveband range; obtaining a sum of the spectral values of all wavebands as an energy of the pixel in the visible light waveband; obtaining a sum of the spectral values of the pixel in all wavebands in the visible light hyperspectral image data, or a sum of the spectral values of the matching spectral sample of the pixel in the same visible light waveband, as a total energy; calculating a ratio coefficient between the energy of the pixel in the visible light waveband and the total energy; mapping the ratio coefficient corresponding to each pixel into the image space to generate the spectral response coefficient map.

[0012] In an optional embodiment, the step of performing spectral reconstruction on each pixel in the visible light hyperspectral image data in combination with the spectral feature fingerprint map and the spectral response coefficient map comprises: for each pixel in the visible light hyperspectral image data, based on a unique identifier corresponding to the pixel in the spectral feature fingerprint map, extracting a spectral sample with the unique identifier from the spectral library, and taking a spectral curve corresponding to the spectral sample as an initial spectral curve of the pixel; performing scaling processing on the initial spectral curve based on a ratio coefficient corresponding to the pixel in the spectral response coefficient map.

[0013] In an optional embodiment, the step of obtaining the hyperspectral image cube with the widened spectral band based on the visible light hyperspectral image data after spectral reconstruction comprises: obtaining wavelength metadata used for spectral reconstruction, and original geospatial metadata of the visible light hyperspectral image data; combining the wavelength metadata, the geospatial metadata, and the visible light hyperspectral image data after spectral reconstruction to obtain the hyperspectral image cube with the widened spectral band.

[0014] In a second aspect, the present application provides a hyperspectral data fusion system for widening spectral bands, which comprises: An acquisition module is configured to acquire visible hyperspectral image data and Vis-NIR visible near-infrared hyperspectral data of a target region; An execution module is configured to perform wavelength domain matching on the Vis-NIR hyperspectral data based on the wave band of the visible hyperspectral image data; A spectrum library construction module is configured to save the Vis-NIR hyperspectral data after wavelength domain matching into a spectrum library; A feature matching analysis module is configured to perform spectrum similarity measurement based on the visible hyperspectral image data and the Vis-NIR hyperspectral data in the spectrum library, to generate a spectrum feature fingerprint map and a spectrum response coefficient map; A reconstruction module is configured to perform spectrum reconstruction on each pixel in the visible hyperspectral image data in combination with the spectrum feature fingerprint map and the spectrum response coefficient map; A fusion generation module is configured to obtain a hyperspectral image cube with a widened spectrum based on the visible hyperspectral image data after spectrum reconstruction.

[0015] The present application provides a hyperspectral spectral data fusion method and system for widening the spectrum wave band, which performs wavelength domain matching on Vis-NIR hyperspectral data based on the wave band of the acquired visible hyperspectral image data, saves the matched Vis-NIR hyperspectral data into a spectrum library, performs spectrum similarity measurement based on the visible hyperspectral image data and the Vis-NIR hyperspectral data in the spectrum library, generates a spectrum feature fingerprint map and a spectrum response coefficient map, performs spectrum reconstruction on each pixel in the visible hyperspectral image data in combination with the spectrum feature fingerprint map and the spectrum response coefficient map, and obtains a hyperspectral image cube with a widened spectrum based on the visible hyperspectral image data after spectrum reconstruction. The present application widens the spectrum range of the final fused hyperspectral image cube based on wavelength domain matching, spectrum feature fingerprint map, spectrum response coefficient map, etc., while retaining the original high spatial resolution. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0017] Figure 1 The flow chart of the hyperspectral spectral data fusion method provided by the embodiments of the present application; Figure 2 The logic idea schematic diagram of the hyperspectral spectral data fusion method provided by the embodiments of the present application; Figure 3 A schematic diagram of the Vis-NIR hyperspectral data collected in the embodiment of the present application is shown in FIG. 3. Figure 4 A schematic diagram of the Vis-NIR hyperspectral data collected in the embodiment of the present application is shown in FIG. 3. Figure 5 The visible light hyperspectral image data before and after fusion in the embodiment of the present application; Figure 6 A structural block diagram of the hyperspectral data fusion system provided in the embodiment of the present application is shown in FIG. 4. Figure 7 A structural block diagram of the electronic device provided in the embodiment of the present application is shown in FIG. 5. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0019] Please refer to Figure 1 A flowchart of the hyperspectral data fusion method for widening the spectral band provided in the embodiment of the present application is shown in FIG. 2. The hyperspectral data fusion method for widening the spectral band can be executed by a hyperspectral data fusion system for widening the spectral band, which can be realized by software and / or hardware and configured in an electronic device, such as a computer device, a server, a programmable logic controller, etc. The detailed steps of the hyperspectral data fusion method for widening the spectral band are described as follows.

[0020] S11, collecting visible light hyperspectral image data and Vis-NIR visible light near-infrared hyperspectral data of a target region.

[0021] S12, performing wavelength domain matching on the Vis-NIR hyperspectral data based on the wave band of the visible light hyperspectral image data.

[0022] S13, saving the Vis-NIR hyperspectral data after wavelength domain matching to a spectral library.

[0023] S14, performing spectral similarity measurement based on the visible light hyperspectral image data and the Vis-NIR hyperspectral data in the spectral library to generate a spectral feature fingerprint map and a spectral response coefficient map.

[0024] S15, performing spectral reconstruction on each pixel in the visible light hyperspectral image data in combination with the spectral feature fingerprint map and the spectral response coefficient map.

[0025] S16, obtaining a hyperspectral image cube with widened spectrum based on the visible light hyperspectral image data after spectral reconstruction.

[0026] Please refer toFigure 2 As shown in the logic diagram, in this embodiment, a field spectrometer (for example, a field spectrometer equipped with a near-infrared sensor) can be used to collect Vis-NIR band data reflected or emitted by a specific feature or target, with a resolution of about 3 nm, to obtain Vis-NIR hyperspectral data. During the collection process, the environmental conditions (such as light, weather) are ensured to be consistent, and necessary calibration and reflectance correction are performed to eliminate the interference of instruments and environmental factors.

[0027] In addition, visible light hyperspectral image data of a large area can be obtained by an aerial or satellite hyperspectral remote sensor. This data is usually already corrected for radiation, atmosphere, reflectance, etc.

[0028] The obtained visible light hyperspectral image data can be as shown in Figure 3 , which is image data, with a wavelength range of 400 nm-1000 nm, a band number of 150, and an image resolution of 3493 4135. The obtained Vis-NIR hyperspectral data can be as shown in Figure 4 , which is single-point data, with a wavelength range of 350 nm-2500 nm, and a band number of 2150.

[0029] Since the Vis-NIR hyperspectral data collected by the field spectrometer has a high spectral resolution (about 3 nm), and the number and width of the bands of the visible light hyperspectral image data are fixed, it is necessary to downsample the Vis-NIR hyperspectral data to align the bands with the visible light hyperspectral image data.

[0030] Specifically, the step of performing wavelength domain matching on the Vis-NIR hyperspectral data based on the bands of the visible light hyperspectral image data can be implemented in the following manner: determining a plurality of bands corresponding to the visible light hyperspectral image data, obtaining the center wavelength and band width of each band; and performing interpolation at the corresponding wavelength points of the Vis-NIR hyperspectral data based on the center wavelength and band width of each band, to realize wavelength domain matching of the Vis-NIR hyperspectral data and the visible light hyperspectral image data.

[0031] In this embodiment, the Vis-NIR hyperspectral data is downsampled based on an interpolation method, which can be linear interpolation, spline interpolation, or cubic convolution interpolation, etc. Specifically, the interpolation method to be used can be determined according to the properties of the original visible light hyperspectral image data.

[0032] The goal of wavelength domain matching is to make each band in the Vis-NIR hyperspectral data after matching correspond to a band in the visible light hyperspectral image data in terms of center wavelength and band width.

[0033] Or, for each band of the visible hyperspectral image data, a weighted average of all Vis-NIR hyperspectral data points falling within the band range is calculated, where the weights can be determined according to the spectral response function or the distance from the band center. The Vis-NIR hyperspectral data is down-sampled based on the weighted average.

[0034] After the Vis-NIR hyperspectral data is down-sampled, it can also be ensured that the down-sampled Vis-NIR hyperspectral data has a high degree of consistency with the corresponding band of the visible hyperspectral image data through visual comparison and statistical analysis (such as correlation coefficient).

[0035] In this embodiment, the above processing method can solve the problem of inconsistent band settings between different spectrometers or sensors, and ensure the comparability of the data in the wavelength dimension.

[0036] In actual application, a suitable resampling method needs to be selected according to the specific situation to ensure the preservation of the original spectral characteristics.

[0037] After the above verification is completed, the Vis-NIR hyperspectral data matched in the wavelength domain can be saved to the spectral library, that is, the standardized spectral database construction is performed, and specifically, the step can be implemented in the following way: A unique identifier is set for each spectral sample in the Vis-NIR hyperspectral data matched in the wavelength domain; the unique identifier of each spectral sample and the spectral value of each band are associated and saved to the spectral library.

[0038] The obtained Vis-NIR hyperspectral data includes a plurality of spectral samples, and each spectral sample can be understood as corresponding to a ground object or target.

[0039] A unique identifier (ID) can be set for each spectral sample, and the spectral sample and its unique identifier are stored in the standardized spectral database (Spectral Library.csv) according to a predefined format and standard.

[0040] For example, the information can be saved in a table form (such as CSV), and each row in the table can correspond to a spectral sample, which can specifically include the following information of the spectral sample: unique identifier, spectral value of band 1, spectral value of band 2, …, spectral value of band N.

[0041] On the basis of the above, the spectral similarity is measured based on the visible hyperspectral image data and the Vis-NIR hyperspectral data in the spectral library to generate a spectral feature fingerprint map and a spectral response coefficient map, and specifically, the step can be implemented in the following way: The spectral angle mapping method is used to obtain the similarity between each pixel in the visible light hyperspectral image data and each spectral sample of the Vis-NIR hyperspectral data in the spectral library, so as to determine the matching spectral sample of each pixel; and the spectral feature fingerprint map and the spectral response coefficient map are generated in combination with the matching spectral sample of each pixel.

[0042] In this embodiment, the step of using the spectral angle mapping method to obtain the similarity between each pixel in the visible light hyperspectral image data and each spectral sample of the Vis-NIR hyperspectral data in the spectral library to determine the matching spectral sample of each pixel can be implemented in the following manner: For each pixel in the visible light hyperspectral image data, the spectral angle mapping method is used to calculate the included angle cosine value between the spectral vector corresponding to the pixel and the spectral vector corresponding to each spectral sample of the Vis-NIR hyperspectral data in the spectral library; and the matching spectral sample is determined from the plurality of spectral samples based on the included angle cosine value between the pixel and each spectral sample.

[0043] The spectral angle mapping method (SAM) is used to calculate the spectral angle between the spectrum of each pixel in the visible light hyperspectral image data and the spectrum of the Vis-NIR hyperspectral data in the spectral library, and the similarity of the spectrum is determined by comparing the size of the spectral angle, so as to realize the spectral similarity measurement.

[0044] In the spectral angle mapping method (SAM), the spectrum is regarded as an N-dimensional vector (N is the number of bands), and the spectral angle (SAM) measures the included angle cosine value between two spectral vectors. The smaller the included angle, the more similar the shape of the two spectra, indicating that they may represent the same substance. The SAM is not sensitive to the absolute brightness of the spectrum, and only focuses on the spectral shape, so it can still maintain good classification and identification performance under the influence of light conditions and terrain.

[0045] For the spectral vector P of each pixel in the visible light hyperspectral image data and the spectral vector R (which can be referred to as a reference spectral vector) of each spectral sample in the spectral library, the spectral angle between the two can be obtained by calculating the dot product of the two divided by the product of their respective modules, and then taking the inverse cosine value, and the specific calculation formula is as follows: α = arccos( (R· P) / (||R|| ||P||) ) For each pixel in the visible light hyperspectral image data, all spectral samples in the spectral library are traversed, and the SAM angle between the pixel and each spectral sample is calculated.

[0046] The spectral sample with the smallest SAM angle (the smallest spectral angle and the highest similarity) is selected as the matching spectral sample of the pixel.

[0047] In the above process, a spectral similarity threshold can also be added to further improve accuracy, that is, if the minimum SAM angle is less than a certain threshold, it indicates that the similarity between the pixel and the corresponding spectral sample is greater than the spectral similarity threshold, and the spectral sample can be determined as the matching spectral sample. However, if the minimum SAM angle corresponding to the pixel is greater than a certain threshold, it indicates that the similarity corresponding to the pixel is less than the spectral similarity threshold, in which case the pixel can be marked as "unclassified", and then other algorithms such as K-Nearest Neighbors (KNN) can be used to assist in processing.

[0048] After determining the matching spectral sample corresponding to each pixel in the visible hyperspectral image data, a spectral feature fingerprint map is generated in combination with the matching spectral sample of each pixel. Specifically, this step can be implemented in the following way: Obtain the unique identifier of the matching spectral sample of each pixel; map the unique identifiers of all matching spectral samples to the image space to generate a spectral feature fingerprint map.

[0049] In this embodiment, each pixel in the visible hyperspectral image data is assigned a unique identifier of its matching spectral sample in the spectral library. These identifiers are mapped to the image space to form a spectral feature fingerprint map. The spectral feature fingerprint map is a single-band image that directly shows which spectral sample type is closest to each pixel, and is the basis for object classification or identification.

[0050] In addition, a spectral response coefficient map is generated in combination with the matching spectral sample of each pixel, which can be implemented in the following way: For each pixel, extract the spectral values of all bands in the visible light band range; obtain the sum of the spectral values of all bands as the energy of the pixel in the visible light band; obtain the sum of the spectral values of the pixel in all bands in the visible hyperspectral image data, or the sum of the spectral values of the matching spectral sample of the pixel in the same visible light band, as the total energy; calculate the ratio coefficient between the energy of the pixel in the visible light band and the total energy; map the ratio coefficient corresponding to each pixel to the image space to generate a spectral response coefficient map.

[0051] In this embodiment, for each pixel, the spectral values of all bands in the visible light band range, i.e. the range of 450nm-680nm, are extracted. The sum (or average) of the spectral values of these bands is calculated as the energy of the pixel in the visible light band, where the energy is understood as the spectral reflectance in the visible light band.

[0052] Then the total energy is calculated, which can be the sum of the spectral values of the pixel in all bands in the visible hyperspectral image data, or the sum of the spectral values of the matching spectral samples in the same visible bands in the spectral library.

[0053] The ratio coefficient between the energy of the pixel and the total energy is calculated, and then mapped into the image space to generate a spectral response coefficient map. The spectral response coefficient map is a single-band image, and each pixel value in the image is the visible band spectral energy ratio coefficient, which provides the relative response intensity information of the ground object in the visible light region, and is used for scaling in subsequent spectral reconstruction. The ratio coefficient reflects the energy intensity of the visible band, which is used to adjust the overall brightness of the reconstructed spectrum to solve the difference in spectral amplitude.

[0054] On this basis, the spectral reconstruction is performed on each pixel in the visible hyperspectral image data in combination with the spectral feature fingerprint map and the spectral response coefficient map. Specifically, the step can be implemented in the following way: For each pixel in the visible hyperspectral image data, based on the unique identifier corresponding to the pixel in the spectral feature fingerprint map, the spectral sample with the unique identifier is extracted from the spectral library, and the spectral curve corresponding to the spectral sample is taken as the initial spectral curve of the pixel; based on the ratio coefficient corresponding to the pixel in the spectral response coefficient map, the initial spectral curve is scaled.

[0055] In this embodiment, according to the fusion result obtained by the spectral feature matching analysis (i.e. the spectral feature fingerprint map and the spectral response coefficient map), the corresponding Vis-NIR hyperspectral data information is extracted from the standardized spectral library, and the spectral curve of each pixel is reconstructed.

[0056] Specifically, for each pixel in the visible hyperspectral image data, according to the unique identifier corresponding to the pixel in the spectral feature fingerprint map, the Vis-NIR hyperspectral data curve corresponding to the unique identifier is accurately extracted from the standardized spectral library, that is, the spectral curve constituted by the corresponding matching spectral sample is taken as the initial spectral curve of the pixel. In this process, the unique identifier determines the shape of the reconstructed spectrum.

[0057] In the process of spectral reconstruction, the initial spectral curve can be optimized by using the scaling based on the spectral energy ratio coefficient to better fit the spectral characteristics of the actual pixel. The ratio coefficient corresponding to the pixel calculated in the spectral response coefficient diagram is used to scale the initial spectral curve extracted according to the unique identifier. The scaling operation can adjust the amplitude of the spectrum to better match the overall spectral characteristics of the original pixel. The ratio coefficient determines the size or intensity of the reconstructed spectrum, thereby realizing the restoration mode of identifier determining shape and ratio determining size. It is ensured that the reconstructed spectrum not only retains the shape characteristics of the original spectrum, but also adjusts its energy intensity.

[0058] Finally, based on the visible hyperspectral image data after spectral reconstruction, a hyperspectral image cube with a widened spectrum can be obtained. Specifically, this step can be realized by the following way: Obtain wavelength metadata for spectral reconstruction and original geospatial metadata of visible hyperspectral image data; combine wavelength metadata, geospatial metadata and visible hyperspectral image data after spectral reconstruction to obtain a hyperspectral image cube with a widened spectrum.

[0059] In this embodiment, the wavelength information for spectral reconstruction, including, for example, start wavelength, end wavelength, number of wavebands, waveband center wavelength or wavelength range, etc., is taken as wavelength metadata. The wavelength metadata is accurately associated with the newly generated visible hyperspectral image data. In this way, it is ensured that each waveband corresponds to its accurate spectral position, which is convenient for subsequent spectral analysis and application.

[0060] In addition, the geospatial metadata (such as geographic coordinate system, projection information, pixel size, image corner coordinates, etc.) of the original visible hyperspectral image data is completely inherited and passed to the newly generated visible hyperspectral image data. It is ensured that the reconstructed visible hyperspectral image data has accurate geographic positioning, so that it can be seamlessly integrated with a geographic information system (GIS) and support spatial analysis.

[0061] Finally, each pixel spectral curve after reconstruction, together with its accurate wavelength metadata and geospatial metadata, is organized into a complete hyperspectral image cube. This cube contains the complete and optimized Vis-NIR hyperspectral data curve of each pixel and carries all the necessary metadata information, marking the successful fusion and output of visible hyperspectral imaging data and Vis-NIR hyperspectral data.

[0062] The wavelength range of the hyperspectral image cube is fully consistent with the wavelength range of the Vis-NIR hyperspectral instrument (350nm-2500nm), and contains accurate wavelength metadata and geospatial metadata, retaining the high spatial resolution of 3493x4135 of the original visible hyperspectral image data. Band optimization ensures the spectral continuity and integrity of the final output data, while the accuracy of the metadata is crucial for subsequent data analysis and application. Figure 5 The left image is the original visible hyperspectral image data, and the right image is the hyperspectral image cube obtained after fusion, which is based on the same inventive concept, please refer to Figure 5 It can be seen that the image spatial data of the two are almost consistent before and after fusion.

[0063] Based on the same inventive concept, please refer to Figure 6 The embodiment of the present application also provides a functional module schematic diagram of a hyperspectral spectral data fusion system for widening the spectral band. The embodiment can divide the functional modules of the hyperspectral spectral data fusion system according to the method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated in one processing module. The integrated module can be realized in the form of hardware or in the form of a software functional module. It should be noted that the division of the modules in the embodiment of the present application is illustrative, and is only a logical functional division. When actually implemented, there can be another division mode.

[0064] For example, in the case of dividing each functional module according to each function, Figure 4 The hyperspectral spectral data fusion system shown is only a device schematic diagram. The hyperspectral spectral data fusion system can include an acquisition module, an execution module, a spectral library construction module, a feature matching analysis module, a reconstruction module, and a fusion generation module. The functions of each functional module of the hyperspectral spectral data fusion system will be described in detail below.

[0065] The acquisition module is used to acquire visible hyperspectral image data and Vis-NIR visible near-infrared hyperspectral data of a target region; The execution module is used to perform wavelength domain matching on the Vis-NIR hyperspectral data based on the band of the visible hyperspectral image data; The spectral library construction module is used to save the Vis-NIR hyperspectral data after wavelength domain matching into a spectral library; The feature matching analysis module is used to perform spectral similarity measurement based on the visible hyperspectral image data and the Vis-NIR hyperspectral data in the spectral library, to generate a spectral feature fingerprint map and a spectral response coefficient map; a reconstruction module, configured to perform spectral reconstruction on each pixel in the visible hyperspectral image data by combining the spectral feature fingerprint and the spectral response coefficient map; a fusion generation module, configured to obtain a hyperspectral image cube with a widened spectrum based on the visible hyperspectral image data after the spectral reconstruction.

[0066] In a possible implementation, the execution module is configured to perform wavelength domain matching in the following manner: determine a plurality of wavebands corresponding to the visible hyperspectral image data, and obtain a central wavelength and a waveband width of each waveband; perform interpolation at corresponding wavelength points of the Vis-NIR hyperspectral data based on the central wavelength and the waveband width of each waveband, to realize wavelength domain matching of the Vis-NIR hyperspectral data and the visible hyperspectral image data.

[0067] In a possible implementation, the spectral library construction module is configured to construct the spectral library in the following manner: set a unique identifier for each spectral sample in the Vis-NIR hyperspectral data after the wavelength domain matching; associate the unique identifier of each spectral sample and the spectral value of each waveband, and save them into the spectral library.

[0068] In a possible implementation, the feature matching analysis module is configured to perform similarity measurement in the following manner: obtain the similarity between each pixel in the visible hyperspectral image data and each spectral sample in the Vis-NIR hyperspectral data in the spectral library by using the spectral angle mapping method, to determine the matching spectral sample of each pixel; combine the matching spectral sample of each pixel to generate the spectral feature fingerprint and the spectral response coefficient map.

[0069] In a possible implementation, the feature matching analysis module is configured to determine the matching spectral sample in the following manner: for each pixel in the visible hyperspectral image data, calculate the included angle cosine value between the spectral vector corresponding to the pixel and the spectral vector corresponding to each spectral sample in the Vis-NIR hyperspectral data in the spectral library by using the spectral angle mapping method; determine the matching spectral sample from the plurality of spectral samples based on the included angle cosine value between the pixel and each spectral sample.

[0070] In a possible implementation, the feature matching analysis module is configured to generate the spectral feature fingerprint in the following manner: obtain the unique identifier of the matching spectral sample of each pixel; Mapping the unique identifiers of all the matched spectral samples into the image space to generate a spectral feature fingerprint map.

[0071] In a possible implementation, the feature matching analysis module is configured to generate the spectral response coefficient map by: For each pixel, extracting spectral values of the pixel in all wavebands in the visible light waveband range; Obtaining a sum of the spectral values in all wavebands as energy of the pixel in the visible light waveband range; Obtaining a sum of the spectral values of the pixel in all wavebands in the visible light hyperspectral image data or a sum of the spectral values of the matched spectral sample of the pixel in the same visible light waveband range as the total energy; Calculating a ratio coefficient between the energy of the pixel in the visible light waveband range and the total energy; Mapping the ratio coefficient corresponding to each pixel into the image space to generate the spectral response coefficient map.

[0072] In a possible implementation, the reconstruction module is configured to perform spectral reconstruction by: For each pixel in the visible light hyperspectral image data, extracting, based on the unique identifier corresponding to the pixel in the spectral feature fingerprint map, a spectral sample with the unique identifier from the spectral library, and taking a spectral curve corresponding to the spectral sample as an initial spectral curve of the pixel; Performing scaling processing on the initial spectral curve based on the ratio coefficient corresponding to the pixel in the spectral response coefficient map.

[0073] In a possible implementation, the fusion generation module is configured to obtain the hyperspectral image cube by: Obtaining wavelength metadata used for spectral reconstruction, and original geospatial metadata of the visible light hyperspectral image data; Combining the wavelength metadata, the geospatial metadata, and the visible light hyperspectral image data after spectral reconstruction to obtain the hyperspectral image cube with the widened spectrum.

[0074] The hyperspectral spectral data fusion system provided in this embodiment can be used to perform the hyperspectral spectral data fusion method in any of the implementation manners in the above embodiments. For details not described in this embodiment, refer to the corresponding description in the above embodiments, which will not be described herein again.

[0075] Please refer to Figure 7This is a structural block diagram of an electronic device provided in an embodiment of the present invention. The electronic device can be a computer device, server, or similar component in a back-end analysis platform. The electronic device includes a memory, a processor, and a communication module. The memory, processor, and communication module are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0076] The memory is used to store computer programs or data. Memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0077] The processor is used to read / write data or programs stored in the memory and to execute the hyperspectral image data fusion method provided in any embodiment of the present invention.

[0078] The communication module is used to establish communication connections between electronic devices and other communication terminals via a network, and to send and receive data via the network.

[0079] It should be understood that, Figure 7 The structure shown is only a schematic diagram of an electronic device; the electronic device may also include components that are larger than those shown. Figure 7 The more or fewer components shown, or having the same Figure 7 The different configurations shown.

[0080] Furthermore, embodiments of the present invention also provide a computer-readable storage medium storing machine-executable instructions, which, when executed, implement the hyperspectral image data fusion method provided in the above embodiments.

[0081] Specifically, the computer-readable storage medium can be a general-purpose storage medium, such as a removable disk or hard disk. When the computer program on the computer-readable storage medium is run, it can execute the aforementioned hyperspectral image data fusion method. The processes involved in the execution of the computer-readable storage medium and its executable instructions can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0082] In the embodiments of the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. The embodiments described above are merely exemplary, for example, the division of units is only a logical function division, and there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0083] In addition, the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0084] In addition, the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0085] It should be noted that, if the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and various media that can store program codes.

[0086] In this document, the terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations.

[0087] The above merely illustrates the embodiments of the present application, and is not used to limit the protection scope of the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for fusion of hyperspectral image data to broaden spectral bands, characterized in that, The method includes: Acquire visible light hyperspectral image data and Vis-NIR visible and near-infrared hyperspectral data of the target area; Wavelength domain matching is performed on the Vis-NIR hyperspectral data based on the bands of the visible light hyperspectral image data. Save the wavelength-domain matched Vis-NIR hyperspectral data to the spectral library; Based on the visible light hyperspectral image data and the Vis-NIR hyperspectral data in the spectral library, a spectral similarity measurement is performed to generate a spectral feature fingerprint map and a spectral response coefficient map; The spectral reconstruction is performed on each pixel in the visible light hyperspectral image data by combining the spectral feature fingerprint map and the spectral response coefficient map; Based on the visible light hyperspectral image data after spectral reconstruction, a hyperspectral image cube with broadened spectrum is obtained.

2. The hyperspectral image data fusion method for broadening the spectral band according to claim 1, characterized in that, The step of performing wavelength domain matching of Vis-NIR hyperspectral data based on the bands of the visible light hyperspectral image data includes: Determine multiple bands corresponding to the visible light hyperspectral image data, and obtain the center wavelength and bandwidth of each band; Based on the center wavelength and bandwidth of each band, interpolation is performed at the corresponding wavelength points of the Vis-NIR hyperspectral data to achieve wavelength domain matching between the Vis-NIR hyperspectral data and the visible light hyperspectral image data.

3. The hyperspectral image data fusion method for broadening the spectral band according to claim 1, characterized in that, The step of saving the wavelength-domain matched Vis-NIR hyperspectral data to the spectral library includes: Set a unique identifier for each spectral sample in the Vis-NIR hyperspectral data after wavelength domain matching; The unique identifier of each spectral sample and the spectral value of each band are associated and saved to the spectral library.

4. The hyperspectral image data fusion method for broadening the spectral band according to claim 1, characterized in that, The step of performing spectral similarity measurement based on the visible light hyperspectral image data and the Vis-NIR hyperspectral data in the spectral library to generate a spectral feature fingerprint map and a spectral response coefficient map includes: The similarity between each pixel in the visible hyperspectral image data and each spectral sample of the Vis-NIR hyperspectral data in the spectral library is obtained by using the spectral angle mapping method, so as to determine the matching spectral sample for each pixel; By combining the matched spectral samples of each pixel, a spectral feature fingerprint map and a spectral response coefficient map are generated.

5. The hyperspectral image data fusion method for broadening the spectral band according to claim 4, characterized in that, The step of obtaining the similarity between each pixel in the visible hyperspectral image data and each spectral sample of the Vis-NIR hyperspectral data in the spectral library using the spectral angle mapping method to determine the matching spectral sample for each pixel includes: For each pixel in the visible hyperspectral image data, the cosine of the angle between the spectral vector corresponding to the pixel and the spectral vector corresponding to each spectral sample of the Vis-NIR hyperspectral data in the spectral library is calculated using the spectral angle mapping method. Based on the cosine value of the angle between the pixel and each of the spectral samples, a matching spectral sample is determined from multiple spectral samples.

6. The hyperspectral image data fusion method for broadening the spectral band according to claim 4, characterized in that, The steps for generating a spectral feature fingerprint by combining the matched spectral samples of each pixel include: Obtain a unique identifier for the matching spectral sample of each pixel; The unique identifiers of all matching spectral samples are mapped to the image space to generate a spectral feature fingerprint.

7. The hyperspectral image data fusion method for broadening the spectral band according to claim 4, characterized in that, The steps for generating a spectral response coefficient map by combining the matched spectral samples of each pixel include: For each pixel, extract the spectral values ​​of that pixel across all bands within the visible light spectrum. The sum of the spectral values ​​of all bands is obtained as the energy of the pixel in the visible light band; The total energy is obtained by summing the spectral values ​​of pixels in all bands in the visible hyperspectral image data, or by summing the spectral values ​​of matching spectral samples of the pixels in the same visible band. Calculate the ratio coefficient between the energy of the pixel in the visible light band and the total energy; The ratio coefficient corresponding to each pixel is mapped to the image space to generate a spectral response coefficient map.

8. The hyperspectral image data fusion method for broadening the spectral band according to claim 3, characterized in that, The step of performing spectral reconstruction on each pixel in the visible light hyperspectral image data by combining the spectral feature fingerprint and the spectral response coefficient map includes: For each pixel in the visible light hyperspectral image data, based on the unique identifier corresponding to the pixel in the spectral feature fingerprint map, a spectral sample with the unique identifier is extracted from the spectral library, and the spectral curve corresponding to the spectral sample is used as the initial spectral curve of the pixel. The initial spectral curve is scaled based on the ratio coefficients corresponding to the pixels in the spectral response coefficient graph.

9. The hyperspectral image data fusion method for broadening the spectral band according to claim 1, characterized in that, The step of obtaining a hyperspectral image cube with broadened spectrum from the visible light hyperspectral image data based on spectral reconstruction includes: Obtain wavelength metadata for spectral reconstruction, as well as the original geospatial metadata of the visible hyperspectral image data; By combining the wavelength metadata, geospatial metadata, and the spectrally reconstructed visible hyperspectral image data, a hyperspectral image cube with a broadened spectrum is obtained.

10. A hyperspectral image data fusion system for broadening spectral bands, characterized in that, The system includes: The acquisition module is used to acquire visible light hyperspectral image data and Vis-NIR visible and near-infrared hyperspectral data of the target area; The execution module is used to perform wavelength domain matching on the Vis-NIR hyperspectral data based on the bands of the visible light hyperspectral image data; The spectral library construction module is used to save Vis-NIR hyperspectral data after wavelength domain matching to the spectral library; The feature matching analysis module is used to perform spectral similarity measurement based on the visible light hyperspectral image data and the Vis-NIR hyperspectral data in the spectral library, so as to generate a spectral feature fingerprint map and a spectral response coefficient map. The reconstruction module is used to perform spectral reconstruction on each pixel in the visible light hyperspectral image data by combining the spectral feature fingerprint map and the spectral response coefficient map; The fusion generation module is used to obtain a hyperspectral image cube with a broadened spectrum based on the spectral reconstructed visible hyperspectral image data.

Images