A hyperspectral image orthorectification method and device, and a storage medium

Abstract

The embodiment of the application provides a hyperspectral image ortho-rectification method and device, and a storage medium, comprising: acquiring full-color image data corresponding to hyperspectral image data to be corrected; obtaining target exterior orientation elements corresponding to the full-color image data by sequentially performing feature extraction and feature matching on the full-color image data; performing space forward intersection calculation and grid construction on the target exterior orientation elements to obtain a regular grid; and performing ortho-rectification on the hyperspectral image data according to the target exterior orientation elements and the regular grid to obtain an ortho-image corresponding to the hyperspectral image. The influence of the types of ground objects in the observation area on the ortho-rectification can be avoided, and the accuracy of the ortho-rectification is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and in particular to a hyperspectral image orthorectification method and device and a storage medium. BACKGROUND

[0002] The current visible light remote sensing image orthorectification method needs to be combined with images to realize the feature point extraction and matching process, and the number of homonym points and the accuracy of image coordinates obtained by the feature point extraction and matching will affect the accuracy of subsequent orthorectification. The existing orthorectification method for recording hyperspectral remote sensing images is to extract and match feature points for airborne hyperspectral images.

[0003] Specifically, three methods can be used to perform full-automatic orthorectification of airborne hyperspectral remote sensing images.

[0004] First, the conventional visible light remote sensing image orthorectification method is directly used to perform feature extraction, feature matching, regional network adjustment, regular grid generation and orthorectification on the airborne hyperspectral image. However, on the one hand, the single-band hyperspectral image has the characteristic of weak texture region, which leads to incomplete feature point matching results and reduces the accuracy of orthorectification; on the other hand, the high spectral resolution of the hyperspectral image limits the improvement of the spatial resolution, and the diversity of the spectrum also makes it possible that the image features of the same ground object in different bands are quite different, which affects the orthorectification process.

[0005] Second, the hyperspectral image is fused and preprocessed with a panchromatic image having a higher spatial resolution than the hyperspectral image and expected to be geometrically registered to obtain a fused image having high spatial resolution and spectral resolution, and then the conventional orthorectification processing flow is performed. However, the additional data preparation and preprocessing work will increase the workload, and the processing result is actually an orthographic image of the fused image, which changes the spectral characteristics of the original image to some extent and is not conducive to the subsequent quantitative analysis and other applications of the orthographic image.

[0006] Third, auxiliary work of feature point extraction and checking of the hyperspectral image based on manual visual interpretation is added in the conventional orthorectification processing flow to ensure that the number of homonym points and the accuracy of image coordinates meet the work requirements. However, the process of feature extraction and matching of the hyperspectral image based on manual visual interpretation increases a lot of workload on the basis of the conventional processing flow, greatly increases the labor cost, and has the possibility of introducing manual interpretation errors. SUMMARY

[0007] The embodiment of the present application provides a hyperspectral image orthorectification method and device, and a storage medium, which can avoid the influence of the ground object type of an observation area on orthorectification, and improve the accuracy of orthorectification.

[0008] The technical solution of the present application is implemented as follows:

[0009] In a first aspect, the embodiment of the present application provides a hyperspectral image orthorectification method, which comprises the following steps:

[0010] Obtaining full-color image data corresponding to the hyperspectral image data to be corrected;

[0011] Obtaining target exterior orientation elements corresponding to the full-color image data by sequentially performing feature extraction and feature matching on the full-color image data;

[0012] Performing space forward intersection calculation and grid construction on the target exterior orientation elements to obtain a regular grid;

[0013] Performing orthorectification on the hyperspectral image data according to the target exterior orientation elements and the regular grid to obtain an orthographic image corresponding to the hyperspectral image.

[0014] In a second aspect, the embodiment of the present application provides a hyperspectral image orthorectification device, which comprises the following units:

[0015] An acquisition unit, configured to acquire full-color image data corresponding to the hyperspectral image data to be corrected;

[0016] A feature extraction and feature matching unit, configured to obtain target exterior orientation elements corresponding to the full-color image data by performing feature extraction and feature matching on the full-color image data;

[0017] A grid generation unit, configured to perform space forward intersection calculation and grid construction on the target exterior orientation elements to obtain a regular grid;

[0018] An orthorectification unit, configured to perform orthorectification on the hyperspectral image data according to the target exterior orientation elements and the regular grid to obtain an orthographic image corresponding to the hyperspectral image.

[0019] In a third aspect, the embodiment of the present application provides a hyperspectral image orthorectification device, which comprises a processor, a memory and a communication bus; the processor implements the hyperspectral image orthorectification method as described above when executing a running program stored in the memory.

[0020] In a fourth aspect, the embodiment of the present application provides a storage medium, which stores a computer program; the computer program is executed by a processor to implement the hyperspectral image orthorectification method as described above.

[0021] This application provides a method, apparatus, and storage medium for orthorectifying hyperspectral images. The method includes: acquiring panchromatic image data corresponding to the hyperspectral image data to be corrected; obtaining target exterior orientation elements corresponding to the panchromatic image data by sequentially performing feature extraction and feature matching on the panchromatic image data; performing spatial forward intersection calculation and grid construction on the target exterior orientation elements to obtain a regular grid; and performing orthorectification on the hyperspectral image data based on the target exterior orientation elements and the regular grid to obtain an orthorectified image corresponding to the hyperspectral image. By employing the above implementation scheme, feature extraction and feature matching are performed on the panchromatic image data corresponding to the hyperspectral image data. Then, the obtained target exterior orientation elements are used as the exterior orientation elements for orthorectification of the hyperspectral image, and subsequent orthorectification is performed. Utilizing the high resolution of the panchromatic image, more corresponding points can be matched, resulting in more target exterior orientation elements, which is unaffected by the type of land cover in the observation area. Furthermore, when the target exterior orientation elements are subsequently used to perform orthorectification on the hyperspectral image data, the spectral characteristics of the hyperspectral image are preserved, improving the accuracy of the orthorectification. Attached Figure Description

[0022] Figure 1 A flowchart of a hyperspectral image orthorectification method provided in this application embodiment;

[0023] Figure 2 An exemplary method for orthorectifying airborne hyperspectral images is provided for embodiments of this application;

[0024] Figure 3 A schematic diagram of the structure of a hyperspectral image orthorectification device provided in this application embodiment. Figure 1 ;

[0025] Figure 4 A schematic diagram of the structure of a hyperspectral image orthorectification device provided in this application embodiment. Figure 2 . Detailed Implementation

[0026] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the scope of this application.

[0027] This application provides a hyperspectral image orthorectification method, such as... Figure 1 As shown, the method may include:

[0028] S101. Obtain the panchromatic image data corresponding to the hyperspectral image data to be corrected.

[0029] The hyperspectral image orthorectification method proposed in this application is applicable to scenarios involving fully automatic orthorectification of airborne hyperspectral remote sensing images.

[0030] In the embodiment of the present application, the hyperspectral image data to be corrected can be stored in the airborne hyperspectral image dataset. In the embodiment of the present application, it is determined whether corresponding panchromatic image data exists for each piece of hyperspectral image data in the airborne hyperspectral image dataset.

[0031] It should be noted that in the embodiment of the present application, corresponding panchromatic image data exists for each piece of hyperspectral image data to be corrected.

[0032] In the embodiment of the present application, the number of hyperspectral image data is two or more, which can be selected according to actual conditions, and the embodiment of the present application does not make specific limitations.

[0033] In the embodiment of the present application, the panchromatic image data is sequentially subjected to feature extraction and feature matching to obtain target exterior orientation elements corresponding to the panchromatic image data.

[0034] In the embodiment of the present application, the panchromatic image data is subjected to feature extraction to obtain feature data corresponding to the panchromatic image data.

[0035] In the embodiment of the present application, a Scale Invariant Feature Transform (SIFT)-CPU or SIFT-Graphics Processing Unit (GPU) algorithm can be used to extract features from each piece of panchromatic image data, and the specific selection can be made according to actual conditions, and the embodiment of the present application does not make specific limitations.

[0036] It should be noted that if the SIFT-CPU algorithm is used to extract features from each piece of panchromatic image data, the feature extraction process includes four steps of scale space extreme value detection, key point positioning, main direction determination and key point feature vector description; and if the SIFT-GPU algorithm is used to extract features from each piece of panchromatic image data, the steps of Gaussian pyramid image construction, key point positioning and main direction determination for each piece of panchromatic image data are performed based on the SIFT-GPU algorithm using GPU parallel algorithm, thereby improving the feature extraction speed.

[0037] It can be understood that the feature extraction of the panchromatic image data in the embodiment of the present application can make the number of obtained feature points larger and more uniformly distributed, and can effectively overcome the deficiency of feature point loss caused by the low radiation brightness value of the shadow area on the single-band image of the hyperspectral image.

[0038] It should be noted that the feature extraction function corresponding to the above SIFT-CPU algorithm or SIFT-GPU algorithm can be stored in a third-party open source library such as OpenCV (cross-platform computer vision and machine learning software), and the above SIFT-CPU algorithm or SIFT-GPU algorithm can be called by calling the feature extraction function in the third-party open source library.

[0039] In the embodiment of the present application, after the feature extraction of the panchromatic image data is performed and the feature data corresponding to the panchromatic image data is obtained, the feature matching of the panchromatic image data is performed based on the feature data, and the matching relationship of the homonym points in the panchromatic image data is obtained.

[0040] Specifically, first, the initial exterior orientation elements of the panchromatic image data are obtained; then, the image center distances between the panchromatic images are calculated according to the initial exterior orientation elements; then, the target image center distances smaller than the image distance threshold are selected from the image center distances; then, the statistical feature data between the target image center distances are calculated; and based on the statistical feature data, the matching relationship of the homonym points in the panchromatic image data is determined.

[0041] In the embodiment of the present application, the statistical feature data can include statistical data such as mean and / or standard deviation, which can be selected according to actual conditions, and the embodiment of the present application is not limited specifically.

[0042] For example, for the panchromatic image data with a difference between the image center distance and the mean value greater than 3 times the standard deviation, it is considered that there is no matching relationship between the panchromatic image data.

[0043] In the embodiment of the present application, the FLANN matching method can be used to perform feature matching on the panchromatic image data with the matching relationship, and the matching relationship of the homonym points in the panchromatic image data is obtained.

[0044] It should be noted that the feature matching function corresponding to the FLANN matching algorithm can also be stored in a third-party open source library such as OpenCV.

[0045] Further, after the matching relationship of the homonym points in the panchromatic image data is determined, the basis matrix corresponding to the matching relationship can be determined; based on the basis matrix, the error matching relationship in the matching relationship is determined; the error matching relationship is deleted from the matching relationship, and the updated matching relationship is obtained.

[0046] In the embodiments of the present application, the fundamental matrix for transforming the left and right image data of the full-color image data pair corresponding to the matching relationship can be calculated according to the 7-point method in the field of computer vision; then, the random sample consensus (RANSAC) method is used to determine the error matching relationship.

[0047] Specifically, after obtaining the fundamental matrix, the epipolar equation corresponding to the feature point coordinates of the left and right image data can be calculated by using formula (1), the epipolar error corresponding to the feature point coordinates is further calculated, and error elimination is performed according to the epipolar error.

[0048] l R :(x L f1+y L f2+f3)x+(x L f4+y L f5+f6)y+(x L f7+y L f8+f9)=0

[0049] l L : (x R f1+y R f2+f3)x+(x R f4+y R f5+f6)y+(x R f7+y R f8+f9)=0 (1)

[0050] Wherein, x L ,y L ,x R ,y R are the feature point coordinates on the left and right image data of the matching pair, and f1, f2,..., f9 are the coefficients of the fundamental matrix in the transformation equation formula (2).

[0051]

[0052] In the embodiments of the present application, after obtaining the matching relationship of the homonymous points of the full-color image data in the full-color image data, the target exterior orientation elements corresponding to the full-color image data are determined based on the matching relationship, the initial exterior orientation elements of the full-color image data, and the initial values of the to-be-determined ground point coordinates corresponding to the full-color image data.

[0053] In the embodiments of the present application, the initial exterior orientation elements corresponding to each panchromatic image data and the initial values of the undetermined ground point coordinates corresponding to each feature point of the panchromatic image data are first determined, and then an error equation is established according to a preset collinearity equation, the matching relationship, the initial exterior orientation elements of the panchromatic image data and the initial values of the undetermined ground point coordinates corresponding to the panchromatic image data are input into the error equation, the equation is solved, and the target exterior orientation elements corresponding to the panchromatic image data are obtained.

[0054] In the embodiments of the present application, the POS auxiliary data provided by the engineering can be used to determine the initial exterior orientation elements corresponding to each panchromatic image data and the initial values of the undetermined ground point coordinates corresponding to each feature point of the panchromatic image data.

[0055] Further, in the process of calculating the target exterior orientation elements, the mis-matched points can also be removed by the weight selection iteration method, and the panchromatic image data that does not meet the block adjustment condition can also be removed. For the hyperspectral image data set with a large ground coverage range, the position-based block block adjustment can be performed.

[0056] Further, after the error matching relationship is removed from the matching relationship to obtain the updated matching relationship, the target exterior orientation elements corresponding to the panchromatic image data can also be determined based on the updated matching relationship, the initial exterior orientation elements of the panchromatic image data and the initial values of the undetermined ground point coordinates corresponding to the panchromatic image data. It should be noted that the process of determining the target exterior orientation elements corresponding to the panchromatic image data based on the updated matching relationship, the initial exterior orientation elements of the panchromatic image data and the initial values of the undetermined ground point coordinates corresponding to the panchromatic image data is consistent with the process of determining the target exterior orientation elements corresponding to the panchromatic image data based on the updated matching relationship, the initial exterior orientation elements of the panchromatic image data and the initial values of the undetermined ground point coordinates corresponding to the panchromatic image data, and thus will not be described here.

[0057] S103, spatial forward intersection calculation and grid construction are performed on the target exterior orientation elements to obtain a regular grid.

[0058] In the embodiments of the present application, the spatial forward intersection calculation is first performed on the target exterior orientation elements to obtain the undetermined ground point coordinates, and the undetermined ground point coordinates are determined as a sparse point cloud. Then, the adaptive filtering of the sparse point cloud is performed to obtain a sparse ground point cloud, and the irregular triangular mesh is constructed based on the sparse ground point cloud. Finally, the regular grid is obtained by interpolation based on the irregular triangular mesh.

[0059] Specifically, the adaptive filtering of the sparse point cloud can be performed by using the multi-scale morphological filtering and the progressive triangular mesh encryption method.

[0060] It should be noted that, considering that the sparse ground point cloud plane coordinates do not overlap with each other, a two-dimensional Delaunay triangulation network can be constructed for the sparse ground point cloud. Linear interpolation is used when performing regular grid interpolation, that is, interpolation is performed on the spatial plane determined by the three points of the triangle.

[0061] It should be noted that, based on the matching relationship, the initial exterior orientation elements of the panchromatic image data, and the initial values of the to-be-determined ground point coordinates corresponding to the panchromatic image data, the process of determining the target exterior orientation elements corresponding to the panchromatic image data and the process of performing spatial forward intersection calculation on the target exterior orientation elements to obtain the to-be-determined ground point coordinates and determining the to-be-determined ground point coordinates as the sparse point cloud together constitute the bundle block adjustment process.

[0062] In S104, orthographic correction is performed on the hyperspectral image data according to the target exterior orientation elements and the regular grid to obtain an orthographic image corresponding to the hyperspectral image.

[0063] In the embodiments of the present application, an interpolation grid of the hyperspectral image is determined according to a preset interpolation interval; first image point coordinates of each grid point in the interpolation grid are obtained; second image point coordinates of each grid point in the interpolation grid are calculated according to the target exterior orientation elements and the regular grid; and orthographic correction is performed on the hyperspectral image data based on the first image point coordinates and the second image point coordinates to obtain an orthographic image.

[0064] Specifically, the process of calculating the second image point coordinates of each grid point in the interpolation grid according to the target exterior orientation elements and the regular grid specifically includes: calculating the X-axis coordinate and the Y-axis coordinate of the to-be-determined ground point corresponding to each grid point according to a preset orthographic image scale parameter; determining the Z-axis coordinate of the to-be-determined ground point corresponding to each grid point according to the X-axis coordinate, the Y-axis coordinate, and the regular grid; and calculating the second image point coordinates of each grid point in the interpolation grid according to the target exterior orientation elements and the Z-axis coordinate.

[0065] It should be noted that the first image point coordinates are the corrected coordinates of the grid points in the interpolation grid, and the second image point coordinates are the uncorrected coordinates of the grid points in the difference grid.

[0066] In the embodiments of the present application, the second image point coordinates can be calculated according to formula (3).

[0067]

[0068] wherein λ is a variable that can be eliminated, f is the principal distance, m'1, m'1, n'1, n'2 are internal orientation transformation coefficients, a1, a2,..., c3 are the rotation matrix obtained by calculating the target exterior orientation elements, I0, J0 are the first image point coordinates, X S , Y S , Z SI, J are image coordinates corresponding to the grid points, and X, Y, Z are three-dimensional coordinates corresponding to the grid points. I, J, X, Y, and Z can all be used as second image point coordinates.

[0069] In the embodiment of the application, after obtaining the first image point coordinates and the second image point coordinates, the first image point coordinates and the second image point coordinates are subjected to bilinear difference value, to obtain the image point coordinates corresponding to other points in the hyperspectral image before correction, and then image point assignment is performed on the image point coordinates before correction, to finally obtain the orthographic image corresponding to the hyperspectral image.

[0070] It should be noted that the process of calculating the second image point coordinates of each grid point in the interpolation grid according to the target exterior orientation elements and the regular grid can specifically include: obtaining ground range data corresponding to the hyperspectral image; constructing a memory regular grid according to the ground range data and the regular grid; and calculating the second image point coordinates of each grid point in the interpolation grid according to the target exterior orientation elements and the memory regular grid. That is, before calculating the second image point coordinates of each grid point in the interpolation grid according to the target exterior orientation elements and the Z-axis coordinates, a memory regular grid is first constructed according to the ground range and the regular grid, to reduce the memory requirement.

[0071] Further, when calculating the second image point coordinates, shared memory parallel programming (Open Multi-Processing, OpenMP) can be used for parallel calculation to improve processing efficiency; and when performing orthographic correction on hyperspectral image data of a super large scale, a block correction strategy can be used.

[0072] It can be understood that the feature extraction and feature matching are performed on the panchromatic image data corresponding to the hyperspectral image data, and then the target exterior orientation elements obtained are used as the exterior orientation elements for orthographic correction of the hyperspectral image, and then the subsequent orthographic correction process is performed. The high-resolution characteristics of the panchromatic image can match more homonymous points, and then more target exterior orientation elements can be obtained, which can not be affected by the types of ground objects in the observation area; and when the target exterior orientation elements are used for orthographic correction of the hyperspectral image data, the spectral characteristics of the hyperspectral image are also retained, which can improve the accuracy of orthographic correction.

[0073] Based on the above embodiment, an embodiment of the application proposes an orthographic correction method for airborne hyperspectral images, as shown in Figure 2 The method can include the following steps.

[0074] 1. Obtain panchromatic image data corresponding to the hyperspectral image data and initial exterior orientation elements corresponding to the panchromatic image data.

[0075] 2. Perform feature extraction on the panchromatic image according to the initial exterior orientation elements, to obtain feature data corresponding to the panchromatic image data.

[0076] 3. performing feature matching on the panchromatic image data based on the feature data to obtain a matching relationship of the same name points in the panchromatic image data;

[0077] It should be noted that step 3 includes the following 3.1-3.5,

[0078] 3.1. calculating image center distances between the panchromatic images according to the initial exterior orientation elements;

[0079] 3.2. screening target image center distances smaller than an image distance threshold from the image center distances;

[0080] 3.3. calculating statistical feature data between the target image center distances;

[0081] 3.4. determining whether the same name points have the matching relationship in the panchromatic image data according to the statistical feature data;

[0082] 3.5. if it is determined that the same name points have the matching relationship in the panchromatic image data, performing feature matching on the panchromatic image data having the matching relationship by using a FLANN matching method to obtain the matching relationship of the same name points in the panchromatic image data.

[0083] 4. obtaining initial values of the to-be-determined ground point coordinates corresponding to each feature point of the panchromatic image data;

[0084] 5. determining target exterior orientation elements and sparse point clouds corresponding to the panchromatic image data according to the initial exterior orientation elements and the initial values of the to-be-determined ground point coordinates corresponding to each feature point of the panchromatic image data;

[0085] It should be noted that step 5 includes the following 5.1-5.3,

[0086] 5.1. establishing an error equation according to a preset collinear equation;

[0087] 5.2. inputting the matching relationship, the initial exterior orientation elements and the initial values of the to-be-determined ground point coordinates corresponding to each feature point of the panchromatic image data into the error equation to solve the target exterior orientation elements;

[0088] 5.3. performing space forward intersection calculation on the target exterior orientation elements to obtain the sparse point clouds;

[0089] 6. generating a regular grid according to the sparse point clouds;

[0090] 7. performing orthographic correction on the hyperspectral image according to the target exterior orientation elements and the regular grid to obtain an orthographic image corresponding to the hyperspectral image;

[0091] It should be noted that step 7 includes the following 7.1-7.3,

[0092] 7.1 Determine the interpolation grid of the hyperspectral image according to the preset interpolation interval; and obtain the corrected image point coordinates corresponding to each grid point in the interpolation grid;

[0093] 7.2. Based on the target's exterior orientation elements, the regular grid, and the corrected image point coordinates, determine the image point coordinates of each grid point before correction;

[0094] 7.3. Based on the coordinates of the first and second image points, orthorectify the hyperspectral image data to obtain an orthorectified image.

[0095] Based on the above embodiments, this application provides a hyperspectral image orthorectification device. For example... Figure 3 As shown, the hyperspectral image orthorectification device 1 includes:

[0096] Acquisition unit 10 is used to acquire panchromatic image data corresponding to the hyperspectral image data to be corrected;

[0097] The feature extraction and feature matching unit 11 is used to obtain the target exterior orientation elements corresponding to the panchromatic image data by performing feature extraction and feature matching on the panchromatic image data;

[0098] The grid generation unit 12 is used to perform spatial forward intersection calculation and grid construction on the target's exterior orientation elements to obtain a regular grid.

[0099] The orthorectification unit 13 is used to orthorectify the hyperspectral image data according to the target exterior orientation elements and the regular grid to obtain the orthorectified image corresponding to the hyperspectral image.

[0100] Optionally, the hyperspectral image orthorectification device 1 further includes: a feature extraction unit, a feature matching unit, and a determination unit;

[0101] The feature extraction unit is used to extract features from the panchromatic image data to obtain feature data corresponding to the panchromatic image data;

[0102] The feature matching unit is used to perform feature matching on the panchromatic image data based on the feature data to obtain the matching relationship of the corresponding points of the panchromatic image data in the panchromatic image data.

[0103] The determining unit is used to determine the target exterior orientation element corresponding to the panchromatic image data based on the matching relationship, the initial exterior orientation element of the panchromatic image data, and the initial value of the coordinates of the undetermined ground point corresponding to the panchromatic image data.

[0104] Optionally, the hyperspectral image orthorectification device 1 further includes: a calculation unit and a screening unit;

[0105] The acquisition unit 10 is further configured to acquire initial exterior orientation elements of the panchromatic image data;

[0106] The calculation unit is configured to calculate image center distances between the panchromatic images according to the initial exterior orientation elements, and calculate statistical feature data between the target image center distances;

[0107] The screening unit is configured to screen target image center distances smaller than an image distance threshold from the image center distances;

[0108] The determination unit is configured to determine a matching relationship of the same name points in the panchromatic image data based on the statistical feature data.

[0109] Optionally, the determination unit is further configured to determine an interpolation grid of the hyperspectral image according to a preset interpolation interval;

[0110] The acquisition unit 10 is further configured to acquire first image point coordinates of each grid point in the interpolation grid;

[0111] The calculation unit is further configured to calculate second image point coordinates of each grid point in the interpolation grid according to the target exterior orientation elements and the regular grid;

[0112] The orthorectification unit 13 is further configured to perform orthorectification on the hyperspectral image data based on the first image point coordinates and the second image point coordinates, to obtain the orthographic image.

[0113] Optionally, the calculation unit is further configured to calculate X-axis coordinates and Y-axis coordinates of a to-be-determined ground point corresponding to each grid point according to a preset orthographic image scale parameter, and calculate the second image point coordinates of each grid point in the interpolation grid according to the target exterior orientation elements and the Z-axis coordinates;

[0114] The determination unit is further configured to determine Z-axis coordinates of the to-be-determined ground point corresponding to each grid point according to the X-axis coordinates, the Y-axis coordinates and the regular grid.

[0115] Optionally, the hyperspectral image orthorectification device 1 further comprises a deletion unit;

[0116] The determination unit is further configured to determine a fundamental matrix corresponding to the matching relationship, determine an error matching relationship in the matching relationship based on the fundamental matrix, and determine target exterior orientation elements of the panchromatic image data based on the updated matching relationship, initial exterior orientation elements of the panchromatic image data and initial values of to-be-determined ground point coordinates corresponding to the panchromatic image data;

[0117] The deleting unit is configured to delete the error matching relationship from the matching relationship to obtain an updated matching relationship.

[0118] Optionally, the hyperspectral image orthorectification device 1 further comprises a constructing unit.

[0119] The obtaining unit 10 is further configured to obtain ground range data corresponding to the hyperspectral image.

[0120] The constructing unit is configured to construct an in-memory regular grid according to the ground range data and the regular grid.

[0121] The calculating unit is further configured to calculate second image point coordinates of each grid point in the interpolation grid according to the target exterior orientation elements and the in-memory regular grid.

[0122] The hyperspectral image orthorectification device provided by the embodiment of the present application obtains full-color image data corresponding to hyperspectral image data to be corrected; the full-color image data is sequentially subjected to feature extraction and feature matching to obtain target exterior orientation elements corresponding to the full-color image data; the target exterior orientation elements are subjected to space forward intersection calculation and grid construction to obtain a regular grid; and the hyperspectral image data is orthorectified according to the target exterior orientation elements and the regular grid to obtain an orthographic image corresponding to the hyperspectral image. As can be seen, the hyperspectral image orthorectification device provided by the embodiment of the present application uses the full-color image data corresponding to the hyperspectral image data to perform feature extraction and feature matching, and then uses the obtained target exterior orientation elements as exterior orientation elements for orthorectification of the hyperspectral image, and then performs subsequent orthorectification process. The high-resolution characteristic of the full-color image can match more homonymous points, and thus more target exterior orientation elements can be obtained, which can not be affected by the types of ground objects in the observation area. When the hyperspectral image data is orthorectified by using the target exterior orientation elements, the spectral characteristics of the hyperspectral image are also retained, which can improve the accuracy of orthorectification.

[0123] Figure 4 The hyperspectral image orthorectification device 1 provided by the embodiment of the present application has the component structure as shown in Figure 2 In actual application, based on the same disclosure concept of the above embodiment, as shown in Figure 4 The hyperspectral image orthorectification device 1 of the embodiment comprises a processor 14, a memory 15 and a communication bus 16.

[0124] In the process of the specific embodiment, the above-mentioned acquisition unit 10, feature extraction and feature matching unit 11, grid generation unit 12, orthorectification unit 13, feature extraction unit, feature matching unit, determination unit, calculation unit, screening unit, deletion unit and construction unit can be implemented by a processor 14 located on the hyperspectral image orthorectification device 1. The processor 14 can be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor. It can be understood that, for different devices, the electronic device used to implement the above-mentioned processor function can also be other, and the present embodiment is not limited specifically.

[0125] In the embodiment of the present application, the communication bus 16 is used to realize the connection and communication between the processor 14 and the memory 15. When the processor 14 executes the running program stored in the memory 15, the following hyperspectral image orthorectification method is realized:

[0126] Obtaining hyperspectral image data corresponding to the full-color image data to be corrected; by sequentially performing feature extraction and feature matching on the full-color image data, obtaining target exterior orientation elements corresponding to the full-color image data; performing spatial forward intersection calculation and grid construction on the target exterior orientation elements to obtain a regular grid; according to the target exterior orientation elements and the regular grid, orthorectifying the hyperspectral image data to obtain an orthographic image corresponding to the hyperspectral image.

[0127] Further, the processor 14 is further used to perform feature extraction on the full-color image data to obtain feature data corresponding to the full-color image data; perform feature matching on the full-color image data based on the feature data to obtain a matching relationship of the same name points of the full-color image data in the full-color image data; and determine the target exterior orientation elements corresponding to the full-color image data based on the matching relationship, initial exterior orientation elements of the full-color image data and initial values of the ground point coordinates to be determined corresponding to the full-color image data.

[0128] Further, the processor 14 is further configured to acquire initial exterior orientation elements of the panchromatic image data; calculate image center distances between the panchromatic images according to the initial exterior orientation elements; select target image center distances smaller than an image distance threshold from the image center distances; calculate statistical feature data between the target image center distances; and determine a matching relationship of the corresponding points in the panchromatic image data based on the statistical feature data.

[0129] Further, the processor 14 is further configured to determine an interpolation grid of the hyperspectral image according to a preset interpolation interval; acquire first image point coordinates of each grid point in the interpolation grid; calculate second image point coordinates of each grid point in the interpolation grid according to the target exterior orientation elements and the regular grid; and perform orthographic correction on the hyperspectral image data based on the first image point coordinates and the second image point coordinates to obtain the orthographic image.

[0130] Further, the processor 14 is further configured to calculate X-axis coordinates and Y-axis coordinates of a to-be-determined ground point corresponding to each grid point according to a preset orthographic image scale parameter; determine Z-axis coordinates of the to-be-determined ground point corresponding to each grid point according to the X-axis coordinates, the Y-axis coordinates and the regular grid; and calculate the second image point coordinates of each grid point in the interpolation grid according to the target exterior orientation elements and the Z-axis coordinates.

[0131] Further, the processor 14 is further configured to determine a fundamental matrix corresponding to the matching relationship; determine an error matching relationship in the matching relationship based on the fundamental matrix; delete the error matching relationship from the matching relationship to obtain an updated matching relationship; and determine target exterior orientation elements of the panchromatic image data based on the updated matching relationship, initial exterior orientation elements of the panchromatic image data and initial values of to-be-determined ground point coordinates corresponding to the panchromatic image data.

[0132] Further, the processor 14 is further configured to acquire ground range data corresponding to the hyperspectral image; construct an in-memory regular grid according to the ground range data and the regular grid; and calculate the second image point coordinates of each grid point in the interpolation grid according to the target exterior orientation elements and the in-memory regular grid.

[0133] The embodiment of the present application provides a storage medium having a computer program stored thereon, the computer readable storage medium stores one or more programs, the one or more programs can be executed by one or more processors, and is applied to a hyperspectral image orthographic correction device, and the computer program implements the hyperspectral image orthographic correction method.

[0134] It should be noted that, in this text, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or apparatus that includes a list of elements not only includes those elements, but also includes other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0135] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and the necessary general hardware platform, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for causing an image display device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present disclosure.

[0136] The above description is only a preferred embodiment of the present application, and is not intended to limit the protection scope of the present application.

Claims

1. A method for orthorectifying hyperspectral images, characterized in that, The method includes: Obtain the panchromatic image data corresponding to the hyperspectral image data to be corrected; Feature extraction is performed on the panchromatic image data to obtain the feature data corresponding to the panchromatic image data; Based on the feature data, feature matching is performed on the panchromatic image data to obtain the matching relationship of the corresponding points of the panchromatic image data in the panchromatic image data. Based on the matching relationship, the initial exterior orientation elements of the panchromatic image data, and the initial values ​​of the coordinates of the undetermined ground points corresponding to the panchromatic image data, the target exterior orientation elements corresponding to the panchromatic image data are determined. Spatial forward intersection calculation and grid construction are performed on the exterior orientation elements of the target to obtain a regular grid; Based on the target exterior orientation elements and the regular grid, the hyperspectral image data is orthorectified to obtain the orthorectified image corresponding to the hyperspectral image.

2. The method according to claim 1, characterized in that, The step of performing feature matching on the panchromatic image data based on the feature data to obtain the matching relationship of corresponding points in the panchromatic image data includes: Obtain the initial exterior orientation elements of the panchromatic image data; Based on the initial exterior orientation elements, calculate the image center distance between the panchromatic images; Target image center distances that are less than the image distance threshold are selected from the image center distances. Calculate the statistical feature data between the center distances of the target images; Based on the statistical feature data, the matching relationship of the corresponding points in the panchromatic image data is determined.

3. The method according to claim 1, characterized in that, The step of performing orthorectification on the hyperspectral image data based on the target exterior orientation elements and the regular grid to obtain an orthorectified image corresponding to the hyperspectral image includes: The interpolation grid of the hyperspectral image is determined according to the preset interpolation interval; and the coordinates of the first image point of each grid point in the interpolation grid are obtained. Based on the target exterior orientation elements and the regular grid, calculate the second image point coordinates of each grid point in the interpolation grid; Based on the coordinates of the first image point and the coordinates of the second image point, orthorectification is performed on the hyperspectral image data to obtain an orthorectified image.

4. The method according to claim 3, characterized in that, The step of calculating the second image point coordinates of each grid point in the interpolation grid based on the target exterior orientation elements and the regular grid includes: Based on the preset orthophoto scale parameters, calculate the X-axis and Y-axis coordinates of the ground point to be determined corresponding to each grid point; Based on the X-axis coordinates, the Y-axis coordinates, and the regular grid, determine the Z-axis coordinates of the ground point to be determined corresponding to each grid point; Based on the target exterior orientation elements and the Z-axis coordinates, calculate the second image point coordinates of each grid point in the interpolation grid.

5. The method according to claim 1, characterized in that, After performing feature matching on the panchromatic image data based on the feature data to obtain the matching relationship of corresponding points in the panchromatic image data, and before determining the target exterior orientation element corresponding to the panchromatic image data based on the matching relationship, the initial exterior orientation element of the panchromatic image data, and the initial value of the coordinates of the undetermined ground point corresponding to the panchromatic image data, the method further includes: Determine the basic matrix corresponding to the matching relationship; Based on the fundamental matrix, the error matching relationship in the matching relationship is determined; Delete the erroneous matching relationship from the matching relationship to obtain the updated matching relationship; Accordingly, determining the target exterior orientation element corresponding to the panchromatic image data based on the matching relationship, the initial exterior orientation element of the panchromatic image data, and the initial values ​​of the coordinates of the undetermined ground points corresponding to the panchromatic image data includes: Based on the updated matching relationship, the initial exterior orientation elements of the panchromatic image data, and the initial values ​​of the coordinates of the undetermined ground points corresponding to the panchromatic image data, the target exterior orientation elements corresponding to the panchromatic image data are determined.

6. The method according to claim 4, characterized in that, The step of calculating the second image point coordinates of each grid point in the interpolation grid based on the target exterior orientation elements and the regular grid includes: Obtain the ground extent data corresponding to the hyperspectral image; Based on the ground area data and the rule grid, a memory rule grid is constructed; Based on the target exterior orientation elements and the memory rule grid, calculate the second image point coordinates of each grid point in the interpolation grid.

7. A hyperspectral image orthorectification device, characterized in that, The device includes: The acquisition unit is used to acquire panchromatic image data corresponding to the hyperspectral image data to be corrected. The feature extraction unit is used to extract features from the panchromatic image data to obtain feature data corresponding to the panchromatic image data. A feature matching unit is used to perform feature matching on the panchromatic image data based on the feature data to obtain the matching relationship of the corresponding points of the panchromatic image data in the panchromatic image data. The determining unit is used to determine the target exterior orientation element corresponding to the panchromatic image data based on the matching relationship, the initial exterior orientation element of the panchromatic image data, and the initial value of the coordinates of the undetermined ground point corresponding to the panchromatic image data. A grid generation unit is used to perform spatial forward intersection calculations and grid construction on the exterior orientation elements of the target to obtain a regular grid. The orthorectification unit is used to orthorectify the hyperspectral image data according to the target exterior orientation elements and the regular grid to obtain the orthorectified image corresponding to the hyperspectral image.

8. A hyperspectral image orthorectification device, characterized in that, The device includes: a processor, a memory, and a communication bus; when the processor executes the running program stored in the memory, it implements the method as described in any one of claims 1-6.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images