RGB-based hyperspectral image correction splicing method and system, and storage medium

Abstract

The invention provides a hyperspectral image correction and splicing method and system based on RGB and a storage medium. The method comprises the steps that an original hyperspectral air strip image file and an original GPS frame file are acquired through a hyperspectral acquisition unit; determining an air strip start-stop frame according to the original hyperspectral air strip image file and the original GPS frame file, cutting out to-be-corrected hyperspectral air strip data from the original hyperspectral air strip image file, and converting the original GPS frame file into three-dimensional coordinate system data; obtaining an RGB contrast image and an earth surface model image corresponding to the original hyperspectral air strip image file, and correcting the to-be-corrected hyperspectral air strip data to obtain corrected hyperspectral air strip data; and obtaining an RGB (Red, Green and Blue) orthographic image, and carrying out air strip mapping, overlapping and splicing processing on the corrected hyperspectral air strip data by taking the RGB orthographic image as a reference to obtain a hyperspectral spliced air strip image file. According to the invention, the splicing precision and integrity of the hyperspectral air strip data are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, system and storage medium for hyperspectral image correction and stitching based on RGB. Background Technology

[0002] Hyperspectral images, with their rich spectral information, can accurately reveal the composition, internal structure, and inherent properties of target objects, and have irreplaceable application value in many fields such as land monitoring, resource exploration, and environmental assessment. With the synergistic development of aerospace technology and spectral sensing technology, line-scan hyperspectral imaging technology based on UAV platforms has become a core means of acquiring hyperspectral data of large-scale land areas due to its high flexibility, low data acquisition cost, and excellent spatiotemporal resolution, driving technological innovation in fields such as agricultural remote sensing, ecological monitoring, and geological exploration. However, current UAV line-scan hyperspectral imaging technology still faces many technical bottlenecks in practical applications. During data acquisition, the raw hyperspectral data generally suffers from spatial distortion and georegistration errors due to factors such as complex terrain, sensor optical distortion, and hardware precision limitations. This leads to a decrease in spatial positioning accuracy, failing to meet the demands of high-precision applications. Furthermore, because the field of view of a single UAV line scan is limited, full-coverage imaging of large scenes requires planning multiple flight strips to collect data sequentially, followed by post-processing stitching of these flight strips. Existing methods for stitching flight strips largely rely on manual planning of the flight strip paths, and require manual adjustment of parameters such as translation, rotation, and scaling to achieve strip alignment during the stitching process. This approach is not only inefficient and unsuitable for large-scale data processing, but also prone to subjective errors introduced by manual operation, leading to inaccurate matching of geographical information between different flight strips and problems such as data breakage, overlap, or misalignment in the stitched data. Furthermore, in areas with significant terrain undulations, when UAVs cruise and scan at a fixed speed, the relative distance and imaging angle between the top of tall buildings and the sensor change rapidly, further exacerbating image compression distortion and severely affecting the integrity and usability of hyperspectral data, thus limiting the large-scale promotion and application of UAV line-scan hyperspectral imaging technology. Summary of the Invention

[0003] This invention provides a hyperspectral image correction and stitching method, system, and storage medium based on RGB, aiming to solve the technical problems of errors and distortions that are easily caused by existing hyperspectral flight strip acquisition and stitching methods.

[0004] To address the aforementioned technical problems, in a first aspect, the present invention provides a hyperspectral image correction and stitching method based on RGB, comprising the following steps: S101. Obtain the original hyperspectral flight strip image file and its corresponding original GPS frame file by capturing images through the hyperspectral acquisition unit; S102. Determine the start and end frames of the flight strip based on the original hyperspectral flight strip image file and the original GPS frame file, and cut out the hyperspectral flight strip data to be corrected from the original hyperspectral flight strip image file based on the start and end frames of the flight strip. At the same time, convert the part of the original GPS frame file corresponding to the hyperspectral flight strip data to be corrected into three-dimensional coordinate system data. S103. Obtain the RGB comparison image and the surface model image corresponding to the original hyperspectral flight strip image file, and correct the hyperspectral flight strip data to be corrected according to the three-dimensional coordinate system data, the RGB comparison image and the surface model image to obtain the corrected hyperspectral flight strip data. S104. Obtain an RGB orthophoto image used as the base image for stitching, perform flight strip mapping, overlapping and stitching processing on different corrected hyperspectral flight strip data based on the RGB orthophoto image, and output a hyperspectral stitched flight strip image file.

[0005] Furthermore, step S102 includes the following sub-steps: S1021. Convert the latitude and longitude coordinates in the original GPS frame file into local planar coordinates with the first point as the origin; S1022. Calculate the heading angle and turning rate per unit distance when the original hyperspectral flight path image file was captured based on the local plane coordinates, and determine whether the original hyperspectral flight path image file was in a straight flight or turning state when it was captured. S1023. Determine the start and end frames of each flight strip in the original hyperspectral flight strip image file according to the action status, and use them as the start and end frames of the flight strip. S1024. Based on the start and end frames of the flight strip, cut out the hyperspectral flight strip data to be corrected from the original hyperspectral flight strip image file. S1025. Extract the GPS information corresponding to the hyperspectral flight strip data to be corrected from the original GPS frame file and convert it into the three-dimensional geographic coordinate system data.

[0006] Furthermore, step S103 includes the following sub-steps: S1031. Determine the spatial range of the hyperspectral flight strip data to be corrected in the three-dimensional geographic coordinate system data, and use it as the project area to be processed. S1032. Based on the project area to be processed, crop the RGB comparison image and the surface model image, and output a cropped surface model file with geographic information; S1033. Based on the shooting parameters of the hyperspectral acquisition unit, generate a sensor angle model containing pixel indexes and horizontal and vertical viewing angles. S1034. By traversing the scan lines of the hyperspectral flight strip data to be corrected, and combining the action state with the sensor angle model, a scan line line direction vector matrix is ​​generated that is unified to the three-dimensional geographic coordinate system data. S1035. Using the sensor position as the starting point and the line-of-sight vector as the direction, construct the endpoints of the light segments for each pixel in the hyperspectral flight strip data to be corrected, in conjunction with the upper and lower bounds of the cropped surface model file. S1036. The intersection point between the ray segment and the ground surface of the hyperspectral acquisition unit is obtained by using a ray tracing algorithm to obtain the real-world coordinates of each pixel; S1037. Integrate the world coordinates of all the aforementioned pixels to generate a pixel geographic coordinate file; S1038. Generate a geographic lookup table based on the pixel geographic coordinate file. The geographic lookup table is used to establish the mapping relationship between the grid of the surface model image and the hyperspectral flight strip data to be corrected. S1039. Based on the geographic lookup table, the original hyperspectral flight strip data to be corrected is resampled, and the corrected hyperspectral flight strip data with geographic information of the three-dimensional geographic coordinate system data is output.

[0007] Furthermore, step S104 includes the following sub-steps: S1041. Convert the RGB orthophoto image to a coordinate system consistent with the original hyperspectral flight strip image file, and scale it to an adapted grayscale image according to the hyperspectral resolution; S1042. Extract the RGB pseudo-color bands from the corrected hyperspectral flight strip data and convert them into a hyperspectral grayscale image; S1043. Using the adapted grayscale image as a reference, feature points are extracted and matched from the hyperspectral grayscale image to obtain feature point pairs; S1044. Calculate the global homography matrix based on the feature point pairs; S1045. The adapted grayscale image is meshed, and the local homography matrix of each mesh is calculated. S1046. Perform steps S1042-S1045 on all the corrected hyperspectral flight strip data that need to be stitched together to obtain the global homography matrix and the local homography matrix of each corrected hyperspectral flight strip data. S1047. Read all the corrected hyperspectral flight strip data that need to be stitched band by band, perform coordinate mapping according to the global homography matrix and the local homography matrix, perform fusion processing on the overlapping areas, integrate all band data, and output the hyperspectral stitched flight strip image file.

[0008] Furthermore, step S1021 also includes: performing symmetrical moving average filtering on the local planar coordinates to smooth and denoise the data and eliminate outliers.

[0009] Furthermore, step S1036 includes the following sub-steps: The coordinates of the two endpoints of the light segments of the pixel are defined as follows: , ,Will , Pixel coordinates mapped to the cropped surface model file ; A discrete successive approximation method is employed, traversing the 2D raster grid cell by cell to progressively approximate the pixels containing the intersection points. Specifically, for the currently traversed pixel, the terrain height *h* of that pixel in the cropped surface model file is substituted into the following expression to calculate the estimated pixel coordinates of the intersection point. : ; in, The height of the ray origin. It is the ray direction vector Quantity, It is a parameter quantity that travels along the ray from the starting point; If the estimated intersection point's pixel coordinates fall within the coordinate range of the previously traversed pixels, the estimated intersection point's coordinates are used as the pixel's real-world coordinates and output.

[0010] Furthermore, step S1041 also includes: performing morphological dilation and cropping processing on the adapted grayscale image based on the range of geographical information of the corrected hyperspectral flight strip data; Step S1044 further includes: performing medium-grained correction processing on the hyperspectral grayscale image based on the global homography matrix.

[0011] Secondly, the present invention also provides an RGB-based hyperspectral image correction and stitching system, comprising: The data acquisition module is used to capture and obtain raw hyperspectral flight strip image files and their corresponding raw GPS frame files through the hyperspectral acquisition unit; The flight strip identification and cropping module is used to determine the start and end frames of the flight strip based on the original hyperspectral flight strip image file and the original GPS frame file, and to crop the hyperspectral flight strip data to be corrected from the original hyperspectral flight strip image file based on the start and end frames of the flight strip. At the same time, the part of the original GPS frame file corresponding to the hyperspectral flight strip data to be corrected is converted into three-dimensional coordinate system data. The hyperspectral correction module is used to acquire an RGB reference image and a surface model image corresponding to the original hyperspectral flight strip image file, and to correct the hyperspectral flight strip data to be corrected based on the three-dimensional coordinate system data, the RGB reference image and the surface model image to obtain corrected hyperspectral flight strip data. The hyperspectral stitching module is used to acquire an RGB orthophoto image as the stitching base map, and to perform flight strip mapping, overlapping and stitching processing on different corrected hyperspectral flight strip data based on the RGB orthophoto image, and output a hyperspectral stitched flight strip image file.

[0012] Thirdly, the present invention also provides a computer device, including: a memory, a processor, and an RGB-based hyperspectral image correction and stitching program stored in the memory and executable on the processor, wherein when the processor executes the RGB-based hyperspectral image correction and stitching program, it implements the steps of the RGB-based hyperspectral image correction and stitching method as described in any of the above embodiments.

[0013] Fourthly, the present invention also provides a storage medium storing an RGB-based hyperspectral image correction and stitching program, wherein when the RGB-based hyperspectral image correction and stitching program is executed by a processor, it implements the steps of the RGB-based hyperspectral image correction and stitching method as described in any of the above embodiments.

[0014] The beneficial effects achieved by this invention are that it proposes a hyperspectral image correction and stitching method based on RGB. This method effectively solves the problems of low efficiency, large subjective error and poor geographic information alignment of traditional methods by using automated flight strip identification and cropping, accurate geographic correction and stitching technology based on RGB orthophotos, thereby improving the stitching accuracy and integrity of hyperspectral flight strip data. Attached Figure Description

[0015] The present invention will now be described in detail with reference to the accompanying drawings. The above and other aspects of the present invention will become clearer and more readily understood through the detailed description following the accompanying drawings. In the drawings: Figure 1 This is a flowchart of the steps of the RGB-based hyperspectral image correction and stitching method provided in the embodiments of the present invention; Figure 2This is a schematic diagram illustrating the process of trimming hyperspectral flight strip data to be corrected according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the endpoint position of the light segment provided in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the process of acquiring corrected hyperspectral flight strip data provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the flight strip splicing process provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of the RGB-based hyperspectral image correction and stitching system provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0017] Example 1 Please refer to Figure 1 , Figure 1 This is a flowchart illustrating the steps of the RGB-based hyperspectral image correction and stitching method provided in this embodiment of the invention. The RGB-based hyperspectral image correction and stitching method includes the following steps: S101. Obtain the original hyperspectral flight strip image file and its corresponding original GPS frame file by capturing images through the hyperspectral acquisition unit.

[0018] As described in the background section of this application, hyperspectral flight strip images are acquired by high-altitude flight equipment such as drones using a hyperspectral acquisition unit with hyperspectral imaging capabilities. In this embodiment of the invention, it is also necessary to acquire the original GPS frame file captured during the acquisition of the original hyperspectral flight strip image file. Generally, the original GPS frame file is acquired simultaneously by the capturing device using other sensors, and its data, such as capture time, location, frame number, and flight attitude, strictly correspond to the situation when the hyperspectral acquisition unit captured the original hyperspectral flight strip image file.

[0019] S102. Determine the start and end frames of the flight strip based on the original hyperspectral flight strip image file and the original GPS frame file, and cut out the hyperspectral flight strip data to be corrected from the original hyperspectral flight strip image file based on the start and end frames of the flight strip. At the same time, convert the part of the original GPS frame file corresponding to the hyperspectral flight strip data to be corrected into three-dimensional coordinate system data.

[0020] Specifically, step S102 includes the following sub-steps: S1021. Convert the latitude and longitude coordinates in the original GPS frame file into local planar coordinates with the origin as the first point; further, step S1021 also includes: performing symmetrical moving average filtering on the local planar coordinates to smooth and eliminate outliers.

[0021] In this embodiment of the invention, the latitude and longitude coordinates are converted into local planar coordinates with the first point as the origin using the equidistant cylindrical approximation method. The coordinate transformation process can be expressed as follows: ; in, , , , This represents the number of latitude and longitude coordinates that need to be converted. These represent the x-coordinate and y-coordinate of the local plane coordinates, respectively. Represents the Earth's radius. These represent the radian values ​​of the origin's coordinates in longitude and latitude, respectively. The radian value representing the reference latitude. These represent the longitude and latitude of the current coordinates, respectively. These represent the radian values ​​of the current coordinates' longitude and latitude, respectively.

[0022] Furthermore, in order to reduce noise interference and improve the stability of subsequent difference calculations, this embodiment of the invention also uses a symmetric moving average filter for coordinate smoothing filtering. Specifically, a forced odd window is created to ensure window symmetry, and edge padding is applied to both ends of the data to reduce boundary effects. Then, a normalized mean convolution kernel is constructed to perform convolution operations on it to achieve efficient smoothing.

[0023] S1022. Calculate the heading angle and turning rate per unit distance when the original hyperspectral flight path image file was captured based on the local plane coordinates, and determine whether the original hyperspectral flight path image file was in a straight flight or turning state when it was captured.

[0024] Specifically, heading angle The turning rate per unit distance is calculated using local planar coordinates. It satisfies the following relationship: ; , ; in, These represent the displacement components of adjacent points on the horizontal and vertical coordinate axes, respectively. The function represents the unwinding heading angle, used to eliminate... Jump.

[0025] When specifically judging the action status, the embodiments of the present invention perform robust threshold calculation on the obtained heading angle and turning rate per unit distance, and perform hysteresis comparison to avoid the problems of outlier sensitivity and jitter of the single threshold method, and provide stable attitude judgment results.

[0026] S1023. Determine the start and end frames of each flight strip in the original hyperspectral flight strip image file according to the action status, and use them as the start and end frames of the flight strip.

[0027] Specifically, in step S1023, overlapping flight strip segments are merged according to morphological closing operations to finally obtain the start frame and end frame of each flight strip.

[0028] S1024. Based on the start and end frames of the flight strip, cut out the hyperspectral flight strip data to be corrected from the original hyperspectral flight strip image file.

[0029] S1025. Extract the GPS information corresponding to the hyperspectral flight strip data to be corrected from the original GPS frame file and convert it into the three-dimensional geographic coordinate system data.

[0030] For the effect achieved in step S102, please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the process of cropping hyperspectral flight strip data to be corrected according to an embodiment of the present invention, wherein... Figure 2 In the image, 'a' represents the original GPS frame file, cropped based on the start and end frames of the flight strip, and converted using the user's actual geographic information to obtain GPS data. Figure 2 In section b, the pseudo-color image of each hyperspectral flight strip to be corrected is cropped from the original hyperspectral flight strip image file based on the start and end frames of the flight strip. Step S102 optimizes the flight strip data by removing abnormal inflection points and invalid short flight strip segments, and converts the latitude and longitude coordinates of the shooting location into corresponding two-dimensional ground coordinates, which facilitates subsequent accurate distance estimation and area calculation. The automatically cropped hyperspectral flight strip data to be corrected can also be quickly adjusted for orientation according to the flight attitude, which is beneficial for subsequent stitching operations.

[0031] S103. Obtain the RGB comparison image and the surface model image corresponding to the original hyperspectral flight strip image file, and correct the hyperspectral flight strip data to be corrected according to the three-dimensional coordinate system data, the RGB comparison image and the surface model image to obtain the corrected hyperspectral flight strip data.

[0032] Specifically, step S103 includes the following sub-steps: S1031. Determine the spatial range of the hyperspectral flight strip data to be corrected in the three-dimensional geographic coordinate system data, and use it as the project area to be processed.

[0033] Specifically, in step S1031, the process involves initially determining the envelope of the track in the planar coordinate system by extracting the maximum and minimum coordinate values ​​from the 3D geographic coordinate system data. Then, considering the field-of-view coverage (half-width) of the hyperspectral acquisition unit and the additional buffer, the area to be processed that needs to be clipped is obtained. The formula for calculating the half-width is: , ; Indicates the horizontal field of view angle. Indicates the horizontal field of view in radians. This represents the maximum height of the ground from the hyperspectral acquisition unit. The purpose of calculating the half-width is to ensure that all pixels are covered by the envelope during cropping.

[0034] S1032. Based on the project area to be processed, crop the RGB comparison image and the surface model image, and output a cropped surface model file with geographic information.

[0035] S1033. Based on the shooting parameters of the hyperspectral acquisition unit, generate a sensor angle model containing pixel indexes and horizontal and vertical viewing angles.

[0036] The sensor angle model is implemented in text format, with each line of the file corresponding to the index of a pixel and its horizontal and vertical viewing angles in radians.

[0037] S1034. By traversing the scan lines of the hyperspectral flight strip data to be corrected, and combining the action state with the sensor angle model, a scan line line direction vector matrix unified to the three-dimensional geographic coordinate system data is generated.

[0038] In this embodiment of the invention, the scan line refers to a row of pixel data acquired by the hyperspectral acquisition unit in a single horizontal scan. It is the basic unit for constructing a complete hyperspectral image. In step S1034, based on the attitude information of each scan line and the sensor model, a line-of-sight vector matrix of each scan line is generated and unified into the line-of-sight vector matrix of the three-dimensional geographic coordinate system data.

[0039] S1035. Using the sensor position as the starting point and the line-of-sight vector as the direction, construct the endpoint of the light segment for each pixel in the hyperspectral flight strip data to be corrected, in conjunction with the upper and lower bounds of the cropped surface model file.

[0040] Specifically, given the sensor position for each scan line and the direction vector of each probe element in that line... Using the upper and lower bounds of the surface model file , Construct the endpoint of the ray segment corresponding to each pixel. , This serves as the initial parameter range for subsequent intersection of ray tracing and surface model data.

[0041] S1036. The intersection point of the ray segment and the ground surface of the hyperspectral acquisition unit is obtained by using a ray tracing algorithm to obtain the real-world coordinates of each pixel.

[0042] Specifically, step S1036 includes the following sub-steps: like Figure 3 As shown, the coordinates of the endpoints of the two ray segments of the pixel are defined as follows: , ,Will , Pixel coordinates mapped to the cropped surface model file ; A discrete successive approximation method is employed, traversing the 2D raster grid cell by cell to progressively approximate the pixels containing the intersection points. Specifically, for the currently traversed pixel, the terrain height *h* of that pixel in the cropped surface model file is substituted into the following expression to calculate the estimated pixel coordinates of the intersection point. : ; in, The height of the ray origin. It is the ray direction vector Quantity, It is a parameter quantity that travels along the ray from the starting point; If the estimated intersection point's pixel coordinates fall within the coordinate range of the previously traversed pixels, the estimated intersection point's coordinates are used as the pixel's real-world coordinates and output.

[0043] S1037. Integrate the world coordinates of all the aforementioned pixels to generate a pixel geographic coordinate file.

[0044] S1038. Generate a geographic lookup table based on the pixel geographic coordinate file. The geographic lookup table is used to establish the mapping relationship between the grid of the surface model image and the hyperspectral flight strip data to be corrected.

[0045] When constructing the mapping relationship, the forward mapping of the original image's coordinate system (rows and columns) to map coordinates (X and Y) is reversed and applied to the target projection grid, thereby obtaining a lookup table of target projection pixels to original image rows and columns.

[0046] S1039. Based on the geographic lookup table, the original hyperspectral flight strip data to be corrected is resampled, and the corrected hyperspectral flight strip data with geographic information of the three-dimensional geographic coordinate system data is output.

[0047] Step S103: Obtaining the physical effect of corrected hyperspectral flight strip data. Figure 4 As shown, according to steps S1031-S1039, ray tracing is used to approximate the true three-dimensional coordinates. During implementation, the hyperspectral flight strip data to be corrected is three-channel, representing the three-dimensional coordinate values ​​of the original hyperspectral data in the true geographic coordinate system. Based on the information of the hyperspectral flight strip data to be corrected, a true geographic lookup table is formed through stretching and interpolation. This geographic lookup table is two-band, representing the row and column indices corresponding to the original hyperspectral image, and is resampled to the target map coordinate system according to the specified spatial resolution and projection. Finally, the original hyperspectral data is sampled based on the geographic lookup table to obtain the corrected hyperspectral flight strip data.

[0048] Furthermore, in this embodiment of the invention, each hyperspectral flight strip data to be corrected obtained in step S102 is processed in the complete manner of steps S1031-S1039. Since each hyperspectral flight strip data to be corrected is obtained by cropping specific flight strip start and end frames, they do not interfere with each other. When step S103 is executed, it can be processed in parallel to improve processing efficiency.

[0049] S104. Obtain an RGB orthophoto image used as the base image for stitching, perform flight strip mapping, overlapping and stitching processing on different corrected hyperspectral flight strip data based on the RGB orthophoto image, and output a hyperspectral stitched flight strip image file.

[0050] Specifically, the RGB orthophoto image is a complete image that has undergone special stitching processing after being captured. It already possesses a unified coordinate system and seamless stitching effect. Its core function is to serve as a stitching base map, guiding the precise alignment of hyperspectral flight strips through feature point matching to achieve large-scale seamless stitching. Step S104 includes the following sub-steps: S1041. Convert the RGB orthophoto image to a coordinate system consistent with the original hyperspectral flight strip image file, and scale it to an adapted grayscale image according to the hyperspectral resolution; step S1041 further includes: performing morphological dilation and cropping processing on the adapted grayscale image according to the range of geographical information of the corrected hyperspectral flight strip data.

[0051] In step S1041, firstly, based on the size of the RGB orthogonal image and the hyperspectral pixel, the RGB orthogonal image is upsampled or downsampled to minimize computational complexity while ensuring no loss of hyperspectral spatial resolution, and to obtain the grayscale image corresponding to the RGB orthogonal image. Secondly, based on the geographical extent of the hyperspectral data of the flight strip in the corresponding original hyperspectral flight strip image file, the grayscale image is cropped after morphological dilation of m pixels. This achieves coarse-grained registration of geographical information while reducing subsequent computational load and minimizing large-scale feature point mismatch issues, resulting in an adapted grayscale image for computation. And record its cutting position.

[0052] S1042. Extract the RGB pseudo-color bands from the corrected hyperspectral flight strip data and convert them into a hyperspectral grayscale image.

[0053] Define the hyperspectral grayscale image obtained in this step as follows: .

[0054] S1043. Using the adapted grayscale image as a reference, feature points are extracted and matched on the hyperspectral grayscale image to obtain feature point pairs.

[0055] Specifically, in step S1043, the two grayscale images obtained in steps S1041 and S1042 are extracted using feature extraction algorithms such as SIFT and SURF. , feature point pairs between .

[0056] In order to improve the accuracy of feature matching, step S1043 can also use the FLANN algorithm to match feature point pairs, and then clean the feature point pairs to remove duplicate, clustered, and low-quality matching points.

[0057] S1044. Calculate the global homography matrix based on the feature point pairs; step S1044 further includes: performing medium-grained correction processing on the hyperspectral grayscale image according to the global homography matrix.

[0058] The global homography matrix can be represented as It satisfies: ; in, , , which are the matching points in the two grayscale images, respectively.

[0059] S1045. The adapted grayscale image is meshed, and the local homography matrix of each mesh is calculated.

[0060] During implementation, in order to improve the matching accuracy of local features, the hyperspectral grayscale image after correction in step S1044 should be processed in the same way as the feature point cleaning in step S1043, and the source feature points and target feature points should be selected according to the inner point array. Next, all feature point pairs are normalized and conditionalized so that the homogeneous coordinates of each set of feature points are substituted into the linear constraint formula of homography transformation, generating two rows of coefficients for each set of points. The coefficients generated for all point pairs are stacked row by row to construct matrix A for the weighted DLT algorithm. Finally, adapt the grayscale image. Meshization is performed by traversing the mesh nodes and solving the local homography matrix using a weighted least squares method based on feature point distances. This process satisfies the following relationship: ; in, It is the grid distance weight parameter matrix.

[0061] The obtained global homography matrix is ​​calculated based on global feature point pairs and is used to describe the global geometric transformation relationship between two images. The obtained local homography matrix is ​​calculated separately for each sub-region by dividing the image into multiple sub-regions through gridding on the basis of global correction. Since a distance-related weight parameter is introduced, feature points closer to the current grid node have higher weights. Therefore, the obtained local homography matrix can more accurately reflect the local transformation characteristics of the region.

[0062] S1046. Perform steps S1042-S1045 on all the corrected hyperspectral flight strip data that need to be stitched together to obtain the global homography matrix and the local homography matrix of each corrected hyperspectral flight strip data.

[0063] It is understandable that, similar to step S103, when processing different corrected hyperspectral flight strip data in steps S1042-S1045, parallel processing can be used to improve processing efficiency.

[0064] S1047. Read all the corrected hyperspectral flight strip data that need to be stitched band by band, perform coordinate mapping according to the global homography matrix and the local homography matrix, perform fusion processing on the overlapping areas, integrate all band data, and output the hyperspectral stitched flight strip image file.

[0065] A band-by-band reading method is used to perform a first mapping using a global homography matrix, followed by a second mapping using a group of local homography matrices. The second mapping employs a reverse mapping sampling method, which involves sampling each pixel on the target canvas. Find its corresponding position in the source image through geometric transformation. Then, the source image is sampled and the value is assigned to that pixel.

[0066] During implementation, overlap between flight strips is inevitable. A direct substitution method can be used to merge the overlapping areas. The flight strip splicing process in step S105 is as follows: Figure 5 As shown, the transformed band data is saved band by band, and finally a hyperspectral stitched flight strip image file is obtained.

[0067] The beneficial effects achieved by this invention are that it proposes a hyperspectral image correction and stitching method based on RGB. This method effectively solves the problems of low efficiency, large subjective error and poor geographic information alignment of traditional methods by using automated flight strip identification and cropping, accurate geographic correction and stitching technology based on RGB orthophotos, thereby improving the stitching accuracy and integrity of hyperspectral flight strip data.

[0068] Example 2 This invention also provides an RGB-based hyperspectral image correction and stitching system 200, please refer to... Figure 6 , Figure 6 This is a schematic diagram of the structure of the RGB-based hyperspectral image correction and stitching system provided in an embodiment of the present invention, which includes: Data acquisition module 201 is used to capture and acquire raw hyperspectral flight strip image files and their corresponding raw GPS frame files through the hyperspectral acquisition unit; The flight strip identification and cropping module 202 is used to determine the start and end frames of the flight strip based on the original hyperspectral flight strip image file and the original GPS frame file, and to crop the hyperspectral flight strip data to be corrected from the original hyperspectral flight strip image file based on the start and end frames of the flight strip, while converting the part of the original GPS frame file corresponding to the hyperspectral flight strip data to be corrected into three-dimensional coordinate system data. The hyperspectral correction module 203 is used to acquire an RGB comparison image and a surface model image corresponding to the original hyperspectral flight strip image file, and to correct the hyperspectral flight strip data to be corrected based on the three-dimensional coordinate system data, the RGB comparison image and the surface model image to obtain corrected hyperspectral flight strip data. The hyperspectral stitching module 204 is used to acquire an RGB orthophoto image as a stitching base map, perform different corrected hyperspectral flight strip data on the basis of the RGB orthophoto image for flight strip mapping, overlapping and stitching, and output a hyperspectral stitched flight strip image file.

[0069] The RGB-based hyperspectral image correction and stitching system 200 can implement the steps in the RGB-based hyperspectral image correction and stitching method in the above embodiments and achieve the same technical effect. Referring to the description in the above embodiments, it will not be repeated here.

[0070] Example 3 This invention also provides a computer device, please refer to... Figure 7 , Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. The computer device 300 includes: a memory 302, a processor 301, and an RGB-based hyperspectral image correction and stitching program stored in the memory 302 and capable of running on the processor 301.

[0071] The processor 301 calls the RGB-based hyperspectral image correction and stitching program stored in the memory 302, and executes the steps in the RGB-based hyperspectral image correction and stitching method provided in this embodiment of the invention. Please refer to... Figure 1 Specifically, it includes the following steps: S101. Obtain the original hyperspectral flight strip image file and its corresponding original GPS frame file by capturing images through the hyperspectral acquisition unit.

[0072] S102. Determine the start and end frames of the flight strip based on the original hyperspectral flight strip image file and the original GPS frame file, and cut out the hyperspectral flight strip data to be corrected from the original hyperspectral flight strip image file based on the start and end frames of the flight strip. At the same time, convert the part of the original GPS frame file corresponding to the hyperspectral flight strip data to be corrected into three-dimensional coordinate system data.

[0073] Specifically, step S102 includes the following sub-steps: S1021. Convert the latitude and longitude coordinates in the original GPS frame file into local planar coordinates with the first point as the origin; further, step S1021 also includes: performing symmetrical moving average filtering on the local planar coordinates to smooth and eliminate outliers. S1022. Calculate the heading angle and turning rate per unit distance when the original hyperspectral flight path image file was captured based on the local plane coordinates, and determine whether the original hyperspectral flight path image file was in a straight flight or turning state when it was captured. S1023. Determine the start and end frames of each flight strip in the original hyperspectral flight strip image file according to the action status, and use them as the start and end frames of the flight strip. S1024. Based on the start and end frames of the flight strip, cut out the hyperspectral flight strip data to be corrected from the original hyperspectral flight strip image file. S1025. Extract the GPS information corresponding to the hyperspectral flight strip data to be corrected from the original GPS frame file and convert it into the three-dimensional geographic coordinate system data.

[0074] S103. Obtain the RGB comparison image and the surface model image corresponding to the original hyperspectral flight strip image file, and correct the hyperspectral flight strip data to be corrected according to the three-dimensional coordinate system data, the RGB comparison image and the surface model image to obtain the corrected hyperspectral flight strip data.

[0075] Specifically, step S103 includes the following sub-steps: S1031. Determine the spatial range of the hyperspectral flight strip data to be corrected in the three-dimensional geographic coordinate system data, and use it as the project area to be processed. S1032. Based on the project area to be processed, crop the RGB comparison image and the surface model image, and output a cropped surface model file with geographic information; S1033. Based on the shooting parameters of the hyperspectral acquisition unit, generate a sensor angle model containing pixel indexes and horizontal and vertical viewing angles. S1034. By traversing the scan lines of the hyperspectral flight strip data to be corrected, and combining the action state with the sensor angle model, a scan line line direction vector matrix is ​​generated that is unified to the three-dimensional geographic coordinate system data. S1035. Using the sensor position as the starting point and the line-of-sight vector as the direction, construct the endpoints of the light segments for each pixel in the hyperspectral flight strip data to be corrected, in conjunction with the upper and lower bounds of the cropped surface model file. S1036. The intersection point between the ray segment and the ground surface of the hyperspectral acquisition unit is obtained by using a ray tracing algorithm to obtain the real-world coordinates of each pixel; S1037. Integrate the world coordinates of all the aforementioned pixels to generate a pixel geographic coordinate file; S1038. Generate a geographic lookup table based on the pixel geographic coordinate file. The geographic lookup table is used to establish the mapping relationship between the grid of the surface model image and the hyperspectral flight strip data to be corrected. S1039. Based on the geographic lookup table, the original hyperspectral flight strip data to be corrected is resampled, and the corrected hyperspectral flight strip data with geographic information of the three-dimensional geographic coordinate system data is output.

[0076] Step S1036 includes the following sub-steps: The coordinates of the two endpoints of the light segments of the pixel are defined as follows: , ,Will , Pixel coordinates mapped to the cropped surface model file ; A discrete successive approximation method is employed, traversing the 2D raster grid cell by cell to progressively approximate the pixels containing the intersection points. Specifically, for the currently traversed pixel, the terrain height *h* of that pixel in the cropped surface model file is substituted into the following expression to calculate the estimated pixel coordinates of the intersection point. : ; in, The height of the ray origin. It is the ray direction vector Quantity, It is a parameter quantity that travels along the ray from the starting point; If the estimated intersection point's pixel coordinates fall within the coordinate range of the previously traversed pixels, the estimated intersection point's coordinates are used as the pixel's real-world coordinates and output.

[0077] S104. Obtain an RGB orthophoto image used as the base image for stitching, perform flight strip mapping, overlapping and stitching processing on different corrected hyperspectral flight strip data based on the RGB orthophoto image, and output a hyperspectral stitched flight strip image file.

[0078] Specifically, step S104 includes the following sub-steps: S1041. Convert the RGB orthophoto image to a coordinate system consistent with the original hyperspectral flight strip image file, and scale it to an adapted grayscale image according to the hyperspectral resolution; step S1041 further includes: performing morphological dilation and cropping processing on the adapted grayscale image according to the geographical information range of the corrected hyperspectral flight strip data. S1042. Extract the RGB pseudo-color bands from the corrected hyperspectral flight strip data and convert them into a hyperspectral grayscale image; S1043. Using the adapted grayscale image as a reference, feature points are extracted and matched from the hyperspectral grayscale image to obtain feature point pairs; S1044. Calculate the global homography matrix based on the feature point pairs; step S1044 further includes: performing medium-grained correction processing on the hyperspectral grayscale image according to the global homography matrix; S1045. The adapted grayscale image is meshed, and the local homography matrix of each mesh is calculated. S1046. Perform steps S1042-S1045 on all the corrected hyperspectral flight strip data that need to be stitched together to obtain the global homography matrix and the local homography matrix of each corrected hyperspectral flight strip data. S1047. Read all the corrected hyperspectral flight strip data that need to be stitched band by band, perform coordinate mapping according to the global homography matrix and the local homography matrix, perform fusion processing on the overlapping areas, integrate all band data, and output the hyperspectral stitched flight strip image file.

[0079] The computer device 300 provided in this embodiment of the invention can implement the steps in the RGB-based hyperspectral image correction and stitching method as described in the above embodiments, and can achieve the same technical effect. Referring to the description in the above embodiments, it will not be repeated here.

[0080] Example 4 This invention also provides a storage medium storing an RGB-based hyperspectral image correction and stitching program. When executed by a processor, the RGB-based hyperspectral image correction and stitching program implements the various processes and steps of the RGB-based hyperspectral image correction and stitching method provided in this invention and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0081] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by hardware related to computer programs or instructions. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0082] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0083] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0084] The embodiments of the present invention have been described above with reference to the accompanying drawings. The disclosed embodiments are merely preferred embodiments of the present invention. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many equivalent changes in form under the guidance of the present invention without departing from the spirit and scope of the claims. All such changes are within the protection scope of the present invention.

Claims

1. A hyperspectral image rectification and stitching method based on RGB, characterized in that, The method comprises the following steps: S101, acquiring an original hyperspectral strip image file and a corresponding original GPS frame file by photographing through a hyperspectral acquisition unit; S102, determining a strip start-stop frame according to the original hyperspectral strip image file and the original GPS frame file, and cutting out to-be-corrected hyperspectral strip data from the original hyperspectral strip image file according to the strip start-stop frame, while converting a part of the original GPS frame file corresponding to the to-be-corrected hyperspectral strip data into three-dimensional coordinate system data; S103, acquiring an RGB contrast image and a ground surface model image corresponding to the original hyperspectral strip image file, correcting the to-be-corrected hyperspectral strip data according to the three-dimensional coordinate system data, the RGB contrast image and the ground surface model image, and obtaining corrected hyperspectral strip data; S104, acquiring an RGB normal photograph image used as a splicing base map, performing strip mapping, overlapping and splicing processing on different corrected hyperspectral strip data based on the RGB normal photograph image, and outputting to obtain a hyperspectral spliced strip image file.

2. The RGB-based hyperspectral image rectification and stitching method according to claim 1, wherein, Step S102 comprises the following sub-steps: S1021, converting longitude and latitude coordinates in the original GPS frame file into local plane coordinates with a first point as an origin; S1022, calculating a heading angle and a unit distance turning rate when the original hyperspectral strip image file is photographed based on the local plane coordinates, and judging an action state of being in a straight flight or turning when the original hyperspectral strip image file is photographed; S1023, determining a start frame and an end frame of each strip in the original hyperspectral strip image file as the strip start-stop frame according to the action state; S1024, cutting out the to-be-corrected hyperspectral strip data from the original hyperspectral strip image file according to the strip start-stop frame; S1025, extracting GPS information corresponding to the to-be-corrected hyperspectral strip data from the original GPS frame file, and converting the GPS information into the three-dimensional geographic coordinate system data.

3. The RGB-based hyperspectral image rectification and stitching method according to claim 2, wherein, Step S103 comprises the following sub-steps: S1031, determining a spatial range of the to-be-corrected hyperspectral strip data in the three-dimensional geographic coordinate system data as a to-be-processed project area; S1032, cutting the RGB contrast image and the ground surface model image according to the to-be-processed project area, and outputting a cut ground surface model file with geographic information; S1033, generating a sensor angle model containing a pixel index and a horizontal and vertical viewing angle based on photographing parameters of the hyperspectral acquisition unit; S1034, generating a scanning line sight direction vector matrix unified to the three-dimensional geographic coordinate system data by traversing scanning lines of the to-be-corrected hyperspectral strip data in combination with the action state and the sensor angle model; S1035, taking a sensor position as a starting point and a sight vector as a direction, and combining upper and lower boundaries of the cut ground surface model file to construct an end point of a light ray segment of each pixel in the to-be-corrected hyperspectral strip data. S1036, obtaining the real world coordinates of each pixel by ray tracing algorithm to find the intersection of the ray segment and the ground surface of the hyperspectral acquisition unit; S1037, integrating the world coordinates of all pixels to generate a pixel geographic coordinate file; S1038, generating a geographic lookup table based on the pixel geographic coordinate file, which is used to establish the mapping relationship between the grid of the ground surface model image and the to-be-corrected hyperspectral strip data; S1039, resampling the original to-be-corrected hyperspectral strip data based on the geographic lookup table, and outputting the corrected hyperspectral strip data with geographic information of the three-dimensional geographic coordinate system data.

4. The RGB-based hyperspectral image rectification and stitching method according to claim 3, wherein, Step S104 includes the following sub-steps: S1041, converting the RGB front camera image to the same coordinate system as the original hyperspectral strip image file, and scaling to an adaptive grayscale image with hyperspectral resolution; S1042, extracting the RGB pseudo-color band from the corrected hyperspectral strip data and converting it to a hyperspectral grayscale image; S1043, taking the adaptive grayscale image as a reference, extracting and matching feature points from the hyperspectral grayscale image to obtain a feature point pair; S1044, calculating a global homography matrix based on the feature point pair; S1045, performing grid processing on the adaptive grayscale image and calculating the local homography matrix of each grid; S1046, performing steps S1042-S1045 on all corrected hyperspectral strip data that need to be spliced to obtain the global homography matrix and the local homography matrix of each corrected hyperspectral strip data; S1047, reading all corrected hyperspectral strip data that need to be spliced wave by wave, and performing coordinate mapping according to the global homography matrix and the local homography matrix, and performing fusion processing on the overlapping area, integrating all band data, and outputting the hyperspectral spliced strip image file.

5. The RGB-based hyperspectral image rectification and stitching method according to claim 2, wherein, Step S1021 further includes: performing symmetric moving average filtering processing on the local plane coordinates to smooth, denoise and eliminate outliers.

6. The RGB-based hyperspectral image rectification and stitching method according to claim 3, wherein, Step S1036 includes the following sub-steps: coordinates of two light ray segment endpoints of the pixel are defined as , , , mapping to pixel coordinates of the cropped terrain model file ; using a discrete step-by-step approximation method, traversing a two-dimensional grid step by step to approach a pixel containing an intersection, wherein, for a currently traversed pixel, the pixel coordinate of an estimated intersection is calculated using the terrain height h of the pixel in the following expression : ； wherein, is the height of the ray origin, is the component of the ray direction vector, is the parameter along the ray from the origin. If the estimated intersection pixel coordinates fall within the coordinate range of the previously traversed pixels, the estimated intersection coordinates are taken as the real world coordinates of the pixel and output.

7. The RGB-based hyperspectral image rectification and stitching method of claim 4, wherein, Step S1041 further includes: performing morphological dilation and cropping processing on the adaptive grayscale image according to the range of geographic information of the corrected hyperspectral strip data; Step S1044 further includes: performing medium-grained correction processing on the hyperspectral grayscale image according to the global homography matrix.

8. An RGB-based hyperspectral image rectification and stitching system, comprising: It includes: A data acquisition module for acquiring an original hyperspectral strip image file and its corresponding original GPS frame file by a hyperspectral acquisition unit; The flight strip identification and cutting module is configured to determine flight strip start and end frames according to the original hyperspectral flight strip image file and the original GPS frame file, and cut out to-be-corrected hyperspectral flight strip data from the original hyperspectral flight strip image file according to the flight strip start and end frames, while converting a part of the original GPS frame file corresponding to the to-be-corrected hyperspectral flight strip data into three-dimensional coordinate system data; The hyperspectral correction module is configured to obtain an RGB reference image and a terrain model image corresponding to the original hyperspectral flight strip image file, correct the to-be-corrected hyperspectral flight strip data according to the three-dimensional coordinate system data, the RGB reference image and the terrain model image, and obtain corrected hyperspectral flight strip data. The hyperspectral stitching module is configured to obtain an RGB orthographic image as a stitching base map, perform flight strip mapping, overlapping and stitching processing on different corrected hyperspectral flight strip data based on the RGB orthographic image, and output a hyperspectral stitched flight strip image file.

9. A computer device, comprising: The memory, the processor and the RGB-based hyperspectral image correction and stitching program stored in the memory and executable on the processor, wherein the processor implements the steps in the RGB-based hyperspectral image correction and stitching method according to any one of claims 1-7 when executing the RGB-based hyperspectral image correction and stitching program. The storage medium stores the RGB-based hyperspectral image correction and stitching program, and the RGB-based hyperspectral image correction and stitching program implements the steps in the RGB-based hyperspectral image correction and stitching method according to any one of claims 1-7 when executed by the processor.

10. A storage medium, characterized by ​

Images