A method and system for processing airborne hyperspectral images

Abstract

The present invention discloses an airborne hyperspectral image processing method, which comprises: step S100: classifying and merging images collected historically by a flight platform based on different shooting states when the flight platform is running; step S200: calculating buffer periods of different shooting states; step S300: when a certain state change of the flight platform is detected in real time, setting a buffer period corresponding to the certain state change for the flight platform; step S400: eliminating hyperspectral images taken from the time point when the state change of the flight platform occurs to the end of the buffer period; collecting spectral data of the eliminated airborne hyperspectral images, and performing coordinate conversion on the spectral data based on POS data to obtain the actual geographic coordinates of the physical object in the image; step S500: resampling the data to output a new image; and in order to better realize the above method flow, an airborne hyperspectral image processing system is also proposed.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to an airborne hyperspectral image processing method and system. Background Art

[0002] The original images of airborne push-broom hyperspectral images are usually affected by factors such as changes in the position and attitude of the flight platform, ground undulations, atmospheric refraction, the rotation and curvature of the earth, etc., which will cause geometric distortion. As a result, the final image is inconsistent with the actual ground objects in terms of coordinate position, relative size, shape, etc. In order to obtain the correct image, the image needs to be corrected for geometric distortion. When the image is geometrically distorted, the scale of the image will often change. When the state of the flight platform changes, the geometric distortion is often the most serious. Compared with the distortion correction of the image generated when the flight state is stable, the geometric distortion correction of the images generated during this period is more complicated in data processing and the image quality obtained will be poorer. Summary of the invention

[0003] The purpose of the present invention is to provide an airborne hyperspectral image processing method to solve the problems raised in the above background technology.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: an airborne hyperspectral image processing method, the processing method comprising:

[0005] Step S100: Classifying and merging the images collected historically by the flight platform based on the different shooting states when the flight platform is operating;

[0006] Step S200: Calculating the buffer periods of different shooting states in the image set obtained after the classification and merging process;

[0007] Step S300: when a certain state change of the flight platform is detected in real time, a buffer period corresponding to the certain state change is set for the flight platform;

[0008] Step S400: removing the hyperspectral images captured by the flight platform from the time when the state changes to the end of the buffer period;

[0009] Step S500: collecting spectral data of the airborne hyperspectral image after elimination, and performing coordinate conversion on the spectral data based on POS data to obtain the actual geographic coordinates of the physical object in the image;

[0010] Step S600: resampling the data obtained in step S500 to output a new image to correct the geometric distortion of the image.

[0011] Furthermore, the classification and merging of images in step S100 includes:

[0012] Step S101: Generate a historical image set from the images collected historically by the flight platform; the processing system monitors the shooting status (H, P) of the flight platform in real time during operation, and captures the time node when the status of the flight platform changes; H represents the flight altitude of the flight platform; P represents the flight attitude of the flight platform when it is at H; the flight attitude includes roll, pitch, and yaw;

[0013] Step S102: for each image in the historical image set, the scale of the image is calculated according to the formula R=U / H, where U represents the focal length, H represents the flight altitude of the flight platform, and R represents the image scale;

[0014] Step S103: in the historical image set, the images before and after the state change are marked for distinction; and the images with the same type of state change are merged to obtain three sub-image sets;

[0015] Image scale refers to the ratio of any line segment on the image to its actual length on the ground. For a central projection image under ideal conditions, that is, the film is horizontal, the ground is horizontal and there is no undulation, the image scale can be expressed by the ratio of focal length to altitude. Due to the undulating terrain and unstable sensor attitude, the scale of each point on the image is different. Therefore, the scale obtained above is an average image scale in a sense. It serves as a data reference for judging the data before and after the flight platform status changes, so as to calculate the recovery period and buffer period later.

[0016] Furthermore, the state change includes three state changes, among which the first state change refers to that the flight altitude H of the flight platform remains unchanged, and the flight attitude P of the flight platform changes; the second state change refers to that the flight attitude P of the flight platform remains unchanged, and the flight altitude H of the flight platform changes; the third state change refers to that the flight attitude P of the flight platform changes, and at the same time, the flight altitude H of the flight platform also changes.

[0017] Furthermore, the calculation of the buffer period for different shooting states in step S200 includes:

[0018] Step S201: the image scale of each image in each sub-image set is obtained, and the time point when the state change occurs is calibrated; the image scale that appears the most times in the image scale number set Y after the state change occurs is used as the reference scale value, and the time point when the first reference scale value appears is obtained, and the recovery period T of the state change is calculated according to the formula T=t'-t; wherein t' represents the time point when the first reference scale value appears, and t represents the time point when the state change occurs;

[0019] Step S202: According to the formula: The buffer period L for this state change to occur is calculated, where n represents the total number of state changes that occur in the sub-image set, T i Indicates the recovery period for the i-th occurrence of this state change;

[0020] When the flying platform is in operation, it usually maintains a stable altitude and position for shooting. During this period, it may be affected by the impact of airflow and wind speed resistance, causing the flying platform to change its state. The probability of geometric distortion of the image when the flying platform changes its state is the highest in actual experience, and the process of correcting the output image is also complicated. Based on this, the shooting state of the flying platform in the historical data is monitored and acquired, and the time point of the state change is captured. This is for the subsequent steps to calculate from the historical data the buffer time required for the flying platform to complete the state change to another stable state under different change states. This buffer time is a reference value for screening the image data before the final geometric distortion correction is performed on the image.

[0021] Furthermore, step S500 includes:

[0022] Step S501: When collecting and recording spectral data, each spectral data is stamped with a UTC timestamp, and the spectral data and the POS data are matched one by one based on the UTC timestamp to achieve time synchronization between the spectral data and the POS data;

[0023] Step S502: in the spectral data after time synchronization, the pixel data of each data is converted from the object point coordinates to the camera center coordinates A in the POS data; the camera center coordinates A are converted to the camera body coordinates B; the camera body coordinates B are converted to the navigation coordinates C; the navigation coordinates C are converted to the map coordinates D;

[0024] Step S503: increasing the map coordinate value INS of the camera body;

[0025] Step S504: Obtain each actual geographic coordinate M of the airborne hyperspectral image according to the formula: M = INS + D*C*B*A. The actual geographic coordinate M is in the form of (X, Y), where X∈(X min , X max ), Y∈(Y min , Y max );X min Indicates the minimum X-axis value among all the actual geographic coordinates obtained. max Indicates the maximum X-axis value among all the actual geographic coordinates obtained; Y min Indicates the minimum Y-axis value among all the actual geographic coordinates obtained; Ymax Indicates the maximum Y-axis value among all actual geographic coordinates obtained; the X-axis range and Y-axis range values ​​appear after spatial synchronization.

[0026] Furthermore, step S600 includes:

[0027] Step S601: Calculate the width W of the new image according to the formula: Among them, pix represents the resolution of the airborne hyperspectral image;

[0028] Step S602: Calculate the height H of the new image according to the formula: pix represents the resolution of the airborne hyperspectral image;

[0029] Step S603: Calculate the corresponding pixel value (T X , T Y ),in, int() means rounding the number in the brackets;

[0030] Step S604: Find the RGB value of each actual geographic coordinate point and assign it to the corresponding pixel (T X , T Y )superior.

[0031] In order to better implement the above method, an airborne hyperspectral image processing system is also proposed, the processing system includes: a database, a data processing module, a buffer period calculation module, a data acquisition module, and a data correction module;

[0032] The database is used to collect the historical image data of the flight platform; and transmit the image data to the data processing module and the data calculation module for data processing and calculation;

[0033] The data processing module is used to receive the image data in the database and capture the state change of the flight platform; it is used to calculate the image scale according to the image data, and mark and classify and merge the image based on the image scale;

[0034] A buffer period calculation module, used for calculating the recovery period and buffer period of the image data processed by the data processing module;

[0035] A data acquisition module is used to collect data collected by the flight platform according to the data calculated in the buffer period calculation module, and then send the collected data to the data correction module for data correction;

[0036] The data correction module is used to receive the data collected by the data acquisition module to correct the image geometric distortion.

[0037] Further, the data processing module includes a scale calculation unit, a state change capture unit, an image marking unit, and an image set merging unit;

[0038] A scale calculation unit, used for receiving the historical image data of the flight platform in the database, and performing image scale calculation on each image in the historical image set;

[0039] A state change capture unit is used to capture and record state changes of the flight platform;

[0040] An image marking unit, used for receiving data in the scale calculation unit and the state change capture unit, and completing a distinguishing mark on the data transmitted in the scale calculation unit based on the capture record in the state change capture unit;

[0041] The image set merging unit is used to classify and merge the images processed by the image marking unit.

[0042] Further, the buffer period calculation module includes a recovery period calculation unit and a buffer period calculation unit; the data correction module includes a geographic coordinate conversion unit and a resampling unit;

[0043] A recovery period calculation unit, used to receive the image data processed by the data processing module to calculate the recovery period; a buffer period calculation unit, used to calculate the buffer period according to the data in the recovery period calculation unit;

[0044] A coordinate conversion unit, used for performing coordinate conversion on the spectral data collected in the data collection module based on the POS data to obtain the actual geographic coordinates of the object in the image;

[0045] The resampling unit is used to resample the data processed by the coordinate conversion unit to generate a new image.

[0046] Compared with the prior art, the beneficial effect achieved by the present invention is that before the geometric distortion correction of the image is performed, the present invention performs a calculation and analysis process of the state buffer period based on the state change characteristics of the flight platform, and screens the images for subsequent image geometric distortion correction, so that the final image after geometric distortion correction is as consistent as possible with the coordinate position, relative size, shape and other aspects of each object on the actual ground, and the presented image quality is as high as possible. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0048] Figure 1It is a flow chart of an airborne hyperspectral image processing method of the present invention;

[0049] Figure 2 It is a structural schematic diagram of an airborne hyperspectral image processing system of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] See also Figure 1-Figure 2 The present invention provides a technical solution: an airborne hyperspectral image processing method, the processing method comprising:

[0052] Step S100: Classifying and merging the images collected historically by the flight platform based on the different shooting states when the flight platform is operating;

[0053] The image classification and merging process includes:

[0054] Step S101: Generate a historical image set from the images collected historically by the flight platform; the processing system monitors the shooting status (H, P) of the flight platform in real time during operation, and captures the time node when the status of the flight platform changes; H represents the flight altitude of the flight platform; P represents the flight attitude of the flight platform when it is at H; the flight attitude includes roll, pitch, and yaw;

[0055] Step S102: for each image in the historical image set, the scale of the image is calculated according to the formula R=U / H, where U represents the focal length, H represents the flight altitude of the flight platform, and R represents the image scale;

[0056] Step S103: in the historical image set, the images before and after the state change are marked for distinction; and the images with the same type of state change are merged to obtain three sub-image sets;

[0057] Among them, the state change includes three kinds of state changes, among which the first kind of state change means that the flight altitude H of the flight platform remains unchanged, and the flight attitude P of the flight platform changes; the second kind of state change means that the flight attitude P of the flight platform remains unchanged, and the flight altitude H of the flight platform changes; the third kind of state change means that the flight attitude P of the flight platform changes, and at the same time, the flight altitude H of the flight platform also changes;

[0058] Step S200: Calculating the buffer periods of different shooting states in the image set obtained after the classification and merging process;

[0059] The calculation of the buffer period for different shooting states includes:

[0060] Step S201: the image scale of each image in each sub-image set is obtained, and the time point when the state change occurs is calibrated; the image scale that appears the most times in the image scale number set Y after the state change occurs is used as the reference scale value, and the time point when the first reference scale value appears is obtained, and the recovery period T of the state change is calculated according to the formula T=t'-t; wherein t' represents the time point when the first reference scale value appears, and t represents the time point when the state change occurs;

[0061] Step S202: According to the formula: The buffer period L for this state change to occur is calculated, where n represents the total number of state changes that occur in the sub-image set, T i Indicates the recovery period for the i-th occurrence of this state change;

[0062] Step S300: when a certain state change of the flight platform is detected in real time, a buffer period corresponding to the certain state change is set for the flight platform;

[0063] Step S400: removing the hyperspectral images captured from the time point when the flight platform changes its state to the end of the buffer period; for example, if the time point when the flight platform changes its state is Z and the buffer period is L, the hyperspectral images captured during the period Z+L are removed;

[0064] Step S500: collecting spectral data of the airborne hyperspectral image after elimination, and performing coordinate conversion on the spectral data based on POS data to obtain the actual geographic coordinates of the physical object in the image;

[0065] Wherein, step S500 includes:

[0066] Step S501: When collecting and recording spectral data, each spectral data is stamped with a UTC timestamp, and the spectral data and the POS data are matched one by one based on the UTC timestamp to achieve time synchronization between the spectral data and the POS data;

[0067] Step S502: in the spectral data after time synchronization, the pixel data of each data is converted from the object point coordinates to the camera center coordinates A in the POS data; the camera center coordinates A are converted to the camera body coordinates B; the camera body coordinates B are converted to the navigation coordinates C; the navigation coordinates C are converted to the map coordinates D; wherein,

[0068]

[0069] H is the altitude, h0 is the ground altitude, fov is the camera field of view, and k is the kth sample in each frame;

[0070]

[0071] Step S503: increasing the map coordinate value INS of the camera body;

[0072] Step S504: Obtain each actual geographic coordinate M of the airborne hyperspectral image according to the formula: M = INS + D*C*B*A. The actual geographic coordinate M is in the form of (X, Y), where X∈(X min , X max ), Y∈(Y min , Y max );X min Indicates the minimum X-axis value among all the actual geographic coordinates obtained. max Indicates the maximum X-axis value among all the actual geographic coordinates obtained; Y min Indicates the minimum Y-axis value among all the actual geographic coordinates obtained; Y max Indicates the maximum Y-axis value among all the actual geographic coordinates obtained;

[0073] Step S600: resampling the data obtained in step S400 to output a new image to correct the geometric distortion of the image.

[0074] Wherein, step S600 includes:

[0075] Step S601: Calculate the width W of the new image according to the formula: Among them, pix represents the resolution of the airborne hyperspectral image;

[0076] Step S602: Calculate the height H of the new image according to the formula: pix represents the resolution of the airborne hyperspectral image;

[0077] Step S603: Calculate the corresponding pixel value (T X , T Y ),in,

[0078] Step S604: Find the RGB value of each actual geographic coordinate point and assign it to the corresponding pixel (T X , T Y )superior.

[0079] In order to better implement the above method, an airborne hyperspectral image processing system is also proposed, the processing system includes: a database, a data processing module, a buffer period calculation module, a data acquisition module, and a data correction module;

[0080] The database is used to collect the historical image data of the flight platform; and transmit the image data to the data processing module and the data calculation module for data processing and calculation;

[0081] The data processing module is used to receive the image data in the database and capture the state change of the flight platform; it is used to calculate the image scale according to the image data, and mark and classify and merge the image based on the image scale;

[0082] Wherein, the data processing module includes a scale calculation unit, a state change capture unit, an image marking unit, and an image set merging unit;

[0083] A scale calculation unit is used to receive the historical image data of the flight platform in the database, and perform image scale calculation on each image in the historical image set; a state change capture unit is used to capture and record the state change of the flight platform; an image marking unit is used to receive the data in the scale calculation unit and the state change capture unit, and complete the distinguishing mark of the data transmitted in the scale calculation unit based on the capture record in the state change capture unit; an image set merging unit is used to classify and merge the images processed by the image marking unit;

[0084] A buffer period calculation module, used for calculating the recovery period and buffer period of the image data processed by the data processing module;

[0085] A data acquisition module is used to collect data collected by the flight platform according to the data calculated in the buffer period calculation module, and then send the collected data to the data correction module for data correction;

[0086] The data correction module is used to receive the data collected by the data acquisition module to correct the image geometric distortion.

[0087] The buffer period calculation module includes a recovery period calculation unit and a buffer period calculation unit; the data correction module includes a geographic coordinate conversion unit and a resampling unit;

[0088] A recovery period calculation unit, used to receive the image data processed by the data processing module to calculate the recovery period; a buffer period calculation unit, used to calculate the buffer period according to the data in the recovery period calculation unit;

[0089] A coordinate conversion unit, used for performing coordinate conversion on the spectral data collected in the data collection module based on the POS data to obtain the actual geographic coordinates of the object in the image;

[0090] The resampling unit is used to resample the data processed by the coordinate conversion unit to generate a new image.

[0091] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0092] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An airborne hyperspectral image processing method, characterized in that: The processing method comprises: Step S100: Classifying and merging the images collected historically by the flight platform based on the different shooting states when the flight platform is operating; Step S200: Calculating the buffer periods of different shooting states in the image set obtained after the classification and merging process; Step S300: when a certain state change of the flight platform is detected in real time, a buffer period corresponding to the certain state change is set for the flight platform; Step S400: removing the hyperspectral images captured by the flight platform from the time when the state changes to the end of the buffer period; Step S500: collecting spectral data of the airborne hyperspectral image after elimination, and performing coordinate conversion on the spectral data based on POS data to obtain the actual geographic coordinates of the physical object in the image; Step S600: resampling the data obtained in step S500 to output a new image to correct the geometric distortion of the image.

2. The method for processing airborne hyperspectral images according to claim 1, characterized in that: The step S500 includes: Step S501: when collecting and recording spectral data, each spectral data is stamped with a UTC timestamp, and the spectral data is matched with the POS data one by one based on the UTC timestamp, so as to achieve time synchronization between the spectral data and the POS data; Step S502: in the spectral data after time synchronization, pixel data of each data is converted from object point coordinates into camera center coordinates A in POS data; the camera center coordinates A are converted into camera body coordinates B; the camera body coordinates B are converted into navigation coordinates C; the navigation coordinates C are converted into map coordinates D; Step S503: increasing the map coordinate value INS of the camera body; Step S504: Obtain each actual geographic coordinate M of the airborne hyperspectral image according to the formula: M = INS + D*C*B*A, and the actual geographic coordinate M is in the form of (X, Y), where X∈(X min , X max ), Y∈(Y min , Y max );X min Indicates the minimum X-axis value among all the actual geographic coordinates obtained. max Indicates the maximum X-axis value among all the actual geographic coordinates obtained; Y min Indicates the minimum Y-axis value among all the actual geographic coordinates obtained; Y max Indicates the maximum Y-axis value among all actual geographic coordinates obtained.

3. The method for processing airborne hyperspectral images according to claim 2, characterized in that: The step S600 includes: Step S601: Calculate the width W of the new image according to the formula: Among them, pix represents the resolution of the airborne hyperspectral image; Step S602: Calculate the height H of the new image according to the formula: pix represents the resolution of the airborne hyperspectral image; Step S603: Calculate the corresponding pixel value (T X , T Y ),in, Step S604: Find the RGB value of each actual geographic coordinate point and assign it to the corresponding pixel (T X , T Y )superior.

4. The method for processing airborne hyperspectral images according to claim 1, characterized in that: The image classification and merging process in step S100 includes: Step S101: Generate a historical image set from the images collected historically by the flight platform; the processing system monitors the shooting status (H, P) of the flight platform in real time during operation, and captures the time node when the status of the flight platform changes; wherein H represents the flight altitude of the flight platform; P represents the flight attitude of the flight platform when it is at H; the flight attitude includes roll, pitch, and yaw; Step S102: for each image in the historical image set, the scale of the image is calculated according to the formula R=U / H, where U represents the focal length, H represents the flight altitude of the flight platform, and R represents the image scale; Step S103: in the historical image set, the images before and after the state change are distinguished and marked; and the images with the same type of state change are merged to obtain three sub-image sets.

5. The method for processing airborne hyperspectral images according to claim 4, characterized in that: State changes include three types of state changes. The first state change refers to that the flight altitude H of the flight platform remains unchanged, while the flight attitude P of the flight platform changes; the second state change refers to that the flight attitude P of the flight platform remains unchanged, while the flight altitude H of the flight platform changes; the third state change refers to that the flight attitude P of the flight platform changes, and at the same time, the flight altitude H of the flight platform also changes.

6. The method for processing airborne hyperspectral images according to claim 4, characterized in that: The calculation of the buffer period for different shooting states in step S200 includes: Step S201: acquiring the image scale of each image in the sub-image set, and calibrating the time point when the state change occurs; taking the image scale that appears most times in the image scale number set Y after the state change occurs as the reference scale value, and acquiring the time point when the first reference scale value appears, and calculating the recovery period T of the state change according to the formula T=t'-t; wherein t' represents the time point when the first reference scale value appears, and t represents the time point when the state change occurs; Step S202: According to the formula: The buffer period L for this state change to occur is calculated, where n represents the total number of state changes that occur in the sub-image set, T i Indicates the recovery period for the i-th occurrence of this state change.

7. An airborne hyperspectral image processing system applied to the method according to any one of claims 1 to 6, characterized in that: The processing system includes: a database, a data processing module, a buffer period calculation module, a data acquisition module, and a data correction module; The database is used to collect historical image data taken by the flight platform; and transmit the image data to the data processing module and the buffer period calculation module for data processing and calculation; The data processing module is used to receive the image data in the database and capture the state change of the flight platform; to calculate the image scale of the image according to the image data, and to mark and classify and merge the image based on the image scale; The buffer period calculation module is used to calculate the recovery period and the buffer period of the image data processed by the data processing module; The data acquisition module is used to collect data collected by the flight platform according to the data calculated in the buffer period calculation module, and then send the collected data to the data correction module for data correction; The data correction module is used to receive the data collected by the data acquisition module to correct the image geometric distortion.

8. The airborne hyperspectral image processing system according to claim 7, characterized in that: The data processing module includes a scale calculation unit, a state change capture unit, an image marking unit, and an image set merging unit; The scale calculation unit is used to receive the historical image data of the flight platform in the database and perform image scale calculation on each image in the historical image set; The state change capturing unit is used to capture and record the state change of the flight platform; The image marking unit is used to receive the data in the scale calculation unit and the state change capture unit, and to complete a distinguishing mark on the data transmitted in the scale calculation unit based on the capture record in the state change capture unit; The image set merging unit is used to classify and merge the images processed by the image marking unit.

9. The airborne hyperspectral image processing system according to claim 7, characterized in that: The buffer period calculation module includes a recovery period calculation unit and a buffer period calculation unit; the data correction module includes a geographic coordinate conversion unit and a resampling unit; The recovery period calculation unit is used to receive the image data processed by the data processing module to calculate the recovery period; the buffer period calculation unit is used to calculate the buffer period according to the data in the recovery period calculation unit; The coordinate conversion unit is used to perform coordinate conversion on the spectral data collected in the data acquisition module based on the POS data to obtain the actual geographic coordinates of the physical object in the image; The resampling unit is used to resample the data processed by the coordinate conversion unit to generate a new image.

Images