Airport runway ice, snow and water detection method and system based on hyperspectral camera

Abstract

The invention relates to the technical field of airport runway safety monitoring, in particular to an airport runway ice, snow and water detection method and system based on a hyperspectral camera. The method comprises the following steps: performing spectrum acquisition by using a hyperspectral camera; performing parameter setting on the hyperspectral camera; establishing a geometric model of rotary scanning, mapping the rectangular image to a sector, solving the distortion problem in the image through a geometric correction algorithm, and restoring the rectangular image to the sector; performing feature analysis and detection on ice, snow and water on the runway; calculating the area proportion of ice, snow and water by adopting an area statistical method; outputting types and area proportions of ice and snow water; the system comprises a hyperspectral camera, a rotary table, an easy-to-bend support and an edge calculation module, through the mode, non-contact detection, classification and area measurement can be carried out on ice, snow and water of a runway, and the detection effect is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of airport runway safety monitoring technology, and in particular to a method and system for detecting ice and snow water on airport runways based on a hyperspectral camera. Background Technology

[0002] The safe operation of airport runways is crucial for air transport. During cold seasons or in severe weather conditions, airport runway surfaces are prone to snow accumulation, ice formation, or water accumulation. Runway ice and snow water pollution can affect the runway friction coefficient, thereby impacting aircraft landings. Currently, the commonly used method for detecting ice and snow water on airport runways is infrared imaging.

[0003] However, existing infrared detection methods struggle to classify and measure the area of ​​ice, snow, and water that is about to freeze on the runway when the temperature differences between them are small, resulting in poor detection performance. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for detecting ice and snow water on airport runways based on a hyperspectral camera. This aims to solve the technical problem that existing infrared detection methods are difficult to classify and measure the area of ​​ice and snow water when the temperature difference between ice blocks, unfrozen snow, and water on the runway that is about to freeze is small, resulting in poor detection performance.

[0005] To achieve the above objectives, the present invention employs a method for detecting ice and snow water on airport runways based on a hyperspectral camera, comprising the following steps: Spectral data acquisition was performed using a hyperspectral camera. Parameter setting for the hyperspectral camera; A geometric model for rotational scanning is established, the rectangular image is mapped to a fan, and the distortion problem in the image is solved by a geometric correction algorithm, thus restoring the rectangular image to a fan. Characteristic analysis and detection of ice and snow water on the runway; The area ratio of ice and snow water was calculated using area statistics methods; The types and area ratios of ice and snow water output.

[0006] In the step of acquiring spectra using a hyperspectral camera: After light passes through the lens and enters the hyperspectral camera, the incident light is dispersed into different wavelengths by the grating and prism; The dispersed spectrum is projected onto a linear array detector, which scans the runway surface line by line, with each line corresponding to the hyperspectral data of a spatial point.

[0007] In the step where, after light enters the hyperspectral camera through the lens, the incident light is dispersed into different wavelengths by a grating and a prism: The wavelength range is 400~1700nm.

[0008] In the step of setting parameters for the hyperspectral camera: Define the camera's installation parameters, including the vertical height from the rotation center to the ground, the camera's pitch angle, and the minimum observation distance corresponding to the center pixel of the scan line; Define the camera's optical parameters, including the camera's vertical field of view and the number of pixels per scan line; Define the camera motion parameters, including the initial rotation angle, rotation step angle, and total number of scan lines; Define camera image metadata, including the width and height of the rectangular image.

[0009] In the steps of establishing a geometric model for rotational scanning, mapping a rectangular image to a sector, and using a geometric correction algorithm to address distortion issues in the image and restore the rectangular image to a sector: Image preprocessing involves calculating key parameters, including the maximum observation distance and the vertical viewing angle offset of each pixel along the scan line. Iterate through each Cartesian coordinate point in the output sector image and calculate its position in the rectangular image in reverse order. Use bilinear interpolation to obtain pixel values ​​for non-integer coordinates.

[0010] Among the steps involved in the characteristic analysis and detection of ice and snow water on the runway: Ice and snow have unique spectral reflection and absorption characteristics in specific wavelength bands, such as near-infrared and short-wave infrared. Snow has a strong reflection peak in the visible light band, while water has an absorption peak in the near-infrared band. By utilizing these wavelength characteristics, ice, snow and water can be directly identified and presented in the corrected fan-shaped image to achieve image segmentation.

[0011] In the step of calculating the area ratio of snowmelt using area statistics methods: Count the number of pixels in the target region in the segmented binary image and multiply it by the actual area of ​​a single pixel.

[0012] The present invention also provides an airport runway ice and snow water detection system based on a hyperspectral camera, including a hyperspectral camera, a turntable, a bendable bracket and an edge computing module; The hyperspectral camera is used to acquire hyperspectral images; The turntable is used to rotate back and forth within an angular sector area to form a sector-shaped imaging area for the runway. The flexible bracket is used to mount the turntable; The edge computing module is used to stitch together hyperspectral images and detect ice and snow water and its area.

[0013] This invention discloses a method and system for detecting ice and snow water on airport runways based on a hyperspectral camera. Utilizing push-broom imaging technology from a line-scan hyperspectral camera, combined with hyperspectral analysis and environmentally adaptable design, it is used for detecting ice and snow water on runways in high-altitude airports. First, the hyperspectral camera is used for spectral acquisition, and its parameters are determined. Then, a geometric model for rotational scanning is established, mapping a rectangular image to a fan-shaped area. A geometric correction algorithm is used to address distortion issues in the image, restoring the rectangular image to a fan-shaped area. The rectangular image is obtained by stitching together line data generated by a line-scan camera. Next, feature analysis and detection of ice and snow water on the runway are performed, and an area statistics method is used to calculate the area ratio of ice and snow water. Finally, the type and area ratio of ice and snow water are output. The system consists of the hyperspectral camera, the turntable, the flexible support, and an edge computing module. The hyperspectral camera employs dispersion... The camera is a 1920×1 line imager, capable of capturing N frames of 1920×1 images simultaneously in a single imaging session, where N represents the number of spectral bands. A turntable is used to mount the hyperspectral camera and is required to output the turntable's rotation angle. During operation, the turntable rotates back and forth within a certain angled fan-shaped area, forming a fan-shaped imaging area of ​​the runway as it rotates. A flexible bracket is used to mount the aforementioned equipment. Since it needs to be installed on both sides of the runway, the design complies with the relevant regulations of the Civil Aviation Administration of China. The edge computing module is directly connected to the hyperspectral camera and the turntable to stitch together the hyperspectral images and detect ice and snow water and its area. Through the above method, non-contact detection, classification, and area measurement of runway ice and snow water are achieved, greatly improving the detection effect. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of the steps of the airport runway ice and snow water detection method based on a hyperspectral camera according to the present invention.

[0016] Figure 2 This is a flowchart of steps S100 of the present invention.

[0017] Figure 3 This is a flowchart of steps S200 of the present invention.

[0018] Figure 4 This is a flowchart of steps S300 of the present invention.

[0019] Figure 5This is a schematic diagram of the structural principle of the airport runway ice and snow water detection system based on a hyperspectral camera according to the present invention.

[0020] 701-Hyperspectral camera, 702-Turntable, 703-Bendable bracket, 704-Edge computing module. Detailed Implementation

[0021] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0022] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0023] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0024] See also Figures 1-4 This invention provides a method for detecting ice and snow water on airport runways based on a hyperspectral camera 701, comprising the following steps: S100: Spectral acquisition is performed using a 701 hyperspectral camera.

[0025] In this embodiment, a hyperspectral camera 701 is used for spectral acquisition. The specific process is as follows: S101: After light passes through the lens and enters the hyperspectral camera 701, the incident light is dispersed into different wavelengths by the grating and prism; S102: The dispersed spectrum is projected onto the linear array detector, scanning the runway surface line by line, with each line corresponding to the hyperspectral data of a spatial point.

[0026] In the above process, after the light enters the hyperspectral camera 701 through the lens, the incident light is dispersed into different wavelengths (wavelength range of 400~1700nm) by the grating and prism. The dispersed spectrum is projected onto the linear array detector (such as CMOS or CCD) and scans the runway surface line by line. Each line corresponds to the hyperspectral data of a spatial point.

[0027] S200: Parameter setting for the hyperspectral camera 701.

[0028] In this embodiment, the parameters of the hyperspectral camera 701 are determined, and the specific process is as follows: S201: Define the camera's installation parameters, including the vertical height from the rotation center to the ground, the camera's pitch angle, and the minimum observation distance corresponding to the center pixel of the scan line; S202: Define the camera's optical parameters, including the camera's vertical field of view and the number of scan line pixels; S203: Define camera motion parameters, including initial rotation angle, rotation step angle, and total number of scan lines; S204: Define camera image metadata, including the width and height of the rectangular image.

[0029] In the above process, the camera installation parameters are determined, including the vertical height (H) from the center of rotation to the ground, and the camera pitch angle (H). (The angle between the optical axis and the horizontal plane) and the minimum observation distance corresponding to the center pixel of the scan line. ) [or calculate ( [ ]; Define the camera's optical parameters, including the camera's vertical field of view ( ) and scan line pixel count (M) (the number of pixels per line); define camera motion parameters, including the initial rotation angle ( (e.g., due north is 0°), the rotation angle is θ, and the rotation step angle is ( (Angle difference between adjacent scan lines) and total number of scan lines (N) (height of the rectangular image); define camera image metadata, including the width (W=M) (pixels) and height of the rectangular image. (pixels).

[0030] S300: Establish a geometric model for rotational scanning, map the rectangular image onto a fan, and use a geometric correction algorithm to solve the distortion problem in the image, restoring the rectangular image to a fan. The rectangular image is obtained by stitching together line data generated by the line scan camera.

[0031] In this embodiment, a geometric model of rotational scanning is established, the rectangular image is mapped to a sector, and a geometric correction algorithm is used to solve the distortion problem in the image, restoring the rectangular image to a sector. The specific process is as follows: S301: Image preprocessing, calculating key parameters, including maximum observation distance and vertical viewing angle offset of each pixel in the scan line; S302: Traverse each Cartesian coordinate point of the output sector image and calculate its position in the rectangular image in reverse; S303: Use bilinear interpolation to obtain pixel values ​​for non-integer coordinates.

[0032] In the above process, image preprocessing is performed, and key parameters are calculated, including the maximum observation distance and the vertical viewing angle offset of each pixel along the scan line. Among them, the maximum observation distance is: , Vertical view offset of each pixel on the scan line: ; Iterate through each Cartesian coordinate point ((x, y)) in the output sector image, and calculate its position (i, j) in the rectangular image. Calculate row i: by inputting the Cartesian coordinates (x, y), obtain the mirror distance and rotation angle. r=sqrt(x 2 +y 2 ) θ = atan2(y, x); Additionally, row i corresponds to angle θ. Since the rows of the rectangular image are arranged in rotational order, we need to know the starting angle θ0 and the angle step Δθ. Assuming the 0th row corresponds to angle θ0, then the angle corresponding to the i-th row is θ_i = θ0 + i * Δθ. Therefore, for a given θ, calculate the row coordinates (scan line index) of the rectangular image. i=(θ-θ0) / Δθ For the j-th pixel on the scan line, its angle with the optical axis is α_j (within the vertical plane containing the optical axis). Note: This angle α_j is in the vertical plane of the camera coordinate system (i.e., along the optical axis), assuming β=0 (i.e., the direction of the optical axis) for the center pixel of the scan line. First, calculate the angle between the ground point and the optical axis: Then calculate the vertical view offset. , Calculate column j on the scan line based on β. Since the center of the scan line (j=M / 2) corresponds to β=0, the top of the scan line (j=0) corresponds to β=FOV / 2, and the bottom of the scan line (j=M-1) corresponds to β=-FOV / 2, therefore: Use bilinear interpolation to obtain pixel values ​​for non-integer coordinates (i, j); in( ) is the input rectangular image.

[0033] S400: Performs feature analysis and detection on ice and snow water on the runway.

[0034] In this embodiment, ice and snow have unique spectral reflection and absorption characteristics in specific bands (such as near-infrared and short-wave infrared). The strong reflection peak of snow is in the visible light band, while the absorption peak of water is in the near-infrared band. By utilizing the band characteristics, ice, snow and water can be directly identified and presented in the fan-shaped image after image correction, thereby achieving the image segmentation effect.

[0035] S500: The area ratio of ice and snow water is calculated using area statistics methods.

[0036] In this embodiment, the number of pixels in the target region in the segmented binary image is counted, and then multiplied by the actual area of ​​a single pixel (calculated based on camera resolution and shooting height). The formula is as follows: Where p is the number of target pixels and GSD is the ground resolution (unit: meters / pixel).

[0037] S600: Outputs the type and area ratio of ice and snow water.

[0038] See also Figure 5 The present invention also provides an airport runway ice and snow water detection system based on a hyperspectral camera 701, including a hyperspectral camera 701, a turntable 702, a bendable bracket 703 and an edge computing module 704. The hyperspectral camera 701 is used to acquire hyperspectral images; The turntable 702 is used to rotate back and forth within an angular sector area to form a sector-shaped imaging area for the runway. The bendable bracket 703 is used to mount the turntable 702; The edge computing module 704 is used to stitch together hyperspectral images and detect ice and snow water and area.

[0039] In this embodiment, the hyperspectral camera 701 is used to acquire hyperspectral images, the turntable 702 rotates back and forth in the angular fan-shaped area to form a fan-shaped imaging area of ​​the runway, the bendable bracket 703 is used to install the turntable 702, and the edge computing module 704 stitches the hyperspectral images and detects the ice and snow water and area. In this system, the hyperspectral camera 701 is installed on both sides of the runway during operation. The hyperspectral camera 701 outputs N 1920×1 images. The hyperspectral camera 701 is installed with a 90° rotation, meaning the 1920 pixel orientation is perpendicular to the runway. The system is then powered on. After power-on, the hyperspectral camera 701 is activated. The turntable 702 checks if it is in its initial position. If the turntable 702 is not in its initial position, it rotates to it. During this process, if the hyperspectral camera 701 has already output image information, the edge control module discards that portion of the image without processing it. If the turntable 702 reaches its initial position and the hyperspectral camera... Once the hyperspectral camera 701 is outputting image information, the edge computing module 704 begins receiving the angle and image information from the turntable 702 and starts detection. The hyperspectral camera 701 continues to rotate with the turntable 702, and the turntable 702 outputs rotation angle information. The hyperspectral camera 701 outputs image information, and the edge computing module 704 continues to receive the above information. Then, the edge computing module 704 uses the deployed ice and snow water detection method to stitch the multispectral images, and processes the stitched images of different bands separately to detect ice and snow water, and further calculates the area of ​​ice and snow water. Finally, the detected types and area ratios of ice and snow water are transmitted through the communication interface.

[0040] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0041] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for detecting ice and snow water on airport runways based on a hyperspectral camera, characterized in that, Includes the following steps: Spectral data acquisition was performed using a hyperspectral camera. Parameter setting for the hyperspectral camera; A geometric model for rotational scanning is established, the rectangular image is mapped to a fan, and the distortion problem in the image is solved by a geometric correction algorithm, thus restoring the rectangular image to a fan. Characteristic analysis and detection of ice and snow water on the runway; The area ratio of ice and snow water was calculated using area statistics methods; The types and area ratios of ice and snow water output.

2. The method for detecting ice and snow water on airport runways based on a hyperspectral camera as described in claim 1, characterized in that, In the steps of spectral acquisition using a hyperspectral camera: After light passes through the lens and enters the hyperspectral camera, the incident light is dispersed into different wavelengths by the grating and prism; The dispersed spectrum is projected onto a linear array detector, which scans the runway surface line by line, with each line corresponding to the hyperspectral data of a spatial point.

3. The method for detecting ice and snow water on airport runways based on a hyperspectral camera as described in claim 2, characterized in that, In the process of light entering the hyperspectral camera through the lens, the incident light is dispersed into different wavelengths by the grating and prism: The wavelength range is 400~1700nm.

4. The method for detecting ice and snow water on airport runways based on a hyperspectral camera as described in claim 1, characterized in that, In the process of setting parameters for a hyperspectral camera: Define the camera's installation parameters, including the vertical height from the rotation center to the ground, the camera's pitch angle, and the minimum observation distance corresponding to the center pixel of the scan line; Define the camera's optical parameters, including the camera's vertical field of view and the number of pixels per scan line; Define the camera motion parameters, including the initial rotation angle, rotation step angle, and total number of scan lines; Define camera image metadata, including the width and height of the rectangular image.

5. The method for detecting ice and snow water on airport runways based on a hyperspectral camera as described in claim 3, characterized in that, In the steps of establishing a geometric model for rotational scanning, mapping a rectangular image to a sector, and using a geometric correction algorithm to address distortion issues in the image and restore the rectangular image to a sector: Image preprocessing involves calculating key parameters, including the maximum observation distance and the vertical viewing angle offset of each pixel along the scan line. Iterate through each Cartesian coordinate point in the output sector image and calculate its position in the rectangular image in reverse order. Use bilinear interpolation to obtain pixel values ​​for non-integer coordinates.

6. The method for detecting ice and snow water on airport runways based on a hyperspectral camera as described in claim 5, characterized in that, In the steps of characterizing and detecting ice and snow water on the runway: Ice and snow have unique spectral reflection and absorption characteristics in specific wavelength bands, such as near-infrared and short-wave infrared. Snow has a strong reflection peak in the visible light band, while water has an absorption peak in the near-infrared band. By utilizing these wavelength characteristics, ice, snow and water can be directly identified and presented in the corrected fan-shaped image to achieve image segmentation.

7. The method for detecting ice and snow water on airport runways based on a hyperspectral camera as described in claim 6, characterized in that, In the step of calculating the area ratio of snowmelt using area statistics methods: Count the number of pixels in the target region in the segmented binary image and multiply it by the actual area of ​​a single pixel.

8. A hyperspectral camera-based airport runway ice and snow water detection system, applied to the hyperspectral camera-based airport runway ice and snow water detection method as described in claim 7, characterized in that, Includes a hyperspectral camera, turntable, flexible support, and edge computing module; The hyperspectral camera is used to acquire hyperspectral images; The turntable is used to rotate back and forth within an angular sector area to form a sector-shaped imaging area for the runway. The flexible bracket is used to mount the turntable; The edge computing module is used to stitch together hyperspectral images and detect ice and snow water and its area.

Images