Hyperspectral image defogging method and device, equipment and storage medium

What is AI technical title?
AI technical title is built by Patsnap AI team. It summarizes the technical point description of the patent document.
By calculating the average pixel values of infrared and visible light, fog intensity maps and fog feature maps, and combining fog abundance, using a fog model for hyperspectral image defog treatment, the defog calculation in the existing technology is solved by solving the problem of large amounts, slow speed and poor results in the existing technology. The problem of this achieves efficient and high-speed fog removal effect.

CN119417744APending Publication Date: 2025-02-11GREE ELECTRIC APPLIANCE INC OF ZHUHAI +1

View PDF 0 Cites 0 Cited by

Patent Information

Application Number: CN202411336239.6
Authority / Receiving Office: CN · China
Patent Type: Applications(China)
Current Assignee / Owner
Filing Date: 2024-09-24
Publication Date: 2025-02-11

AI Technical Summary

Technical Problem

In the prior art, the high-spectral image defog removal method has a large calculation amount, slow processing speed, and poor fog removal effect, making it difficult to obtain clear and fog-free hyperspectral images in high-speed photography.

Method used

By obtaining the hyperspectral image to be defogged, calculating the average pixel values of infrared and visible light, calculating the fog intensity map, obtaining the fog feature map and fog abundance, and defogging the image using the fog model.

Benefits of technology

Efficient hyperspectral image defogging processing is achieved, reducing the calculation amount, improving the processing speed, and maintaining good defogging performance under different scenes and fog concentration conditions, restoring the detailed information of the image.

✦ Generated by Eureka AI based on patent content.

Smart Images

Figure CN119417744A_ABST

Patent Text Reader

Abstract

The embodiment of the invention provides a hyperspectral image defogging method and device, equipment and a storage medium, and the method comprises the steps: obtaining a to-be-defogged hyperspectral image, the hyperspectral image comprising images of a plurality of channels; respectively calculating an average value of pixel values of each pixel in all channels of an infrared band and an average value of pixel values of each pixel in all channels of a visible light band in the hyperspectral image to be defogged to obtain an infrared average pixel image and a visible light average pixel image; calculating a fog intensity graph by using the infrared average pixel image and the visible light average pixel image; calculating the fog abundance of each channel; and defogging the to-be-defogged hyperspectral image by using a fog model according to the fog intensity graph and the fog abundance to obtain a defogged hyperspectral image. According to the embodiment of the invention, the calculation amount of the defogging of the hyperspectral image is greatly reduced, the defogging processing speed is improved, and a clear and fogless hyperspectral image can be obtained in a high-speed photographing environment.

Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a hyperspectral image defogging method and device, an electronic device and a storage medium. Background Art

[0002] Hyperspectral sensors are capable of recording hundreds of spectral channels and are a rapidly developing remote sensing technology that is widely used in many fields such as urban management, environmental protection, and agricultural monitoring. However, under harsh atmospheric conditions, the absorption and scattering of reflected light by atmospheric particles will cause the contrast of the collected hyperspectral images to decrease and the visibility to deteriorate. These problems seriously affect subsequent practical applications such as target recognition.

[0003] In order to solve the interference of fog on image quality, the computer vision field has proposed a variety of defogging methods for grayscale or color natural images, and the remote sensing field has also developed a variety of defogging technologies. However, existing defogging methods generally have problems such as large computational complexity, slow processing speed, and poor defogging effect, which makes it difficult to obtain clear and fog-free hyperspectral images in high-speed photography. Summary of the invention

[0004] The embodiment of the present application provides a hyperspectral image defogging method to solve the problem that the defogging method in the prior art has large computational complexity, slow processing speed, poor defogging effect, and difficulty in obtaining clear and fog-free hyperspectral images in high-speed photography.

[0005] Correspondingly, the embodiments of the present application also provide a hyperspectral image defog device, an electronic device and a storage medium to ensure the implementation and application of the above method.

[0006] In order to solve the above problems, the present application discloses a hyperspectral image defogging method, which includes:

[0007] Acquire a hyperspectral image to be defogged, wherein the hyperspectral image includes images of several channels;

[0008] Calculate the average pixel value of each pixel in all channels of the infrared band and the average pixel value of each pixel in all channels of the visible light band in the hyperspectral image to be defogged, and obtain an infrared average pixel image and a visible light average pixel image;

[0009] Calculate a fog intensity map using the infrared average pixel image and the visible light average pixel image;

[0010] Obtaining a fog feature map according to a first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map;

[0011] The fog abundance of each channel is calculated according to the pixel values corresponding to the dense fog pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixel points of the fog intensity map, and the pixel values corresponding to the fog-free pixel points of the fog intensity map, wherein the dense fog pixel points and the fog-free pixel points are obtained according to the fog feature map;

[0012] According to the fog intensity map and the fog abundance, the hyperspectral image to be defogged is defogged using a fog model to obtain a defogged hyperspectral image.

[0013] In an optional embodiment of the present application, obtaining a fog feature map according to the first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map includes:

[0014] Extracting a second color space image of the hyperspectral image to be defogged;

[0015] Converting the second color space image into the first color space image, wherein the first color space image at least includes hue component data and lightness component data;

[0016] The fog characteristic map is obtained by performing calculations using the hue component data, the brightness component data and the fog intensity map.

[0017] In an optional embodiment of the present application, the dense fog pixel point and the fog-free pixel point are obtained according to the fog feature map, including:

[0018] Selecting a pixel point with the highest pixel value from the fog feature map as the dense fog pixel point;

[0019] A pixel point with the lowest pixel value is selected from the fog feature map as the fog-free pixel point.

[0020] In an optional embodiment of the present application, defogging the hyperspectral image to be defogged using a fog model according to the fog intensity map and the fog abundance includes:

[0021] Performing data expansion on the fog abundance of each channel to obtain an expanded fog abundance, wherein the size of the expanded fog abundance matches the size of the hyperspectral image to be defogged;

[0022] The hyperspectral image to be defogged is defogged using a fog model according to the fog intensity map and the expanded fog abundance.

[0023] In an optional embodiment of the present application, the calculating of the fog intensity map using the infrared average pixel image and the visible light average pixel image includes:

[0024] The visible light average pixel image is subtracted from the infrared average pixel image to obtain the fog intensity map.

[0025] In an optional embodiment of the present application, after subtracting the visible light average pixel image from the infrared average pixel image to obtain the fog intensity map, the method further includes:

[0026] Traversing the fog intensity values corresponding to all pixels in the fog intensity map, and filtering out invalid pixels whose fog intensity values are less than zero;

[0027] The fog intensity value corresponding to the invalid pixel point is set to a preset fog intensity value.

[0028] In an optional embodiment of the present application, after acquiring the hyperspectral image to be defogged, the method further includes:

[0029] Converting the hyperspectral image to be defogged into a one-dimensional hyperspectral image;

[0030] According to a preset mapping method, the pixel value range of the pixel points in the one-dimensional hyperspectral image is mapped to a normalized interval;

[0031] The one-dimensional hyperspectral image is dimensionalized to obtain a normalized hyperspectral image to be defogged.

[0032] In an optional embodiment of the present application, after obtaining the normalized hyperspectral image to be defogged, the method further includes:

[0033] The normalized hyperspectral image to be defogged is subjected to an adaptive grayscale transformation to obtain a grayscale transformed hyperspectral image to be defogged.

[0034] In an optional embodiment of the present application, after obtaining the hyperspectral image to be defogged after grayscale transformation, the method further includes:

[0035] The image of a preset channel in the hyperspectral image to be defogged after the grayscale transformation is removed, and the image of the preset channel does not have image information.

[0036] In an optional embodiment of the present application, after obtaining the defogging hyperspectral image, the method further includes:

[0037] Selecting a gamma correction coefficient according to the defogged hyperspectral image;

[0038] According to the gamma correction coefficient, an exponential operation is performed on the pixel value corresponding to each pixel point in the defogged hyperspectral image to obtain a gamma-corrected defogged image.

[0039] In an optional embodiment of the present application, the fog abundance of each channel is calculated according to the pixel values corresponding to the dense fog pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixel points of the fog intensity map, and the pixel values corresponding to the fog-free pixel points of the fog intensity map, including:

[0040] The fog abundance of each channel is calculated according to the simplified fog model according to the pixel values corresponding to the dense fog pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixels of the fog intensity map, and the pixel values corresponding to the fog-free pixels of the fog intensity map.

[0041] The present application also discloses a hyperspectral image defogging device, the device comprising:

[0042] An image acquisition module, used for acquiring a hyperspectral image to be defogged, wherein the hyperspectral image includes images of several channels;

[0043] The average pixel image calculation module is used to calculate the average pixel value of each pixel in all channels of the infrared band and the average pixel value of each pixel in all channels of the visible light band in the hyperspectral image to be defogged, so as to obtain an infrared average pixel image and a visible light average pixel image;

[0044] A fog intensity map calculation module, used to calculate a fog intensity map using the infrared average pixel image and the visible light average pixel image;

[0045] A fog feature map calculation module, used for obtaining a fog feature map according to a first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map;

[0046] A fog abundance calculation module, for calculating the fog abundance of each channel according to the pixel values corresponding to the dense fog pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixel points of the fog intensity map, and the pixel values corresponding to the fog-free pixel points of the fog intensity map, wherein the dense fog pixel points and the fog-free pixel points are obtained according to the fog feature map;

[0047] The defogging module is used to defog the hyperspectral image to be defogged using a fog model according to the fog intensity map and the fog abundance, so as to obtain a defogged hyperspectral image.

[0048] The embodiment of the present application also discloses an electronic device, including: a processor; and a memory, on which executable code is stored. When the executable code is executed, the processor executes the hyperspectral image defogging method as described in one or more of the embodiments of the present application.

[0049] The embodiments of the present application also disclose one or more machine-readable media on which executable codes are stored. When the executable codes are executed, a processor executes the hyperspectral image defogging method as described in one or more of the embodiments of the present application.

[0050] Compared with the prior art, the embodiments of the present application have the following advantages:

[0051] In an embodiment of the present application, a hyperspectral image to be defogged is obtained, and the hyperspectral image includes images of several channels; the average value of the pixel value of each pixel in the hyperspectral image to be defogged in all channels of the infrared band and the average value of the pixel value of each pixel in all channels of the visible light band are calculated respectively to obtain an infrared average pixel image and a visible light average pixel image; the infrared average pixel image and the visible light average pixel image are used to calculate a fog intensity map; a fog feature map is obtained according to the first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map; the fog abundance of each channel is calculated according to the pixel value corresponding to each channel of the dense fog pixel point of the hyperspectral image to be defogged, the pixel value corresponding to each channel of the fog-free pixel point of the hyperspectral image to be defogged, the pixel value corresponding to each channel of the fog-free pixel point of the hyperspectral image to be defogged, the pixel value corresponding to the dense fog pixel point of the fog intensity map, and the pixel value corresponding to the fog-free pixel point of the fog intensity map, and the dense fog pixel point and the fog-free pixel point are obtained according to the fog feature map; according to the fog intensity map and the fog abundance, the hyperspectral image to be defogged is defogged using a fog model to obtain a defogged hyperspectral image. Compared with the traditional deep learning or machine learning defogging method, the embodiment of the present application does not need to rely on the training of deep learning models for hyperspectral images with and without fog. Instead, it can achieve efficient hyperspectral image defogging by performing matrix operations on the hyperspectral images to be defogged. The embodiment of the present application not only greatly reduces the amount of calculation for hyperspectral image defogging and improves the defogging processing speed, but also has good defogging performance in different scenes and different fog concentration conditions, and can restore the detailed information of hyperspectral images, ensuring that clear and fog-free hyperspectral images can be obtained even in high-speed photography environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flowchart of the steps of an embodiment of a hyperspectral image defogging method of the present application;

[0053] Figure 2 It is a flow chart of an embodiment of a hyperspectral image defogging method of the present application;

[0054] Figure 3AThis is a schematic diagram of AVIRIS data before defogging in an embodiment of a hyperspectral image defogging method of the present application;

[0055] Figure 3B This is a schematic diagram of AVIRIS data after defogging in an embodiment of a hyperspectral image defogging method of the present application;

[0056] Figure 4A This is a schematic diagram of GF data before defogging in an embodiment of a hyperspectral image defogging method of the present application;

[0057] Figure 4B This is a schematic diagram of GF data after defogging according to an embodiment of a hyperspectral image defogging method of the present application;

[0058] Figure 5 It is a structural block diagram of an embodiment of a hyperspectral image defogging device of the present application;

[0059] Figure 6 It is a schematic diagram of the structure of a device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0061] Hyperspectral sensors can record hundreds of spectral channels and carry rich spectral information. They are an increasingly popular remote sensing technology and have been widely used in many fields such as urban management, environmental protection, and agricultural monitoring. However, under harsh atmospheric conditions, the absorption and scattering of reflected light by atmospheric particles can cause the collected hyperspectral images to have low contrast and poor visibility. These effects seriously interfere with subsequent practical applications such as target recognition.

[0062] In order to solve the interference of fog on image quality, a variety of defogging methods for grayscale or color natural images have been proposed in the field of computer vision. These methods can be roughly divided into two categories: defogging methods based on atmospheric scattering models and defogging methods based on image contrast enhancement. Defogging methods based on atmospheric scattering models usually follow three steps: first calculate the transmission matrix of foggy images, then estimate the global atmospheric light, and finally estimate high-quality images by solving the atmospheric scattering model. In contrast, defogging methods based on image contrast enhancement aim to improve image visibility by improving image details.

[0063] In the field of remote sensing, a variety of defogging methods have also been proposed. Early defogging methods focused on image enhancement techniques for multispectral remote sensing images, including histogram matching, retinal enhancement, homomorphic filtering, and wavelet transform. In order to improve the defogging performance, defogging methods such as trilateral filtering have been proposed to eliminate haze in multispectral remote sensing images. Although various defogging methods have been proposed in the field of computer vision, existing defogging methods generally have problems such as large defogging calculation, slow defogging speed, and poor defogging performance, which makes it impossible to obtain clear and haze-free hyperspectral images in high-speed photography.

[0064] Therefore, the embodiment of the present application proposes a fast and effective hyperspectral image defogging method, aiming to overcome the shortcomings of the prior art and provide a solution that can achieve clear and fog-free hyperspectral image acquisition in high-speed photography.

[0065] Reference Figure 1 , is a flowchart of a method for defogging a hyperspectral image according to an embodiment of the present invention, comprising the following steps:

[0066] Step 101: Acquire a hyperspectral image to be defogged, wherein the hyperspectral image includes images of several channels.

[0067] In step 101, firstly, a hyperspectral image to be defogged is obtained. The hyperspectral image to be defogged includes images of multiple channels. In one example, the hyperspectral image obtained by the AVIRIS (Airborne Visible / Infrared Imaging Spectrometer) satellite contains images of 224 channels, and the image pixels of each channel are 512×512; while the hyperspectral image obtained by the Gaofen (GF) 5 remote sensing satellite contains images of 330 channels, and the image pixels of each channel are 512×512. The hyperspectral image also contains images of multiple bands, and each band corresponds to a different channel. In one example, the visible light band of the hyperspectral image to be defogged in the AVIRIS data corresponds to channels 1-30, and the infrared band corresponds to channels 180-210; while the visible light band of the hyperspectral image to be defogged in the GF data corresponds to channels 1-61, and the infrared band corresponds to channels 251-287.

[0068] In an optional embodiment, after obtaining the hyperspectral image to be defogged, the hyperspectral image to be defogged may be preprocessed, and the preprocessing may include normalization, adaptive grayscale transformation, and removal of images of preset channels, etc. This embodiment does not impose any limitation on the method of preprocessing the hyperspectral image to be defogged.

[0069] Step 102: Calculate the average pixel value of each pixel in all channels of the infrared band and the average pixel value of each pixel in all channels of the visible light band in the hyperspectral image to be defogged, and obtain an infrared average pixel image and a visible light average pixel image.

[0070] In step 102, all images of the hyperspectral image to be defogged in the visible light band and the infrared band are extracted to calculate the infrared average pixel image and the visible light average pixel image. Taking AVIRIS data as an example, AVIRIS data needs to extract images of channels 1-30 to calculate the visible light average pixel image, and extract images of channels 180-210 to calculate the infrared average pixel image. Then, the average value of the pixel values of all channels of the infrared band for each pixel in the hyperspectral image to be defogged and the average value of the pixel values of all channels of the visible light band for each pixel are calculated to obtain a single infrared average pixel image and a single visible light average pixel image.

[0071] In an optional embodiment, the infrared average pixel image and the visible light average pixel image may be calculated using the following formula (1).

[0072]

[0073] Among them, when k is 1, it means the calculation of the average pixel image of visible light, represents the average pixel image of visible light, Indicates the number of visible light bands; when k is 2, it indicates the calculation of the infrared average pixel image. represents the infrared average pixel image, Indicates the number of infrared bands; a indicates the ath image in the process of calculating the sum of pixel values of all channels of each pixel in the infrared band or visible light band, such as Represents the first image of the visible light band, Represents the first image of the infrared band.

[0074] Step 103: Calculate a fog intensity map using the infrared average pixel image and the visible light average pixel image.

[0075] In step 103, the fog intensity map is calculated using the infrared average pixel image and the visible light average pixel image. Since the wavelength of the infrared band is longer than that of the visible light band, the infrared band has a stronger penetrating ability than the visible light band. Compared with the image of the visible light band, there is almost no fog in the image of the infrared band. Therefore, in an optional embodiment, the fog intensity map can be obtained by subtracting the visible light average pixel image from the infrared average pixel image.

[0076] In one embodiment, the fog intensity map may be calculated according to the following formula (2).

[0077]

[0078] Among them, F is the fog intensity map, I 1 is the average pixel image of visible light, I 2 is the infrared average pixel image.

[0079] Step 104: Obtain a fog feature map according to the first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map.

[0080] In step 104, a fog feature map is calculated based on the first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map. The larger the pixel value in the fog feature map, the higher the fog density at that location. Optionally, the first color space image is an HSV image. Since dense fog has the characteristics of high hue and high brightness in the image, the fog feature map can be calculated using the hue component data and brightness component data in the HSV image and the fog intensity map. The specific calculation formula can be the following formula (3).

[0081] E=F·V·H (3)

[0082] Among them, E is the fog feature map, F is the fog intensity map, V is the brightness component data, and H is the hue component data. The fog feature map E can be obtained by multiplying the three.

[0083] In one embodiment, before calculating the fog feature map, it is first necessary to extract the second color space image of the hyperspectral image to be defogged from the hyperspectral image to be defogged, and the second color space image can be an RGB image. For example, the RGB bands of AVIRIS data are 29, 19, and 12, and the RGB bands of GF data are 59, 38, and 20. An image combination of the corresponding channels is proposed as an RGB image. Afterwards, the second color space image is converted into the first color space image. In one example, the RGB image of the hyperspectral image to be defogged is converted into an HSV image.

[0084] Step 105, calculate the fog abundance of each channel according to the pixel values corresponding to the dense fog pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixels of the fog intensity map, and the pixel values corresponding to the fog-free pixels of the fog intensity map, wherein the dense fog pixels and the fog-free pixels are obtained according to the fog feature map.

[0085] In step 105, since the larger the pixel value in the fog feature map, the denser the fog is, the coordinates of the dense fog pixel points and the fog-free pixel points can be obtained according to the fog feature map. Specifically, the two pixel points with the highest pixel value and the lowest pixel value in the fog feature map can be selected to represent the dense fog pixel points and the fog-free pixel points, respectively.

[0086] Then, the fog abundance of each channel is calculated according to the pixel values corresponding to the dense fog pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixels of the fog intensity map, and the pixel values corresponding to the fog-free pixels of the fog intensity map. Specifically, the pixel values corresponding to the dense fog pixels and the pixel values corresponding to the fog-free pixels of the fog intensity map are obtained, and then the pixel values corresponding to the dense fog pixels and the pixel values corresponding to the fog-free pixels of each channel of the hyperspectral image to be defogged are extracted, and the fog abundance corresponding to the image of each channel is calculated respectively. In one example, the fog abundance obtained for the hyperspectral image to be defogged in the AVIRIS data is 1×224 data, that is, the fog abundance corresponding to the images of 224 channels respectively.

[0087] In one embodiment, the fog abundance corresponding to each channel may be calculated using the following formula (4).

[0088]

[0089] Among them, I(x 1 ,y 1 ) represents the pixel value of the dense fog pixel point in the hyperspectral image to be defogged corresponding to this channel, I(x 2 ,y 2 ) represents the pixel value of the non-fog pixel point in the hyperspectral image to be defogged corresponding to this channel, F(x 1 ,y 1 ) represents the pixel value corresponding to the dense fog pixel point in the fog intensity map, F(x 2 ,y 2 ) represents the pixel value corresponding to the fog-free pixel point in the fog intensity map.

[0090] Step 106: Defogging the hyperspectral image to be defogged using a fog model according to the fog intensity map and the fog abundance to obtain a defogged hyperspectral image.

[0091] In step 106, based on the fog intensity map and fog abundance calculated above, the fog model is used to defog the hyperspectral image to be defogged, thereby obtaining a defogged hyperspectral image. In an optional embodiment, the fog model can be expressed as the following formula (5).

[0092] I(x)=O(x)+A(x)F(x) (5)

[0093] Wherein, I(x) represents the hyperspectral image to be defogged, O(x) represents the hyperspectral image after defog, A(x) represents the fog abundance, and F(x) represents the fog intensity map. The embodiment of the present application only needs to perform simple matrix operations on the hyperspectral image to be defogged to achieve efficient hyperspectral image defogging processing.

[0094] In an optional embodiment, before calculating the defogged hyperspectral image, it is necessary to expand the fog abundance data so that its size matches the hyperspectral image to be defogged. For example, in the AVIRIS data, the size of the hyperspectral image to be defogged is 512×512×224, and the fog abundance data obtained is 1×224, and the fog abundance data needs to be expanded to 512×512×224. Specifically, each pixel value of each channel in the fog abundance image can be set to the fog abundance of the channel.

[0095] In an embodiment of the present application, a hyperspectral image to be defogged is obtained, and the hyperspectral image includes images of several channels; the average value of the pixel value of each pixel in the hyperspectral image to be defogged in all channels of the infrared band and the average value of the pixel value of each pixel in all channels of the visible light band are calculated respectively to obtain an infrared average pixel image and a visible light average pixel image; the infrared average pixel image and the visible light average pixel image are used to calculate a fog intensity map; a fog feature map is obtained according to the first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map; the fog abundance of each channel is calculated according to the pixel value corresponding to each channel of the dense fog pixel point of the hyperspectral image to be defogged, the pixel value corresponding to each channel of the fog-free pixel point of the hyperspectral image to be defogged, the pixel value corresponding to each channel of the fog-free pixel point of the hyperspectral image to be defogged, the pixel value corresponding to the dense fog pixel point of the fog intensity map, and the pixel value corresponding to the fog-free pixel point of the fog intensity map, and the dense fog pixel point and the fog-free pixel point are obtained according to the fog feature map; according to the fog intensity map and the fog abundance, the hyperspectral image to be defogged is defogged using a fog model to obtain a defogged hyperspectral image. Compared with the traditional deep learning or machine learning defogging method, the embodiment of the present application does not need to rely on the training of deep learning models for hyperspectral images with and without fog. Instead, it can achieve efficient hyperspectral image defogging by performing matrix operations on the hyperspectral images to be defogged. The embodiment of the present application not only greatly reduces the amount of calculation for hyperspectral image defogging and improves the defogging processing speed, but also has good defogging performance in different scenes and different fog concentration conditions, and can restore the detailed information of hyperspectral images, ensuring that clear and fog-free hyperspectral images can be obtained even in high-speed photography environments.

[0096] In an optional embodiment of the present application, obtaining a fog feature map according to the first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map includes:

[0097] Extracting a second color space image of the hyperspectral image to be defogged;

[0098] Converting the second color space image into the first color space image, wherein the first color space image at least includes hue component data and lightness component data;

[0099] The fog characteristic map is obtained by performing calculations using the hue component data, the brightness component data and the fog intensity map.

[0100] In this embodiment, when calculating the fog feature map, it is first necessary to extract the second color space image of the hyperspectral image to be defogged, convert the second color space image into a first color space image, the first color space image at least includes hue component data and lightness component data, and then use the hue component data, lightness component data and fog intensity map to calculate and obtain the fog feature map. In one example, the first color space image is an HSV image, and the second color space image is an RGB image. The image combination of the channels corresponding to the R, G, and B bands is extracted as an RGB image. Afterwards, the RGB image of the hyperspectral image to be defogged is converted into an HSV image, and the fog feature map can be calculated using the hue component data and lightness component data in the HSV image and the fog intensity map.

[0101] The embodiment of the present application utilizes the characteristics of dense fog in the image with high hue and high brightness, converts the second color space image into the first color space image, and calculates the fog feature map for the subsequent selection of dense fog pixels and fog-free pixels, and the calculation process is convenient and fast.

[0102] In an optional embodiment of the present application, the dense fog pixel point and the fog-free pixel point are obtained according to the fog feature map, including:

[0103] Selecting a pixel point with the highest pixel value from the fog feature map as the dense fog pixel point;

[0104] A pixel point with the lowest pixel value is selected from the fog feature map as the fog-free pixel point.

[0105] In this embodiment, after obtaining the fog feature map, since the larger the pixel value in the fog feature map is, the higher the fog density is at that location, the coordinates of the dense fog pixel point and the fog-free pixel point can be obtained according to the fog feature map. The pixel point with the highest pixel value can be selected from the fog feature map as the dense fog pixel point, and the pixel point with the lowest pixel value can be selected from the fog feature map as the fog-free pixel point.

[0106] The embodiment of the present application selects dense fog pixels and fog-free pixels from the fog feature map for subsequent calculation of fog abundance.

[0107] In an optional embodiment of the present application, defogging the hyperspectral image to be defogged using a fog model according to the fog intensity map and the fog abundance includes:

[0108] Performing data expansion on the fog abundance of each channel to obtain an expanded fog abundance, wherein the size of the expanded fog abundance matches the size of the hyperspectral image to be defogged;

[0109] The hyperspectral image to be defogged is defogged using a fog model according to the fog intensity map and the expanded fog abundance.

[0110] Before defogging the hyperspectral image to be defogged, the present embodiment needs to first perform data expansion on the fog abundance so that its size matches that of the hyperspectral image to be defogged. For example, in the AVIRIS data, the size of the hyperspectral image to be defogged is 512×512×224, and the fog abundance data obtained is 1×224, and the fog abundance data needs to be expanded to 512×512×224. In the present embodiment, data expansion is performed on the fog abundance of each channel to obtain the expanded fog abundance, and the size of the expanded fog abundance matches the size of the hyperspectral image to be defogged, and then the hyperspectral image to be defogged is defogged using the fog model based on the fog intensity map and the expanded fog abundance. Specifically, during data expansion, each pixel value of the image of each channel in the fog abundance image can be set to the fog abundance of the channel.

[0111] In the embodiment of the present application, after data expansion of the fog abundance is performed, the hyperspectral image to be defogged is defogged, so that the data size of the fog abundance matches the hyperspectral image to be defogged, which is convenient for defog calculation.

[0112] In an optional embodiment of the present application, it includes:

[0113] The visible light average pixel image is subtracted from the infrared average pixel image to obtain the fog intensity map.

[0114] Since the wavelength of the infrared band is longer than that of the visible light band, the infrared band has stronger penetrating power than the visible light band. Compared with the image of the visible light band, there is almost no fog in the image of the infrared band. Therefore, when calculating the fog intensity map, this embodiment subtracts the visible light average pixel image from the infrared average pixel image to obtain the fog intensity map. In one embodiment, the fog intensity map can be calculated according to the above formula (2).

[0115] The embodiment of the present application utilizes the characteristic that the infrared band has stronger penetrating power than the visible light band to calculate the fog intensity map, thereby obtaining the fog intensity map for use in the subsequent defogging calculation process.

[0116] In an optional embodiment of the present application, after subtracting the visible light average pixel image from the infrared average pixel image to obtain the fog intensity map, the method further includes:

[0117] Traversing the fog intensity values corresponding to all pixels in the fog intensity map, and filtering out invalid pixels whose fog intensity values are less than zero;

[0118] The fog intensity value corresponding to the invalid pixel point is set to a preset fog intensity value.

[0119] In this embodiment, since the pixel values of the fog intensity map need to be positive values, invalid pixels in the fog intensity map need to be processed after the fog intensity map is obtained, and the fog intensity values corresponding to the invalid pixels are less than zero. The fog intensity values corresponding to all pixels in the fog intensity map are traversed, and the invalid pixels whose fog intensity values are less than zero are screened out, and then the fog intensity values corresponding to the invalid pixels are set to the preset fog intensity values. In one example, the preset fog intensity value can be zero, that is, the fog intensity values corresponding to the invalid pixels are reset to zero.

[0120] The embodiment of the present application processes invalid pixels in the fog intensity map to prevent errors from occurring in subsequent calculations, thereby affecting the defogging effect.

[0121] In an optional embodiment of the present application, after acquiring the hyperspectral image to be defogged, the method further includes:

[0122] Converting the hyperspectral image to be defogged into a one-dimensional hyperspectral image;

[0123] According to a preset mapping method, the pixel value range of the pixel points in the one-dimensional hyperspectral image is mapped to a normalized interval;

[0124] The one-dimensional hyperspectral image is dimensionalized to obtain a normalized hyperspectral image to be defogged.

[0125] After acquiring the hyperspectral image to be defogged, the present embodiment first needs to preprocess the hyperspectral image to be defogged, and the preprocessing may include normalization. Specifically, the multi-dimensional hyperspectral image to be defogged is converted into a one-dimensional hyperspectral image, and then the pixel value range of the pixel points in the one-dimensional hyperspectral image is mapped to the normalized interval according to a preset mapping method. In one example, the normalized interval may be 0 to 1. In this example, if the pixel value range of the pixel points in the one-dimensional hyperspectral image is 0 to 255, the pixel value range of the pixel points in the one-dimensional hyperspectral image after processing is changed to 0 to 1. Afterwards, the one-dimensional hyperspectral image is dimensionalized and restored to its original dimension to obtain the normalized hyperspectral image to be defogged.

[0126] The embodiment of the present application performs normalization processing on the hyperspectral image to be defogged, which can effectively reduce errors in subsequent calculations and improve calculation accuracy.

[0127] In an optional embodiment of the present application, after obtaining the normalized hyperspectral image to be defogged, the method further includes:

[0128] The normalized hyperspectral image to be defogged is subjected to an adaptive grayscale transformation to obtain a grayscale transformed hyperspectral image to be defogged.

[0129] This embodiment can further perform adaptive grayscale transformation on the normalized hyperspectral image to be defogged. Specifically, the normalized hyperspectral image to be defogged is adaptively transformed to obtain the hyperspectral image to be defogged after grayscale transformation.

[0130] The embodiment of the present application performs adaptive grayscale transformation on the normalized hyperspectral image to be defogged, thereby improving the contrast of the hyperspectral image to be defogged, which facilitates subsequent defog processing.

[0131] In an optional embodiment of the present application, after obtaining the hyperspectral image to be defogged after grayscale transformation, the method further includes:

[0132] The image of a preset channel in the hyperspectral image to be defogged after the grayscale transformation is removed, and the image of the preset channel does not have image information.

[0133] This embodiment can remove the images of preset channels in the hyperspectral image to be defogged after grayscale transformation during preprocessing. In the hyperspectral image to be defogged, due to severe water and atmospheric absorption, there is no image information on the images of these channels, and the pixel values corresponding to each pixel point are empty, which will interfere with the subsequent calculation process. Therefore, it is necessary to remove the images of these channels to reduce interference factors. For example, for GF data, the hyperspectral image to be defogged needs to remove images of 25 bands (numbered 193-200, numbered 246-262).

[0134] The embodiment of the present application can reduce the influence of the images of preset channels on the defogging process by removing the images of these channels that do not have image information.

[0135] In an optional embodiment of the present application, after obtaining the defogging hyperspectral image, the method further includes:

[0136] Selecting a gamma correction coefficient according to the defogged hyperspectral image;

[0137] According to the gamma correction coefficient, an exponential operation is performed on the pixel value corresponding to each pixel point in the defogged hyperspectral image to obtain a gamma-corrected defogged image.

[0138] In this embodiment, in order to solve the problems of low contrast and low brightness of the hyperspectral image after preliminary defogging, after obtaining the hyperspectral image after defogging, gamma correction can be performed on the hyperspectral image after defogging to improve the contrast of the hyperspectral image after defogging. Specifically, according to the hyperspectral image after defogging, a gamma correction coefficient is selected, and according to the gamma correction coefficient, an exponential operation is performed on the pixel value corresponding to each pixel point in the hyperspectral image after defogging to increase the pixel value distribution range of the image, thereby improving the contrast of the image, and thus obtaining a defogging image after gamma correction.

[0139] The embodiment of the present application performs gamma correction on the defogged hyperspectral image, which can improve the contrast of the image, restore the detail information of the hyperspectral image, and ensure the acquisition of a clear and fog-free hyperspectral image.

[0140] In an optional embodiment of the present application, the fog abundance of each channel is calculated according to the pixel values corresponding to the dense fog pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixel points of the fog intensity map, and the pixel values corresponding to the fog-free pixel points of the fog intensity map, including:

[0141] The fog abundance of each channel is calculated according to the simplified fog model according to the pixel values corresponding to the dense fog pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixels of the fog intensity map, and the pixel values corresponding to the fog-free pixels of the fog intensity map.

[0142] In this embodiment, when calculating fog abundance, the fog abundance of each channel can be calculated according to the pixel values corresponding to the dense fog pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixels of the fog intensity map, and the pixel values corresponding to the fog-free pixels of the fog intensity map, according to the simplified fog model. The simplified fog model can be expressed as the above formula (4).

[0143] The embodiment of the present application calculates the fog abundance of each channel according to a simplified fog model, and can achieve efficient hyperspectral image defogging processing by performing matrix operations on the hyperspectral image to be defogged.

[0144] In an embodiment of the present application, a hyperspectral image to be defogged is obtained, and the hyperspectral image includes images of several channels; the average value of the pixel value of each pixel in the hyperspectral image to be defogged in all channels of the infrared band and the average value of the pixel value of each pixel in all channels of the visible light band are calculated respectively to obtain an infrared average pixel image and a visible light average pixel image; the infrared average pixel image and the visible light average pixel image are used to calculate a fog intensity map; a fog feature map is obtained according to the first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map; the fog abundance of each channel is calculated according to the pixel value corresponding to each channel of the dense fog pixel point of the hyperspectral image to be defogged, the pixel value corresponding to each channel of the fog-free pixel point of the hyperspectral image to be defogged, the pixel value corresponding to each channel of the fog-free pixel point of the hyperspectral image to be defogged, the pixel value corresponding to the dense fog pixel point of the fog intensity map, and the pixel value corresponding to the fog-free pixel point of the fog intensity map, and the dense fog pixel point and the fog-free pixel point are obtained according to the fog feature map; according to the fog intensity map and the fog abundance, the hyperspectral image to be defogged is defogged using a fog model to obtain a defogged hyperspectral image. Compared with the traditional deep learning or machine learning defogging method, the embodiment of the present application does not need to rely on the training of deep learning models for hyperspectral images with and without fog. Instead, it can achieve efficient hyperspectral image defogging by performing matrix operations on the hyperspectral images to be defogged. The embodiment of the present application not only greatly reduces the amount of calculation for hyperspectral image defogging and improves the defogging processing speed, but also has good defogging performance in different scenes and different fog concentration conditions, and can restore the detailed information of hyperspectral images, ensuring that clear and fog-free hyperspectral images can be obtained even in high-speed photography environments.

[0145] In order to enable those skilled in the art to better understand the hyperspectral image defogging method proposed in the embodiment of the present application, the following is combined with Figure 2 , the specific process of a hyperspectral image defogging method in an embodiment of the present application is described.

[0146] Reference Figure 2 , is a flow chart of an embodiment of a hyperspectral image defogging method of the present application.

[0147] Step 201: input a hyperspectral image to be defogged.

[0148] In step 201, a hyperspectral image to be defogged is first inputted, and the hyperspectral image to be defogged is preprocessed. The preprocessing may include normalization, adaptive grayscale transformation, and removal of images of preset channels.

[0149] Step 202: Calculate the average visible light pixel image in the visible light band and the average infrared pixel image in the infrared band of the hyperspectral image to be defogged, and calculate the fog intensity map by taking the difference.

[0150] In step 202, all images of the hyperspectral image to be defogged in the visible light band and the infrared band are extracted, and the visible light average pixel image of the hyperspectral image to be defogged in the visible light band and the infrared average pixel image of the infrared band are calculated according to the above formula (1), and the fog intensity map is calculated by difference according to the above formula (2).

[0151] Step 203: perform HSV color space transformation on the RGB image of the hyperspectral image to be defogged, multiply the hue component data, the brightness component data and the fog intensity map to obtain a fog feature map, and automatically extract dense fog pixels and fog-free pixels.

[0152] In step 203, the RGB band image of the hyperspectral image to be defogged is extracted, and the RGB image of the hyperspectral image to be defogged is transformed into the HSV color space. The hue component data (H), the brightness component data (V) and the fog intensity map are multiplied according to the above formula (3) to obtain a fog feature map. The coordinates of the pixel points with the highest and lowest pixel values in the fog feature map are respectively selected as the dense fog pixel points and the fog-free pixel points.

[0153] Step 204: Calculate the fog abundance in different bands using the fog intensity difference between the dense fog pixel points and the fog-free pixel points in the fog intensity map.

[0154] In step 204, the fog abundance of different bands is calculated according to the simplified fog model represented by the above formula (4) using the fog intensity difference between the dense fog pixel points and the fog-free pixel points in the fog intensity map.

[0155] Step 205: According to the fog model, the fog intensity map and the fog abundance are used to reconstruct the defogged hyperspectral image.

[0156] In step 205, according to the fog model represented by the above formula (5), the fog intensity map and the fog abundance are used to reconstruct the defogged hyperspectral image.

[0157] the following Figure 3A to Figure 4B The defogging result obtained by defogging a hyperspectral image using a hyperspectral image defogging method according to an embodiment of the present application is demonstrated so that those skilled in the art can have a clearer understanding of the technical effect of the embodiment of the present application.

[0158] Reference Figure 3A , is a schematic diagram of AVIRIS data before defogging in an embodiment of a hyperspectral image defogging method of the present application.

[0159] Figure 3A The hyperspectral images of foggy days taken by the AVIRIS satellite, i.e., the hyperspectral images to be defogged, are displayed.

[0160] Reference Figure 3B , is a schematic diagram of AVIRIS data after defogging in an embodiment of a hyperspectral image defogging method of the present application.

[0161] Figure 3B A hyperspectral image defogging method according to an embodiment of the present application is presented. Figure 3A The defogging result of the hyperspectral image to be defogged is shown in FIG. 1 . It can be seen that the hyperspectral image defogging method according to the embodiment of the present application can restore the detail information of the hyperspectral image.

[0162] Reference Figure 4A , is a schematic diagram of GF data before defogging in an embodiment of a hyperspectral image defogging method of the present application.

[0163] Figure 4A The hyperspectral images of foggy days taken by Gaofen (GF) 5 remote sensing satellite, i.e., the hyperspectral images to be defogged, are displayed.

[0164] Reference Figure 4B , is a schematic diagram of GF data after defogging in an embodiment of a hyperspectral image defogging method of the present application.

[0165] Figure 4B A hyperspectral image defogging method according to an embodiment of the present application is presented. Figure 4A The defogging results of the hyperspectral image to be defogged are shown in FIG. 1 . It can be seen that a hyperspectral image defogging method according to an embodiment of the present application can restore the detail information of the hyperspectral image and has good defogging performance in different scenes and under different fog concentration conditions.

[0166] In an embodiment of the present application, a hyperspectral image to be defogged is obtained, and the hyperspectral image includes images of several channels; the average value of the pixel value of each pixel in the hyperspectral image to be defogged in all channels of the infrared band and the average value of the pixel value of each pixel in all channels of the visible light band are calculated respectively to obtain an infrared average pixel image and a visible light average pixel image; the infrared average pixel image and the visible light average pixel image are used to calculate a fog intensity map; a fog feature map is obtained according to the first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map; the fog abundance of each channel is calculated according to the pixel value corresponding to each channel of the dense fog pixel point of the hyperspectral image to be defogged, the pixel value corresponding to each channel of the fog-free pixel point of the hyperspectral image to be defogged, the pixel value corresponding to each channel of the fog-free pixel point of the hyperspectral image to be defogged, the pixel value corresponding to the dense fog pixel point of the fog intensity map, and the pixel value corresponding to the fog-free pixel point of the fog intensity map, and the dense fog pixel point and the fog-free pixel point are obtained according to the fog feature map; according to the fog intensity map and the fog abundance, the hyperspectral image to be defogged is defogged using a fog model to obtain a defogged hyperspectral image. Compared with the traditional deep learning or machine learning defogging method, the embodiment of the present application does not need to rely on the training of deep learning models for hyperspectral images with and without fog. Instead, it can achieve efficient hyperspectral image defogging by performing matrix operations on the hyperspectral images to be defogged. The embodiment of the present application not only greatly reduces the amount of calculation for hyperspectral image defogging and improves the defogging processing speed, but also has good defogging performance in different scenes and different fog concentration conditions, and can restore the detailed information of hyperspectral images, ensuring that clear and fog-free hyperspectral images can be obtained even in high-speed photography environments.

[0167] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0168] On the basis of the above embodiments, this embodiment further provides a hyperspectral image defogger device, which is applied to electronic devices such as terminal devices and servers.

[0169] Reference Figure 5 , shows a structural block diagram of an embodiment of a hyperspectral image defogging device of the present application, which may specifically include the following modules:

[0170] An image acquisition module 501 is used to acquire a hyperspectral image to be defogged, wherein the hyperspectral image includes images of several channels;

[0171] The average pixel image calculation module 502 is used to calculate the average pixel value of each pixel in all channels of the infrared band and the average pixel value of each pixel in all channels of the visible light band in the hyperspectral image to be defogged, so as to obtain an infrared average pixel image and a visible light average pixel image;

[0172] A fog intensity map calculation module 503, configured to calculate a fog intensity map using the infrared average pixel image and the visible light average pixel image;

[0173] A fog feature map calculation module 504 is used to obtain a fog feature map according to the first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map;

[0174] A fog abundance calculation module 505 is used to calculate the fog abundance of each channel according to the pixel values corresponding to the dense fog pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixel points of the fog intensity map, and the pixel values corresponding to the fog-free pixel points of the fog intensity map, wherein the dense fog pixel points and the fog-free pixel points are obtained according to the fog feature map;

[0175] The defogging module 506 is configured to defog the hyperspectral image to be defogged using a fog model according to the fog intensity map and the fog abundance, so as to obtain a defogged hyperspectral image.

[0176] In an optional embodiment of the present application, the fog feature map calculation module 504 includes:

[0177] A second color space image extraction submodule, used for extracting a second color space image of the hyperspectral image to be defogged;

[0178] A first color space image conversion submodule, used to convert the second color space image into the first color space image, wherein the first color space image at least includes hue component data and lightness component data;

[0179] The fog feature calculation submodule is used to use the hue component data, the brightness component data and the fog intensity map to perform calculations to obtain the fog feature map.

[0180] In an optional embodiment of the present application, the device includes:

[0181] A dense fog pixel point selection module is used to select the pixel point with the highest pixel value from the fog feature map as the dense fog pixel point;

[0182] The fog-free pixel point selection module is used to select the pixel point with the lowest pixel value from the fog feature map as the fog-free pixel point.

[0183] In an optional embodiment of the present application, the demisting module 506 includes:

[0184] A data expansion submodule, used for performing data expansion on the fog abundance of each channel to obtain an expanded fog abundance, wherein the size of the expanded fog abundance matches the size of the hyperspectral image to be defogged;

[0185] The fog model defogging submodule is used to use the fog model to defog the hyperspectral image to be defogged according to the fog intensity map and the expanded fog abundance.

[0186] In an optional embodiment of the present application, the fog intensity map calculation module 503 is further used to:

[0187] The visible light average pixel image is subtracted from the infrared average pixel image to obtain the fog intensity map.

[0188] In an optional embodiment of the present application, the device further includes:

[0189] An invalid pixel point screening module is used to traverse the fog intensity values corresponding to all pixel points in the fog intensity map and screen out invalid pixel points whose fog intensity values are less than zero;

[0190] The fog intensity value setting module is used to set the fog intensity value corresponding to the invalid pixel point to a preset fog intensity value.

[0191] In an optional embodiment of the present application, the device further includes:

[0192] A one-dimensional hyperspectral image conversion module, used for converting the hyperspectral image to be defogged into a one-dimensional hyperspectral image;

[0193] A normalization module, used for mapping the pixel value range of the pixel points in the one-dimensional hyperspectral image to a normalization interval according to a preset mapping method;

[0194] The dimensionality restoration module is used to perform dimensionality restoration on the one-dimensional hyperspectral image to obtain a normalized hyperspectral image to be defogged.

[0195] In an optional embodiment of the present application, the device further includes:

[0196] The grayscale transformation module is used to perform adaptive grayscale transformation on the normalized hyperspectral image to be defogged to obtain the hyperspectral image to be defogged after grayscale transformation.

[0197] In an optional embodiment of the present application, the device further includes:

[0198] The channel removal module is used to remove the image of a preset channel in the hyperspectral image to be defogged after the grayscale transformation, wherein the image of the preset channel has no image information.

[0199] In an optional embodiment of the present application, the device further includes:

[0200] A correction coefficient setting module, used for selecting a gamma correction coefficient according to the defogged hyperspectral image;

[0201] The gamma correction module is used to perform an exponential operation on the pixel value corresponding to each pixel point in the defogged hyperspectral image according to the gamma correction coefficient to obtain a gamma-corrected defogged image.

[0202] In an optional embodiment of the present application, the fog abundance calculation module 505 is further used to:

[0203] The fog abundance of each channel is calculated according to the simplified fog model according to the pixel values corresponding to the dense fog pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixels of the fog intensity map, and the pixel values corresponding to the fog-free pixels of the fog intensity map.

[0204] In an embodiment of the present application, a hyperspectral image to be defogged is obtained, and the hyperspectral image includes images of several channels; the average value of the pixel value of each pixel in the hyperspectral image to be defogged in all channels of the infrared band and the average value of the pixel value of each pixel in all channels of the visible light band are calculated respectively to obtain an infrared average pixel image and a visible light average pixel image; the infrared average pixel image and the visible light average pixel image are used to calculate a fog intensity map; a fog feature map is obtained according to the first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map; the fog abundance of each channel is calculated according to the pixel value corresponding to each channel of the dense fog pixel point of the hyperspectral image to be defogged, the pixel value corresponding to each channel of the fog-free pixel point of the hyperspectral image to be defogged, the pixel value corresponding to each channel of the fog-free pixel point of the hyperspectral image to be defogged, the pixel value corresponding to the dense fog pixel point of the fog intensity map, and the pixel value corresponding to the fog-free pixel point of the fog intensity map, and the dense fog pixel point and the fog-free pixel point are obtained according to the fog feature map; according to the fog intensity map and the fog abundance, the hyperspectral image to be defogged is defogged using a fog model to obtain a defogged hyperspectral image. Compared with the traditional deep learning or machine learning defogging method, the embodiment of the present application does not need to rely on the training of deep learning models for hyperspectral images with and without fog. Instead, it can achieve efficient hyperspectral image defogging by performing matrix operations on the hyperspectral images to be defogged. The embodiment of the present application not only greatly reduces the amount of calculation for hyperspectral image defogging and improves the defogging processing speed, but also has good defogging performance in different scenes and different fog concentration conditions, and can restore the detailed information of hyperspectral images, ensuring that clear and fog-free hyperspectral images can be obtained even in high-speed photography environments.

[0205] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0206] The present application embodiment provides one or more machine-readable media on which instructions are stored, and when executed by one or more processors, an electronic device executes one or more of the methods described in the above embodiments. In the present application embodiment, the electronic device includes various types of devices such as terminal devices and servers (clusters).

[0207] The embodiments of the present disclosure may be implemented as a device configured as desired using any appropriate hardware, firmware, software, or any combination thereof, and the device may include electronic devices such as terminal devices and servers (clusters). Figure 6 An exemplary apparatus 600 that may be used to implement various embodiments described in this application is schematically illustrated.

[0208] For one embodiment, Figure 6An exemplary apparatus 600 is shown having one or more processors 602, a control module (chip set) 604 coupled to at least one of the processor(s) 602, a memory 606 coupled to the control module 604, a non-volatile memory (NVM) / storage device 608 coupled to the control module 604, one or more input / output devices 610 coupled to the control module 604, and a network interface 612 coupled to the control module 604.

[0209] The processor 602 may include one or more single-core or multi-core processors, and the processor 602 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 600 can be used as a terminal device, server (cluster), etc. described in the embodiments of the present application.

[0210] In some embodiments, the apparatus 600 may include one or more computer-readable media (e.g., memory 606 or NVM / storage device 608) having instructions 614 and one or more processors 602 combined with the one or more computer-readable media and configured to execute the instructions 614 to implement a module to perform the actions described in the present disclosure.

[0211] For one embodiment, control module 604 may include any suitable interface controller to provide any suitable interface to at least one of processor(s) 602 and / or any suitable device or component in communication with control module 604 .

[0212] The control module 604 may include a memory controller module to provide an interface to the memory 606. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0213] The memory 606 may be used, for example, to load and store data and / or instructions 614 for the device 600. For one embodiment, the memory 606 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 606 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0214] For one embodiment, control module 604 may include one or more input / output controllers to provide an interface to NVM / storage device 608 and input / output device(s) 610 .

[0215] For example, NVM / storage 608 may be used to store data and / or instructions 614. NVM / storage 608 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0216] NVM / storage device 608 may include storage resources that are physically part of the device on which apparatus 600 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 608 may be accessed via input / output device(s) 610 over a network.

[0217] (One or more) input / output devices 610 may provide an interface for the apparatus 600 to communicate with any other appropriate device, and the input / output device 610 may include a communication component, an audio component, a sensor component, etc. The network interface 612 may provide an interface for the apparatus 600 to communicate through one or more networks, and the apparatus 600 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0218] For one embodiment, at least one of the processor(s) 602 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 604. For one embodiment, at least one of the processor(s) 602 may be packaged together with the logic of one or more controllers of the control module 604 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 602 may be integrated on the same die with the logic of one or more controllers of the control module 604. For one embodiment, at least one of the processor(s) 602 may be integrated on the same die with the logic of one or more controllers of the control module 604 to form a system-on-chip (SoC).

[0219] In various embodiments, the device 600 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the device 600 may have more or fewer components and / or different architectures. For example, in some embodiments, the device 600 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0220] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information, etc. are stored in a memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0221] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0222] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0223] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable hyperspectral image defogging terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable hyperspectral image defogging terminal device generate instructions for implementing the processes in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0224] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable hyperspectral image defogging terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0225] These computer program instructions can also be loaded onto a computer or other programmable hyperspectral image defogging terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0226] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0227] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0228] The above is a detailed introduction to a hyperspectral image defogging method and device, an electronic device and a storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A hyperspectral image defogging method, characterized in that: The method comprises: Acquire a hyperspectral image to be defogged, wherein the hyperspectral image includes images of several channels; Calculate the average pixel value of each pixel in all channels of the infrared band and the average pixel value of each pixel in all channels of the visible light band in the hyperspectral image to be defogged, and obtain an infrared average pixel image and a visible light average pixel image; Calculate a fog intensity map using the infrared average pixel image and the visible light average pixel image; Obtaining a fog feature map according to a first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map; The fog abundance of each channel is calculated according to the pixel values corresponding to the dense fog pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixel points of the fog intensity map, and the pixel values corresponding to the fog-free pixel points of the fog intensity map, wherein the dense fog pixel points and the fog-free pixel points are obtained according to the fog feature map; According to the fog intensity map and the fog abundance, the hyperspectral image to be defogged is defogged using a fog model to obtain a defogged hyperspectral image.

2. The method according to claim 1, characterized in that The step of obtaining a fog feature map according to the first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map includes: Extracting a second color space image of the hyperspectral image to be defogged; Converting the second color space image into the first color space image, wherein the first color space image at least includes hue component data and lightness component data; The fog characteristic map is obtained by performing calculations using the hue component data, the brightness component data and the fog intensity map.

3. The method according to claim 2, characterized in that The dense fog pixel points and the fog-free pixel points are obtained according to the fog feature map, including: Selecting a pixel point with the highest pixel value from the fog feature map as the dense fog pixel point; A pixel point with the lowest pixel value is selected from the fog feature map as the fog-free pixel point.

4. The method according to claim 1, characterized in that: The step of defogging the hyperspectral image to be defogged using a fog model according to the fog intensity map and the fog abundance includes: Performing data expansion on the fog abundance of each channel to obtain an expanded fog abundance, wherein the size of the expanded fog abundance matches the size of the hyperspectral image to be defogged; The hyperspectral image to be defogged is defogged using a fog model according to the fog intensity map and the expanded fog abundance.

5. The method according to claim 1, characterized in that The step of calculating a fog intensity map using the infrared average pixel image and the visible light average pixel image comprises: The visible light average pixel image is subtracted from the infrared average pixel image to obtain the fog intensity map.

6. The method according to claim 5, characterized in that After obtaining the fog intensity map by subtracting the visible light average pixel image from the infrared average pixel image, the method further includes: Traversing the fog intensity values corresponding to all pixels in the fog intensity map, and filtering out invalid pixels whose fog intensity values are less than zero; The fog intensity value corresponding to the invalid pixel point is set to a preset fog intensity value.

7. The method according to claim 1, characterized in that After acquiring the hyperspectral image to be defogged, the method further includes: Converting the hyperspectral image to be defogged into a one-dimensional hyperspectral image; According to a preset mapping method, the pixel value range of the pixel points in the one-dimensional hyperspectral image is mapped to a normalized interval; The one-dimensional hyperspectral image is dimensionalized to obtain a normalized hyperspectral image to be defogged.

8. The method according to claim 7, characterized in that After obtaining the normalized hyperspectral image to be defogged, the method further includes: The normalized hyperspectral image to be defogged is subjected to an adaptive grayscale transformation to obtain a grayscale transformed hyperspectral image to be defogged.

9. The method according to claim 8, characterized in that After obtaining the hyperspectral image to be defogged after grayscale transformation, the method further includes: The image of a preset channel in the hyperspectral image to be defogged after the grayscale transformation is removed, and the image of the preset channel does not have image information.

10. The method according to claim 1, characterized in that After obtaining the defogging hyperspectral image, the method further includes: Selecting a gamma correction coefficient according to the defogged hyperspectral image; According to the gamma correction coefficient, an exponential operation is performed on the pixel value corresponding to each pixel point in the defogged hyperspectral image to obtain a gamma-corrected defogged image.

11. The method according to claim 1, characterized in that: The calculating of fog abundance of each channel according to the pixel values corresponding to the dense fog pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixel points of the fog intensity map, and the pixel values corresponding to the fog-free pixel points of the fog intensity map, includes: The fog abundance of each channel is calculated according to the simplified fog model according to the pixel values corresponding to the dense fog pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixels of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixels of the fog intensity map, and the pixel values corresponding to the fog-free pixels of the fog intensity map.

12. A hyperspectral image defogging device, characterized in that: The device comprises: An image acquisition module, used for acquiring a hyperspectral image to be defogged, wherein the hyperspectral image includes images of several channels; The average pixel image calculation module is used to calculate the average pixel value of each pixel in all channels of the infrared band and the average pixel value of each pixel in all channels of the visible light band in the hyperspectral image to be defogged, so as to obtain an infrared average pixel image and a visible light average pixel image; A fog intensity map calculation module, used to calculate a fog intensity map using the infrared average pixel image and the visible light average pixel image; A fog feature map calculation module, used for obtaining a fog feature map according to a first color space image corresponding to the hyperspectral image to be defogged and the fog intensity map; A fog abundance calculation module, for calculating the fog abundance of each channel according to the pixel values corresponding to the dense fog pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the fog-free pixel points of the hyperspectral image to be defogged in each channel, the pixel values corresponding to the dense fog pixel points of the fog intensity map, and the pixel values corresponding to the fog-free pixel points of the fog intensity map, wherein the dense fog pixel points and the fog-free pixel points are obtained according to the fog feature map; The defogging module is used to defog the hyperspectral image to be defogged using a fog model according to the fog intensity map and the fog abundance, so as to obtain a defogged hyperspectral image.

13. An electronic device, characterized in that: include: processor; and A memory having executable codes stored thereon, which, when executed, enable the processor to execute the hyperspectral image defogging method as described in one or more of claims 1-11.

14. One or more machine-readable media having executable codes stored thereon, which, when executed, cause a processor to execute the hyperspectral image defogging method as claimed in one or more of claims 1-11.