Fast Detection Method, Device, Equipment and Medium for Morning and Evening Fog Based on Terrain Constraint and Deep Learning

Abstract

The present invention discloses a method, device, equipment and medium for rapid detection of morning and evening fog based on terrain constraint and deep learning. The method includes: Step 1, obtaining DEM data of the research area and H8 / AHI data at morning and evening times; Step 2, preprocessing the data obtained in Step 1, which includes: performing band cropping and removing useless bands on the H8 / AHI data, and then combining the remaining band data and DEM data by channel to obtain fused AHI-DEM data; Step 3, inputting the AHI-DEM data into a pre-trained deep learning-based morning and evening fog detection model, and outputting the morning and evening fog detection result of the research area. The present invention can accurately, efficiently and rapidly realize the detection of morning and evening land fog.

Description

Technical Field

[0001] The present invention relates to the field of environmental monitoring and meteorological forecasting, and particularly to a method, device, equipment and medium for quickly detecting morning and evening fog based on terrain constraint and deep learning. Background Art

[0002] Fog, namely tiny water droplets suspended in the air, is likely to occur at morning and evening times, which will have an important impact on visibility and seriously threaten traffic safety; air pollutants are also easily dissolved in fog droplets, affecting people's respiratory health; in addition, fog droplets adhering to the surface of transmission lines will form a conductive layer, triggering the "pollution flashover" phenomenon and threatening power safety. Precise fog detection means are an important prerequisite for reducing fog losses and impacts.

[0003] Traditional land fog detection mainly relies on the data of meteorological observation stations. Limited by the spatio-temporal resolution, it cannot form continuous regional detection results and has low spatio-temporal representativeness. Remote sensing images have the characteristics of spatial continuity and high temporal resolution, which can effectively make up for the defects of station detection. The current remote sensing fog detection algorithms are mainly developed for daytime and nighttime. Daytime fog detection mainly relies on spectral information. At this time, there is sufficient light on the remote sensing image, which is easy for fog recognition and extraction; in nighttime fog detection, the difference in brightness temperature between the 3.7μm and 11μm bands is currently widely used. This method is simple and efficient, but it is only applicable to nighttime fog detection and has poor distinguishability between fog and low clouds. At morning and evening times, due to uneven illumination, the spectral information of fog is relatively complex, and it is extremely easy to mutate with time under the influence of illumination. Therefore, the fog detection means developed based on daytime and nighttime cannot be directly applied to morning and evening fog detection.

[0004] At the same time, the occurrence of fog is also affected by altitude and terrain. Dense fog accumulates near the ground, and the base height of the fog is low, with the characteristic of being grounded, which makes it easy to be affected by terrain undulations. Combining DEM with traditional fog detection indicators such as R0.86 / R0.64, R0.86 / R1.6 and BT11 - BT3.9 improves the accuracy of cloud, haze and clear sky recognition, but due to the large differences in scale and spectral texture features between altitude and terrain, their combination has a certain degree of difficulty. Summary of the Invention

[0005] The purpose of the present invention is to provide a method, device, equipment and medium for quickly detecting morning and evening fog based on terrain constraint and deep learning, which can accurately, efficiently and quickly realize morning and evening land fog detection in view of the deficiencies of the prior art.

[0006] To achieve the above technical purpose, the present invention adopts the following technical solutions:

[0007] A method for quickly detecting morning and evening fog based on terrain constraint and deep learning includes:

[0008] Step 1: Obtain the DEM data of the study area and the H8 / AHI data at dawn and dusk

[0009] Step 2: Preprocess the data obtained in Step 1, including: performing band cropping and removing useless bands on the H8 / AHI data, and then combining the remaining band data and the DEM data by channel to obtain the fused AHI-DEM data

[0010] Step 3: Input the AHI-DEM data into a pre-trained deep learning-based dawn and dusk fog detection model, and output the dawn and dusk fog detection results of the study area

[0011] Furthermore, the dawn and dusk fog detection model is based on the U-Net network, and in the first 3 encoding layers, an SE-Net attention module is introduced between the convolutional layer and the pooling layer

[0012] Furthermore, the H8 / AHI data obtained in Step 1 includes 16 bands. After removing the useless bands, 7 bands with numbers 3, 5, 6, 7, 11, 13, and 14 are retained, and the corresponding wavelengths are: 0.64μm, 1.60μm, 2.30μm, 3.90μm, 8.60μm, 10.4μm, 11.2μm

[0013] Furthermore, for the pre-trained deep learning-based dawn and dusk fog detection model, the method for making training samples is as follows

[0014] Based on the different lighting conditions on the east and west sides of the study area at the moment of the passing of the twilight line, the study area is divided into a daytime sub-region, a nighttime sub-region, and a dawn and dusk sub-region

[0015] Based on the texture spectral characteristics, perform fog discrimination annotation on the pixels in the daytime sub-region. Based on the bright temperature difference characteristics of the dual channels of numbers 7 and 11, perform fog discrimination annotation on the pixels in the nighttime sub-region. Based on the motion characteristics, perform fog discrimination annotation on the pixels in the dawn and dusk sub-region to obtain the fog mask data of the study area. The AHI-DEM data of the study area and the fog mask data form a training sample

[0016] Furthermore, after performing fog discrimination annotation on the pixels based on the texture spectral characteristics, bright temperature difference characteristics, and motion characteristics, further correct the annotation of the pixels according to the visibility and relative air temperature in the ground observation data

[0017] Furthermore, the dawn and dusk moment refers to the moment when the solar altitude angle is in the range of [0°, 25°]

[0018] Furthermore, when training the dawn / dusk fog detection model using training samples, the Adam gradient descent algorithm and batch normalization training are introduced to adjust the training strategy.

[0019] A fast dawn / dusk fog detection device based on terrain constraint and deep learning, comprising:

[0020] An original data acquisition module, configured to: acquire DEM data of the study area and H8 / AHI data at dawn / dusk;

[0021] A data preprocessing module, configured to: preprocess the acquired DEM data and H8 / AHI data at dawn / dusk, including: performing band clipping and useless band removal on the H8 / AHI data, and then combining the remaining band data and DEM data by channel to obtain fused AHI-DEM data;

[0022] A dawn / dusk fog detection model, based on a deep learning network, configured to: intelligently obtain the dawn / dusk fog detection result of the study area according to the preprocessed AHI-DEM data.

[0023] An electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor implements the fast dawn / dusk fog detection method described in any one of the above.

[0024] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the fast dawn / dusk fog detection method described in any one of the above is implemented.

[0025] Beneficial effects

[0026] The dawn / dusk fog detection model of the present invention based on terrain constraint and enhanced channel domain attention mechanism quickly detects the dawn / dusk fog in the study area. Compared with the prior art, the beneficial effects are as follows:

[0027] It can detect large areas and wide ranges of fog; the detection result of the fog area edge is more refined, the fog area boundary is clearer, it can effectively solve the problem of misdetection of low clouds / mid-level clouds in the fog dissipation stage, near the dawn / dusk line and the daytime area far from the dawn / dusk line, has high reliability and good generalization, the detection result is relatively stable, is not prone to misdetection, and very well solves the difficult problem that it is difficult to separate fog from the ground especially near the dawn / dusk line during the dawn / dusk period; it is fast in detecting dawn / dusk fog, simple and efficient, with less human intervention, can obtain the detection result near real-time, better meets the requirements of real-time detection of large-scale land dawn / dusk fog, and has good application prospects. Description of the drawings

[0028] Figure 1 It is a flowchart of a fast dawn / dusk fog detection algorithm based on terrain constraint and deep learning;

[0029] Figure 2 It is the flowchart for the production of the AHI-DEM-Fm dataset;

[0030] Figure 3 It is the structural diagram of the DDFDC-Net model; Note: The features extracted by the SE-Net are highlighted in orange blocks, and BN and Relu are located after the convolutional layers of the entire DDFDC-Net architecture. Specific implementation manners

[0031] The embodiments of the present invention will be described in detail below. Based on the technical solutions of the present invention, detailed implementation manners and specific operation processes are given, and the technical solutions of the present invention are further explained and illustrated.

[0032] This embodiment provides a fast detection method for morning and evening fog based on terrain constraint and deep learning. Referring to Figure 1 as shown, it includes the following steps:

[0033] Step 1: Obtain DEM data of the study area and H8 / AHI data at morning and evening times;

[0034] The H8 / AHI data is remote sensing image data obtained by the AHI sensor (Advanced Himawari Imager) carried by the geostationary meteorological satellite H8 (Himawari-8), and is abbreviated as H8 / AHI data.

[0035] Definition of morning and evening times: According to the solar altitude angle (the angle between the incident direction of sunlight and the ground plane, complementary to the solar zenith angle, the higher the solar altitude angle, the stronger the downward radiation of the sun, and the solar altitude angle is the highest at noon), screen the remote sensing images containing morning and evening times in the study area. The solar zenith angle can be directly obtained in the AHI data, and the solar altitude angle at morning and evening times is defined as [0°, 25°], that is, the solar zenith angle [65°, 90°].

[0036] The DEM (Digital Elevation Model data) is the SRTM1-DEM data released by the United States Geological Survey.

[0037] Step 2: Preprocess the data obtained in Step 1, including: performing band cropping and removing useless bands on the H8 / AHI data, and then combining the remaining band data and DEM data by channel to obtain the fused AHI-DEM data;

[0038] Preprocess the H8 / AHI data, DEM data, and ground station data used subsequently in this embodiment:

[0039] (1) The preprocessing steps of H8 / AHI data include:

[0040] Step A1: Radiometric calibration. Convert the DN values in the original H8 / AHI images into reflectance values or brightness temperatures. There are 16 bands in the original AHI data, among which bands 1 - 6 are calibrated to reflectance and bands 7 - 16 are calibrated to brightness temperature;

[0041] Step A2: Projection conversion. Uniformly convert the calibrated AHI data to the WGS - 84 coordinate system, and at the same time resample the data spatial resolution to 2 km;

[0042] Step A3: Band cropping and land - sea mask cropping. There are 16 bands in the original AHI data, and band cropping is required to eliminate useless bands. The 7 retained bands are 3 (0.64μm), 5 (1.60μm), 6 (2.30μm), 7 (3.90μm), 11 (8.60μm), 13 (10.4μm), 14 (11.2μm);

[0043] In this embodiment, the width of the AHI data in the study area is large and does not include some ocean areas. Use the Subset Data tool in ENVI5.3 to perform mask processing on the AHI data with the help of the standard SHP file of the Chinese land boundary, and assign pixel values of 0 to the ocean part.

[0044] (2) DEM data

[0045] DEM (Digital Elevation Model data) is the SRTM1 - DEM data released by the United States Geological Survey. The original spatial resolution of the data is 30 m. To be consistent with the AHI data, project it to the WGS - 84 coordinate system and resample it to 2 km.

[0046] (3) Ground observation data

[0047] Ground observation data refers to the all - element Micaps - diamond1 mapping data recorded by ground meteorological stations. Ground stations with "visibility" less than or equal to 1 km and "present weather phenomenon" coded as 40 - 49 are marked as foggy stations; among them: among foggy stations, stations with visibility between 50 - 200 m are marked as thick fog stations, stations with visibility between 200 - 500 m are marked as medium fog stations, stations with visibility between 500 - 1000 m are recorded as light fog stations, and stations with visibility greater than 1 km are non - fog stations; in addition, stations with "visibility" of 1 - 5 km and "relative air humidity greater than 90%" are marked as light fog stations.

[0048] (4) DEM and AHI channel fusion

[0049] Taking the DEM as a channel, it is fused with the cropped AHI data, and the data dimension changes from the original 7 channels to 8 channels, and finally the AHI-DEM data including the fusion at dawn and dusk is constructed.

[0050] Step 3: Input the AHI-DEM data into the pre-trained deep learning-based dawn and dusk fog detection model, and the dawn and dusk fog detection results of the study area are output.

[0051] Before using the pre-trained deep learning-based dawn and dusk fog detection model, it needs to be obtained by pre-training the deep learning network with training samples. The production of the training sample dataset, the construction and training of the model, etc. are introduced below.

[0052] (1) Construct the dawn and dusk dataset AHI-DEM and make the fog mask dataset Fm. Refer to Figure 2 as shown.

[0053] There is no H8 / AHI fog mask dataset in the currently publicly available remote sensing training datasets. At the same time, to refine the recognition accuracy of the model at the fog area edge, the DEM is added to the AHI data. Based on the spectral, texture, and motion characteristics of fog, the surface, and clouds using ground observation site data, AHI data, and DEM data, fog pixels are labeled pixel by pixel in the original image to establish the AHI-DEM-Fm dataset for training and verification.

[0054] Step B1: Construct the dataset AHI-DEM including dawn and dusk. Screen the remote sensing images of the study area including dawn and dusk according to the solar altitude angle (the angle between the incident direction of sunlight and the ground plane, complementary to the solar zenith angle. The higher the solar altitude angle, the stronger the downward radiation of the sun, and the solar altitude angle is the highest at noon). The solar zenith angle can be directly obtained in the AHI data. The solar altitude angle at dawn and dusk is defined as (0°, 25°), that is, the solar zenith angle (65°, 90°).

[0055] Step B2: perform band clipping and land and sea mask clipping. There are 16 bands in the original AHI data. Extra bands will slow down the model training speed. Therefore, band clipping is required to remove useless bands. The 7 bands retained are 3 (0.64μm), 5 (1.60μm), 6 (2.30μm), 7 (3.90μm), 11 (8.60μm), 13 (10.4μm), and 14 (11.2μm). The AHI data has a large width and will include part of the ocean area in one image data. The Subset Data tool in ENVI5.3 is used to mask the AHI data with the help of the standard Chinese land boundary SHP file, and the pixel value of the ocean part is assigned to 0. The DEM is used as a channel and merged with the clipped AHI data. The data dimension is changed from the original 7 channels to 8 channels, and finally the AHI-DEM containing the dawn and dusk dataset is constructed.

[0056] Step B3: Create a fog mask dataset Fm. Using the ROI tool in ENVI5.3, mark the fog pixels on the AHI-DEM dataset at dawn and dusk, create a fog mask dataset Fm, and store it as a binary single-channel tif format image.

[0057] Among them, affected by the rotation of the earth, due to the large image width of the study area (104.5°~136°E), the lighting conditions on the east and west sides of the twilight line are different when the twilight line passes. In the morning, the east side of the twilight line is in the daytime, and the west side is in the night; at dusk, the west side of the twilight line is in the daytime, and the east side is in the night. According to different lighting conditions, the study area at dawn and dusk is divided into three types: daytime area, nighttime area and dawn and dusk area. Affected by sunlight, various types of objects in the daytime area contain emission and reflection features, while at night they only contain emission features. With the help of TSD (Texture and Spectral Difference), BTD (Bright Temperature Difference), and VI (Visual Interpretation) methods, the pixels are identified as fog at three different times: daytime area, nighttime area, and dawn and dusk area.

[0058] TSD features are the texture spectrum differences between fog, clouds and ground surface during the day. The fog is close to the ground, with a smooth top, small fluctuations, clear and neat boundaries, and uniform texture and neat and continuous edges in the image. The cloud types are complex, with violent fluctuations on the top, and the image shows rough texture and broken boundaries. The area is well illuminated during the day, and the reflectivity of clouds and fog is significantly higher than that of the ground surface. The brightness contrast between clouds and fog and the ground surface in the image is obvious. The low reflectivity of the ground surface and the difference in texture between clouds and fog can be used as the basis for distinguishing fog in the daytime area in the images at dawn and dusk.

[0059] BTD feature is the brightness temperature difference between channels 7 and 11 It is mainly applied to night fog discrimination. The difficulty of night fog discrimination lies in separating the ground surface from the fog. Since the nighttime reflectance band is not available, and because the fog is close to the ground surface, there is almost no difference between the fog and the ground surface in the nighttime brightness temperature band. For night fog, its emissivity in the 3.7μm channel is ~0.8, while its emissivity in the 11μm channel is ~1. Therefore, at night, the difference between the two channels of the fog is ~(-4)K, while for the ground surface, there is no obvious difference in the signal values of the two channels at night, and the difference between the two channels is ~0;

[0060] The VI feature is the motion feature of the fog. In the area where the twilight line passes, it is difficult to separate the fog. However, compared with clouds and the ground surface, the fog has unique motion features; over time, the ground surface is stationary, while the clouds mainly show drifting motion, and the motion speed of the clouds is significantly higher than that of the fog. The motion feature of the fog in the time-series remote sensing images is mainly manifested as the expansion or contraction of the area, and the center and internal structure of the fog area remain stable within a certain period of time. By analyzing the motion features of the ground objects in the adjacent 30-minute remote sensing images, the fog area can be effectively discriminated, which is mainly used to discriminate the fog with the same surface spectral information at dawn and dusk.

[0061] Step B4: Calibrate the fog mask dataset Fm by combining ground observation data. Since there will be deviations in the fog areas manually labeled, it is necessary to borrow ground observation data for calibration, eliminate the areas where the label indicates fog but there is no fog at the corresponding ground station, and ensure the accuracy of the fog mask dataset Fm.

[0062] Step B5: Establish the AHI-DEM-Fm dataset. To improve the training efficiency of the model and the learning effect of the edge information, it is necessary to crop the AHI-DEM dataset at dawn and dusk and the fog mask dataset Fm. The cropping method is 30% overlapping cropping (adjacent pictures have an overlapping degree of 30%). The original AHI-DEM is cropped into an 8-channel 256*256 pixel tif format image, and the mask data Fm is cropped into a single-channel 256*256 pixel tif format image to form the AHI-DEM-Fm dataset.

[0063] (2) Establish the DDFDC-Net model, that is: based on the U-net network, and in the first 3 encoding layers, introduce the SE-Net attention module between the convolutional layer and the pooling layer to eliminate the influence of different solar elevation angles in the DEM data and the same remote sensing image on the spectral features of the pixels, and improve the generalization performance of the network. Refer to Figure 3 As shown, specifically:

[0064] Step C1: On the left side of the DDFDC-Net network is the encoding stage (referred to as the encoder), which consists of 5 groups of convolutional layers (CL), 4 groups of pooling layers (MPL), and 3 groups of SE-Net modules (placed between the first three convolutional layers and pooling layers). The convolutional layers extract the feature information of morning and evening fog in the AHI-DEM dataset containing morning and evening hours using a 3*3 convolutional kernel. After each convolution, a batch normalization training (BN) operation is performed, followed by non-linear activation using the Relu function. Then, the SE-Net module is used for channel attention learning to learn the weights of each feature channel. The weights are multiplied with the corresponding channels to obtain feature channels with increased channel attention weights, and then it enters the pooling layer for learning.

[0065] Step C2: The pooling layer selects the maximum value of 4 pixels in the window as the current pixel after pooling through a 2*2 pooling window, thereby reducing the dimension of the feature information and completing the downsampling of the feature map.

[0066] Step C3: On the right side of the network is the decoding stage (referred to as the decoder) composed of 4 groups of convolutional layers and 4 groups of upsampling layers, which is used to gradually restore the size of the feature map. The feature maps obtained in each decoding stage are fused with the corresponding feature maps in the encoding stage using skip connections to establish the connection of context feature information, compensate for the information lost in the downsampling pooling layer, and complete the model's spatial context language.

[0067] (3) Train the DDFDC-Net model, that is, train to obtain a rapid morning fog detection model.

[0068] In this embodiment, the Adam gradient descent algorithm and batch normalization training (BN) are introduced to adjust the training strategy, accelerate the convergence speed of model training, and avoid overfitting, thereby obtaining the optimal network model.

[0069] Step D1: Randomly divide the AHI-DEM-Fm dataset into a training set and a validation set at a ratio of 7:3 (training set: 2184 images, validation set: 936 images) as input data to train and validate the DDFDC-Net model.

[0070] Step D2: The configuration parameters of the training platform are: Graphics Processing Unit (GPU): NVIDIA GeForce RTX 2080Ti, programming language: Python 3.6.6, framework: PyTorch 1.5.1, computing platform (CUDA): 10.0; based on this, the model parameters are designed.

[0071] Step D2.1 Set the model parameter Batch size = 8, that is, 8 images are used in each training batch (the number is related to the performance of the computer and GPU).

[0072] Step D2.2 Set the model parameter Epoch = 1500, which is the maximum number of epochs for model iteration (the model terminates iteration after reaching this number of epochs);

[0073] Step D2.3 Set the model parameter α initial = 0.0035, which is the initial learning rate. Here: The magnitude of the learning rate affects the speed of parameter update. If the learning rate is too large, the model may not converge finally; if the learning rate is too small, the convergence speed of the model will be too slow and the training time will increase significantly. The parameter update speed is fast at the beginning of model training and slow when the model converges. Therefore, during the training process, the model learning rate needs to be updated dynamically to ensure that the learning rate is relatively large at the beginning of model training and relatively small when the model converges;

[0074] Step D2.4 Set the model parameter Decaying Epoch = 1440, which is the epoch when the learning rate starts to be adjusted. After this number of epochs, the dynamically adjusted learning rate α is (according to the following formula):

[0075]

[0076] Step D3: Output the preliminary prediction result. The last convolutional layer uses the sigmoid function to output the target probability of each pixel predicted by the model. Set the threshold 0.5 to complete the classification of target pixels / non-target pixels. Pixels with a value lower than the threshold 0.5 are set as target pixels, and the target mask classification result map predicted by the model is obtained to complete the initial classification of pixels.

[0077] Step D4: Compare the prediction result with the ground truth and calculate the deviation between the initial classification and the real fog mask. Calculate the gradient of each parameter using the error backpropagation algorithm according to the deviation, and dynamically adjust and obtain a new set of parameters according to the gradient.

[0078] As a preferred solution, the optimizer for model parameter update selects the Adam optimizer, and batch normalization training (BN) is added between each pooling layer (CL) and non-linear activation layer (Relu) to improve the training speed and shorten the training time;

[0079] To prevent model overfitting, only the optimal epoch model parameters are retained during the training process: If the training accuracy IOU of the current epoch is higher than that of the previous epoch, the model parameters of the current epoch are adopted. At the same time, to test the model effect and obtain the optimal model, the training process interval is set to 25 epochs, and the validation accuracy is obtained once every 25 epochs using the validation set. The validation accuracy evaluation metrics are defined as: intersection over union IOU, accuracy Accurary, and harmonic mean F1-Score, which are defined as follows:

[0080]

[0081]

[0082] Among them: Ground Truth represents the total number of foggy pixels in the image predicted by the model, Ground Truth represents the total number of foggy pixels in the corresponding label of the image; Correctly classified pixels are the number of correctly classified pixels, and Total pixels are the total number of classified pixels; Precision represents the proportion of pixels predicted to be foggy to all foggy pixels, and Recall represents the proportion of pixels predicted to be foggy to all foggy pixels. The above detection indicators range from 0 to 1. The larger the IOU, the higher the segmentation accuracy.

[0083] Step D5: Repeat the above steps D2-D4 and iterate continuously until the verification accuracy no longer increases significantly, the model training is successful, and the morning and evening fog rapid detection model is obtained.

[0084] The morning and evening fog rapid detection model obtained through the above training can now be used in step 3 to perform morning and evening fog rapid detection on the corresponding study area according to the newly fused AHI-DEM data to obtain fog detection results.

[0085] Next, the accuracy of the DDFDC-Net model detection results of the present invention is verified:

[0086] The ground observation data corresponding to the morning and evening satellite images from the China Meteorological Administration from November 17, 2016 to January 3, 2017 at 8:00 and 17:00 were selected to verify the accuracy of the DDFDC-Net algorithm.

[0087] The detection results at 8:00 on November 18, 2016 show that there is a large area of ​​fog in the central and western parts of the study area, and the algorithm detection results are highly consistent with the ground observation data. From the detection results, the fog area has an increasing and decreasing trend, indicating that DDFDC-Net can effectively capture the development process of fog.

[0088] The detection results at 17:00 on November 18, 2016 show that there is a large area of ​​fog in the central and northeastern parts of the study area. The algorithm detection results are highly consistent with the ground observation data. The satellite detection results show that the area of ​​fog is increasing from 15:30 to 17:00, and the algorithm can effectively capture the evolution of dusk fog. Comparing the satellite image and the algorithm detection results, it is found that the algorithm misdetects the daytime area on the left side of the twilight line at 16:30. Analysis of terrain characteristics shows that the terrain in this area is undulating and the low cloud coverage rate is high. The algorithm misjudged the low cloud as fog. The algorithm missed the nighttime area on the right side of the twilight line at 17:00. In addition, the fog detection results from 15:30 to 17:00 for 90 minutes detected more than 80% of the fog area in the study area.

[0089] Select the detection accuracy of the ground observation data quantitative verification algorithm, and use the common index evaluation system, including the probability of detection (POD), false alarm ratio (FAR), and critical success index (CSI) to evaluate the detection results. The above indexes are defined as follows:

[0090]

[0091]

[0092] where: N X is the number of detections, and its subscript X is the type of detection index, including H, M, and F (H means that the satellite detection result is consistent with the ground observation result, that is, a correct detection; M means that there is no fog in the satellite detection result but the ground data shows fog, that is, a missed detection; F means that there is fog in the satellite detection result but the ground data shows no fog, that is, a false detection). The range of the above detection indexes is 0-1. The larger the POD, the higher the detection accuracy; the larger the CSI, the more effective the method.

[0093] Table 1 shows the fog detection accuracy at 8:00 on randomly selected 5 days from November 18, 2016 to January 3, 2017. The average correct rate of satellite fog detection at 8:00 on the 5 days is 84.0%, the average false alarm rate is 16.4%, and the average critical success index is 72.0%. The correct rate of the algorithm is relatively high. Among them, the false detection rate of satellite fog detection on January 3, 2017 is the highest, and the critical success index is the smallest.

[0094] Table 1 Fog detection accuracy at 8:00 in the morning

[0095]

[0096] Table 2 shows the fog detection accuracy at 17:00 on randomly selected 5 days from November 17, 2016 to January 3, 2017. The average correct rate of the detection results at 17:00 on the 5 days is 83.7%, the average false detection rate is 15.8%, and the average critical success index is 72.6%. The algorithm has relatively high accuracy and reliability at both dawn and dusk.

[0097] Table 2 Fog detection accuracy at 17:00 in the evening

[0098]

[0099] In order to further verify the seasonal adaptability of the DDFDC-Net model, 8 morning fog cases and 5 dusk fog cases from February to October 2017 were randomly selected, totaling 13 fog cases, combined with nearly synchronous ground observation data to quantitatively verify the fog detection accuracy of the DDFDC-Net model. Among them: there are fewer dusk fog cases from April to July, and the fog area is smaller, resulting in fewer dusk fog cases than morning fog cases in seasonal verification. Seasonal changes lead to changes in fog formation time and the crossing time of the morning and evening line. The experiment selects the fog case with the largest fog area at dawn and dusk and combines it with ground observation data to verify the accuracy of the algorithm. For example, the ground verification data at 08:00 in February and March, and the ground observation data at 05:00 from May to October are selected to quantitatively verify the fog detection accuracy of the model. The accuracy of the detection results is shown in Tables 3 and 4, and it can be seen that:

[0100] (1) The POD, FAR and CSI of the 8 morning fog cases were 66.8%, 45.1% and 43.6%, respectively. The accuracy of the 5 dusk fog cases was 68.0%, 32.3% and 49.8%, respectively. The overall accuracy of the algorithm is good.

[0101] (2) The algorithm shows higher accuracy for seasons and months close to the training data, such as POD of 76.6%, FAR of 33.7%, and CSI of 55.2% for morning fog cases in February, March, and October. However, compared with the accuracy of fog detection results in winter in Table 1, the three indicators drop by nearly 7%, 17%, and 17%, respectively.

[0102] (3) The detection accuracy of the algorithm gradually decreases in the distant seasons and months. For example, the POD, FAR and CSI of the morning fog cases detected from May to September are 60.8%, 52% and 36.7%, respectively. Compared with the winter fog detection accuracy in Table 1, the three indicators decrease by nearly 23%, 36% and 35%, respectively.

[0103] (4) The accuracy of the algorithm detection results decreases as the time from the algorithm training data increases. The main reason is that the DDFDC-Net model is greatly affected by the training data. The fog spectral characteristics change with the seasons, resulting in poor detection results for the seasonal model not covered by the training data. If training data sets are created for each season to train the DDFDC-Net seasonal model, better fog detection results are expected.

[0104] (5) By comparing the algorithm detection results and the original remote sensing images, it was found that the algorithm's detection accuracy in summer and autumn was significantly affected by the cloud cover in the study area. When there was only a single weather phenomenon, fog, in the study area, the algorithm's detection accuracy in summer was extremely high. The main reason was that the spectral textures of the ground surface and fog / clouds were very different in summer. However, since there were more low clouds (liquid water clouds) in summer, they were easily misjudged as fog, resulting in a decrease in detection accuracy. In autumn, there were more clouds in the study area, and the misjudgment rate also increased.

[0105] Table 3 Detection accuracy of morning fog in different seasons

[0106]

[0107]

[0108] Table 4 Detection accuracy of dusk fog in different seasons

[0109]

[0110] Comparative verification of detection accuracy of U-Net and DDFDC-Net

[0111] We randomly selected two morning fog cases at 08:00 on November 18 and December 12, 2016, and two dusk fog cases at 17:00 on November 18 and December 11, 2016, and conducted a qualitative comparative analysis of the detection results of the U-Net model and the DDFDC-Net model. We found that:

[0112] (1) The DDFDC-Net model has more refined fog edge detection results, and the fog boundary is clearer; U-Net misjudges low clouds in the daytime area with high solar altitude in the image, while the DDFDC-Net model does not misjudge;

[0113] (2) The fog is in the dissipation stage. Both DDFDC-Net and U-Net have misjudgments, but the misjudgment range of DDFDC-Net is smaller. The DDFDC-Net model misjudges the surface and medium-high clouds as fog in a smaller area.

[0114] (3) There is a large area of ​​light fog in the eastern and central areas of the study area. This area is located in the night area on the right side of the terminator. The detection effects of the DDFDC-Net and U-Net models are good. Both models have good detection capabilities for large areas of light fog at night.

[0115] (4) Although the ground verification data showed fog, it was contaminated by low and medium clouds above. Both models missed detections in this area, but the U-Net refinement of the edge of the fog area was slightly coarse, and a small part of the low-level clouds were misjudged as fog, which was not misjudged by the DDFDC-Net model;

[0116] (5) Compared with U-net, DDFDC-Net has a certain improvement on the false detection of low clouds over certain areas. However, since a certain area is located in the transition zone between the second and third steps, the terrain features in the area are extremely complex and a large part of the low clouds are still misjudged as fog by both models.

[0117] The above embodiments are the preferred embodiments of the present application. Those of ordinary skill in the art can also make various transformations or improvements based on this. Without departing from the general concept of the present application, these transformations or improvements should all fall within the scope of protection required by the present application.

Claims

1. A fast detection method for morning and evening fog based on terrain constraints and deep learning, characterized in that, it includes: Step 1: Obtain DEM data of the study area and H8 / AHI data at morning and evening times; Step 2: Preprocess the data obtained in Step 1, including: performing band cropping and removing useless bands on the H8 / AHI data, and then combining the remaining band data and DEM data by channel to obtain fused AHI-DEM data; Step 3: Input the AHI-DEM data into a pre-trained deep learning-based morning and evening fog detection model, and output the morning and evening fog detection results of the study area; The morning and evening fog detection model specifically includes: Step C1: The left side of the network is the encoding stage, which consists of 5 groups of convolutional layers, 4 groups of pooling layers and 3 groups of SE-Net modules; the convolutional layers extract the feature information of morning and evening fog in the AHI-DEM dataset containing morning and evening times through a 3*3 convolutional kernel, perform batch normalization training operations after each convolution, then use the Relu function for non-linear activation, and then use the SE-Net module for channel attention learning, learning the weights of each feature channel, multiplying the weights with the corresponding channels to obtain feature channels with increased channel attention weights, and then entering the pooling layer for learning; Step C2: The pooling layer selects the maximum value of 4 pixels in the window as the current pixel after pooling through a 2*2 pooling window, so as to reduce the dimension of the feature information and complete the downsampling of the feature map; Step C3: The right side of the network is a decoding stage composed of 4 groups of convolutional layers and 4 groups of upsampling layers, which is used to gradually restore the size of the feature map; among them, the feature map obtained in each decoding stage is fused with the corresponding feature map in the encoding stage by using skip links to establish the connection of context feature information and compensate for the information lost in the downsampling pooling layer to complete the model space context language.

2. The fast detection method for morning and evening fog according to claim 1, characterized in that, The H8 / AHI data obtained in Step 1 includes 16 bands. After removing the useless bands, 7 bands with numbers 3, 5, 6, 7, 11, 13, and 14 are retained, and the corresponding wavelengths are: 0.64μm, 1.60μm, 2.30μm, 3.90μm, 8.60μm, 10.4μm, 11.2μm.

3. The fast detection method for morning and evening fog according to claim 2, characterized in that, For the pre-trained deep learning-based morning and evening fog detection model, the method for making its training samples is: Based on the different lighting conditions on the east and west sides of the study area at the moment of the passing of the terminator, the study area is divided into a daytime sub-region, a nighttime sub-region and a morning and evening sub-region; Based on the texture spectral characteristics, the pixels in the daytime sub-region are marked for fog discrimination, based on the bright temperature difference characteristics of the dual channels numbered 7 and 11, the pixels in the nighttime sub-region are marked for fog discrimination, and based on the motion characteristics, the pixels in the morning and evening sub-region are marked for fog discrimination to obtain the fog mask data of the study area. The AHI-DEM data of the study area and the fog mask data form a training sample.

4. The fast detection method for morning and evening fog according to claim 3, It is characterized in that after the fog discrimination annotation of pixels is carried out based on texture spectral features, bright temperature difference features and motion features, the annotation of pixels is further corrected according to the visibility and relative air temperature in the ground observation data.

5. The rapid detection method of morning and evening fog according to claim 1, It is characterized in that the morning and evening time refers to the time when the solar altitude angle is in the range of [0°, 25°].

6. The rapid detection method of morning and evening fog according to claim 1, It is characterized in that when using training samples to train the morning and evening fog detection model, the Adam gradient descent algorithm and batch normalization training are introduced to adjust the training strategy.

7. A rapid detection device for morning and evening fog based on terrain constraint and deep learning, It is characterized in that including: An original data acquisition module, used for: acquiring DEM data of the research area and H8 / AHI data at morning and evening times; A data preprocessing module, used for: preprocessing the acquired DEM data and H8 / AHI data at morning and evening times, including: performing band clipping and useless band removal on the H8 / AHI data, and then combining the remaining band data and DEM data by channel to obtain the fused AHI-DEM data; A morning and evening fog detection model, based on a deep learning network, used for: intelligently obtaining the morning and evening fog detection result of the research area according to the preprocessed AHI-DEM data; The morning and evening fog detection model specifically includes: Step C1: The left side of the network is the encoding stage, which consists of 5 groups of convolutional layers, 4 groups of pooling layers and 3 groups of SE-Net modules; the convolutional layers extract the feature information of morning and evening fog in the AHI-DEM dataset containing morning and evening times through 3*3 convolutional kernels, perform batch normalization training operations after each convolution, then use the Relu function for non-linear activation, and then use the SE-Net module for channel attention learning, learning the weights of each feature channel, multiplying the weights with the corresponding channels to obtain feature channels with increased channel attention weights, and then entering the pooling layer for learning; Step C2: The pooling layer selects the maximum value of 4 pixels in the window as the current pixel after pooling through a 2*2 pooling window, so as to reduce the dimension of the feature information and complete the downsampling of the feature map; Step C3: The right side of the network is a decoding stage composed of 4 groups of convolutional layers and 4 groups of upsampling layers, used to gradually restore the size of the feature map; among them, the feature map obtained in each decoding stage is fused with the corresponding encoding stage feature map by using skip links to establish the connection of context feature information, compensate for the information lost in the downsampling pooling layer, and complete the model space context language.

8. An electronic device, including a memory and a processor, and a computer program is stored in the memory, It is characterized in that when the computer program is executed by the processor, the processor realizes the rapid detection method of morning and evening fog as described in any one of claims 1 to 6.

9. A computer-readable storage medium, on which a computer program is stored, It is characterized in that when the computer program is executed by a processor, it realizes the rapid detection method of morning and evening fog as described in any one of claims 1 to 6.

Images