Satellite remote sensing sea fog identification method based on knowledge and data dual drive

Abstract

The invention discloses a satellite remote sensing sea fog identification method based on knowledge and data dual drive, and belongs to the technical field of remote sensing image processing. The method comprises the following steps: integrating a multispectral satellite image, atmosphere reanalysis data and vertical observation data by using a multi-source data collaborative preprocessing technology, and providing comprehensive input for a model; through knowledge data fusion and knowledge feature fusion, physical indexes such as sea air temperature difference and wind speed are converted into features which can be directly learned by the model; a space-time attention mechanism module, a knowledge guidance attention mechanism module and physical constraint loss are introduced into the model. According to the method, the high precision of sea fog identification is ensured, the prediction result is more interpretable due to the constraint of physical rules, and the cloud-fog misjudgment rate is remarkably reduced. In addition, the requirement for the data annotation amount is lower, model convergence is faster, and the method has excellent cross-sea-area generalization ability and robustness. Therefore, more reliable and efficient technical support is provided for sea fog monitoring and forecasting.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image processing, and particularly relates to a satellite remote sensing sea fog identification method based on knowledge and data double driving. BACKGROUND

[0002] Sea fog is a disastrous weather phenomenon that occurs in the lower atmosphere over the sea or coastal areas, with water droplets or ice crystals suspended in the atmospheric boundary layer, resulting in horizontal visibility of less than 1 km, mainly in the form of advection fog. Its fog droplet size distribution is complex (radius size range from less than 1 μm to 50-80 μm), significantly changing the optical properties of the atmosphere, causing visibility deterioration and air quality decline. Due to its high concentration, wide range, and long duration, sea fog poses a serious threat to maritime shipping, fishing production, energy development, and military activities. Research has shown that its social and economic losses are comparable to those of typhoons.

[0003] Sea fog monitoring methods have evolved from manual observation, ground station observation to satellite remote sensing detection. Among them, meteorological satellite remote sensing images have become the core data source for sea fog monitoring due to their high temporal resolution, wide coverage, and multi-spectral observation advantages. At the same time, deep learning technology has applied the ideas of computer vision to the field of remote sensing image interpretation, making breakthrough progress. Especially in the field of cloud and fog recognition, deep learning can significantly improve the accuracy and efficiency of cloud and fog detection, making intelligent recognition based on satellite images the current mainstream method.

[0004] Due to the complex physical processes involved in the formation and dissipation of sea fog, such as sea surface temperature, temperature gradient, wind speed, humidity, etc., and the significant regional and seasonal regularities, current detection methods based on deep learning cannot fully meet the needs of high-precision sea fog identification. Such detection methods mainly have the following limitations: first, as a pure data-driven computer vision solution, the model is difficult to incorporate meteorological physical mechanism knowledge (such as the thermodynamic criterion of advection fog), resulting in a lack of explainability in the decision-making process; second, the multi-spectral channels (FY-4A contains 14 bands), absolute geographic location, and time stamp of remote sensing images are not fully exploited, and the effective correlation between the spectral characteristics of sea fog in remote sensing images and its spatio-temporal distribution regularity cannot be established; third, the scarcity of historical samples of sea fog events (usually only thousands of labeled samples) and the spectral similarity of fog-cloud in the thermal infrared band (such as the overlap of 11 μm and 3.7 μm brightness temperature difference), together resulting in insufficient training data and low labeling confidence, severely restricting the model's generalization ability. These defects are essentially a deep-seated contradiction between the data-driven paradigm and physical mechanisms, and small sample reality. SUMMARY

[0005] The present application aims at the defects of the existing pure data-driven sea fog identification model in cross-region generalization, expression of physical mechanism and robustness under complex weather conditions, and provides a satellite remote sensing sea fog identification method based on knowledge and data double driving, which constructs a sea fog detection method that can be explained to a certain extent, small sample adaptation and cross-region robustness through three-level embedding mechanisms of knowledge data fusion, knowledge feature fusion and knowledge network fusion, so as to improve the identification precision

[0006] The technical scheme adopted by the present application is:

[0007] A satellite remote sensing sea fog identification method based on knowledge and data double driving, comprising the following steps:

[0008] Step 1, determine the target area, and obtain two types of satellite observation data and atmospheric reanalysis data of the target area, wherein the satellite observation data includes multispectral image and cloud-aerosol observation data of geostationary meteorological satellite;

[0009] Step 2, multi-source data collaborative preprocessing is performed on the two types of satellite observation data and atmospheric reanalysis data of the target area, the two types of satellite observation data are integrated into a unified input tensor through space-time matching and resampling; and the cloud-aerosol observation data preprocessing includes sea fog reclassification;

[0010] Step 3, taking the preprocessed multispectral image of the geostationary meteorological satellite and the atmospheric reanalysis data as sample data, constructing the sample label of the sample data based on the reclassified cloud-aerosol observation data, and constructing a training data set;

[0011] Step 4, feature extraction and fusion are performed on the sample data of the training data set, including:

[0012] According to the set time period, the multispectral image of the sample data is gradually time-advanced and pixel-by-pixel synthesized to obtain the channel feature of the multispectral image;

[0013] Extracting the space-time feature in the atmospheric reanalysis data of the sample data; and extracting the multi-item feature enhancement index of the sample data, including cloud top height, sea surface air-sea temperature difference and fog occurrence probability;

[0014] The space-time feature, index feature and channel feature of the multispectral image are spliced to obtain the fusion feature of the sample data;

[0015] Based on a group of pre-defined sea fog formation meteorological factors (meteorological factors strongly related to sea fog formation), the meteorological parameter values of each sea fog formation meteorological factor are extracted from the atmospheric reanalysis data of the sample data to form a meteorological parameter enhancement vector of the sample data;

[0016] Step 5, a sea fog recognition model is constructed, which comprises in sequence: a backbone network, a space-time attention mechanism module (STM module), a knowledge-guided attention module, a fusion module and a sea fog classification layer;

[0017] The input of the backbone network is the fusion feature of the sample data, and the output feature map X of the backbone network is sent into the space-time attention mechanism module and the knowledge-guided attention module respectively;

[0018] The space-time attention mechanism module is used to obtain a channel weight vector and a space attention weight vector of the output feature map X, and based on the two weights, a first attention feature map of the output feature map X is obtained, which is the output feature map of the space-time attention mechanism module;

[0019] The knowledge-guided attention module is used to obtain a knowledge attention weight vector of the meteorological parameter enhancement vector, and based on the space attention weight vector obtained by the space-time attention mechanism module, a second attention map of the output feature map X is obtained, which is the output map of the knowledge-guided attention module;

[0020] The first and second attention feature maps are fused by the fusion module, and the obtained fusion feature map is input into the sea fog classification layer;

[0021] The sea fog classification layer is used to output the predicted sea fog probability of each pixel;

[0022] Step 6, the sea fog recognition model is trained based on the training data set to obtain a sea fog recognition model for the target area;

[0023] The loss function used in the model training is: wherein, is the cross-entropy loss of the sea fog recognition model, is a set balance hyperparameter, is a regularization term related to the sea fog formation meteorological factor, when at least one constraint of the sea fog formation meteorological factor is met, the value of is set to the sea fog probability predicted by the sea fog recognition model, otherwise it is set to 0.

[0024] Further, the sea fog formation meteorological factor includes: sea-air temperature difference, relative humidity, wind speed and wind direction.

[0025] Preferably, is specifically set as:

[0026]

[0027] wherein, represents the sea fog probability predicted by the sea fog recognition model; represents the input sea surface wind speed at a specified height; represents a preset wind speed threshold.

[0028] In the sea fog recognition model constructed in this invention, the backbone network extracts features from the input spatiotemporal metadata vector (i.e., the fusion features of the sample data); and through the spatiotemporal attention mechanism module and the knowledge-guided attention mechanism (KGAM), meteorological knowledge is used as prior information to generate attention weights and is explicitly injected into the network to improve the robustness of the model under complex meteorological conditions; during model training, physical constraint loss (PCL) is introduced. Based on the introduced regularization term, predictions that do not conform to physical laws (such as sea fog occurring under high wind speeds) are penalized, forcing the model to comply with physical rules and improving the credibility of the results.

[0029] Furthermore, in step 2, the multi-source data collaborative preprocessing includes: radiometric calibration, projection transformation, and image format conversion of the multispectral image; image format conversion and sea fog reclassification of the cloud-aerosol observation data; image format conversion of the atmospheric reanalysis dataset, and resampling it to the pixel grid of the multispectral image.

[0030] Furthermore, the spatiotemporal characteristics include: sea surface temperature, air temperature (air temperature referenced to altitude), wind speed (wind speed referenced to altitude), and relative humidity.

[0031] Furthermore, sea surface temperature is the temperature of the sea surface itself or the water at a specified height above the sea surface; air temperature is the air temperature at a height of about 2 meters above sea level; and wind speed is the wind speed at a height of about 10 meters above sea level.

[0032] Furthermore, the channel characteristics of multispectral images include: brightness temperature of visible and near-infrared light, short-wave infrared light, mid-wave infrared light, water vapor, and long-wave infrared light.

[0033] Furthermore, in step 4, the step time is set to 1 hour.

[0034] Furthermore, the reclassification of sea fog is as follows:

[0035] (1) If the height difference between the bottom of the layer and the nominal sea level is less than or equal to If the current layer is classified as sea fog; then, , The first height deviation value is set;

[0036] (2) If the height difference between the surface of the layer and the nominal sea level exceeds the sea level When the surface / subsurface of the layer is above the nominal sea surface elevation 0, it is considered a candidate for sea fog. If the 532 nm attenuated backscatter of the candidate sea fog layer is greater than or equal to the threshold threshold, the current layer is classified as sea fog.

[0037] (3) If the height difference between the bottom of the layer and the nominal sea surface elevation is greater than 0.5 km, then the current layer is classified as non-fog.

[0038] Further, in step 4, the cloud top height, the sea-air temperature difference, and the fog occurrence probability are specifically as follows:

[0039] Cloud top height wherein, the unit is kilometer, is the long-wave infrared channel brightness temperature value, is the clear sky brightness temperature synthesized by the characteristics of each channel of the multi-spectral image, is the temperature lapse rate of the atmosphere;

[0040] Sea-air temperature difference wherein, is the sea surface air temperature, is the sea surface temperature;

[0041] Fog occurrence probability wherein, is the relative humidity, is the natural base, is a preset adjustable coefficient for representing the inhibition strength of wind speed on fog formation.

[0042] Further, the spatio-temporal attention mechanism module includes a position encoder, a channel attention generator, a spatial attention generator, and a feature weighting output layer;

[0043] wherein, the position encoder sequentially includes a fully connected layer , an activation function , and a fully connected layer , for mapping the input data of the sea fog recognition model (i.e., the input spatio-temporal metadata vector) to a high-dimensional position embedding vector , wherein, is the batch size, is the dimension of the high-dimensional position embedding vector , and is the real number field;

[0044] The channel attention generator maps the high-dimensional position embedding vector to a channel weight vector through a fully connected layer and a Sigmoid activation function, wherein represents the Sigmoid activation function, represents the number of channels;

[0045] ​The spatial attention generator generates spatial attention weights by a convolution layer 、 and an activation function wherein , is the output feature map of the backbone network, and H and W are the height and width of the feature map respectively;

[0046] The feature weighting output layer is used to perform element-wise multiplication of the feature map X with the channel weight vector and the spatial attention weight to obtain the first attention feature map wherein , represents an element-wise multiplication operation.

[0047] Further, the knowledge-guided attention module specifically comprises:

[0048] The meteorological parameter enhancement vector Q of the sample data is subjected to feature extraction by a convolution layer and an adaptive average pooling layer to obtain a meteorological knowledge feature vector K: wherein represents a two-dimensional adaptive average pooling layer;

[0049] The attention weight vector of the meteorological knowledge feature vector K is extracted by a fully connected layer: The output feature map of the backbone network is subjected to feature fusion by a feature fusion layer to obtain a second attention feature map . This explicit fusion manner enables the model to dynamically adjust its internal feature representation according to meteorological knowledge, thereby exhibiting stronger robustness under complex meteorological conditions such as strong wind.

[0050] Further, the present application further comprises step 6, model evaluation is performed on the trained sea fog recognition model, and the sea fog recognition model meeting the model evaluation index is used for sea fog recognition of a target area;

[0051] wherein the model evaluation index comprises an average intersection over union , precision , recall and F1 score, and each index specifically comprises:

[0052] The average intersection over union is: ;

[0053] The precision is: ;

[0054] ​Recall rate : ;

[0055] F1 score: ;

[0056] wherein, is the intersection over union of the th class, is the single class intersection over union, (True Positive) is the number of sea fog pixels correctly identified, (False Positive) is the number of non-sea fog pixels misjudged as sea fog, (False Negative) is the number of sea fog pixels missed.

[0057] The technical solution provided by the present application at least brings the following beneficial effects:

[0058] The present application can improve the sea fog recognition accuracy to a certain extent, reduce the mist-cloud misjudgment rate based on physical rule constraints, has certain explainability, guarantees the generalization of cross-sea areas through the normalization mechanism, and reduces the demand of the model on data labeling quantity with the assistance of prior knowledge, so that the model converges faster. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0060] Figure 1 A processing process schematic diagram of a satellite remote sensing sea fog recognition method based on knowledge and data double driving provided by the present application embodiment.

[0061] Figure 2 It is a FY-4A L1 multi-spectral image preprocessing result graph.

[0062] Figure 3 It is a CALIPSO laser radar data preprocessing flowchart.

[0063] Figure 4 It is a knowledge feature fusion feature enhancement tensor schematic diagram. DETAILED DESCRIPTION

[0064] In order to make the purposes, technical solutions, and advantages of the embodiments of the present application clearer, the following will describe the technical solutions in the embodiments of the present application in a detailed and complete manner with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings can be arranged and designed using different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application.

[0065] The embodiments of the present application disclose a satellite remote sensing sea fog identification method based on knowledge and data double driving, which deeply fuses meteorological field knowledge and deep learning model through a three-level embedding mechanism, aiming to solve the core technical problems such as poor cross-regional generalization, insufficient robustness, and low prediction result reliability of the existing pure data driven model. First, the embodiments of the present application use multi-source data collaborative preprocessing technology to integrate multi-spectral satellite images, atmospheric reanalysis data, and vertical observation data, providing comprehensive input for the model. On this basis, through knowledge data fusion and knowledge feature fusion, the sea-air temperature difference, wind speed and other physical indicators are converted into features that can be directly learned by the model, realizing the internalization of knowledge at the data level. Secondly, the embodiments of the present application introduce a spatio-temporal attention mechanism module, a knowledge-guided attention mechanism module and a physical constraint loss at the network architecture level. The spatio-temporal attention mechanism module solves the problem of inconsistent cross-regional data distribution; the knowledge-guided attention mechanism improves the robustness of the model under complex weather conditions; and the physical constraint loss punishes unreasonable predictions to ensure the physical reliability of the results. The method of the embodiments of the present application ensures high precision of sea fog identification, and the prediction results are more interpretable due to the constraint of physical rules, and significantly reduce the cloud-fog misjudgment rate. In addition, the method of the embodiments of the present application has lower demand for data annotation quantity, faster model convergence, and superior cross-sea area generalization ability and robustness. These beneficial effects provide more reliable and efficient technical support for sea fog monitoring and forecasting.

[0066] In one embodiment, the specific implementation process of the satellite remote sensing sea fog identification method based on knowledge and data double driving provided by the embodiments of the present application is as shown in Figure 1 , which includes:

[0067] Step 1: Selection of research area (target area)

[0068] The embodiments of the present application focus on complex sea fog prone areas, and the selected research area of the present embodiment is: coordinates 105°~130°E, 18°~41°N.

[0069] Step 2: Multi-source data collaborative preprocessing

[0070] The input main data of the present application example is FY-4A L1 multi-spectral image (spatial resolution: 0.5-4 km); the input auxiliary data includes ERA5 reanalysis data (sea surface temperature sst, 2m air temperature t2m, 10m wind speed w10m, relative humidity RH) and CALIPSO satellite CALIOP sensor data (VFM, 532nm and 1064nm backscatter coefficient). The data preprocessing mode and process are as follows:

[0071] (1) FY-4A L1 multi-spectral image preprocessing. Download FY-4A L1 data, the downloaded data product is FY4A imager full disc 4km L1 data, which is converted into GeoTIFF image after radiation calibration, projection conversion and definition of coordinate reference system. The image channel example of the research area is as shown in Figure 2 .

[0072] (2) CALIPSO data preprocessing. The spaceborne polarization lidar (Cloud-Aerosol Lidar with Orthogonal Polarization, CALIOP) carried by CALIPSO satellite can transmit laser pulses at 532nm and 1064nm with a repetition frequency of 20Hz, which can penetrate clouds and obtain atmospheric vertical profile information. The present application example uses the 532nm and 1064nm backscatter coefficients of Level 1 product and the vertical feature mask (VFM) of Level 2 product. Among them, the vertical classification of VFM data product includes clean atmosphere, cloud, aerosol, planetary boundary layer, subsurface, etc., but there is no fog classification. In remote sensing observation, the determination of sea fog is whether the cloud layer contacts the sea surface. However, there are two kinds of misjudgment conditions for determining the existence of sea fog based on VFM cloud height data: 1) the fog bottom height of sea fog is low, and the classification algorithm of CALIOP cannot effectively distinguish the sea surface from the sea fog, so the fog near the sea surface is misjudged as the sea surface (surface) or subsurface; 2) when the sea fog top height is relatively close to the low cloud bottom height and the sea fog thickness is large, the algorithm will judge the sea fog connected with low cloud and aerosol as cloud (cloud). Therefore, the present application example proposes the following determination for the identification of low cloud and heavy fog based on CALIPSO satellite data. The specific determination standard network diagram is as shown in Figure 3 .

[0073] 1) bin is the basic vertical resolution unit in CALIPSO data product. The standard bin of CALIPSO has a vertical resolution of 30 meters, that is, each bin represents a 30-meter-thick atmospheric layer. If the bottom of the cloud layer is higher than the sea surface by no more than 2 bins (60m), the layer is classified as sea fog.

[0074] 2) If the CALIOP detected surface profile is at least 2 bins (60 m) higher than the nominal sea surface height, and the CALIOP surface / subsurface is higher than 0 is considered as a candidate of sea fog. If the 532 nm attenuated backscatter of this candidate sea fog layer is greater than a predefined threshold (0.03 km-1sr-1), then the layer will be classified as sea fog. -1 SR -1 ), the layer will be classified as sea fog.

[0075] 3) If the cloud layer bottom is more than 2 bins (60 m) above sea surface, then the layer is classified as non-sea fog.

[0076] (3) ERA5 reanalysis data preprocessing. The ERA5 reanalysis data used in the present application mainly includes sea surface temperature (sst), 2m air temperature (t2m), 10m wind speed (w10m), relative humidity (RH). Due to the inconsistency of spatial resolution, after format conversion, the nearest neighbor resampling method is used to resample the ERA5 reanalysis data to the FY-4A pixel grid.

[0077] Step 3: Knowledge data fusion

[0078] With the preprocessed multi-spectral images of geostationary meteorological satellites and atmospheric reanalysis data as sample data, and the reclassified cloud-aerosol observation data as the corresponding sample label, a training data set is constructed. In this embodiment, combined with meteorological observation knowledge, texture and spectral information of multi-channel data observation and CALIPSO satellite vertical observation data, sea fog sample labels are made, and knowledge is generated, labeled or processed to train data, and the knowledge is internalized in the training data.

[0079] Step 4: Knowledge feature fusion

[0080] This step expands the knowledge feature dimension according to the knowledge mined from existing literature, combined with input main data and input auxiliary data.

[0081] (1) Sea surface temperature knowledge. The sea fog in China's coastal area is mainly advection cooling fog, and the cold sea surface is the cold source of the cooling of warm and humid air; if the water temperature is too high, the air temperature is difficult to be reduced to below the dew point temperature, and the sea fog will be difficult to form; the large fog area in the northwest Pacific Ocean appears in the sea area with an average water temperature lower than 20°C, and the fog gradually decreases in the sea area with a water temperature higher than 20°C, and there is no fog in the area with a water temperature higher than 25°C.

[0082] (2) Sea-air temperature difference knowledge. The formation of sea fog is not only restricted by the sea surface water temperature, but also closely related to the difference between the air temperature and the water temperature. Because only when the air temperature is higher than the water temperature within a certain limit, the air temperature can gradually decrease to reach supersaturation and form fog. The air temperature and water temperature difference of China's coastal advection cooling fog is between 0.5°C and 3°C. If the temperature difference is greater than 5°C or less than-0.1°C, sea fog cannot be formed in general.

[0083] (3) Air humidity knowledge. When the temperature condition is met, the air humidity is also a key factor for the formation of sea fog. According to the overall transport formula for calculating the sensible heat and latent heat transport of the water surface, the atmosphere releases the sensible heat to the sea, and the air temperature and water temperature difference is required to be positive, and the saturated water vapor condenses into fog, and the water vapor pressure of the air mass is required to be greater than the saturated water vapor pressure of the sea surface, so that the larger air humidity is also one of the conditions for the formation of sea fog.

[0084] (4) Wind and fog correlation knowledge. The sea surface advection fog is formed by the cooling and condensation of warm and humid air flowing on the cold sea surface. Only the wind can blow the warm and humid air to the cold sea surface. Therefore, in addition to the influence of temperature and humidity conditions, the wind (especially the specific wind direction and moderate wind force) is also an important factor for the formation of sea fog. It is generally considered that the wind of 2-4 levels is most conducive to the development of advection fog. This is because the wind less than 2 levels is not conducive to the transportation of warm and humid air, and the wind greater than 6 levels greatly increases the momentum exchange of the sea surface, the mixed layer thickens, and the water vapor cannot be accumulated on the water surface, and the fog is easy to dissipate or lift to low clouds.

[0085] In addition to the above fusion of time and space characteristics, the examples of the present application select three characteristic enhancement indexes of sea-air temperature difference, cloud top height index, and fog occurrence probability as new effective features.

[0086] 1) Sea-air temperature difference.

[0087] (1)

[0088] In the formula, The sea-air temperature difference value is represented by Tsd; The sea surface air temperature is represented by Tsa, which usually refers to the air temperature about 2m high on the sea level; The sea surface temperature is represented by Tsw, which usually refers to the temperature of the sea surface or the uppermost layer of water of several meters.

[0089] 2) Cloud top height index (CTH). According to the cloud top height, the medium and high clouds can be quickly removed and determined. The basic principle is to use the difference between the cloud top temperature and the sea surface temperature and the vertical decrement rate of the troposphere, that is, the temperature decreases by 6.5℃ per 1km of altitude, to inversely calculate the cloud top height to separate the medium and high clouds. Since the sea surface temperature and the channel brightness temperature do not belong to the same dimension, the present examples adopt the maximum brightness temperature of a single pixel per hour for 15 days to replace the sea surface temperature in the corresponding period. Assuming that the single pixel is not covered by the cloud layer at least once in 15 days, and the sea surface brightness temperature changes little in 15 days, the following formula can be used to calculate the cloud top height:

[0090] (2)

[0091] In the formula, For long-wave infrared channel brightness temperature value, its wavelength range is about 8 to 14 microns, in the modified waveband range, the cloud layer is considered as an approximate black body, and the main radiation received by the satellite sensor comes from the cloud top or the ground itself, so the value is closer to the actual temperature of the cloud top, and the example of the application takes the channel with a center wavelength of 10.8um of the FY4A satellite AGRI sensor as the BT value, For the calculated clear-sky brightness temperature, For the temperature gradient of the atmosphere, in the embodiment, it is set to

[0092] 3) The index of the fog occurrence probability empirical formula.

[0093] (3)

[0094] Fog occurrence probability wherein, is the relative humidity, is the input sea surface wind speed at a specified height, that is, the wind speed w10m at the sea surface 10 meters; is the natural base, is a preset adjustable coefficient for determining the strength of the inhibitory effect of wind speed on fog formation, which is a negative number, reflecting that the probability of fog occurrence decreases when the wind speed increases, and the greater the absolute value, the stronger the inhibitory effect of wind speed on fog, in the embodiment, k=-0.1 is taken as a reference value which is widely used and can effectively reflect the inhibitory effect of wind speed.

[0095] The FY4A L1 multi-channel data, the spatio-temporal feature data and the remote sensing index data are spliced to fuse the knowledge features into the original channel data, and a feature enhanced tensor is obtained as shown in Figure 4 .

[0096] Step 5: Knowledge network fusion

[0097] (1) Spatio-temporal attention mechanism

[0098] Firstly, the formation and dissipation of sea fog are closely related to ocean currents, land-sea thermal differences and topography, and the seasonal variation is also very obvious, therefore, a spatio-temporal attention mechanism (Spatio-Temporal Attention Mechanism, STAM) is designed to replace the traditional normalization layer and single channel attention mechanism. The STAM aims to use the spatio-temporal metadata (such as longitude, latitude and month) of the input data as prior knowledge to dynamically guide the network to perform channel and spatial double weighting on the feature map, thereby enhancing the adaptability of the model to different geographical regions and seasonal environments.

[0099] ​The STAM module consists of a position encoder, a channel attention generator and a spatial attention generator. The position encoder converts the spatio-temporal metadata into a high-dimensional embedding vector, the channel attention generator learns the channel weights based on the vector, and the spatial attention generator directly learns the spatial weights from the feature map itself.

[0100] 1) Position encoding: First, the spatio-temporal metadata vector M passes through a position encoder composed of a fully connected layer (Linear), generating a high-dimensional position embedding vector P

[0101] (4)

[0102] where, is the dimension of the embedding vector.

[0103] 2) Channel attention generation: The position embedding vector P is input into a channel attention generator composed of a fully connected layer and a Sigmoid activation function. This module maps the position information to a weight vector in the channel dimension

[0104] (5)

[0105] 3) Spatial attention generation: Unlike channel attention, spatial attention weights are directly learned from the feature map X (the output feature map of the backbone network). It goes through a series of convolutional layers (Conv2d) and activation functions, and finally generates a weight map with the same spatial dimension as the feature map

[0106] (6)

[0107] Here, take a two-layer convolutional network as an example, Conv1 and Conv2 usually use 1x1 convolution to adjust the number of channels.

[0108] 4) Feature weighting and output: Finally, the feature map X is simultaneously multiplied element-wise with the channel weight and the spatial weight to obtain the final weighted feature map

[0109] (7)

[0110] where, is the feature map of the intermediate layer of the network, denotes the element-wise multiplication operation.

[0111] (2) Knowledge-guided attention mechanism

[0112] To make up for the lack of data or uneven distribution and the poor interpretability of pure data-driven models, especially under complex weather conditions, the embodiment of the present application introduces a knowledge-guided attention mechanism (KGAM). This mechanism explicitly injects meteorological domain knowledge into the neural network, enhancing the model's attention to key features. The specific implementation is as follows:

[0113] 1) Knowledge feature extraction. A set of meteorological factors strongly related to sea fog formation is defined in advance, including sea-air temperature difference, relative humidity, wind speed, and wind direction. These meteorological data can be obtained from atmospheric reanalysis data sets to form a meteorological knowledge feature vector K.

[0114] (8)

[0115] In the formula, Q is the meteorological parameter enhancement vector.

[0116] 2) Attention weight generation. The knowledge feature vector K is taken as an additional input to generate an attention weight vector .

[0117] (9)

[0118] 3) Feature fusion. The generated knowledge attention weight is multiplied element by element with the previously calculated spatial attention weight and the feature map , thereby weighting the features to obtain the final weighted feature map :

[0119] (10)

[0120] This explicit fusion method enables the model to dynamically adjust its internal feature representation according to meteorological knowledge, thereby exhibiting stronger robustness under complex weather conditions such as strong winds.

[0121] (3) Physical constraint loss

[0122] To reduce unreasonable predictions, the present application designs a physics-constrained loss (PCL) as a regular term for the model. This loss function forces the model to adhere to basic physical laws in its prediction results, thereby improving the credibility of the results. The specific implementation is as follows:

[0123] 1) Define physical constraints. Determine the physical constraint conditions related to sea fog phenomena, such as sea-air temperature difference constraints, high wind speed constraints, etc.

[0124] Sea-air temperature difference constraint. Sea fog usually forms when the sea-air temperature difference is less than zero or close to zero, so when the model predicts the presence of sea fog, the corresponding sea-air temperature difference should meet this condition.

[0125] High wind speed constraint. When the wind speed is too high, it is not easy to form or maintain sea fog. Therefore, under high wind speed (such as greater than 10 m / s) conditions, the probability of the model predicting sea fog should be penalized.

[0126] 2) Loss function construction. Incorporate physical constraints into the loss function and construct an additional regularization term :

[0127] (11)

[0128] where, is the standard cross-entropy loss, is the balance hyperparameter. The specific form of can be a penalty function. For example:

[0129] (12)

[0130] where, represents the model's predicted sea fog probability (i.e., the output pixel value); represents the input wind speed value; represents the preset wind speed threshold.

[0131] 3) Model optimization. During the training process, the model not only optimizes the classification accuracy, but also is guided by the physical constraints, so that the final prediction result is more in line with scientific laws, significantly improving the credibility of the algorithm.

[0132] Through the above steps, the present application deeply integrates domain knowledge and data-driven methods, effectively solving the problems of poor cross-regional adaptability, insufficient robustness under complex weather conditions, and unreasonable prediction in sea fog recognition, and ultimately constructing a sea fog recognition algorithm with superior performance and high credibility.

[0133] That is, in the embodiments of the present application, the constructed sea fog recognition model successively includes: a backbone network (for example, a backbone network of ResNet50-Unet is selected), a spatio-temporal attention mechanism module, a knowledge-guided attention module, a fusion module and a sea fog classification decoder; wherein the input of the backbone network is the fusion features of the sample data, and the output feature map X of the backbone network is respectively sent to the spatio-temporal attention mechanism module and the knowledge-guided attention module; the spatio-temporal attention mechanism module is used to obtain a channel weight vector and a spatial attention weight vector of the output feature map X, and a first attention feature map of the output feature map X is obtained based on the two weight vectors; the knowledge-guided attention module is used to obtain a knowledge attention weight vector of the meteorological parameter enhancement vector, and a second attention map of the output feature map X is obtained based on the spatial attention weight vector obtained by the spatio-temporal attention mechanism module; the first and second attention feature maps are fused through the fusion module, and the obtained fusion feature map is input into the sea fog classification decoder; the sea fog classification decoder is used to output the predicted sea fog probability of each pixel.

[0134] Step 6: Training and evaluation

[0135] This step describes the training settings and performance evaluation indicators of the model.

[0136] (1) Training

[0137] The preprocessed data set is input into the model, and training is performed through the following configurations:

[0138] 1) Optimizer. The AdamW optimizer is adopted, the initial learning rate is 0.001, and the cosine annealing (CosineAnnealing) learning rate scheduler is used to ensure the stable convergence of the model.

[0139] 2) Batch size. The batch size is set to 8.

[0140] 3) Hardware configuration. The training process is performed on a computing platform equipped with a high-performance GPU.

[0141] (2) Evaluation

[0142] After the model training is completed, the performance of the model is evaluated using an independent test data set. The main evaluation indicators aim to comprehensively measure the accuracy and precision of sea fog recognition, and the calculation formula is as follows:

[0143] 1) Mean Intersection over Union (mIoU). It measures the average segmentation performance of the model on all classes. For sea fog recognition (two classes: sea fog and non-sea fog), mIoU is the average value of the IoU of each class.

[0144] (13)

[0145] wherein, is the total number of classes, is the Intersection over Union of the th class, which is calculated as follows:

[0146] (14)

[0147] wherein, TP (True Positive) is the number of correctly identified sea fog pixels, FP (False Positive) is the number of non-sea fog pixels misjudged as sea fog, and FN (False Negative) is the number of missed sea fog pixels.

[0148] 2) Precision. Measures the proportion of pixels predicted by the model as sea fog that are actually sea fog.

[0149] (15)

[0150] 3) Recall. Measures the proportion of all true sea fog pixels that are correctly identified by the model.

[0151] (16)

[0152] 4) F1-Score. Combines precision and recall, which is the harmonic mean of the two.

[0153] (17)

[0154] The comparison results of the method of the present application and some other advanced sea fog recognition algorithms are shown in Table 1.

[0155] U-Net: U-Net is a classic semantic segmentation network, whose name comes from the unique "U" shaped symmetric architecture, which consists of a down-sampling (encoder) and an up-sampling (decoder) path. This encoding-decoding structure has shown great superiority in medical imaging and computer vision fields, and is still used as the backbone network of many advanced neural networks. Its core feature is the skip connection, which directly transmits high-resolution information in the encoder to the decoder, ensuring accurate pixel-level positioning.

[0156] ResNet-Unet: ResNet-Unet is a residual block that replaces the traditional convolutional layer in the encoder based on U-Net. This combination effectively alleviates the gradient vanishing problem in deep network training, while retaining the superior encoding-decoding structure and skip connection of U-Net, thus having stronger feature extraction capability and better performance, and is also the backbone network adopted in the method of the present application.

[0157] S2D-Net: S2D-Net is a deep encoding-decoding framework specially used for satellite remote sensing sea fog identification. Its backbone network is also based on U-Net, and the core idea of this network is to fuse inter-spectral and intra-spectral two kinds of heterogeneous difference features. It captures the shape features of sea fog through the inter-spectral difference module (PIDM), and extracts its fine texture and appearance information through the intra-spectral difference module (ISDM), aiming to obtain more robust and accurate segmentation results.

[0158] As can be seen from Table 1, the KD-FogNet algorithm proposed in the embodiments of the present application has a leading advantage in various indicators compared with other similar algorithms.

[0159] Table 1 Comparison of algorithm indicators

[0160]

[0161] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

[0162] The above only describes some embodiments of the present application. For those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application.

Claims

1. A satellite remote sensing sea fog identification method based on knowledge and data double driving, characterized in that, The method comprises the following steps: Step 1, determining a target area, and obtaining two types of satellite observation data and atmospheric reanalysis data of the target area, wherein the satellite observation data comprises multispectral images and cloud-aerosol observation data of geostationary meteorological satellites; Step 2, performing multi-source data collaborative preprocessing on the two types of satellite observation data and atmospheric reanalysis data of the target area, integrating the two types of satellite observation data into a unified input tensor through space-time matching and resampling; and the preprocessing of the cloud-aerosol observation data comprises sea fog reclassification; Step 3, taking the preprocessed multispectral images of the geostationary meteorological satellite and the atmospheric reanalysis data as sample data, constructing sample labels of the sample data based on the reclassified cloud-aerosol observation data, and constructing a training data set; Step 4, performing feature extraction and fusion on the sample data of the training data set, comprising: performing pixel synthesis on the multispectral images of the sample data according to a set time period to obtain channel features of the multispectral images; extracting space-time features in the atmospheric reanalysis data of the sample data; and extracting a plurality of feature enhancement indexes of the sample data, including cloud top height, sea surface air-sea temperature difference and fog occurrence probability; splicing the space-time features, index features and channel features of the multispectral images to obtain fusion features of the sample data; based on a group of meteorological factors for forming sea fog, extracting meteorological parameter values of each meteorological factor for forming sea fog from the atmospheric reanalysis data of the sample data to form a meteorological parameter enhancement vector of the sample data; Step 5, constructing a sea fog recognition model, which comprises in sequence a backbone network, a space-time attention mechanism module, a knowledge-guided attention module, a fusion module and a sea fog classification decoder; wherein the input of the backbone network is the fusion features of the sample data, and the output feature map X of the backbone network is sent into the space-time attention mechanism module and the knowledge-guided attention module; the space-time attention mechanism module is used for obtaining a channel weight vector and a spatial attention weight vector of the output feature map X, and obtaining a first attention feature map of the output feature map X based on the two weights; the knowledge-guided attention module is used for obtaining a knowledge attention weight vector of the meteorological parameter enhancement vector, and obtaining a second attention map of the output feature map X based on the spatial attention weight vector obtained by the space-time attention mechanism module; the first and second attention feature maps are fused through the fusion module, and the obtained fusion feature map is input into the sea fog classification decoder; the sea fog classification decoder is used for outputting the predicted sea fog probability of each pixel; Step 6, performing model training on the sea fog recognition model based on the training data set to obtain a sea fog recognition model for the target area. The loss function used in model training is: wherein, is the cross-entropy loss of the sea fog recognition model, is a set balance hyperparameter, is a regularization term about the sea fog formation meteorological factors, when at least one constraint of the sea fog formation meteorological factors is satisfied, the value of is set to the sea fog probability predicted by the sea fog recognition model, otherwise 0.

2. The method of claim 1, wherein, In step 2, the multi-source data collaborative preprocessing comprises: performing radiation calibration, projection conversion and image format conversion on the multispectral images; performing image format conversion and sea fog reclassification on the cloud-aerosol observation data; performing image format conversion on the atmospheric reanalysis data set, and resampling it to the pixel grid of the multispectral images.

3. The method of claim 1, wherein, The space-time features include sea surface temperature, air temperature, wind speed and relative humidity.

4. The method of claim 1, wherein, The channel characteristics of the multispectral image include: brightness temperature of visible light, near-infrared light, short-wave infrared light, medium-wave infrared light, water vapor and long-wave infrared light.

5. The method of claim 1, wherein, The sea fog formation meteorological factors include: sea-air temperature difference, relative humidity, wind speed and wind direction.

6. The method of claim 5, wherein, During model training, the regularization term Specifically, the following settings are made: ； wherein, represents a sea fog probability predicted by the sea fog recognition model; represents an input sea surface wind speed at a specified height; represents a preset wind speed threshold.

7. The method of claim 1, wherein, The sea fog reclassification specifically is: If the height difference between the bottom of the layer and the nominal sea surface elevation is less than or equal to then the current layer is classified as sea fog if the layer is classified as sea fog; wherein, , is a first height bias value set. If the surface of the layer differs from the nominal sea surface elevation by more than the sea surface , and the surface / subsurface of the layer is higher than the nominal sea surface elevation 0 by more than the sea surface , and the surface / subsurface of the layer is higher than the nominal sea surface elevation 0 by more than the sea surface , and the surface / subsurface of the layer is higher than the nominal sea surface elevation 0 by more than the sea surface , and the surface / subsurface of the layer is higher than the nominal sea surface elevation 0 by more than the sea surface , and the surface / subsurface of the layer is higher than the nominal sea surface elevation 0 by more than the sea surface , and the surface / subsurface of the layer is higher than the nominal sea surface elevation 0 by more than the sea surface <000 If the height difference between the bottom of the layer and the nominal sea surface elevation is greater than then the current layer classification is not sea fog.

8. The method of claim 1, wherein, In step 4, the cloud top height, the sea surface air-sea temperature difference and the fog occurrence probability are specifically: Cloud top height wherein, is in kilometer, is the long-wave infrared channel brightness temperature value, is the clear-sky brightness temperature synthesized by the characteristics of each channel of the multi-spectral image, is the temperature lapse rate of the atmosphere; sea-air temperature difference wherein, is the sea surface air temperature, is the sea surface temperature; fog occurrence probability wherein, RH is the relative humidity, ln is the natural logarithm, a is a preset adjustable coefficient for characterizing the strength of the inhibitory effect of wind speed on fog formation.

9. The method of claim 1, wherein, The spatio-temporal attention mechanism module includes a position encoder, a channel attention generator, a spatial attention generator and a feature weighting output layer; The position encoder includes, in sequence, fully connected layers. Activation function and fully connected layer Features used to fuse sample data input to the sea fog recognition model Mapping to high-dimensional position embedding vector , ,in, For batch size, High-dimensional position embedding vector Dimensions For the real number field; The channel attention generator maps the high-dimensional positional embedding vector into a channel weight vector by a fully connected layer with a sigmoid activation function , wherein denotes the sigmoid activation function, denotes the number of channels; The spatial attention generator generates spatial attention weights by a convolution layer , and an activation function wherein , is an output feature map of the backbone network, and H and W are height and width of the feature map , respectively. The feature weighting output layer is used to perform element-wise multiplication of the feature map X with a channel weight vector and spatial attention weights to obtain the final weighted feature map and as the output feature map of the spatio-temporal attention mechanism module, , wherein, denotes an element-wise multiplication operation.

10. The method of claim 1, wherein, Further comprising step 6, model evaluation is performed on the trained sea fog identification model, and then the sea fog identification model meeting the model evaluation index is used for sea fog identification of a target area; Among them, the model evaluation indexes include: average intersection over union , precision , recall and F1 score, and each index is specifically: Average intersection over union : ; precision : ; recall rate : ; F1 score: ; wherein, is the intersection over union ratio for the first class, is the intersection over union ratio for the second class, is the intersection over union ratio for the single class, is the number of correctly identified sea fog pixels, is the number of non-sea fog pixels that were misidentified as sea fog, is the number of missed sea fog pixels.

Images