A deep neural network feature enhancement method based on spatiotemporal metadata fingerprints

Abstract

This invention discloses a deep neural network feature enhancement method based on spatiotemporal metadata fingerprints, relating to the fields of artificial intelligence and spatiotemporal data processing. The method includes: parsing the absolute geographic location and observation time metadata of multidimensional spatiotemporal data; constructing a geospatial fingerprint tensor aligned with the spatial dimensions of the visual feature map; constructing a periodic and continuous temporal phase fingerprint vector using trigonometric function transformations; fusing the spatiotemporal fingerprint with visual features through cross-modal projection to generate a spatiotemporal attention mask; and dynamically calibrating the basic features using this mask. This invention effectively solves the problem of absolute spatiotemporal information loss caused by translation invariance in traditional convolutional neural networks, significantly improving the prediction accuracy of the model in non-stationary spatiotemporal scenarios.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and spatiotemporal data processing technology, and in particular to a deep neural network feature enhancement method based on spatiotemporal metadata fingerprints. Background Technology

[0002] With the rapid development of deep learning, Convolutional Neural Networks (CNNs) have become one of the core technologies for processing image, video, and remote sensing data. CNNs and their variants (such as ResNet, U-Net, and DeepLab) extract features by sliding convolutional kernels with shared weights across images, exhibiting significant "translation invariance." This means that regardless of where the target object appears in the image, the feature responses extracted by the network are similar, thus enabling consistent classification results. This property is highly advantageous for general object recognition tasks.

[0003] However, in scenarios with complex spatiotemporal attributes, such as sea fog detection and traffic prediction, data distribution exhibits significant spatiotemporal nonstationarity. This nonstationarity makes traditional feature extraction mechanisms based on translation invariance difficult to effectively capture spatiotemporal evolution patterns, thus having certain limitations. Specifically, this is reflected in the following two aspects:

[0004] (1) Spatial nonstationarity: The same visual texture may represent completely different physical meanings in different geographical locations. For example, in meteorological satellite cloud images, cloud clusters with similar spectral characteristics (high reflectivity, low brightness temperature) may indicate typhoons or convective clouds if they appear in tropical seas; while the same texture, if it appears in a high-latitude cold current convergence area, is very likely advection fog. Existing CNN models are not sensitive to absolute location and therefore cannot make use of this geographical prior knowledge;

[0005] (2) Temporal non-stationarity: The same state may have different evolution trends at different time periods. For example, in traffic forecasting, the same road congestion density is normal if it occurs during the morning rush hour (8:00), but abnormal if it occurs at night (2:00). In sea fog monitoring, the formation of sea fog is strictly controlled by the diurnal cycle. Strong solar radiation in the afternoon will disrupt the stable stratification of sea fog. At this time, even if a fog-like texture is observed, it is very likely to be broken clouds rather than sea fog.

[0006] To incorporate spatiotemporal information, existing deep learning methods typically employ a simple "channel concatenation" approach, directly concatenating latitude and longitude coordinates and linear time values ​​(e.g., 0-23 hours) as additional channels into the network's input layer. However, this simplistic approach suffers from three significant drawbacks in practical applications:

[0007] (1) Loss of absolute location information in deep networks: Modern convolutional neural networks typically involve multiple downsampling (pooling) or large stride convolution operations. As the number of network layers increases, the spatial resolution of the feature maps gradually decreases, and the fine geographic coordinate information spliced ​​from the original input layers is gradually "diluted" or even lost. This makes it difficult for deep semantic features to perceive their corresponding precise geographic locations, thus making it difficult to use geographic distribution priors (such as land-sea background, ocean current distribution) to assist in decision-making;

[0008] (2) Temporal periodicity breaks caused by linear encoding: Existing time encoding often directly uses normalized linear values ​​(e.g., normalizing 0-23 hours to [0,1]). This method disrupts the physical periodic continuity of time. For example, 23:59 (value close to 1) and 00:01 (value close to 0) are only 2 minutes apart in physical time, but they are far apart in numerical space. This "artificially created numerical mutation" will cause the model's predictive performance to drop significantly at the boundary of the cycle (such as midnight or the end of the year), and it will be unable to accurately capture the evolutionary pattern across cycles;

[0009] (3) Lack of dynamic adjustment mechanism: Existing fusion methods are usually static, meaning that the network processes visual textures in the same way regardless of the time or location. There is a lack of a "gating mechanism" or "filter" that can dynamically adjust the response weights to specific textures based on the current spatiotemporal context. For example, in non-sea fog seasons (such as summer) or non-fog periods (such as afternoon), the network should automatically suppress false edge responses that resemble sea fog, but existing static models cannot achieve this adaptive feature recalibration.

[0010] Therefore, there is an urgent need for a feature enhancement method that can explicitly and effectively encode absolute spatiotemporal information, solve the problem of time periodicity, and deeply integrate it with visual features, so as to break through the performance bottleneck of existing technologies in complex non-stationary spatiotemporal scenarios. Summary of the Invention

[0011] The purpose of this invention is to provide a deep neural network feature enhancement method based on spatiotemporal metadata fingerprints. This method achieves dynamic recalibration of deep features by constructing a "geospatial fingerprint" aligned with visual features and a "periodic temporal fingerprint" that maintains numerical continuity, thereby endowing the neural network with the ability to perceive the absolute spatiotemporal environment.

[0012] To achieve the above objectives, the present invention provides the following technical solution: a deep neural network feature enhancement method based on spatiotemporal metadata fingerprints, comprising the following steps:

[0013] Step 1, Multidimensional Data Parsing and Metadata Extraction: Obtain the multidimensional spatiotemporal data to be processed, parse the metadata attached to the data, extract the absolute geographical location information covering the data collection area and the absolute timestamp information of the data collection time, and obtain multidimensional spatiotemporal sequence data;

[0014] Step 2, Basic Visual Feature Extraction: Input the multidimensional spatiotemporal sequence data into the backbone of the deep neural network, and extract the basic feature tensor through convolution operations. Its dimensions are Where C is the number of channels, and H and W are the height and width;

[0015] Step 3, Geospatial Fingerprint Tensor Construction: Based on absolute geographic location information, construct a geospatial fingerprint tensor. Spatial dimension A consistent grid coordinate matrix is ​​used; longitude and latitude values ​​are normalized to a preset interval (e.g., [-1, 1]), and a geospatial fingerprint tensor composed of longitude and latitude channels is constructed. ;

[0016] Step 4, Construction of Multi-Scale Periodic Time Fingerprint: Parse the absolute timestamp information to determine at least one time cycle dimension required for the target task (such as daily, weekly, or yearly cycles); for each time cycle dimension, set its corresponding baseline period constant T, and use a periodic mapping function to map linear time values ​​into two-dimensional phase vectors on the unit circle to maintain the numerical continuity of the time encoding at the beginning and end of the cycle; concatenate the generated at least one two-dimensional phase vector to construct a multi-scale composite time fingerprint vector. That is, if there exists only one two-dimensional phase vector, then That is, the two-dimensional phase vector itself;

[0017] Step 5, Cross-modal spatiotemporal-semantic fusion: Integrating the temporal fingerprint vector Perform spatial broadcasting, extending it to include geospatial fingerprint tensors. With the same spatial dimension, a temporal fingerprint vector is obtained. ; will ; for time fingerprint vector Geospatial fingerprint tensor With fundamental feature tensor Feature fusion processing is performed to obtain multimodal fused latent features; for example, the time fingerprint vector is used to obtain multimodal fused latent features. Geospatial fingerprint tensor With fundamental feature tensor The features are stitched together along the channel dimension and projected onto the latent space through a convolutional layer to obtain multimodal fusion latent features.

[0018] Step 6, Dynamic Weight Calibration and Feature Output: Based on the multimodal fusion features, a spatiotemporal attention mask is generated using an activation function (such as sigmoid); then the spatiotemporal attention mask is combined with the basic feature tensor. Perform element-wise product and output spatiotemporal augmented features through residual connection. .

[0019] Furthermore, the backbone network is a convolutional neural network based on residual networks.

[0020] Furthermore, in step 4, the periodic mapping function is transformed using trigonometric functions, expanded using Fourier series, or derived from Gaussian radial basis functions.

[0021] Furthermore, in step 4, when using trigonometric function transformation to convert to a periodic mapping function, specifically mapping the linear time value to a two-dimensional phase vector on the unit circle using trigonometric function transformation includes:

[0022] Define the number of event scales required for the target task as N, and the period constant for any scale as... This indicates that k is the event scale identifier;

[0023] For timestamp values ​​at the current scale Construct a two-dimensional phase vector at scale k .

[0024] Furthermore, the event scales of the target task include: diurnal variation scale and seasonal variation scale.

[0025] Furthermore, in step 5, the multimodal fusion latent features specifically include:

[0026] geospatial fingerprint tensor With time fingerprint vector By concatenating along the channel dimension, we obtain the spatiotemporal prior tensor. ;

[0027] The spatiotemporal prior tensor is converted through a convolution operation with a 1×1 kernel. Basic feature tensor The latent space is mapped separately to obtain the projected visual feature embedding and physical feature embedding;

[0028] Then, by splicing the visual feature embedding and the physical feature embedding through the channel dimension, feature fusion is performed to obtain multimodal fusion latent features.

[0029] Furthermore, in step 5, the feature fusion method of visual feature embedding and physical feature embedding is replaced with additive fusion or fusion based on the cross-attention mechanism of Transformer.

[0030] Furthermore, in step 6, spatiotemporal enhancement features ,in, It represents the Hadamah accumulation. For dimensionality recovery, there are 1×1 convolutional layers, i.e., residual connections with 1×1 convolutional layers. This refers to a spatiotemporal attention mask.

[0031] Furthermore, the multidimensional spatiotemporal data to be processed is satellite metadata.

[0032] Furthermore, based on the spatiotemporal enhancement features obtained in step 6 It can be used for sea fog identification in data acquisition areas, such as clear skies, sea fog, low clouds, and mid-to-high clouds.

[0033] The technical solution provided by this invention brings at least the following beneficial effects:

[0034] (1) Solving the problem of time discontinuity: This invention uses trigonometric function (sine / cosine) transformation to encode time. Compared with traditional linear normalization, it can perfectly express the periodic cyclical characteristics of time, eliminate numerical abrupt changes at the boundary of the cycle, and significantly improve the model's ability to capture periodic patterns (such as daily variations and seasonal variations).

[0035] (2) Imparting absolute spatial perception capability: By embedding geospatial fingerprints, the limitation of translation invariance of convolutional neural networks is overcome, enabling the model to learn the prior distribution characteristics of specific geographical areas.

[0036] (3) Strong versatility and scalability: The multi-scale time fingerprint construction strategy proposed in this method is not limited to a specific field. The period constant can be flexibly configured according to actual needs (such as 24 hours or 365 days), which is applicable to various spatiotemporal prediction scenarios such as meteorology, transportation, and agriculture. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1This is a schematic diagram illustrating the geometric principle of constructing time phase fingerprint vectors;

[0039] Figure 2 This is a schematic diagram of the internal structure of the Spatiotemporal Fingerprint Feature Enhancement Module (STAM).

[0040] Figure 3 This is a comparison chart showing the implementation effect of the present invention in a sea fog monitoring application scenario. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be described in detail and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Generally, the components of the embodiments of the present invention described and shown in the accompanying drawings can be arranged and designed using different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present invention.

[0042] Because the occurrence and evolution of sea fog exhibit significant spatiotemporal heterogeneity, for example, its formation is strictly controlled by the diurnal cycle (usually forming at night and dissipating in the afternoon), and it displays distinctly different texture properties in nearshore and offshore areas. To enable deep neural networks to distinguish between "clouds" and "fog" with similar spectral features but in different spatiotemporal contexts, this embodiment designs a Spatiotemporal Attention Module (STAM). This module is embedded as a plug-in in the shallow feature extraction stage of the backbone network, injecting explicit geolocation and temporal cycle priors into the network to achieve "physical perception recalibration" of sea fog features. Specifically, the implementation steps of a deep neural network feature enhancement method based on spatiotemporal metadata fingerprints provided in this embodiment include:

[0043] Step S1: Multidimensional data parsing and metadata extraction

[0044] Acquire multidimensional spatiotemporal data to be processed, parse the metadata attached to the data, extract the absolute geographical location information covering the data collection area and the absolute timestamp information of the data collection time, and obtain multidimensional spatiotemporal sequence data;

[0045] In this embodiment, the multidimensional spatiotemporal data to be processed is the multi-channel observation data of the FY-4A satellite, i.e., satellite metadata, which includes visible light, shortwave infrared, longwave infrared and other channels.

[0046] Step S2, Basic Visual Feature Extraction

[0047] This step is used to extract shallow visual features from multidimensional spatiotemporal sequence data.

[0048] First, the multidimensional spatiotemporal sequence data obtained in step S1 is input into the backbone network of a deep neural network for feature extraction. In this embodiment, the backbone network used is a convolutional neural network based on ResNet, and the output of the shallow layers of the network is selected as the basic visual feature tensor. (i.e., visual features of sea fog), its dimensions are At this time It mainly contains texture and edge information, but lacks global spatiotemporal context.

[0049] Step S3, Fingerprinting of Spatiotemporal Features

[0050] To transform discrete satellite metadata into a continuous, physically meaningful tensor representation, this step includes: geospatial fingerprint tensor construction, multi-scale periodic temporal fingerprint construction, and cross-modal spatiotemporal-semantic fusion.

[0051] (1) Construction of geospatial fingerprint tensor:

[0052] The distribution of sea fog is greatly influenced by ocean currents, water depth, and distance from the shore. This embodiment extracts the longitude λ of each pixel (i, j) based on the affine transformation matrix of the input satellite data (TIFF). i,j and latitude φ i,j Construct a tensor with the same spatial dimensions as its input image, and normalize the longitude and latitude values ​​to a preset interval [-1,1]:

[0053] (1)

[0054] In the formula, A and B represent the longitude and latitude matrices, respectively; H and W are the height and width of the input image, respectively; and R is the real number field. This provides the model with an absolute position reference, which helps the network learn the fog formation prior for specific sea areas (such as the confluence of warm and cold currents).

[0055] (2) Construction of multi-scale periodic temporal fingerprints:

[0056] For tasks with complex periodicity, this invention employs a frequency division coding strategy. It assumes the task involves N different event scales (e.g., short period T). short and long-period T long For the k-th scale, the period constant is set to T. k The current event's value at this scale is t. k Then construct the subvector V k , Figure 1This is a schematic diagram illustrating the geometric principles of constructing time-phase fingerprint vectors, demonstrating the comparative mapping relationship between linear time coding and periodic time coding:

[0057] (2)

[0058] This formula maps one-dimensional time to a two-dimensional unit circle, ensuring that t k =0 and t k =T k The feature vectors at different times are continuous and consistent. The final composite temporal fingerprint vector V time Concatenate all subvectors:

[0059] (3)

[0060] The sea fog in this embodiment exhibits a strong diurnal variation pattern (typically forming at night, peaking in the early morning, and dissipating in the afternoon) and seasonal characteristics. Directly using linear time values ​​would disrupt the periodic continuity of time (e.g., the proximity of 23:00 and 00:00). Therefore, this embodiment maps the timestamp t to reflect the characteristics of changes in solar radiation angle and seasonal sea surface temperature variations. The observation hours of the satellite data are analyzed. and accumulated days They are encoded as follows:

[0061] Daily variation fingerprint (V) diurnal ): Used to capture the solar cycle of sea fog formation and dissipation.

[0062] (4)

[0063] Seasonal fingerprints (V) seasonal ): Used to capture the seasonal background that influences sea-temperature differences.

[0064] (5)

[0065] The above scalar vector is broadcast. , Expanding the vector into a tensor with the same dimensions as the feature map space, and then concatenating them, yields the temporal fingerprint vector. :

[0066] (6)

[0067] (3) Cross-modal spatiotemporal-semantic fusion

[0068] The geospatial fingerprint tensor can be first... With time fingerprint vector The layers are stitched together along the channel dimension to form a complete sea fog spatiotemporal prior tensor. :

[0069] (7)

[0070] Then the sea fog spacetime prior tensor With fundamental feature tensor The features are concatenated along the channel dimension and projected onto the latent space through a convolutional layer to obtain multimodal fusion latent features (referred to as fusion features). ), thereby utilizing the constructed Extracted visual features of sea fog Perform dynamic weighting.

[0071] Visual characteristics of sea fog In this embodiment, the visual features of sea fog are first processed using 1*1 convolution. With the spacetime prior tensor of sea fog Mapped to the latent space respectively To integrate physical environment information with image texture information:

[0072] (8)

[0073] In the formula, and These represent the visual feature embedding and physical feature embedding after projection, respectively.

[0074] Step S4, Dynamic Weight Calibration and Feature Output

[0075] This step is attention-guided feature recalibration, including fog sensitivity weight generation, residual correction, and output, specifically as follows:

[0076] Sea Fog Sensitivity Weight Generation: Feature Fusion The sub-network, incorporating 3x3 convolutions, aims to capture spatiotemporal correlations within local neighborhoods (e.g., determining whether the texture at the current time and location matches sea fog features). A spatial attention map is then generated using Sigmoid activation. :

[0077] (9)

[0078] The weight graph A st It indicates which regions in the image have a higher probability of fogging under the current spatiotemporal conditions.

[0079] Residual Correction and Output: Finally, this embodiment applies attention weights to the original features and outputs the corrected features F through residual connections. out :

[0080] (10)

[0081] In the formula, This represents the Hadamard product (i.e., element-wise multiplication). For 1x1 convolutions used for dimension recovery, the specific design of this module in this embodiment is as follows: Figure 2 As shown.

[0082] Furthermore, this embodiment also includes step S5, training and evaluation:

[0083] This step describes the model's training setup and performance evaluation metrics.

[0084] (1) Training

[0085] Input the preprocessed dataset into the model and train it using the following configuration:

[0086] 1) Optimizer. The AdamW optimizer is used with an initial learning rate of 0.001, and a cosine annealing learning rate scheduler is used to ensure stable convergence of the model.

[0087] 2) Batch size. Set the batch size to 8.

[0088] 3) Hardware configuration. The training process is carried out on a computing platform equipped with a high-performance GPU.

[0089] (2) Assessment

[0090] To verify the effectiveness of the proposed STAM module in actual sea fog monitoring, ablation experiments were conducted on a dataset containing 1342 FY-4A images. The baseline model (based on the ResNet-UNet model) was compared with the baseline+STAM model (the method of this invention), and the model performance was evaluated using the following metrics, calculated as follows:

[0091] 1) Mean Intersection over Union (mIoU). Measures the model's average segmentation performance across all categories. For sea fog recognition (categories include: clear sky, sea fog, low clouds, and mid-to-high clouds), mIoU is the average IoU for each category.

[0092] (11)

[0093] In the formula, The total number of categories, For the first The intersection-over-union (IoU) ratio for each category is calculated as follows:

[0094] (14)

[0095] In the formula, TP (True Positive) is the number of correctly identified sea fog pixels, FP (False Positive) is the number of non-sea fog pixels that were misclassified as sea fog, and FN (False Negative) is the number of sea fog pixels that were missed.

[0096] 2) Precision. Measured by the proportion of pixels that the model predicts as sea fog, but which are actually sea fog.

[0097] (15)

[0098] 3) Recall. Measured by the proportion of all real fog pixels that are correctly identified by the model.

[0099] (16)

[0100] 4) F1-Score. It combines precision and recall and is the harmonic mean of the two.

[0101] (17)

[0102] The evaluation results of the spatiotemporal feature enhancement effect index of the STAM module in this invention are shown in Table 1, and the comparison results of the inference graph and the label are as follows: Figure 3 As shown in Table 1, introducing STAM on top of the baseline significantly improved model performance. mIoU increased from 76.26% to 78.03%, an improvement of 1.77%; mAcc, Pre, Recall, and F1 index increased by 1.33%, 1.46%, 1.52%, and 1.22%, respectively. Figure 3 The results show that by introducing STAM on top of the baseline, the model better captures the boundaries and regional extent of sea fog events occurring at different times and locations in the northern South China Sea and the Yellow and Bohai Seas, and reduces false alarms about low clouds. Overall, STAM effectively establishes the spatiotemporal correlation of sea fog occurrence. By explicitly modeling geographic location (latitude and longitude) and time cycle (daily / seasonal variation) characteristics, the model successfully captures the high-incidence patterns of sea fog in specific sea areas (such as cold coastal waters) and at specific times, significantly enhancing the prior perception ability of sea fog spatial distribution, thereby reducing false alarms in atypical areas.

[0103] Table 1 Comparison Results of Ablation Experiments

[0104]

[0105] It should be noted that the specific implementation of this invention may also include the following variations:

[0106] Replacement of time coding function: In addition to sine / cosine functions, Fourier series expansion or Gaussian radial basis function (RBF) can also be used as periodic mapping function.

[0107] Replacement of fusion methods: Cross-modal fusion is not limited to channel concatenation, but can also use additive fusion or cross-attention based on Transformer.

[0108] Network placement adjustment: The Spatiotemporal Fingerprint Embedding Module (STAM) can be placed not only at the input of the backbone network, but also at each layer of the bottleneck layer or the decoder, to build a multi-scale spatiotemporal enhancement pyramid.

[0109] The present invention provides a deep neural network feature enhancement method based on spatiotemporal metadata fingerprints, which can effectively solve the problem of loss of absolute spatiotemporal information caused by translation invariance in traditional convolutional neural networks, and significantly improve the prediction accuracy of the model in non-stationary spatiotemporal scenarios.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0111] The above descriptions are merely some embodiments of the present invention. Those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the scope of protection of the present invention.

Claims

1. A deep neural network feature enhancement method based on spatiotemporal metadata fingerprints, characterized in that, Includes the following steps: Step 1: Obtain the multidimensional spatiotemporal data to be processed, parse the metadata attached to the data, extract the absolute geographical location information covering the data collection area and the absolute timestamp information of the data collection time, and obtain multidimensional spatiotemporal sequence data; Step 2: Input the multidimensional spatiotemporal sequence data into the backbone network of the deep neural network, and extract the basic feature tensor through convolution operation. Its dimensions are Where C is the number of channels, and H and W are the height and width; Step 3: Based on absolute geographic location information, establish a tensor with the basic feature tensor. A grid coordinate matrix with consistent spatial dimensions; normalizing longitude and latitude values ​​to a preset interval, and constructing a geospatial fingerprint tensor composed of longitude and latitude channels. ; Step 4: Parse the absolute timestamp information to determine at least one time cycle dimension required for the target task; for each time cycle dimension, set its corresponding reference period constant T, and use a periodic mapping function to map the linear time values ​​into a two-dimensional phase vector on the unit circle; concatenate all two-dimensional phase vectors to obtain a multi-scale composite time fingerprint vector. ; Step 5, convert the time fingerprint vector Perform spatial broadcasting, expanding it to include geospatial fingerprint tensors. With the same spatial dimension, a temporal fingerprint vector is obtained. ; Time fingerprint vector Geospatial fingerprint tensor With fundamental feature tensor Feature fusion processing is performed to obtain multimodal fused latent features; Step 6: Based on the multimodal fusion features, generate a spatiotemporal attention mask using an activation function; then combine the spatiotemporal attention mask with the basic feature tensor. Perform element-wise multiplication and output spatiotemporal augmented features through residual connections. .

2. The method as described in claim 1, characterized in that, The backbone network is a convolutional neural network based on residual networks.

3. The method as described in claim 1, characterized in that, In step 4, the periodic mapping function is obtained by trigonometric function transformation, Fourier series expansion or Gaussian radial basis function.

4. The method as described in claim 3, characterized in that, In step 4, when using trigonometric function transformation to convert to a periodic mapping function, specifically mapping the linear time value to a two-dimensional phase vector on the unit circle using trigonometric function transformation includes: Define the number of event scales required for the target task as N, and the period constant for any scale as... This indicates that k is the event scale identifier; For timestamp values ​​at the current scale Construct a two-dimensional phase vector at scale k .

5. The method as described in claim 4, characterized in that, The event scales for the target task include: diurnal variation scale and seasonal variation scale.

6. The method as described in claim 1, characterized in that, In step 5, the multimodal fusion of latent features specifically includes: geospatial fingerprint tensor With time fingerprint vector By concatenating along the channel dimension, we obtain the spatiotemporal prior tensor. ; The spatiotemporal prior tensor is converted through a convolution operation with a 1×1 kernel. Basic feature tensor The latent space is mapped separately to obtain the projected visual feature embedding and physical feature embedding; Then, by splicing the visual feature embedding and the physical feature embedding through the channel dimension, feature fusion is performed to obtain multimodal fusion latent features.

7. The method as described in claim 6, characterized in that, In step 5, the feature fusion method of visual feature embedding and physical feature embedding is replaced with additive fusion or fusion based on the cross-attention mechanism of Transformer.

8. The method as described in claim 1, characterized in that, In step 6, spatiotemporal enhancement features ,in, It represents the Hadamah accumulation. For 1×1 convolutional layers used for dimension recovery, This refers to a spatiotemporal attention mask.

9. The method as described in claim 1, characterized in that, The multidimensional spatiotemporal data to be processed is satellite metadata.

10. The method as described in claim 9, characterized in that, Based on the spatiotemporal enhancement features obtained in step 6 It can be used for sea fog identification in data acquisition areas.

Images