Three-dimensional wind field prediction method and system based on multi-modal complementary fusion learning

Abstract

The invention provides a three-dimensional wind field prediction method and system based on multi-modal complementary fusion learning. The method comprises the following steps: S1, acquiring remote sensing observation data and numerical simulation data; s2, obtaining standardized remote sensing features and simulation features; s3, obtaining a unified scene representation; s4, splicing the unified scene representation with the to-be-predicted space-time coordinates, inputting the spliced scene representation and the to-be-predicted space-time coordinates into a physical enhancement decoder, and outputting three-dimensional wind speed vectors at the corresponding space-time coordinates; and S5, iteratively optimizing parameters of the bimodal encoder, the cross-modal attention fusion module and the physical enhancement decoder to form a closed-loop prediction model. According to the method, multi-modal data complementation and physical information deep fusion are realized, through innovating a neural network architecture and a constraint mechanism, the prediction precision under a sparse data condition is remarkably improved, the physical credibility of a result is enhanced, and a technical support is provided for intelligent development of the wind power industry.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of information and communication engineering and weather prediction, and relates to a three-dimensional wind field prediction method and system based on multi-modal complementary fusion learning. BACKGROUND

[0002] With the breakthrough of deep learning technology, data-driven intelligent prediction provides a new paradigm for fine assessment of wind energy resources in the wind power industry. Traditional numerical weather prediction (NWP) and computational fluid dynamics (CFD) methods are complex and parameter-sensitive, and are difficult to meet the real-time high-precision demand; pure data-driven models rely on dense labeled data and lack physical consistency, and are prone to failure in actual scenarios with sparse observation sites.

[0003] Physical information neural network (PINN) realizes the fusion of data and physical laws by embedding physical equations, although progress has been made, but in the extreme sparse scenario where the coverage rate of measurement points is less than 30%, the prediction uncertainty is still large. The information complementary advantage of multi-modal deep learning provides a way to solve this problem, numerical simulation and remote sensing observation data can be complementary from different dimensions.

[0004] Currently, the field of wind field prediction is experiencing a paradigm shift from single modality to multi-modality, from pure data-driven to physical information-enhanced, but key problems such as cross-modal fusion architecture design, data and physical constraint balance have not been solved. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a three-dimensional wind field prediction method and system based on multi-modal complementary fusion learning, which aims to solve the core problems of data sparsity, modality heterogeneity and lack of physical consistency in existing wind field prediction, and realize high-precision, physically credible three-dimensional wind field prediction under sparse data conditions.

[0006] The method comprises the following steps:

[0007] Step S1, acquiring remote sensing observation data and numerical simulation data, the remote sensing observation data is in two-dimensional image format, containing ground or low-altitude wind speed related information obtained by satellites and radar equipment; the numerical simulation data is in three-dimensional grid format, coming from a meteorological model or a computational fluid dynamics simulation, containing three-dimensional wind field related information from the ground to high altitude;

[0008] Step S2, based on the remote sensing observation data and the numerical simulation data, deep features of the two data sources are extracted through a dual-modal encoder, and feature space alignment is realized through a projection layer to obtain standardized remote sensing features and simulation features;

[0009] Step S3, input the standardized remote sensing features and simulation features into a cross-modal attention fusion module, taking the remote sensing features as queries and the simulation features as key values, generate dynamic attention weights by calculating the correlation between the two modalities, and perform adaptive fusion on the dual-modal features based on the weights to obtain a unified scene representation;

[0010] Step S4, concatenate the unified scene representation with the spatiotemporal coordinates (x, y, z, t) to be predicted, and input it into a physical enhancement decoder. The physical enhancement decoder adopts a five-layer fully connected neural network structure, integrates the power law model of wind profile, the Davenport coherence function model, and the Kaimal wind spectrum model as frequency domain physical constraints during the training process, and outputs the three-dimensional wind speed vector at the corresponding spatiotemporal coordinates;

[0011] Step S5, based on the deviation between the actual observation data and the prediction results, iteratively optimize the parameters of the dual-modal encoder, the cross-modal attention fusion module, and the physical enhancement decoder to form a data-efficient and physically consistent closed-loop prediction model, denoted as a dual-modal physical information fusion neural network model.

[0012] Step S2 includes the following steps:

[0013] S21, pre-process the remote sensing observation data. To address the inconsistencies in semantic expression, spatial structure representation, and significant changes in physical credibility at different height layers between remote sensing observation data and numerical simulation data, a dual-modal structured preprocessing method based on height perception consistency constraint is proposed to achieve semantic alignment and physically consistent expression at the height structure level. First, the original two-dimensional remote sensing data is normalized by channel to obtain normalized remote sensing data :

[0014] ,

[0015] wherein and represent the mean and standard deviation of the Cth channel, is the original remote sensing data, is the normalized remote sensing data; secondly, based on the physical characteristics of the continuous change of wind speed with height, a preset height weight mapping function is used to ensure that the remote sensing height perception features conform to the actual wind field vertical distribution law, and the normalized remote sensing data is subjected to height perception weighting processing:

[0016] ,

[0017] ,

[0018] wherein e represents the natural constant, represents the pre-processed remote sensing data, represents the influence weight of different height layers on remote sensing observation data, a represents the parameter of the height weight function, and H represents the image height, represents an element-wise weighting operation: multiplying the output of the height weight mapping function by element by element, realizing the weighted adjustment of the height dimension, so as to introduce the height structure information into the two-dimensional remote sensing data at the channel semantic level, and finally output the remote sensing feature data containing the height semantic information, the data format is [B, C, H, W], wherein B is the batch size, C is the input channel number, corresponding to the wind speed size, wind direction angle and data quality identifier, and W is the image width, as the remote sensing mode input of the dual-mode encoder;

[0019] S22, the dual-mode encoder includes a remote sensing data encoder and a numerical simulation data encoder;

[0020] The pre-processed remote sensing data is processed by using the remote sensing data encoder, and the remote sensing data encoder includes a lightweight two-layer two-dimensional convolution architecture, which is a first convolution layer and a second convolution layer. The first convolution layer is used to extract basic spatial features, and the pre-processed remote sensing data is subjected to a two-dimensional convolution operation to extract the spatial texture and local wind speed gradient of the wind field, and the formula is:

[0021] ,

[0022] wherein, is the size of the convolution kernel, is the step distance, is the padding value, which ensures that the size of the output feature map is consistent with the input. is the convolution kernel parameter of the first convolution layer, is the bias vector, and i and j are the position weight parameters of the first layer convolution, and are the height and width of the image output feature respectively;

[0023] Then, the activation function is used to enhance the nonlinear expression, and the ReLU activation function is applied to the output feature of the first convolution layer to relieve the gradient disappearance problem and enhance the nonlinear fitting ability of the model, and the formula is:

[0024] ,

[0025] The second convolution layer is used to deepen the feature representation: the activated feature is subjected to a second two-dimensional convolution to aggregate local features and extract more abstract spatial correlation features of the wind field, and the formula is:

[0026] ,

[0027] in, These are the features after activation in the first convolutional layer. Step size For fill value, These are the kernel parameters for the second convolutional layer. The bias vector is the final output. ;

[0028] The output of the second convolutional layer Apply the ReLU activation function again, as shown in the formula:

[0029] ,

[0030] Finally, the features after activation of the second layer are analyzed. Perform two-dimensional global average pooling operation The two-dimensional feature map is compressed into a one-dimensional feature vector, preserving global spatial statistical information. The formula is as follows:

[0031] ,

[0032] in This represents the feature vector after pooling;

[0033] Pooled feature vectors Spatial dimensional differences are eliminated by calculating the global mean, focusing on the overall distribution characteristics of the wind field. Then, a linear projection layer is used to... The feature vectors are mapped to fixed dimensions, and then normalized layer by layer to eliminate feature distribution offset, ensuring comparability with numerical simulation features, and finally outputting 32-dimensional standardized remote sensing features.

[0034] S23. To address the problem of redundant high-dimensional information in numerical simulation data and its disadvantage in cross-modal alignment, a high-structure compression preprocessing is performed on the numerical simulation data. First, the numerical simulation data is subjected to hierarchical weighted summation along the height dimension: ,

[0035] in The weight coefficient corresponding to height z. These are the original values ​​of the numerical simulation data at (x, y, z) in three-dimensional space. This is highly compressed two-dimensional simulation data. The number of layers is determined by the height dimension, so as to achieve the mapping of three-dimensional height information to two-dimensional structural expression while maintaining the height change trend;

[0036] To address the issue of varying physical reliability of wind fields at different altitudes, weighting coefficients are determined using the wind speed gradient along the altitude direction.

[0037] ,

[0038] in Let [B, F, L] be the vertical gradient of the wind speed U at height z to enhance the modeling ability for regions with significant height changes; the input format is [B, F, L]. x ,L y ,L z The data, where F=5 represents the number of physical variables, including horizontal wind speed u, horizontal wind speed v, vertical wind speed w, temperature T, and pressure p. The x-axis represents the number of grid points in the x-axis direction, indicating the number of discrete grid points in the 3D numerical simulation along the horizontal x-axis. This defines the resolution in the horizontal space. The number of grid points in the y-direction represents the number of discrete grid points in the 3D numerical simulation along the horizontal y-axis. Together they constitute the grid size of the horizontal space. The vertical layer number indicates the number of layers in the numerical simulation 3D mesh along the vertical height z-axis, used to characterize the wind field structure at different heights.

[0039] To address the issue of inconsistent trends in height structure variation in bimodal data, a height consistency constraint loss is constructed using the following formula. :

[0040] ,

[0041] in and Here, represents the spatial gradient of the remote sensing data and the spatial gradient of the simulated data, respectively, and h represents the vertical height dimension of the wind field. This formula is used to constrain the consistency of the spatial gradient variation trend between the remote sensing height perception features and the simulated height compression features. By minimizing the height consistency constraint loss function, alignment optimization of the two modalities at the height structure level is achieved. Through the height perception consistency constraint preprocessing, the remote sensing data and the numerical simulation data achieve consistent representation at the height semantic, spatial structure, and statistical distribution levels, thereby improving the feature learning stability and fusion accuracy of the two-modal encoder.

[0042] S24. The preprocessed numerical simulation data is processed using a numerical simulation data encoder containing two layers of three-dimensional convolution.

[0043] To address the need for joint modeling of the local spatial structure of a three-dimensional wind field and the physical correlation between vertical layers, the first layer of the numerical simulation data encoder performs the following three-dimensional convolution operation on the input data:

[0044] ,

[0045] in, This represents the feature volume output by the first 3D convolutional layer. This refers to the preprocessed numerical simulation data; Represents a three-dimensional convolution operation; Represents the three-dimensional convolution kernel weight matrix; Indicates the bias term;

[0046] To enhance the ability to represent the nonlinear physical relationships of complex wind fields, an activation function is applied to the output of the first-layer 3D convolution:

[0047] ,

[0048] in This represents the activated three-dimensional feature volume;

[0049] To further extract higher-level 3D spatial coupling features, a 3D convolution operation is performed again on the activated feature volume:

[0050] ,

[0051] in This represents the output feature volume of the second-layer 3D convolution. This represents the weight matrix of the second-layer convolutional kernel. Indicates the corresponding bias term;

[0052] Apply an activation function to the output of the second 3D convolution layer:

[0053] ,

[0054] in This represents the final three-dimensional spatial feature representation result;

[0055] To address the issue of high dimensionality of 3D feature volumes and the large computational complexity of their direct participation in cross-modal fusion, a 3D global average pooling operation is performed on the feature volumes:

[0056] ,

[0057] in This represents the compressed global three-dimensional statistical feature vector. Represents the original elements of the three-dimensional feature volume, namely the activated feature values ​​at positions i (x-dimensional), j (y-dimensional), and k (z-dimensional);

[0058] To achieve comparability with remote sensing modal features in terms of dimensionality and distribution, the global three-dimensional statistical feature vector is projected through a linear mapping layer:

[0059] ,

[0060] in This represents the final eigenvector of the numerical simulation mode. Represents the projection weight matrix. This represents the projection bias term, and standardization is performed on the eigenvectors:

[0061]

[0062] in This represents the standardized feature vector. This represents the mean of each dimension of the feature vector. The standard deviation is used to eliminate the impact of scale differences among different physical variables on cross-modal fusion. Through the above process, numerical simulation data is transformed into a structured representation that is dimensionally compact, semantically stable, and comparable to remote sensing features, while preserving the physical structure information of the three-dimensional wind field. This improves the stability, physical consistency, and prediction accuracy of dual-modal feature fusion.

[0063] S25. Spatial alignment of remote sensing features and simulated features is achieved through learnable linear transformations and layer normalization. The comparability of remote sensing features and simulated features is ensured by standardizing feature scales and flexibly adjusting feature distribution.

[0064] Spatial alignment of remote sensing and simulated features is achieved through a dual-modal feature adaptive alignment mechanism. This mechanism is based on learnable linear transformations and layer normalization. First, the statistical parameters of the remote sensing and simulated features are calculated separately.

[0065] ,

[0066] ,

[0067] ,

[0068] ,

[0069] in and , where are the mean and standard deviation of the remote sensing features, and B is the batch size. This represents the remote sensing feature of the b-th sample in the d-th dimension (the original feature elements output by the encoder from the remote sensing data). and These are the mean and standard deviation of the simulated features, respectively. For the b-th sample, the d-th dimension is the numerical simulation feature (feature element output by the encoder from the original simulation data);

[0070] Then, through independent, learnable affine parameters, precise matching of feature distributions is achieved by flexible adjustment, as shown in the formula:

[0071] ,

[0072] ,

[0073] in, These are learnable affine transformation parameters for remote sensing and simulated features, which are then optimized in conjunction with parameters from other modules during model training. These are remote sensing features obtained by feature extraction, aggregation, and normalization of preprocessed data using a dual-modal encoder. , These are the aligned remote sensing features and simulated features, respectively.

[0074] This mechanism eliminates the numerical scale differences between remote sensing and simulated features caused by different data sources and extraction methods through a dual operation of standardizing the unified scale and adjusting the distribution using learnable parameters. It also corrects feature distribution offsets through flexible adjustments, ultimately resulting in aligned features. and Being in the same semantic space, they are comparable in terms of feature dimensions, numerical distribution, and physical semantics, laying the foundation for the relevance calculation of the subsequent cross-modal attention fusion module.

[0075] Step S3 includes the following steps:

[0076] S31. Regarding the aligned remote sensing features By querying the projection matrix Dimension mapping and semantic extraction are performed to obtain the query vector Q, as shown in the formula:

[0077] ,

[0078] in, This is a bias term used to introduce nonlinear offset, which enhances the representational ability of key wind field information in remote sensing features.

[0079] Secondly, regarding the aligned simulated features via the key projection matrix Sum projection matrix We perform feature transformations to obtain the key vector K and value vector V, as shown in the following formulas:

[0080] ,

[0081] ,

[0082] in and As a bias term, the key vector K focuses on the physical structure features in the numerical simulation data, while the value vector V retains the detailed information in the simulation data that can be used to supplement sparse areas of the remote sensing data.

[0083] S32. Due to the different sources, imaging mechanisms, and physical modeling methods of remote sensing data and numerical simulation data, there are significant inconsistencies in their feature space distribution, semantic expression, and scale range. Direct feature fusion can easily lead to semantic mismatch and fusion instability, thus affecting the accuracy of 3D wind field prediction. This invention introduces a cross-modal semantic alignment step after bimodal encoding. By constructing a similarity calculation method based on an attention mechanism, it achieves consistent alignment of remote sensing features and numerical simulation features in the semantic space.

[0084] To measure the semantic relevance between the query vector and the key vector, the dot product similarity is calculated:

[0085] ,

[0086] Where S represents the cross-modal semantic similarity matrix. This represents the transpose of the key vector matrix. To prevent the dot product result from becoming too large and causing gradient instability when the feature dimension is large, the similarity result is scaled:

[0087] ,

[0088] in This represents the scaled semantic similarity matrix. The feature dimension of the key vector. To balance the numerical scaling effects of different dimensions and stabilize the training process, the scaled similarity matrix is ​​normalized using Softmax.

[0089] ,

[0090] Where A represents the cross-modal semantic alignment weight matrix;

[0091] We obtain cross-modal alignment features by using a semantic weight matrix to perform a weighted summation of the value vectors.

[0092] ,

[0093] in This represents the cross-modal alignment feature representation obtained under the semantic guidance of numerical simulation for remote sensing modes. This feature simultaneously incorporates remote sensing observation information and numerical simulation physical semantic information, achieving cross-modal semantic alignment.

[0094] S33. The scaled similarity is converted into attention weights using the sigmoid function. The attention weights are driven by the real-time content and contextual features of the input data, and dynamically reflect the relative reliability of the aligned remote sensing features and the aligned simulated features.

[0095] S34. After completing cross-modal semantic alignment, the remote sensing modal features have obtained semantic guidance information from the numerical simulation modality, but the two still exist as independent features. If only a single modal feature is used for prediction, it is difficult to fully utilize the complementary advantages of the two modalities in terms of observation accuracy and physical consistency; if they are simply spliced ​​and fused, redundant information is easily introduced and noise interference is amplified.

[0096] Therefore, this invention proposes a cross-modal fusion step, which constructs a joint feature representation that combines remote sensing observation information and numerical simulation physical constraints through adaptive fusion modeling of aligned features.

[0097] To avoid excessively weakening of the original semantic information of remote sensing during cross-modal alignment, a residual fusion mechanism is introduced:

[0098] ,

[0099] in For residual fusion characteristics, For cross-modal alignment features, These are the original features of remote sensing.

[0100] To further model the higher-order nonlinear coupling relationship between the two modes, a nonlinear mapping is performed on the residual fusion features:

[0101] ,

[0102] in This represents the final joint features after cross-modal fusion; Represents the fusion mapping weight matrix; Indicates the corresponding bias term; This represents a non-linear activation function.

[0103] To enhance the ability to distinguish the importance of different fusion feature channels, a channel weight adjustment mechanism is introduced for the fusion features:

[0104] ,

[0105] ,

[0106] in This represents the importance weight vector for each channel; Represents the weight mapping matrix; This indicates an element-wise weighted operation; This represents the fusion feature after weight adjustment;

[0107] Finally, the cross-modal joint feature representation is obtained. :

[0108] ,

[0109] This fusion feature serves as the input to the subsequent 3D wind field prediction and decoding module, representing the integrated expression of remote sensing observation information and numerical simulation physical constraint information.

[0110] S35. After multimodal fusion, the explicit correlation between features and their corresponding spatiotemporal locations is often ignored, making it difficult for the model to accurately depict the spatiotemporal continuity and dynamic evolution of the wind field, thus affecting the prediction accuracy.

[0111] Therefore, after cross-modal fusion, this invention introduces a spatiotemporal coordinate guidance mechanism, which explicitly introduces spatiotemporal location information to perform position-aware enhanced modeling of the fused features;

[0112] The unified scene representation is combined with spatiotemporal coordinate information. The spatial coordinate information comes from the pixel location of the remote sensing image or the spatial index (x, y, z) of the numerical simulation grid node, and the temporal coordinate information comes from the temporal index corresponding to the time step of the numerical simulation. Therefore, each fused feature unit corresponds to a set of spatiotemporal coordinates:

[0113] ,

[0114] Where x and y represent the horizontal spatial coordinates. Indicates the height layer index or the actual height value. Indicates a time step index or timestamp;

[0115] To establish the correspondence between spatial location and feature information, the fused features are concatenated with spatiotemporal coordinate information: ,

[0116] Concat represents the concatenation operation. This indicates that the cross-modal joint feature representation is finally obtained after the cross-modal attention fusion module. This indicates the features after splicing.

[0117] Step S4 includes the following steps: S41. Construct a five-layer fully connected neural network, with 200 neurons in each layer. Use the Tanh activation function to alleviate the vanishing gradient problem, and then concatenate the features. Input network performs layer-by-layer feature transformation: , , , , , Among them, hi W represents the output activation value of the i-th hidden layer, where i = 1, 2, 3, 4. i and b i Let be the weight matrix and bias vector of the i-th layer, respectively. Tanh(·) is the hyperbolic tangent activation function, with an output range between [-1, 1], which helps alleviate the gradient vanishing problem. The final output is a three-dimensional wind speed vector, which includes wind speed components along the x, y, and z directions; S42, Introduced wind profile constraint loss The power-law model ensures the reasonableness of the wind speed distribution at different altitudes in the prediction results, and the wind profile constraint loss... This is achieved by calculating the deviation between the predicted average wind speed and the wind speed at the reference height using a model. , in, For a dual-modal physical information fusion neural network model in high-level Predicted average wind speed, For reference height The average wind speed at that location This is the surface roughness index, with a typical value range of 0.1 to 0.3. The number of height layers involved in the calculation; S43. Introducing coherence function constraint loss The correlation prediction accuracy of wind speed fluctuations at different spatial locations is evaluated based on the Davenport coherence function model, and the coherence function constraint loss is used. This is achieved by calculating the deviation between the predicted coherence function value and the theoretical value: , , in, The coherence function value predicted by the physical information fusion neural network model. The coherence function value is derived from the theory of physical information fusion neural network model. Let J be the Euclidean distance between points J and K. and These are the horizontal coherence attenuation coefficient and the vertical coherence attenuation coefficient, respectively. and Let be the average wind speeds at heights j and k, respectively; exp represents the natural exponential function, y j y k Let z be the horizontal coordinates of the j-th observation point and the k-th observation point, respectively. j z kLet be the vertical coordinates of the j-th observation point and the k-th observation point, respectively, and let f be the frequency of the wind speed fluctuation. This ensures that the frequency domain energy distribution of the wind speed fluctuation conforms to physical laws. The wind spectrum constraint loss... This is achieved by calculating the deviation between the predicted power spectral density and the theoretical spectral value: , in The one-sided power spectral density predicted by the model. This is the theoretical value for the Kaimal spectrum. Let V be the variance of wind speed fluctuations. Where i represents the number of frequency points to consider, and i is the relevant parameter. S45. The data fitting loss is weighted and combined with the wind profile constraint loss, coherence function constraint loss, and wind spectrum constraint loss to form a total loss function. The decoder parameters are optimized through backpropagation, and a three-dimensional wind speed vector containing wind speed components in the x, y, and z directions is output.

[0118] Step S5 includes the following steps: S51. The stochastic gradient descent optimization algorithm is adopted. Based on the total loss function, the backpropagation algorithm is used to traverse the bimodal encoder, cross-modal attention fusion module and physical enhancement decoder layer by layer to calculate the gradient of all learnable parameters, including the data fitting loss gradient, wind profile constraint loss gradient, coherence function constraint loss gradient and wind spectrum constraint loss gradient. S52. Update the convolution kernel parameters and projection matrix parameters of the bimodal encoder, the query projection matrix, key projection matrix, and bias term parameters of the cross-modal attention fusion module, and the weight matrix and bias vector of the physical enhancement decoder according to the preset learning rate. Update the bimodal encoder parameters: adjust all learnable parameters according to the gradient descent direction, and the update formula is: , in This represents the parameters to be updated. These are the original parameters. for gradient, Let be the learning rate for the k-th training round; Update the parameters of the cross-modal attention fusion module: Adjust the query projection matrix according to the update formula. Key projection matrix Value projection matrix The updated parameters are as follows: , , , in This represents the updated query projection matrix. This represents the updated key projection matrix. This represents the projection matrix of the updated values. , , These are the gradients of the query projection matrix, the key projection matrix, and the value projection matrix, respectively. Update the physical enhancement decoder parameters: Adjust the weight matrix W of the five fully connected neural networks according to the update formula. dec1 W dec2 W dec3 W dec4 W dec5 ; S53. Iterate the training until the total loss function converges, and verify the prediction accuracy of the model in the sparse measurement data scenario to ensure that the prediction results meet both the data fitting accuracy requirements and the physical law constraints. S54. Save the optimized model parameters to form a closed-loop prediction model that can be directly used for 3D wind field prediction in any spatiotemporal coordinate. The closed-loop prediction model supports resolution-free on-demand prediction. Save all parameters of the converged bimodal encoder, cross-modal attention fusion module and physical enhancement decoder in binary format to generate an independent model parameter file. The file contains parameter name, dimension, value and optimized learning rate. The closed-loop prediction model supports inputting arbitrary spatiotemporal coordinates (x, y, z, t), where x and y are horizontal spatial coordinates (unit: m), z is the height coordinate (unit: m), and t is the time coordinate (unit: s). It can adaptively adjust the output resolution according to actual needs and output the three-dimensional wind speed vector (u, v, w) at the corresponding coordinates, realizing on-demand prediction with flexible resolution.

[0119] This invention also provides a three-dimensional wind field prediction system based on multimodal complementary fusion learning, including a data acquisition module, a bimodal encoding module, a cross-modal fusion module, a physical enhancement decoding module, and a model optimization module, used to execute the following methods: The data acquisition module is used to collect and preprocess remote sensing observation data and numerical simulation data, and output a standardized data source that meets the input format requirements; The bimodal encoding module includes a remote sensing data encoder and a numerical simulation data encoder, which are used to extract depth features from the two data sources respectively, and to achieve feature space standardization through a feature alignment mechanism; The cross-modal fusion module is used to dynamically evaluate the reliability of bimodal features through an attention mechanism, generate adaptive fusion weights and complete feature integration, and output a unified scene representation.

[0120] The physical enhancement decoding module is used to receive scene representation and spatiotemporal coordinate stitched data, and output three-dimensional wind speed prediction results through a fully connected network with physical constraints.

[0121] The model optimization module is used to iteratively adjust the parameters of each module based on the prediction bias, optimize the total loss function, and ensure the model's prediction accuracy and physical consistency.

[0122] The present invention also provides an electronic device, including a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method.

[0123] The present invention also provides a storage medium storing a computer program or instructions that, when the computer program or instructions are run on a computer, execute the steps of the method described.

[0124] This method innovates a dual-modal parallel coding architecture, which uses lightweight two-dimensional / three-dimensional convolutional networks to adapt to remote sensing observation (two-dimensional discrete) and numerical simulation (three-dimensional continuous) data respectively. The feature alignment mechanism solves the modal heterogeneity problem and efficiently extracts complementary features.

[0125] A cross-modal dynamic attention fusion mechanism is proposed, which dynamically evaluates the reliability of the data source through a query-key-value attention architecture and generates adaptive fusion weights, overcoming the limitations of traditional fixed-weight fusion and improving adaptability to complex scenarios.

[0126] We construct a frequency domain physical constraint-enhanced decoding paradigm that deeply integrates the wind field-specific power-law model, coherence function, and wind spectrum model into the loss function, ensuring the physical consistency and interpretability of the prediction results, while supporting resolution-free on-demand prediction.

[0127] A closed-loop optimization system that is both data efficient and physically consistent is formed. The system alleviates the data sparsity problem through multimodal complementarity and ensures the reliability of predictions through physical constraints, thereby significantly improving the prediction accuracy in scenarios where the measurement point coverage is less than 30%.

[0128] Beneficial Effects: This invention significantly improves the prediction accuracy of 3D wind fields under sparse data conditions through multimodal data complementarity and deep fusion of physical information, solving the problems of insufficient physical consistency in traditional data-driven models and poor data adaptability in pure physical models. Its lightweight network architecture ensures real-time prediction, the dynamic attention fusion mechanism enhances adaptability to complex scenarios, and frequency domain physical constraints ensure the physical reliability of the prediction results. This method and system can provide high-precision technical support for wind energy resource assessment and wind farm operation scheduling in the wind power industry, promoting the intelligent development of the wind power industry and possessing significant engineering application value and scientific research significance. Attached Figure Description

[0129] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0130] Figure 1 This is an overall framework diagram of a dual-modal physical information fusion neural network (DM-PIFNN) provided in an embodiment of this application.

[0131] Figure 2 This is a dual-mode encoder provided in one embodiment of this application.

[0132] Figure 3 This is a cross-modal attention fusion module provided in one embodiment of this application.

[0133] Figure 4 This is a physical enhancement decoder provided in one embodiment of this application. Detailed Implementation

[0134] This embodiment provides a three-dimensional wind field prediction method based on multimodal complementary fusion learning, including the following steps:

[0135] S1. Acquire remote sensing observation data and numerical simulation data. The remote sensing observation data is in two-dimensional image format and contains information related to surface or low-altitude wind speed acquired by satellite or radar equipment. The numerical simulation data is in three-dimensional grid format and comes from meteorological models or computational fluid dynamics simulations. It contains information related to three-dimensional wind fields from the ground to the upper atmosphere.

[0136] S2. Based on the remote sensing observation data and numerical simulation data, depth features of the two data sources are extracted by a dual-modal encoder, and feature space alignment is achieved through a projection layer to obtain standardized remote sensing features and simulation features.

[0137] S3. Input the standardized remote sensing features and simulated features into the cross-modal attention fusion module, using remote sensing features as queries and simulated features as keys, generate dynamic attention weights by calculating the intermodal correlation, and adaptively fuse the dual-modal features based on the weights to obtain a unified scene representation;

[0138] S4. The unified scene representation is concatenated with the spatiotemporal coordinates (x, y, z, t) to be predicted and input into the physical enhancement decoder. The decoder adopts a five-layer fully connected neural network structure. During the training process, the wind profile power law model, Davenport coherence function model and Kaimal wind spectrum model are integrated as frequency domain physical constraints, and the three-dimensional wind speed vector at the corresponding spatiotemporal coordinates is output.

[0139] S5. Based on the deviation between actual observation data and prediction results, iteratively optimize the parameters of the dual-modal encoder, cross-modal attention fusion module and physical enhancement decoder to form a data-efficient and physically consistent closed-loop prediction model.

[0140] Step S2 includes the following steps:

[0141] S21. Preprocess the remote sensing observation data to obtain data with the input format [B,C,H,W], where B is the batch size, C is the number of input channels, corresponding to wind speed, wind direction angle and data quality label, and H and W are the image height and width, respectively.

[0142] S22. The preprocessed remote sensing observation data is processed using a lightweight encoder containing two layers of two-dimensional convolution. Basic spatial feature extraction, feature enhancement representation, feature aggregation and projection normalization operations are performed sequentially to output 32-dimensional remote sensing features.

[0143] S23. Preprocess the numerical simulation data to obtain an input format of [B,F,L] x ,L y ,L z The data is as follows: B is the batch size, F=5 is the number of physical variables, including horizontal wind speed u, horizontal wind speed v, vertical wind speed w, temperature T, and pressure p, L. x L y L z These represent the number of grid points in the x-direction, the number of grid points in the y-direction, and the number of vertical layers, respectively.

[0144] S24. The preprocessed numerical simulation data is processed using a lightweight encoder containing two layers of three-dimensional convolution. Three-dimensional spatial feature extraction, feature deepening representation, three-dimensional global average pooling and projection normalization operations are performed sequentially to output 32-dimensional simulation features.

[0145] S25. Spatial alignment of remote sensing features and simulated features is achieved through learnable linear transformation and layer normalization. The alignment transformation ensures comparability of the two features by standardizing the feature scale and flexibly adjusting the feature distribution.

[0146] Step S3 includes the following steps:

[0147] S31. Extract query information from remote sensing features by querying the projection matrix, extract matching information from simulated features by key projection matrix, and introduce bias terms to achieve nonlinear transformation.

[0148] S32. Calculate the dot product similarity between the query vector and the key vector, and perform scaling to stabilize the training process;

[0149] S33. The scaled similarity is converted into attention weights using the sigmoid function. The weights are driven by the real-time content and contextual features of the input data, and dynamically reflect the relative reliability of the two data sources.

[0150] S34. The remote sensing features and simulation features are weighted and fused using the attention weights to generate a unified scene representation containing dual-modal complementary information.

[0151] S35. Combine the unified scene representation with the spatiotemporal coordinate information to establish the correspondence between spatial location and feature information, providing an input basis for decoding and prediction.

[0152] Step S4 includes the following steps:

[0153] S41. Construct a five-layer fully connected neural network, with 200 neurons in each layer. Use the Tanh activation function to alleviate the gradient vanishing problem. Input the concatenated features into the network for layer-by-layer feature transformation.

[0154] S42. Introduce wind profile constraint loss. Based on the power law model, ensure the reasonableness of the wind speed distribution at different heights in the prediction results. The constraint loss is achieved by calculating the deviation between the average wind speed predicted by the model and the wind speed at the reference height.

[0155] S43. Introduce coherence function constraint loss and evaluate the correlation prediction accuracy of wind speed fluctuations at different locations in space based on the Davenport coherence function model. The constraint loss is achieved by calculating the deviation between the predicted coherence function value and the theoretical value.

[0156] S44. Introduce wind spectrum constraint loss. Based on the Kaimal spectrum model, ensure that the frequency domain energy distribution of wind speed fluctuations conforms to physical laws. The constraint loss is achieved by calculating the deviation between the predicted power spectral density and the theoretical spectral value.

[0157] S45. The data fitting loss is weighted and combined with the above three physical constraint losses to form a total loss function. The decoder parameters are optimized through backpropagation, and a three-dimensional wind speed vector containing wind speed components in the x, y, and z directions is output.

[0158] Step S5 includes the following steps:

[0159] S51. The stochastic gradient descent optimization algorithm is adopted to calculate the gradient of the parameters of each module based on the total loss function;

[0160] S52. Update the convolution kernel parameters and projection matrix parameters of the dual-modal encoder, the query projection matrix, key projection matrix and bias term parameters of the cross-modal attention fusion module, and the weight matrix and bias vector of the physical enhancement decoder according to the preset learning rate.

[0161] S53. Iterate training until the loss function converges, verify the prediction accuracy of the model in sparse measurement data scenarios, and ensure that the prediction results meet both the data fitting accuracy requirements and physical constraints.

[0162] S54. Save the optimized model parameters to form a closed-loop prediction model that can be directly used for three-dimensional wind field prediction in any spatiotemporal coordinates. The model supports on-demand prediction with flexible resolution.

[0163] like Figure 1 As shown, conv2D represents a two-dimensional convolutional layer, BN represents batch normalization, ReLU represents an activation function, GAP represents global average pooling, 2D CNN represents a two-dimensional convolutional neural network, 3D CNN represents a three-dimensional convolutional neural network, Conv3d represents a three-dimensional convolutional layer, Linear represents a linear layer, cross-modal attention fusion module represents a cross-modal attention fusion module, and concat represents feature concatenation. Figure 1 The diagram shows the overall framework of the Dual-Modal Physical Information Fusion Neural Network (DM-PIFNN). The framework accepts two types of data input: first, remote sensing observation data, typically presented in two-dimensional image format, containing surface or low-altitude wind speed information acquired by satellites, radars, and other equipment; second, numerical simulation data, from meteorological models (such as WRF) or computational fluid dynamics (CFD) simulations, presented in three-dimensional grid format, containing complete three-dimensional wind field information from the ground to the upper atmosphere. These two types of data represent different perspectives on the wind field—remote sensing data reflects actual observations but has limited spatial resolution, while simulation data contains a complete three-dimensional structure but contains model errors, as shown in Table 1.

[0164] Table 1

[0165]

[0166] Secondly, the framework's processing flow follows a logical chain of encoding-fusion-decoding: the dual-modal encoder first extracts feature representations from the two data sources respectively, then intelligently integrates these features through a cross-modal fusion module, and finally converts the fused features into specific wind field predictions through a physically enhanced decoder. Notably, the decoder not only receives the fused features but also the spatiotemporal coordinates (x, y, z, t) to be predicted, enabling it to query wind speed values ​​at any location and time.

[0167] The mathematical expression of the entire architecture can be summarized as follows: given an input data pair and query coordinates c, the model outputs the three-dimensional wind speed U at those coordinates. This process is achieved by learning three key mappings: the encoder maps the raw data to the feature space, the fusion unit maps different features to a unified representation, and the decoder maps the fused features to physical quantities.

[0168] likeFigure 2 As shown, conv2D represents a two-dimensional convolutional layer, BN represents batch normalization, ReLU represents the activation function, GAP represents global average pooling, 2D CNN represents a two-dimensional convolutional neural network, 3D CNN represents a three-dimensional convolutional neural network, and Conv3d represents a three-dimensional convolutional layer. Figure 2 This is a concise and efficient bimodal encoder. The core objective of this encoder is to convert two inputs—remote sensing observation data (two-dimensional image format) and numerical simulation data (three-dimensional mesh format)—which have different physical meanings and heterogeneous data structures, into feature vectors with uniform dimensions and comparable semantics, thus providing a foundation for subsequent cross-modal fusion.

[0169] Table 2

[0170]

[0171] Table 2 shows the format, processing method, and output form of the dual-modal input data in DM-PIFNN. Remote sensing observation data is input into the network in the form of two-dimensional images, and spatial wind field representation is obtained through convolutional feature extraction; numerical simulation data is input in the form of three-dimensional grids, and vertical wind field structure information is extracted through a three-dimensional feature encoding network. The two types of features are uniformly represented and learned in the fusion module, providing high-quality prior constraints for subsequent three-dimensional wind field inversion.

[0172] Considering the limited computational resources and small training data scale in practical applications, this method abandons complex deep network structures and instead adopts lightweight shallow convolutional networks. The remote sensing encoder contains only two 2D convolutional layers, and the analog encoder contains only two 3D convolutional layers, with the output feature dimension of each encoder being uniformly 32-dimensional. This design significantly reduces model complexity and improves training stability and inference speed while ensuring basic feature extraction capabilities.

[0173] The overall architecture adopts a dual-stream parallel processing mode: two encoders run independently, each processing its corresponding data modality; at the end of the encoder, feature alignment is achieved through simple linear projection and layer normalization. The final output 32-dimensional feature vector will be directly input into the subsequent cross-modal attention fusion module.

[0174] Figure 3 In this context, Q (Query) is the query vector, K (Key) is the key vector, V (Value) is the value vector, and FC... Q FC K FC V These are the fully connected layers that generate Q, K, and V, respectively. Figure 3The cross-modal attention fusion module, which transforms feature dimensions, is the core innovative component of this method, undertaking the crucial task of intelligently integrating bimodal information. In practical applications of wind field prediction, remote sensing data and numerical simulation data each have their advantages and disadvantages. Remote sensing data reflects real measurements but is limited by spatial resolution and weather conditions, while numerical simulations contain physical laws but suffer from model errors and parameterization uncertainties. Traditional methods typically employ simple feature stitching or weighted averaging, which cannot adapt to the dynamic changes in data reliability under different spatiotemporal conditions.

[0175] This module innovatively introduces an attention mechanism, dynamically evaluating the relative reliability of two data sources in different scenarios through learning, thereby achieving adaptive intelligent fusion. Specifically, the module uses aligned remote sensing features as queries and simulated features as key-value pairs, generating attention weights by calculating the correlation between the two. These weights determine the degree of dependence on the two features during the fusion process.

[0176] like Figure 4 As shown, the Neural network module represents the neural network module, X / Y / Z are spatial coordinates (X: longitude / horizontal, Y: latitude / horizontal, Z: altitude / vertical), T is the time coordinate (the time information corresponding to the wind field), F is the feature input, U is the wind speed (the core prediction result of the model output, usually containing three-dimensional wind speed components), and S is the wind speed. ij Let S be the coherence function of wind speed at two points (describing the correlation of wind speed fluctuations at any two points in space). jk The power spectral density of wind speed (describes the energy distribution of wind speed fluctuations at different frequencies), the physical information module, and the wind profile model (a physical model describing the variation of wind speed with height, such as a power-law wind profile).

[0177] The coherence function model (a physical model describing the correlation between wind speeds at two points in space, such as the Davenport coherence function) and the feedback module (which feeds back the constraint results from the physical information module to the neural network module to optimize model training) are the core prediction modules of this method. The graph-based physical enhancement decoder is responsible for converting the fused multimodal features into physically meaningful 3D spatiotemporal wind field predictions. The input is the concatenated features. This decoder uses a five-layer fully connected neural network, with each layer containing 200 neurons.

[0178] Expected verification results:

[0179] Within the 6 / 12 / 24-hour full-time period, the path error is reduced by 25%~32% compared to ECMWF HRES, and the center wind speed RMSE is reduced by 45%~55% compared to ECMWF HRES.

[0180] Training set: 120 typhoon cases in the Northwest Pacific from 2010 to 2020 (including tropical storms to super typhoons, covering westward, northward, and turning paths), extracting typhoon center location, maximum wind speed, and minimum air pressure data at 6-hour intervals from CMA to BST; matching the three-dimensional wind field (U / V components), air pressure, and temperature data (0.25°×0.25° resolution, 1-hour lead time, 200~1000hPa pressure layer) of the corresponding region in ERA5 (500km×500km radius of typhoon center).

[0181] Validation set: 30 typical typhoons from 2021 to 2023 that were not included in the training set, with data types consistent with the training set. The results are shown in Table 3.

[0182] Table 3

[0183]

[0184] This embodiment also provides a three-dimensional wind field prediction system based on multimodal complementary fusion learning, including a data acquisition module, a bimodal encoding module, a cross-modal fusion module, a physical enhancement decoding module, and a model optimization module, used to execute the following methods: The data acquisition module is used to collect and preprocess remote sensing observation data and numerical simulation data, and output standardized data sources that meet the input format requirements; The bimodal encoding module includes a remote sensing data encoder and a numerical simulation data encoder, which are used to extract depth features from the two data sources respectively, and to achieve feature space standardization through a feature alignment mechanism; The cross-modal fusion module is used to dynamically evaluate the reliability of bimodal features through an attention mechanism, generate adaptive fusion weights and complete feature integration, and output a unified scene representation.

[0185] The physical enhancement decoding module is used to receive scene representation and spatiotemporal coordinate stitched data, and output three-dimensional wind speed prediction results through a fully connected network with physical constraints.

[0186] The model optimization module is used to iteratively adjust the parameters of each module based on the prediction bias, optimize the total loss function, and ensure the model's prediction accuracy and physical consistency.

[0187] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory stores program code that, when executed by the processor, causes the processor to perform the steps of the method.

[0188] This embodiment also provides a storage medium storing a computer program or instructions that, when run on a computer, execute the steps of the method described.

[0189] This invention provides a three-dimensional wind field prediction method and system based on multimodal complementary fusion learning. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A three-dimensional wind field prediction method based on multimodal complementary fusion learning, characterized in that, Includes the following steps: Step S1: Acquire remote sensing observation data and numerical simulation data. The remote sensing observation data is in two-dimensional image format and contains information related to surface or low-altitude wind speed acquired by satellite or radar equipment. The numerical simulation data is in three-dimensional grid format and comes from meteorological models or computational fluid dynamics simulations, containing information related to three-dimensional wind fields from the ground to the upper atmosphere. Step S2: Based on the remote sensing observation data and numerical simulation data, the depth features of the two data sources are extracted by a dual-modal encoder, and the feature space is aligned by a projection layer to obtain the standardized remote sensing features and simulation features. Step S3: Input the standardized remote sensing features and simulated features into the cross-modal attention fusion module, using remote sensing features as queries and simulated features as keys, generate dynamic attention weights by calculating intermodal correlations, and adaptively fuse the dual-modal features based on the weights to obtain a unified scene representation; Step S4: The unified scene representation is concatenated with the spatiotemporal coordinates (x, y, z, t) to be predicted and input into the physical enhancement decoder. The physical enhancement decoder adopts a five-layer fully connected neural network structure. During the training process, the wind profile power law model, Davenport coherence function model and Kaimal wind spectrum model are integrated as frequency domain physical constraints, and the three-dimensional wind speed vector at the corresponding spatiotemporal coordinates is output. Step S5: Based on the deviation between the actual observation data and the prediction results, iteratively optimize the parameters of the bimodal encoder, the cross-modal attention fusion module, and the physical enhancement decoder to form a closed-loop prediction model, denoted as the bimodal physical information fusion neural network model.

2. The method as described in claim 1, characterized in that, Step S2 includes the following steps: S21. Preprocess the remote sensing observation data. First, perform channel normalization on the raw two-dimensional remote sensing data to obtain normalized remote sensing data. : ， in and Let represent the mean and standard deviation of the C-th channel, respectively. The raw remote sensing data, The data is the normalized remote sensing data; Based on a preset height weight mapping function The normalized remote sensing data is subjected to highly perceptual weighting processing: ， ， Where e represents the natural constant. This represents the preprocessed remote sensing data. This indicates the weight of the influence of different altitude layers on remote sensing observation data. The parameters of the height weighting function are represented by H, where H represents the image height. This represents element-wise weighted operations: Mapping function for height weight Output and Element-wise multiplication is used to achieve weighted adjustment of the height dimension, thereby introducing height structure information into the two-dimensional remote sensing data at the channel semantic level. The final output is remote sensing feature data containing height semantic information, with the data format [B,C,H,W], where B is the batch size, C is the number of input channels, corresponding to wind speed, wind direction angle and data quality label, and W is the image width, which serves as the remote sensing modality input of the dual-modal encoder. S22, The dual-mode encoder includes a remote sensing data encoder and a numerical simulation data encoder; A remote sensing data encoder is used to process preprocessed remote sensing data. The remote sensing data encoder consists of a lightweight two-layer two-dimensional convolutional architecture, namely a first convolutional layer and a second convolutional layer. The first convolutional layer is used to extract basic spatial features from the preprocessed remote sensing data. Perform two-dimensional convolution operation Extracting wind field spatial texture and local wind speed gradient, the formula is: ， in, The kernel size is [size]. Step size For fill value, These are the convolution kernel parameters for the first convolutional layer. Let i be the bias vector, and j be the positional weight parameters of the first convolutional layer. and These represent the height and width of the image output features, respectively; Then, activation functions are used to enhance the nonlinear representation, and the output features of the first convolutional layer are applied. Applying the ReLU activation function, the formula is as follows: ， The second convolutional layer is used to deepen the feature representation: after activation, the features are processed... Perform a second two-dimensional convolution to aggregate local features and extract more abstract spatial correlation features of the wind field. The formula is as follows: ， in, These are the features after activation in the first convolutional layer. Step size For fill value, These are the kernel parameters for the second convolutional layer. The bias vector is the final output. ; The output of the second convolutional layer Apply the ReLU activation function again, as shown in the formula: ， Finally, the features after activation of the second layer are analyzed. Perform two-dimensional global average pooling operation The two-dimensional feature map is compressed into a one-dimensional feature vector, preserving global spatial statistical information. The formula is as follows: ， in This represents the feature vector after pooling; Pooled feature vectors Spatial dimensional differences are eliminated by calculating the global mean, followed by a linear projection layer. The feature vectors are mapped to fixed dimensions, and then normalized to eliminate feature distribution offset, finally outputting 32-dimensional standardized remote sensing features. S23. Perform high-structure compression preprocessing on the numerical simulation data. First, perform layered weighted summation on the numerical simulation data along the height dimension: ， in The weight coefficient corresponding to height z. These are the original values ​​of the numerical simulation data at (x, y, z) in three-dimensional space. This is highly compressed two-dimensional simulation data. The number of layers in the height dimension; The weighting coefficients are determined by the wind speed gradient along the vertical direction. ， in Let z be the vertical gradient of the wind speed U at height z; the input format is [B,F,L] x ,L y ,L z The data, where F=5 represents the number of physical variables, including horizontal wind speed u, horizontal wind speed v, vertical wind speed w, temperature T, and pressure p. The number of grid points in the x-direction. The number of grid points in the y-direction. Indicates the number of vertical layers; The high consistency constraint loss is constructed using the following formula. : ， in and These are the spatial gradients of the remote sensing data and the simulated data, respectively, where h is the vertical height dimension of the wind field; S24. The preprocessed numerical simulation data is processed using a numerical simulation data encoder containing two layers of three-dimensional convolution. The first layer of the numerical simulation data encoder performs the following three-dimensional convolution operation on the input data: ， in, This represents the feature volume output by the first 3D convolutional layer. This refers to the preprocessed numerical simulation data; Represents a three-dimensional convolution operation; Represents the three-dimensional convolution kernel weight matrix; Indicates the bias term; Apply an activation function to the output of the first 3D convolutional layer: ， in This represents the activated three-dimensional feature volume; Perform a 3D convolution operation again on the activated feature volume: ， in This represents the output feature volume of the second-layer 3D convolution. This represents the weight matrix of the second-layer convolutional kernel. Indicates the corresponding bias term; Apply an activation function to the output of the second 3D convolution layer: ， in This represents the final three-dimensional spatial feature representation result; Perform a three-dimensional global average pooling operation on the feature volume: ， in This represents the compressed global three-dimensional statistical feature vector. The primitive elements representing the three-dimensional feature volume; Projecting the global 3D statistical feature vector through a linear mapping layer: ， in This represents the final eigenvector of the numerical simulation mode. Represents the projection weight matrix. This represents the projection bias term, and standardization is performed on the eigenvectors: ， in This represents the standardized feature vector. This represents the mean of each dimension of the feature vector. Indicates the corresponding standard deviation; S25. Spatial alignment of remote sensing features and simulated features is achieved through learnable linear transformations and layer normalization. Spatial alignment of remote sensing and simulated features is achieved through a dual-modal feature adaptive alignment mechanism. First, the statistical parameters of the remote sensing and simulated features are calculated separately: ， ， ， ， in and , where are the mean and standard deviation of the remote sensing features, and B is the batch size. Let b be the remote sensing feature of the b-th sample and d-th dimension. and These are the mean and standard deviation of the simulated features, respectively. Let b be the numerical simulation feature of the b-th sample and d-th dimension; Then, feature distribution matching is achieved through flexible adjustment using independent learnable affine parameters, as shown in the formula: ， ， in, Learnable affine transformation parameters for remote sensing and simulated features. These are remote sensing features obtained by feature extraction, aggregation, and normalization of preprocessed data using a dual-modal encoder. , These are the aligned remote sensing features and simulated features, respectively.

3. The method as described in claim 2, characterized in that, Step S3 includes the following steps: S31. Regarding the aligned remote sensing features By querying the projection matrix Dimension mapping and semantic extraction are performed to obtain the query vector Q, as shown in the formula: ， in, For bias terms; Secondly, regarding the aligned simulated features via the key projection matrix Sum projection matrix We perform feature transformations to obtain the key vector K and value vector V, as shown in the following formulas: ， ， in and For bias terms; S32. Calculate dot product similarity: ， Where S represents the cross-modal semantic similarity matrix. This represents the transpose of the key vector matrix; Scaling the similarity results: ， in This represents the scaled semantic similarity matrix. This represents the feature dimension of the key vector; Softmax normalization is applied to the scaled similarity matrix: ， Where A represents the cross-modal semantic alignment weight matrix; We obtain cross-modal alignment features by using a semantic weight matrix to perform a weighted summation of the value vectors. ， in This represents the cross-modal aligned feature representation of remote sensing modalities obtained under the semantic guidance of numerical simulation. S33. The scaled similarity is converted into attention weights using the sigmoid function. The attention weights are driven by the real-time content and contextual features of the input data, and dynamically reflect the relative reliability of the aligned remote sensing features and the aligned simulated features. S34. Introduce a residual fusion mechanism: ， in For residual fusion characteristics, For cross-modal alignment features, These are the original features of remote sensing. Perform nonlinear mapping on the residual fusion features: ， in This represents the final joint features after cross-modal fusion; Represents the fusion mapping weight matrix; Indicates the corresponding bias term; Represents a nonlinear activation function; Introduce a channel weight adjustment mechanism for the fused features: ， ， in This represents the importance weight vector for each channel; Represents the weight mapping matrix; This indicates an element-wise weighted operation; This represents the fusion features after weight adjustment; Finally, the cross-modal joint feature representation is obtained. : ； S35. Introduce a spatiotemporal coordinate guidance mechanism to perform position-aware enhancement modeling of fused features by explicitly introducing spatiotemporal location information. The unified scene representation is combined with spatiotemporal coordinate information. The spatial coordinate information comes from the pixel location of the remote sensing image or the spatial index (x, y, z) of the numerical simulation grid node, and the temporal coordinate information comes from the temporal index corresponding to the time step of the numerical simulation. Each fused feature unit corresponds to a set of spatiotemporal coordinates: ； The fused features are then combined with spatiotemporal coordinate information: ， Concat represents the concatenation operation. This indicates that the cross-modal joint feature representation is finally obtained after the cross-modal attention fusion module. This indicates the features after splicing.

4. The method as described in claim 3, characterized in that, Step S4 includes the following steps: S41. Construct a five-layer fully connected neural network and concatenate the features. Input network performs layer-by-layer feature transformation: ， ， ， ， ， Among them, h i W represents the output activation value of the i-th hidden layer, where i = 1, 2, 3, 4. i and b i Let be the weight matrix and bias vector of the i-th layer, respectively, and let Tanh(·) be the hyperbolic tangent activation function. This is the final output three-dimensional wind speed vector; S42, Introduced wind profile constraint loss The power-law model ensures the reasonableness of the wind speed distribution at different altitudes in the prediction results, and the wind profile constraint loss... This is achieved by calculating the deviation between the predicted average wind speed and the wind speed at the reference height using a model. ， in, For a dual-modal physical information fusion neural network model in high-level Predicted average wind speed, For reference height The average wind speed at that location The surface roughness index, The number of height layers involved in the calculation; S43. Introducing coherence function constraint loss The correlation prediction accuracy of wind speed fluctuations at different spatial locations is evaluated based on the Davenport coherence function model, and the coherence function constraint loss is used. This is achieved by calculating the deviation between the predicted coherence function value and the theoretical value: ， ， in, The coherence function value predicted by the physical information fusion neural network model. The coherence function value is derived from the theory of physical information fusion neural network model. Let J be the Euclidean distance between points J and K. and These are the horizontal coherence attenuation coefficient and the vertical coherence attenuation coefficient, respectively. and Let be the average wind speeds at heights j and k, respectively; exp represents the natural exponential function, y j y k Let z be the horizontal coordinates of the j-th observation point and the k-th observation point, respectively. j z k Let be the vertical coordinates of the j-th observation point and the k-th observation point, respectively, and let f be the frequency of wind speed fluctuation. The wind spectrum constraint loss is... This is achieved by calculating the deviation between the predicted power spectral density and the theoretical spectral value: ， in The one-sided power spectral density predicted by the model. This is the theoretical value for the Kaimal spectrum. Let V be the variance of wind speed fluctuations. Where i represents the number of frequency points to consider, and i is the relevant parameter. S45. The data fitting loss is weighted and combined with the wind profile constraint loss, coherence function constraint loss, and wind spectrum constraint loss to form a total loss function. The decoder parameters are optimized through backpropagation, and a three-dimensional wind speed vector containing wind speed components in the x, y, and z directions is output.

5. The method as described in claim 4, characterized in that, Step S5 includes the following steps: S51. The stochastic gradient descent optimization algorithm is adopted. Based on the total loss function, the backpropagation algorithm is used to traverse the bimodal encoder, cross-modal attention fusion module and physical enhancement decoder layer by layer to calculate the gradient of all learnable parameters, including the data fitting loss gradient, wind profile constraint loss gradient, coherence function constraint loss gradient and wind spectrum constraint loss gradient. S52. Update the convolution kernel parameters and projection matrix parameters of the bimodal encoder, the query projection matrix, key projection matrix, and bias term parameters of the cross-modal attention fusion module, and the weight matrix and bias vector of the physical enhancement decoder according to the preset learning rate. Update the bimodal encoder parameters: adjust all learnable parameters according to the gradient descent direction, and the update formula is: ， in This represents the parameters to be updated. These are the original parameters. for gradient, Let be the learning rate for the k-th training round; Update the parameters of the cross-modal attention fusion module: Adjust the query projection matrix according to the update formula. Key projection matrix Value projection matrix The updated parameters are as follows: ， ， ， in This represents the updated query projection matrix. This represents the updated key projection matrix. This represents the projection matrix of the updated values. , , These are the gradients of the query projection matrix, the key projection matrix, and the value projection matrix, respectively. Update the physical enhancement decoder parameters: Adjust the weight matrix W of the five fully connected neural networks according to the update formula. dec1 W dec2 W dec3 W dec4 W dec5 ; S53. Iterate the training until the total loss function converges, and verify the prediction accuracy of the model in the sparse measurement data scenario to ensure that the prediction results meet both the data fitting accuracy requirements and the physical law constraints. S54. Save the optimized model parameters to form a closed-loop prediction model that can be directly used for 3D wind field prediction in any spatiotemporal coordinate. The closed-loop prediction model supports resolution-free on-demand prediction. Save all parameters of the converged bimodal encoder, cross-modal attention fusion module and physical enhancement decoder in binary format to generate an independent model parameter file. The file contains parameter name, dimension, value and optimized learning rate. The closed-loop prediction model supports inputting arbitrary spatiotemporal coordinates (x, y, z, t) and outputting a three-dimensional wind speed vector (u, v, w) at the corresponding coordinates.

6. A three-dimensional wind field prediction system based on multimodal complementary fusion learning, characterized in that, The system comprises a data acquisition module, a bimodal encoding module, a cross-modal fusion module, a physical enhancement decoding module, and a model optimization module, for performing the method described in any one of claims 1 to 5: the data acquisition module is used to collect and preprocess remote sensing observation data and numerical simulation data, and output a standardized data source that conforms to the input format requirements; the bimodal encoding module includes a remote sensing data encoder and a numerical simulation data encoder, which are used to extract depth features from the two data sources respectively, and to achieve feature space standardization through a feature alignment mechanism; the cross-modal fusion module is used to dynamically evaluate the reliability of the bimodal features through an attention mechanism, generate adaptive fusion weights and complete feature integration, and output a unified scene representation.

7. The system as described in claim 6, characterized in that, The physical enhancement decoding module is used to receive scene representation and spatiotemporal coordinate stitched data, and output three-dimensional wind speed prediction results through a fully connected network with physical constraints.

8. The system as described in claim 7, characterized in that, The model optimization module is used to iteratively adjust the parameters of each module based on the prediction bias, optimize the total loss function, and ensure the prediction accuracy and physical consistency of the model.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing program code that, when executed by the processor, causes the processor to perform the steps of the method as described in any one of claims 1 to 5.

10. A storage medium, characterized in that, It stores a computer program or instructions that, when run on a computer, perform the steps of the method as described in any one of claims 1 to 5.

Images