Computer-implemented method for marine surface wind downscaling

Abstract

A computer-implemented method for ocean surface wind (OSW) downscaling. The method comprises the following steps: obtaining an input image with a group of wind data; an OSW downscaling model based on a TransUNet framework is constructed; generating an output image having a set of resulting wind data from the input image using the OSW downscaling model; and before generating the output image using the OSW downscaling model, training the OSW downscaling model according to a plurality of sets of training samples, where the OSW downscaling model is trained by optimizing model parameters of the OSW downscaling model in the sense of minimizing a loss function to improve resolution.

Description

Technical Field

[0001] The present application relates to the field of atmospheric sciences, and in particular, to a computer-implemented method for downscaling ocean surface winds (OSW). Background Art

[0002] Current OSW products cannot provide reliable and accurate wind fields under high-speed wind conditions (>15m / s). For example, during Typhoon Mangkhut in 2018, the Cross-Calibrated Multi-Platform (CCMP) significantly underestimated wind speeds when wind speeds exceeded 15m / s, compared to the best track datasets (JTWC, CMA, HKO, and TOKYO). Figure 1a-1c Another disadvantage of current OSW products is their low spatial resolution. Such disadvantages limit their application in various fields, such as estimation of wind energy capacity, deployment of offshore wind power stations, and operation of weather forecasting.

[0003] The current limitations are mainly due to the ability of the instruments on board the satellite to detect extreme weather conditions. The performance of the CCMP, which combines OSW measurements from microwave radiometers and scatterometers, is adversely affected by rainy weather. This is why there are few observations near the center of a typhoon, such as Figure 1a-1c At the same time, the current OSW products do not include synthetic aperture radar (SAR) data and CYGNSS OSW data that have outstanding performance in rainy days. The main problem of current OSW products (such as CCMP) is the poor performance under extreme weather conditions (especially when the wind speed exceeds 15m / s). Another disadvantage is the low spatial resolution (about 25km). Summary of the invention

[0004] The present disclosure provides a computer-implemented method for OSW downscaling, comprising:

[0005] Obtain a set of input images of wind data;

[0006] Construct an OSW downscaling model based on the TransUNet architecture;

[0007] generating an output image of a set of resulting wind data having an increased resolution from the input image using the OSW downscaling model; and

[0008] Before using the OSW downscaling model to generate the output image, the OSW downscaling model is trained according to a plurality of sets of training samples, wherein the OSW downscaling model is trained by optimizing model parameters of the OSW downscaling model in the sense of minimizing a loss function.

[0009] In some embodiments, the plurality of sets of training samples include a first set of wind data samples and a second set of wind data samples, the first set of wind data samples including at least one set of SAR OSW data, and the second set of wind data samples including at least one set of CCMPOSW data.

[0010] In some embodiments, the typhoon eye in both the first set of wind data samples and the second set of wind data samples are centered within their image frames, respectively, so as to coordinate the differences caused by the time lag between the first set of wind data samples and the second set of wind data samples.

[0011] In some embodiments, if a pixel of a wind image in a first set of wind data samples is less than a first threshold and if a predetermined proportion of surrounding pixels exceeds a second threshold, the pixel is determined to be the eye of a typhoon in the wind image of the first set of wind data samples.

[0012] In some embodiments, if a pixel of a wind image in a second set of wind data samples is less than a first threshold and if a predetermined proportion of surrounding pixels exceeds a second threshold, the pixel is determined to be the eye of a typhoon in the wind image of the second set of wind data samples.

[0013] In certain embodiments, training of the OSW downscaling model includes pairing the wind images of the first set of wind data samples with the wind images of the second set of wind data samples based on a date when the wind images of the first set of wind data samples and the wind images of the second set of wind data samples are obtained.

[0014] In some embodiments, all data of wind images of the paired first set of wind data samples and wind images of the paired second set of wind data samples are divided into a training data set, a validation data set, and a test data set, and the training data set is configured to update model parameters of the OSW downscaling model in the sense of minimizing a loss function.

[0015] In some embodiments, the validation dataset is configured to mitigate overfitting, and the test dataset is configured to estimate the final uncertainty of the OSW downscaling model. In some embodiments, the first set of wind data samples and the second set of wind data samples in the training dataset are respectively augmented by data augmentation before being input into the OSW downscaling model.

[0016] In certain embodiments, the second set of wind data samples has been further upsampled using bilinear interpolation before being input into the OSW downscaling model.

[0017] In some embodiments, in the OSW downscaling model, all ReLU activation functions are replaced by Leaky ReLU activation functions.

[0018] In some embodiments, the loss function is determined by: applying a first mask to result wind data samples obtained by using the OSW downscaling model to calculate a first mean square error (MSE) value between a first set of wind data samples in a training dataset and result wind data in an area where wind data in the first set of wind data samples exist; applying a second mask to both result wind data samples obtained by using the OSW downscaling model and a second set of wind data samples in the training dataset to calculate a second MSE value between the second set of wind data samples and result wind data samples obtained by using the OSW downscaling model in an area where the first set of wind data samples are absent; and combining the first MSE value with the second MSE value.

[0019] In some embodiments, the loss function used for training is expressed as follows:

[0020]

[0021] Where n1 is the number of pixels with SAR wind, n2 is the number of pixels without SAR wind, N is the total number of pixels in the image frame, i represents the i-th pixel, is the i-th SAR wind data, is the i-th CCMP wind data, is the i-th resulting wind data obtained from the TransUNet architecture; if SAR wind data is available at the i-th pixel, then mask i is 1; otherwise, mask i is zero.

[0022] In some embodiments, the step of training the OSW downscaling model is stopped when the validation loss obtained from the loss function does not improve for a predetermined consecutive period.

[0023] The method obtained by the present disclosure achieves high accuracy and high resolution, with better performance under high-speed wind conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. The accompanying drawings and corresponding specific embodiments are only used to provide a better understanding of the present disclosure and do not constitute any limitation on the scope of the present disclosure defined in the claims. In particular:

[0025] Figure 1a-Figure 1c Figure 2 shows the wind field retrieved from CCMP during Typhoon Mangkhut in 2018, where Figure 1a The wind field at 12:00 UT on September 15, 2018 is shown. Figure 1b shows the number of observations used in the processing of the CCMP wind data at 12:00 UT on 15 September 2018, and Figure 1cThe maximum wind speed near the typhoon center from different data sources including the Joint Typhoon Warning Center (JTWC), China Meteorological Administration (CMA), Hong Kong Observatory (HKO) and Japan Meteorological Agency (TOKYO) are shown.

[0026] Figure 2 A flowchart of a training method for an OSW model and an inference method for an OSW downscaling model according to an embodiment of the present disclosure is shown.

[0027] Figure 3 A diagram showing the TransUNet architecture according to one embodiment of the present disclosure.

[0028] Figure 4 Comparisons between CCMP wind fields (left column), wind fields generated by an OSW downscaling model trained by the method of the present disclosure (middle column), and SAR wind fields (right column) are illustrated.

[0029] Figure 5 The total wind speed curve between CCMP wind speed, SAR wind speed and wind speed generated by the OSW downscaling model trained by the method of the present disclosure is illustrated ( Figure 4 The gray dashed line in the figure shows the comparison of the two images. DETAILED DESCRIPTION

[0030] The present disclosure will be described more fully below with reference to the accompanying drawings. However, the present disclosure may be implemented in many different forms and should not be construed as limited to the embodiments described herein.

[0031] The present disclosure aims to provide a method of integrating OSW from different observations and increasing the resolution of OSW with improved accuracy using advanced artificial intelligence techniques.

[0032] The main purpose of the present disclosure is to provide accurate OSW data so as to make good preparations for the deployment of offshore wind turbines. Another potential use of the present disclosure is to provide accurate weather forecasts to government departments and logistics companies to respond to dangerous emergencies. For the public, accurate weather information can help them make more efficient outdoor activity plans.

[0033] like Figure 2 As shown in FIG. 1 , the present disclosure provides a training method for an OSW model and an inference method for an OSW downscaling model. Figure 2As shown in , in order to train the OSW downscaling model, the training method includes: obtaining a set of wind data samples, wherein the set of wind data samples includes a first set of wind data and a second set of wind data; constructing an OSW downscaling model based on the TransUNet architecture; inputting the first set of wind data and the second set of wind data into the OSW downscaling model; outputting a set of result wind data; adjusting the OSW downscaling model by using a loss function to obtain a trained OSW downscaling model. For the OSW downscaling model, the inference method includes: obtaining an input image with a set of wind data; using the trained OSW downscaling model, generating an output image with a set of result wind data according to the input image.

[0034] According to an embodiment of the present disclosure, a training method for an OSW model is as follows: Figure 2 As shown in , the OSW downscaling model is trained by optimizing the model parameters of the OSW downscaling model in the sense of minimizing the loss function. Figure 2 As shown in , multiple groups of training samples include a first group of wind data samples and a second group of wind data samples, the first group of wind data samples is a group of SAROSW data, and the second group of wind data samples is a group of CCMPOSW data. The group of SAR OSW data and the group of CCMPOSW data are input into the typhoon eye search algorithm to find the center of the group of data. The centered SAR OSW data and the centered CCMPOSW data can be obtained from the group of SAR OSW data and the group of CCMPOSW data, respectively, wherein the centered SAR OSW and the centered CCMP are called typhoon eye, and the typhoon eye in both the group of SAR OSW data and the group of CCMPOSW data is centered in their image frames. When determining the eye of a typhoon, if a pixel of the wind image in the group of SAR OSW data is less than a first threshold and if a predetermined proportion of the surrounding pixels exceeds a second threshold, then the one pixel is determined as the eye of the typhoon in the group of SAR OSW data of the wind image; if a pixel of the wind image in the group of CCMPOSW data is less than a first threshold and if a predetermined proportion of the surrounding pixels exceeds a second threshold, then the one pixel is determined as the eye of the typhoon in the group of CCMPOSW data of the wind image.

[0035] The centered SAR OSW data and the centered CCMPOSW data are then matched or paired according to the corresponding typhoon eye to obtain multiple sets of matched wind data samples. Then, the multiple sets of matched wind data samples are divided into three sets of data, including a test data set, a training data set, and a validation data set. The training data set is configured to update the model parameters by minimizing the loss function, the validation data set is configured to mitigate overfitting, and the test data set is configured to estimate the final uncertainty of the OSW downscaling model.

[0036] Regarding the training data set, the data in the training set is expanded through data augmentation and then input into the TransUNet architecture for training. At the same time, the test set and the validation set are also sent to the TransUNet architecture for training, thereby obtaining a trained TransUNet architecture. The trained TransUNet architecture can increase the resolution and correct the underestimation of typhoon wind speed, and can be used to build an integrated OSW downscaling model. In some embodiments, all ReLU activation functions in the TransUNet architecture are replaced by the Leaky ReLU activation function in the OSW downscaling model. Although the original version of TransUNet uses ReLU as the activation function, in some embodiments of the present application, the ReLU activation function is replaced with the Leaky ReLU activation function because, in some embodiments, U wind and V wind as described below, which can have negative values, are used in the OSW downscaling model.

[0037] The present disclosure also provides an OSW downscaling model and an OSW downscaling method trained by the above method.

[0038] The sample set comprises OSW data.OSW data can be obtained from the National Oceanic and Atmospheric Administration (NOAA), and the OSW data covers the typhoons of several years (such as from 2015 to 2024), comprising OSW data from Sentinel-1, RADARSAT and RADATSAT-2 satellites.Alternatively, the data covering several years or the data from any other source from ASCAT, Sentinel-1, CYGNSS and RADARSAT-2 can be used.OSW data are from multiple sources, comprising ASCAT, Sentinel-1, CYGNSS, RADARSAT-2, COWVR and PIESAT SAR.For example, the OSW data derived from 1075 SARs covering most typhoons in the world can be used as a group of SAR OSW data.In addition, the data from a group of CCMPOSW data can be used.This group of CCMPOSW data combines satellite wind inversion, on-site wind measurement and the background wind field from numerical weather analysis, and provides a global wind vector without gaps over the ocean. CCMP uses a variational approach to ensure that data-rich regions and background wind fields are closely aligned in time and space.

[0039] In certain embodiments, a typhoon eye search algorithm as described below is provided. The typhoon eye search algorithm comprises the following steps. First, two key thresholds are calculated: Threshold 1 representing the 50th percentile of the wind speed, and Threshold 2 representing the 90th percentile of the wind speed. All pixels are then scanned at speeds below Threshold 1. Each pixel contains the wind speed at its location in the OSW data and scanning means setting a FOR loop to traverse all pixels whose speeds are below Threshold 1. For each such pixel, it is evaluated whether 50% of the pixels within a given window size exhibit speeds exceeding the above-mentioned Threshold 2. The window sizes used in certain embodiments range from 3×3 to 11×11, with a step size of 2 pixels. If such a pixel meets the criteria under any window size, such a pixel is determined to be the eye of the typhoon.

[0040] Subsequently, the above-mentioned typhoon eye search algorithm is applied to the SAR OSW data and its corresponding CCMPOSW data to accurately locate the typhoon eye in both the SAROSW data and the CCMPOSW data. After the typhoon eye is identified, the typhoon eye in the CCMP wind data is centered in the 40×40 matrix, and the typhoon eye in the SAR wind data is centered in the 160×160 matrix. The above-mentioned alignment process ensures that the typhoon eye of the CCMP wind data and the SAR wind data in the two sets of data are centered in the matrix. The above-mentioned typhoon eye search process helps to coordinate the differences caused by the time lag between the CCMP wind data and the SAR wind data.

[0041] The typhoon eye search algorithm can effectively locate the center of the typhoon. Considering that some SAR observations only cover part of the typhoon but not the typhoon center, manual verification can be performed to ensure perfect results. In particular, after applying the typhoon eye search algorithm, manual verification is performed, and any data in which the typhoon eye search algorithm does not accurately identify the typhoon center is discarded. After applying the typhoon eye search algorithm and manual verification, the CCMP wind data can be optionally further upsampled to a 160×160 matrix as input using bilinear interpolation. Before applying the CCMP wind data to the OSW downscaling mode, the CCMP wind data can be upsampled to facilitate the later calculation of the loss function.

[0042] In an inference method for OSW downscaling according to certain embodiments, the method includes obtaining an input image with a set of wind data. The wind data in the set of wind data is input into an OSW downscaling model built based on the TransUNet architecture. In the wind data, the horizontal wind speed can be represented by two wind components (U wind and V wind) contained in the SAR OSW data and the CCMPOSW data, or by the total wind speed and wind direction. In such embodiments, the image band can be composed of a first band designated as U wind, a second band designated as V wind, and a third band designated as the total wind speed. Wind direction is excluded from the band due to its periodicity, which poses a major challenge to deep learning algorithms. On the contrary, in certain embodiments of the present disclosure, wind direction can be derived from U wind and V wind. The third band is set as the total wind speed together with U wind and V wind, aiming to enhance the ability of the algorithm to learn absolute wind speed.

[0043] In certain embodiments, after separating multiple groups of matching wind data samples, two main strategies can be used to expand the training data set by data enhancement. First, the typhoon center can be located at various positions in the image. For CCMPOSW data, the x-coordinate and y-coordinate of the typhoon center in the image frame can be adjusted from the 1st pixel to the 20th pixel, or the 1st pixel to the 40th pixel, or the 1st pixel to the 60th pixel, with a step size of specific pixels (such as 3 pixels, 5 pixels, 7 pixels, 9 pixels). For SAR OSW data, the coordinates of the typhoon center in the image are configured to be used for alignment according to the corresponding CCMPOSW data. The x-coordinate and y-coordinate of the SAR OSW typhoon center are scaled to several times the x-coordinate and y-coordinate of the CCMP typhoon center, such as three times, four times, five times. Secondly, the image of the training data set can be rotated 90 °, 180 ° and 270 °. This rotation is only applied to the data centered by the typhoon in the image. In certain embodiments, as an enhancement to these augmentations, the size of the set of wind data used for training may be increased, for example, by 75, 85, 95, or 100 times.

[0044] For the deep learning architecture and training process, the TransUNet architecture is applied as the OSW downscaling model. The TransUNet architecture has been mentioned in Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., ... & Zhou, Y. (2021). Transunet: Transformers make strong encoders for medical imagesegmentation. arXiv preprint arXiv: 2102.04306 (Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., ... & Zhou, Y. (2021). Transunet: Transformers make strong encoders for medical imagesegmentation. arXiv preprint arXiv: 2102.04306). The TransUNet architecture can effectively capture global context and large-scale latent information, while also enabling accurate capture of detailed local features. The goal is to produce wind field output with enhanced spatial resolution and typhoon-corrected wind speed, which is different from the segmentation task that usually produces class probabilities.

[0045] The TransUNet architecture consists of an encoder for encoding (downsampling) and a decoder for decoding (upsampling). Figure 3 The TransUNet architecture is shown in .

[0046] The TransUNet architecture is constructed by adding a transformer layer on the basis of UNet. The transformer layer includes an encoder part and a decoder part. The encoder part includes a transformer encoder and a convolutional attention mechanism module. The transformer encoder is configured to use the CNN-transformer algorithm to extract the features of the input image into feature maps and position information. The decoder part includes multiple convolutional layers and one or more upsampling layers for convolving and upsampling the feature maps and position information from the encoder part.

[0047] During the encoding process, a CNN can be used as a feature extractor to generate feature maps for the input. Then, for each level of the feature extractor, the output feature map is then connected to the decoder path of the same level. The feature map is then labeled (vectorized) as a linear projection of the 2D embedding of the shape. The embedding is pre-trained and will preserve the positional information of the feature map. Finally, the output is reshaped in preparation for the upsampling path. The Unet architecture is used to combine the CNN feature maps from the encoder to the global context features encoded by the transformer through skip connections.

[0048] During decoding, the input from the CNN-Transformer encoder is upsampled by a 3×3 convolutional layer with Leaky ReLU activation and sent to the third-level CNN feature extractor to obtain the resulting feature map. The resulting feature map is then extended again through a 3×3 convolutional layer with Leaky ReLU activation, and its output is then sent to the second-level CNN feature extractor. These steps are repeated, and the output is a mask of shape (C, H, W), where C = 3, H = image height, and W = image width. C = 3 because the image band contains U wind, V wind, and total wind speed.

[0049] The use of Leaky ReLU activation allows negative values ​​to be generated in the Uwind and Vwind outputs.

[0050] SAR wind data is used as the ground truth. However, it only covers a portion of the input image frame. In order to incorporate the constraints outside the SAR data coverage area, the upsampled CCMP wind data is used in the loss function calculation outside the SAR data coverage area, but with a smaller weight.

[0051] To calculate the loss function, a mask is first applied to the output to calculate the MSE value between the SAR wind and the output in the area where SAR wind data is present. Because the SAR OSW only covers a portion of the input image frame, the mask is configured to be derived from the SAR OSW and the output is obtained from TransUNet. Next, another mask is applied to both the upsampled CCMP input and output to calculate the MSE value in the area where SAR wind data is missing. Finally, the two MSE values ​​are combined to define the loss function for training the TransUNet architecture. The loss function is expressed as follows:

[0052]

[0053] Where n1 is the number of pixels with SAR wind, n2 is the number of pixels without SAR wind, N is the total number of pixels in the image frame, i represents the i-th pixel, is the i-th SAR wind data, is the i-th CCMP wind data, is the i-th resulting wind data obtained from the TransUNet architecture; if SAR wind data is available at the i-th pixel, then mask i is 1; otherwise mask i is zero.

[0054] The same loss function is used for both training loss and validation loss. In the test dataset, the root mean square deviation (RMSD) is quantified only in the region where SAR wind is available in order to quantify the test error. The RMSD is defined using the following equation:

[0055]

[0056] where n1 is the number of pixels with SAR wind, is the i-th SAR wind data, is the i-th CCMP wind data, is the i-th resulting wind data obtained from the TransUNet architecture; if SAR wind data is available at the i-th pixel, then mask i is 1; otherwise mask i is zero.

[0057] During network training, the validation loss is calculated with the validation dataset after each epoch and a checkpoint is saved only if the validation loss has decreased. If the validation loss does not improve for a predetermined number of consecutive epochs (such as 15, 20, 25, 30 consecutive epochs), the training will be stopped to prevent overfitting.

[0058] The OSW downscaling model obtained in the embodiments of the present disclosure has a promising match with SAR observations over regions where SAR winds are available. Figure 4 As shown in the figure, the top two rows show two horizontal wind speeds (U wind and V wind refer to the wind speed in two directions), and the bottom row shows the total wind speed. On average, in the test set, the RMSD between CCMP and SAR is 11.43m / s, while based on the method of the present application using the deep learning algorithm, the RMSE can be reduced to 5.13m / s. Outside the SAR coverage, the simulation shows a reasonable match with the CCMP input, such as Figure 5 In general, the present disclosure can successfully improve the spatial resolution of CCMP and correct its underestimation of typhoons. In addition, the OSW downscaling model of the embodiment of the present disclosure can provide a complete SAR-like wind field without any data gaps.

[0059] The main advantages of the present invention over the prior art can be described from the following three aspects.

[0060] First, it has the characteristics of high accuracy and high resolution. Compared with existing products, the integrated OSW product of the present application has better performance in high-speed wind conditions, with a resolution of 1 km and near real-time (NRT) delay.

[0061] Secondly, it involves physical atmospheric models for accurate weather forecasting. With accurate and integrated OSW data, the accuracy of weather forecasts and predictions can be improved. Compared with the current operational forecasting system, the accuracy of weather forecasts can be improved by 20% or even more.

[0062] The present disclosure also provides a computer system, which includes a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the training method for the OSW downscaling model as described herein.

[0063] The computer system may be a server or a terminal. The server may be an independent physical server or a terminal. It may be a server cluster or a distributed system composed of multiple physical servers, or it may provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms. The terminal may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart wearable device, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via a wired communication method or a wireless communication method, and the present disclosure does not limit this. It will be understood by those skilled in the art that the above-mentioned computer system structure does not limit the computer system, and may include more or fewer components, or combine certain components, or arrange different components. The computer system may also include a display unit, etc., which will not be described in detail here. In particular, in an embodiment, the processor in the computer system will load the executable file corresponding to the process of one or more public programs into the memory according to the instruction, and the processor will run the executable file stored in the memory to implement various functions.

[0064] The present disclosure also provides a computer program product, which is configured to implement the training method for the OSW downscaling model as described herein.

[0065] The embodiments or elements shown in this disclosure (including the specific instructions and materials utilized in the examples) are intended to be illustrative rather than restrictive. They allow for a wide range of changes, adjustments or adaptations consistent with the basic concepts of the disclosure. It is important to clarify that all drawings depicted are for illustration purposes only; they are not to scale, nor are they exact replicas of actual devices.

[0066] Where not explicitly described, the various embodiments described in conjunction with the drawings or their various aspects and features may be combined or exchanged with each other without limiting or expanding the scope of the described disclosure, as long as such combination or exchange is meaningful and within the meaning of the present disclosure. The advantages described with respect to a specific embodiment of the present disclosure or with respect to a specific drawing are also the advantages of other embodiments of the present disclosure in any applicable case.

[0067] References

[0068] Cui, Z., Pu, Z., Tallapragada, V., Atlas, R., & Ruf, C.S. (2019). A Preliminary Impact Study of CYGNSS Ocean Surface Wind Speeds on Numerical Simulations of Hurricanes. Geophysical Research Letters, 46(5), 2984–2992. (Cui, Z., Pu, Z., Tallapragada, V., Atlas, R., & Ruf, C.S. (2019). A Preliminary Impact Study of CYGNSS Ocean Surface Wind Speeds on Numerical Simulations of Hurricanes. Geophysical Research Letters, 46(5), 2984–2992. https: / / doi.org / 10.1029 / 2019GL082236)

[0069] Duan, B., Zhang, W., Yang, X., Dai, H., & Yu, Y. (2017). Assimilation of Typhoon Wind Field Retrieved from Scatterometer and SAR Based on the Huber Norm Quality Control. Remote Sensing, 9(10), 987. https: / / doi.org / 10.3390 / rs9100987

[0070] Guo, C., Ai, W., Hu, S., Du, X., & Chen, N. (2022). Sea Surface Wind Direction Retrieval Based on Convolution Neural Network and Wavelet Analysis. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15, 3868–3876. https: / / doi.org / 10.1109 / JSTARS.2022.3173001

[0071] Li, J., Hong, D., Gao, L., Yao, J., Zheng, K., Zhang, B., & Chanussot, J. (2022). Deep learning in multimodal remote sensing data fusion: A comprehensive review. International Journal of Applied Earth Observation and Geoinformation, 112, 102926. https: / / doi.org / 10.1016 / j.jag.2022.102926

[0072] Mao, H., Kathuria, D., Duffield, N., & Mohanty, BP (2019). Gap Filling of High-Resolution Soil Moisture for SMAP / Sentinel-1: A Two-Layer Machine Learning-Based Framework. Water Resources Research, 55(8), 6986–7009. (Mao, H., Kathuria, D., Duffield, N., & Mohanty, BP (2019). Gap Filling of High-Resolution Soil Moisture for SMAP / Sentinel-1: A Two-Layer Machine Learning-Based Framework. Water Resources Research, 55(8), 6986–7009. https: / / doi.org / 10.1029 / 2019WR024902)

[0073] Munir, A., Blasch, E., Kwon, J., Kong, J., & Aved, A. (2021). Artificial Intelligence and Data Fusion at the Edge. IEEE Aerospace and Electronic Systems Magazine, 36(7), 62–78. (Munir, A., Blasch, E., Kwon, J., Kong, J., & Aved, A. (2021). Artificial Intelligence and Data Fusion at the Edge. IEEE Aerospace and Electronic Systems Magazine, 36(7), 62–78. https: / / doi.org / 10.1109 / MAES.2020.3043072)

[0074] Pu, Z., Wang, Y., Li, X., Ruf, C., Bi, L., & Mehra, A. (2022). Impacts of Assimilating CYGNSS Satellite Ocean-Surface Wind on Prediction of Landfalling Hurricanes with the HWRFModel. Remote Sensing, 14(9), 2118. https: / / doi.org / 10.3390 / rs14092118

[0075] Wang, M., Wang, D., Xiang, Y., Liang, Y., Xia, R., Yang, J., et al. (2023). Fusion of ocean data from multiple sources using deep learning: Utilizing sea temperature as an example. Frontiers in Marine Science, 10. Retrieved from https: / / www.frontiersin.org / articles / 10.3389 / fmars.2023.1112065

[0076] Wang, Y., Shi, X., Lei, L., & Fung, JC-H. (2022). Deep Learning Augmented Data Assimilation: Reconstructing Missing Information with Convolutional Autoencoders. Monthly Weather Review, 150(8), 1977–1991. (Wang, Y., Shi, X., Lei, L., & Fung, JC-H. (2022). Deep Learning Augmented Data Assimilation: Reconstructing Missing Information with Convolutional Autoencoders. Monthly Weather Review, 150(8), 1977–1991. https: / / doi.org / 10.1175 / MWR-D-21-0288.1)

[0077] Zanchetta, A., & Zecchetto, S. (2021). Wind direction retrieval from Sentinel-1 SAR images using ResNet. Remote Sensing of Environment, 253, 112178. (Zanchetta, A., & Zecchetto, S. (2021). Wind direction retrieval from Sentinel-1 SAR images using ResNet. Remote Sensing of Environment, 253, 112178. https: / / doi.org / 10.1016 / j.rse.2020.112178)

[0078] Zecchetto, S., De Biasio, F., Valle, A. della, Quattrocchi, G., Cadau, E., & Cucco, A. (2016). Wind Fields From C- and X-BandSAR Images at VV Polarization in Coastal Area (Gulf of Oristano, Italy). IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(6), 2643–2650. Sensing,9(6),2643–2650.https: / / doi.org / 10.1109 / JSTARS.2016.2538322)

[0079] Zhou, L., Zheng, G., Li, X., Yang, J., Ren, L., Chen, P., et al. (2017). An Improved Local Gradient Method for SeaSurface Wind Direction Retrieval from SAR Imagery. Remote Sensing, 9(7), 671. (Zhou, L., Zheng, G., Li, X., Yang, J., Ren, L., Chen, P., et al. (2017). An Improved Local Gradient Method for SeaSurface Wind Direction Retrieval from SAR Imagery. Remote Sensing, 9(7), 671. https: / / doi.org / 10.3390 / rs9070671)

[0080] Atlas, R., R. N. Hoffman, J. Ardizzone, S. M. Leidner, J. C. Jusem, D. K. Smith, and D. Gombos, 2011: A cross-calibrated, multiplatform ocean surface wind velocity product for meteorological and oceanographic applications. Bull. Amer. Meteor. Computer Reviews in Social Sciences, 92, 157-174. (Atlas, R., R. N. Hoffman, J. Ardizzone, S. M. Leidner, J. C. Jusem, D. K. Smith, and D. Gombos, 2011: A cross-calibrated, multiplatform ocean surface wind velocity product for meteorological and oceanographic applications. Bull. Amer. Meteor. Soc., 92, 157-174. https: / / doi.org / 10.1175 / 2010BAMS2946.1)

[0081] Mears, C.A., Scott, J., Wentz, F.J., Ricciardulli, L., Leidner, S.M., Hoffman, R., & Atlas, R. (2019). A near-real-time version of the cross-calibrated multiplatform (CCMP) ocean surface wind velocity data set. Journal of Geophysical Research: Oceans, 124(10), 6997-7010.

[0082] Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., ... & Zhou, Y. (2021). Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306.

[0083] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, AN, ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.

[0084] Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015:18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18 (pp. 234-241). Springer International Publishing.

Claims

1. A computer-implemented method for downscaling ocean surface wind (OSW), comprising: Obtain a set of input images of wind data; Construct an ocean surface wind downscaling model based on the TransUNet architecture; generating an output image having a set of resulting wind data of increased resolution from the input image using the ocean surface wind downscaling model; as well as Before using the ocean surface wind downscaling model to generate the output image, the ocean surface wind downscaling model is trained according to multiple groups of training samples, wherein the ocean surface wind downscaling model is trained by optimizing model parameters of the ocean surface wind downscaling model in the sense of minimizing a loss function.

2. The computer-implemented method of claim 1, wherein: The plurality of sets of training samples include a first set of wind data samples and a second set of wind data samples, the first set of wind data samples being a set of synthetic aperture radar (SAR) ocean surface wind data, and the second set of wind data samples being a set of cross-calibrated multi-platform (CCMP) ocean surface wind data.

3. The computer-implemented method of claim 2, wherein: The typhoon eye in both the first set of wind data samples and the second set of wind data samples are located at the center of their image frames, respectively, so as to train the ocean surface wind downscaling model and coordinate the differences caused by the time lag between the first set of wind data samples and the second set of wind data samples.

4. The computer-implemented method of claim 3, wherein: If a pixel of the wind image of the first set of wind data samples is less than a first threshold and if a predetermined proportion of surrounding pixels exceeds a second threshold, the pixel is determined to be the eye of the typhoon in the wind image of the first set of wind data samples.

5. The computer-implemented method of claim 4, wherein: If a pixel of the wind image of the second set of wind data samples is less than a first threshold and if a predetermined proportion of surrounding pixels exceeds a second threshold, the pixel is determined to be the eye of the typhoon in the wind image of the second set of wind data samples.

6. The computer-implemented method of claim 5, wherein: The training of the ocean surface wind downscaling model includes, when the wind images of the first set of wind data samples and the wind images of the second set of wind data samples are obtained, pairing the wind images of the first set of wind data samples with the wind images of the second set of wind data samples based on date.

7. The computer-implemented method of claim 6, wherein: All data of the paired wind images of the first set of wind data samples and the paired wind images of the second set of wind data samples are divided into a training data set, a validation data set and a test data set, wherein the training data set is configured to update the model parameters of the ocean surface wind downscaling model in the sense of minimizing a loss function.

8. The computer-implemented method of claim 7, wherein: The validation dataset is configured to mitigate overfitting, and the test dataset is configured to estimate final uncertainty of the ocean surface wind downscaling model.

9. The computer-implemented method of claim 7, wherein: The first set of wind data samples and the second set of wind data samples in the training data set are respectively augmented by data augmentation before being input into the ocean surface wind downscaling model.

10. The computer-implemented method of claim 9, wherein: The second set of wind data samples have been further upsampled using bilinear interpolation before being input into the ocean surface wind downscaling model.

11. The computer-implemented method of claim 10, wherein: In the ocean surface wind prediction model, all ReLU activation functions are replaced by Leaky ReLU activation functions.

12. The computer-implemented method of claim 1, wherein: The loss function is determined by: applying a first mask to result wind data samples obtained by using the ocean surface wind downscaling model to calculate a first mean square error (MSE) value between a first set of wind data samples in the training data set and result wind data in an area where wind data in the first set of wind data samples exist; applying a second mask to the result wind data samples obtained by using the ocean surface wind downscaling model and a second set of wind data samples in the training data set to calculate a second MSE value between the second set of wind data samples and the result wind data samples obtained by using the ocean surface wind downscaling model in an area where the first set of wind data samples are absent; and combining the first MSE value with the second MSE value.

13. The computer-implemented method of claim 12, wherein: The loss function used for training is expressed as follows: Where n1 is the number of pixels with SAR wind, n2 is the number of pixels without SAR wind, N is the total number of pixels in the image frame, i represents the i-th pixel, is the i-th SAR wind data, is the i-th CCMP wind data, is the i-th resulting wind data obtained from the TransUNet architecture; if the SAR wind data is available at the i-th pixel, then mask i is 1; otherwise, mask i is zero.

14. The computer-implemented method of claim 12, wherein: The step of stopping training the ocean surface wind downscaling model when the validation loss obtained from the loss function does not improve for a predetermined continuous period.

Images