Multi-modal information fusion sea surface wind speed inversion method, device and system

Abstract

The invention provides a multi-modal information fusion sea surface wind speed inversion method, device and system, and the method comprises the steps: obtaining satellite navigation data, wind speed reference data and significant wave height data, performing matching and quality control on the wind speed reference data and the significant wave height data with the satellite navigation data to obtain an initial data set; recombining the initial data set according to a track to obtain a target data set, and dividing the target data set into a training set, a verification set and a test set; and constructing a wind speed inversion model by using the spatial-temporal correlation of continuous time delay-Doppler diagram observation values in the same trajectory and combining with auxiliary feature information. According to the invention, the extracted information can be fused with various auxiliary feature information, the inversion precision is improved, and the influence characteristics of the ocean state on the wind speed inversion can be quantitatively analyzed.

Description

Technical Field

[0001] The present invention relates to the field of atmospheric science and technology, and particularly to a multi-modal information fusion method, device and system for retrieving sea surface wind speed. Background Art

[0002] As an important meteorological parameter, the sea surface wind field indirectly affects the Earth's climate through the action of ocean circulation. Accurately detecting and studying it is of great significance. Traditional sea surface wind field detection means include stations, buoys, meteorological remote sensing satellites, etc. They have defects such as limited measurement range, high cost, and high power consumption. In recent years, the theory of retrieving sea surface wind speed by Global Navigation Satellite System Reflectometry (GNSS-R) technology has been continuously improved and developed, providing a sea surface wind field detection means with a large coverage range, high spatio-temporal resolution, and low cost. Spaceborne GNSS-R sea surface wind speed retrieval mainly extracts signal features by analyzing the continuous Delay-Doppler Map (DDM), and establishes a model to map the signal features to the reference wind speed. Traditional methods include waveform matching and Geophysical Model Functions (GMF) methods.

[0003] In recent years, studies on GNSS-R sea surface wind speed retrieval have found that factors such as ocean state, observation geometry, and receiver hardware parameters affect the retrieval accuracy. Especially in the low wind speed environment, the influence of swell on sea surface roughness is significant. Traditional methods are difficult to accurately establish the complex relationship between features and wind speed. Scholars have gradually adopted optimization algorithms and neural network models to explore the mapping relationship between features and wind speed, significantly improving the retrieval accuracy. With the in-depth study of GNSS-R sea surface wind speed retrieval, it has been proved that data-driven neural network models can extract more effective information from a single DDM, and further improve the retrieval accuracy through multi-modal feature fusion. Existing studies have shown that the spatio-temporal correlation of adjacent observations along the same trajectory can effectively reduce the retrieval bias. However, using only single DDM features will lose strong spatio-temporal correlation information. Currently, some scholars have tried to use trajectory information to improve the wind speed retrieval accuracy. However, the models used do not consider ocean state information such as swell, resulting in ambiguity in accuracy in the low wind speed range.

[0004] For the problem of poor retrieval accuracy in the existing related technologies, no effective solution has been proposed yet. Summary of the Invention

[0005] The present invention provides a multi-modal information fusion method, device and system for retrieving sea surface wind speed, so as to solve the defect of poor retrieval accuracy in the existing related technologies and achieve high-precision wind speed retrieval.

[0006] In a first aspect, the present invention provides a multi-modal information fusion sea surface wind speed inversion method, comprising: Acquiring satellite navigation data, wind speed reference data and significant wave height data, and matching and quality controlling the wind speed reference data and the significant wave height data with the satellite navigation data, respectively, to obtain an initial data set; Reorganizing the initial data set according to the trajectory to obtain a target data set, and dividing the target data set into a training set, a validation set, and a test set; The wind speed inversion model is constructed by using the spatiotemporal correlation of continuous delay-Doppler map observations in the same trajectory and combining auxiliary feature information. The wind speed inversion model is trained using the training set and the validation set, and the wind speed inversion model is performed performance evaluation and error analysis using the test set; The delay-Doppler diagram data to be measured is obtained, and the wind speed is predicted by the wind speed inversion model to obtain a prediction result.

[0007] According to a multi-modal information fusion sea surface wind speed inversion method provided by the present invention, the wind speed reference data and the significant wave height data are matched with the satellite navigation data and quality controlled to obtain an initial data set, including: Matching the wind speed reference data and the significant wave height data with the satellite navigation data based on the latitude and longitude of the sampling point and time; The matched data is screened according to preset screening conditions to obtain the initial data set.

[0008] According to a multi-modal information fusion sea surface wind speed inversion method provided by the present invention, the initial data set is reorganized according to the trajectory to obtain a target data set, including: Distinguishing observation samples of different trajectories in the target data set by using trajectory labels; The different trajectory data are divided into sub-trajectories containing a target number of observation samples to obtain the target data set.

[0009] According to a multi-modal information fusion sea surface wind speed inversion method provided by the present invention, different trajectory data are divided into sub-trajectories containing a target number of observation samples, including: For each trajectory, if the total number of observation samples in the trajectory is an integer multiple of the target number, then the trajectory is evenly divided into a number of sub-trajectories starting from the first observation sample in the trajectory; If the total number of observation samples in the trajectory is not an integer multiple of the target number, the last group of observation samples whose number does not reach the target number is discarded.

[0010] A multimodal information fusion sea surface wind speed inversion method provided by the present invention uses the spatio-temporal correlation of consecutive time-delay Doppler map observations in the same trajectory, combines auxiliary feature information, and constructs a wind speed inversion model, including: Capture the spatio-temporal correlation information of the time-delay Doppler map data through a long short-term memory network and a BP neural network to obtain DDM sequence features; Obtain auxiliary feature information, and preprocess the auxiliary feature information to obtain auxiliary sequence features; Concatenate the DDM sequence features and the auxiliary sequence features, and construct the wind speed inversion model.

[0011] A multimodal information fusion sea surface wind speed inversion method provided by the present invention uses the training set and the validation set to train the wind speed inversion model, including: Set the training period for training the wind speed inversion model; Use the training set and the validation set, combine the L2 regularization strategy and the early stopping method to train the wind speed inversion model.

[0012] A multimodal information fusion sea surface wind speed inversion method provided by the present invention uses the test set to perform performance evaluation and error analysis on the wind speed inversion model, including: Determine the error index of each sub-trajectory based on the wind speed inversion model; Determine the overall error of the wind speed inversion model according to the average value of the error indexes of all sub-trajectories.

[0013] For a multimodal information fusion sea surface wind speed inversion method provided by the present invention, the error index of the sub-trajectory includes at least one of the root mean square error, the overall deviation, and the mean absolute percentage error of each sub-trajectory.

[0014] In a second aspect, the present invention also provides a multimodal information fusion sea surface wind speed inversion device, including: An acquisition module, configured to acquire satellite navigation data, wind speed reference data, and significant wave height data, and match and perform quality control on the wind speed reference data and the significant wave height data with the satellite navigation data respectively to obtain an initial data set; A recombination module, configured to recombine the initial data set according to the trajectory to obtain a target data set, and divide the target data set into a training set, a validation set, and a test set; A construction module, configured to use the spatio-temporal correlation of consecutive time-delay Doppler map observations in the same trajectory, combine auxiliary feature information, and construct a wind speed inversion model; A training module, configured to train the wind speed inversion model by using the training set and the validation set, and perform performance evaluation and error analysis on the wind speed inversion model by using the test set; A prediction module, configured to obtain the to-be-measured time-delay Doppler map data, and perform wind speed prediction through the wind speed inversion model to obtain a prediction result.

[0015] In a third aspect, the present invention further provides a multi-modal information fusion sea surface wind speed inversion system, including: A data download and storage module, configured to obtain satellite navigation data, wind speed reference data, and significant wave height data; A data preprocessing module, configured to match and perform quality control on the wind speed reference data and the significant wave height data with the satellite navigation data respectively to obtain an initial data set; A data integration and generation module, configured to reorganize the initial data set according to trajectories to obtain a target data set, and divide the target data set into a training set, a validation set, and a test set; A wind speed inversion model generation module, configured to construct a wind speed inversion model by using the spatio-temporal correlation of continuous time-delay Doppler map observations in the same trajectory and combining auxiliary feature information, and train the wind speed inversion model by using the training set and the validation set; An accuracy evaluation and result output module, configured to perform performance evaluation and error analysis on the wind speed inversion model by using the test set, obtain the to-be-measured time-delay Doppler map data, and perform wind speed prediction through the wind speed inversion model to obtain a prediction result.

[0016] In a fourth aspect, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the program, it implements the multi-modal information fusion sea surface wind speed inversion method as described in the first aspect above.

[0017] In a fifth aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the multi-modal information fusion sea surface wind speed inversion method as described in the first aspect above.

[0018] In a sixth aspect, the present invention further provides a computer program product, including a computer program, where when the computer program is executed by a processor, it implements the multi-modal information fusion sea surface wind speed inversion method as described in the first aspect above.

[0019] Compared with the prior art, the present invention has the following beneficial effects: The multi-modal information fusion sea surface wind speed inversion method provided by the present invention extracts the spatio-temporal correlation information between consecutive observation values on the observation trajectory, and fuses the extracted information with a variety of auxiliary feature information to improve the inversion accuracy. It can also quantitatively analyze the influence characteristics of the ocean state on the wind speed inversion. Compared with the prior art, this method shows obvious advantages in various evaluation indexes, obtains more accurate and reliable wind speed inversion results, can meet the needs of atmospheric research related to the sea surface wind field using a large amount of GNSS-R satellite data, and solves the problem of poor inversion accuracy existing in the existing related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0021] Figure 1 is a flowchart of the multi-modal information fusion sea surface wind speed inversion method provided by the present invention; Figure 2 is a schematic diagram of the process of sea surface wind speed inversion in an embodiment of the present invention; Figure 3 is a schematic diagram of building a wind speed inversion model in an embodiment of the present invention; Figure 4 is a structural block diagram of the multi-modal information fusion sea surface wind speed inversion device provided by the present invention; Figure 5 is a schematic diagram of the structure of the multi-modal information fusion sea surface wind speed inversion system provided by the present invention; Figure 6 is a schematic diagram of the structure of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention with reference to the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0023] The present invention provides a multi-modal information fusion sea surface wind speed inversion method, Figure 1 is a flowchart of the multi-modal information fusion sea surface wind speed inversion method provided by the present invention, as Figure 1 shown, the method includes the following steps: Step S101: Obtain satellite navigation data, wind speed reference data, and significant wave height data, and match and perform quality control on the wind speed reference data and significant wave height data with the satellite navigation data respectively to obtain an initial data set; Step S102: Reorganize the initial data set according to the trajectory to obtain a target data set, and divide the target data set into a training set, a validation set, and a test set; Step S103: Utilize the spatio-temporal correlation of consecutive delay-Doppler map observations in the same trajectory, combined with auxiliary feature information, to construct a wind speed inversion model; Step S104: Train the wind speed inversion model using the training set and the validation set, and perform performance evaluation and error analysis on the wind speed inversion model using the test set; Step S105: Obtain the to-be-measured delay-Doppler map data, and perform wind speed prediction through the wind speed inversion model to obtain a prediction result.

[0024] In this method, first, obtain Cyclone Global Navigation Satellite System (CYGNSS) data, hereinafter referred to as satellite navigation data for convenience of description; and obtain wind speed reference data and significant wave height data, match the wind speed reference data and significant wave height data with the satellite navigation data respectively, and perform quality control to obtain an initial data set. Then, utilize the initial data set after the above quality control to reorganize according to the trajectory, and divide the reorganized target data set into a training set, a validation set, and a test set. Since the sea surface is rough, the ocean state, observation geometry, and receiver hardware parameters will also affect the accuracy of wind speed inversion. Especially in a low wind speed environment, the swell caused by the non-local wind field has a particularly significant impact on the sea surface roughness, directly affecting the accuracy of wind speed inversion. Therefore, this method utilizes the spatio-temporal correlation of consecutive delay-Doppler map (DDM) observations in the same trajectory, integrates auxiliary feature information such as the ocean state, observation geometry, and receiver hardware parameters, and constructs a wind speed inversion model. Then, train the wind speed inversion model using the training set and the validation set, perform performance evaluation and error analysis on the wind speed inversion model using the test set, and fine-tune the wind speed inversion model to improve the prediction accuracy of the wind speed inversion model. Finally, obtain the to-be-measured delay-Doppler map data, and perform wind speed prediction through the fine-tuned wind speed inversion model to obtain a prediction result.

[0025] In the above process, aiming at the problem that most current multi-modal neural networks only take a single time-delay Doppler map as input, do not fully utilize the strong spatio-temporal correlation between the time-delay Doppler maps of continuous observations on the same trajectory, and few models fuse the observation trajectory and ocean state information at the same time, resulting in a loss of wind speed inversion accuracy, a wind speed inversion model is constructed. This model can effectively extract the spatio-temporal correlation information between continuous observations on the observation trajectory, can fuse the extracted information with a variety of auxiliary feature information to improve the inversion accuracy, and can also quantitatively analyze the influence characteristics of the ocean state on wind speed inversion. Compared with the existing technology, this method shows obvious advantages in various evaluation indexes, obtains more accurate and reliable wind speed inversion results, can meet the needs of atmospheric research related to the sea surface wind field using a large amount of GNSS-R satellite data, and solves the problem of poor inversion accuracy existing in the existing related technologies.

[0026] The following uses the CYGNSS L1 level 2.1 version data product, and the time span is from August 1, 2018 to December 31, 2019 for description. Specifically, the following embodiments mainly use DDM, DDM uncertainty, DDM eigenvalue LES and NBRCS, sampling point longitude and latitude, sampling point signal incident angle, and range corrected gain (RCG) as input features.

[0027] In addition, the following embodiments use the European Centre for Medium-Range Weather Forecasts (ECMWF) ERA5 wind speed vector product of the corresponding period as the reference wind speed, and the significant wave height data as auxiliary data. ECMWF ERA5 data is a global gridded product based on a numerical weather prediction model, and is one of the reanalysis data products with the highest accuracy at present, and is also a commonly used reference data in the GNSS-R sea surface wind speed inversion scenario. The wind speed vector and significant wave height information used in the following embodiments are shown in Table 1: Table 1 Auxiliary feature names and their meanings

[0028] Figure 2 is a schematic diagram of the process of sea surface wind speed inversion in the embodiments of the present invention. As Figure 2 shown, in some of these embodiments, in step S101, the wind speed reference data and the significant wave height data are respectively matched and quality-controlled with the satellite navigation data to obtain an initial data set, including: matching the wind speed reference data and the significant wave height data with the satellite navigation data based on the sampling point longitude and latitude and time respectively; screening the matched data according to preset screening conditions to obtain the initial data set.

[0029] Exemplarily, satellite-borne CYGNSS data and ECMWF ERA5 data are batch obtained and stored, and the wind speed reference data and significant wave height data are respectively matched with the CYGNSS data based on the longitude, latitude and time of the CYGNSS data sampling points. Since the spatio-temporal resolutions of the CYGNSS data and the ECMWF data are different, matching needs to be performed based on the longitude, latitude and time of the CYGNSS sampling points. The wind speed vector matching is bilinear interpolation in space and linear interpolation in time; the significant wave height matching adopts the nearest neighbor method, with a spatial threshold of 25 km and a time threshold of 30 minutes, taking into account both the matching accuracy and the sample quality.

[0030] In this embodiment, the quality control conditions are as follows: the sample value is non-empty (non-Nan); the matched wind speed, significant wave height, and DDM eigenvalue are positive; the sample quality control label quality_flags is equal to 0; samples where the star sensor is blocked and the CYGNSS satellite attitude cannot be determined are deleted; the RCG is greater than 3.

[0031] In some of these embodiments, in step S102, the initial data set is reorganized according to the trajectory to obtain the target data set, including: differentiating the observation samples of different trajectories in the target data set through the trajectory label; dividing the data of different trajectories into sub-trajectories containing the target number of observation samples to obtain the target data set.

[0032] In this embodiment, dividing the data of different trajectories into sub-trajectories containing the target number of observation samples includes: for each trajectory, if the total number of observation samples in the trajectory is an integer multiple of the target number, then starting from the first observation sample in the trajectory, it is evenly divided into several sub-trajectories; if the total number of observation samples in the trajectory is not an integer multiple of the target number, then the last group of observation samples whose quantity does not reach the target number is discarded.

[0033] Specifically, for 8 data files per day, the observation samples of different trajectories are distinguished by the trajectory label track_id. track_id represents the sequence of continuously tracking a satellite for a period of time during signal acquisition, and its value range is [1, n], where n is the total number of trajectories. Samples with the same track_id value belong to the same time-continuous observation trajectory. Further, the data of different trajectories are divided into sub-trajectories containing m samples. Suppose a DDM observation trajectory has p observation samples. If p is an integer multiple of m, then starting from the first observation sample in the trajectory, every m samples are divided into a group as sub-trajectories , if p is not an integer multiple of m, then the observation samples of the last sub-trajectory that is less than one are discarded and not used.

[0034] Exemplarily, different trajectory data are divided into sub-trajectories each containing 9 samples. Suppose a DDM observation trajectory has p samples. If p is an integer multiple of 9, then starting from the first sample in the trajectory, every 9 samples are grouped into a sub-trajectory. , the sub-trajectory is denoted as:

[0035]

[0036] wherein, denotes the th sub-trajectory. Each sub-trajectory contains 9 samples, and these 9 samples are denoted as ; denotes the total number of divided sub-trajectories. If p is not an integer multiple of 9, then the observed samples less than one sub-trajectory at the end are discarded.

[0037] After recombination by trajectory, considering the large data time span and large data volume per satellite per day, downsampling is performed again. For each satellite data file per day, a random number q is generated within 0 - 6000, and q samples are randomly and non-repeatedly selected. If the total number of samples is less than q, then all are retained. The final experimental samples are 1,312,789. Similarly, the data from August 1, 2018 to July 31, 2019 is used as the training set, with a total number of samples of 863,080; the data from August 1 to September 30, 2019 is used as the validation set, with a total number of samples of 183,050; the data from October 1 to December 31, 2019 is used as the test set, with a total number of samples of 266,659.

[0038] In some of these embodiments, in step S103, using the spatio-temporal correlation of consecutive time-delay Doppler map observations in the same trajectory, combined with auxiliary feature information, a wind speed inversion model is constructed, including: capturing the spatio-temporal correlation information of time-delay Doppler map data through a Long Short-Term Memory (LSTM) network and a BP neural network to obtain DDM sequence features; obtaining auxiliary feature information and preprocessing the auxiliary feature information to obtain auxiliary sequence features; splicing the DDM sequence features and the auxiliary sequence features, and constructing a wind speed inversion model. The wind speed inversion model includes several parts such as trajectory data feature extraction, auxiliary feature processing, multi-information fusion, and wind speed regression prediction.

[0039] Specifically, Figure 3 is a schematic diagram of building a wind speed inversion model in an embodiment of the present invention. As Figure 3 shown, in the trajectory data feature extraction part, an LSTM network and a BP neural network are respectively used to capture the spatio-temporal correlation information of the DDM sequence and obtain auxiliary feature information. In the DDM sequence feature extraction part, the input For m consecutive DDM observation values on the same trajectory (in the above-mentioned embodiment, the value of m is 9), the shape is , where 256 represents the size. Input the DDM sequence into m parallel LSTM layers. Each LSTM layer contains two layers, and the number of neurons in each layer is 64 and 32 respectively. The return_sequences=True is set for both layers of the network, and the activation function is the Relu function. After passing through the two LSTM layers, a fully connected layer is connected, with the number of neurons being 100 and the activation function being the Tanh function. The finally output DDM sequence feature is a feature map of 256×m×100.

[0040] For the auxiliary feature processing part, 8 pieces of auxiliary feature information related to the ocean state, observation geometry, and receiver hardware parameters are selected. During the preprocessing of the auxiliary feature information, two fully connected layers are used, with the number of neurons being 80 and 160 respectively, and the activation functions being the Tanh function and the Relu function respectively, and the output shape is the auxiliary sequence feature of 256×m×160.

[0041] Concatenate the DDM sequence feature and the auxiliary sequence feature and input them into the fully connected layer. The model has a total of three fully connected layers, with the number of neurons being 256, 128, and 64 respectively. In order to increase the non-linear expression ability of the model and alleviate the gradient disappearance, the activation functions of the three layers are the tanh function, the Relu function, and the Relu function respectively. The finally output shape is [256,m], representing m wind speed inversion values corresponding to a DDM sequence feature and the auxiliary sequence feature vector.

[0042] On this basis, in step S104, use the training set and the validation set to train the wind speed inversion model, including: setting the training period for training the wind speed inversion model; using the training set and the validation set, combining the L2 regularization strategy and the early stopping method to train the wind speed inversion model.

[0043] Exemplarily, the Adam optimizer is used in the training process, and the training period (epochs) is set to 256. The L2 regularization strategy is adopted in each neural network layer of the wind speed inversion model to prevent overfitting. In addition, the early stopping method is also adopted to prevent overfitting, with a tolerance of 10 epochs, that is, if the validation set loss value does not decrease for 10 consecutive epochs, the training stops and the network weights are restored to the state when the validation set loss is the lowest.

[0044] Correspondingly, the performance of the wind speed inversion model is evaluated and the error is analyzed by using the test set, including: determining the error index of each sub-trajectory based on the wind speed inversion model; and determining the overall error of the wind speed inversion model according to the average value of the error indexes of all sub-trajectories. Among them, the error index of the sub-trajectory includes at least one of the root mean square error (RMSE), the overall bias (Bias), and the mean absolute percentage error (MAPE) of each sub-trajectory.

[0045] Exemplarily, for the calculation of the inversion accuracy of the wind speed inversion model, the error index of each sub-trajectory is calculated first and then averaged. For the th sub-trajectory, its , , are defined as:

[0046]

[0047]

[0048] Among them, represents the th sub-trajectory, represents the root mean square error of the th sub-trajectory, represents the overall bias of the th sub-trajectory, represents the mean absolute percentage error of the th sub-trajectory, represents the number of samples included in the th sub-trajectory, , respectively represent the th inversion wind speed and the corresponding reference wind speed in the th sub-trajectory. The overall error is the average value of the errors of all sub-trajectories. Assuming there are sub-trajectories in total, the overall error can be written as:

[0049]

[0050]

[0051] Among them, represents the overall root mean square error, represents the overall bias of the model, It represents the overall mean absolute percentage error, and m represents the number of sub-trajectories. Through the above evaluation method, in this embodiment, the RMSE of the wind speed inversion model is 1.23 m / s, the Bias is -0.041 m / s, and the MAPE is 20.8%. The wind speed inversion model is used to predict the wind speed for the to-be-measured DDM data. After obtaining the new CYGNSS DDM data, the wind speed can be directly inverted using this wind speed inversion model, and the global distribution can be given.

[0052] In summary, for the problems that the existing neural network methods do not fully utilize the spatio-temporal correlation information between consecutive observations on the observation trajectory and do not consider the influence of ocean state information on the wind speed inversion accuracy. Compared with other neural network models, this method utilizes the spatio-temporal correlation between adjacent DDMs in the same observation trajectory, and at the same time integrates information such as observation geometry, receiver hardware characteristics, and ocean state to achieve high-precision wind speed inversion.

[0053] The present invention also provides a multi-modal information fusion sea surface wind speed inversion device. The multi-modal information fusion sea surface wind speed inversion device provided by the present invention will be described below. The multi-modal information fusion sea surface wind speed inversion device described below can be mutually referred to the multi-modal information fusion sea surface wind speed inversion method described above. Figure 4 is the structural block diagram of the multi-modal information fusion sea surface wind speed inversion device provided by the present invention, as Figure 4 shown, the device includes: An acquisition module 401, configured to acquire satellite navigation data, wind speed reference data, and significant wave height data, and match and perform quality control on the wind speed reference data and the significant wave height data with the satellite navigation data respectively to obtain an initial data set; A recombination module 402, configured to recombine the initial data set according to the trajectory to obtain a target data set, and divide the target data set into a training set, a validation set, and a test set; A construction module 403, configured to utilize the spatio-temporal correlation of consecutive delay-Doppler map observations in the same trajectory, and combine auxiliary feature information to construct a wind speed inversion model; A training module 404, configured to train the wind speed inversion model using the training set and the validation set, and perform performance evaluation and error analysis on the wind speed inversion model using the test set; A prediction module 405, configured to acquire the to-be-measured delay-Doppler map data, and perform wind speed prediction through the wind speed inversion model to obtain a prediction result.

[0054] When the present device is in use, first, the acquisition module 401 acquires satellite navigation data, wind speed reference data, and significant wave height data, matches the wind speed reference data and the significant wave height data with the satellite navigation data respectively, and performs quality control to obtain an initial data set. Then, the recombination module 402 uses the initial data set after the above-mentioned quality control to perform recombination according to the trajectory, and divides the recombined target data set into a training set, a validation set, and a test set. Due to the rough sea surface, the ocean state, the observation geometry, and the receiver hardware parameters will also affect the accuracy of wind speed inversion. Especially in a low wind speed environment, the swell caused by the non-local wind field has a particularly significant impact on the sea surface roughness, directly affecting the accuracy of wind speed inversion. Therefore, the construction module 403 of the present device utilizes the spatio-temporal correlation of the continuous Delay-Doppler Map (DDM) observations in the same trajectory, fuses auxiliary feature information such as the ocean state, the observation geometry, and the receiver hardware parameters, and constructs a wind speed inversion model. Then, the training module 404 uses the training set and the validation set to train the wind speed inversion model, uses the test set to perform performance evaluation and error analysis on the wind speed inversion model, and fine-tunes the wind speed inversion model to improve the prediction accuracy of the wind speed inversion model. Finally, the prediction module 405 acquires the to-be-measured Delay-Doppler Map data, and performs wind speed prediction through the fine-tuned wind speed inversion model to obtain a prediction result.

[0055] In the above process, aiming at the problem that most current multi-modal neural networks only take a single Delay-Doppler Map as the input, do not fully utilize the strong spatio-temporal correlation between the continuous Delay-Doppler Maps observed on the same trajectory, and few models simultaneously fuse the observation trajectory and the ocean state information, resulting in a loss of wind speed inversion accuracy, a wind speed inversion model is constructed. This model can effectively extract the spatio-temporal correlation information between the continuous observations on the observation trajectory, can fuse the extracted information with various auxiliary feature information to improve the inversion accuracy, and can also quantitatively analyze the influence characteristics of the ocean state on wind speed inversion. Compared with the existing technologies, the present device shows obvious advantages in various evaluation indexes, obtains more accurate and reliable wind speed inversion results, can meet the needs of atmospheric studies related to the sea surface wind field using a large amount of GNSS-R satellite data, and solves the problem of poor inversion accuracy existing in the existing related technologies.

[0056] The present invention also provides a multi-modal information fusion sea surface wind speed inversion system, Figure 5 which is a schematic structural diagram of the multi-modal information fusion sea surface wind speed inversion system provided by the present invention. As Figure 5 shown, the system includes: A data download and storage module, which is used to acquire satellite navigation data, wind speed reference data, and significant wave height data; A data preprocessing module, which is used to match and perform quality control on wind speed reference data and significant wave height data with satellite navigation data respectively to obtain an initial data set; A data integration and generation module, which is used to reorganize the initial data set according to trajectories to obtain a target data set, and divide the target data set into a training set, a validation set and a test set; A wind speed inversion model generation module, which is used to utilize the spatio-temporal correlation of consecutive time-delay Doppler map observations in the same trajectory, combine with auxiliary feature information to construct a wind speed inversion model, and train the wind speed inversion model by using the training set and the validation set; An accuracy evaluation and result output module, which is used to perform performance evaluation and error analysis on the wind speed inversion model by using the test set, obtain the time-delay Doppler map data to be measured, and perform wind speed prediction through the wind speed inversion model to obtain a prediction result.

[0057] Figure 6 An entity structure diagram of an electronic device is exemplified, as Figure 6 shown. The electronic device may include: a processor 601, a communications interface 602, a memory 603, and a communication bus 604. Among them, the processor 601, the communications interface 602, and the memory 603 complete mutual communication through the communication bus 604. The processor 601 can call the logical instructions in the memory 603 to execute a multi-modal information fusion sea surface wind speed inversion method, and this method includes: Obtain satellite navigation data, wind speed reference data and significant wave height data, and match and perform quality control on the wind speed reference data and the significant wave height data with the satellite navigation data respectively to obtain an initial data set; Reorganize the initial data set according to trajectories to obtain a target data set, and divide the target data set into a training set, a validation set and a test set; Utilize the spatio-temporal correlation of consecutive time-delay Doppler map observations in the same trajectory, combine with auxiliary feature information to construct a wind speed inversion model; Train the wind speed inversion model by using the training set and the validation set, and perform performance evaluation and error analysis on the wind speed inversion model by using the test set; Obtain the time-delay Doppler map data to be measured, and perform wind speed prediction through the wind speed inversion model to obtain a prediction result.

[0058] In addition, when the logical instructions in the above-mentioned memory 603 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0059] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-modal information fusion sea surface wind speed inversion method provided by the above-mentioned various methods. The method includes: Obtain satellite navigation data, wind speed reference data, and significant wave height data, and respectively match and perform quality control on the wind speed reference data and significant wave height data with the satellite navigation data to obtain an initial data set; Reorganize the initial data set according to the trajectory to obtain a target data set, and divide the target data set into a training set, a validation set, and a test set; Utilize the spatio-temporal correlation of consecutive time-delay Doppler map observations in the same trajectory, combined with auxiliary feature information, to construct a wind speed inversion model; Train the wind speed inversion model using the training set and the validation set, and perform performance evaluation and error analysis on the wind speed inversion model using the test set; Obtain the to-be-measured time-delay Doppler map data, and perform wind speed prediction through the wind speed inversion model to obtain a prediction result.

[0060] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the multi-modal information fusion sea surface wind speed inversion method provided by the above-mentioned various methods. The method includes: Obtain satellite navigation data, wind speed reference data, and significant wave height data, and respectively match and perform quality control on the wind speed reference data and significant wave height data with the satellite navigation data to obtain an initial data set; Reorganize the initial data set according to the trajectory to obtain a target data set, and divide the target data set into a training set, a validation set, and a test set; Utilize the spatio-temporal correlation of consecutive time-delay Doppler map observations in the same trajectory, and combine with auxiliary feature information to construct a wind speed inversion model; Use the training set and the validation set to train the wind speed inversion model, and use the test set to conduct performance evaluation and error analysis on the wind speed inversion model; Obtain the time-delay Doppler map data to be measured, and perform wind speed prediction through the wind speed inversion model to obtain the prediction result.

[0061] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0062] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, also by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., including several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A multi-modal information fusion method for retrieving sea surface wind speed, characterized in that, Including: Obtain satellite navigation data, wind speed reference data, and significant wave height data, and match and perform quality control on the wind speed reference data and the significant wave height data with the satellite navigation data respectively to obtain an initial data set; Reorganize the initial data set according to trajectories to obtain a target data set, and divide the target data set into a training set, a validation set, and a test set; Utilize the spatio-temporal correlation of consecutive time-delay Doppler map observations in the same trajectory, combined with auxiliary feature information, to construct a wind speed inversion model; Train the wind speed inversion model using the training set and the validation set, and perform performance evaluation and error analysis on the wind speed inversion model using the test set; Obtain the to-be-measured time-delay Doppler map data, and perform wind speed prediction through the wind speed inversion model to obtain a prediction result.

2. The multimodal information fusion sea surface wind speed inversion method according to claim 1, characterized in that Match and perform quality control on the wind speed reference data and the significant wave height data with the satellite navigation data respectively to obtain an initial data set, including: Match the wind speed reference data and the significant wave height data with the satellite navigation data respectively based on the sampling point longitude and latitude and time; Filter the matched data according to preset filtering conditions to obtain the initial data set.

3. The multimodal information fusion sea surface wind speed inversion method according to claim 1, wherein Reorganize the initial data set according to trajectories to obtain a target data set, including: Distinguish the observation samples of different trajectories in the target data set through trajectory labels; Divide the data of different trajectories into sub-trajectories containing a target number of observation samples to obtain the target data set.

4. The multimodal information fusion sea surface wind speed inversion method according to claim 3, characterized in that Dividing the data of different trajectories into sub-trajectories containing a target number of observation samples includes: For each trajectory, if the total number of observation samples in the trajectory is an integer multiple of the target number, evenly divide it into several sub-trajectories starting from the first observation sample in the trajectory; If the total number of observation samples in the trajectory is not an integer multiple of the target number, discard the last group of observation samples whose quantity does not reach the target number.

5. The multimodal information fusion sea surface wind speed inversion method according to claim 1, wherein Utilize the spatio-temporal correlation of consecutive time-delay Doppler map observations in the same trajectory, combined with auxiliary feature information, to construct a wind speed inversion model, including: Capture the spatio-temporal correlation information of the time-delay Doppler map data through a long short-term memory network and a BP neural network to obtain DDM sequence features; Obtain auxiliary feature information, and perform preprocessing on the auxiliary feature information to obtain auxiliary sequence features; Concatenate the DDM sequence features and the auxiliary sequence features, and construct the wind speed inversion model.

6. The multimodal information fusion sea surface wind speed inversion method according to claim 1, characterized in that Train the wind speed inversion model using the training set and the validation set, including: Set the training epochs for training the wind speed inversion model; Utilize the training set and the validation set, combined with the L2 regularization strategy and early stopping method, to train the wind speed inversion model.

7. The multimodal information fusion sea surface wind speed inversion method according to claim 1, characterized in that Perform performance evaluation and error analysis on the wind speed inversion model using the test set, including: Determine the error index of each sub-trajectory based on the wind speed inversion model; Determine the overall error of the wind speed inversion model according to the average value of the error indexes of all sub-trajectories.

8. The multi-modal information fusion sea surface wind speed inversion method according to claim 7, wherein, The error metrics of the sub-trajectories include at least one of the root mean square error, the overall deviation, and the mean absolute percentage error of each of the sub-trajectories.

9. A multimodal information fusion sea surface wind speed inversion device, characterized in that, Comprising: An acquisition module, configured to acquire satellite navigation data, wind speed reference data, and significant wave height data, and perform matching and quality control on the wind speed reference data and the significant wave height data with the satellite navigation data respectively to obtain an initial data set; A recombination module, configured to recombine the initial data set according to trajectories to obtain a target data set, and divide the target data set into a training set, a validation set, and a test set; A construction module, configured to construct a wind speed inversion model by using the spatio-temporal correlation of consecutive time-delay Doppler map observations in the same trajectory and combining auxiliary feature information; A training module, configured to train the wind speed inversion model by using the training set and the validation set, and perform performance evaluation and error analysis on the wind speed inversion model by using the test set; A prediction module, configured to acquire time-delay Doppler map data to be measured, and perform wind speed prediction through the wind speed inversion model to obtain a prediction result.

10. A multi-modal information fusion sea surface wind speed inversion system, characterized in that, Comprising: A data download and storage module, configured to acquire satellite navigation data, wind speed reference data, and significant wave height data; A data preprocessing module, configured to perform matching and quality control on the wind speed reference data and the significant wave height data with the satellite navigation data respectively to obtain an initial data set; A data integration generation module, configured to recombine the initial data set according to trajectories to obtain a target data set, and divide the target data set into a training set, a validation set, and a test set; A wind speed inversion model generation module, configured to construct a wind speed inversion model by using the spatio-temporal correlation of consecutive time-delay Doppler map observations in the same trajectory and combining auxiliary feature information, and train the wind speed inversion model by using the training set and the validation set; An accuracy evaluation and result output module, configured to perform performance evaluation and error analysis on the wind speed inversion model by using the test set, acquire time-delay Doppler map data to be measured, and perform wind speed prediction through the wind speed inversion model to obtain a prediction result.

Images