A super-resolution reconstruction method, device, equipment and medium for sea surface height
Through deep learning model, multi-channel features of sea surface height, wind speed and sea surface temperature are extracted, and high-resolution reconstruction of sea surface height is achieved, solving the problem of insufficient small-scale monitoring resolution of sea surface dynamics process in the existing technology, and improving monitoring Accuracy.
Patent Information
- Application Number
- CN202411023593.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-29
AI Technical Summary
The existing sea surface height monitoring methods cannot effectively handle the sea surface dynamics process at small scales, resulting in insufficient global monitoring resolution of the sea surface flow field and the inability to accurately monitor the sea surface dynamics process at smaller scales.
Deep learning technology is used to construct an image super-resolution model, and the high-resolution heavy-resolution heavy-resolution heavy-resolution heavy-resolution sea surface altitude, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature are extracted. structure.
Compared with existing statistical and interpolation schemes, deep learning models can more accurately reconstruct high-resolution sea surface heights, improving the resolution and accuracy of monitoring small-scale sea surface dynamics processes.
Smart Images

Figure CN118552410B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of ocean data processing technology, and in particular to a method, device, equipment and medium for super-resolution reconstruction of sea surface height. Background Art
[0002] As a variable describing the physical state of the ocean, sea surface height plays an important role in the Earth system, regulating the Earth's climate and the function of the marine ecosystem. It is widely used to estimate geostrophic currents and analyze sea surface dynamic processes. Sea surface dynamic processes cover a wide range of scales, with the smallest even reaching several kilometers.
[0003] However, current observation and processing methods cannot fully handle these scales. In the past, global monitoring of sea surface flow fields has been achieved by converting sea surface height maps into flow field maps using the geostrophic balance method. These sea surface height maps are derived from the statistical interpolation results of along-track satellite altimeter data. Due to gaps and repetitions between satellite scanning orbits, the effective resolution of the interpolated sea surface height maps is limited to tens of kilometers, which cannot be used to monitor smaller-scale sea surface dynamic processes. Summary of the invention
[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.
[0005] A first aspect of an embodiment of the present application provides a super-resolution reconstruction method for sea surface height, the super-resolution reconstruction method for sea surface height comprising:
[0006] Obtain the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured;
[0007] The coarse-resolution sea surface height, sea surface meridional wind speed, sea surface latitudinal wind speed and high-resolution sea surface temperature of the area to be measured are input into a preset deep learning model, so that the deep learning model extracts different-scale features between the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface latitudinal wind speed and high-resolution sea surface temperature of the area to be measured, and reconstructs the high-resolution sea surface height according to the different-scale features.
[0008] An embodiment of the present application provides a super-resolution reconstruction method for sea surface height. The method introduces deep learning technology to construct an image super-resolution model. Multi-channel and different-scale feature information of coarse-resolution sea surface height, sea surface meridional wind speed, sea surface latitudinal wind speed and high-resolution sea surface temperature is extracted through the deep model. The mapping relationship between the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface latitudinal wind speed and high-resolution sea surface temperature and the high-resolution sea surface height is inferred through the multi-channel and different-scale feature information, thereby realizing high-resolution reconstruction of the sea surface height. Compared with the existing statistical and interpolation schemes, the sea surface height reconstructed by the method is more accurate.
[0009] In some embodiments of the present application, the deep learning model includes a first element convolution stacking module, a second element convolution stacking module and a feature processing module, wherein the first element convolution stacking module includes three stacking submodules, the second element convolution stacking module includes two stacking submodules, and the stacking submodule includes a plurality of parallel subnetworks, each subnetwork has a convolution layer and an activation layer, and the convolution layer in each subnetwork has a different convolution kernel from the convolution layers in other subnetworks;
[0010] The coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured are input into a preset deep learning model, so that the deep learning model extracts different scale features between the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured, and reconstructs the high-resolution sea surface height according to the different scale features, including:
[0011] The coarse-resolution sea surface height data, sea surface meridional wind speed data and sea surface zonal wind speed data of the area to be measured are respectively input into a corresponding stacking submodule in the first element convolution stacking module, so as to extract the stacking features of the corresponding input data through the corresponding stacking submodule, and stack the stacking features of the three stacking submodules to obtain a stacking feature map output by the first element convolution stacking module; the stacking feature refers to the feature obtained by extracting and stacking a plurality of different size features of the corresponding input data by the stacking submodule;
[0012] Inputting the high-resolution sea surface temperature data of the area to be measured into the second element convolution stacking module, extracting the stacking features of the high-resolution sea surface temperature data through the first stacking submodule of the second element convolution stacking module, performing bilinear interpolation down-sampling on the stacking features of the high-resolution sea surface temperature data, and obtaining a high-resolution feature map of the high-resolution sea surface temperature data, and extracting the stacking features of the high-resolution feature map through the second stacking submodule of the second element convolution stacking module, and performing bilinear interpolation down-sampling on the stacking features of the high-resolution feature map to obtain a low-resolution feature map;
[0013] The stacked feature map and the low-resolution feature map are stacked in the channel dimension to obtain a multi-channel feature map, and the multi-channel feature map is input into the feature processing module so that the feature processing module reconstructs high-resolution sea surface height data according to the multi-channel feature map.
[0014] In some embodiments of the present application, the stacking submodule includes three parallel sub-networks;
[0015] The expression of the first element convolution stacking module extracting the stacked feature map is:
[0016]
[0017] in, They represent the stacking characteristics of the coarse-resolution sea surface height data, sea surface meridional wind speed data, and sea surface zonal wind speed data extracted by the stacking submodule, respectively. represents the mapping function of the stacking operation, Represents the stacked feature map extracted by the first element convolution stack module;
[0018] The stacking submodule extracts the corresponding stacking feature expression as follows:
[0019]
[0020] in, Indicates one of the following element data: sea surface height data, sea surface meridional wind speed data, and sea surface zonal wind speed data with coarse resolution. Respectively The convolution kernel mapping function, represents the ReLU activation function, represents the mapping function of the stacked submodule, Indicates the stacking submodule extracted The stacking characteristics of
[0021] The expression for extracting high-resolution feature maps and low-resolution feature maps by the second element convolution stacking module is:
[0022]
[0023]
[0024] in, represents high-resolution sea surface temperature data, Both represent the mapping function of bilinear interpolation downsampling, They respectively represent the high-resolution feature map and low-resolution feature map extracted by the second element convolution stacking module.
[0025] In some embodiments of the present application, the feature processing module includes a high-fidelity convolution feature extraction module and an output module;
[0026] The step of inputting the multi-channel feature map into the feature processing module so that the feature processing module reconstructs high-resolution sea level height data according to the multi-channel feature map comprises:
[0027] Extracting corresponding features from the multi-channel feature map according to the high-fidelity convolutional feature extraction module;
[0028] Inputting the features extracted from the multi-channel feature map by the high-fidelity convolution feature extraction module into the output module, so as to reconstruct high-resolution sea surface height data through the output module;
[0029] Among them, the expression of extracting features by the high-fidelity convolution feature extraction module is:
[0030]
[0031] in, represents the input features of the high-fidelity convolutional feature extraction module, represents the mapping function of the high-fidelity convolutional feature extraction module, Represents the mapping function of the Inception module, represents the mapping function of the stacking operation, Represents the mapping function of CBAM convolutional attention, Represents the addition of corresponding elements of two feature map tensors of the same size, represents the output features of the high-fidelity convolutional feature extraction module, represents the mapping function of bilinear interpolation downsampling, Represents the mapping function of bilinear interpolation upsampling;
[0032] Among them, the expression of the Inception module is:
[0033]
[0034] in, Represents the output features of the Inception module, Represents the mapping function of the Inception module, Represents the input features of the Inception module, represents the mapping function of the maximum pooling layer, Respectively The convolution kernel mapping function.
[0035] In some embodiments of the present application, the number of the high-fidelity convolutional feature extraction modules is seven;
[0036] The extracting corresponding features from the multi-channel feature map according to the high-fidelity convolution feature extraction module includes:
[0037] The multi-channel feature map is input into the first high-fidelity convolutional feature extraction module to obtain the output feature of the first high-fidelity convolutional feature extraction module. ;
[0038] The characteristics After bilinear interpolation downsampling, the image is input to the second high-fidelity convolution feature extraction module to obtain the output features of the second high-fidelity convolution feature extraction module. ;
[0039] The characteristics After bilinear interpolation downsampling, the data is input into the third high-fidelity convolution feature extraction module to obtain the output features of the third high-fidelity convolution feature extraction module. ;
[0040] The characteristics After bilinear interpolation downsampling, the image is input to the fourth high-fidelity convolution feature extraction module to obtain the output features of the fourth high-fidelity convolution feature extraction module. ;
[0041] The characteristics After bilinear interpolation upsampling and feature The stacked features are input into the fifth high-fidelity convolutional feature extraction module to obtain the output features of the fifth high-fidelity convolutional feature extraction module. ;
[0042] The characteristics After bilinear interpolation upsampling and feature The stacked features are input into the sixth high-fidelity convolutional feature extraction module to obtain the output features of the sixth high-fidelity convolutional feature extraction module. ;
[0043] The characteristics After bilinear interpolation upsampling and feature The stacked features are input into the seventh high-fidelity convolutional feature extraction module to obtain the output features of the seventh high-fidelity convolutional feature extraction module. , the feature As the high-fidelity convolution feature extraction module, corresponding features are extracted from the multi-channel feature map.
[0044] In some embodiments of the present application, the expression for reconstructing high-resolution sea surface height data by the output module is:
[0045]
[0046] in, Represents the high-resolution sea surface height data reconstructed by the output module, represents the input features of the output module, represents the mapping function of the output module, represents the mapping function of the BN layer, Represents the ReLU activation function.
[0047] In some embodiments of the present application, the loss function of the deep learning model is:
[0048]
[0049]
[0050] in, Represents the mapping function of the L1 loss function, Represents the mapping relationship between the input data and output data of the deep learning model, Represents coarse-resolution sea surface height data, sea surface meridional wind speed data, sea surface zonal wind speed data and high-resolution sea surface temperature data, Indicates the first The weights and biases of the layers, represents the high-resolution sea surface height data reconstructed by the deep learning model, Represents high-resolution sea surface height data as a label.
[0051] A second aspect of an embodiment of the present application provides a super-resolution reconstruction device for sea level, the super-resolution reconstruction device for sea level comprising:
[0052] A data acquisition unit, used to acquire the sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured;
[0053] A data processing unit is used to input the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface latitudinal wind speed and high-resolution sea surface temperature of the area to be measured into a preset deep learning model, so that the deep learning model extracts different scale features among the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface latitudinal wind speed and high-resolution sea surface temperature of the area to be measured, and reconstructs the high-resolution sea surface height according to the different scale features.
[0054] The electronic device provided in the third aspect of an embodiment of the present application includes: at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute a super-resolution reconstruction method of sea surface height according to the first aspect mentioned above.
[0055] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a super-resolution reconstruction method of sea surface height according to the first aspect.
[0056] It can be understood that the beneficial effects of the second to fourth aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0058] Figure 1 It is a flow chart of a super-resolution reconstruction method of sea surface height provided by an embodiment of the present application;
[0059] Figure 2 It is a schematic diagram of the training process of a deep learning model provided by an embodiment of the present application;
[0060] Figure 3 It is a structural diagram of a first element convolution stacking module provided by an embodiment of the present application;
[0061] Figure 4 is a structural diagram of an Inception module provided by an embodiment of the present application;
[0062] Figure 5 is a structural diagram of a CA channel attention module provided by an embodiment of the present application;
[0063] Figure 6 is a structural diagram of an SA spatial attention module provided by an embodiment of the present application;
[0064] Figure 7 It is a structural diagram of a CBAM convolutional attention module provided by an embodiment of the present application;
[0065] Figure 8 is a structural diagram of a high-fidelity convolutional feature extraction module provided by an embodiment of the present application;
[0066] Fig. 9 is a structural diagram of an output module provided by an embodiment of the present application;
[0067] Fig.10 An embodiment of the present application provides a structural diagram of a deep learning model;
[0068] Fig.11 It is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0069] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0070] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0071] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0072] Reference Figure 1 An embodiment of the present application provides a super-resolution reconstruction method for sea surface height, and the super-resolution reconstruction method for sea surface height includes steps S110 to S120:
[0073] Step S110, obtaining the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured.
[0074] Step S120, input the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface latitudinal wind speed and high-resolution sea surface temperature into a preset deep learning model, so that the deep learning model extracts different scale features between the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface latitudinal wind speed and high-resolution sea surface temperature, and reconstructs the high-resolution sea surface height according to the different scale features.
[0075] In step S110, the area to be tested refers to the area in the ocean where the sea surface height needs to be detected. In step S110, four types of data are provided, namely, the sea surface height with coarse resolution, the sea surface meridional wind speed, the sea surface zonal wind speed, and the sea surface temperature with high resolution. In step S120, the high-resolution data reconstruction of the sea surface height will be completed using these four types of data using super-resolution reconstruction technology. It should be noted that in this field, the coarse resolution and high resolution of the sea surface height can be distinguished by a threshold, and the high resolution of the sea surface temperature can also be distinguished by a threshold.
[0076] In step S120, the method introduces image super-resolution technology to construct a deep learning model, extracts multi-channel and different-scale feature information of coarse-resolution sea surface height data, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature through the deep model, and infers the mapping relationship between coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature and high-resolution sea surface height through multi-channel and different-scale feature information, thereby realizing high-resolution reconstruction of sea surface height. Compared with existing statistical and interpolation schemes, the method can reconstruct high-resolution sea surface height more accurately and efficiently.
[0077] Reference Figure 2 The deep learning model preset in the above step S120 is trained in the following manner:
[0078] Step S210, construct a deep learning model including a first element convolution stacking module, a second element convolution stacking module and a feature processing module, the first element convolution stacking module includes three stacking sub-modules, the second element convolution stacking module includes two stacking sub-modules, the stacking sub-module includes multiple parallel sub-networks, wherein each sub-network has a convolution layer and an activation layer and the convolution layer in each sub-network has a different convolution kernel from the convolution layers in other sub-networks.
[0079] Step S220, the coarse-resolution sea surface height data, sea surface meridional wind speed data and sea surface latitudinal wind speed data serving as training data are respectively input into a corresponding stacking submodule in the first element convolution stacking module, so as to extract the stacking features of the corresponding input data through a corresponding stacking submodule, and stack all the stacking features to obtain a stacking feature map output by the first element convolution stacking module; the stacking feature refers to the feature obtained by extracting multiple different size features of the corresponding input data and stacking them by the stacking submodule.
[0080] Step S230, input the high-resolution sea surface temperature data as training data into the second element convolution stacking module, extract the stacking features of the high-resolution sea surface temperature data through the first stacking submodule of the second element convolution stacking module, perform bilinear interpolation down-sampling on the stacking features of the high-resolution sea surface temperature data, and obtain a high-resolution feature map of the high-resolution sea surface temperature data, extract the stacking features of the high-resolution feature map through the second stacking submodule, and perform bilinear interpolation down-sampling on the stacking features of the high-resolution feature map to obtain a low-resolution feature map.
[0081] Step S240: stack the stacked feature map and the low-resolution feature map in the channel dimension to obtain a multi-channel feature map, and input the multi-channel feature map into the feature processing module so that the feature processing module reconstructs high-resolution sea surface height data according to the multi-channel feature map.
[0082] In the above step S210, the deep learning model includes the following three parts:
[0083] The first element convolution stacking module is composed of three stacking submodules in parallel, and the three stacking submodules are used to process the coarse-resolution sea surface height data, the sea surface meridional wind speed data, and the sea surface zonal wind speed data respectively. The stacking submodule has multiple parallel subnetworks, each of which has a convolution layer and an activation layer, and the convolution layer in each subnetwork has a different convolution kernel from the convolution layer in other subnetworks, which is used to extract the multi-scale features of each element data, such as extracting the multi-scale features of the coarse-resolution sea surface height data.
[0084] The second element convolution stacking module includes two stacking sub-modules. The second element convolution stacking module acts on high-resolution sea surface temperature data to extract high-resolution features and low-resolution features of the high-resolution sea surface temperature data.
[0085] The feature processing module is used to reconstruct high-resolution sea surface height data based on the fused features between the first element convolution stacking module and the second element convolution stacking module.
[0086] In step S220 to step S230, the training data includes: coarse resolution sea surface height data, sea surface meridional wind speed data, sea surface zonal wind speed data, and high resolution sea surface temperature data. The training data here can be historical data of the area to be tested, and the data source is detailed in the subsequent embodiments.
[0087] The coarse-resolution sea surface height data, sea surface meridional wind speed data, and sea surface zonal wind speed data are respectively input into a corresponding stacking submodule in the first element convolution stacking module, so as to extract the stacking features of the corresponding input data through a corresponding stacking submodule, and stack all the stacking features to obtain a stacking feature map output by the first element convolution stacking module. Taking the coarse-resolution sea surface height data as an example, the coarse-resolution sea surface height data is input into a corresponding stacking submodule, and multiple parallel subnetworks of the stacking submodule respectively extract features of corresponding scales therefrom, and then stack them to obtain a stacking feature map (features of multiple scales) corresponding to the coarse-resolution sea surface height data output by the stacking submodule.
[0088] The high-resolution sea surface temperature data is input into the second element convolution stacking module, the stacking features of the high-resolution sea surface temperature data are first extracted through the first stacking submodule of the second element convolution stacking module, the stacking features of the high-resolution sea surface temperature data are bilinearly interpolated and down-sampled to obtain a high-resolution feature map of the high-resolution sea surface temperature data, the stacking features of the high-resolution feature map are extracted through the second stacking submodule, the stacking features of the high-resolution feature map are bilinearly interpolated and down-sampled to obtain a low-resolution feature map.
[0089] The high-resolution sea surface temperature data and the other three elements are processed separately. This is because the sea surface temperature data is high-resolution, so the feature extraction of the stacking submodule is performed twice. To ensure sufficient information extraction, the original high-resolution temperature data is first subjected to feature extraction of the stacking submodule, and then its resolution is reduced to the same resolution as the coarse-resolution sea surface height, and then input into the stacking submodule for feature extraction. This can avoid the loss of high-resolution detail information caused by reducing the resolution of the sea surface temperature at the beginning and then extracting features.
[0090] In some embodiments, the feature processing module in step S240 includes a high-fidelity convolution feature extraction module and an output module;
[0091] Step S240 inputs the multi-channel feature map into the feature processing module, so that the feature processing module reconstructs high-resolution sea surface height data according to the multi-channel feature map, including steps S241 to S242:
[0092] Step S241, extracting corresponding features from the multi-channel feature map according to the high-fidelity convolution feature extraction module;
[0093] Step S242: input the features extracted from the multi-channel feature map by the high-fidelity convolution feature extraction module into the output module, so as to reconstruct high-resolution sea surface height data through the output module.
[0094] Among them, the expression of the high-fidelity convolution feature extraction module is:
[0095]
[0096] in, represents the input features of the high-fidelity convolutional feature extraction module, represents the mapping function of the high-fidelity convolutional feature extraction module, Represents the mapping function of the Inception module, represents the mapping function of the stacking operation, Represents the mapping function of CBAM convolutional attention, Represents the addition of corresponding elements of two feature map tensors of the same size, represents the output features of the high-fidelity convolutional feature extraction module, represents the mapping function of bilinear interpolation downsampling, Represents the mapping function of bilinear interpolation upsampling;
[0097] The expression of the Inception module is:
[0098]
[0099] in, Represents the output features of the Inception module, Represents the mapping function of the Inception module, Represents the input features of the Inception module, represents the mapping function of the maximum pooling layer, Respectively The convolution kernel mapping function.
[0100] This embodiment is provided with a high-fidelity convolution feature extraction module, through which the following beneficial effects can be achieved:
[0101] In order to reduce the computational cost, the feature map after feature extraction by one layer of Inception module is downsampled to reduce the size of the feature map to reduce the computational cost and feature redundancy, so that feature extraction by more layers of Inception modules can be performed; after feature extraction, it is upsampled to the original size through bilinear interpolation and channel stacked with the feature map of the original size to retain the initial detail information.
[0102] There is a CBAM convolutional attention layer in the high-fidelity convolutional feature extraction module. The channel attention in CBAM can focus on the channels that play a greater role in the multi-channel feature map, while the spatial attention layer can focus on the areas in the same channel that play an important role in the super-resolution of the sea surface height.
[0103] In addition, the high-fidelity convolutional feature extraction module is equipped with an Inception module, which extracts features in parallel by stacking convolution kernels of different sizes and splicing feature maps of different scales to obtain richer and more diverse feature representations.
[0104] For specific details, please refer to the subsequent embodiments.
[0105] In some embodiments, extracting corresponding features from the multi-channel feature map according to the high-fidelity convolution feature extraction module in step S241 includes steps S2411 to S2417:
[0106] Step S2411: Input the multi-channel feature map into the first high-fidelity convolution feature extraction module to obtain the output feature map of the first high-fidelity convolution feature extraction module. .
[0107] Step S2412: After bilinear interpolation downsampling, the image is input to the second high-fidelity convolution feature extraction module to obtain the output features of the second high-fidelity convolution feature extraction module. .
[0108] Step S2413: After bilinear interpolation downsampling, the data is input into the third high-fidelity convolution feature extraction module to obtain the output features of the third high-fidelity convolution feature extraction module. .
[0109] Step S2414: After bilinear interpolation downsampling, the image is input to the fourth high-fidelity convolution feature extraction module to obtain the output features of the fourth high-fidelity convolution feature extraction module. .
[0110] Step S2415: Set the feature After bilinear interpolation upsampling and feature The stacked features are input into the fifth high-fidelity convolutional feature extraction module to obtain the output features of the fifth high-fidelity convolutional feature extraction module. .
[0111] Step S2416: After bilinear interpolation upsampling and feature The stacked features are input into the sixth high-fidelity convolutional feature extraction module to obtain the output features of the sixth high-fidelity convolutional feature extraction module. .
[0112] Step S2417: After bilinear interpolation upsampling and feature The stacked features are input into the seventh high-fidelity convolutional feature extraction module to obtain the output features of the seventh high-fidelity convolutional feature extraction module. .
[0113] This embodiment involves a total of 7 high-fidelity convolutional feature extraction modules, which are connected in a shape of a letter U. This has the following advantages:
[0114] Channels at the same level in the U-shape are stacked. These connections allow direct transmission of information between the encoding layer and the decoding layer. This helps solve the gradient vanishing problem and helps the model better restore image details.
[0115] Through skip connections, the U-shaped structure can pass the features extracted in the encoder stage directly to the decoder stage. This design allows the decoder to utilize richer feature representations, helps alleviate the problem of information loss, and improves the performance of the model.
[0116] See the subsequent embodiments for details.
[0117] Reference Figures 3 to 10 To facilitate the understanding of those skilled in the art, the present application provides a super-resolution reconstruction method of sea surface height, which specifically includes the following steps S910 to S940:
[0118] Step S910: Obtain input data and label data to construct a data set for training and testing a deep learning model. Specifically, steps S911 to S915 are included:
[0119] Step S911: Use the daily average high-resolution (0.05°×0.05°) sea surface temperature data, the daily average coarse-resolution (0.25°×0.25°) sea surface height data, the sea surface meridional wind speed data, and the sea surface zonal wind speed data as input data, and use the daily average high-resolution (0.08°×0.08°) sea surface height data as labels.
[0120] The above data are downloaded from 2010 to 2019 Daily resolution data, including samples, that is Sample data at time instants, samples of all elements are The coverage area is consistent with the longitude range. ( )arrive ( ), latitude range ( )arrive ( ); high resolution (approx. ) Sea surface temperature data comes from OSTIA operational sea surface temperature and ice analysis products; sea surface wind speed (meridional and latitudinal) comes from the Advanced Scatterometer (ASCAT) data product with a resolution of 0.25°×0.25° provided by Ifremer / cersat; coarse resolution ( ) comes from the DUACS SSH satellite sea surface height product; high resolution (approximately )Sea surface height data comes from GLORYS12V1 (Global Ocean Physics Reanalysis) global ocean physics reanalysis product;
[0121] Among them, four elements of data, including coarse-resolution sea surface height, sea surface wind speed (longitudinal and latitudinal), and high-resolution sea surface temperature, are used as input data for the deep learning model; high-resolution sea surface height data are used as labels.
[0122] Step S912: Pair the input data obtained in step S911 corresponding to time and space with the label to obtain a data set of high-resolution and low-resolution data pairs. , dataset The packaging method is:
[0123]
[0124] Among them, the data set Each sample in is denoted as , Expressed as The sample elements, and the spatial coverage of all samples is longitude arrive ,latitude arrive ; Indicates arrive samples, matrix Each row in represents a sample, and each sample contains 5 elements, represented by Indicates arrive The first four are four input elements: coarse-resolution sea surface height data, sea surface meridional wind speed data, sea surface zonal wind speed data, high-resolution sea surface temperature data, and the last one is the high-resolution sea surface height data of the label.
[0125] Step S913: The above-organized data set according to The ratio is divided into training sets , validation set With the test set ; The specific division method is: select 2010 to 2017, that is, samples as training set , 2018 samples as validation set , 365 samples in 2019 as the test set .
[0126] Step S914: training set obtained from step S913 , validation set With the test set Randomly pick samples, as a small batch of input , the input mini-batch Each element of the MinMax normalization method is performed to normalize The normalized small batch of data is recorded as , the MinMax normalization formula is as follows:
[0127]
[0128] in, Represents an input feature in the batch, where is the sea surface height data, is the sea surface meridional wind speed data, is the sea surface zonal wind speed data, High-resolution sea surface temperature, is high-resolution sea surface height data (label); For the training set The minimum value of the elements, For the training set The maximum value of the elements, each element after normalization of the batch is recorded as .
[0129] Step S915: The Convert to a four-dimensional tensor format , abbreviated as , the purpose is to unify all elements into the same dimension; time refers to the time point corresponding to the element, that is, the batch contains samples, corresponding to the number , each sample corresponds to a time point; the channel number at this time One-to-one correspondence with the input elements; longitude and latitude represent the spatial position index of the grid, and the longitude range arrive , latitude range arrive , and the interval between the two is the spatial resolution of the corresponding element.
[0130] Step S920: Build a deep learning model.
[0131] The expression of the deep learning model is as follows:
[0132]
[0133] in, Represents the mapping relationship between the input and output of the deep learning model, The input is normalized data, and the output is a batch of super-resolution sea surface height data , where each output result corresponds one-to-one to the input sample. Indicates the model The weights and biases of the layers. Deep learning model training specifically includes the following steps:
[0134] Build a deep learning environment, install the open source deep learning framework Pytorch based on the Python programming language on the server, and build a deep learning model based on the Pytorch framework. The following (1)-(4) introduces: the first element convolution stacking module, the second element convolution stacking module, the high-fidelity convolution feature extraction module, and the output module.
[0135] (1) Construct the first element convolution stacking module. Figure 3 The module is provided with 3 identical stacking submodules, each of which processes the input element on a channel and outputs a feature map corresponding to a certain element data containing 3 channels, and then stacks the features of the 3 channels output by the 3 stacking submodules in the channel dimension to obtain the stacked features, thereby constructing the first element convolution stacking module.
[0136] Taking the stacking submodule CNNStack-1 as an example, this module processes coarse-resolution sea level height data. The structure in CNNStack-1 is set as 3 layers of convolution operations in parallel in the channel dimension, which are used to extract feature information of different scales of the input data. The number of data channels for inputting a certain element is 1. The first layer consists of 1 1×1 convolution kernel (step size is 1×1, no edge padding) and 1 ReLU activation layer. The second layer consists of 1 1×1 convolution kernel (step size is 1×1, no edge padding), 1 ReLU activation layer, 1 convolution kernel of size 3×3 (step size is 1×1, edge padding 1 layer, and the padding value is set to be consistent with the adjacent value). and 1 ReLU activation layer. The third layer consists of a convolution kernel of size 1×1 (with a step size of 1×1 and no edge padding), a ReLU activation layer, a convolution kernel of size 5×5 (with a step size of 1×1 and 2 layers of edge padding, and the padding value is set to be consistent with the adjacent value) and a ReLU activation layer. The edge padding ensures that the size of the feature map after the convolution operation remains unchanged. After the three layers of parallel and independent convolution operations are completed, the results of the three layers of convolution operations are stacked up in the channel dimension to obtain the stacked features of the sea level height containing 3 channels. The activation layer uses the ReLU activation function, taking the maximum value of 0 and the input, which can increase the nonlinearity of the model and the sparsity of the network. Represents the input of the function, and the ReLU activation function expression is:
[0137]
[0138] The stacking submodule CNNStack-j module is used to process the The stacking submodule of feature data is expressed as:
[0139]
[0140] Among them, the output of the CNNStack-j module is recorded as the feature map , size is , , 160 , the number of grids in the length and width directions of the feature map are indivual; Represents the mapping function of the stacked submodule; It means that the feature maps of three sizes of a certain element after three layers of convolution operations in parallel in the channel dimension are stacked up in the channel dimension, and the sizes of the other dimensions remain unchanged; represents the ReLU activation function, , and Respectively represent 1×1, 3×3 and 5×5 convolution kernel mapping functions; Represents a certain input element after normalization of a certain input batch, is the coarse-resolution sea surface height data, is the sea surface meridional wind speed data, It is the sea surface zonal wind speed data.
[0141] Finally, the processing results of the three CNNStack-j modules are stacked in the channel dimension to obtain the output of the entire first element convolution stacking module, which is expressed as:
[0142]
[0143] in, Represents the output feature map of the first element convolution stack module, with size ; Represents the mapping function of the first element convolutional stack module.
[0144] (2) Construct the second element convolution stacking module to extract the high-resolution features and low-resolution features of the high-resolution sea surface temperature data to obtain high-resolution features and low-resolution features. The module can extract high-resolution sea surface temperature element data High-resolution features (0.08°×0.08°) and low-resolution features (0.25°×0.25°), the expression is as follows:
[0145]
[0146]
[0147] in, represents high-resolution features, represents low-resolution features, represents the mapping function of the stacked submodule, and Represents the mapping function of the bilinear interpolation downsampling layer.
[0148] After a CNNStack-4 module and a bilinear interpolation downsampling layer in the second element convolution stack module, the high-resolution sea surface temperature data The length and width of the sample are downsampled to approximately the original , output high-resolution features , size is , , , The expression is as follows:
[0149]
[0150] in, represents high-resolution features, Represents the mapping function of the CNNStack-4 module, Represents the mapping function of the first layer Bilinear Downsampling bilinear interpolation downsampling layer;
[0151] The high-resolution features obtained After a CNNStack-4 module and a bilinear interpolation downsampling layer, the high-resolution features are The length and width of the downsampled size are approximately , get low-resolution features , size is The length and width dimensions of the grid number and the output of the above stacked submodule are recorded as feature maps are consistent, so they can be stacked in the channel dimension, The expression is as follows:
[0152]
[0153] in, represents low-resolution features, Represents the mapping function of the CNNStack-4 module, Represents the mapping function of the bilinear interpolation downsampling layer.
[0154] Step S924: construct a high-fidelity convolution feature extraction module. The expression of the high-fidelity convolution feature extraction module is:
[0155]
[0156] in, represents the output feature map of the high-fidelity convolutional feature extraction module, represents the mapping function of the high-fidelity convolutional feature extraction module, Represents the mapping function of the Inception module, represents the mapping function of the CBAM convolutional attention module, is a bilinear interpolation downsampling layer that reduces the size to half of the input. It is a bilinear interpolation upsampling layer that increases the size by two times and restores it to its original size. It represents the multi-channel feature map initially input into the high-fidelity convolutional feature extraction module, and ⊕ represents the addition of corresponding elements of two feature map tensors of the same size.
[0157] (3) The following is a detailed introduction to the high-fidelity convolutional feature extraction module:
[0158] The three convolutional layers and one pooling layer are connected in parallel in the channel dimension, and the results of each layer are stacked in the channel dimension to construct the Inception module. Represents the multi-channel feature map of the input, Indicates the number of channels of the input feature map, Indicates the number of channels of the output feature map; the first convolutional layer contains 1×1 convolution kernel, 1 ReLU activation layer, the output feature map is channels; the second convolutional layer contains 1×1 convolution kernel, 1 ReLU activation layer, 3×3 convolution kernels (with a step size of 1×1, 1 layer of edge padding, and the padding value is set to be consistent with the adjacent value), 1 layer of ReLU activation layer, and the output feature map contains channels; the third convolutional layer contains 1×1 convolution kernel, 1 ReLU activation layer, 5×5 convolution kernels (with a step size of 1×1, 2 layers of edge padding, and the padding value is set to be consistent with the adjacent value), 1 layer of ReLU activation layer, and the output feature map contains channels; the 4th pooling layer includes 1 pooling layer (the number of input and output channels is the same), 1×1 convolution kernel, 1 ReLU activation layer, and the output feature map contains channels; the above 4 layers of results will be stacked up in the channel dimension, so the number of channels of the output feature map is , refer to Figure 4 , the expression of the Inception module is as follows:
[0159]
[0160] in, Indicates the output feature map of the Inception module, Represents the mapping function of the Inception module, Represents the multi-channel feature map of the input Inception module, Represents the mapping function of the maximum pooling layer.
[0161] The Inception layer can simultaneously use convolution kernels of different sizes for feature extraction, thereby capturing image features of different scales. Specifically, the Inception module extracts features in parallel by stacking convolution kernels of different sizes and concatenates feature maps of different scales to obtain richer and more diverse feature representations.
[0162] Construct a CBAM convolutional attention module. The input multi-channel feature map first passes through a CA channel attention module, and the channel attention map is weighted summed with the data. Then it is input into a SA spatial attention module, and the spatial attention map is weighted summed with the data to get the final result, which is the process of the CBAM convolutional attention module.
[0163] The number of channels of the input and output feature maps of the CBAM convolutional attention module is consistent; first, the input data passes through a CA channel attention module to obtain the weighted result, and then passes through a SA spatial attention module to obtain the final weighted result;
[0164] Reference Figure 5 , the CA channel attention module inputs the feature map, passes through 1 layer of global maximum pooling layer and 1 layer of global average pooling layer in the length and width dimensions, and then inputs them into the same MLP multi-layer perceptron with 3 fully connected layers, and adds the output results of the two perceptrons element by element, and finally outputs the channel attention feature map through the sigmoid activation layer; the expression of the CA channel attention module is:
[0165]
[0166] in, represents the output feature map of the CA channel attention module, represents the CA channel attention module mapping function, Represents the multi-channel feature map input to the CA channel attention module; Represents the mapping function of the global average pooling layer in the length and width dimensions, using the function in the Pytroch function library To achieve this, the expression is as follows:
[0167]
[0168]
[0169] in, Represents the multi-channel feature map of the input, represents the feature map that is first average-pooled in the length dimension, Indicates that After average pooling in the width dimension, the feature map is Output of the average pooling layer in length and width dimensions, input multi-channel feature map Size , Size , Size ; It will be used here and in the following examples to refer to a feature map of a multi-channel with different sizes. The number of channels, the number of grids in the length direction, and the number of grids in the width direction; The "2" in the function means averaging in the second dimension, that is, the length dimension, and the "3" means averaging in the third dimension, that is, the width dimension. Indicates that the sizes of the remaining dimensions remain unchanged.
[0170] Similarly, Represents the mapping function of the global average pooling layer in the length and width dimensions, which is provided by the function in the Pytroch library To achieve this, the expression is:
[0171]
[0172]
[0173] The size of the input and output feature maps of the MLP multilayer perceptron remains unchanged; Represents the sigmoid activation function layer, and the expression is as follows:
[0174]
[0175] in, Represents the input of the activation function.
[0176] The above channel attention feature map The size is With size Input feature map Multiplying the corresponding dimensions is equivalent to giving all grid points on the corresponding channel the same weight value.
[0177] Reference Figure 6 , and then input into the SA spatial attention module; first, it passes through a global maximum pooling layer of 1 channel dimension and a global average pooling layer of 1 channel dimension, and then these two results are stacked in the channel dimension, and the number of channels is , and then through a 7×7 convolution layer, the dimension is reduced to channels, and then pass through a sigmoid activation layer to obtain the spatial attention feature map. The output feature map size of the spatial attention module is ; The SA spatial attention module can be expressed as:
[0178]
[0179] in, Represents the output feature map of the SA spatial attention module, represents the mapping function of the SA spatial attention module, Represents the multi-channel feature map input to the SA spatial attention module, Represents a 7×7 convolution kernel (with a stride of 1×1, 3 layers of edge padding, and the padding value is set to be consistent with the adjacent value). Represents the channel global average pooling layer, the expression is:
[0180]
[0181] in, represents the feature map after channel global average pooling, "1" means averaging in the first dimension, that is, the channel dimension, Size .
[0182] Similarly, Represents the channel global maximum pooling layer, the expression is:
[0183]
[0184] It represents the feature map after global maximum pooling of the channel. "1" means taking the maximum value in the first dimension, that is, the channel dimension. Size .
[0185] Finally, the spatial attention feature map Multiply it with the corresponding grid of the feature map of the input SA spatial attention module of this module to obtain the output features of the final CBAM module. There is only one channel, and the value of each grid represents the importance of the grid in the plane space, and the multiplication of the corresponding grids is equivalent to giving a spatial weight to the same grid of all channels in the feature map batch; the complete CBAM convolutional attention module function expression is:
[0186]
[0187] in, represents the output feature map of the CBAM convolutional attention module, represents the mapping function of the CBAM convolutional attention module, Represents the multi-channel feature map initially input to the CBAM convolutional attention module, Indicates multiplication of corresponding dimensions.
[0188] Reference Figure 7 ,From the above steps, we can see that the number of input and output channels of the CBAM convolutional attention module is consistent, so, Size and Input Feature Map The sizes are consistent, both .
[0189] Reference Figure 8 After the input data (multi-channel feature map) is input into the Inception module, the output feature map goes through two paths. Path 1 is input into an Inception module again, and Path 2 is input The bilinear interpolation downsampling layer reduces the size to half of the input, and then passes through three Inception modules in sequence, and then inputs The bilinear interpolation upsampling layer doubles the size and restores it to its original size. The output feature maps of the two paths are then stacked in the channel dimension and passed through a CBAM convolutional attention module before being combined with the multi-channel feature map that was initially input into the high-fidelity convolutional feature extraction module. By adding each element together, we get a high-fidelity convolution feature extraction module. The expression of the high-fidelity convolution feature extraction module is:
[0190]
[0191] in, represents the output feature map of the high-fidelity convolutional feature extraction module, represents the mapping function of the high-fidelity convolutional feature extraction module, Represents the mapping function of the Inception module, represents the mapping function of the CBAM convolutional attention module, is a bilinear interpolation downsampling layer that reduces the size to half of the input. It is a bilinear interpolation upsampling layer that increases the size by two times and restores it to its original size. Represents the multi-channel feature map initially input to the high-fidelity convolutional feature extraction module.
[0192] The beneficial effects of setting a high-fidelity convolution feature extraction module in this embodiment include:
[0193] In order to reduce the computational cost, the feature map after feature extraction by one layer of Inception module is downsampled to reduce the size of the feature map to reduce the computational cost and feature redundancy, so that feature extraction can be performed through more layers of Inception layers; after feature extraction, it is upsampled to the original size through bilinear interpolation and channel stacked with the feature map of the original size to retain the initial detail information.
[0194] The channel attention in CBAM can focus on the channels that play a greater role in the multi-channel feature map, while the spatial attention layer can focus on the areas in the same channel that play an important role in the super-resolution of the sea surface height.
[0195] (4) Reference Fig. 9 , construct the output module, where the output module expression is as follows:
[0196]
[0197] in, represents the output feature map of the output module, represents the mapping function of the output module, A multi-channel feature map representing the input to the output module.
[0198] After the multi-channel feature map is input into 4 5×5 convolution kernels (the stride is 1×1, the edge is padded with 2 layers, and the padding value is set to be consistent with the adjacent value), the input feature map is reduced to 4 channels.
[0199] The 4 channels are passed through 4 batch normalization layers, and the 4 channel feature maps are batch normalized to obtain normalized feature maps.
[0200] The normalized feature map is sequentially passed through a ReLU activation layer and a 3×3 convolution kernel (with a step size of 1×1, 2 layers of edge padding, and the padding value is set to be consistent with the adjacent value), and the feature map is reduced to 1 channel to obtain the output module. The expression of the output module is as follows:
[0201]
[0202] in, represents the output feature map of the output module, represents the mapping function of the output module, Multi-channel feature map representing the input-output module.
[0203] The output feature map of the first element convolution stacking module is combined with the low-resolution features and high-resolution features, the high-fidelity convolution feature extraction module and the output module to obtain a deep learning model.
[0204] Reference Fig.10 , the data processing process of the deep learning model includes the following steps S921 to S929:
[0205] Step S921: Input the coarse-resolution sea surface height data, sea surface meridional wind speed data, and sea surface zonal wind speed data into the first element convolution stacking module to obtain three output feature maps extracted by the first element convolution stacking module. Input the high-resolution sea surface temperature data into the second element convolution stacking module to obtain the low-resolution features output by the second element convolution stacking module.
[0206] Step S922: stack the three output feature maps and low-resolution features of step S921 in the channel dimension to obtain a multi-channel feature map .
[0207] Step S923: The After a high-fidelity convolutional feature extraction module (HP-CNN-1), the output feature map of the high-fidelity convolutional feature extraction module is obtained. .
[0208] Step S924: The feature map obtained in step S923 ,enter The bilinear interpolation downsampling layer reduces the size to half of the input, and then passes through a high-fidelity convolutional feature extraction module (HP-CNN-2) to obtain the output feature map of the high-fidelity convolutional feature extraction module. .
[0209] Step S925: The feature map obtained in step S924 ,enter The bilinear interpolation downsampling layer reduces the size to half of the input, and then passes through a high-fidelity convolutional feature extraction module (HP-CNN-3) to obtain the output feature map of the high-fidelity convolutional feature extraction module. .
[0210] Step S926: The feature map obtained in step S925 ,enter The bilinear interpolation downsampling layer reduces the size to half of the input, and then passes through a high-fidelity convolutional feature extraction module (HP-CNN-4) to obtain the output feature map of the high-fidelity convolutional feature extraction module. .
[0211] Step S927: The feature map obtained in step S926 ,enter The bilinear interpolation upsampling layer increases the size to twice the input, and the feature map obtained in step S925 The output feature map of the high-fidelity convolutional feature extraction module is obtained by stacking in the channel dimension and then passing through a high-fidelity convolutional feature extraction module (HP-CNN-5). .
[0212] Step S928: The feature map obtained in step S927 ,enter The bilinear interpolation upsampling layer increases the size to twice the input, and the feature map obtained in step S924 The output feature map of the high-fidelity convolutional feature extraction module is obtained by stacking in the channel dimension and then passing through a high-fidelity convolutional feature extraction module (HP-CNN-6). .
[0213] Step S929: The feature map obtained in step S928 ,enter The bilinear interpolation upsampling layer increases the size to twice the input, and the feature map obtained in step S923 The output feature map of the high-fidelity convolutional feature extraction module is obtained by stacking in the channel dimension and then passing through a high-fidelity convolutional feature extraction module (HP-CNN-7). .
[0214] Finally, the feature map obtained in step S929 The length and width of the input feature map are upsampled to approximately the original , represents the grid resolution of the target high resolution, Represents the grid resolution of the coarse-resolution input feature, and finally passes through an output module, the input BD5 The bilinear interpolation upsampling layer is used to obtain a deep learning model. The above involves a total of 7 high-fidelity convolutional feature extraction modules, which are connected in a U shape. The advantages of this are:
[0215] The channels at the same level in the above U-shape are stacked. These connections allow direct transmission of information between the encoding layer (i.e. the left side of the U) and the decoding layer (the right side of the U). This helps solve the gradient vanishing problem and helps the model better restore image details.
[0216] Through skip connections, the U-shaped structure can pass the features extracted in the encoder stage directly to the decoder stage. This design allows the decoder to utilize richer feature representations, helps alleviate the problem of information loss, and improves the performance of the model.
[0217] From the above steps, we can see that, except for the upsampling layer and the downsampling layer in the deep learning model, all convolution operations in each submodule adopt edge padding according to the size of the convolution kernel, ensuring that the length and width of the grids of the input and output of each submodule are consistent, and the output batch size is also fixed to 16. At the same time, all upsampling layers and downsampling layers will not change the number of channels of the feature map. According to the module connection order, the specific settings of the number of input and output feature map channels of each submodule are given:
[0218] (1) Number of channels of CNNstack input feature map , the number of channels of the output feature map ; (2) Number of channels of CNNstack-4 input feature map , the number of channels of the output feature map ; (3) Number of channels of HP-CNN-1 input feature map , the number of channels of the output feature map ; (4) Number of channels of HP-CNN-2 input feature map , the number of channels of the output feature map ; (5) Number of channels of HP-CNN-3 input feature map , the number of channels of the output feature map ; (6) Number of channels of HP-CNN-4 input feature map , the number of channels of the output feature map ; (7) Number of channels of HP-CNN-5 input feature map , the number of channels of the output feature map ; (8) Number of channels of HP-CNN-6 input feature map , the number of channels of the output feature map ; (9) Number of channels of HP-CNN-7 input feature map , the number of channels of the output feature map ; (10) The number of channels of the Output input feature map and , number of output channels ;
[0219] The mapping relationship of the deep learning model of this embodiment is summarized as follows:
[0220]
[0221] in, Represents the mapping relationship between the input and output of the deep learning model. The input is a batch of multi-factor data that has been normalized. , the samples in the output batch correspond to the super-resolution sea surface height data batch , Indicates the model The weights and biases of the layers.
[0222] Step S930, using the data set obtained in step S910 to train the deep learning model obtained in step S920 to obtain a trained model, specifically includes the following steps S931 to S932:
[0223] Step S931: Select L1 loss as the loss function of the deep learning model obtained in step S920. The expression of L1 loss function is as follows:
[0224]
[0225] in, is a batch of super-resolution sea surface height data, is the normalized high-resolution sea surface height data used as labels;
[0226] Step S932: Based on step S931, the obtained training set The data is input into the deep learning model in batches for training and fitting The parameters of each layer in , and obtain a trained deep learning model.
[0227] Furthermore, step S932 specifically includes the following steps S9321 to S9322:
[0228] Step S9321: Set the basic hyperparameters for training the deep learning model under the Pytorch framework.
[0229] Specifically: The number of initialization iterations is , is a positive integer, and the initial learning rate is , the initial batch size is , the Glorot normal distribution initializer is used to randomly initialize the weights of each layer of the network, and the iterative optimizer is Adam. Then the step training set The data is input into the deep learning model in batches for training and fitting The parameters of each layer in .
[0230] Step S9322: Save the model parameters after training to obtain a trained deep learning model.
[0231] Step S940, using the trained deep learning model to perform super-resolution reconstruction of sea surface height, that is, obtaining high-resolution sea surface height data, specifically includes the following steps:
[0232] Step S941: The test set obtained in step S910 The data in the table are normalized using the MinMax normalization method to interval.
[0233] Step S942: Input the normalized test set data obtained in step S910 into the trained deep learning model in batches, and output the super-resolution height data of each batch.
[0234] When implementing it, you can also choose RMSE root mean square error as the evaluation indicator of the model quality. The expression of RMSE is as follows:
[0235]
[0236] in, and The dimensions are , so the length dimension grid number is , the number of grids in the width dimension is ; Calculate the RMSE root mean square error between the output results of each batch and the true results, average the RMSE of all batches, and obtain the final RMSE to evaluate the model effect.
[0237] An embodiment of the present application provides a super-resolution reconstruction device for sea level height, and the super-resolution reconstruction device for sea level height includes a data acquisition unit 1100 and a data processing unit 1200:
[0238] The data acquisition unit 1100 is used to obtain the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured.
[0239] The data processing unit 1200 is used to input the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface latitudinal wind speed and high-resolution sea surface temperature of the area to be measured into a preset deep learning model, so that the deep learning model extracts the different scale features between the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface latitudinal wind speed and high-resolution sea surface temperature of the area to be measured, and reconstructs the high-resolution sea surface height according to the different scale features.
[0240] It should be noted that this embodiment and the above method embodiment are based on the same inventive concept, so the relevant contents of the above method embodiment are also applicable to this embodiment and will not be described in detail here.
[0241] like Fig.11 The present application also provides an electronic device, which includes:
[0242] At least one memory; at least one processor; at least one program; the program is stored in the memory, and the processor executes at least one program to implement the super-resolution reconstruction method of sea surface height described above in the present disclosure.
[0243] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.
[0244] The electronic device according to the embodiment of the present application is described in detail below.
[0245] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;
[0246] The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the program code is stored in the memory 1700, and the processor 1600 calls and executes the super-resolution reconstruction method of the sea surface height in the embodiment of the present application.
[0247] Input / output interface 1800, used to implement information input and output;
[0248] Communication interface 1900, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);
[0249] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );
[0250] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .
[0251] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned super-resolution reconstruction method of sea surface height.
[0252] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0253] The embodiments described in this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.
[0254] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0255] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0256] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0257] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0258] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0259] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0260] The above is a specific description of the preferred implementation of the embodiments of the present application, but the embodiments of the present application are not limited to the above-mentioned implementation methods. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the embodiments of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the embodiments of the present application.
Claims
1. A super-resolution reconstruction method for sea surface height, characterized in that: The super-resolution reconstruction method of sea surface height comprises: Obtain the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured; The coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured are input into a preset deep learning model, so that the deep learning model extracts different scale features between the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured, and reconstructs the high-resolution sea surface height according to the different scale features; the deep learning model includes a first element convolution stacking module, a second element convolution stacking module and a feature processing module, wherein the first element convolution stacking module includes three stacking submodules, the second element convolution stacking module includes two stacking submodules, and the stacking submodule includes a plurality of parallel subnetworks, each subnetwork has a convolution layer and an activation layer, and the convolution layer in each subnetwork has a different convolution kernel from the convolution layers in other subnetworks; The coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured are input into a preset deep learning model, so that the deep learning model extracts different scale features between the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured, and reconstructs the high-resolution sea surface height according to the different scale features, including: The coarse-resolution sea surface height data, sea surface meridional wind speed data and sea surface zonal wind speed data of the area to be measured are respectively input into a corresponding stacking submodule in the first element convolution stacking module, so as to extract the stacking features of the corresponding input data through the corresponding stacking submodule, and stack the stacking features of the three stacking submodules to obtain a stacking feature map output by the first element convolution stacking module; the stacking feature refers to the feature obtained by extracting and stacking a plurality of different size features of the corresponding input data by the stacking submodule; Inputting the high-resolution sea surface temperature data of the area to be measured into the second element convolution stacking module, extracting the stacking features of the high-resolution sea surface temperature data through the first stacking submodule of the second element convolution stacking module, performing bilinear interpolation down-sampling on the stacking features of the high-resolution sea surface temperature data, and obtaining a high-resolution feature map of the high-resolution sea surface temperature data, and extracting the stacking features of the high-resolution feature map through the second stacking submodule of the second element convolution stacking module, and performing bilinear interpolation down-sampling on the stacking features of the high-resolution feature map to obtain a low-resolution feature map; The stacked feature map and the low-resolution feature map are stacked in the channel dimension to obtain a multi-channel feature map, and the multi-channel feature map is input into the feature processing module so that the feature processing module reconstructs high-resolution sea surface height data according to the multi-channel feature map.
2. The super-resolution reconstruction method of sea level according to claim 1, characterized in that: The stacking submodule includes three parallel subnetworks; The expression of the first element convolution stacking module extracting the stacked feature map is: ; in, They represent the stacking characteristics of the coarse-resolution sea surface height data, sea surface meridional wind speed data, and sea surface zonal wind speed data extracted by the stacking submodule, respectively. represents the mapping function of the stacking operation, Represents the stacked feature map extracted by the first element convolution stack module; The stacking submodule extracts the corresponding stacking feature expression as follows: ; in, Indicates one of the following element data: sea surface height data, sea surface meridional wind speed data, and sea surface zonal wind speed data with coarse resolution. Respectively The convolution kernel mapping function, represents the ReLU activation function, represents the mapping function of the stacked submodule, Indicates the stacking submodule extracted The stacking characteristics of The expression for extracting high-resolution feature maps and low-resolution feature maps by the second element convolution stacking module is: ; ; in, represents high-resolution sea surface temperature data, Both represent the mapping function of bilinear interpolation downsampling, They respectively represent the high-resolution feature map and low-resolution feature map extracted by the second element convolution stacking module.
3. The super-resolution reconstruction method of sea level according to claim 1, characterized in that: The feature processing module includes a high-fidelity convolution feature extraction module and an output module; The step of inputting the multi-channel feature map into the feature processing module so that the feature processing module reconstructs high-resolution sea level height data according to the multi-channel feature map comprises: Extracting corresponding features from the multi-channel feature map according to the high-fidelity convolutional feature extraction module; Inputting the features extracted from the multi-channel feature map by the high-fidelity convolution feature extraction module into the output module, so as to reconstruct high-resolution sea surface height data through the output module; Among them, the expression of extracting features by the high-fidelity convolution feature extraction module is: ; in, represents the input features of the high-fidelity convolutional feature extraction module, represents the mapping function of the high-fidelity convolutional feature extraction module, Represents the mapping function of the Inception module, represents the mapping function of the stacking operation, Represents the mapping function of CBAM convolutional attention, Represents the addition of corresponding elements of two feature map tensors of the same size, represents the output features of the high-fidelity convolutional feature extraction module, represents the mapping function of bilinear interpolation downsampling, Represents the mapping function of bilinear interpolation upsampling; Among them, the expression of the Inception module is: ; in, Represents the output features of the Inception module, Represents the mapping function of the Inception module, Represents the input features of the Inception module, represents the mapping function of the maximum pooling layer, Respectively The convolution kernel mapping function.
4. The super-resolution reconstruction method of sea level according to claim 3, characterized in that: The number of the high-fidelity convolutional feature extraction modules is seven; The extracting corresponding features from the multi-channel feature map according to the high-fidelity convolution feature extraction module includes: The multi-channel feature map is input into the first high-fidelity convolution feature extraction module to obtain the output feature of the first high-fidelity convolution feature extraction module. ; The characteristics After bilinear interpolation downsampling, the image is input to the second high-fidelity convolution feature extraction module to obtain the output features of the second high-fidelity convolution feature extraction module. ; The characteristics After bilinear interpolation downsampling, the data is input into the third high-fidelity convolution feature extraction module to obtain the output features of the third high-fidelity convolution feature extraction module. ; The characteristics After bilinear interpolation downsampling, the image is input to the fourth high-fidelity convolution feature extraction module to obtain the output features of the fourth high-fidelity convolution feature extraction module. ; The characteristics After bilinear interpolation upsampling and feature The stacked features are input into the fifth high-fidelity convolutional feature extraction module to obtain the output features of the fifth high-fidelity convolutional feature extraction module. ; The characteristics After bilinear interpolation upsampling and feature The stacked features are input into the sixth high-fidelity convolutional feature extraction module to obtain the output features of the sixth high-fidelity convolutional feature extraction module. ; The characteristics After bilinear interpolation upsampling and feature The stacked features are input into the seventh high-fidelity convolutional feature extraction module to obtain the output features of the seventh high-fidelity convolutional feature extraction module. , the feature As the corresponding features extracted by the high-fidelity convolutional feature extraction module from the multi-channel feature map.
5. The super-resolution reconstruction method of sea level according to claim 4, characterized in that: The expression for reconstructing high-resolution sea surface height data by the output module is: ; in, Represents the high-resolution sea surface height data reconstructed by the output module, represents the input features of the output module, represents the mapping function of the output module, represents the mapping function of the BN layer, Represents the ReLU activation function.
6. The super-resolution reconstruction method of sea level according to claim 1, characterized in that: The loss function of the deep learning model is: ; ; in, Represents the mapping function of the L1 loss function, Represents the mapping relationship between the input data and output data of the deep learning model, Represents coarse-resolution sea surface height data, sea surface meridional wind speed data, sea surface zonal wind speed data and high-resolution sea surface temperature data, Indicates the first The weights and biases of the layers, represents the high-resolution sea surface height data reconstructed by the deep learning model, Represents high-resolution sea surface height data as a label.
7. A super-resolution reconstruction device for sea level, characterized in that: The super-resolution reconstruction device of the sea level comprises: A data acquisition unit, used to acquire the sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured; A data processing unit, used for inputting the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured into a preset deep learning model, so that the deep learning model extracts different scale features between the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured, and reconstructs the high-resolution sea surface height according to the different scale features; the deep learning model includes a first element convolution stacking module, a second element convolution stacking module and a feature processing module, wherein the first element convolution stacking module includes three stacking submodules, the second element convolution stacking module includes two stacking submodules, and the stacking submodule includes a plurality of parallel subnetworks, each subnetwork has a convolution layer and an activation layer, and the convolution layer in each subnetwork has a different convolution kernel from the convolution layers in other subnetworks; The coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured are input into a preset deep learning model, so that the deep learning model extracts different scale features between the coarse-resolution sea surface height, sea surface meridional wind speed, sea surface zonal wind speed and high-resolution sea surface temperature of the area to be measured, and reconstructs the high-resolution sea surface height according to the different scale features, including: The coarse-resolution sea surface height data, sea surface meridional wind speed data and sea surface zonal wind speed data of the area to be measured are respectively input into a corresponding stacking submodule in the first element convolution stacking module, so as to extract the stacking features of the corresponding input data through the corresponding stacking submodule, and stack the stacking features of the three stacking submodules to obtain a stacking feature map output by the first element convolution stacking module; the stacking feature refers to the feature obtained by extracting and stacking a plurality of different size features of the corresponding input data by the stacking submodule; Inputting the high-resolution sea surface temperature data of the area to be measured into the second element convolution stacking module, extracting the stacking features of the high-resolution sea surface temperature data through the first stacking submodule of the second element convolution stacking module, performing bilinear interpolation down-sampling on the stacking features of the high-resolution sea surface temperature data, and obtaining a high-resolution feature map of the high-resolution sea surface temperature data, and extracting the stacking features of the high-resolution feature map through the second stacking submodule of the second element convolution stacking module, and performing bilinear interpolation down-sampling on the stacking features of the high-resolution feature map to obtain a low-resolution feature map; The stacked feature map and the low-resolution feature map are stacked in the channel dimension to obtain a multi-channel feature map, and the multi-channel feature map is input into the feature processing module so that the feature processing module reconstructs high-resolution sea surface height data according to the multi-channel feature map.
8. An electronic device, characterized in that: include: at least one control processor and a memory for communicatively coupling with the at least one control processor; The memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can perform the super-resolution reconstruction method of sea surface height as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the super-resolution variability reconstruction method of sea surface height according to any one of claims 1 to 6.
Citation Information
Patent Citations
Sea surface height abnormal data fusion method based on satellite altimeter and tide station
CN117009913A
Marine salinity super-resolution reconstruction method based on deep learning SROSRN model
CN117934274A
Sea surface height super-resolution reconstruction method based on diffusion model
CN117974444A
Super resolution imaging method using collaborative learning
KR102406287B1
Model training method and apparatus, super-resolution reconstruction method, device and medium
WO2023184913A1
