A method, device and electronic device for spatial downscaling of ocean surface temperature
The method uses a quality enhancement network and Unet network cascade to efficiently convert low-resolution SST images to high-resolution images, addressing the computational inefficiencies of existing models by reducing GPU memory usage and processing time while maintaining accuracy.
Patent Information
- Application Number
- CN202411253082.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-09
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-09-09
AI Technical Summary
Existing generative diffusion models for spatial downscaling of ocean surface temperature consume excessive GPU memory and computing resources, and take a long time to convert low-resolution SST images into high-resolution images.
A method involving a quality enhancement network, a target model, and a Unet network is used to preprocess and cascade these networks for spatial downscaling, where the quality enhancement network enhances image quality, and the Unet network optimizes super-resolution images, reducing computational complexity and improving processing speed.
The method efficiently converts low-resolution SST images to high-resolution images, reducing GPU memory usage and processing time while maintaining accurate and realistic results, with improved training efficiency and robustness.
Smart Images

Figure CN118761910B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition technology, and in particular to a method, device and electronic equipment for spatial downscaling of ocean surface temperature. Background Art
[0002] Spatial downscaling of sea surface temperature (SST) is the process of converting low-resolution SST images into high-resolution SST images. The purpose is to more accurately describe and reproduce the temperature distribution characteristics of local sea areas, so as to better understand and predict climate change, analyze the health of marine ecosystems, and predict changes in fishery resources.
[0003] The generative diffusion model is a generative neural network model that is widely used in existing computer vision images, such as in the process of spatial downscaling of ocean surface temperature. However, this model will occupy more GPU memory, consume more computing resources, and take a long time when used. Summary of the invention
[0004] In view of this, an embodiment of the present invention provides a method, device and electronic device for spatial downscaling of ocean surface temperature, so as to alleviate the problem that the existing network model takes a long time to convert low-resolution SST images into high-resolution SST images.
[0005] According to a first aspect, an embodiment of the present invention provides a method for spatial downscaling of sea surface temperature, comprising:
[0006] Acquire a first SST data set, wherein the first SST data set includes a plurality of SST images;
[0007] Sending the SST images in the first SST dataset into a quality enhancement network for resolution conversion, and outputting a second SST dataset, wherein the image resolution of the SST images in the second SST dataset is higher than the image resolution of the SST images in the first SST dataset;
[0008] Sending the second SST data set to a target model to generate super-resolution images, and outputting a third SST data set, wherein the resolution of the SST images in the third SST data set is higher than the resolution of the SST images in the second SST data set;
[0009] The third SST dataset is sent to the Unet network for super-resolution image optimization, and the optimized super SST dataset is output.
[0010] The ocean surface temperature spatial downscaling method provided in the present application alleviates the problem of long time consumption caused by directly performing ocean surface temperature spatial downscaling through a target model (such as a diffusion model) in the prior art by using a quality enhancement network for preprocessing and cascading the quality enhancement network, a target model, and a Unet network.
[0011] Optionally, before acquiring the first SST data set, the method includes:
[0012] Acquire a first preprocessed data set and a second preprocessed data set, wherein the first preprocessed data set and the second preprocessed data set both include a plurality of image data, and a resolution of the first preprocessed data set is lower than a resolution of the second preprocessed data set;
[0013] The first preprocessed data set and the second preprocessed data set are subjected to maximum and minimum normalization processing, so that the data in the first preprocessed data set and the second preprocessed data set tend to the target interval, and the first SST data set is output.
[0014] Optionally, the step of sending the SST images in the first SST dataset to a quality enhancement network for resolution conversion and outputting a second SST dataset comprises:
[0015] Determine initial data of the SST images in the first SST data set, wherein the initial data includes the number of SST images, the number of channels of the SST images, the longitudinal resolution of the SST images, and the lateral resolution of the SST images;
[0016] The initial data is sent to the quality enhancement network for data conversion, and the second SST data set is output. The second SST data set includes the number of SST images after data conversion, the number of channels of the SST images after data conversion, the longitudinal resolution of the SST images after data conversion, and the lateral resolution of the SST images after data conversion.
[0017] Optionally, the quality enhancement network includes a Conv layer, a Mish activation function, a ChannelAttnBlock and an upsampling layer Upsample which are cascaded in sequence, wherein the number of ChannelAttnBlocks is at least 2.
[0018] Optionally, the step of sending the initial data into the quality enhancement network for data conversion and outputting the second SST data set further includes:
[0019] The mean absolute error between the second SST data set and the first SST data set is iteratively calculated until the mean absolute error converges.
[0020] Optionally, before the second SST data set is fed into a target model for super-resolution image generation and the third SST data set is output, the target model is trained, and the training steps of the target model include:
[0021] Acquire a first SST training data set and a second SST training data set, wherein the first SST training data set and the second SST training data set include a plurality of image data, the data in the first SST training data set and the second SST training data set correspond to each other, and the resolution of the image data in the first SST training data set is lower than the resolution of the image data in the second SST training data set;
[0022] Sampling data in the first SST training data set and the second SST training data set to determine a target sampling value;
[0023] Performing noise sampling on the data in the first SST training data set and the second SST training data set using a normal distribution to determine sampling noise;
[0024] Calculating a mean square error based on the data in the second SST data set, the target sampling value, and the sampling noise;
[0025] The mean square error is iterated until the mean square error converges to obtain the target model.
[0026] Optionally, the third SST dataset is sent to a Unet network for super-resolution image optimization, and an optimized super-resolution SST dataset is output, including:
[0027] The mean square error in the third SST data set is iteratively calculated until the mean square error converges.
[0028] According to a second aspect, an embodiment of the present invention provides a device for spatial downscaling of sea surface temperature, the device comprising:
[0029] An acquisition module, configured to acquire a first SST data set, wherein the first SST data set includes a plurality of SST images;
[0030] A conversion module, used for sending the SST images in the first SST data set into a quality enhancement network for resolution conversion, and outputting a second SST data set, wherein the image resolution of the SST images in the second SST data set is higher than the image resolution of the SST images in the first SST data set;
[0031] A super-resolution module, used for sending the second SST data set into a target model to generate super-resolution images, and outputting a third SST data set, wherein the resolution of the SST images in the third SST data set is higher than the resolution of the SST images in the second SST data set;
[0032] The optimization module is used to send the third SST data set into the Unet network for super-resolution image optimization and output the optimized super SST data set.
[0033] The ocean surface temperature spatial downscaling device provided in the present application uses an acquisition module to acquire image data of the ocean surface temperature to be optimized, i.e., a first SST data set, and sends the first SST data set to a conversion module, and uses a quality enhancement network in the conversion module to enhance the image quality of the image data in the first SST data set, and outputs a second SST data set. Thereafter, the output second SST data set is sent to a super-resolution module for image resolution conversion to obtain a third SST data set, i.e., a super-resolution image, and finally the third SST data set is sent to an optimization module for image optimization, and a final optimized image is output; by using the quality enhancement network in the conversion module to enhance the image quality of the input image, the data processing pressure of subsequent modules is reduced, the GPU video memory is released, and the data processing speed is improved.
[0034] Optionally, the device further comprises:
[0035] A first preprocessing module is used to obtain a first preprocessing data set and a second preprocessing data set, wherein the first preprocessing data set and the second preprocessing data set both include a plurality of image data, and the resolution of the first preprocessing data set is lower than the resolution of the second preprocessing data set;
[0036] The second preprocessing module is used to perform maximum and minimum normalization processing on the first preprocessing data set and the second preprocessing data set, so that the data in the first preprocessing data set and the second preprocessing data set tend to the target interval, and output the first SST data set.
[0037] According to a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the ocean surface temperature spatial downscaling method described in the first aspect or any one of the embodiments of the first aspect by executing the computer instructions. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The features and advantages of the present invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present invention in any way. In the accompanying drawings:
[0039] Figure 1 A schematic diagram of the flow chart of the ocean surface temperature spatial downscaling method provided in an embodiment of the present application.
[0040] Figure 2A schematic flow chart of steps S21-S22 in the ocean surface temperature spatial downscaling method provided in an embodiment of the present application.
[0041] Figure 3 A schematic flow chart of steps S31-S35 in the ocean surface temperature spatial downscaling method provided in an embodiment of the present application.
[0042] Figure 4 This is a schematic diagram of an SST image obtained without using the ocean surface temperature spatial downscaling method provided in an embodiment of the present application.
[0043] Figure 5 A schematic diagram of an SST image in a super-resolution SST dataset output using the ocean surface temperature spatial downscaling method provided in an embodiment of the present application.
[0044] Figure 6 A curve chart of the RMSE evaluation index of the model corresponding to the ocean surface temperature spatial downscaling method provided in the embodiment of the present application and the existing model.
[0045] Figure 7 A curve chart of the MAE evaluation index of the corresponding model of the ocean surface temperature spatial downscaling method provided in the embodiment of the present application and the existing model.
[0046] Figure 8 A curve chart of the PSNR evaluation index of the corresponding model of the ocean surface temperature spatial downscaling method provided in the embodiment of the present application and the existing model.
[0047] Fig. 9 A curve chart of the Bias evaluation index of the model corresponding to the ocean surface temperature spatial downscaling method provided in the embodiment of the present application and the existing model.
[0048] Fig.10 A schematic diagram of the structure of the ocean surface temperature spatial downscaling device provided in an embodiment of the present application.
[0049] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0050] Reference numerals
[0051] 1-acquisition module; 2-conversion module; 3-super-resolution module; 4-optimization module; 21-processor; 20-memory. DETAILED DESCRIPTION
[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0053] The embodiments of the present application provide a method, device and electronic device for spatial downscaling of ocean surface temperature. Figure 1 As shown, it is a flow chart of the spatial downscaling method of the sea surface temperature provided in the embodiment of the present application. The spatial downscaling method of the sea surface temperature provided in the embodiment of the present application is applied to an intelligent terminal or a control system. The intelligent terminal or the control system obtains the image data of the sea surface temperature in advance or the image data of the sea surface temperature uploaded in real time by the image sensor, and downscales the image data to obtain more accurate temperature information of the sea surface temperature, so as to more accurately describe and reproduce the temperature distribution characteristics of the local sea area, so as to facilitate researchers to better understand and predict climate change, analyze the health of the marine ecosystem and predict fishery resources. At the same time, in the spatial downscaling method, device and electronic device of the sea surface temperature provided in the present application, a quality enhancement network, a target model, a Unet network and a cascade of a quality enhancement network, a target model and a Unet network are used to alleviate the problem that when using traditional models, more GPU video memory is occupied, more computing resources are consumed and time-consuming during data processing in the traditional model.
[0054] Continue reading Figure 1 The method for spatial downscaling of sea surface temperature provided in the embodiment of the present application comprises the following steps:
[0055] S1, obtaining a first SST data set, where the first SST data set includes a plurality of SST images.
[0056] In this embodiment, the first SST data set is a collection of temperature images of the ocean surface in the target sea area collected by an image acquisition device and transmitted to an intelligent terminal (such as a PC, a cloud storage center), wherein the SST images in the first SST data set are low-resolution images.
[0057] S2, sending the SST images in the first SST dataset into the quality enhancement network for resolution conversion, and outputting a second SST dataset, wherein the image resolution of the SST images in the second SST dataset is higher than the image resolution of the SST images in the first SST dataset.
[0058] In this embodiment, the quality enhancement network performs resolution conversion on the SST images in the first SST data set so that the low-resolution image format meets the high-resolution image format, thereby reducing subsequent data processing steps, unifying the data format, and alleviating some data processing pressure. In addition, the quality enhancement network can also adjust the SST images through iterative learning to improve the brightness, contrast, color distortion and noise of the SST images, so as to enhance the details of the temperature changes in the image, which is beneficial to the prediction of climate and the analysis of marine ecology.
[0059] S3, sending the second SST data set to the target model for super-resolution image generation, and outputting a third SST data set, wherein the resolution of the SST images in the third SST data set is higher than the resolution of the SST images in the second SST data set.
[0060] S4, sending the third SST dataset into the Unet network for super-resolution image optimization, and outputting the optimized super-resolution SST dataset.
[0061] In this embodiment, the first SST dataset, the second SST dataset, and the third SST dataset all include a plurality of image data and image information for characterizing the sea surface temperature, wherein the SST image in the first SST dataset may be image data pre-stored in a designated storage module / terminal. The target model may be a diffusion model, and a higher resolution SST image is obtained by using the diffusion model to reconstruct the SST image. In this embodiment, the image data in the third SST dataset is optimized using a Unet network, so that an SST image containing more details (i.e., image data in a super-resolution SST dataset) is obtained after obtaining a high-resolution SST image.
[0062] The ocean surface temperature spatial downscaling method provided in the present application alleviates the problem of long time consumption caused by directly performing ocean surface temperature spatial downscaling through a target model (such as a diffusion model) in the prior art by using a quality enhancement network for preprocessing and a cascade of the quality enhancement network, a target model, and a Unet network.
[0063] The ocean surface temperature spatial downscaling method provided in the embodiment of the present application can generate accurate and realistic high-resolution SST data, such as: image data in an optimized super-resolution SST data set. At the same time, in the present application, a more efficient quality enhancement network is first used to restore most of the simple high-resolution SST structural content (such as: texture details of the SST image), and then a generative diffusion model is used to generate and reconstruct the complex details of the high-resolution SST. By distributing most of the computational complexity in the original downscaling task to the more efficient quality enhancement network, the training pressure and training time of the generative diffusion model are reduced, thereby greatly improving the training efficiency and running speed of the diffusion model. In addition, the SST image fusion optimization is completed through the Unet network. First, the generative diffusion model is used to generate multiple super-resolution SST images, such as the images in the third SST data set. Then these super-resolution SST images are input into the post-processing Unet, and these super-resolution SST images are fused while improving the optimization robustness. Finally, the final super-resolution SST image is output, so that the final downscaling result can remain relatively stable as much as possible under the same low-resolution SST input. The method provided in this application can constrain the randomness in the generative diffusion model, so as to ensure that the SST image is basically consistent with the high-resolution SST image after super-resolution after each input of the same low-resolution SST image, and there will be no significant difference.
[0064] The ocean surface temperature spatial downscaling method provided in this application also includes:
[0065] S01, acquiring a first preprocessed data set and a second preprocessed data set, wherein the first preprocessed data set and the second preprocessed data set both include a plurality of image data, and a resolution of the first preprocessed data set is lower than a resolution of the second preprocessed data set.
[0066] S02, performing maximum and minimum normalization processing on the first preprocessed data set and the second preprocessed data set, so that the data in the first preprocessed data set and the second preprocessed data set tend to the target interval, and outputting the first SST data set.
[0067] In this embodiment, the first preprocessed data set and the second preprocessed data set may be a plurality of image data (including SST images) of a specified target sea surface temperature collected in advance by an image sensor, wherein the first preprocessed data set includes low-resolution SST image data and the second preprocessed data set includes high-resolution SST image data.
[0068] Specifically, the first preprocessed dataset may be a low-resolution SST dataset. The second preprocessed dataset can be a high-resolution SST dataset , among which the low-resolution SST dataset With high-resolution SST dataset Both contain n data, and use low-resolution SST datasets With high-resolution SST dataset The maximum and minimum normalization processing can be:
[0069] Need to explain, and The following relationship needs to be satisfied:
[0070]
[0071] Preferably, the maximum and minimum normalization method used in this application requires respectively and The data range in is normalized to (0, 1).
[0072] Among them, the maximum and minimum normalization can be to set the maximum value of the current data set to , the minimum value is , then for the data , the normalized result for:
[0073]
[0074] In this embodiment, using maximum and minimum normalization to process data can improve the accuracy and effectiveness of the data, reduce data overfitting, and save model training time.
[0075] In this embodiment, if Figure 2 As shown, step S2 also includes the following steps:
[0076] S21, determine the initial data of the SST image in the first SST data set, the initial data including the number of SST images, the number of channels of the SST image, the longitudinal resolution of the SST image, and the lateral resolution of the SST image.
[0077] S22, sending the initial data into the quality enhancement network for data conversion, and outputting a second SST data set, the second SST data set includes the number of SST images after data conversion, the number of channels of the SST images after data conversion, the longitudinal resolution of the SST images after data conversion, and the lateral resolution of the SST images after data conversion.
[0078] In this embodiment, resolution conversion is performed through a quality enhancement network to predict and restore high-frequency details lost in a low-resolution image, so as to improve the clarity and quality of the image.
[0079] In this embodiment, the quality enhancement network includes a Conv layer, a Mish activation function, a ChannelAttnBlock and an upsample layer Upsample which are cascaded in sequence, wherein the number of ChannelAttnBlocks is at least 2. After the initial data is sent to the quality enhancement network for data conversion and the second SST data set is output, the mean absolute error between the second SST data set and the first SST data set needs to be iteratively calculated until the mean absolute error converges.
[0080] In this embodiment, the Conv layer, the Mish activation function, the ChannelAttnBlock and the upsample layer Upsample are cascaded so that the neural network can process the image data more finely, so as to improve the efficiency and accuracy of the image processing task.
[0081] Specifically, the quality enhancement network provided in this application is used to super-resolve low-resolution SST to high-resolution SST. The purpose of this process is to restore most of the simple SST texture content that characterizes small and medium-scale processes in the process, thereby reducing the workload of subsequent generation of diffusion models. Optionally, in addition to restoring most of the simple SST texture content that characterizes small and medium-scale processes, the output of the quality enhancement network may also include information such as larger-scale ocean dynamic processes, local approximate sea temperature zones, and simple sea temperature distribution characteristics.
[0082] Specifically, the format of the input SST image data is (B, 1, Lh, Lw), where B, 1, Lh, and Lw are Batchsize, the number of channels, the longitudinal resolution of the low-resolution SST, and the lateral resolution of the low-resolution SST, respectively. It passes through a Conv layer and a Mish activation function, 6 consecutive ChannelAttnBlocks, and finally an upsampling layer Upsample, and finally the output is a super-resolution SST of (B, 1, Hh, Hw), where Hh and Hw are Batchsize, the number of channels, the longitudinal resolution of the super-resolution SST, and the lateral resolution of the super-resolution SST, respectively.
[0083] In this embodiment, multiple ChannelAttnBlocks (channel attention modules) are used, the purpose of which is to enhance the more important channel information in the input features (input SST image data) while suppressing noisy or unimportant channel information to optimize the quality of the output SST image data.
[0084] In this embodiment, the training efficiency is improved by optimizing the quality enhancement network, and the model output parameters are restricted to improve the model data processing speed and alleviate the data processing pressure.
[0085] Optionally, the purpose of using at least 2 ChannelAttnBlocks is to process the target data or specified data features in a targeted manner so that the network can converge faster, thereby improving the processing speed of image data and alleviating data processing pressure.
[0086] Optionally, the training process of the quality enhancement network provided in the present application can be to train each low-resolution SST data And the corresponding high-resolution SST data To process, please follow the steps below:
[0087] (1) Input into EQN (quality enhancement network), and get super-resolved SST after EQN processing .
[0088] (2) Calculation and The mean absolute error between them is used as the loss function. The calculation of the mean absolute error can refer to the existing calculation method, which will not be repeated here.
[0089] (3) Optimize the loss function and repeat the above process until convergence to obtain a trained quality enhancement network.
[0090] The ocean surface temperature spatial downscaling method provided in the embodiment of the present application uses a quality enhancement network to pre-process the first SST data set, which can restore most of the simple SST texture content that characterizes small and medium-scale processes, thereby reducing the workload of subsequent target models in processing data.
[0091] In this embodiment, the target model may be a diffusion model, such as Figure 3 As shown, step S3 also includes:
[0092] S31, obtaining a first SST training data set and a second SST training data set, wherein the first SST training data set and the second SST training data set include a plurality of image data, the data in the first SST training data set and the second SST training data set correspond to each other, and the resolution of the image data in the first SST training data set is lower than the resolution of the image data in the second SST training data set.
[0093] S32, sampling the data in the first SST training data set and the second SST training data set to determine a target sampling value.
[0094] S33, performing noise sampling on the data in the first SST training data set and the second SST training data set using a normal distribution to determine sampling noise;
[0095] S34, calculating a mean square error based on the data in the second SST data set, the target sampling value and the sampling noise.
[0096] S35, iterating the mean square error until the mean square error converges to obtain the target model.
[0097] Among them, the first SST training data set and the second SST training data set pre-collect SST images in the target area within a period of time, and the image data in the first SST training data set and the second SST training data set correspond one to one. There is a difference in resolution between the two, but the image data in the second SST training data set is used as the basis for restoring the image data in the first SST training data set, so as to simplify the difficulty of data processing and alleviate the pressure of data processing.
[0098] Specifically, in this embodiment, the training target model (ie, diffusion model) may be:
[0099] Obtaining low-resolution SST datasets With high-resolution SST dataset , low-resolution SST dataset With high-resolution SST dataset Contains n data, among which, low-resolution SST dataset With high-resolution SST dataset Satisfies the following relationship:
[0100]
[0101] set up =1000, is a pre-set sequence (which can be extracted from pre-stored data), and .make ,in, Used to indicate pre-set weights.
[0102] For each sample And the corresponding , and proceed as follows:
[0103] (1) From Sample a value in .
[0104] (2) From the normal distribution Sampling a noise .
[0105] (3) Calculation , denoted as ,in Denoted as the image condition vector set sample of the high-resolution SST dataset The weight information of Denoted as the introduced random disturbance noise.
[0106] (4) Input the trained EQN network to obtain the super-resolution SST
[0107] (5) Calculation and The mean square error of .
[0108] (6) Optimize the error and repeat the above process until convergence.
[0109] Specifically, the sampling process for generating a diffusion model can be:
[0110] set up =1000, is a pre-set sequence, and .
[0111] make , It is the Unet neural network trained in the above steps.
[0112] From the normal distribution Sampling a noise .
[0113] according to For each value in the sequence , perform the following steps:
[0114] (1) From the normal distribution Sampling a noise .
[0115] (2) Calculation , ,in It is represented as the weight of the random disturbance noise introduced in the tth step.
[0116] (3) After iterative calculation in step S2 above, the final , which includes super-resolution SST.
[0117] In this embodiment, step S4 can reduce the amount of data introduced and alleviate data processing pressure by iteratively calculating the mean square error in the third SST data set until the mean square error converges.
[0118] Specifically, the Unet network can be used for low-resolution SST datasets With high-resolution SST dataset , which contain n data, among which Let number be the number of super-resolution SSTs to be input to Unet.
[0119] For each sample And the corresponding , and proceed as follows:
[0120] (1) Low-resolution SST Input the EQN trained in step S2 above to obtain the super-resolution SST , where low-resolution SST For reference Figure 4 shown.
[0121] Will The generative diffusion model trained in input 3 is sampled to obtain super-resolution SST .
[0122] (2) Repeat process (1) number times to obtain number sheets
[0123] (3) Number Input the post-processing Unet to get the final downscaling result result.
[0124] (4) Calculate result and The mean square error of .
[0125] (5) Optimize the error and repeat the above process until convergence.
[0126] Optional, for low-resolution SST datasets With high-resolution SST dataset , which contain n data, among which .
[0127] Low-resolution SST Input the trained EQN in 2 to get the super-resolution SST .
[0128] Will The generative diffusion model trained in input 3 is sampled to obtain super-resolution SST .
[0129] Repeat number times (1) to get number sheets
[0130] Number Input post-processing Unet to get the final downscaling result result, where the final downscaling result can be referred to Figure 5 .
[0131] In this embodiment, the resolution of low-resolution data (SST images in the first SST data set) is converted through a quality enhancement network, and then a target model (such as a generated diffusion model) is used to output a super-resolution image. Finally, the super-resolution image is optimized and output to complete the conversion of the low-resolution image to the super-resolution image. In the above process, since the quality enhancement network is used to pre-process the input SST image, the data processing pressure in the subsequent use of the target model and image optimization process is alleviated, and the conversion process of the super-resolution image is accelerated. The subsequent use of the target model and optimization of the super-resolution image are to ensure the image quality of the final output super-resolution image. In order to facilitate understanding of the advantages of the ocean surface temperature spatial downscaling method provided in the present application, as Figure 6-Figure 9 As shown, in this embodiment, the existing models (RCAN, ESRGAN, LDM) are introduced to compare the prediction capabilities with the model provided by this application (DSSR). Based on the comparison results (curve graph), it can be seen that the performance of the model provided by this application in super-resolution image prediction is better than that of the existing models after multiple iterative learning. Figure 6-Figure 9 The horizontal axis is used to represent the sample processing time, and the vertical axis is used to represent different evaluation indicators: peak signal-to-noise ratio (PSNR), root mean square error (RMSE), mean absolute error (MAE), and bias.
[0132] Accordingly, please refer to Fig.10 , an ocean surface temperature spatial downscaling device provided by an embodiment of the present invention comprises:
[0133] Acquisition module 1, used for acquiring a first SST data set, the first SST data set includes a plurality of SST images, for details, refer to step S1;
[0134] Conversion module 2, used for sending the SST images in the first SST data set into the quality enhancement network for resolution conversion, and outputting a second SST data set, wherein the image resolution of the SST images in the second SST data set is higher than the image resolution of the SST images in the first SST data set, for details, refer to step S2;
[0135] The super-resolution module 3 is used to send the second SST data set to the target model for super-resolution image generation, and output a third SST data set. The resolution of the SST image in the third SST data set is higher than the resolution of the SST image in the second SST data set. For details, refer to step S3.
[0136] Optimization module 4, used for sending the third SST data set into the Unet network for super-resolution image optimization, and outputting the optimized super-resolution SST data set. For details, refer to step S4;
[0137] Optionally, the device provided by the present application further includes:
[0138] A first preprocessing module is used to obtain a first preprocessing data set and a second preprocessing data set, wherein the first preprocessing data set and the second preprocessing data set both include a plurality of image data, and the resolution of the first preprocessing data set is lower than the resolution of the second preprocessing data set. For details, refer to step S01;
[0139] The second preprocessing module is used to perform maximum and minimum normalization processing on the first preprocessing data set and the second preprocessing data set, so that the data in the first preprocessing data set and the second preprocessing data set tend to the target interval, and output the first SST data set. For details, refer to step S02.
[0140] The ocean surface temperature spatial downscaling device provided in the present application uses an acquisition module to acquire image data of the ocean surface temperature to be optimized, i.e., a first SST data set, and sends the first SST data set to a conversion module. The image data in the first SST data set is enhanced in image quality by a quality enhancement network in the conversion module, and a second SST data set is output. The output second SST data set is then sent to a super-resolution module for image resolution conversion to obtain a third SST data set, i.e., a super-resolution image. Finally, the third SST data set is sent to an optimization module for image optimization, and a final optimized image is output. The image quality of the input image is enhanced by using the quality enhancement network in the conversion module, thereby reducing the data processing pressure of subsequent modules, releasing GPU video memory, and improving data processing speed.
[0141] The embodiment of the present invention further provides an electronic device, such as Fig.11 As shown, the electronic device may include a processor 21 and a memory 20, wherein the processor 21 and the memory 20 may be connected via a bus or in other ways.
[0142] The processor 21 may be a central processing unit (CPU). The processor 21 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.
[0143] The memory 20 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the ocean surface temperature spatial downscaling method in the embodiment of the present invention (for example, Fig.10 The processor 21 executes various functional applications and data processing of the processor 21 by running the non-transient software programs, instructions and modules stored in the memory 20, that is, the ocean surface temperature spatial downscaling method in the above method embodiment is implemented.
[0144] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created by the processor 21, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0145] The one or more modules are stored in the memory 20 and when executed by the processor 21, perform the following steps: Figures 1 to 3 The ocean surface temperature spatial downscaling method in the illustrated embodiment.
[0146] For details of the above electronic equipment, please refer to Figures 1 to 9 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.
[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by hardware related to computer program instructions, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.
[0148] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A method for spatial downscaling of sea surface temperature, characterized in that: include: Acquire a first SST data set, wherein the first SST data set includes a plurality of SST images; Sending the SST images in the first SST data set into a quality enhancement network for resolution conversion, and outputting a second SST data set, including: determining initial data of the SST images in the first SST data set, the initial data including the number of SST images, the number of channels of the SST images, the longitudinal resolution of the SST images, and the lateral resolution of the SST images; sending the initial data into the quality enhancement network for data conversion, and outputting the second SST data set, the second SST data set including the number of SST images after data conversion, the number of channels of the SST images after data conversion, the longitudinal resolution of the SST images after data conversion, and the lateral resolution of the SST images after data conversion; the image resolution of the SST images in the second SST data set is higher than the image resolution of the SST images in the first SST data set; Sending the second SST data set to a target model to generate super-resolution images, and outputting a third SST data set, wherein the resolution of the SST images in the third SST data set is higher than the resolution of the SST images in the second SST data set; Sending the third SST dataset into the Unet network for super-resolution image optimization, and outputting an optimized super-resolution SST dataset; The quality enhancement network includes a Conv layer, a Mish activation function, a ChannelAttnBlock and an upsampling layer Upsample which are cascaded in sequence, wherein there are at least 2 ChannelAttnBlocks, and the target model is a diffusion model.
2. The method for spatial downscaling of sea surface temperature according to claim 1, characterized in that: Before obtaining the first SST data set, the method includes: Acquire a first preprocessed data set and a second preprocessed data set, wherein the first preprocessed data set and the second preprocessed data set both include a plurality of image data, and a resolution of the first preprocessed data set is lower than a resolution of the second preprocessed data set; The first preprocessed data set and the second preprocessed data set are subjected to maximum and minimum normalization processing, so that the data in the first preprocessed data set and the second preprocessed data set tend to the target interval, and the first SST data set is output.
3. The method for spatial downscaling of sea surface temperature according to claim 1, characterized in that: The step of sending the initial data into the quality enhancement network for data conversion and outputting the second SST data set further includes: The mean absolute error between the second SST data set and the first SST data set is iteratively calculated until the mean absolute error converges.
4. The method for spatial downscaling of sea surface temperature according to claim 1, characterized in that: Before sending the second SST data set to the target model for super-resolution image generation and outputting the third SST data set, the target model is trained. The training steps of the target model include: Acquire a first SST training data set and a second SST training data set, wherein the first SST training data set and the second SST training data set include a plurality of image data, the data in the first SST training data set and the second SST training data set correspond to each other, and the resolution of the image data in the first SST training data set is lower than the resolution of the image data in the second SST training data set; Sampling the data in the first SST training data set and the second SST training data set to determine a target sampling value; Performing noise sampling on the data in the first SST training data set and the second SST training data set using a normal distribution to determine sampling noise; Calculating a mean square error based on the data in the second SST data set, the target sampling value, and the sampling noise; The mean square error is iterated until the mean square error converges to obtain the target model.
5. The method for spatial downscaling of sea surface temperature according to claim 1, characterized in that: The third SST dataset is sent to the Unet network for super-resolution image optimization, and the optimized super-resolution SST dataset is output, including: The mean square error in the third SST data set is iteratively calculated until the mean square error converges.
6. A spatial downscaling device for sea surface temperature, characterized in that: The device comprises: An acquisition module, configured to acquire a first SST data set, wherein the first SST data set includes a plurality of SST images; A conversion module, used for sending the SST images in the first SST data set into a quality enhancement network for resolution conversion and outputting a second SST data set, comprising: determining initial data of the SST images in the first SST data set, the initial data comprising the number of SST images, the number of channels of the SST images, the longitudinal resolution of the SST images and the lateral resolution of the SST images; sending the initial data into the quality enhancement network for data conversion, and outputting the second SST data set, the second SST data set comprising the number of SST images after data conversion, the number of channels of the SST images after data conversion, the longitudinal resolution of the SST images after data conversion and the lateral resolution of the SST images after data conversion; the image resolution of the SST images in the second SST data set is higher than the image resolution of the SST images in the first SST data set; A super-resolution module, used for sending the second SST data set into a target model to generate super-resolution images, and outputting a third SST data set, wherein the resolution of the SST images in the third SST data set is higher than the resolution of the SST images in the second SST data set; An optimization module, used for sending the third SST dataset into a Unet network for super-resolution image optimization, and outputting an optimized super-resolution SST dataset; The quality enhancement network includes a Conv layer, a Mish activation function, a ChannelAttnBlock and an upsampling layer Upsample which are cascaded in sequence, wherein there are at least 2 ChannelAttnBlocks, and the target model is a diffusion model.
7. The ocean surface temperature spatial downscaling device according to claim 6, characterized in that: The device also includes: A first preprocessing module is used to obtain a first preprocessing data set and a second preprocessing data set, wherein the first preprocessing data set and the second preprocessing data set both include a plurality of image data, and the resolution of the first preprocessing data set is lower than the resolution of the second preprocessing data set; The second preprocessing module is used to perform maximum and minimum normalization processing on the first preprocessing data set and the second preprocessing data set, so that the data in the first preprocessing data set and the second preprocessing data set tend to the target interval, and output the first SST data set.
8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the ocean surface temperature spatial downscaling method according to any one of claims 1 to 5 by executing the computer instructions.
