Training method of data super-division model and data super-division processing method

Abstract

The invention provides a data super-division model training method and a data super-division processing method, and relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like. The method comprises the following steps: respectively mapping sample low-resolution data and sample high-resolution data in a sample data pair to a corresponding low-resolution potential space and a corresponding high-resolution potential space; determining a first correlation coefficient between the sample data pair mapped by the current time frame and the sample data pair mapped by the previous time frame; determining a second correlation coefficient between the sample data pair mapped by the current time frame and the sample data pair mapped by the next time frame; model training is carried out based on the low-resolution potential space, the high-resolution potential space, the first correlation coefficient and the second correlation coefficient, a trained data super-resolution model is obtained, and the data super-resolution model is used for converting low-resolution data into high-resolution data.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to technical fields such as computer vision, deep learning, and large models, and in particular to a data super-resolution model training method and a data super-resolution processing method. Background Art

[0002] Super-resolution (SR) refers to the process of using optics and related optical knowledge to restore image details and other data information based on known image information. In simple terms, it is to increase the resolution of the image to prevent its image quality from deteriorating. Image super-resolution technology can be applied to the fields of image compression, remote sensing imaging, video perception, etc. For example, climate data can be super-resolution processed. The super-resolution processing of climate data is to convert low-resolution climate data into a high-resolution format, thereby providing more detailed and accurate information for a specific area. Summary of the invention

[0003] The present invention provides a data super-resolution model training method and a data super-resolution processing method.

[0004] According to a first aspect of the present disclosure, a method for training a data super-resolution model is provided, including: mapping sample low-resolution data and sample high-resolution data in a sample data pair to corresponding low-resolution latent space and high-resolution latent space, respectively; determining a first correlation coefficient between the sample data pair mapped in a current time frame and the sample data pair mapped in a previous time frame; determining a second correlation coefficient between the sample data pair mapped in the current time frame and the sample data pair mapped in a subsequent time frame; performing model training based on the low-resolution latent space, the high-resolution latent space, and the first correlation coefficient and the second correlation coefficient to obtain a trained data super-resolution model, wherein the data super-resolution model is used to convert low-resolution data into high-resolution data.

[0005] According to a second aspect of the present disclosure, a data super-resolution processing method is provided, comprising: obtaining low-resolution data to be super-resolution; inputting the low-resolution data into a data super-resolution model, and outputting high-resolution data corresponding to the low-resolution data, wherein the data super-resolution model is trained using the method described in any implementation method of the first aspect.

[0006] According to a third aspect of the present disclosure, a training device for a data super-resolution model is provided, comprising: a mapping module, configured to map sample low-resolution data and sample high-resolution data in a sample data pair to corresponding low-resolution latent space and high-resolution latent space, respectively; a first coefficient determination module, configured to determine a first correlation coefficient between the sample data pair mapped in a current time frame and the sample data pair mapped in a previous time frame; a second coefficient determination module, configured to determine a second correlation coefficient between the sample data pair mapped in the current time frame and the sample data pair mapped in a subsequent time frame; a training module, configured to perform model training based on the low-resolution latent space, the high-resolution latent space, the first correlation coefficient and the second correlation coefficient to obtain a trained data super-resolution model, wherein the data super-resolution model is used to convert low-resolution data into high-resolution data.

[0007] According to a fourth aspect of the present disclosure, a data super-resolution processing device is provided, comprising: an acquisition module, configured to acquire low-resolution data to be super-resolution processed; a super-resolution module, configured to input the low-resolution data into a data super-resolution model, and output high-resolution data corresponding to the low-resolution data, wherein the data super-resolution model is trained using the method described in any implementation method of the first aspect.

[0008] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any implementation manner of the first aspect or the second aspect.

[0009] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method described in any implementation manner of the first aspect or the second aspect.

[0010] According to a seventh aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the computer program implements the method described in any implementation manner of the first aspect or the second aspect.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0013] Figure 1is an exemplary system architecture diagram in which the present disclosure may be applied;

[0014] Figure 2 is a flowchart of an embodiment of a method for training a data super-resolution model according to the present disclosure;

[0015] Figure 3 is a flowchart of another embodiment of a method for training a data super-resolution model according to the present disclosure;

[0016] Figure 4 yes Figure 3 A flowchart of the model training steps in Figure 1;

[0017] Figure 5-1 It is a schematic diagram of the process of mapping data into latent space;

[0018] Figure 5-2 is a schematic diagram of the bidirectional time alignment process;

[0019] Figure 5-3 It is a schematic diagram of the model training process;

[0020] Figure 6 is a flow chart of an embodiment of a data super-resolution processing method according to the present disclosure;

[0021] Figure 7 is a structural schematic diagram of an embodiment of a training device for a data super-resolution model according to the present disclosure;

[0022] Figure 8 is a structural schematic diagram of an embodiment of a data super-resolution processing device according to the present disclosure;

[0023] Fig. 9 It is a block diagram of an electronic device used to implement the data super-resolution model training method or data super-resolution processing method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0025] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] Figure 1An exemplary system architecture 100 is shown to which an embodiment of a data super-resolution model training method, a data super-resolution processing method, a data super-resolution model training device, and a data super-resolution processing device of the present disclosure can be applied.

[0027] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0028] The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send information, etc. Various client applications can be installed on the terminal devices 101, 102, 103.

[0029] The terminal devices 101, 102, 103 may be hardware or software. When the terminal devices 101, 102, 103 are hardware, they may be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. When the terminal devices 101, 102, 103 are software, they may be installed in the above electronic devices. They may be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.

[0030] The server 105 can provide various services. For example, the server 105 can analyze and process the sample data pairs obtained from the terminal devices 101, 102, and 103, and generate processing results (such as a trained data super-resolution model).

[0031] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, for providing distributed services), or it can be implemented as a single software or software module. No specific limitation is made here.

[0032] It should be noted that the data super-resolution model training method and data super-resolution processing method provided in the embodiments of the present disclosure are generally executed by the server 105. Accordingly, the data super-resolution model training device and data super-resolution processing device are generally arranged in the server 105.

[0033] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0034] Continue to refer Figure 2 , which shows a process 200 of an embodiment of a method for training a data super-resolution model according to the present disclosure. The method for training a data super-resolution model comprises the following steps:

[0035] Step 201 : Map sample low-resolution data and sample high-resolution data in a sample data pair to corresponding low-score latent space and high-score latent space respectively.

[0036] In this embodiment, the execution subject of the training method of the data super-resolution model (for example Figure 1 The server 105 shown in the figure will map the sample low-resolution data and the sample high-resolution data in the sample data pair to the corresponding low-score latent space and high-score latent space respectively. Here, the above-mentioned execution subject will first obtain the sample data pair for model training, and the sample data pair includes the sample low-resolution data and the sample high-resolution data corresponding to the sample low-resolution data.

[0037] Then, the sample low-resolution data and the sample high-resolution data are mapped to the corresponding low-score latent space and high-score latent space respectively. The low-score latent space is the latent space where the low-resolution data is located after mapping. The low-score latent space includes the low-score latent representation of the sample low-resolution data at each moment. For example, a low-score encoder can be used to generate the low-score latent representation of the sample low-resolution data at each moment. The high-score latent space is the latent space of the high-resolution data after mapping. The high-score latent space includes the high-score latent representation of the sample high-resolution data at each moment. For example, a high-score encoder can be used to generate the high-score latent representation of the sample high-resolution data at each moment.

[0038] Optionally, the low-score latent representation in the low-score latent space can be associated with the high-score latent representation in the high-score latent space through a shared codebook, thereby ensuring consistent representation of data of different resolutions in the latent space.

[0039] Step 202: determine a first correlation coefficient between a sample data pair mapped in the current time frame and a sample data pair mapped in the previous time frame.

[0040] In this embodiment, the above-mentioned execution entity will determine the first correlation coefficient between the sample data pair mapped in the current time frame and the sample data pair mapped in the previous time frame. The sample data pair mapped in the current time frame is the low-score potential representation corresponding to the sample low-resolution data of the current time frame and the high-score potential representation corresponding to the sample high-resolution data. The sample data pair mapped in the previous time frame is the low-score potential representation corresponding to the sample low-resolution data of the previous time frame of the current time frame and the high-score potential representation corresponding to the sample high-resolution data. For example, the high-score potential representation of the current time frame can be predicted through a forward time alignment network based on the low-score potential representation, high-score potential representation and low-score potential representation of the sample low-resolution data of the current time frame after the sample data pair mapped in the previous time frame, and then the first correlation coefficient is generated based on the numerical relationship between the predicted high-score potential representation and the high-score potential representation generated by the encoder.

[0041] Step 203: determine a second correlation coefficient between the sample data pair after mapping in the current time frame and the sample data pair after mapping in the next time frame.

[0042] In this embodiment, the above-mentioned execution entity will determine the second correlation coefficient between the sample data pair mapped in the current time frame and the sample data pair mapped in the next time frame. The sample data pair mapped in the current time frame is the low-score potential representation corresponding to the sample low-resolution data of the current time frame and the high-score potential representation corresponding to the sample high-resolution data. The sample data pair mapped in the next time frame is the low-score potential representation corresponding to the sample low-resolution data of the next time frame after the current time frame and the high-score potential representation corresponding to the sample high-resolution data. For example, the high-score potential representation of the current time frame can be predicted through a backward time alignment network based on the low-score potential representation, high-score potential representation and low-score potential representation of the sample low-resolution data of the current time frame after mapping the sample data pair in the next time frame, and then the second correlation coefficient is generated based on the numerical relationship between the predicted high-score potential representation and the high-score potential representation generated by the encoder.

[0043] Step 204 , performing model training based on the low-score latent space, the high-score latent space, the first correlation coefficient, and the second correlation coefficient to obtain a trained data super-resolution model.

[0044] In this embodiment, the above-mentioned execution subject will train the initial super-resolution model based on the low-resolution latent space, the high-resolution latent space, the first correlation coefficient, and the second correlation coefficient to obtain a trained data super-resolution model, wherein the data super-resolution model is used to convert low-resolution data into high-resolution data. Here, the initial super-resolution model includes a low-resolution encoder, a super-resolution network, and a high-resolution decoder. Specifically, the super-resolution network can be trained in the latent space first, that is, the low-resolution potential representation in the low-resolution latent space, the high-resolution potential representation in the high-resolution latent space, the first correlation coefficient, and the second correlation coefficient are used to train the super-resolution network to obtain a trained super-resolution network. Then, the low-resolution encoder, the trained super-resolution network, and the high-resolution decoder are trained using sample data to obtain a trained data super-resolution model.

[0045] The training method of the data super-resolution model provided in the embodiment of the present disclosure first maps the sample low-resolution data and the sample high-resolution data in the sample data pair to the corresponding low-resolution latent space and high-resolution latent space respectively; then determines the first correlation coefficient between the sample data pair after the mapping of the current time frame and the sample data pair after the mapping of the previous time frame; then determines the second correlation coefficient between the sample data pair after the mapping of the current time frame and the sample data pair after the mapping of the next time frame; finally, based on the low-resolution latent space, the high-resolution latent space and the first correlation coefficient and the second correlation coefficient, the model is trained to obtain the trained data super-resolution model. The training method of the data super-resolution model in this embodiment, when processing the sample data pair, respectively determines the correlation between the data of the current time frame and the data of the previous time frame and the data of the next time frame, thereby utilizing the correlation in the time series, improving the detail accuracy of the reconstructed high-resolution data, and further improving the accuracy and precision of the data super-resolution model.

[0046] In some optional implementations of this embodiment, the sample data pair includes: sample low-resolution climate data and sample high-resolution climate data. That is, the sample data pair is a climate data pair, and the climate data pair is used as sample data for model training, so that the trained data super-resolution model can generate more accurate high-resolution climate data when super-resolution processing is performed on climate data.

[0047] In addition, in the technical solutions involved in this disclosure, the acquisition, storage, use, processing, transportation, provision and disclosure of user personal information involved are in compliance with the relevant laws and regulations and do not violate public order and good morals.

[0048] Continue to refer Figure 3 , Figure 3 A process 300 of another embodiment of a method for training a data super-resolution model according to the present disclosure is shown. The method for training a data super-resolution model comprises the following steps:

[0049] Step 301, using a low-score encoder to generate a low-score potential representation of the sample low-resolution data at each moment.

[0050] In this embodiment, since the low-resolution latent space obtained after mapping includes the low-resolution latent representation of the sample low-resolution data at each time, the execution body of the training method of the data super-resolution model (e.g. Figure 1 The server 105 shown in the figure will use a low-score encoder to generate a low-score potential representation of the sample low-resolution data at each moment, thereby achieving mapping of the sample low-resolution data.

[0051] Specifically, a vector quantized variational autoencoder (VQ-VAE) model can be used to map the sample low-resolution (LR) data to a low-resolution latent space. LR The potential representations of (t) at time t are:

[0052] Z LR (t) = q(φ LR (I LR (t)));

[0053] Among them, φ LR is the encoder function for the sample low-resolution data, and q() represents the vector quantization operation.

[0054] Step 302, using a high-score encoder to generate a high-score potential representation of the sample high-resolution data at each time.

[0055] In this embodiment, since the high-score latent space obtained after mapping includes the high-score latent representation of the sample high-resolution data at each moment, the above-mentioned execution entity will use the high-score encoder to generate the high-score latent representation of the sample high-resolution data at each moment, thereby realizing the mapping of the sample high-resolution data.

[0056] Specifically, a vector quantized variational autoencoder (VQ-VAE) model can be used to map sample high-resolution (HR) data to a high-resolution latent space. HR The potential representations of (t) at time t are:

[0057] Z HR (t) = q(φ HR (I HR (t)));

[0058] Among them, φ HR is the encoder function for the sample low-resolution data, and q() represents the vector quantization operation.

[0059] By mapping sample data pairs to corresponding latent spaces respectively, the influence of spatial noise of the data is reduced.

[0060] In some optional implementations of this embodiment, the training method of the data super-resolution model further includes: associating the low-score potential representation at each moment in the low-score latent space with the high-score potential representation at each moment in the high-score latent space through a codebook. The q() in the above formula is used to discretize the continuous potential output into a shared codebook. Thus, the shared codebook ensures the consistent representation of data of different resolutions in the latent space.

[0061] Step 303: determine a target high-score latent representation associated with the low-score latent representation of the current time frame through the codebook.

[0062] In this embodiment, since the low-score potential representations at each moment in the low-score potential space have been associated with the high-score potential representations at each moment in the high-score potential space through the code book, the high-score potential representation associated with the low-resolution data of the sample in the current time frame can be determined through the code book, that is, the target high-score potential representation Z HR (t).

[0063] Step 304 , determining a first predicted high-score potential representation of the current time frame based on the low-score potential representation and high-score potential representation of the sample data pair mapped in the previous time frame and the low-score potential representation of the sample low-resolution data of the current time frame.

[0064] In this embodiment, the execution subject predicts the high-score potential representation of the current time frame through the forward time alignment network based on the low-score potential representation and high-score potential representation of the sample data pair after mapping in the previous time frame and the low-score potential representation of the sample low-resolution data of the current time frame, and records it as the first predicted high-score potential representation, which can be specifically expressed as follows:

[0065]

[0066] Among them, ψ F is the forward alignment network, Z LR (t-1) is the low-scoring potential representation of the previous time frame, Z HR (t-1) is the high-resolution latent representation of the previous time frame, Z LR (t) is the low-scoring potential representation of the current time frame, is the first predicted high-scoring potential representation obtained for the prediction.

[0067] Step 305: Generate a first correlation coefficient based on the numerical relationship between the first predicted high-scoring latent representation and the target high-scoring latent representation.

[0068] In this embodiment, the execution subject will obtain the high-scoring potential representation based on the prediction. and the high-scoring latent representation Z generated by the encoder HR (t) to generate the first correlation coefficient λ1.

[0069] Thus, the association relationship between the sample data after mapping in the current time frame and the sample data after mapping in the previous time frame is determined through the above steps.

[0070] Step 306, determine a second predicted high-score potential representation of the current time frame based on the low-score potential representation, the high-score potential representation of the sample data pair after mapping in the next time frame and the low-score potential representation of the sample low-resolution data of the current time frame.

[0071] In this embodiment, the execution subject predicts the high-score potential representation of the current time frame through the backward time alignment network according to the low-score potential representation, high-score potential representation of the sample data pair after mapping in the next time frame and the low-score potential representation of the sample low-resolution data of the current time frame, and records it as the second predicted high-score potential representation, which can be specifically expressed as follows:

[0072]

[0073] Among them, ψ B is the backward alignment network, Z LR (t+1) is the low-score potential representation of the next time frame, Z HR (t+1) is the high-resolution potential representation of the next time frame, Z LR (t) is the low-scoring potential representation of the current time frame, The second predicted high-scoring potential representation obtained for the prediction.

[0074] Step 307: Generate a second correlation coefficient based on the numerical relationship between the second predicted high-scoring potential representation and the target high-scoring potential representation.

[0075] In this embodiment, the execution subject will obtain the high-scoring potential representation based on the prediction. and the high-scoring latent representation Z generated by the encoder HR (t) to generate a second correlation coefficient λ2.

[0076] Thus, the association relationship between the sample data after mapping in the current time frame and the sample data after mapping in the next time frame is determined through the above steps.

[0077] Step 308: Perform model training based on the low-score latent space, the high-score latent space, the first correlation coefficient, and the second correlation coefficient to obtain a trained data super-resolution model.

[0078] Step 308 is basically the same as step 204 in the aforementioned embodiment. The specific implementation method can refer to the aforementioned description of step 204, which will not be repeated here.

[0079] from Figure 3 It can be seen that Figure 2 Compared with the corresponding embodiment, the training method of the data super-resolution model in this embodiment first reduces the influence of spatial noise of the data through a unified latent space mapping. And through a shared code book, the consistent representation of data of different resolutions in the latent space is ensured. Then, through the two forward and backward time alignment networks, the correlation of adjacent time frames in the sample data is captured, so that the model can make full use of the bidirectional correlation in the time series and enhance the time alignment effect, thereby improving the detail performance of the high-resolution output.

[0080] Continue to refer Figure 4 , Figure 4 Shows Figure 3 A process 400 of a model training step in FIG. The model training step includes:

[0081] Step 401: using an initial super-resolution network to convert a low-score latent representation of a current time frame into a first high-score latent representation.

[0082] In this embodiment, the initial super-resolution network is first used to convert the low-score potential representation of the current time frame into a high-score potential representation, which is recorded as the first high-score potential representation. The initial super-resolution network is used to convert the low-score potential representation in the latent space into a high-score potential representation. The specific implementation is as follows:

[0083]

[0084] Among them, Z LR (t) is the low-score latent representation of the current time frame, ψ ST is the initial super-resolution network, is the first highest scoring potential representation.

[0085] Step 402: Train the initial super-resolution network based on the first high-score potential representation, the target high-score potential representation, the first correlation coefficient, and the second correlation coefficient to obtain a trained target super-resolution network.

[0086] In this embodiment, in the first stage of model training, the initial super-resolution network is trained using the first high-score potential representation, the target high-score potential representation, the first correlation coefficient, and the second correlation coefficient, thereby obtaining a trained target super-resolution network.

[0087] In some optional implementations of this embodiment, step 402 further includes:

[0088] Step 4021, constructing a first loss function based on the first correlation coefficient and the second correlation coefficient.

[0089] Here, the first correlation coefficient and the second correlation coefficient are used as regularization terms to construct the loss function, and the first loss function is obtained, so that the first loss function L includes not only the standard MSE term, but also the regularization terms of the forward and backward time alignment networks. The specific implementation is as follows:

[0090]

[0091] Step 4022, using a first loss function to calculate a first loss value between the first high-scoring potential representation and the target high-scoring potential representation.

[0092] Then, the first loss function L is used to calculate and Z HR (t), that is, the first loss value.

[0093] Step 4023: Adjust the parameters of the initial super-resolution network based on the first loss value to obtain a trained target super-resolution network.

[0094] Afterwards, the parameters of the initial super-resolution network are adjusted based on the calculated first loss value, thereby obtaining a trained target super-resolution network.

[0095] During the training of the initial super-resolution network, preliminary training is first performed in the latent space, combined with a bidirectional time alignment network as a regularization term, thereby making full use of the correlation of the time series.

[0096] Step 403: Use the sample data to train the initial super-resolution model to obtain a trained data super-resolution model.

[0097] In this embodiment, the sample data pairs are used to train the initial super-resolution model to obtain a trained data super-resolution model, wherein the initial super-resolution model includes a low-resolution encoder, a target super-resolution network, and a high-resolution decoder corresponding to the high-resolution encoder. Here, the initial sample data pairs are used to train the initial super-resolution model as a whole, thereby obtaining a trained data super-resolution model.

[0098] In some optional implementations of this embodiment, step 403 further includes:

[0099] Step 4031, using the target super-resolution network to convert the low-score latent representation of the current time frame into a second high-score latent representation.

[0100] That is, the target super-resolution network trained in step 402 is used to convert the low-score potential representation of the current time frame into a high-score potential representation, that is, the second highest-score potential representation.

[0101] Step 4032: Decode the second high-score latent representation using a high-score decoder to obtain high-score reconstructed data.

[0102] Then, a high-score decoder is used to decode the second high-score potential representation. The decoding process is the data reconstruction process, so as to obtain the corresponding high-score reconstructed data. The high-score reconstructed data is high-resolution data obtained by reconstructing the sample low-resolution data.

[0103] Step 4033, based on the second loss value between the high-resolution reconstructed data and the sample high-resolution data, adjust the parameters of the low-resolution encoder, the target super-resolution network and the high-resolution decoder to obtain a trained data super-resolution model.

[0104] Finally, the loss value between the high-resolution reconstructed data and the sample high-resolution data corresponding to the low-resolution data of the sample in the sample data pair is calculated, and the parameters of the initial super-resolution model are adjusted based on the loss value, that is, the parameters of the low-resolution encoder, the target super-resolution network and the high-resolution decoder are adjusted to obtain the trained data super-resolution model.

[0105] In some optional implementations of this embodiment, the second loss function includes a latitude weighted mean square error; and step 4043 further includes: calculating a second loss value between the high-resolution reconstructed data and the sample high-resolution data based on the latitude weighted mean square error formula; adjusting the parameters of the low-resolution encoder, the target super-resolution network, and the high-resolution decoder based on the second loss value.

[0106] In this implementation, the latitude weighted mean square error (LWMSE) is used as the second loss function, and the latitude weighted mean square error is used to calculate the second loss value between the high-resolution reconstruction data and the sample high-resolution data. The specific implementation is as follows:

[0107]

[0108] Where T is time, V is the variable, s is the scaling factor from low to high, H and W are the horizontal and vertical numbers of the earth grid, respectively, and L(h) is the latitude weighting factor used to adjust the area differences of different latitude regions:

[0109]

[0110] Among them, lat(h) represents the factor of latitude.

[0111] Then, based on the second loss value, the parameters of the low-score encoder, the target super-score network, and the high-score decoder are adjusted to obtain a trained data super-score model.

[0112] from Figure 4As can be seen in the figure, the model training step first conducts preliminary training in the latent space during the training of the initial super-resolution network, and combines the bidirectional time alignment network as a regularization term, thereby making full use of the correlation of the time series. Then, during the overall training of the super-resolution model, the encoder and decoder components are unfrozen, and the latitude-weighted mean square error is used as the loss function, thereby optimizing the output accuracy of the model in the sample data domain.

[0113] Furthermore, in an application scenario of the data super-resolution model disclosed in the present invention, Figure 5-1 The process of mapping data to the latent space is shown. First, the sample low-resolution data I in the sample data pair is converted by a low-resolution encoder. LR (t) is mapped to the low-score potential space to obtain the corresponding low-score potential representation Z LR (t). The high-resolution data I in the sample data pair is converted by the high-resolution encoder HR (t) is mapped to the high-score potential space to obtain the corresponding high-score potential representation Z HR (t).

[0114] Further, Figure 5-2 The process of bidirectional temporal alignment is shown, with the low-score latent representation Z after mapping to the current frame. LR (t) and the low-score potential representation Z after mapping to the previous frame LR (t-1), high score potential representation Z HR (t-1), predict the high-scoring latent representation of the current time frame through the forward time alignment network Then based on and the high-score potential representation Z obtained by the high-score encoder HR (t) to generate the first correlation coefficient λ1.

[0115] The low-score potential representation Z after mapping for the current frame LR (t) and the low-score potential representation Z after mapping to the next frame LR (t+1), high-score potential representation Z HR (t+1), predict the high-score latent representation of the current time frame through the backward time alignment network Then based on and the high-score potential representation Z obtained by the high-score encoder HR (t) to generate a second correlation coefficient λ2.

[0116] Further, Figure 5-3 The model training process is shown. The initial super-resolution network is used to transform Z LR (t-1), Z LR (t), Z LR (t+1) converted into high-score latent representation A loss function is constructed based on the first correlation coefficient λ1 and the second correlation coefficient λ2, and the loss function is used to calculate and Z HR (t), Z HR (t-1), Z HR The loss value between (t+1) is used to adjust the parameters of the initial super-resolution network to obtain the trained super-resolution network.

[0117] Finally, the trained super-resolution network is used to generate Z LR (t) corresponding Using the high-resolution decoder corresponding to the high-resolution encoder Decode and reconstruct Based on and I HR The loss value between (t) is used to adjust the parameters of the data super-resolution model (including the low-resolution encoder, the target super-resolution network and the high-resolution decoder) to obtain the trained data super-resolution model.

[0118] Continue to refer Figure 6 , which shows a process 600 of an embodiment of a data super-resolution processing method according to the present disclosure. The data super-resolution processing method comprises the following steps:

[0119] Step 601, obtaining low-resolution data to be super-resolved.

[0120] In this embodiment, the execution subject of the data super-resolution processing method (for example Figure 1 The server 105 shown in the figure will obtain low-resolution data to be super-resolution processed, and the low-resolution data can be low-resolution data that needs to be super-resolution processed in various fields. For example, when the data super-resolution processing method is applied to the field of image compression, the low-resolution data can be a compressed low-resolution image.

[0121] Step 602: input the low-resolution data into a data super-resolution model, and output high-resolution data corresponding to the low-resolution data.

[0122] In this embodiment, the above-mentioned execution subject inputs the low-resolution data into the data super-resolution model, and outputs the high-resolution data corresponding to the low-resolution data. The data super-resolution model can be trained using the method described in the above embodiment. Since the data super-resolution model can convert the low-resolution data into the corresponding high-resolution data, the low-resolution data to be processed is input into the data super-resolution model, and the corresponding high-resolution data can be output.

[0123] In some optional implementations of this embodiment, the data super-resolution model includes a low-resolution encoder, a target super-resolution network and a high-resolution decoder; and step 602 further includes:

[0124] Step 6021, input the low-resolution data into the low-score encoder, and output the target low-score potential representation.

[0125] The low-resolution data is input into the low-score encoder, which maps the low-resolution data to the corresponding low-score latent space to obtain a low-score potential representation of the low-resolution data in the low-score latent space, i.e., the target low-score potential representation.

[0126] Step 6022, input the target low-score latent representation into the target super-resolution network, and output the converted high-score latent representation.

[0127] The target low-score latent representation is then input into the target super-resolution network, which converts the target low-score latent representation into a high-score latent representation to obtain a converted high-score latent representation.

[0128] Step 6023, input the converted high-resolution potential representation into a high-resolution decoder, and output high-resolution data.

[0129] Finally, the converted high-resolution potential representation is input into the high-resolution decoder, which decodes and reconstructs it and outputs the high-resolution data corresponding to the low-resolution data.

[0130] Therefore, through the low-resolution encoder, target super-resolution network and high-resolution decoder in the data super-resolution model, high-resolution data corresponding to the low-resolution data can be accurately generated.

[0131] In some optional implementations of this embodiment, the low-resolution data includes low-resolution climate data, and the high-resolution data includes high-resolution climate data. That is, the climate data can be super-resolved using a data super-resolution model to convert the low-resolution climate data into a high-resolution format, thereby providing more detailed and accurate climate information for a specific area.

[0132] The data super-resolution processing method provided in the embodiment of the present disclosure first obtains low-resolution data to be super-resolution processed; then the low-resolution data is input into the data super-resolution model, and the high-resolution data corresponding to the low-resolution data is output. The data super-resolution processing method in this embodiment converts the low-resolution data into a high-resolution format through a pre-trained data super-resolution model, thereby improving the accuracy of data super-resolution processing.

[0133] In an application scenario, the data super-resolution model, also known as the Temporal-Enhanced Super-Resolution (TESR) framework, significantly improves the accuracy and detail performance of high-resolution climate data when super-resolution processing is performed on climate data by making full use of time correlation. Experimental results on multiple climate data sets (such as CMIP6 and ERA5) show that the data super-resolution model in this embodiment has achieved the lowest root mean square error (RMSE) among all tested climate variables. Specifically, the RMSE of the TESR framework on the five climate variables of Z500, T850, T2m, U10, and V10 are 1077.73, 5.41, 6.02, 4.22, and 4.23, respectively. The following lists the RMSE of the TESR framework and the prior art VRT (Video Restoration Techniques, video processing technology), SwinIR super-resolution method, and ClimaX deep learning model on the five climate variables of Z500, T850, T2m, U10, and V10.

[0134]

[0135] It can be seen that the TESR framework performs better on all variables, especially on Z500 and T2m variables, with RMSE reduced by 10.69 and 0.09, respectively.

[0136] Further references Figure 7 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a training device for a data super-resolution model. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0137] like Figure 7As shown, the training device 700 of the data super-resolution model of this embodiment includes: a mapping module 701, a first coefficient determination module 702, a second coefficient determination module 703 and a training module 704. The mapping module 701 is configured to map the sample low-resolution data and the sample high-resolution data in the sample data pair to the corresponding low-resolution latent space and high-resolution latent space respectively; the first coefficient determination module 702 is configured to determine the first correlation coefficient between the sample data pair mapped in the current time frame and the sample data pair mapped in the previous time frame; the second coefficient determination module 703 is configured to determine the second correlation coefficient between the sample data pair mapped in the current time frame and the sample data pair mapped in the next time frame; the training module 704 is configured to perform model training based on the low-resolution latent space, the high-resolution latent space, the first correlation coefficient and the second correlation coefficient to obtain the trained data super-resolution model, wherein the data super-resolution model is used to convert low-resolution data into high-resolution data.

[0138] In the data super-resolution model training device 700, the specific processing of the mapping module 701, the first coefficient determination module 702, the second coefficient determination module 703 and the training module 704 and the technical effects thereof can be referred to in Figure 2 The relevant descriptions of steps 201 - 204 in the corresponding embodiment are not repeated here.

[0139] In some optional implementations of this embodiment, the low-score latent space includes low-score latent representations of sample low-resolution data at each moment, and the high-score latent space includes high-score latent representations of sample high-resolution data at each moment; and the mapping module is further configured to: generate low-score latent representations of sample low-resolution data at each moment using a low-score encoder; and generate high-score latent representations of sample high-resolution data at each moment using a high-score encoder.

[0140] In some optional implementations of this embodiment, the training device 700 of the above-mentioned data super-resolution model also includes: an association module, configured to associate the low-score potential representation at each moment in the low-score latent space with the high-score potential representation at each moment in the high-score latent space through a code book.

[0141] In some optional implementations of the present embodiment, the first coefficient determination module is further configured to: determine a target high-score potential representation associated with a low-score potential representation of the current time frame through a code book; determine a first predicted high-score potential representation of the current time frame based on the low-score potential representation, high-score potential representation of the sample data pair after mapping from the previous time frame, and the low-score potential representation of the sample low-resolution data of the current time frame; and generate a first association coefficient based on the numerical relationship between the first predicted high-score potential representation and the target high-score potential representation.

[0142] In some optional implementations of the present embodiment, the second coefficient determination module is further configured to: determine the second predicted high-score potential representation of the current time frame based on the low-score potential representation, high-score potential representation of the sample data pair after mapping to the next time frame, and the low-score potential representation of the sample low-resolution data of the current time frame; and generate a second correlation coefficient based on the numerical relationship between the second predicted high-score potential representation and the target high-score potential representation.

[0143] In some optional implementations of this embodiment, the training module includes: a conversion submodule, configured to use an initial super-resolution network to convert a low-score potential representation of the current time frame into a first high-score potential representation; a first training submodule, configured to train the initial super-resolution network based on the first high-score potential representation, the target high-score potential representation, the first correlation coefficient and the second correlation coefficient to obtain a trained target super-resolution network; a second training submodule, configured to use sample data to train the initial super-resolution model to obtain a trained data super-resolution model, wherein the initial super-resolution model includes a low-score encoder, a target super-resolution network, and a high-score decoder corresponding to the high-score encoder.

[0144] In some optional implementations of this embodiment, the first training submodule is further configured to: construct a first loss function based on the first correlation coefficient and the second correlation coefficient; use the first loss function to calculate a first loss value between the first high-score potential representation and the target high-score potential representation; adjust the parameters of the initial super-resolution network based on the first loss value to obtain the trained target super-resolution network.

[0145] In some optional implementations of the present embodiment, the second training submodule includes: a conversion unit, configured to use a target super-resolution network to convert a low-score potential representation of the current time frame into a second high-score potential representation; a decoding unit, configured to use a high-score decoder to decode the second high-score potential representation to obtain high-score reconstruction data; an adjustment unit, configured to adjust the parameters of the low-score encoder, the target super-resolution network and the high-score decoder based on a second loss value between the high-score reconstruction data and the sample high-resolution data to obtain a trained data super-resolution model.

[0146] In some optional implementations of this embodiment, the second loss function includes a latitude weighted mean square error; and the adjustment unit is further configured to: calculate a second loss value between the high-resolution reconstructed data and the sample high-resolution data based on the latitude weighted mean square error formula; and adjust the parameters of the low-resolution encoder, the target super-resolution network, and the high-resolution decoder based on the second loss value.

[0147] In some optional implementations of this embodiment, the sample data pair includes: sample low-resolution climate data and sample high-resolution climate data.

[0148] Further references Figure 8As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a data super-resolution processing device. Figure 6 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0149] like Figure 8 As shown, the data super-resolution processing device 800 of this embodiment includes: an acquisition module 801 and a super-resolution module 802. The acquisition module 801 is configured to acquire low-resolution data to be super-resolution processed; the super-resolution module 802 is configured to input the low-resolution data into the data super-resolution model, and output the high-resolution data corresponding to the low-resolution data, wherein the data super-resolution model is trained by the method described in the above embodiment.

[0150] In the data super-resolution processing device 800, the specific processing of the acquisition module 801 and the super-resolution module 802 and the technical effects thereof can be referred to in Figure 6 The relevant descriptions of steps 601-602 in the corresponding embodiment are not repeated here.

[0151] In some optional implementations of this embodiment, the data super-resolution model includes a low-resolution encoder, a target super-resolution network and a high-resolution decoder; and the super-resolution module is further configured to: input the low-resolution data into the low-resolution encoder, and output the target low-resolution potential representation; input the target low-resolution potential representation into the target super-resolution network, and output the converted high-score potential representation; input the converted high-score potential representation into the high-score decoder, and output the high-resolution data.

[0152] In some optional implementations of this embodiment, the low-resolution data includes low-resolution climate data, and the high-resolution data includes high-resolution climate data.

[0153] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0154] Fig. 9 A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0155] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0156] Multiple components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0157] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as a training method for a data super-resolution model or a data super-resolution processing method. For example, in some embodiments, a training method for a data super-resolution model or a data super-resolution processing method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the training method for the data super-resolution model or the data super-resolution processing method described above may be executed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute a data super-resolution model training method or a data super-resolution processing method in any other appropriate manner (for example, by means of firmware).

[0158] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0159] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0160] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0161] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0162] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0163] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0164] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0165] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for training a data super-resolution model, comprising: Mapping the sample low-resolution data and the sample high-resolution data in the sample data pair to the corresponding low-score latent space and high-score latent space respectively; Determine a first correlation coefficient between a sample data pair mapped in a current time frame and a sample data pair mapped in a previous time frame; Determine a second correlation coefficient between the sample data pair mapped in the current time frame and the sample data pair mapped in the next time frame; Model training is performed based on the low-score latent space, the high-score latent space, the first correlation coefficient, and the second correlation coefficient to obtain a trained data super-resolution model, wherein the data super-resolution model is used to convert low-resolution data into high-resolution data.

2. The method according to claim 1, wherein: The low-score latent space includes the low-score latent representation of the sample low-resolution data at each time, and the high-score latent space includes the high-score latent representation of the sample high-resolution data at each time; as well as The mapping of the sample low-resolution data and the sample high-resolution data in the sample data pair to the corresponding low-score latent space and high-score latent space respectively includes: Generate a low-score potential representation of the sample low-resolution data at each moment using a low-score encoder; A high-score encoder is used to generate a high-score potential representation of the sample high-resolution data at each moment.

3. The method according to claim 2, further comprising: The low-score latent representation at each moment in the low-score latent space is associated with the high-score latent representation at each moment in the high-score latent space through a codebook.

4. The method according to claim 3, wherein: The determining of a first correlation coefficient between a sample data pair mapped in a current time frame and a sample data pair mapped in a previous time frame comprises: Determine, by means of the codebook, a target high-score latent representation associated with the low-score latent representation of the current time frame; Determine a first predicted high-score potential representation of the current time frame based on the low-score potential representation and the high-score potential representation of the sample data pair mapped from the previous time frame and the low-score potential representation of the sample low-resolution data of the current time frame; A first correlation coefficient is generated based on a numerical relationship between the first predicted high-scoring latent representation and the target high-scoring latent representation.

5. The method according to claim 4, wherein: The determining of a second correlation coefficient between the sample data pair mapped in the current time frame and the sample data pair mapped in the next time frame comprises: Determine a second predicted high-score potential representation of the current time frame based on the low-score potential representation and the high-score potential representation of the sample data pair mapped from the next time frame and the low-score potential representation of the sample low-resolution data of the current time frame; A second correlation coefficient is generated based on the numerical relationship between the second predicted high-scoring potential representation and the target high-scoring potential representation.

6. The method according to claim 5, wherein: The performing model training based on the low-score latent space, the high-score latent space, the first correlation coefficient, and the second correlation coefficient to obtain a trained data super-resolution model includes: Converting the low-score latent representation of the current time frame into a first high-score latent representation using an initial super-resolution network; Training the initial super-resolution network based on the first high-score potential representation, the target high-score potential representation, the first correlation coefficient, and the second correlation coefficient to obtain a trained target super-resolution network; The sample data is used to train the initial super-resolution model to obtain a trained data super-resolution model, wherein the initial super-resolution model includes the low-resolution encoder, the target super-resolution network, and a high-resolution decoder corresponding to the high-resolution encoder.

7. The method according to claim 6, wherein: The step of training the initial super-resolution network based on the first high-score potential representation, the target high-score potential representation, the first correlation coefficient, and the second correlation coefficient to obtain a trained target super-resolution network includes: Constructing a first loss function based on the first correlation coefficient and the second correlation coefficient; Calculating a first loss value between the first high-scoring potential representation and the target high-scoring potential representation using the first loss function; The parameters of the initial super-resolution network are adjusted based on the first loss value to obtain a trained target super-resolution network.

8. The method according to claim 7, wherein: The step of training the initial super-resolution model using the sample data to obtain a trained data super-resolution model comprises: Converting the low-scoring latent representation of the current time frame into a second high-scoring latent representation using the target super-resolution network; Decoding the second high-score potential representation using the high-score decoder to obtain high-score reconstructed data; Based on a second loss value between the high-resolution reconstruction data and the sample high-resolution data, the parameters of the low-resolution encoder, the target super-resolution network and the high-resolution decoder are adjusted to obtain a trained data super-resolution model.

9. The method according to claim 8, wherein: The second loss function includes a latitude-weighted mean square error; and The adjusting the parameters of the low-resolution encoder, the target super-resolution network, and the high-resolution decoder based on the second loss value between the high-resolution reconstruction data and the sample high-resolution data comprises: Calculating a second loss value between the high-resolution reconstruction data and the sample high-resolution data based on the latitude weighted mean square error formula; The parameters of the low-score encoder, the target super-score network, and the high-score decoder are adjusted based on the second loss value.

10. The method according to claim 1, wherein: The sample data pair includes: sample low-resolution climate data and sample high-resolution climate data.

11. A data super-resolution processing method, comprising: Obtain low-resolution data to be super-resolved; The low-resolution data is input into a data super-resolution model, and high-resolution data corresponding to the low-resolution data is output, wherein the data super-resolution model is trained using the method described in any one of claims 1-10.

12. The method according to claim 11, wherein: The data super-resolution model includes a low-resolution encoder, a target super-resolution network and a high-resolution decoder; as well as The step of inputting the low-resolution data into a data super-resolution model and outputting high-resolution data corresponding to the low-resolution data comprises: Input the low-resolution data into the low-score encoder, and output a target low-score potential representation; Inputting the target low-score potential representation into the target super-resolution network, and outputting a converted high-score potential representation; The converted high-score potential representation is input into the high-score decoder, and the high-resolution data is output.

13. The method according to claim 11, wherein: The low-resolution data includes low-resolution climate data, and the high-resolution data includes high-resolution climate data.

14. A training device for a data super-resolution model, comprising: A mapping module, configured to map the sample low-resolution data and the sample high-resolution data in the sample data pair to the corresponding low-score latent space and high-score latent space respectively; A first coefficient determination module, configured to determine a first correlation coefficient between a sample data pair mapped in a current time frame and a sample data pair mapped in a previous time frame; A second coefficient determination module is configured to determine a second correlation coefficient between the sample data pair mapped in the current time frame and the sample data pair mapped in the next time frame; The training module is configured to perform model training based on the low-score latent space, the high-score latent space, the first correlation coefficient, and the second correlation coefficient to obtain a trained data super-resolution model, wherein the data super-resolution model is used to convert low-resolution data into high-resolution data.

15. The device according to claim 14, wherein: The low-score latent space includes the low-score latent representation of the sample low-resolution data at each time instant, and the high-score latent space includes the high-score latent representation of the sample high-resolution data at each time instant; and The mapping module is further configured to: Generate a low-score potential representation of the sample low-resolution data at each moment using a low-score encoder; A high-score encoder is used to generate a high-score potential representation of the sample high-resolution data at each moment.

16. The apparatus according to claim 15, further comprising: The association module is configured to associate the low-score latent representation at each moment in the low-score latent space with the high-score latent representation at each moment in the high-score latent space through a code book.

17. The device according to claim 16, wherein: The first coefficient determination module is further configured to: Determine, by means of the codebook, a target high-score latent representation associated with the low-score latent representation of the current time frame; Determine a first predicted high-score potential representation of the current time frame based on the low-score potential representation and the high-score potential representation of the sample data pair mapped from the previous time frame and the low-score potential representation of the sample low-resolution data of the current time frame; A first correlation coefficient is generated based on a numerical relationship between the first predicted high-scoring latent representation and the target high-scoring latent representation.

18. The device according to claim 17, wherein: The second coefficient determination module is further configured to: Determine a second predicted high-score potential representation of the current time frame based on the low-score potential representation and the high-score potential representation of the sample data pair mapped from the next time frame and the low-score potential representation of the sample low-resolution data of the current time frame; A second correlation coefficient is generated based on the numerical relationship between the second predicted high-scoring potential representation and the target high-scoring potential representation.

19. The device according to claim 18, wherein: The training module includes: a conversion submodule configured to convert the low-score latent representation of the current time frame into a first high-score latent representation using an initial super-resolution network; A first training submodule is configured to train the initial super-resolution network based on the first high-score potential representation, the target high-score potential representation, the first association coefficient, and the second association coefficient to obtain a trained target super-resolution network; The second training submodule is configured to use the sample data to train the initial super-resolution model to obtain a trained data super-resolution model, wherein the initial super-resolution model includes the low-score encoder, the target super-resolution network and a high-score decoder corresponding to the high-score encoder.

20. The device according to claim 19, wherein The first training submodule is further configured to: Constructing a first loss function based on the first correlation coefficient and the second correlation coefficient; Calculating a first loss value between the first high-scoring potential representation and the target high-scoring potential representation using the first loss function; The parameters of the initial super-resolution network are adjusted based on the first loss value to obtain a trained target super-resolution network.

21. The device according to claim 20, wherein: The second training submodule includes: a conversion unit configured to convert the low-score latent representation of the current time frame into a second high-score latent representation using the target super-resolution network; a decoding unit, configured to decode the second high-score potential representation using the high-score decoder to obtain high-score reconstruction data; The adjustment unit is configured to adjust the parameters of the low-resolution encoder, the target super-resolution network and the high-resolution decoder based on a second loss value between the high-resolution reconstruction data and the sample high-resolution data to obtain a trained data super-resolution model.

22. The device according to claim 21, wherein The second loss function includes a latitude-weighted mean square error; and The adjustment unit is further configured to: Calculating a second loss value between the high-resolution reconstruction data and the sample high-resolution data based on the latitude weighted mean square error formula; The parameters of the low-score encoder, the target super-score network, and the high-score decoder are adjusted based on the second loss value.

23. The device according to claim 14, wherein: The sample data pair includes: sample low-resolution climate data and sample high-resolution climate data.

24. A data super-resolution processing device, comprising: An acquisition module is configured to acquire low-resolution data to be super-resolved; The super-resolution module is configured to input the low-resolution data into a data super-resolution model and output high-resolution data corresponding to the low-resolution data, wherein the data super-resolution model is trained using the method described in any one of claims 1-10.

25. The device according to claim 24, wherein: The data super-resolution model includes a low-resolution encoder, a target super-resolution network and a high-resolution decoder; and The super-resolution module is further configured to: Input the low-resolution data into the low-score encoder, and output a target low-score potential representation; Inputting the target low-score potential representation into the target super-resolution network, and outputting a converted high-score potential representation; The converted high-score potential representation is input into the high-score decoder, and the high-resolution data is output.

26. The device according to claim 24, wherein: The low-resolution data includes low-resolution climate data, and the high-resolution data includes high-resolution climate data.

27. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10 or claims 11-13.

28. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method of any one of claims 1-10 or claims 11-13.

29. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1-10 or claims 11-13.

Images