A method and apparatus for reconstructing typhoon data from synthetic aperture radar.

Abstract

This application provides a method and apparatus for reconstructing synthetic aperture radar (SAR) typhoon data. The method involves acquiring an initial SAR typhoon image to be reconstructed; preprocessing the initial SAR typhoon image to identify target typhoon regions, resulting in a typhoon image to be reconstructed; using pre-trained local and global branch networks, extracting features from the typhoon image to be reconstructed, resulting in an aggregated typhoon image; using a pre-constructed aggregation network, aggregating features from the aggregated typhoon image, resulting in an aggregated typhoon image; and adjusting the size of the aggregated typhoon image according to the size of the initial SAR typhoon image to obtain the reconstructed typhoon image. This allows for the reconstruction of typhoon data within a portion of the target typhoon region in a typhoon image, ensuring the quality of SAR typhoon data used for predicting sea surface wind speed, and indirectly improving the accuracy of typhoon intensity estimation and typhoon structure information.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for reconstructing synthetic aperture radar typhoon data. Background Technology

[0002] SAR (Spectroscopic Radiography) is an active Earth observation system that can be installed on aircraft, satellites, spacecraft, and other flying platforms to conduct all-weather, 24 / 7 Earth observations and has a certain degree of surface penetration capability. Therefore, SAR systems have unique advantages in applications such as disaster monitoring, environmental monitoring, marine monitoring, resource exploration, crop yield estimation, surveying and mapping, and military applications. In particular, SAR satellites can be used to observe typhoons, enabling the monitoring of typhoon intensity and structure. However, extreme wind speeds can cause saturation of C-band SAR signals, and the influence of rainfall further degrades the data quality of SAR images under typhoon conditions, directly affecting the estimation of typhoon intensity and structure. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a method and apparatus for reconstructing synthetic aperture radar typhoon data, which can reconstruct typhoon data in a portion of the target typhoon area in a typhoon image to ensure the quality of SAR typhoon data, thereby indirectly improving the accuracy of typhoon intensity estimation and structure information.

[0004] This application provides a method for reconstructing synthetic aperture radar typhoon data, the method comprising:

[0005] Acquire the initial Synthetic Aperture Radar (SAR) typhoon image to be reconstructed;

[0006] The initial SAR typhoon image is preprocessed to determine the target typhoon region in the initial SAR typhoon image, thereby obtaining the typhoon image to be reconstructed.

[0007] Using pre-trained local branch networks and global branch networks, feature extraction is performed on the typhoon image to be reconstructed to obtain the typhoon image to be aggregated; wherein, the typhoon image to be aggregated includes local features and global features extracted from the typhoon image to be reconstructed;

[0008] The typhoon image to be aggregated is processed by feature aggregation through a pre-constructed aggregation network to obtain an aggregated typhoon image.

[0009] Based on the size of the initial SAR typhoon image, the size of the aggregated typhoon image is adjusted to obtain a reconstructed typhoon image.

[0010] In one possible implementation, the step of using pre-trained local branch networks and global branch networks to extract features from the typhoon image to be reconstructed, respectively, to obtain the typhoon image to be aggregated, includes:

[0011] By utilizing multiple convolutional layers in the local branch network, the local features between local pixels in the typhoon image to be reconstructed are extracted by performing multiple first convolutional processes on the typhoon image to be reconstructed, and the typhoon data in the target typhoon area is reconstructed to obtain the first reconstructed typhoon image.

[0012] By utilizing multiple attention mechanism convolutional layers (CoT Block layers) in the global branch network, the typhoon image to be reconstructed is subjected to multiple second convolutional processes to extract global features between global pixels in the typhoon image to be reconstructed, and the typhoon data in the target typhoon area is reconstructed to obtain the second reconstructed typhoon image.

[0013] A third convolution process is performed on the first reconstructed typhoon image and the second reconstructed typhoon image to fuse the local features in the first reconstructed typhoon image and the global features in the second reconstructed typhoon image to obtain the typhoon image to be aggregated.

[0014] In one possible implementation, the step of utilizing multiple convolutional layers in the local branch network to perform multiple first convolutional processes on the typhoon image to be reconstructed, extracting local features between local pixels in the typhoon image to be reconstructed, and reconstructing typhoon data within the target typhoon area to obtain a first reconstructed typhoon image includes:

[0015] By utilizing multiple convolutional layers in the local branch network and referencing typhoon data within a first reference region surrounding the target typhoon region, the local features between local pixels in the typhoon image to be reconstructed are extracted by performing multiple first convolutional processes on the typhoon image to be reconstructed, and the typhoon data within the target typhoon region is reconstructed to obtain the first reconstructed typhoon image.

[0016] In one possible implementation, the second reconstructed typhoon image is obtained by utilizing multiple attention mechanism convolutional layers (CoT Block layers) in the global branch network to perform multiple second convolutional processes on the typhoon image to be reconstructed, extracting global features between global pixels in the typhoon image to be reconstructed, and reconstructing typhoon data within the target typhoon region, including:

[0017] Using multiple CoT Block layers in the global branch network, and referencing typhoon data in a second reference region associated with the target typhoon region, the global features between global pixels in the typhoon image to be reconstructed are extracted by performing multiple second convolution processes on the typhoon image to be reconstructed, and the typhoon data in the target typhoon region is reconstructed to obtain a second reconstructed typhoon image; wherein, the correlation between the second reference region and the target typhoon region is greater than a preset correlation threshold.

[0018] In one possible implementation, the CoT Block layer includes a first convolutional sub-layer, a second convolutional sub-layer, an ECA module, and an attention mechanism (COT) layer; utilizing multiple CoT Block layers in the global branch network, referencing typhoon data within a second reference region associated with the target typhoon region, and performing multiple second convolutional processes on the typhoon image to be reconstructed, global features between global pixels in the typhoon image to be reconstructed are extracted to reconstruct the typhoon data within the target typhoon region, resulting in a second reconstructed typhoon image, including:

[0019] For each CoT Block layer, the first convolutional sub-layer in the CoT Block layer is used to refer to the typhoon data in the second reference area associated with the target typhoon area to perform a fourth convolutional process on the typhoon image to be reconstructed, so as to obtain the first feature map.

[0020] The first feature map is used to calculate the similarity of the CoT Block layer to obtain the second feature map.

[0021] Using the second convolutional sub-layer in the CoT Block layer, and referencing typhoon data in a second reference region associated with the target typhoon region, the second feature map is subjected to a fifth convolutional process to obtain a third feature map; using the ECA module in the CoT Block layer, the channel information of different pixel channels of the third feature map is captured to obtain a second reconstructed typhoon image.

[0022] In one possible implementation, the aggregation network includes at least one first sub-aggregation layer and at least one second sub-aggregation layer; the step of performing feature aggregation processing on the typhoon image to be aggregated through the pre-constructed aggregation network to obtain an aggregated typhoon image includes:

[0023] Using the at least one first sub-aggregation layer, the typhoon image to be aggregated is subjected to a first aggregation process to obtain a first sub-typhoon feature map;

[0024] Using the at least one second sub-aggregation layer, the typhoon image to be aggregated is subjected to a second aggregation process to obtain a second sub-typhoon feature map;

[0025] Based on the feature maps of the first and second sub-typhoons, an aggregated typhoon image is obtained.

[0026] In one possible implementation, the preprocessing of the initial typhoon image to determine the target typhoon region in the initial typhoon image and obtain the typhoon image to be reconstructed includes:

[0027] The initial SAR typhoon image is preprocessed, and the target typhoon areas with invalid data quality identifiers in the initial SAR typhoon image are determined according to the data quality identifiers corresponding to each pixel position in the initial SAR typhoon image, so as to obtain the typhoon image to be reconstructed.

[0028] In one possible implementation, the reconstruction method further includes:

[0029] The reconstructed typhoon image is input into a pre-trained image refinement model, and the target typhoon image corresponding to the reconstructed typhoon image is generated by the image refinement model; wherein the image precision of the target typhoon image is lower than that of the reconstructed typhoon image.

[0030] In one possible implementation, the image refinement model is trained through the following steps:

[0031] Obtain a training image set and a test image set for training the image refinement model; wherein, the training image set includes multiple training typhoon images and a training label image for each training typhoon image; the test image set includes multiple test typhoon images and a test label image for each test typhoon image.

[0032] Multiple training typhoon images from the training image set are used as input features, and the training label image of each training typhoon image is used as output features to train the pre-constructed image generation network, thus obtaining a preliminary generation network.

[0033] Using the test image set, the total loss value of the loss value group of the preliminary generation network is determined through a pre-constructed image discrimination network; wherein, the loss value group includes reconstruction loss value, perceptual loss value, style loss value, and adversarial loss value;

[0034] If the total loss value is greater than the preset loss threshold, the network parameters of the initially generated network are adjusted with reference to the total loss value until the total loss value is less than or equal to the preset loss threshold, thereby obtaining the image refinement model.

[0035] This application embodiment also provides a reconstruction device for synthetic aperture radar typhoon data, the reconstruction device comprising:

[0036] The image acquisition module is used to acquire the initial synthetic aperture radar (SAR) typhoon image to be reconstructed;

[0037] The region determination module is used to preprocess the initial SAR typhoon image, determine the target typhoon region in the initial SAR typhoon image, and obtain the typhoon image to be reconstructed.

[0038] The feature extraction module is used to extract features from the typhoon image to be reconstructed using pre-trained local branch networks and global branch networks, respectively, to obtain the typhoon image to be aggregated; wherein, the typhoon image to be aggregated includes local features and global features extracted from the typhoon image to be reconstructed;

[0039] The aggregation module is used to perform feature aggregation processing on the typhoon image to be aggregated through a pre-built aggregation network to obtain an aggregated typhoon image.

[0040] The size adjustment module is used to adjust the size of the aggregated typhoon image according to the size of the initial SAR typhoon image, so as to obtain the reconstructed typhoon image.

[0041] In one possible implementation, when the feature extraction module is used to extract features from the typhoon image to be reconstructed using pre-trained local branch networks and global branch networks respectively, to obtain the typhoon image to be aggregated, the feature extraction module is used to:

[0042] By utilizing multiple convolutional layers in the local branch network, the local features between local pixels in the typhoon image to be reconstructed are extracted by performing multiple first convolutional processes on the typhoon image to be reconstructed, and the typhoon data in the target typhoon area is reconstructed to obtain the first reconstructed typhoon image.

[0043] By utilizing multiple attention mechanism convolutional layers (CoT Block layers) in the global branch network, the typhoon image to be reconstructed is subjected to multiple second convolutional processes to extract global features between global pixels in the typhoon image to be reconstructed, and the typhoon data in the target typhoon area is reconstructed to obtain the second reconstructed typhoon image.

[0044] A third convolution process is performed on the first reconstructed typhoon image and the second reconstructed typhoon image to fuse the local features in the first reconstructed typhoon image and the global features in the second reconstructed typhoon image to obtain the typhoon image to be aggregated.

[0045] In one possible implementation, when the feature extraction module utilizes multiple convolutional layers in the local branch network to perform multiple first convolutional processes on the typhoon image to be reconstructed, extracting local features between local pixels in the typhoon image to be reconstructed, and reconstructing typhoon data within the target typhoon region to obtain a first reconstructed typhoon image, the feature extraction module is used to:

[0046] By utilizing multiple convolutional layers in the local branch network and referencing typhoon data within a first reference region surrounding the target typhoon region, the local features between local pixels in the typhoon image to be reconstructed are extracted by performing multiple first convolutional processes on the typhoon image to be reconstructed, and the typhoon data within the target typhoon region is reconstructed to obtain the first reconstructed typhoon image.

[0047] In one possible implementation, when the feature extraction module performs multiple second convolutional processes on the typhoon image to be reconstructed using multiple attention mechanism convolutional layers (CoT Block layers) in the global branch network to extract global features between global pixels in the typhoon image to be reconstructed, and reconstructs the typhoon data within the target typhoon region to obtain the second reconstructed typhoon image, the feature extraction module is used to:

[0048] Using multiple CoT Block layers in the global branch network, and referencing typhoon data in a second reference region associated with the target typhoon region, the global features between global pixels in the typhoon image to be reconstructed are extracted by performing multiple second convolution processes on the typhoon image to be reconstructed, and the typhoon data in the target typhoon region is reconstructed to obtain a second reconstructed typhoon image; wherein, the correlation between the second reference region and the target typhoon region is greater than a preset correlation threshold.

[0049] In one possible implementation, the CoT Block layer includes a first convolutional sub-layer, a second convolutional sub-layer, an ECA module, and an attention mechanism (COT) layer; when the feature extraction module utilizes multiple CoT Block layers in the global branch network, referencing typhoon data in a second reference region associated with the target typhoon region, to extract global features between global pixels in the typhoon image to be reconstructed by performing multiple second convolutional processes on the typhoon image to be reconstructed, and reconstructs the typhoon data within the target typhoon region to obtain a second reconstructed typhoon image, the feature extraction module is used to:

[0050] For each CoT Block layer, the first convolutional sub-layer in the CoT Block layer is used to refer to the typhoon data in the second reference area associated with the target typhoon area to perform a fourth convolutional process on the typhoon image to be reconstructed, so as to obtain the first feature map.

[0051] The first feature map is used to calculate the similarity of the CoT Block layer to obtain the second feature map.

[0052] Using the second convolutional sublayer in the CoT Block layer, and referencing typhoon data in a second reference region associated with the target typhoon region, the second feature map is subjected to a fifth convolutional process to obtain the third feature map;

[0053] The ECA module in the CoT Block layer is used to capture the channel information of different pixel channels of the third feature map to obtain the second reconstructed typhoon image.

[0054] In one possible implementation, the aggregation network includes at least one first sub-aggregation layer and at least one second sub-aggregation layer; when the aggregation module performs feature aggregation processing on the typhoon image to be aggregated through the pre-built aggregation network to obtain an aggregated typhoon image, the aggregation module is used to:

[0055] Using the at least one first sub-aggregation layer, the typhoon image to be aggregated is subjected to a first aggregation process to obtain a first sub-typhoon feature map;

[0056] Using the at least one second sub-aggregation layer, the typhoon image to be aggregated is subjected to a second aggregation process to obtain a second sub-typhoon feature map;

[0057] Based on the feature maps of the first and second sub-typhoons, an aggregated typhoon image is obtained.

[0058] In one possible implementation, when the region determination module preprocesses the initial SAR typhoon image to determine the target typhoon region in the initial SAR typhoon image and obtains the typhoon image to be reconstructed, the region determination module is used to:

[0059] The initial SAR typhoon image is preprocessed, and the target typhoon areas with invalid data quality identifiers in the initial SAR typhoon image are determined according to the data quality identifiers corresponding to each pixel position in the initial SAR typhoon image, so as to obtain the typhoon image to be reconstructed.

[0060] In one possible implementation, the reconstruction apparatus further includes an image generation module, the image generation module being used for:

[0061] The reconstructed typhoon image is input into a pre-trained image refinement model, and a target typhoon image corresponding to the reconstructed typhoon image is generated through the image refinement model; wherein, the image precision of the target typhoon image is higher than that of the reconstructed typhoon image.

[0062] In one possible implementation, the reconstruction apparatus further includes a model training module, which is used to train the image refinement model through the following steps:

[0063] Obtain a training image set and a test image set for training the image refinement model; wherein, the training image set includes multiple training typhoon images and a training label image for each training typhoon image; the test image set includes multiple test typhoon images and a test label image for each test typhoon image.

[0064] Multiple training typhoon images from the training image set are used as input features, and the training label image of each training typhoon image is used as output features to train the pre-constructed image generation network, thus obtaining a preliminary generation network.

[0065] Using the test image set, the total loss value of the loss value group of the preliminary generation network is determined through a pre-constructed image discrimination network; wherein, the loss value group includes reconstruction loss value, perceptual loss value, style loss value, and adversarial loss value;

[0066] If the total loss value is greater than the preset loss threshold, the network parameters of the initially generated network are adjusted with reference to the total loss value until the total loss value is less than or equal to the preset loss threshold, thereby obtaining the image refinement model.

[0067] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the synthetic aperture radar typhoon data reconstruction method described above are performed.

[0068] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the synthetic aperture radar typhoon data reconstruction method described above.

[0069] The synthetic aperture radar (SAR) typhoon data reconstruction method and apparatus provided in this application embodiment acquire an initial SAR typhoon image to be reconstructed; preprocess the initial SAR typhoon image to determine the target typhoon region in the initial SAR typhoon image, obtaining a typhoon image to be reconstructed; use pre-trained local branch networks and global branch networks to extract features from the typhoon image to be reconstructed, respectively, to obtain an aggregated typhoon image; wherein the aggregated typhoon image includes local features and global features extracted from the typhoon image to be reconstructed; perform feature aggregation processing on the aggregated typhoon image through a pre-constructed aggregation network to obtain an aggregated typhoon image; adjust the size of the aggregated typhoon image according to the size of the initial SAR typhoon image to obtain a reconstructed typhoon image. In this way, typhoon data within a portion of the target typhoon region in a typhoon image can be reconstructed to ensure the quality of typhoon data used for predicting sea surface wind speed, thereby indirectly improving the accuracy of the prediction results.

[0070] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0071] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0072] Figure 1 A flowchart illustrating a method for reconstructing synthetic aperture radar typhoon data provided in this application embodiment;

[0073] Figure 2 This is a schematic diagram of a CoT Block layer structure provided in an embodiment of this application;

[0074] Figure 3 This is a schematic diagram of a typhoon data reconstruction process provided in an embodiment of this application;

[0075] Figure 4 This is one of the structural schematic diagrams of a synthetic aperture radar typhoon data reconstruction device provided in the embodiments of this application;

[0076] Figure 5 A second schematic diagram of a synthetic aperture radar typhoon data reconstruction device provided in this application embodiment;

[0077] Figure 6This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0079] Research has revealed that Synthetic Aperture Radar (SAR), an active Earth observation system, can capture the sea surface wind field of typhoons, enabling detailed monitoring of typhoon structures. However, the extreme wind speeds generated by typhoons can cause saturation of the C-band SAR signal, affecting the data quality of the SAR images acquired by the system, resulting in lower-quality SAR images.

[0080] Based on this, this application provides a method for reconstructing synthetic aperture radar typhoon data. By reconstructing typhoon data that does not meet the conditions in the initial typhoon image, the quality of typhoon data in the reconstructed typhoon image used for wind speed prediction is guaranteed, which helps to improve the data quality of typhoon data in the reconstructed typhoon image and thus indirectly improves the accuracy of wind speed prediction results.

[0081] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for reconstructing synthetic aperture radar typhoon data provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the method for reconstructing synthetic aperture radar typhoon data includes:

[0082] S101. Acquire the initial Synthetic Aperture Radar (SAR) typhoon image to be reconstructed.

[0083] S102. Preprocess the initial SAR typhoon image to determine the target typhoon region in the initial SAR typhoon image and obtain the typhoon image to be reconstructed.

[0084] S103. Using pre-trained local branch networks and global branch networks, feature extraction is performed on the typhoon image to be reconstructed to obtain the typhoon image to be aggregated.

[0085] S104. The typhoon image to be aggregated is processed by feature aggregation through a pre-constructed aggregation network to obtain an aggregated typhoon image.

[0086] S105. Based on the size of the initial SAR typhoon image, adjust the size of the aggregated typhoon image to obtain the reconstructed typhoon image.

[0087] This application provides a method for reconstructing synthetic aperture radar (SAR) typhoon data. The method involves acquiring an initial SAR typhoon image to be reconstructed; preprocessing the initial SAR typhoon image to identify those whose typhoon data does not meet data quality control requirements and needs to undergo target typhoon region feature extraction; using pre-trained local and global branch networks to extract features from the typhoon image to be reconstructed, resulting in an aggregated typhoon image; and then aggregating the typhoon data obtained from different reconstruction methods in the aggregated typhoon image using an aggregation network to obtain an aggregated typhoon image. Here, during image semantic segmentation, the aggregation network obtains some image features through convolutional layers, but the decoder needs to restore these features to the original image size before it can reconstruct each pixel of the original image. This method enables the reconstruction of typhoon data within the target typhoon region from the initial typhoon image with poor data quality, resulting in a reconstructed typhoon image with high accuracy.

[0088] In step S101, typhoon images acquired by Synthetic Aperture Radar (SAR) are obtained. Since the quality of typhoon images acquired by SAR is uncertain—that is, during the acquisition of typhoon images, SAR may be affected by extreme conditions (e.g., typhoons, heavy rainfall, etc.) leading to SAR signal oversaturation—resulting in inaccurate typhoon data for some typhoon areas in the acquired typhoon images. Therefore, further reconstruction processing of the typhoon images acquired by SAR is required. At this point, the typhoon images acquired by SAR can be determined as the initial SAR typhoon images.

[0089] Here, for the initial SAR typhoon image, only a portion of the typhoon data in the initial SAR typhoon image will be affected, resulting in inaccuracy and low quality. Therefore, when reconstructing the data, it is only necessary to reconstruct the low-quality typhoon data.

[0090] In step S102, the initial SAR typhoon image is preprocessed to determine the target typhoon area in the initial typhoon image that needs to be reconstructed, and the target typhoon area is marked to obtain the typhoon image to be reconstructed.

[0091] Here, the target typhoon area is the area in the initial typhoon image where there is typhoon data that does not meet the preset conditions; the typhoon data that does not meet the data quality control can refer to inaccurate or poor-quality typhoon data, and the judgment of inaccuracy and poor quality can be based on the data accuracy.

[0092] In one embodiment, step S102 includes: preprocessing the initial typhoon image, determining the target typhoon area in the initial SAR typhoon image whose data quality identifier is invalid based on the data quality identifier corresponding to each pixel position in the initial SAR typhoon image, and obtaining the typhoon image to be reconstructed.

[0093] In this step, each pixel location in the initial SAR typhoon image carries a data quality identifier. At this time, the target typhoon area with invalid data quality identifier can be determined from the initial SAR typhoon image based on the data quality identifier corresponding to each pixel location, thus obtaining the typhoon image to be reconstructed.

[0094] Here, the data quality identifiers include 0, 1, 2, and 3; where 0 is the valid identifier, indicating that the data at this pixel location is valid typhoon data; 1 is the land identifier, indicating that the data at this pixel location is land data; 2 is the sea ice identifier, indicating that the data at this pixel location is sea ice data; and 3 is the invalid identifier, indicating that the data at this pixel location is invalid typhoon data.

[0095] It should be noted that determining the target typhoon area in the initial SAR typhoon image based on the data quality identifier carried in the initial SAR typhoon image is only one implementation method. If there is no data quality identifier in the initial SAR typhoon image, the target typhoon area can also be determined by the following methods.

[0096] In step S103, features are extracted from the typhoon image to be reconstructed using pre-trained local branch networks and global branch networks, respectively. After feature extraction, the first reconstructed typhoon image obtained by the local branch network and the second reconstructed typhoon image obtained by the global branch network are integrated to obtain the typhoon image to be aggregated, which needs to be processed by the aggregation network for feature aggregation.

[0097] The typhoon image to be aggregated includes local and global features extracted from the typhoon image to be reconstructed.

[0098] In one implementation, step S103 includes:

[0099] S1031. Using multiple convolutional layers in the local branch network, the local features between local pixels in the typhoon image to be reconstructed are extracted by performing multiple first convolutional processes on the typhoon image to be reconstructed, and the typhoon data in the target typhoon area is reconstructed to obtain the first reconstructed typhoon image.

[0100] In this step, the local branch network includes multiple convolutional layers capable of performing the first convolutional processing on the typhoon image to be reconstructed. The typhoon image to be reconstructed is input into the local branch network, and the first convolutional processing is performed multiple times on the typhoon image to be reconstructed through the multiple convolutional layers in the local branch network. Then, through multiple convolutional operations, the local features between each local pixel in the typhoon image to be reconstructed are extracted, and the typhoon data in the target typhoon region in the typhoon image to be reconstructed is reconstructed to obtain the first reconstructed typhoon image.

[0101] For each target typhoon region in the first reconstructed typhoon image, during the first convolution process, the typhoon data in the target typhoon region needs to be reconstructed by combining the image pixel values ​​of the surrounding and neighboring areas.

[0102] In one implementation, step S1031 includes: using multiple convolutional layers in the local branch network, referencing typhoon data in a first reference region surrounding the target typhoon region, and performing multiple first convolutional processes on the typhoon image to be reconstructed to reconstruct the typhoon data within the target typhoon region, thereby obtaining a first reconstructed typhoon image.

[0103] In this step, for each convolutional layer in the local branch network, when performing the first convolutional processing on the typhoon image to be reconstructed using the convolutional layer, the typhoon data in the first reference area located around the target typhoon area is referenced to obtain the typhoon data in the target typhoon area. The first convolutional processing is performed one by one in each convolutional layer to reconstruct the typhoon data in each target typhoon area and obtain the first reconstructed typhoon image.

[0104] In one implementation, a local branch network is trained through the following steps:

[0105] Obtain a network training set for training the local branch network, wherein the network training set includes multiple training network images and network label images for each training network image; input the multiple pre-processed training network images in the network training set as input features, and use the network label images of each training network image as output features, stack them into small batches and input them into the pre-constructed local branch network to train the pre-constructed local branch network, thereby obtaining the trained local branch network.

[0106] S1032. Using multiple attention mechanism convolutional layers (CoT Block layers) in the global branch network, perform multiple second convolutional processes on the typhoon image to be reconstructed, extract global features between global pixels in the typhoon image to be reconstructed, reconstruct typhoon data in the target typhoon area, and obtain the second reconstructed typhoon image.

[0107] In this step, the global branch network includes multiple attention mechanism convolutional layers (CoT Block layers) capable of performing a second convolutional process on the typhoon image to be reconstructed. The typhoon image to be reconstructed is input into the global branch network, and through the multiple attention mechanism convolutional layers in the global branch network, the typhoon image to be reconstructed undergoes multiple second convolutional processes. Then, through multiple convolutional operations, global features between global pixels in the typhoon image to be reconstructed are extracted, and typhoon data in the target typhoon region in the typhoon image to be reconstructed is reconstructed to obtain the second reconstructed typhoon image.

[0108] For each target typhoon region in the second reconstructed typhoon image, during the second convolution process, it is necessary to combine the second reference region in the typhoon image to be reconstructed, which has a high degree of correlation with the target typhoon region, to reconstruct the typhoon data in the target typhoon region.

[0109] In one implementation, step S1032 includes: using multiple CoT Block layers in the global branch network, referencing typhoon data in a second reference region associated with the target typhoon region, performing multiple second convolution processes on the typhoon image to be reconstructed, extracting global features between global pixels in the typhoon image to be reconstructed, reconstructing typhoon data within the target typhoon region, and obtaining a second reconstructed typhoon image.

[0110] In this step, for each CoT Block layer in the global branch network, when performing the second convolution processing on the typhoon image to be reconstructed using the CoT Block layer, the typhoon data in the second reference area of ​​the image to be reconstructed with a correlation degree greater than a preset correlation degree threshold is referenced to obtain the typhoon data in the target typhoon area through convolution. The second convolution processing is performed one by one in each CoT Block layer to reconstruct the typhoon data in each target typhoon area and obtain the second reconstructed typhoon image.

[0111] Wherein, the correlation between the second reference area and the target typhoon area is greater than a preset correlation threshold.

[0112] Here, the multiple CoT Block layers included in the global branch network can not only extract global features from the input typhoon image to be reconstructed, but also perform multiple downsampling processes on the typhoon image to be reconstructed, so as to reduce the amount of data processing in subsequent processes.

[0113] Please see Figure 2 , Figure 2 This is a schematic diagram of a CoT Block layer structure provided in an embodiment of this application. Figure 2 As shown, the CoT Block layer 2a includes a first convolutional sub-layer 2b, an attention mechanism (COT) layer 2c, a second convolutional layer 2d, and an ECA module 2e.

[0114] In one implementation, the CoT Block layer includes a first convolutional sub-layer, a second convolutional sub-layer, an ECA module, and an attention mechanism (COT) layer; the process of utilizing multiple CoT Block layers in the global branch network, referencing typhoon data in a second reference region associated with the target typhoon region, and performing multiple second convolutional processes on the typhoon image to be reconstructed to extract global features between global pixels in the typhoon image to be reconstructed, reconstructing the typhoon data within the target typhoon region, and obtaining a second reconstructed typhoon image includes:

[0115] Step 1: For each CoT Block layer, using the first convolutional sub-layer in the CoT Block layer, and referencing typhoon data in the second reference region associated with the target typhoon region, perform a fourth convolutional process on the typhoon image to be reconstructed to obtain a first feature map.

[0116] In this step, for each CoT Block layer, the first convolutional sub-layer in the CoT Block layer is used to refer to the typhoon data in the second reference area associated with the target typhoon area to perform a fourth convolutional process on the typhoon image to be reconstructed, so as to obtain the first feature map; wherein, the first convolutional sub-layer is a 1×1 convolutional layer.

[0117] Step 2: Calculate the similarity of the first feature map using the COT layer in the CoT Block layer to obtain the second feature map.

[0118] In this step, the self-similarity matrix of the first feature map is calculated through the COT layer in the CoT Block layer, and then the self-similarity matrix is ​​multiplied with the first feature map to obtain the second feature map calculated by the self-attention mechanism.

[0119] Step 3: Using the second convolutional sub-layer in the CoT Block layer, and referencing typhoon data in the second reference region associated with the target typhoon region, perform a fifth convolution on the second feature map to obtain the third feature map.

[0120] In this step, the second convolutional sub-layer in the CoT Block layer is used again to refer to the typhoon data in the second reference area associated with the target typhoon area to perform a fifth convolution on the second feature map to obtain the third feature map; wherein, the second convolutional sub-layer is a 1×1 convolutional layer.

[0121] Here, the Efficient Channel Attention (ECA) module is a local cross-channel interaction strategy that does not reduce dimensionality, effectively avoiding the impact of dimensionality reduction on the learning effect of channel attention. It is used to enhance the channel features of the input feature map (e.g., the third feature map).

[0122] Step 4: Using the ECA module in the CoT Block layer, capture the channel information of different pixel channels of the third feature map to obtain the second reconstructed typhoon image.

[0123] In this step, the ECA module in the CoT Block layer is used to capture the channel information of different pixel channels in the third feature map in order to extract global features containing more information, and thus obtain a second reconstructed typhoon image containing global features.

[0124] In one implementation, the global branch network is trained through the following steps:

[0125] Obtain a network training set for training the global branch network, wherein the network training set includes multiple training network images and network label images for each training network image; use the multiple preprocessed training network images in the network training set as input features and the network label images for each training network image as output features, stack them into small batches and input them into the pre-constructed global branch network to train the pre-constructed global branch network, thereby obtaining the trained global branch network.

[0126] S1033. Perform a third convolution process on the first reconstructed typhoon image and the second reconstructed typhoon image to obtain the typhoon image to be aggregated.

[0127] In this step, after obtaining the first reconstructed typhoon image and the second reconstructed typhoon image, the first reconstructed typhoon image and the second reconstructed typhoon image are integrated to obtain the typhoon image to be aggregated for feature aggregation processing.

[0128] Here, since the original size of the typhoon image to be reconstructed is large, if the original size of the typhoon image to be reconstructed is reconstructed, the large amount of data processing will put a lot of processing pressure on the local branch network and the global branch network, resulting in low processing efficiency. Therefore, in order to avoid putting data processing pressure on the local branch network and the global branch network, the typhoon image to be reconstructed can be downsampled before reconstructing the typhoon data of the target typhoon area, and then the downsampled typhoon image to be reconstructed can be reconstructed.

[0129] In step S104, a pre-built aggregation network is used to perform feature aggregation processing on the features from the first reconstructed typhoon image and the second reconstructed typhoon image in the typhoon image to be aggregated, so as to obtain the typhoon image to be aggregated.

[0130] In one embodiment, the aggregation network includes at least one first sub-aggregation layer and at least one second sub-aggregation layer; step S104 includes:

[0131] S1041. Using the at least one first sub-aggregation layer, perform a first aggregation process on the typhoon image to be aggregated to obtain a first sub-typhoon feature map.

[0132] Here, the aggregation network can specifically include 5 aggregation layers, of which 4 can be the first sub-aggregation layer and 1 can be the second sub-aggregation layer (this is just an example and can be adjusted according to the actual situation).

[0133] In this step, the typhoon images to be aggregated are input into at least one first sub-aggregation layer. In each first sub-aggregation layer, the typhoon images to be aggregated are subjected to a first aggregation process. Then, the typhoon images to be aggregated after the first aggregation process output by each first sub-aggregation layer are aggregated again through a convolution operation to obtain the first sub-typhoon feature map.

[0134] Here, before aggregating the first aggregated typhoon images output by each first sub-aggregation layer through convolution operations, it is necessary to calculate the first aggregated typhoon images output by each first sub-aggregation layer through the activation function (RELU). The main function of the activation function is to perform nonlinear transformation on the first aggregated typhoon images to solve the problem of insufficient expression and classification ability of the linear model (e.g., the first sub-aggregation layer).

[0135] S1042. Using the at least one second sub-aggregation layer, perform a second aggregation process on the typhoon image to be aggregated to obtain a second sub-typhoon feature map.

[0136] In this step, the typhoon image to be aggregated is input into the second sub-aggregation layer. In each second sub-aggregation layer, the typhoon image to be aggregated is subjected to a second aggregation process to obtain the second sub-typhoon feature map.

[0137] S1043. Based on the first sub-typhoon feature map and the second sub-typhoon feature map, an aggregated typhoon image is obtained.

[0138] In this step, the first and second sub-typhoon feature maps are processed using the following formula to obtain the aggregated typhoon image.

[0139] x' = x2 × β + x1 × (1 - β);

[0140] Where x' is the aggregated typhoon image, x1 is the first sub-typhoon feature map, x2 is the second sub-typhoon feature map, and β is an adaptive threshold used to control the weights assigned to the first and second sub-aggregation layers by the aggregation context module, so that more information valuable for reconstruction is output by the aggregation layer and some redundant information is discarded.

[0141] Here, in order to make the sub-aggregation layer suitable for aggregating typhoon images, the dilation rate in the dilated convolution of each sub-aggregation layer is reduced before using each sub-aggregation layer. Each sub-aggregation layer uses four dilated convolution rates. The typhoon images to be aggregated are copied, transformed, and stitched together in each sub-aggregation layer to obtain the output typhoon images of each sub-aggregation layer.

[0142] In step S105, in order to avoid changing the size of the typhoon image during the data reconstruction process, so that the size of the reconstructed typhoon image does not match the size of the initial typhoon image, it is also necessary to adjust the size of the aggregated typhoon image with reference to the size of the initial typhoon image (for example, if the size of the aggregated typhoon image is small, the aggregated typhoon image can be upsampled) to obtain the reconstructed typhoon image.

[0143] In the process of upsampling aggregated typhoon images, in addition to restoring the size of the aggregated typhoon image through the upsampling layer, convolutional layers and bilinear interpolation layers can also be set to assist in upsampling the aggregated typhoon image in order to better restore the aggregated typhoon image to the size of the initial typhoon image.

[0144] Here, in order to improve the image accuracy and clarity of the reconstructed typhoon image, a pre-trained image refinement model is used after obtaining the reconstructed typhoon image to improve the image accuracy of the output image.

[0145] In one embodiment, the reconstruction method further includes: inputting the reconstructed typhoon image into a pre-trained image refinement model, and generating a target typhoon image corresponding to the reconstructed typhoon image through the image refinement model.

[0146] In this step, the reconstructed typhoon image is input into a pre-trained image refinement model. The image refinement model can refer to the reconstructed typhoon image to generate a target typhoon image corresponding to the reconstructed typhoon image. The image accuracy of the obtained target typhoon image is higher than that of the reconstructed typhoon image.

[0147] In one implementation, the image refinement model is trained through the following steps:

[0148] Step a: Obtain the training image set and the test image set for training the image refinement model.

[0149] The training image set includes multiple training typhoon images and training label images for each training typhoon image; the test image set includes multiple test typhoon images and test label images for each test typhoon image.

[0150] Here, we can train an image refinement model by constructing an adversarial generative network. The adversarial generative network includes an image generation network and an image discrimination network. The image generation network is used to generate images with higher image accuracy, while the image discrimination network is used to judge the realism of the images generated by the image generation network.

[0151] Step b: Using multiple training typhoon images from the training image set as input features and the training label image of each training typhoon image as output features, train the pre-constructed image generation network to obtain a preliminary generation network.

[0152] Step c: Using the test image set, determine the total loss value of the loss value group of the preliminary generation network through the pre-constructed image discrimination network.

[0153] In this step, multiple test typhoon images from the test image set are input into the preliminary generation network, which generates a reference typhoon image corresponding to each test typhoon image. Then, the reference typhoon image and the test label image of each test typhoon image are input into the image discrimination network, which judges the authenticity of the reference typhoon image and the test label image to determine the total loss value of the loss value group of the preliminary generation network.

[0154] The loss set includes reconstruction loss, perceptual loss, style loss, and adversarial loss. Reconstruction loss minimizes the absolute error between the generated image and the real image. Perceptual loss extracts relevant information from the image using a deep convolutional neural network (CNN), extracting both structural and textural features. Perceptual loss calculates the structural and textural information of the predicted and labeled values ​​and calculates the error, further reducing the absolute error between the generated image and the real image. Style loss uses a Gram matrix to capture the image's structural information, assisting in optimizing the reconstruction loss. Adversarial loss adds a discriminator to determine whether the image originates from the model or from the real world, causing the probability distribution of the predicted image to converge towards the probability distribution of the real image, thus solving the problem of blurred predicted images caused by reconstruction loss.

[0155] Step d: If the total loss value is greater than the preset loss threshold, adjust the network parameters of the initially generated network with reference to the total loss value until the total loss value is less than or equal to the preset loss threshold to obtain the image refinement model.

[0156] Please see Figure 3 , Figure 3 This is a schematic diagram illustrating a typhoon data reconstruction process provided in an embodiment of this application. Figure 3 As shown, the typhoon image 3a to be reconstructed, obtained after preprocessing the initial SAR typhoon image, is input into the local branch network 3b and the global branch network 3c, respectively. Through multiple convolutional layers 3b-1 included in the local branch network 3b, the typhoon image 3a to be reconstructed undergoes multiple first convolutional processes to reconstruct the typhoon data within the target typhoon region in the typhoon image 3a, resulting in the first reconstructed typhoon image y1. Through multiple attention mechanism convolutional layers 3c-1 included in the global branch network 3c, the typhoon image 3a to be reconstructed undergoes multiple second convolutional processes to reconstruct the typhoon data within the target typhoon region in the typhoon image 3a, resulting in the second reconstructed typhoon image y2. Then, the additionally set convolutional network 3d performs a third convolutional process on the first reconstructed typhoon image y1 and the second reconstructed typhoon image y2 to obtain the typhoon image x to be aggregated.

[0157] The typhoon image x to be aggregated is input into the aggregation network 3e, which includes at least one first sub-aggregation layer 3e-1 and at least one second sub-aggregation layer 3e-2. In each first sub-aggregation layer 3e-1, the typhoon image x to be aggregated is subjected to a first aggregation process, and the typhoon images to be aggregated after the first aggregation process output by each first sub-aggregation layer are aggregated again through a convolution operation to obtain a first sub-typhoon feature map x1. In at least one second sub-aggregation layer 3e-2, the typhoon image x to be aggregated is subjected to a second aggregation process to obtain a second sub-typhoon feature map x2. Based on the first sub-typhoon feature map x1 and the second sub-typhoon feature map x2, the aggregated typhoon image x' is obtained. The size of the aggregated typhoon image x' is adjusted by upsampling with reference to the initial SAR typhoon image to obtain a reconstructed typhoon image x''. The reconstructed typhoon image x'' is input into the image refinement model 3f to obtain a high-precision target typhoon image 3g.

[0158] The synthetic aperture radar (SAR) typhoon data reconstruction method provided in this application involves: acquiring an initial SAR typhoon image to be reconstructed; preprocessing the initial SAR typhoon image to determine the target typhoon region within it, thus obtaining a typhoon image to be reconstructed; using pre-trained local and global branch networks to extract features from the typhoon image to be reconstructed, resulting in an aggregated typhoon image; wherein the aggregated typhoon image includes local and global features extracted from the initial SAR typhoon image; performing feature aggregation processing on the aggregated typhoon image using a pre-constructed aggregation network, resulting in an aggregated typhoon image; and adjusting the size of the aggregated typhoon image according to the size of the initial SAR typhoon image to obtain a reconstructed typhoon image. This method allows for the reconstruction of typhoon data within a portion of the target typhoon region in a typhoon image, ensuring the quality of typhoon data used for predicting sea surface wind speeds, thereby indirectly improving the accuracy of prediction results.

[0159] Please see Figure 4 , Figure 5 , Figure 4 This is one of the structural schematic diagrams of a synthetic aperture radar typhoon data reconstruction device provided in the embodiments of this application. Figure 5 This is a second schematic diagram of a synthetic aperture radar typhoon data reconstruction device provided in an embodiment of this application. Figure 4 As shown, the reconstruction device 400 includes:

[0160] Image acquisition module 410 is used to acquire the initial synthetic aperture radar (SAR) typhoon image to be reconstructed;

[0161] The region determination module 420 is used to preprocess the initial SAR typhoon image, determine the target typhoon region in the initial SAR typhoon image, and obtain the typhoon image to be reconstructed.

[0162] The feature extraction module 430 is used to extract features from the typhoon image to be reconstructed using pre-trained local branch networks and global branch networks to obtain the typhoon image to be aggregated; wherein, the typhoon image to be aggregated includes local features and global features extracted from the typhoon image to be reconstructed.

[0163] The aggregation module 440 is used to perform feature aggregation processing on the typhoon image to be aggregated through a pre-built aggregation network to obtain an aggregated typhoon image.

[0164] The size adjustment module 450 is used to adjust the size of the aggregated typhoon image according to the size of the initial SAR typhoon image, so as to obtain the reconstructed typhoon image.

[0165] Furthermore, when the feature extraction module 430 uses pre-trained local branch networks and global branch networks to extract features from the typhoon image to be reconstructed to obtain the typhoon image to be aggregated, the feature extraction module 430 is used to:

[0166] By utilizing multiple convolutional layers in the local branch network, the local features between local pixels in the typhoon image to be reconstructed are extracted by performing multiple first convolutional processes on the typhoon image to be reconstructed, and the typhoon data in the target typhoon area is reconstructed to obtain the first reconstructed typhoon image.

[0167] By utilizing multiple attention mechanism convolutional layers (CoT Block layers) in the global branch network, the typhoon image to be reconstructed is subjected to multiple second convolutional processes to extract global features between global pixels in the typhoon image to be reconstructed, and the typhoon data in the target typhoon area is reconstructed to obtain the second reconstructed typhoon image.

[0168] A third convolution process is performed on the first reconstructed typhoon image and the second reconstructed typhoon image to fuse the local features in the first reconstructed typhoon image and the global features in the second reconstructed typhoon image to obtain the typhoon image to be aggregated.

[0169] Furthermore, when the feature extraction module 430 utilizes multiple convolutional layers in the local branch network to perform multiple first convolutional processes on the typhoon image to be reconstructed, extracting local features between local pixels in the typhoon image to be reconstructed, and reconstructing typhoon data within the target typhoon region to obtain the first reconstructed typhoon image, the feature extraction module 430 is used to:

[0170] By utilizing multiple convolutional layers in the local branch network and referencing typhoon data within a first reference region surrounding the target typhoon region, the local features between local pixels in the typhoon image to be reconstructed are extracted by performing multiple first convolutional processes on the typhoon image to be reconstructed, and the typhoon data within the target typhoon region is reconstructed to obtain the first reconstructed typhoon image.

[0171] Furthermore, when the feature extraction module 430 performs multiple second convolution processes on the typhoon image to be reconstructed using multiple attention mechanism convolutional layers (CoT Block layers) in the global branch network to extract global features between global pixels in the typhoon image to be reconstructed, and reconstructs the typhoon data in the target typhoon area to obtain the second reconstructed typhoon image, the feature extraction module 430 is used to:

[0172] Using multiple CoT Block layers in the global branch network, and referencing typhoon data in a second reference region associated with the target typhoon region, the global features between global pixels in the typhoon image to be reconstructed are extracted by performing multiple second convolution processes on the typhoon image to be reconstructed, and the typhoon data in the target typhoon region is reconstructed to obtain a second reconstructed typhoon image; wherein, the correlation between the second reference region and the target typhoon region is greater than a preset correlation threshold.

[0173] Furthermore, the CoT Block layer includes a first convolutional sub-layer, a second convolutional sub-layer, an ECA module, and an attention mechanism (COT) layer; when the feature extraction module 430 utilizes multiple CoT Block layers in the global branch network, referencing typhoon data in a second reference region associated with the target typhoon region, to extract global features between global pixels in the typhoon image to be reconstructed by performing multiple second convolutional processes on the typhoon image to be reconstructed, and reconstructs the typhoon data within the target typhoon region to obtain a second reconstructed typhoon image, the feature extraction module 430 is used to:

[0174] For each CoT Block layer, the first convolutional sub-layer in the CoT Block layer is used to refer to the typhoon data in the second reference area associated with the target typhoon area to perform a fourth convolutional process on the typhoon image to be reconstructed, so as to obtain the first feature map.

[0175] The first feature map is used to calculate the similarity of the CoT Block layer to obtain the second feature map.

[0176] Using the second convolutional sublayer in the CoT Block layer, and referencing typhoon data in a second reference region associated with the target typhoon region, the second feature map is subjected to a fifth convolutional process to obtain the third feature map;

[0177] The ECA module in the CoT Block layer is used to capture the channel information of different pixel channels of the third feature map to obtain the second reconstructed typhoon image.

[0178] Furthermore, the aggregation network includes at least one first sub-aggregation layer and at least one second sub-aggregation layer; when the aggregation module 440 performs feature aggregation processing on the typhoon image to be aggregated through the pre-constructed aggregation network to obtain an aggregated typhoon image, the aggregation module 440 is used to:

[0179] Using the at least one first sub-aggregation layer, the typhoon image to be aggregated is subjected to a first aggregation process to obtain a first sub-typhoon feature map;

[0180] Using the at least one second sub-aggregation layer, the typhoon image to be aggregated is subjected to a second aggregation process to obtain a second sub-typhoon feature map;

[0181] Based on the feature maps of the first and second sub-typhoons, an aggregated typhoon image is obtained.

[0182] Furthermore, when the region determination module 420 preprocesses the initial SAR typhoon image to determine the target typhoon region in the initial SAR typhoon image and obtains the typhoon image to be reconstructed, the region determination module 420 is used to:

[0183] The initial SAR typhoon image is preprocessed, and the target typhoon areas with invalid data quality identifiers in the initial SAR typhoon image are determined according to the data quality identifiers corresponding to each pixel position in the initial SAR typhoon image, so as to obtain the typhoon image to be reconstructed.

[0184] Furthermore, such as Figure 5 As shown, the reconstruction device 400 further includes an image generation module 460, which is used for:

[0185] The reconstructed typhoon image is input into a pre-trained image refinement model, and a target typhoon image corresponding to the reconstructed typhoon image is generated through the image refinement model; wherein, the image precision of the target typhoon image is higher than that of the reconstructed typhoon image.

[0186] Furthermore, such as Figure 5As shown, the reconstruction device 400 further includes a model training module 470, which is used to train the image refinement model through the following steps:

[0187] Obtain a training image set and a test image set for training the image refinement model; wherein, the training image set includes multiple training typhoon images and a training label image for each training typhoon image; the test image set includes multiple test typhoon images and a test label image for each test typhoon image.

[0188] Multiple training typhoon images from the training image set are used as input features, and the training label image of each training typhoon image is used as output features to train the pre-constructed image generation network, thus obtaining a preliminary generation network.

[0189] Using the test image set, the total loss value of the loss value group of the preliminary generation network is determined through a pre-constructed image discrimination network; wherein, the loss value group includes reconstruction loss value, perceptual loss value, style loss value, and adversarial loss value;

[0190] If the total loss value is greater than the preset loss threshold, the network parameters of the initially generated network are adjusted with reference to the total loss value until the total loss value is less than or equal to the preset loss threshold, thereby obtaining the image refinement model.

[0191] The synthetic aperture radar (SAR) typhoon data reconstruction apparatus provided in this application acquires an initial SAR typhoon image to be reconstructed; preprocesses the initial SAR typhoon image to determine the target typhoon region within it, obtaining a typhoon image to be reconstructed; uses pre-trained local and global branch networks to extract features from the typhoon image to be reconstructed, obtaining an aggregated typhoon image; wherein the aggregated typhoon image includes local and global features extracted from the typhoon image to be reconstructed; performs feature aggregation processing on the aggregated typhoon image using a pre-constructed aggregation network, obtaining an aggregated typhoon image; adjusts the size of the aggregated typhoon image according to the size of the initial SAR typhoon image, obtaining a reconstructed typhoon image. In this way, typhoon data within a portion of the target typhoon region in a typhoon image can be reconstructed, ensuring the quality of typhoon data used for predicting sea surface wind speed, thereby indirectly improving the accuracy of prediction results.

[0192] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 600 includes a processor 610, a memory 620, and a bus 630.

[0193] The memory 620 stores machine-readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 and the memory 620 communicate via the bus 630. When the machine-readable instructions are executed by the processor 610, they can perform the operations described above. Figure 1 The steps of the synthetic aperture radar typhoon data reconstruction method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0194] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the synthetic aperture radar typhoon data reconstruction method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0195] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0196] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0197] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0198] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0199] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0200] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of reconstructing synthetic aperture radar typhoon data, characterized by, The reconstruction method comprises: acquiring an initial Synthetic Aperture Radar (SAR) typhoon image to be reconstructed; preprocessing the initial SAR typhoon image to determine a target typhoon region in the initial SAR typhoon image, and obtaining a typhoon image to be reconstructed; extracting features from the typhoon image to be reconstructed by using a pre-trained local branch network and a global branch network, and obtaining a typhoon image to be aggregated; wherein the typhoon image to be aggregated comprises local features and global features extracted from the typhoon image to be reconstructed; performing feature aggregation processing on the typhoon image to be aggregated by using a pre-constructed aggregation network, and obtaining an aggregated typhoon image; adjusting the size of the aggregated typhoon image according to the size of the initial SAR typhoon image, and obtaining a reconstructed typhoon image after reconstruction is completed; extracting features from the typhoon image to be reconstructed by using a pre-trained local branch network and a global branch network, and obtaining a typhoon image to be aggregated, comprising: extracting local features between each local pixel in the typhoon image to be reconstructed by performing multiple first convolution processing on the typhoon image to be reconstructed by using multiple convolution layers in the local branch network, reconstructing typhoon data in the target typhoon region, and obtaining a first reconstructed typhoon image; extracting global features between global pixels in the typhoon image to be reconstructed by performing multiple second convolution processing on the typhoon image to be reconstructed by using multiple CoT Block layers in the global branch network, reconstructing typhoon data in the target typhoon region, and obtaining a second reconstructed typhoon image; performing third convolution processing on the first reconstructed typhoon image and the second reconstructed typhoon image, fusing the local features in the first reconstructed typhoon image and the global features in the second reconstructed typhoon image, and obtaining a typhoon image to be aggregated; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer; the CoT Block layer comprises a first convolution sub-layer, a second convolution sub-layer, an ECA module, and a COT layer ​ ​ The second feature map is subjected to fifth convolution processing by referring to the typhoon data in a second reference region associated with the target typhoon region, to obtain a third feature map. The ECA module in the CoT Block layer is used to capture channel information of different pixel channels of the third feature map, to obtain a second reconstructed typhoon image.

2. The reconstitution method of claim 1, wherein, The plurality of convolution layers in the local branch network are used to extract local features between each local pixel in the to-be-reconstructed typhoon image by performing multiple first convolution processes on the to-be-reconstructed typhoon image, to reconstruct the typhoon data in the target typhoon region, to obtain a first reconstructed typhoon image. The plurality of convolution layers in the local branch network are used to extract local features between each local pixel in the to-be-reconstructed typhoon image by performing multiple first convolution processes on the to-be-reconstructed typhoon image, to reconstruct the typhoon data in the target typhoon region, to obtain a first reconstructed typhoon image.

3. The reconstitution method of claim 1, wherein, The aggregation network includes at least one first sub-aggregation layer and at least one second sub-aggregation layer; the to-be-aggregated typhoon image is subjected to feature aggregation processing by the pre-constructed aggregation network, to obtain an aggregated typhoon image, including: The at least one first sub-aggregation layer is used to perform first aggregation processing on the to-be-aggregated typhoon image, to obtain a first sub-typhoon feature map; The at least one second sub-aggregation layer is used to perform second aggregation processing on the to-be-aggregated typhoon image, to obtain a second sub-typhoon feature map; The aggregated typhoon image is obtained based on the first sub-typhoon feature map and the second sub-typhoon feature map.

4. The reconstitution method of claim 1, wherein, The initial SAR typhoon image is preprocessed to determine the target typhoon region in the initial SAR typhoon image, to obtain a to-be-reconstructed typhoon image, including: The initial SAR typhoon image is preprocessed to determine the target typhoon region in the initial SAR typhoon image whose data quality identifier is an invalid identifier, to obtain a to-be-reconstructed typhoon image.

5. The reconstitution method of claim 1, wherein, The reconstruction method further includes: The reconstructed typhoon image is input into a pre-trained image refinement model, and the image refinement model is used to generate a target typhoon image corresponding to the reconstructed typhoon image; the image accuracy of the target typhoon image is lower than that of the reconstructed typhoon image.

6. The reconstitution method of claim 5, wherein, The image refinement model is trained by the following steps: A training image set and a test image set used for training the image refinement model are obtained; the training image set includes multiple training typhoon images and a training label image of each training typhoon image; the test image set includes multiple test typhoon images and a test label image of each test typhoon image; The multiple training typhoon images in the training image set are used as input features, and the training label image of each training typhoon image is used as an output feature, to train a pre-constructed image generation network, to obtain a preliminary generation network; The total loss value of the loss value group of the preliminary generation network is determined by using the test image set and a pre-constructed image discrimination network, wherein the loss value group comprises a reconstruction loss value, a perception loss value, a style loss value and an adversarial loss value; If the total loss value is greater than a preset loss threshold, the network parameters of the preliminary generation network are adjusted with reference to the total loss value until the total loss value is less than or equal to the preset loss threshold, thereby obtaining an image refinement model.

7. A reconstruction apparatus of synthetic aperture radar typhoon data, characterized by, The reconstruction device comprises: An image acquisition module is configured to acquire an initial synthetic aperture radar (SAR) typhoon image to be reconstructed; A region determination module is configured to preprocess the initial SAR typhoon image and determine a target typhoon region in the initial SAR typhoon image, thereby obtaining a typhoon image to be reconstructed; A feature extraction module is configured to use a pre-trained local branch network and a global branch network to respectively extract features from the typhoon image to be reconstructed, thereby obtaining a typhoon image to be aggregated; wherein the typhoon image to be aggregated comprises local features and global features extracted from the typhoon image to be reconstructed; An aggregation module is configured to use a pre-constructed aggregation network to perform feature aggregation on the typhoon image to be aggregated, thereby obtaining an aggregated typhoon image; A size adjustment module is configured to adjust the size of the aggregated typhoon image according to the size of the initial SAR typhoon image, thereby obtaining a reconstructed typhoon image; When the feature extraction module is configured to use a pre-trained local branch network and a global branch network to respectively extract features from the typhoon image to be reconstructed, the feature extraction module is configured to: Use multiple convolution layers in the local branch network to extract local features between each local pixel in the typhoon image to be reconstructed by performing multiple first convolution operations on the typhoon image to be reconstructed, thereby reconstructing typhoon data in the target typhoon region and obtaining a first reconstructed typhoon image; Use multiple attention mechanism convolution layers (CoT Block layers) in the global branch network to extract global features between global pixels in the typhoon image to be reconstructed by performing multiple second convolution operations on the typhoon image to be reconstructed, thereby reconstructing typhoon data in the target typhoon region and obtaining a second reconstructed typhoon image; Perform a third convolution operation on the first reconstructed typhoon image and the second reconstructed typhoon image, thereby fusing the local features in the first reconstructed typhoon image and the global features in the second reconstructed typhoon image, and obtaining a typhoon image to be aggregated; The CoT Block layer comprises a first convolutional sub-layer, a second convolutional sub-layer, an ECA module, and an attention mechanism (COT) layer; the feature extraction module is configured to perform multiple second convolutional processing on the to-be-reconstructed typhoon image by using multiple attention mechanism convolutional layers (CoT Block layers) in the global branch network, extract global features between global pixels in the to-be-reconstructed typhoon image, reconstruct typhoon data in the target typhoon region, and obtain a second reconstructed typhoon image. For each CoT Block layer, the first convolutional sub-layer in the CoT Block layer is configured to perform fourth convolutional processing on the to-be-reconstructed typhoon image by referring to typhoon data in a second reference region associated with the target typhoon region, and obtain a first feature map; wherein the correlation degree between the second reference region and the target typhoon region is greater than a preset correlation degree threshold; The COT layer in the CoT Block layer is configured to perform similarity calculation on the first feature map, and obtain a second feature map; The second convolutional sub-layer in the CoT Block layer is configured to perform fifth convolutional processing on the second feature map by referring to typhoon data in a second reference region associated with the target typhoon region, and obtain a third feature map; The ECA module in the CoT Block layer is configured to capture channel information of different pixel channels of the third feature map, and obtain a second reconstructed typhoon image.

Images