Mesoscale convection image segmentation method based on small target recognition network

Abstract

The present invention provides a mesoscale convective image segmentation method based on a small target recognition network, belonging to the field of image processing technology, the method comprising obtaining original cloud top brightness temperature data; preprocessing the original cloud top brightness temperature data and dividing it into a training set and a test set; constructing and initializing a small target recognition network based on feature alignment and redistribution; training the small target recognition network using the training set; obtaining a mesoscale convective image segmentation result based on the trained small target recognition network; after reaching a threshold of the number of training times, inputting the test set into the trained small target recognition network to determine whether the current small target recognition network meets the index requirements; inputting the test set into the small target recognition network that meets the index requirements to obtain the final mesoscale convective image segmentation. The present invention solves the problem that the traditional method is slow in identifying mesoscale convective systems MCSs and only targets local areas.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a mesoscale convection image segmentation method based on a small target recognition network. Background Art

[0002] Mesoscale convective systems (MCSs) are storm cells or storm groups formed by the interaction of multiple independent convective units composed of single updrafts and downdrafts. The main characteristics are that the length scale in one or more directions is greater than 100 kilometers and the duration is more than three hours. They are mainly concentrated in mid-latitudes, especially over tropical Africa, the Amazon River and the western Pacific warm pool. Most of the catastrophic weather such as extreme rainfall and storms in these areas are related to mesoscale convective systems MCSs. In order to study the climatological characteristics of mesoscale convective systems MCSs, it is necessary to be able to quickly and accurately identify mesoscale convective systems MCSs, especially in the early stages of mesoscale convective systems MCSs.

[0003] Traditional identification of mesoscale convective systems (MCSs) is mainly based on infrared satellite (IR) data or radar reflectivity data, using statistical learning and computer vision and other related technologies. Among them, the threshold method is widely used in the identification of mesoscale convective systems (MCSs). In this method, the temperature or radar reflectivity threshold must first be determined to distinguish between areas that meet the threshold criteria and areas that do not meet the threshold criteria in the infrared satellite data or radar data. Subsequently, the regional coverage threshold of the mesoscale convective system (MCSs) is determined, and the areas that meet the regional coverage threshold are classified as mesoscale convective systems (MCSs).

[0004] For radar reflectivity mosaic data, the three-dimensional storm automatic monitoring and storm cell identification SCIT named TITAN is effective in identifying small-scale supercell storms or isolated storm cells, but is not accurate enough in identifying mesoscale convective systems MCSs with more complex structures and morphologies. Subsequently, the improved algorithm based on storm cell identification SCIT can correctly identify single storms with reflection coefficients ≥40dBZ and ≥50dBZ, with an accuracy of 68% and 96%, respectively. In the existing technology, radar reflectivity data is used to identify mesoscale convective system MCSs in the United States using the threshold method, where the radar reflectivity threshold is set to 40 dBZ and the area threshold is set to 100km 2 Due to the limitation of a single characteristic parameter, the recognition effect for reflectivity coefficient ≥ 35dBZ is still poor. In addition, the recognition of mesoscale convective systems (MCSs) is also limited by the effective range of radar data.

[0005] With the continuous exploration of multi-dimensional characteristic parameters of convective systems, researchers have gradually adopted the method of multi-parameter joint extraction of system characteristics to study the identification of mesoscale convective systems MCSs. For example, the existing technology combines TRMM satellite data and LIS lightning data to identify and verify the subtropical severe convective system in South America. Houze uses geosynchronous satellite and ground radar data to identify MCSs globally based on three GPM radar data indicators: deep convective core (DCC), wide convective core (WCC) and deep wide convective core (DWC). The existing technology uses contour frequency maps to statistically analyze Kuband and NEXRAD radar reflectivity data in the United States during the warm season of 2014-2016 for MCSs identification. The existing technology combines cloud top infrared brightness temperature data and GridRad radar data, and uses the FLEXTRKR algorithm to identify MCSs east of the Rocky Mountains in the United States. The existing technology uses infrared satellite data to identify global MCSs, and the temperature thresholds are set to 233 K and 241 K, respectively. It can be seen that the threshold-based method is highly subjective, and the setting of the threshold will vary due to different atmospheric conditions. In addition, using this method requires more computing resources and time.

[0006] From the perspective of computer vision, the recognition of mesoscale convective systems (MCSs) can be viewed as an image semantic segmentation task. The infrared data of the local area and the corresponding mesoscale convective system (MCSs) labels obtained by the threshold method are visualized, as shown in Figure 2. Figure 1 As shown in the figure. Semantic segmentation, also known as pixel-level classification, is a basic task, which is to segment the region of interest (foreground object) in an image from the image itself (background). Since the beginning of the deep learning era, convolutional neural networks (CNNs) have been used for semantic segmentation. Among them, the introduction of Unet has greatly promoted the progress of deep learning technology in the field of image semantic segmentation. Unet is an encoder-decoder architecture model, in which the encoder is used to extract spatial features and the decoder restores these features to the shape of the original image. This architecture has achieved excellent segmentation results in the field of semantic segmentation. Subsequently, a large number of Unet-based networks such as Unet++, MalUnet, U2Net, etc. were proposed for semantic segmentation.

[0007] In the field of atmospheric science, there are also a large number of neural networks based on Unet for segmenting meteorological elements. For example, the prior art proposed a CloudU-Net model for day and night cloud image segmentation. This method replaces ordinary convolution with dilated convolution, so that as much contextual information as possible can be extracted while maintaining inference efficiency. Subsequently, the prior art optimized the CloudU-Net network structure and proposed the CloudU-Netv2 model. This method changes the upsampling in CloudU-Net to bilinear upsampling, and adds position and channel attention modules to improve the recognition ability of feature representation. The prior art proposed a new deep convolutional neural network MA-SegCloud for cloud image segmentation, which designs a multi-branch asymmetric convolution module to capture contextual information of a larger area and uses an attention mechanism to enhance the features of cloud images. The prior art built a deep convolutional neural network (MesCoSNet) based on the architecture of RetinaNet, which can better identify MCSs in specific areas over Russia. The prior art proposed a Dugs-UNet model and designed a Shape Stream module to extract edge feature information of MCSs to improve the segmentation accuracy of convection.

[0008] Deficiencies of existing technologies:

[0009] 1. Traditional convection methods are based on temperature and area thresholds; these methods have high accuracy in convection identification, but consume a lot of computing resources and have a slow convection identification speed.

[0010] 2. Deep learning models have made significant progress in the field of MCSs identification. However, existing deep learning models do not take into account the large differences in MCS domain coverage and the uneven spatial distribution of MCSs. Directly applying these models to the MCSs identification task may lead to problems such as insufficient MCSs identification and blurred edge detection. Summary of the invention

[0011] In view of the above-mentioned deficiencies in the prior art, the present invention provides a mesoscale convective image segmentation method based on a small target recognition network, which solves the problems of high computing resource consumption, slow recognition speed and small recognition area in the traditional method for identifying mesoscale convective systems (MCSs).

[0012] In order to achieve the above objectives, the technical solution adopted by the present invention is: a mesoscale convection image segmentation method based on a small target recognition network, comprising the following steps:

[0013] S1. Obtain original cloud top brightness temperature data;

[0014] S2, preprocessing the original cloud top brightness temperature data and dividing it into a training set and a test set;

[0015] S3, building a small target recognition network based on feature alignment and redistribution;

[0016] S4, using the training set to train the small target recognition network;

[0017] S5. Based on the trained small target recognition network, the mesoscale convection image segmentation result is obtained;

[0018] S6, after reaching the threshold of the number of training times in S4 and S5, the test set is input into the trained small target recognition network to determine whether the current small target recognition network meets the index requirements. If so, the parameters of the current small target recognition network are saved and the process goes to S7, otherwise, the process returns to S4;

[0019] S7. Input the test set into the small target recognition network that meets the index requirements to obtain the final mesoscale convection image segmentation.

[0020] The beneficial effects of the present invention are as follows: the present invention constructs a network model for accurately identifying convection in the atmospheric system, and uses the constructed mesoscale convective system MCSs identification data set as a training set for learning. First, the backbone network of UNet is used to extract hierarchical features, and then a feature alignment and redistribution module FADM is constructed to aggregate multi-scale information in the downsampling stage, aiming to improve the model's perception of small-scale MCSs, prevent small-scale features from being lost during the downsampling process of the model, and provide sufficient guidance for accurately identifying small-scale MCSs. In addition, in order to alleviate the decrease in recognition accuracy caused by the blurred boundaries of the mesoscale convective system MCSs, the present invention proposes an asymmetric feature recovery module AFRM to extract and retain the spatial edge features of convection from different spatial dimensions, and accurately identify the edge information of the mesoscale convective system MCSs.

[0021] Further, the S2 comprises the following steps:

[0022] S201, setting a temperature threshold of the mesoscale convective system MCSs, and marking the area with a temperature less than the temperature threshold from the original cloud top brightness temperature data;

[0023] S202, setting the regional coverage threshold of the mesoscale convective system MCSs, screening the areas marked in S201 from the regional coverage threshold, selecting areas with a coverage area greater than the regional coverage threshold, and marking the areas that meet both the temperature threshold and the regional coverage threshold as mesoscale convective system MCSs areas, wherein the mesoscale convective system MCSs area is marked as 1, and the non-mesoscale convective system MCSs area is marked as 0;

[0024] S203, converting the original cloud top brightness temperature data and the marked data obtained in S202 into an image format;

[0025] S204, according to the converted image format, fill the cloud top brightness temperature data and label data with 0;

[0026] S205, cropping the filled cloud top brightness temperature data and label data in the first dimension;

[0027] S206. Divide into a training set and a test set according to the cropping result, wherein both the training set and the test set include cloud top brightness temperature data and mesoscale convective system MCSs labels in image format.

[0028] The beneficial effect of the above further solution is that the present invention can construct the convection identification data set required by the model through the above design.

[0029] Furthermore, the S3 comprises the following steps:

[0030] S301, for the training set and the test set divided by the mesoscale convective system MCSs identification, the label data in step S204 corresponding to the original image is converted into a binary grayscale image;

[0031] S302, converting the converted binary grayscale image into a tensor format tensor, and normalizing the original image and all pixel values ​​in the label data using maximum and minimum normalization;

[0032] S303: construct a small target recognition network according to the normalization processing result of S302.

[0033] The beneficial effect of the above further scheme is that the present invention improves the stability of the model training process and accelerates the model convergence by preprocessing the data set.

[0034] Furthermore, the small target recognition network includes:

[0035] An encoder, used to extract hierarchical pyramid features of the image after the normalization process in S302, wherein the encoder includes four dual convolution blocks and downsampling;

[0036] Feature alignment and redistribution module FADM, used to align the hierarchical pyramid features and redistribute them to the decoder;

[0037] A decoder is used to decode and restore the pyramid features aligned at different levels, wherein the decoder includes a transposed convolution and an asymmetric feature recovery module AFRM, and the asymmetric feature recovery module AFRM is used to extract and retain the spatial edge features of convection from different spatial dimensions to identify the edge information of mesoscale convection systems MCSs.

[0038] Furthermore, the S4 comprises the following steps:

[0039] S401, input the images normalized by S302 into the encoder to extract hierarchical pyramid features, and output the feature map of each layer of the encoder ;

[0040] S402: feature map Input to the feature alignment and redistribution module FADM, and perform alignment processing to obtain the redistributed output ,Among them, the feature alignment and redistribution module FADM includes four parts:,feature alignment, multi-scale feature extraction, global attention and feature,redistribution;

[0041] S403, in the decoder part, the feature map output by the fourth layer of the encoder Or the feature map output by the previous layer of the decoder Input to the transposed convolution, we get ,in, represents the output of the transposed convolution in the i-th layer decoder;

[0042] S404, will and Input into the asymmetric feature recovery module AFRM to extract and retain the spatial edge features of convection from different spatial dimensions, identify the edge information of the mesoscale convection system MCSs, and obtain the output of the i-th layer decoder ,in, represents the output of the i-th layer decoder;

[0043] S405, The result of mesoscale convection image segmentation is obtained by inputting it into the segmentation head SegHead, thus completing the training of small target recognition network.

[0044] Furthermore, the S402 includes the following steps:

[0045] S4021. In the feature alignment part, each layer feature map of the input encoder , using the feature scale of one of the layers as a standard to align the feature information of different layers except the one of the layers, using the bilinear interpolation method to upsample the low-resolution layers except the one of the layers to the same resolution as the one of the layers, and using the adaptive average pooling method to downsample the high-resolution layers except the one of the layers twice, so that the features of the high-resolution layers are consistent with those of the one of the layers, and a feature map with uniform spatial scale is obtained;

[0046] S4022, connecting each feature map in series in the channel dimension, and performing feature fusion by using convolution to obtain a fusion result;

[0047] S4023. In the multi-scale feature extraction part, the multi-scale spatial information is extracted using convolution kernels of different sizes for the fusion result, and the multi-scale spatial information is spliced ​​in the channel dimension and then compressed to obtain the compressed result. ;

[0048] S4024, in the global attention part, performing a convolution operation on the fusion result to obtain a global feature map, and activating the global feature map to obtain a global weight after feature alignment;

[0049] S4025, the aligned global weight and compression result Multiply them together to get the overall output ,in, Represents the overall output;

[0050] S4026. In the feature redistribution section, Split the channel dimension to get the redistributed output .

[0051] The beneficial effects of the above further scheme are: the present invention can effectively avoid information loss and improve the segmentation accuracy of small-scale MCS by designing a feature alignment and reallocation module FADM. The present invention aggregates multi-scale features by using the feature alignment and reallocation module to provide rich spatial information and semantic details.

[0052] Furthermore, the S404 includes the following steps:

[0053] S4041, will and Concatenate in the channel dimension and get ,in, Indicates the result after splicing;

[0054] S4042, for the first branch, The characteristic channels are compressed to obtain , and is passed to a convolutional block to extract vertical features and concentrate horizontal features. Passed to another convolutional block to extract horizontal features and concentrate on vertical features. Represents the compressed features;

[0055] S4043, adding the feature maps of different dimensions processed by S4042 by pixel to obtain an edge space feature map ;

[0056] S4044, for the second branch, use maximum pooling compression The spatial representation of , and use point-by-point convolution to obtain the pixel-by-pixel spatial weight;

[0057] S4045. Activate the spatial weight to obtain the output of the second branch ,in, represents the output of the second branch;

[0058] S4046, will and edge space feature map Multiply and adjust the feature information through convolution operations and activation functions;

[0059] S4047, loop through S4041 to S4046, and obtain the i-th layer decoder output by identifying edge information of mesoscale convective systems MCSs.

[0060] The beneficial effect of the above further solution is that the asymmetric feature recovery module can detect and retain the spatial edge features of the object from data of different dimensions, and enhance the extracted features through adaptive weights.

[0061] Furthermore, the loss function of the small target recognition network is expressed as follows:

[0062]

[0063] in, represents the loss function, and Both represent loss weights, represents the weighted joint Jacob loss, represents the weighted binary cross entropy loss, It represents the result of mesoscale convection image segmentation. Represents label data.

[0064] The beneficial effect of the above further solution is that the present invention can optimize the small target recognition network through the above loss function. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 Schematic diagram of the mesoscale convective system MCSs identification dataset constructed by the present invention.

[0066] Figure 2 This is a schematic diagram of the network structure proposed by the present invention.

[0067] Figure 3 Schematic diagram of the structure of the asymmetric feature recovery module AFRM.

[0068] Figure 4 Schematic diagram of the structure of the feature alignment and redistribution module FADM.

[0069] Figure 5 This is a qualitative result diagram of the present invention on the mesoscale convective system MCSs identification data set.

[0070] Figure 6 This is another qualitative result diagram of the present invention on the mesoscale convective system MCSs identification dataset.

[0071] Figure 7 This is a qualitative result diagram of the present invention on the mesoscale convective system MCSs identification dataset in the local area.

[0072] Figure 8 This is a qualitative result diagram of the ablation experiment of the present invention.

[0073] Fig. 9 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0074] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0075] Example

[0076] Before describing the present invention, the following terms are explained:

[0077] MCSs: Mesoscale convective systems.

[0078] AFRM: Feature Alignment Distribution Module, feature alignment redistribution module.

[0079] FADM: Asymmetric Feature Recovery Modul, asymmetric feature recovery module.

[0080] The present invention relates to the field of image target recognition research, and in particular to the rapid recognition of strong convection in atmospheric elements. It mainly solves the problem that the traditional method is slow in recognizing MCSs and only targets local areas. The traditional threshold-based MCSs recognition method has higher recognition accuracy and has excellent effects when applied to local areas. Due to the limitation of computing resources, if you want to recognize MCS in a large range (tropical and subtropical areas), it takes a lot of time.

[0081] In view of the shortcomings of existing technologies, such as Figure 1 As shown in the figure, the present invention proposes a small target recognition network (MFE-Net) based on feature alignment and redistribution. Figure 2 The schematic diagram of the MFE-Net network structure proposed in the present invention. Its input data is Img, and its output is the recognition result graph Output. Among them, the resolution of Img is C×W×H, C represents the number of channels, H represents the height of the image, W represents the width of the image, and the data set resolution size is: 3×512×512. The network of the present invention includes a dual convolution backbone network to extract hierarchical features, and also includes a feature alignment and redistribution module FADM for collecting multi-scale information in the downsampling stage, and an asymmetric feature recovery module AFRM to extract and retain the spatial edge features of convection from different spatial dimensions, and accurately identify the edge information of MCSs. The core and main innovation of the MCSs recognition network of the present invention lies in the feature alignment and redistribution module (FADM) and the asymmetric feature recovery module (AFRM). Among them, Img is the training set constructed by S206.

[0082] like Fig. 9 As shown, the present invention provides a mesoscale convection image segmentation method based on a small target recognition network, and its implementation method is as follows:

[0083] S1. Obtain original cloud top brightness temperature data;

[0084] In this embodiment, original global cloud top brightness temperature data (European Union Cloud Archive User Service (CLAUS) project dataset) is obtained, and the data range is (30°S–30°N, 180°W–180°E).

[0085] S2. Preprocess the original cloud top brightness temperature data and divide it into training set and test set. The implementation method is as follows:

[0086] S201, setting a temperature threshold of the mesoscale convective system MCSs, and marking the area with a temperature less than the temperature threshold from the original cloud top brightness temperature data;

[0087] S202, setting the regional coverage threshold of the mesoscale convective system MCSs, screening the areas marked in S201 from the regional coverage threshold, selecting areas with a coverage area greater than the regional coverage threshold, and marking the areas that meet both the temperature threshold and the regional coverage threshold as mesoscale convective system MCSs areas, wherein the mesoscale convective system MCSs area is marked as 1, and the non-mesoscale convective system MCSs area is marked as 0;

[0088] S203, converting the original cloud top brightness temperature data and the marked data obtained in S202 into an image format;

[0089] S204, according to the converted image format, fill the cloud top brightness temperature data and label data with 0;

[0090] S205, cropping the filled cloud top brightness temperature data and label data in the first dimension;

[0091] S206. Divide into a training set and a test set according to the cropping result, wherein both the training set and the test set include cloud top brightness temperature data and mesoscale convective system MCSs labels in image format.

[0092] In this embodiment, the specific implementation steps of the effective small target recognition network based on feature alignment and redistribution (MFE-Net) proposed by the present invention are as follows:

[0093] A1. Identify mesoscale convective systems (MCSs) from raw data. In this step, the threshold method is used to identify and mark mesoscale convective systems (MCSs) in cloud top brightness temperature data. First, the temperature threshold of mesoscale convective systems (MCSs) is set to 233K, and the areas with temperatures less than 233K are marked from the raw data.

[0094] A2. Set the regional coverage threshold of mesoscale convective systems (MCSs) to 5000 km 2 The marked areas are further screened using the area coverage threshold, and the areas with a coverage area greater than 5000km are selected. 2 The areas that meet both the temperature threshold and the area coverage threshold are marked as mesoscale convective systems (MCSs), where the MCSs area is marked as 1 and the non-MCS area is marked as 0.

[0095] A3. Use the drawing tool BaseMap to convert the original cloud top brightness temperature data and the annotation data obtained in A2 into image format. The size of a single image is 1536*322, and there are a total of 11,680 images.

[0096] A4. Fill the cloud top brightness temperature data and label data with 0. The size of the filled image is 1536*512.

[0097] A5. Crop the filled cloud top brightness temperature data and label data in the first dimension. Each image can be cropped into three 512*512 images. The total amount of data after cropping is 35040.

[0098] A6. Divide the training set and the test set. The present invention uses a ratio of 8:2 to split the data set, and the data volumes of the training set and the test set are 28032 and 7008 respectively.

[0099] S3. Construct a small target recognition network based on feature alignment and redistribution. The implementation method is as follows:

[0100] S301, for the training set and the test set divided by the mesoscale convective system MCSs identification, the label data in step S204 corresponding to the original image is converted into a binary grayscale image;

[0101] S302, converting the converted binary grayscale image into a tensor format tensor, and normalizing the original image and all pixel values ​​in the label data using maximum and minimum normalization;

[0102] S303: construct a small target recognition network according to the normalization processing result of S302.

[0103] In this embodiment, the small target recognition network includes:

[0104] An encoder, used to extract hierarchical pyramid features of the image after the normalization process in S302, wherein the encoder includes four dual convolution blocks and downsampling;

[0105] Feature alignment and redistribution module FADM, used to align the hierarchical pyramid features and redistribute them to the decoder;

[0106] A decoder is used to decode and restore the pyramid features aligned at different levels, wherein the decoder includes a transposed convolution and an asymmetric feature recovery module AFRM, and the asymmetric feature recovery module AFRM is used to extract and retain the spatial edge features of convection from different spatial dimensions to identify the edge information of mesoscale convection systems MCSs.

[0107] In this embodiment, a small target recognition network based on feature alignment and redistribution is constructed and initialized, and the training set preprocessed by S2 is respectively sent to the small target recognition network, specifically including:

[0108] B1. For the mesoscale convective system MCSs identification dataset, the original image is not processed, and the corresponding label mask is converted into a binary grayscale image;

[0109] B2, then convert the image processed by B1 into tensor format tensor, and use maximum and minimum normalization to normalize all pixel values ​​in the original image and label mask to 0 to 1;

[0110] B3. Construct a convection identification network;

[0111] In this embodiment, the small target recognition network includes an encoder, a feature alignment and redistribution module (FADM), and a decoder embedded with an asymmetric feature recovery module (AFRM), and the overall model is an encoding-decoding architecture. The encoder of the small target recognition network consists of four dual convolution blocks and a downsampling layer, which is used to extract the hierarchical pyramid features of the input image. The feature alignment and redistribution module is inserted between the encoder and the decoder to align and redistribute the pyramid features extracted by the encoder to the decoder. The decoder is stacked by transposed convolution and asymmetric feature recovery modules and a target recognition head, which is used to decode and recover the pyramid features of different levels step by step.

[0112] In this embodiment, the input of the small target recognition network is the image data obtained in S206, represented as Img. The image Img is passed to the encoder to extract feature information and obtain feature maps of each layer, which are represented as feature maps. ( ), where i represents the i-th layer of the encoder. The feature map is then The input feature is fed into the feature alignment and redistribution module FADM, which aligns the input features and redistributes them in the channel dimension to ensure that all encoder outputs can be well utilized in the decoding stage. The outputs of the feature alignment and redistribution module FADM are represented as ( ), Indicates the redistribution output. In the decoding stage, the decoder converts the output of the last layer of the encoder And the output of the feature alignment and redistribution module As input, the output of the first layer of the decoder is obtained. The input of the second layer of the decoder comes from the output of the first layer and the output of the feature alignment and redistribution module. The input of the third layer of the decoder comes from the output of the second layer and the output of the feature alignment and redistribution module Finally, the output of the third layer of the decoder is input into the small target recognition head to obtain the final mesoscale convective system MCSs recognition graph Output, and then the difference between the prediction graph Output and the original image mask is constrained by Jacob loss and cross entropy loss, and the loss calculation value is passed into the network for back propagation to automatically adjust the network parameters.

[0113] In this embodiment, after every 10 trainings, the test set is passed to the small target recognition network for testing to determine whether the Dice index of the current test of the network is the highest. If so, the current model parameters are saved, otherwise the next round of training is performed.

[0114] S4. Use the training set to train the small target recognition network. The implementation method is as follows:

[0115] S401, input the images normalized by S302 into the encoder to extract hierarchical pyramid features, and output the feature map of each layer of the encoder ;

[0116] S402: feature map Input to the feature alignment and redistribution module FADM, and perform alignment processing to obtain the redistributed output , where the feature alignment and redistribution module FADM includes four parts: feature alignment, multi-scale feature extraction, global attention, and feature redistribution. Its implementation method is as follows:

[0117] S4021. In the feature alignment part, each layer feature map of the input encoder , using the feature scale of one of the layers as a standard to align the feature information of different layers except the one of the layers, using the bilinear interpolation method to upsample the low-resolution layers except the one of the layers to the same resolution as the one of the layers, and using the adaptive average pooling method to downsample the high-resolution layers except the one of the layers twice, so that the features of the high-resolution layers are consistent with those of the one of the layers, and a feature map with uniform spatial scale is obtained;

[0118] S4022, connecting each feature map in series in the channel dimension, and performing feature fusion by using convolution to obtain a fusion result;

[0119] S4023. In the multi-scale feature extraction part, the multi-scale spatial information is extracted using convolution kernels of different sizes for the fusion result, and the multi-scale spatial information is spliced ​​in the channel dimension and then compressed to obtain the compressed result. ;

[0120] S4024, in the global attention part, performing a convolution operation on the fusion result to obtain a global feature map, and activating the global feature map to obtain a global weight after feature alignment;

[0121] S4025, the aligned global weight and compression result Multiply them together to get the overall output ,in, Represents the overall output;

[0122] S4026. In the feature redistribution section, Split the channel dimension to get the redistributed output ;

[0123] S403, in the decoder part, the feature map output by the fourth layer of the encoder Or the feature map output by the previous layer of the decoder Input to the transposed convolution, we get ,in, represents the output of the transposed convolution in the i-th layer decoder;

[0124] S404, and Input into the asymmetric feature recovery module AFRM to extract and retain the spatial edge features of convection from different spatial dimensions, identify the edge information of the mesoscale convection system MCSs, and obtain the output of the i-th layer decoder ,in, Represents the output of the i-th layer decoder, which is implemented as follows:

[0125] S4041, will and Concatenate in the channel dimension and get ,in, Indicates the result after splicing;

[0126] S4042, for the first branch, The characteristic channels are compressed to obtain , and is passed to a convolutional block to extract vertical features and concentrate horizontal features. Passed to another convolutional block to extract horizontal features and concentrate on vertical features. Represents the compressed features;

[0127] S4043, adding the feature maps of different dimensions processed by S4042 by pixel to obtain an edge space feature map ;

[0128] S4044, for the second branch, use maximum pooling compression The spatial representation of , and use point-by-point convolution to obtain the pixel-by-pixel spatial weight;

[0129] S4045. Activate the spatial weight to obtain the output of the second branch ,in, represents the output of the second branch;

[0130] S4046, will and edge space feature map Multiply and adjust the feature information through convolution operations and activation functions;

[0131] S4047, looping through S4041 to S4046, obtaining the i-th layer decoder output by identifying edge information of the mesoscale convective system MCSs;

[0132] S405, The result of mesoscale convection image segmentation is obtained by inputting it into the segmentation head SegHead, thus completing the training of small target recognition network.

[0133] In this embodiment, image data is input and network training is performed, which specifically includes:

[0134] C1. The input of the network is the image data after normalization in S302. The image is passed to the encoder to extract feature information, and the feature maps of the four layers are output respectively, which are expressed as ( ). Where i represents the i-th layer of the encoder.

[0135] C2. Then The feature alignment and redistribution module is passed to avoid information loss and improve the feature extraction effect of small-scale MCSs. Aggregating multi-scale features in this way can provide rich spatial information and semantic details. The feature alignment and redistribution module consists of four main parts: feature alignment, multi-scale feature extraction, global attention, and feature redistribution.

[0136] C21. Input the pyramid features generated by the encoder in the feature alignment part ( Using the second layer ( ) as a standard to align different layers ( ) feature information. Low resolution and Use bilinear interpolation to upsample to The same resolution enables the network to learn global features and retain certain spatial information for aggregation. At the same time, the adaptive average pooling technique is used to Downsample twice to make it The characteristics of the process can be expressed as follows:

[0137]

[0138] in, and Respectively represent The output feature maps and feature map sizes of the encoder layers, and Represent bilinear interpolation and adaptive average pooling respectively. By performing upsampling and downsampling operations, a uniform feature map can be obtained in the spatial scale. Subsequently, the four feature maps are concatenated in the channel dimension and The convolution is used to perform feature fusion to obtain the fusion result.

[0139] C22, in the multi-scale feature alignment part, the fusion result is used , and The convolution kernels of different sizes are used to extract multi-scale spatial information. In this process, the dilated convolution technique and the grouped convolution technique are used to further improve the perceptual field of the convolution process. Then, the obtained multi-scale spatial information is spliced ​​in the channel dimension and used The convolution of is used to compress the channel and obtain the compressed result SP.

[0140] C23. In the global attention branch, the fusion result is Convolution is performed to obtain a global feature map. Subsequently, the feature map is activated with Sigmoid to obtain the global weights after feature alignment, and then these weights are multiplied with the compression result SP obtained by the multi-scale feature extraction module to output .

[0141] C24, will Split on the channel dimension and get ( ). The data flow of the entire module can be expressed by the following formula:

[0142]

[0143]

[0144]

[0145]

[0146] in, Indicates The output feature map of the encoder layer, Indicates connections by channel, and Represent bilinear interpolation and adaptive average pooling, respectively. Indicates the process of splitting by channel, ( ) indicates that the convolution kernel size is The convolution operation, Represents the activation function, mapping the result between 0 and 1.

[0147] C3. In the decoder part, the input of the first layer of decoder comes from the output of the last layer of encoder ,Will or Input to the transposed convolution to get In the decoding stage, the decoder converts the output of the last layer of the encoder into And the output of the feature alignment and redistribution module As input, the output of the first layer of the decoder is obtained. The input of the second layer of the decoder comes from the output of the first layer and the output of the feature alignment and redistribution module. The input of the third layer of the decoder comes from the output of the second layer and the output of the feature alignment and redistribution module .

[0148] C4. and The asymmetric feature recovery module AFRM is input to output the result of the i-th layer decoder. Specifically, the asymmetric feature recovery module has two branches.

[0149] C41. First, and Concatenate in the channel dimension to get F, where Represents the result after splicing.

[0150] C42. For the first branch, use The convolution of F is used to compress the feature channels of F. , then, Pass to a Convolutional blocks are used to extract vertical features, and horizontal feature changes are concentrated to capture horizontal edge texture differences. Deliver to The convolution block extracts features in the horizontal direction, focuses on processing texture detail changes in the vertical direction, and retains the edge structure in the vertical direction. The feature maps obtained in different dimensions are added pixel by pixel to obtain the edge space feature map. .

[0151] C43. For the second branch, first use the maximum pooling to compress the spatial representation of the feature map F, then use point-by-point convolution to obtain the pixel-by-pixel spatial weight, and use PRelu to activate the spatial weight to obtain the output of this branch. ,in, Represents the output of the second branch.

[0152] C44, will and edge space feature map Multiply and pass Convolution and PRelu activation functions adjust the feature information to obtain the output of the i-th layer decoder , steps C41 to C44 can be expressed by the formula:

[0153]

[0154]

[0155]

[0156]

[0157]

[0158] in, and They represent the outputs of the feature alignment and redistribution module FADM and the decoding stage, Indicates channel connection, Include Convolution, batch normalization BatchNorm (and PRelu activation function, It means that the convolution kernel of the convolution block is , refers to convolution, represents the addition operation, represents the maximum set operation, Include Convolution and batch normalization BatchNorm.

[0159] C44, loop execution C41 to C44 to obtain the output of the third layer decoder ,in, Represents the output of the third layer decoder.

[0160] C5, then Input into the segmentation head SegHead to obtain the final recognition result Output, where SegHead consists of Convolution composition.

[0161] S5. Based on the trained small target recognition network, the mesoscale convection image segmentation result is obtained;

[0162] In this embodiment, the final predicted segmentation map Output of step C5 and the loss of the label data mask in step S302 are calculated. The present invention uses a weighted joint Jacobian loss function and a weighted binary cross entropy (BCE) loss function to train the small target recognition network MFE-Net. The loss function formula is expressed as:

[0163]

[0164] in, represents the loss function, and Both represent loss weights, represents the weighted joint Jacob loss, represents the weighted binary cross entropy loss, It represents the result of mesoscale convection image segmentation. Represents label data.

[0165] S6, after reaching the threshold of the number of training times in S4 and S5, the test set is input into the trained small target recognition network to determine whether the current small target recognition network meets the index requirements. If so, the parameters of the current small target recognition network are saved and the process goes to S7, otherwise, the process returns to S4;

[0166] In this embodiment, after S4 and S5 are trained 10 times in sequence, the test set is passed into the image MCSs small target recognition network for testing to determine whether the Dice coefficient Dice index of the small target recognition network currently tested is the highest. If so, the current network parameters are saved, otherwise jump to S4 for the next round of training.

[0167] S7. Input the test set into the small target recognition network that meets the index requirements to obtain the final mesoscale convection image segmentation.

[0168] In this embodiment, the best performing small target recognition network is taken out during the test, and then the test set is sequentially fed into the network to calculate the average intersection over union (mIoU) and average overlap (mDice) indicators, and save the MCSs recognition graph Output. The specific calculation formulas for mIoU and mDice are as follows:

[0169]

[0170]

[0171] Among them, P represents the set of network prediction results of MFE-Net, G represents the set of corresponding true labels, k represents the number of categories, and i represents the traversed category.

[0172] In order to verify the effectiveness of the proposed method, the proposed method is compared with other advanced methods. In order to make a fair comparison, the official codes released by other methods are used, and the experimental settings are followed, in which all methods are implemented in the same computing environment, and quantitative and qualitative analysis are performed simultaneously. Table 1 shows the quantitative comparison results of mDice and mIoU on 11 different network structures of the mesoscale convective system MCSs dataset.

[0173] Table 1

[0174]

[0175] Among them, the average overlap mDice is used to calculate the similarity of two samples, and the general value range is [0,1]. The closer the average overlap mDice is to 1, the better. In Table 1, the percentage counting method is used to expand the measurement index by 100 times. The average intersection over union (mIoU) is used to measure the correlation between the true and predicted values. The higher the correlation, the better. Table 1 shows the quantitative results of the mesoscale convective system MCSs data, where the best results are marked in red. It can be found that the present invention obtains the highest average overlap mDice and average intersection over union (mIoU), and the average overlap mDice and average intersection over union (mIoU) obtained by the small target network MFE-Net are 89.68 and 81.29, respectively. In terms of average overlap mDice, the small target network MFE-Net is 1.71 and 2.54 higher than the most competitive U-Net and Unet++, respectively. The results show that the small target network MFE-Net can better solve the problems of uneven distribution and large scale changes of mesoscale convective systems (MCSs). Compared with previous methods, the small target network MFE-Net can provide better segmentation results.

[0176] In this embodiment, Figure 5 , Figure 6 and Figure 7 The results of MCSs recognition in the global and local areas are shown respectively. The figure compares the overall segmentation results of the present invention and other advanced methods. Obviously, in terms of overall segmentation performance, the segmentation results of the small target recognition network MFE-Net proposed in the present invention are more accurate than those of other methods. Figure 5 and Figure 6 It can be seen that most of the comparison methods have obvious over-segmentation. For example, ShelfNet, DeepLabV_3+, BisNetV_2 and other methods used for small target segmentation can only segment the general outline of the mesoscale convective system MCSs, and the segmentation effect of edge details is poor. Methods based on Unet structure (UNet, Unet++, MalUNet, etc.) perform well in terms of overall segmentation results. However, when pasting the mesoscale convective system MCSs, there is still segmentation loss on the target edge. Figure 7 In the figure, only several methods that perform well in quantitative indicators are shown in order to analyze the results from the perspective of segmentation details. The locations where each method has major segmentation problems are marked with red boxes. For very small MCS, most methods show segmentation failure or over-segmentation problems. Unet++ also has problems with small target error segmentation. The segmentation results of U-Net are relatively good, but there is also a problem of boundary information loss. Overall, the small target network MFE-Net shows a clear segmentation advantage.

[0177] In order to study the effectiveness of the feature alignment and redistribution module FADM and the asymmetric feature recovery module AFRM in improving performance, the feature alignment and redistribution module FADM and the asymmetric feature recovery module AFRM are removed successively for ablation experiments. Figure 8 As shown in the figure, compared with the complete small target recognition network MFE-Net network model, Table 2 gives the mDice index and mIoU index results of the ablation experiment. Combined with Table 2, it can be seen that after removing the feature alignment and redistribution module FADM and the asymmetric feature recovery module AFRM, the indicators have decreased. This proves the effectiveness of the feature alignment and redistribution module FADM and the asymmetric feature recovery module AFRM in terms of indicators.

[0178] Table 2

[0179]

[0180] from Figure 8 It was found that after removing the feature alignment and redistribution module FADM, the segmentation feature information would become redundant, resulting in over-segmentation and texture loss. After removing the asymmetric feature recovery module AFRM, the edge space feature loss increased significantly, resulting in unclear segmentation of adjacent MCS. After removing the feature alignment and redistribution module FADM and the asymmetric feature recovery module AFRM, the over-segmentation of the mesoscale convective system MCSs can be clearly observed. Finally, the ablation experiment proves that the two modules proposed in the present invention improve the segmentation accuracy of the mesoscale convective system MCSs. The feature alignment and redistribution module FADM can extract and align multi-scale features in spatial dimensions, providing rich spatial information and semantic details. The asymmetric feature recovery module AFRM can extract and retain the spatial edge features of objects from data of different spatial dimensions, and enhance the extracted features using adaptive weights. In order to more intuitively prove the alignment redistribution module FADM and the feature alignment and redistribution module AFRM.

[0181] In summary, the traditional threshold-based mesoscale convective system MCSs identification method has higher recognition accuracy and has excellent effects when applied in local areas. Due to the limitation of computing resources, if you want to identify MCS in a large range (tropical and subtropical areas), it takes a lot of time. The present invention constructs a small target recognition network (MFE-Net) based on feature alignment and redistribution to identify mesoscale convective system MCSs worldwide. In addition, the present invention also constructs a convection recognition dataset for training and testing small target recognition networks. The small target recognition network mainly includes two core modules: feature alignment and redistribution module FADM and asymmetric feature recovery module AFRM. First, the backbone network of UNet is used to extract hierarchical features, and then the feature alignment and redistribution module FADM is constructed to aggregate multi-scale information in the downsampling stage, aiming to improve the model's perception of small-scale mesoscale convective system MCSs, prevent small-scale features from being lost during the downsampling process of the model, and provide sufficient guidance for accurately identifying small-scale MCSs. In addition, in order to alleviate the decrease in recognition accuracy caused by the blurred boundaries of mesoscale convective systems MCSs, an asymmetric feature recovery module AFRM is proposed to extract and retain the spatial edge features of convection from different spatial dimensions, and accurately identify the edge information of mesoscale convective systems MCSs. Finally, it is proved on the constructed data set that the present invention can quickly and accurately identify mesoscale convective systems MCSs on a global scale. Compared with other deep learning models, the small target recognition network proposed in the present invention has better performance, which solves the problems of high computing resource consumption, slow recognition speed and small recognition area in the traditional method of mesoscale convective system MCSs recognition.

[0182] It should be noted that the above specific embodiments are exemplary, and those skilled in the art can come up with various solutions inspired by the disclosure of the present invention, and these solutions also belong to the disclosure scope of the present invention and fall within the protection scope of the present invention. Those skilled in the art should understand that the present invention description and its drawings are illustrative and do not constitute a limitation of the claims. The protection scope of the present invention is defined by the claims and their equivalents.

Claims

1. A mesoscale convective image segmentation method based on a small target recognition network, characterized in that: The following steps are involved: S1. Obtain original cloud top brightness temperature data; S2, preprocessing the original cloud top brightness temperature data and dividing it into a training set and a test set; S3, building a small target recognition network based on feature alignment and redistribution; S4, using the training set to train the small target recognition network; S5. Based on the trained small target recognition network, the mesoscale convection image segmentation result is obtained; S6, after reaching the threshold of the number of training times in S4 and S5, the test set is input into the trained small target recognition network to determine whether the current small target recognition network meets the index requirements. If so, the parameters of the current small target recognition network are saved and the process goes to S7, otherwise, the process returns to S4; S7, input the test set into the small target recognition network that meets the index requirements, and obtain the final mesoscale convection image segmentation; The S3 comprises the following steps: S301, for the training set and the test set divided by the mesoscale convective system MCSs identification, the label data in step S204 corresponding to the original image is converted into a binary grayscale image; S302, converting the converted binary grayscale image into a tensor format tensor, and normalizing the original image and all pixel values ​​in the label data using maximum and minimum normalization; S303, constructing a small target recognition network according to the normalization processing result of S302; The small target recognition network includes: An encoder, used to extract hierarchical pyramid features of the image after normalization processing in S302, wherein the encoder includes four dual convolution blocks and downsampling; Feature alignment and redistribution module FADM, used to align the hierarchical pyramid features and redistribute them to the decoder; A decoder, used for decoding and restoring the pyramid features aligned at different levels, wherein the decoder includes a transposed convolution and an asymmetric feature recovery module AFRM, wherein the asymmetric feature recovery module AFRM is used for extracting and retaining the spatial edge features of convection from different spatial dimensions to identify the edge information of the mesoscale convection system MCSs; The S4 comprises the following steps: S401, input the images normalized by S302 into the encoder to extract hierarchical pyramid features, and output the feature map of each layer of the encoder ; S402: feature map Input to the feature alignment and redistribution module FADM, and perform alignment processing to obtain the redistributed output ,Among them, the feature alignment and redistribution module FADM includes four parts:,feature alignment, multi-scale feature extraction, global attention and feature,redistribution; S403, in the decoder part, the feature map output by the fourth layer of the encoder Or the feature map output by the previous layer of the decoder Input to the transposed convolution, we get ,in, represents the output of the transposed convolution in the i-th layer decoder; S404, and Input into the asymmetric feature recovery module AFRM to extract and retain the spatial edge features of convection from different spatial dimensions, identify the edge information of the mesoscale convection system MCSs, and obtain the output of the i-th layer decoder ,in, represents the output of the i-th layer decoder; S405, Input to the segmentation head SegHead to obtain the mesoscale convection image segmentation result and complete the training of the small target recognition network; The S402 includes the following steps: S4021. In the feature alignment part, each layer feature map of the input encoder , using the feature scale of one of the layers as a standard to align the feature information of different layers except the one of the layers, using the bilinear interpolation method to upsample the low-resolution layers except the one of the layers to the same resolution as the one of the layers, and using the adaptive average pooling method to downsample the high-resolution layers except the one of the layers twice, so that the features of the high-resolution layers are consistent with those of the one of the layers, and a feature map with uniform spatial scale is obtained; S4022, connecting each feature map in series in the channel dimension, and performing feature fusion by using convolution to obtain a fusion result; S4023. In the multi-scale feature extraction part, the multi-scale spatial information is extracted using convolution kernels of different sizes for the fusion result, and the multi-scale spatial information is spliced ​​in the channel dimension and then compressed to obtain the compressed result. ; S4024, in the global attention part, performing a convolution operation on the fusion result to obtain a global feature map, and activating the global feature map to obtain a global weight after feature alignment; S4025, the aligned global weight and compression result Multiply them together to get the overall output ,in, Represents the overall output; S4026. In the feature redistribution section, Split the channel dimension to get the redistributed output ; The S404 comprises the following steps: S4041, will and Concatenate in the channel dimension and get ,in, Indicates the result after splicing; S4042, for the first branch, The characteristic channels are compressed to obtain , and is passed to a convolutional block to extract vertical features and concentrate horizontal features. Passed to another convolutional block to extract horizontal features and concentrate on vertical features. Represents the compressed features; S4043, adding the feature maps of different dimensions processed by S4042 by pixel to obtain an edge space feature map ; S4044, for the second branch, use maximum pooling compression The spatial representation of , and use point-by-point convolution to obtain the pixel-by-pixel spatial weight; S4045. Activate the spatial weight to obtain the output of the second branch ,in, represents the output of the second branch; S4046, will and edge space feature map Multiply and adjust the feature information through convolution operations and activation functions; S4047, loop through S4041 to S4046, and obtain the i-th layer decoder output by identifying edge information of mesoscale convective systems MCSs.

2. The mesoscale convection image segmentation method based on a small target recognition network according to claim 1 is characterized in that: The S2 comprises the following steps: S201, setting a temperature threshold of the mesoscale convective system MCSs, and marking the area with a temperature less than the temperature threshold from the original cloud top brightness temperature data; S202, setting the regional coverage threshold of the mesoscale convective system MCSs, screening the areas marked in S201 from the regional coverage threshold, selecting areas with a coverage area greater than the regional coverage threshold, and marking the areas that meet both the temperature threshold and the regional coverage threshold as mesoscale convective system MCSs areas, wherein the mesoscale convective system MCSs area is marked as 1, and the non-mesoscale convective system MCSs area is marked as 0; S203, converting the original cloud top brightness temperature data and the marked data obtained in S202 into an image format; S204, according to the converted image format, fill the cloud top brightness temperature data and label data with 0; S205, cropping the filled cloud top brightness temperature data and label data in the first dimension; S206. Divide into a training set and a test set according to the cropping result, wherein both the training set and the test set include cloud top brightness temperature data and mesoscale convective system MCSs labels in image format.

3. The method for mesoscale convection image segmentation based on a small target recognition network according to any one of claims 1-2, characterized in that: The expression of the loss function of the small target recognition network is as follows: ； in, represents the loss function, and Both represent loss weights, represents the weighted joint Jacob loss, represents the weighted binary cross entropy loss, It represents the result of mesoscale convection image segmentation. Represents label data.

Images