Convection initial generation intelligent identification method and device based on multi-scale neural network

Abstract

The application relates to a convection initial generation intelligent identification method and device based on a multi-scale neural network, which comprises the following steps: acquiring multi-channel data of an FY-4A satellite, performing time alignment and channel fusion, then expanding and uniformly cutting a window of a preset size to obtain a plurality of interest domain window data; processing each interest domain window data by using a multi-scale feature extraction module to obtain four features of different scales; processing the four features of different scales by using a multi-scale feature fusion module to obtain full-scale features; inputting the full-scale features corresponding to each interest domain window data into a classification layer to obtain corresponding convection initial generation identification results; and splicing all the convection initial generation identification results to obtain a final identification result. The method is based on a multi-scale convolutional neural network, integrates multi-channel fusion data, explores the advantages of a sliding window method in convection initial generation identification, and effectively improves the identification precision and prediction timeliness of convection initial generation.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of meteorological satellite data processing and artificial intelligence, and relates to a convection initial birth intelligent identification method and device based on a multi-scale neural network. BACKGROUND

[0002] Severe convective weather is sudden, severe weather, and has great destructive power, often accompanied by thunderstorm gales, hail, tornadoes, and local heavy rainfall. Severe convective weather is one of the most destructive weather disasters. Convection initial birth is the initial signal of severe convective weather, and the monitoring of convection initial birth is crucial for the prediction and early warning of severe convective weather.

[0003] Three observation methods are used for convection initial birth identification: automatic station observation for numerical prediction, radar observation and satellite observation for threshold identification and extrapolation. Although it performs well in medium and long-term prediction, it is difficult to meet the needs of convection initial birth identification and prediction due to the grid resolution being much larger than the target scale of convection initial birth. Radar and satellite-based convection initial birth identification methods are the mainstream methods. With the improvement of satellite spatio-temporal resolution, compared with the radar identification method with limited coverage, the satellite identification method with wider coverage has become a key means for convection initial birth identification, which can achieve more detailed convection initial birth identification.

[0004] Currently, satellite-based convection initial birth identification methods can be divided into two categories: physical-driven feature threshold method and data-driven machine learning method. The feature threshold method is based on satellite channels or channel combinations to capture convection initial birth features and form a fixed discriminant index combination to identify convection initial birth. The mainstream convection initial birth identification technology is based on 8 independent convection initial birth prediction indexes from geostationary orbit environmental operational satellite data. The existing data in China is based on FY-4A satellite 4KM resolution data.

[0005] However, existing convection initial birth identification methods such as the blending method and the threshold method mainly use artificial threshold setting, which is limited by the experience of forecasters, and still has deficiencies in identification accuracy and generalization. It is difficult to apply to daily business. The reason is that the artificial threshold-based convection initial birth identification has a certain degree of uncertainty, and the artificial model has weak robustness in different business environments. It is difficult for artificial to process pixel features in satellite cloud images in batches.

[0006] In recent years, with the leap of computing power and the accumulation of massive satellite data, machine learning methods have shown significant advantages in the task of convective initial identification based on their ability to learn complex nonlinear tasks. Machine learning methods can automatically learn the evolution law from historical data, learn the complex relationship between independent variables through special network design, and avoid the over-reliance on statistical models and simplifying assumptions of traditional methods. It is worth noting that existing machine learning-based convective initial identification methods are divided into two methods: identification methods based on random forests, support vector machines and other ensemble learning methods, and identification methods based on deep learning such as fully convolutional networks. The method based on ensemble learning still relies on the prior forecast index to train a more accurate threshold combination, although it has strong interpretability, but inherits the defects of manually set threshold, and has weak spatial representation ability. While the identification method based on deep learning, although it increases the image processing and other spatial feature extraction methods, however, for the small target of convective initial, still faces the problem of little target information in the image and signal easily covered by the background, which will greatly affect the identification accuracy. SUMMARY

[0007] In view of the problems existing in the above-mentioned traditional method, the present application provides a multi-scale neural network-based intelligent identification method and device for convective initial, which can effectively improve the identification accuracy and prediction timeliness of convective initial.

[0008] In order to achieve the above-mentioned purpose, the embodiments of the present application adopt the following technical solutions:

[0009] On the one hand, a multi-scale neural network-based intelligent identification method for convective initial is provided, comprising the following steps:

[0010] Step 1: Obtain multi-channel data of FY-4A satellite, generate domain of interest data through time alignment and channel fusion, expand and uniformly cut with a preset size window to obtain a plurality of domain of interest window data.

[0011] Step 2: A multi-scale feature extraction module is used to extract multi-scale features from each domain of interest window data, and four different scale features corresponding to each domain of interest window data are obtained; the multi-scale feature extraction module is a backbone network based on bottleneck residual block design.

[0012] Step 3: The four different scale features corresponding to each domain of interest window data are input into a multi-scale feature fusion module to obtain corresponding full-scale features; the multi-scale feature fusion module is used to fuse adjacent scale features from bottom to top by using two multi-scale fusion up-sampling modules, integrate effective features in adjacent scale features, and integrate the obtained effective features and large-scale features by using a joint decision module, and then weighted decision is made to obtain full-scale features.

[0013] Step 4: input the full-scale features corresponding to each interest domain window data into a classification layer to obtain corresponding convection initial identification results, splice the convection initial identification results corresponding to all interest domain window data to obtain the final convection initial identification results of the whole image.

[0014] In another aspect, a convection initial intelligent identification device based on a multi-scale neural network is also provided, comprising:

[0015] An interest domain window data determination unit is configured to acquire multi-channel data of a FY-4A satellite, generate interest domain data through time alignment and channel fusion, expand and uniformly cut with a preset size window to obtain a plurality of interest domain window data;

[0016] A multi-scale feature extraction unit is configured to extract multi-scale features of each interest domain window data by using a multi-scale feature extraction module to obtain four different scale features corresponding to each interest domain window data; the multi-scale feature extraction module is a backbone network designed based on a bottleneck residual block;

[0017] A multi-scale feature fusion unit is configured to input the four different scale features corresponding to each interest domain window data into a multi-scale feature fusion module to obtain corresponding full-scale features; the multi-scale feature fusion module is configured to fuse adjacent scale features from bottom to top by using two multi-scale fusion up-sampling modules, integrate effective features in the adjacent scale features, integrate the obtained effective features and large-scale features by using a joint decision module, and obtain full-scale features through decision weighting;

[0018] A convection initial intelligent identification unit is configured to input the full-scale features corresponding to each interest domain window data into a classification layer to obtain corresponding convection initial identification results, splice the convection initial identification results corresponding to all interest domain window data to obtain the final convection initial identification results of the whole image.

[0019] One of the above technical solutions has the following advantages and beneficial effects:

[0020] The method comprises: acquiring multi-channel data of a FY-4A satellite, generating interest domain data through time alignment and channel fusion, expanding and uniformly cutting a window of a preset size to obtain a plurality of interest domain window data; performing multi-scale feature extraction on each interest domain window data by using a multi-scale feature extraction module to obtain four features of different scales corresponding to each interest domain window data; inputting the four features of different scales corresponding to each interest domain window data into a multi-scale feature fusion module to obtain corresponding full-scale features; inputting the full-scale features corresponding to each interest domain window data into a classification layer to obtain corresponding convection initial identification results; and splicing the convection initial identification results corresponding to all interest domain window data to obtain a final convection initial identification result of the whole image. Based on the multi-scale convolutional neural network, the satellite channel fusion data features are integrated, the advantages of the sliding window method in convection initial identification are explored, and the identification accuracy and the prediction timeliness of the convection initial are effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 FIG. 1 is a flowchart of a convection initial intelligent identification method based on a multi-scale neural network in an embodiment;

[0023] Figure 2 FIG. 2 is a general block diagram and a component module schematic diagram of a convection initial intelligent identification model based on a multi-scale neural network in an embodiment, wherein Figure 2 (a) in FIG. 2 is a general block diagram of the convection initial intelligent identification model based on the multi-scale neural network, Figure 2 (b) in FIG. 2 is a structural block diagram of C1, Figure 2 (c) in FIG. 2 is a structural block diagram of C2, Figure 2 (d) in FIG. 2 is a structural block diagram of C3, Figure 2 (e) in FIG. 2 is a structural block diagram of C4, Figure 2 (f) in FIG. 2 is a structural block diagram of C5;

[0024] Figure 3 FIG. 3 is a window sample collection example diagram in an embodiment;

[0025] Figure 4 FIG. 4 is a window sample example diagram in an embodiment;

[0026] Figure 5A structure diagram of a multi-scale fusion up-sampling module in an embodiment;

[0027] Figure 6 A structure diagram of a joint decision module in an embodiment;

[0028] Figure 7 A loss function descent graph in an embodiment. DETAILED DESCRIPTION

[0029] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the specification only for the purpose of describing specific embodiments and is not intended to limit the present application.

[0031] It should be noted that the term "embodiment" mentioned herein means that the specific features, structures or characteristics described in combination with the embodiment can be included in at least one embodiment of the present application. The phrase is shown at various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment to other embodiments. Those skilled in the art can understand that the embodiments described herein can be combined with other embodiments. The term "and / or" used herein refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0032] The embodiments of the present application will be described in detail below in combination with the drawings in the embodiments of the present application.

[0033] In one embodiment, as shown in Figure 1 a convection initial birth intelligent identification method based on a multi-scale neural network can include the following processing steps 1 to 4:

[0034] Step 1: Obtain multi-channel data of FY-4A satellite, generate domain of interest data through time alignment and channel fusion, expand and uniformly cut with a preset size window to obtain a plurality of domain of interest window data.

[0035] Specifically, the multi-channel data of FY-4A satellite includes: FY-4A brightness temperature first-level product data, the first-level product data includes low-layer mid-infrared channel brightness temperature, high-layer mid-infrared channel brightness temperature, high-layer water vapor channel brightness temperature, Low layer water vapor channel brightness temperature, Long wave infrared channel brightness temperature, Long wave infrared channel brightness temperature, Long wave infrared channel brightness temperature, Long wave infrared channel brightness temperature.

[0036] The interest domain data includes: channel difference data (channel difference data) and Channel brightness temperature difference, and Channel brightness temperature difference, and Channel brightness temperature difference); time change data (time change data) Channel brightness temperature / Change, and Brightness temperature difference Change, and Brightness temperature difference Change, and Brightness temperature difference Change).

[0037] The channel fusion generates the interest domain data, which includes calculating the channel difference data between channels by direct difference method and calculating the time change data by front-back frame difference method.

[0038] Step 2: multi-scale feature extraction module is used to perform multi-scale feature extraction on each interest domain window data to obtain four different scale features corresponding to each interest domain window data; the multi-scale feature extraction module is a backbone network based on bottleneck residual block.

[0039] Specifically, the multi-scale feature extraction module includes five layer block groups connected in turn , which is designed based on bottleneck residual block. The multi-scale feature extraction module is 8 interest domain data in the sample, and the output is different features of different scales obtained by each layer block group in turn from large to small ; the four different scale features here refer to the features output by the layer block group .

[0040] Step 3: input the four different scale features corresponding to each interest domain window data into the multi-scale feature fusion module to obtain corresponding full-scale features; the multi-scale feature fusion module is used to fuse adjacent scale features from bottom to top by using two multi-scale fusion up-sampling modules, integrate effective features in adjacent scale features, integrate the obtained effective features and large-scale features by using a joint decision module, and obtain full-scale features by decision weighting.

[0041] Specifically, the multi-scale fusion up-sampling module fuses adjacent scale features from bottom to top and integrates effective features in adjacent scale features. The first multi-scale fusion up-sampling module fuses the features output by the multi-scale feature extraction module , integrates effective features in adjacent scale features, and obtains first up-sampling features; the second multi-scale fusion up-sampling module fuses the first up-sampling features and the features output by the multi-scale feature extraction module , integrates effective features in adjacent scale features, and obtains second up-sampling features.

[0042] The joint decision module is used to integrate, decision weight, and fuse full-scale features by using the first up-sampling features, the second up-sampling features, and the features .

[0043] Step 4: input the full-scale features corresponding to each interest domain window data into the classification layer to obtain corresponding convection initial identification results, and splice the convection initial identification results corresponding to all interest domain window data to obtain the final convection initial identification result of the whole image.

[0044] The convection initial intelligent identification model based on the multi-scale neural network is composed of the multi-scale feature extraction module, the multi-scale feature fusion module, and the classification layer. The overall block diagram of the convection initial intelligent identification model based on the multi-scale neural network is shown in Figure 2 (a), Figure 2 (b) is a structural block diagram of C1, Figure 2 (c) is a structural block diagram of C2, Figure 2 (d) is a structural block diagram of C3, Figure 2 (e) is a structural block diagram of C4, Figure 2 (f) is a structural block diagram of C5.

[0045] The convection initial birth intelligent identification method based on the multi-scale neural network comprises the following steps: acquiring multi-channel data of a FY-4A satellite, generating interest domain data through time alignment and channel fusion, expanding and uniformly cutting a preset size window to obtain a plurality of interest domain window data; performing multi-scale feature extraction on each interest domain window data by using a multi-scale feature extraction module to obtain four different scale features corresponding to each interest domain window data; inputting the four different scale features corresponding to each interest domain window data into a multi-scale feature fusion module to obtain corresponding full-scale features; inputting the full-scale features corresponding to each interest domain window data into a classification layer to obtain corresponding convection initial birth identification results; and splicing the convection initial birth identification results corresponding to all the interest domain window data to obtain the final convection initial birth identification result of the whole image. Based on the multi-scale convolutional neural network, the satellite channel fusion data features are integrated, the advantages of the sliding window method in the convection initial birth identification are explored, and the identification precision and the prediction timeliness of the convection initial birth are effectively improved.

[0046] In one embodiment, the interest domain data comprises channel difference data and time change data; step 1 comprises:

[0047] Step 1-1: reading FY-4A brightness temperature first-level product data, the first-level product data comprising low-layer mid-wave infrared channel brightness temperature, high-layer mid-wave infrared channel brightness temperature, high-layer water vapor channel brightness temperature, low-layer water vapor channel brightness temperature, long-wave infrared channel brightness temperature, long-wave infrared channel brightness temperature, long-wave infrared channel brightness temperature, long-wave infrared channel brightness temperature.

[0048] Step 1-2: time alignment of the FY-4A multi-channel data; wherein the time alignment comprises , , time interval data alignment.

[0049] Specifically, , , The time interval data alignment formula is calculated as follows:

[0050] ;

[0051] Wherein, is the current time of the image, is each time of the first ten frames, is a minimum value function, is an absolute value function, is defined as the time point after the selection is defined as the time point after the selection is defined as the time point before the selection is defined as the time point before the selection is defined as the time point before the selection is defined as the time point before the selection is defined as the time point before the selection

[0052] Step 1-3, the time-aligned multi-channel data is calculated by using a direct difference method to calculate the inter-channel channel difference data, and a front-back frame difference method to calculate the time variation data, and the channel difference data and the time variation data are normalized.

[0053] Specifically, the channel fusion generates the interest domain data, which includes using a direct difference method to calculate the inter-channel channel difference data, and using a front-back frame difference method to calculate the time variation data.

[0054] Wherein, the calculation process of the channel difference interest domain data calculated by using the direct difference method is: reading the time-aligned multi-channel data, calculating the channel difference by using the direct difference method, and the calculation formula is:

[0055]

[0056]

[0057]

[0058] Wherein, is the channel brightness temperature difference

[0059] is the channel brightness temperature difference

[0060] is the channel brightness temperature difference

[0061] are high-level water vapor channel brightness temperature, long-wave infrared channel brightness temperature, long-wave infrared channel brightness temperature, long-wave infrared channel brightness temperature, is a subtraction operation, is the difference between two channel data, is the data of two channels.

[0062] ​​​​​​​​​​​The calculation formula is based on the difference between consecutive frames. , , The formula for calculating time-varying interest region data is:

[0063] ;

[0064] ;

[0065] ;

[0066] ;

[0067] ;

[0068] ;

[0069] in, , , They are respectively Time-varying data Time-varying data Time-varying data, for Channel brightness temperature Change data, for Channel brightness temperature Change data, for and Channel Bright Temperature Difference Change data, for and Channel Bright Temperature Difference Change data, for and Channel Bright Temperature Difference Change data, Data at the initial time point, , , They are respectively , , Data at that time;

[0070] Subsequently, the data in the interest domain is normalized using the following formula:

[0071] ;

[0072] in, For channel Normalized data

[0073] ;

[0074] For channel primary product data, For channel maximum value, For channel minimum value.

[0075] Step 1-4: The obtained interest domain data is expanded and uniformly cut by a preset size window to obtain a plurality of interest domain window data.

[0076] Specifically, window sample collection: a 80*80 grid size window is set, and 230K brightness temperature threshold and 0.3 proportion threshold of maximum brightness temperature gradient and minimum brightness temperature gradient change are used as indexes to extract window samples from the FY-4A primary product data obtained in step 1-1, and the extraction formula is:

[0077] ; ;

[0078] Among them, is the x, y axis coordinates of the to-be-selected window center, is the brightness temperature data corresponding to the (x, y) coordinate pair, respectively, low-layer mid-wave infrared channel brightness temperature, high-layer mid-wave infrared channel brightness temperature, high-layer water vapor channel brightness temperature, low-layer water vapor channel brightness temperature, long-wave infrared channel brightness temperature, long-wave infrared channel brightness temperature, long-wave infrared channel brightness temperature, long-wave infrared channel brightness temperature, is a gradient operation, is a minimum value operation, is a maximum value operation. The results are shown in Figure 3 , Figure 4 . Figure 4 CH07 is low-layer mid-wave infrared channel brightness temperature, CH08 is high-layer mid-wave infrared channel brightness temperature, CH09 is high-layer water vapor channel brightness temperature, CH10 is low-layer water vapor channel brightness temperature, CH11 is long-wave infrared channel brightness temperature, CH13 is long-wave infrared channel brightness temperature, CH12 is Long-wave infrared channel brightness temperature, CH14 is Brightness temperature of long-wave infrared channel.

[0079] In one embodiment, the multi-scale feature extraction module includes five sequentially connected block groups. Composition; layer group Includes: one convolutional layer and one average pooling layer; block group Each group contains three bottleneck residual blocks of 3, 4, and 6 respectively. Each bottleneck residual block consists of a convolutional layer with a 1×1 kernel, a convolutional layer with a 3×3 kernel, and a convolutional layer with a 1×1 kernel. Step 2 includes: applying layer block groups to the interest domain data. The process involves processing the data and then passing the results through a block group. The second-scale feature is obtained through processing; the second-scale feature is then processed using a block group. Processing is performed to obtain third-scale features; these third-scale features are then grouped into blocks. Processing yields 03. Fourth-scale feature; the fourth-scale feature is then grouped into layers. The process yields the fifth-scale features.

[0080] Specifically, a multi-scale feature extraction module is designed: the multi-scale feature extraction module consists of 5 sequentially connected block groups. The system consists of eight interest domains from a sample as input and outputs different features of varying scales obtained sequentially from each block group. ;

[0081] Among them, the block group Consisting of a convolutional layer With an average pooling layer The convolutional layer contains 64 dimensions. The convolution kernel is used to obtain coarse-grained features after inputting data from eight interest regions in the sample. The calculation process is as follows:

[0082] ;

[0083] in, Represents a convolutional layer CC convolution kernel, For convolution operations, To modify the activation function of the linear unit, For the standardized function, For average convolutional pooling operations, For the first i The data in the eight interest domains are as follows: ;

[0084] ;

[0085] ;

[0086] ;

[0087] ;

[0088] ;

[0089] ;

[0090] .

[0091] Layer group Each layer group contains 3, 4, 6, three bottleneck residual blocks respectively, and the bottleneck residual block contains three convolutional layers;

[0092] The convolution kernel of the first convolutional layer and the third convolutional layer ;

[0093] The convolution kernel of the second convolutional layer ;

[0094] Feature Input Obtain features , Input Obtain , Input Obtain , Input Obtain , the calculation process is:

[0095] ;

[0096] Wherein, is a bottleneck residual block, l indicates that the operation is performed l times , , indicates the feature, , is a convolution operation, and + is an addition operation, is a rectified linear unit activation function, is a standardization function.

[0097] The eight domain data of interest are input into the multi-scale feature extraction module, and the output is multi-scale features , the number of channels and the size change are: the number of channels of the output feature map of the layer , the size of the output feature map , the number of channels of the output feature map of the layer , the size of the output feature map , the number of channels of the output feature map of the layer , the size of the output feature map , the number of channels of the output feature map of the layer , the size of the output feature map , the number of channels of the output feature map of the layer , the size of the output feature map .

[0098] In one embodiment, the multi-scale feature fusion module includes two multi-scale fusion up-sampling modules and a joint decision module; step 3 includes: inputting the fifth scale feature, the fourth scale feature and the third scale feature into the first multi-scale fusion up-sampling module to obtain a first up-sampling feature; inputting the first up-sampling feature, the third scale feature and the second scale feature into the second multi-scale fusion up-sampling module to obtain a second up-sampling feature; inputting the first up-sampling feature, the second up-sampling feature and the second scale feature into the joint decision module to obtain a full-scale feature.

[0099] In one embodiment, as Figure 5As shown, the multi-scale fusion up-sampling module includes a high-scale feature pooling branch, a low-scale feature pooling branch and a convolution branch; the fifth-scale feature, the fourth-scale feature and the third-scale feature are input into the first multi-scale fusion up-sampling module to obtain first up-sampling features, including: the third-scale feature is input into the high-scale feature pooling branch, and after average pooling, point convolution, batch normalization processing and ReLU function activation, high-scale pooled features are obtained; the fourth-scale feature is input into the convolution branch, and after convolution and batch normalization processing, convolution features are obtained; the fifth-scale feature is input into the low-scale feature pooling branch, and after maximum pooling, point convolution, batch normalization processing and ReLU function activation, low-scale pooled features are obtained; the high-scale pooled features and the convolution features are spliced, and after point convolution and batch normalization processing, first intermediate fusion features are obtained; the first intermediate fusion features and the high-scale pooled features are multiplied pixel by pixel to obtain second intermediate fusion features; after batch normalization processing, the second intermediate fusion features are added to the third-scale feature after batch normalization processing to obtain the first up-sampling features.

[0100] Specifically, the multi-scale feature fusion module includes two multi-scale fusion up-sampling modules 、 and a joint decision module .

[0101] The two multi-scale fusion up-sampling modules fuse adjacent scale features from bottom to top, integrate effective features in adjacent scale features, the joint decision module integrates effective features in the up-sampling modules, decides weighting, and fuses full-scale features; wherein the multi-scale fusion up-sampling module 、 has the same structure, and each multi-scale fusion up-sampling module includes a high-scale feature pooling branch, a low-scale feature pooling branch and a convolution branch; wherein the high-scale feature pooling branch includes an average pooling layer and a convolution layer (m∈{1,2}, corresponding to 、 respectively); the low-scale feature pooling branch includes a maximum pooling layer , a convolution layer and an up-sampling convolution layer (m∈{1,2}, corresponding to 、 respectively); the convolution branch includes two convolution layers (m∈{1,2}, corresponding to 、 respectively).

[0102] Joint decision-making module It includes one channel decision branch, one spatial decision branch, and one filtering branch; wherein, the channel decision branch contains an upsampling layer. Two max pooling layers , With two convolutional layers , The spatial decision branch contains an upsampling layer. The filtering branch contains a convolutional layer. .

[0103] Features from three adjacent scales are upsampled to obtain a multi-scale fused feature, the size of which is similar to the intermediate scale features. Figure 1 To.

[0104] Two multi-scale fusion upsampling modules obtain two upsampled features, which are then compared with the feature map of the maximum size. Input the joint decision-making module and output a weighted feature with dimensions similar to the feature map. Consistent.

[0105] The largest scale feature among the three scales After pooling layer With convolutional layers And standardize to obtain downscaling and upscaling features. The intermediate scale features among the three scales After convolutional layer Obtain filtering features and with dimensionality enhancement features By splicing, fusion features can be obtained. The feature of the smallest scale in the multi-scale system. After passing through the pooling layer With convolutional layers A weight calculator composed of interconnected components calculates the weights. ; and after passing through the convolutional layer Dimensionality reduction fusion features Multiply by the channels to obtain the weighted features. Minimum Scale Features After the upsampling layer Features after and The features are summed and standardized to obtain the multi-scale fused features. The result is either input into the next multi-scale fusion upsampling module or directly output, and the calculation process is as follows:

[0106] ;

[0107] ;

[0108] ;

[0109] ;

[0110] ;

[0111] ;

[0112] ;

[0113] );

[0114] wherein, i ∈{3,4}, corresponding to the multi-scale feature extracted from the multi-scale feature extraction module , m∈{1,2}, respectively corresponding to , , the convolution layer corresponding to the convolution kernel , , , the up-sampling convolution layer corresponding to the transpose convolution kernel , is an average pooling operation, is a maximum pooling operation, is a convolution operation, is a rectified linear unit activation function, is a standardization function, is a channel-wise weighting operation.

[0115] Two multi-scale fusion up-sampling modules output two up-sampling features and feature , which are respectively input into three branches of the joint decision module, in the channel decision branch, the smaller scale passes through the up-sampling layer and is standardized to obtain the up-scale feature , which then passes through the maximum pooling layer and the convolution layer to obtain the channel decision component , and the larger scale passes through the maximum pooling layer and the convolution layer to obtain the channel decision component , the component is multiplied by the component element by element, and then passes through the standardization and the activation function to obtain the channel weight ; the up-scale feature and the fusion feature The input space decision branch is multiplied element by element and passes through an up-sampling layer , and the spatial weight is obtained after normalization ; the feature passes through the convolution layer in the filtering branch , and the filtered feature is obtained , and is multiplied by the channel weight , the spatial weight , and is normalized and activated to obtain the decision weighted feature , and the calculation process is as follows:

[0116] ;

[0117] ;

[0118] ;

[0119] ;

[0120] ;

[0121] ;

[0122] ;

[0123] wherein the convolution layer corresponds to the convolution kernel , , the up-sampling convolution layer corresponds to the transpose convolution kernel , is an average pooling operation, is a maximum pooling operation, is a convolution operation, is a rectified linear unit activation function, is a normalization function, is a channel weighting operation, is a pixel-by-pixel multiplication operation.

[0124] In the high-scale feature pooling branch, the pooling layer inputs the feature map channel number , corresponding to the feature , the output feature map channel number is unchanged, and the size is reduced to 20x20, the input feature map channel number of the convolution layer is , the convolution kernel size is , and the output feature map channel number is ; in the low-scale feature pooling branch, the pooling layer The number of channels in the input feature map is Corresponding features The number of output feature map channels remains the same, but the size is reduced to 1×1, and the convolutional layer... The number of channels in the input feature map is Corresponding features kernel size Number of output feature map channels Upsampling convolutional layer The number of channels in the input feature map is kernel size Number of output feature map channels The size is increased to 20×20; in the convolutional branch, the convolutional layer The number of channels in the input feature map is Corresponding features kernel size Number of output feature map channels Convolutional layer The number of channels in the input feature map is Corresponding fusion features kernel size Number of output feature map channels Output multi-scale fusion features .

[0125] In the high-scale feature pooling branch, the pooling layer The number of channels in the input feature map is Corresponding features The number of output feature map channels remains the same, but the size is reduced to 40×40, and the convolutional layer... The number of channels in the input feature map is kernel size Number of output feature map channels In the low-scale feature pooling branch, the pooling layer The number of channels in the input feature map is Corresponding features The number of output feature map channels remains the same, but the size is reduced to 1×1, and the convolutional layer... The number of channels in the input feature map is Corresponding fusion features kernel size Number of output feature map channels Upsampling convolutional layer The number of channels in the input feature map is kernel size Number of output feature map channels , the size is expanded to 40x40; in the convolution branch, the input feature map channel number of the convolution layer , the corresponding feature , the convolution kernel size , the output feature map channel number , the input feature map channel number of the convolution layer , the corresponding fusion feature , the convolution kernel size , the output feature map channel number , the output multi-scale fusion feature .

[0126] In one embodiment, as shown in Figure 6 , the joint decision module includes a channel decision branch, a spatial decision branch and a filtering branch; the first up-sampling feature, the second up-sampling feature and the second scale feature are input into the joint decision module to obtain a full-scale feature, including: inputting the second scale feature into the filtering branch to obtain a high-scale fusion feature through convolution and batch normalization processing; inputting the second up-sampling feature into the spatial decision branch to obtain a medium-scale fusion feature through maximum pooling, point convolution, batch normalization processing and ReLU function activation; inputting the first up-sampling feature into the channel decision branch to obtain a low-scale fusion feature through maximum pooling, point convolution, batch normalization processing and ReLU function activation; multiplying the medium-scale fusion feature and the low-scale fusion feature pixel by pixel, then performing batch normalization processing and ReLU function activation to obtain a third intermediate fusion feature; multiplying the third intermediate fusion feature and the high-scale fusion feature pixel by pixel, then performing batch normalization processing to obtain a fourth intermediate fusion feature; performing channel-wise weighting operation on the first up-sampling feature and the second up-sampling feature, then performing batch normalization processing and ReLU function activation to obtain a fifth intermediate fusion feature; performing batch normalization processing on the fourth intermediate fusion feature, then performing channel-wise weighting operation on the fifth intermediate fusion feature, and performing batch normalization processing and ReLU function activation on the obtained result to obtain the full-scale feature.

[0127] Specifically, in the channel decision branch of the joint decision module , the input feature map channel number of the up-sampling layer , the corresponding fusion feature , the output feature map channel number , the size is expanded to 40x40, the input feature map channel number of the maximum pooling layer ​​​​, the output feature map channel number is unchanged, the size is reduced to 1x1, the convolution layer The input feature map channel number is , the convolution kernel size is , the output feature map channel number is , the maximum pooling layer The input feature map channel number is , the corresponding fusion feature , the output feature map channel number is unchanged, the size is reduced to 1x1, the convolution layer The input feature map channel number is , the convolution kernel size is , the output feature map channel number is ; in the spatial decision branch, the up-sampling layer The input feature map channel number is , the output feature map channel number is , and the size is expanded to 80x80; in the filter branch, the convolution layer The input feature map channel number is , the corresponding feature , the convolution kernel size is , and the output feature map channel number is ; finally, the multi-scale fusion feature is output.

[0128] In one embodiment, the classification layer in step 4 includes a convolution layer with a convolution kernel of 3x3, a batch normalization layer, and a ReLU activation function.

[0129] Specifically, the classification layer includes a convolution layer, the input feature map channel number is , the convolution kernel size is , the output feature map channel number is , the convolution step is , and the output feature map size is 80x80. The feature obtained by the multi-scale feature fusion module is input into the classification layer CL , and the output result is a convection initial recognition effect diagram P :

[0130] ;

[0131] wherein, is the classification layer convolution kernel, is the rectified linear unit activation function, is the standardization function, is the convolution operation.

[0132] In one embodiment, the convection initial birth intelligent recognition model based on the multi-scale neural network is composed of a multi-scale feature extraction module, a multi-scale feature fusion module, and a classification layer. In the training process of the convection initial birth intelligent recognition model, the FocalLoss loss function is used to calculate the loss between the model prediction value and the convection initial birth true value.

[0133] ;

[0134] wherein, is the FocalLoss loss function, is the convection initial birth pixel value at a certain index, is the pixel value at the same index in the model recognition effect map, is the horizontal and vertical coordinate index, is the number of pixels contained in the label, and are two hyper-parameters.

[0135] Specifically, the specific process of training the convection initial birth intelligent recognition model based on the multi-scale neural network includes:

[0136] Step P1: Construct a training sample set.

[0137] Step P1-1, window sample collection: Set a window size of 80x80 grid, and use the 230K brightness temperature threshold and the 0.3 proportion threshold of the maximum brightness temperature gradient and the minimum brightness temperature gradient change as indicators to extract window samples from the FY-4A first-level product data obtained in step 1-1. The extraction formula is:

[0138] ; ;

[0139] wherein, is the x and y axis coordinates of the selected window center, is the brightness temperature data corresponding to the coordinates, x, y low layer mid-infrared channel brightness temperature, high layer mid-infrared channel brightness temperature, high layer water vapor channel brightness temperature, low layer water vapor channel brightness temperature, long wave infrared channel brightness temperature, long wave infrared channel brightness temperature, long wave infrared channel brightness temperature, long wave infrared channel brightness temperature, is the gradient operation, is the minimum value operation, is the maximum value operation.​​

[0140] Step P1-2, reading label data: reading the convective initial label data, the label data format is consistent with the FY-4A multi-channel data format, wherein the convective initial pixel and the non-convective initial pixel are marked.

[0141] Step P1-3, positive and negative sample division: the sample containing the convective initial pixel in the window is defined as the positive sample, and the sample not containing the convective initial pixel is defined as the negative sample; for the same time, the positive and negative samples in the interest domain data are collected in a fixed ratio of 1:2, and the overall samples are divided into a training sample set, a verification sample set and a test sample set in a ratio of 8:1:1.

[0142] Step P2, designing a multi-scale feature extraction module, a multi-scale feature fusion module and a classification layer to form a convective initial recognition model based on a multi-scale neural network.

[0143] Step P3, initializing model parameters, setting learning rate, batch size, training round number hyperparameters: adopting batch training, taking a training sample with a dimension of from the training sample set each time to input the network, training and optimizing the network parameters, the parameter in the training process is recorded as the weight in the convolution kernel, using the FocalLoss loss function to calculate the loss between the model prediction effect map and the convective initial true value, learning the parameters through the error back propagation algorithm, and using the SGD optimizer to optimize the back propagation process; when the data set is iterated for one round, the verification sample set is used to evaluate the current model effect, and the hyperparameter setting is optimized; when the training is completed, the model parameters are saved;

[0144] Step P3-1, initializing the model and inputting the model: setting the learning rate as , the batch size as 2, epoch and the training round number as 100, adopting batch training, taking a training sample with a dimension of from the training sample set each time to input the network;

[0145] Step P3-2, loss function calculation and back propagation: using the FocalLoss loss function to calculate the loss between the model prediction effect value and the convective initial true value, the loss function The calculation formula is as follows:

[0146] ;

[0147] Wherein, is the convective initial pixel value at a certain index, is the pixel value at the same index in the model recognition effect map, is the horizontal and vertical coordinate index, The number of pixels contained in the label, and The set hyperparameters are 0.2, 5, respectively;

[0148] The parameters in the training process are recorded as weights in the convolution kernel , the calculated loss value is used to learn the parameters through the error back propagation algorithm, and the SGD optimizer is used to optimize the back propagation process; when the data set is iterated for one round, the validation sample set is used to evaluate the current model effect, the hyperparameter setting is optimized, the model parameters are saved when the training is completed, and the training loss reduction function is as shown in Figure 7 .

[0149] Step P4, using a sliding window method to identify the convection initial generation.

[0150] Step P4-1, based on the input satellite multi-channel data, re-time alignment and fusion to obtain the interest domain data, expand and cut with a window size of 80*80 to obtain the interest domain window data.

[0151] Step P4-1-1, time alignment of the input satellite multi-channel data.

[0152] Step P4-1-2, generate interest domain data.

[0153] Step P4-1-3, expand and cut the interest domain data: obtain the size of the interest domain data, and expand the length and width of the image to a multiple of 80, the expansion formula is:

[0154] ;

[0155] Wherein, the number of pixels of the long axis and the width axis of the original image, the number of pixels of the long axis and the width axis of the expanded image, when the number of pixels of the long axis and the width axis of the original image is a multiple of 80, the image is not expanded;

[0156] Expand the image and cut it with a window size of 80*80 to uniformly divide the expanded image, and obtain the coordinates of the upper left corner of the window in the original image corresponding to:

[0157] ;

[0158] Wherein, is an integer.

[0159] Step P4-2, read the model parameters, input the interest domain window data one by one, extract and fuse multi-scale features, and merge the window segmentation results;

[0160] Step P4-2-1, Extract multi-scale features: After reading the model parameters, for each sample within the window... In the block group of the input multi-scale feature extraction module Features at different scales were obtained, arranged from largest to smallest as follows: , , , , ;sample go through Medium convolution kernel , Standardization and After the function is activated, the output encoded features are determined. Then, the large-scale features are used as input to the next layer block group to extract blocks and corresponding convolutional kernels. Convolution, after standardization, and activation by the corresponding activation function, yields the following results: , , , The calculation formula is:

[0161] ;

[0162] ;

[0163] ; ;

[0164] ;

[0165] in, For the input sample frame, To output encoded features, , include: ;

[0166] ;

[0167] ;

[0168] ;

[0169] ;

[0170] ;

[0171] ;

[0172] ;

[0173] , is a convolution kernel, represents a convolution operation, is a rectified linear unit activation function, is a normalization function.

[0174] Step P4-2-2, fuse multi-scale features: fuse the multi-scale features obtained in step P4-2-1 , , , in order, the first of which is , , input into the module , to perform spatial average pooling on the largest scale feature , and then perform convolution, normalization, and activation function activation on the convolution layer , concatenate the features after convolution and normalization , and obtain the fusion features containing high-scale information, then perform convolution, normalization, and compression information on the convolution layer , multiply the smallest scale feature after spatial maximum pooling, convolution layer convolution, normalization, and activation function activation, to obtain a feature map that initially contains channel attention and high-scale layer information, add the feature map containing attention and high-scale information to the feature map obtained by the upsampling layer , , and then normalize to obtain the multi-scale fusion feature map of this layer, denoted as , wherein the convolution layer corresponding convolution kernel is , , , the upsampling convolution layer corresponding transpose convolution kernel is , and the calculation process is as follows:

[0175] ;

[0176] Similarly, input the output , , into the multi-scale fusion upsampling module , to obtain , and the calculation process is as follows:

[0177] ;

[0178] The convolution layer corresponding convolution kernel is , ,​ up-sampling convolutional layer the corresponding transposed convolution kernel is ;

[0179] then , , input the joint decision module , after the up-sampling layer , respectively after the pooling layer , the convolutional layer , normalization and activation function activation, and after the pooling layer , the convolutional layer , normalization and activation function activation multiply by channel, obtain channel weight, and multiply by element, and after up-sampling , normalization and activation function activation, obtain spatial weight; after the convolutional layer , normalization and activation function activation multiply by channel weight and spatial weight in turn, to obtain the feature map after decision fusion , the corresponding convolution kernel is , up-sampling convolutional layer the corresponding transposed convolution kernel is , the calculation process is:

[0180] ;

[0181] final feature input after the classification layer CL output the effect picture of the initial recognition of the convection P , the calculation process is:

[0182] ;

[0183] wherein, is the classification layer convolution kernel, is the rectified linear unit activation function, is the normalization function.

[0184] Step P4-2-3, splicing the segmentation result: according to the coordinates of the upper left corner of the cutting window on the corresponding expanded image, sequentially sort the window images to form the final recognition result of the whole image.

[0185] It should be understood that, although the above Figure 1The steps are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated in this document, there is no strict order in which these steps are executed; they can be performed in other orders. Furthermore, the above... Figure 1 At least some of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0186] In one embodiment, a convection nascent intelligent recognition device based on a multi-scale neural network is also provided, including:

[0187] The interest domain window data determination unit is used to acquire multi-channel data from the FY-4A satellite, generate interest domain data through time alignment and channel fusion, expand and uniformly cut the data using a preset-sized window to obtain several interest domain window data.

[0188] The multi-scale feature extraction unit is used to perform multi-scale feature extraction on the data of each interest domain window using the multi-scale feature extraction module, so as to obtain four features of different scales corresponding to each interest domain window data; the multi-scale feature extraction module is a backbone network designed based on bottleneck residual blocks.

[0189] The multi-scale feature fusion unit is used to input the features of four different scales corresponding to the data of each interest domain window into the multi-scale feature fusion module to obtain the corresponding full-scale features; the multi-scale feature fusion module is used to fuse adjacent scale features from bottom to top using two multi-scale fusion upsampling modules, integrate the effective features in adjacent scale features, and use a joint decision module to integrate the obtained effective features and large-scale features, and make decision weighting to obtain full-scale features.

[0190] The convection primordia recognition unit is used to input the full-scale features corresponding to each interest domain window data into the classification layer to obtain the corresponding convection primordia recognition results, and to stitch together the convection primordia recognition results corresponding to all interest domain window data to obtain the final convection primordia recognition result of the entire image.

[0191] In one embodiment, the domain of interest data includes: channel difference data and time variation data; the domain of interest window data determination unit is used to read the brightness temperature level 1 product data of FY-4A, the level 1 product data including... Low-level mid-wave infrared channel brightness temperature, High-rise mid-wave infrared channel brightness temperature, Brightness of high-rise water vapor channel Brightness temperature of lower-level water vapor channels Long-wave infrared channel brightness temperature, Long-wave infrared channel brightness temperature, Long-wave infrared channel brightness temperature, Brightness temperature of long-wave infrared channels; time alignment of FY-4A multi-channel data; including time alignment... , , Time interval data alignment; for time-aligned multi-channel data, the channel difference data between channels is calculated using the direct interpolation method, and the time change data is calculated using the frame interpolation method. The channel difference data and the time change data are normalized; the obtained domain of interest data is expanded and uniformly divided using a window of preset size to obtain several domain of interest window data.

[0192] In one embodiment, the multi-scale feature extraction module includes five sequentially connected block groups. Composition; layer group Includes: one convolutional layer and one average pooling layer; block group Each group contains 3, 4, and 6 bottleneck residual blocks, each consisting of a 1×1 convolutional layer, a 3×3 convolutional layer, and a 1×1 convolutional layer. The multi-scale feature extraction unit is also used to apply block groups to the interest domain data. The process involves processing the data and then passing the results through a block group. The second-scale feature is obtained through processing; the second-scale feature is then processed using a block group. Processing is performed to obtain third-scale features; these third-scale features are then grouped into blocks. The process yields fourth-scale features; these fourth-scale features are then grouped into blocks. The process yields the fifth-scale features.

[0193] In one embodiment, the multi-scale feature fusion module includes two multi-scale fusion upsampling modules and a joint decision module; the multi-scale feature fusion unit is further configured to input a fifth-scale feature, a fourth-scale feature, and a third-scale feature into the first multi-scale fusion upsampling module to obtain a first upsampled feature; input the first upsampled feature, the third-scale feature, and a second-scale feature into the second multi-scale fusion upsampling module to obtain a second upsampled feature; and input the first upsampled feature, the second upsampled feature, and the second-scale feature into the joint decision module to obtain full-scale features.

[0194] In one embodiment, the multi-scale fusion upsampling module includes a high-scale feature pooling branch, a low-scale feature pooling branch, and a convolution branch. The multi-scale feature fusion unit is further configured to input a third-scale feature into the high-scale feature pooling branch, and after average pooling, point convolution, batch normalization, and ReLU activation, obtain a high-scale pooled feature; input a fourth-scale feature into the convolution branch, and after convolution and batch normalization, obtain a convolutional feature; input a fifth-scale feature into the low-scale feature pooling branch, and after max pooling, point convolution, batch normalization, and ReLU activation, obtain a low-scale pooled feature; concatenate the high-scale pooling feature with the convolutional feature, and after point convolution and batch normalization, obtain a first intermediate fusion feature; multiply the first intermediate fusion feature and the high-scale pooling feature pixel by pixel to obtain a second intermediate fusion feature; and add the second intermediate fusion feature after batch normalization to the batch-normalized third-scale feature to obtain a first upsampled feature.

[0195] In one embodiment, the joint decision module includes: a channel decision branch, a spatial decision branch, and a filtering branch; a multi-scale feature fusion unit, further configured to input a second-scale feature into the filtering branch, and after convolution and batch normalization, obtain a high-scale fusion feature; input the second upsampled feature into the spatial decision branch, and after max pooling, point convolution, batch normalization, and ReLU activation, obtain a mid-scale fusion feature; input the first upsampled feature into the channel decision branch, and after max pooling, point convolution, batch normalization, and ReLU activation, obtain a low-scale fusion feature; and input the mid-scale feature into the filtering branch, and after max pooling, point convolution, batch normalization, and ReLU activation, obtain a low-scale fusion feature; and input the second upsampled feature into the spatial decision ... The fused feature and the low-scale fused feature are multiplied pixel-by-pixel, then batch normalized and activated by the ReLU function to obtain the third intermediate fused feature; the third intermediate fused feature is multiplied pixel-by-pixel with the high-scale fused feature and then batch normalized to obtain the fourth intermediate fused feature; the first upsampled feature and the second upsampled feature are weighted by channel, then batch normalized and activated by the ReLU function to obtain the fifth intermediate fused feature; the fourth intermediate fused feature is batch normalized and then weighted by channel with the fifth intermediate fused feature, and the result is batch normalized and activated by the ReLU function to obtain the full-scale feature.

[0196] In one embodiment, the classification layer in the nascent intelligent recognition unit includes a convolutional layer with a 3×3 kernel, a batch normalization layer, and a ReLU activation function.

[0197] In one embodiment, a convection nascent intelligent recognition model based on a multi-scale neural network is constructed from a multi-scale feature extraction module, a multi-scale feature fusion module, and a classification layer. During the training process of the convection nascent intelligent recognition model, the FocalLoss loss function is used to calculate the loss between the model's predicted values ​​and the ground truth values ​​of the convection nascent events. The FocalLoss loss function is:

[0198] ;

[0199] in, FocalLoss loss function The initial pixel value of the convection at a certain index. To identify the pixel value at the same index in the model's rendering, For x and y coordinate index, The number of pixels contained in the label. and These are two super coefficients.

[0200] It is understood that for a detailed explanation of the convection nascent intelligent recognition device based on multi-scale neural networks, please refer to the corresponding explanations of the various embodiments of the convection nascent intelligent recognition method based on multi-scale neural networks above, and will not be repeated here. Each module in the above-mentioned convection nascent intelligent recognition device based on multi-scale neural networks can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in hardware or independently of a device with data processing capabilities, or stored in software in the memory of the aforementioned device, so that the processor can call and execute the operations corresponding to each module. The aforementioned device can be, but is not limited to, various types of data processing computer devices already existing in the art.

[0201] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0202] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and all such modifications and improvements fall within the scope of protection of this application.

Claims

1. A method for identifying a convection birth based on a multi-scale neural network, characterized in that, The method comprises the steps of: Step 1: obtaining multi-channel data of FY-4A satellite, generating domain of interest data through time alignment and channel fusion, expanding and uniformly cutting with a preset size window to obtain a plurality of domain of interest window data; Step 2: using a multi-scale feature extraction module to extract multi-scale features from each domain of interest window data to obtain four different scale features corresponding to each domain of interest window data; the multi-scale feature extraction module is a backbone network based on a bottleneck residual block; Step 3: inputting the four different scale features corresponding to each domain of interest window data into a multi-scale feature fusion module to obtain corresponding full-scale features; The multi-scale feature fusion module is used to fuse adjacent scale features from bottom to top by using two multi-scale fusion up-sampling modules, integrate effective features in adjacent scale features, integrate the obtained effective features and large-scale features by using a joint decision module, and obtain full-scale features by decision weighting; Step 4: inputting the full-scale features corresponding to each domain of interest window data into a classification layer to obtain corresponding convection initial identification results, and splicing the convection initial identification results corresponding to all domain of interest window data to obtain final convection initial identification results of the whole image; The multi-scale feature extraction module comprises five sequentially connected layer block groups Composition; layer block group Comprise: a convolution layer, an average pooling layer; layer block group Each group respectively contains 3, 4, 6, three bottleneck residual blocks, and the bottleneck residual block is composed of a convolution layer with a convolution kernel of 1*1, a convolution layer with a convolution kernel of 3*3 and a convolution layer with a convolution kernel of 1*1; step 2 comprises: performing processing on the domain of interest data performing processing on the domain of interest data performing processing on the domain of interest data employing layer block groups processing to obtain third scale features; adopting layer block group for the third scale feature processing to obtain a fourth scale feature; adopting the fourth scale feature with layer block group processing to obtain a fifth scale feature; The multi-scale feature fusion module comprises two multi-scale fusion up-sampling modules and a joint decision module; step 3 comprises: inputting the fifth scale feature, the fourth scale feature and the third scale feature into the first multi-scale fusion up-sampling module to obtain first up-sampling features; inputting the first up-sampling features, the third scale feature and the second scale feature into the second multi-scale fusion up-sampling module to obtain second up-sampling features; inputting the first up-sampling features, the second up-sampling features and the second scale feature into the joint decision module to obtain full-scale features.

2. The method of claim 1, wherein the multi-scale neural network is a convolutional neural network. The domain of interest data comprises channel difference data and time change data; step 1 comprises: Step 1-1: read FY-4A brightness temperature primary product data, the primary product data includes low layer mid-wave infrared channel brightness temperature, high layer mid-wave infrared channel brightness temperature, high layer water vapor channel brightness temperature, low layer water vapor channel brightness temperature, long-wave infrared channel brightness temperature, long-wave infrared channel brightness temperature, long-wave infrared channel brightness temperature, long-wave infrared channel brightness temperature; Step 1-2: Time alignment is performed on the FY-4A multi-channel data; wherein the time alignment includes 、 、 time interval data alignment; Step 1-3: calculating the channel difference data between channels by using a direct difference method on the time-aligned multi-channel data, calculating the time change data by using a front-back frame difference method, and performing normalization processing on the channel difference data and the time change data; Step 1-4: expanding the obtained domain of interest data and uniformly cutting with a preset size window to obtain a plurality of domain of interest window data.

3. The method of claim 1, wherein the multi-scale neural network is a convolutional neural network. The multi-scale fusion up-sampling module comprises a high-scale feature pooling branch, a low-scale feature pooling branch and a convolution branch; inputting the fifth scale feature, the fourth scale feature and the third scale feature into the first multi-scale fusion up-sampling module to obtain first up-sampling features, which comprises: inputting the third scale feature into the high-scale feature pooling branch to obtain high-scale pooling features after average pooling, point convolution, batch normalization processing and ReLU function activation; inputting the fourth scale feature into the convolution branch to obtain convolution features after convolution and batch normalization processing; The fifth scale feature is input into the low scale feature pooling branch, and after maximum pooling, point convolution, batch normalization processing and ReLU function activation, low scale pooling features are obtained; After the high scale pooling features and the convolution features are spliced and then subjected to point convolution and batch normalization processing, first intermediate fusion features are obtained; The first intermediate fusion features and the low scale pooling features are multiplied pixel by pixel to obtain second intermediate fusion features; After the second intermediate fusion features are subjected to batch normalization processing and then added to the fifth scale features subjected to upsampling and batch normalization processing, first upsampling features are obtained.

4. The method of claim 1, wherein the multi-scale neural network-based convection incipience intelligence identification method is characterized by, The joint decision module includes a channel decision branch, a spatial decision branch and a filtering branch; the first upsampling features, the second upsampling features and second scale features are input into the joint decision module to obtain full scale features, including: The second scale features are input into the filtering branch, and after convolution and batch normalization processing, high scale fusion features are obtained; The second upsampling features are input into the spatial decision branch, and after maximum pooling, point convolution, batch normalization processing and ReLU function activation, medium scale fusion features are obtained; The first upsampling features are input into the spatial decision branch, and after upsampling, maximum pooling, point convolution, batch normalization processing and ReLU function activation, low scale fusion features are obtained; After the medium scale fusion features and the low scale fusion features are multiplied pixel by pixel, batch normalization processing and ReLU function activation are performed to obtain third intermediate fusion features; After the third intermediate fusion features and the high scale fusion features are multiplied pixel by pixel, batch normalization processing is performed to obtain fourth intermediate fusion features; After the first upsampling features are subjected to upsampling, the second upsampling features are subjected to channel weighting operation, and the obtained results are input into the spatial decision branch, and after upsampling, batch normalization processing and ReLU function activation, fifth intermediate fusion features are obtained; After the fourth intermediate fusion features are subjected to batch normalization processing, the fifth intermediate fusion features are subjected to channel weighting operation, and the obtained results are subjected to batch normalization processing and ReLU function activation to obtain full scale features.

5. The method of claim 1, wherein the multi-scale neural network-based convection incipience intelligence identification method is characterized by, The classification layer in step 4 includes a convolution layer with a convolution kernel of 3x3, a batch normalization layer and a ReLU activation function.

6. The method of claim 1, wherein, The convection cell birth intelligent recognition model based on the multi-scale neural network is composed of the multi-scale feature extraction module, the multi-scale feature fusion module and the classification layer, and in the training process of the convection cell birth intelligent recognition model, a FocalLoss loss function is used to calculate the loss between the model prediction effect value and the convection cell birth true value. The FocalLoss loss function is: wherein, is a FocalLoss loss function, is a pixel value of the first stream at a certain index, is a pixel value of the second stream at the same index, are horizontal and vertical coordinate indices, is a number of pixels contained in the label, and are two hyper-parameters.

7. A convection incipience intelligent identification device based on a multi-scale neural network, characterized in that, including: The interest domain window data determination unit is configured to obtain multi-channel data of the FY-4A satellite, generate interest domain data through time alignment and channel fusion, expand and uniformly cut with a preset size window to obtain a plurality of interest domain window data; The multiscale feature extraction unit is configured to perform multiscale feature extraction on each interest domain window data by using a multiscale feature extraction module to obtain four features of different scales corresponding to each interest domain window data; the multiscale feature extraction module is a backbone network designed based on a bottleneck residual block; wherein the multiscale feature extraction module includes five layer block groups connected in sequence The layer block group is composed of The layer block group includes a convolutional layer and an average pooling layer Each group includes three, four, and six bottleneck residual blocks, respectively, and each bottleneck residual block is composed of a convolutional layer with a 1*1 convolution kernel, a convolutional layer with a 3*3 convolution kernel, and a convolutional layer with a 1*1 convolution kernel; specifically, the layer block group is used to process the interest domain data to obtain a first scale feature; the second scale feature is processed by the layer block group to obtain a second scale feature; the second scale feature is processed by the layer block group to obtain a third scale feature; the third scale feature is processed by the layer block group to obtain a fourth scale feature; the fourth scale feature is processed by the layer block group to obtain a fifth scale feature; the fifth scale feature is processed by the layer block group The multiscale feature fusion unit is configured to input four features of different scales corresponding to each interest domain window data into a multiscale feature fusion module to obtain corresponding full-scale features; the multiscale feature fusion module is configured to fuse adjacent scale features from bottom to top by using two multiscale fusion up-sampling modules, integrate effective features in the adjacent scale features, integrate the obtained effective features and large-scale features by using a joint decision module, and obtain full-scale features by decision weighting; the multiscale feature fusion module includes two multiscale fusion up-sampling modules and one joint decision module; specifically, the fifth scale feature, the fourth scale feature and the third scale feature are input into the first multiscale fusion up-sampling module to obtain first up-sampling features; the first up-sampling features, the third scale feature and the second scale feature are input into the second multiscale fusion up-sampling module to obtain second up-sampling features; and the first up-sampling features, the second up-sampling features and the second scale feature are input into the joint decision module to obtain full-scale features; The convection initial generation intelligent identification unit is configured to input the full-scale features corresponding to each interest domain window data into a classification layer to obtain corresponding convection initial generation identification results, and splice the convection initial generation identification results corresponding to all interest domain window data to obtain a final convection initial generation identification result of the whole image.

Images