High-resolution thunderstorm cloud recognition method and device based on deep learning, electronic equipment and storage medium

Through deep learning technology, thunderstorm cloud recognition is used to use satellite low light, infrared and GNSS-R data, which solves the problems of night recognition difficulties and insufficient detailed information recognition, and achieves efficient and accurate thunderstorm cloud recognition effect.

CN119087542APending Publication Date: 2024-12-06ELLIPSPACE (BEIJING) TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411012312.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art is difficult to accurately identify thunderstorm clouds under night low light conditions, and the identification process of cloud clusters in satellite cloud maps requires professional knowledge. It is impossible to fully discover detailed information or nuances in the data, and it is impossible to accurately locate the location of thunderstorm clouds.

Method used

Using deep learning-based methods, radiation calibration, brightness temperature inversion and wind speed inversion are performed by obtaining real-time satellite low light data, infrared data and GNSS-R data, and input to the thunderstorm cloud recognition model completed for identification, using multiple The convolution layer, pooling layer, convolution layer and segmentation layer can be separated at each depth for feature extraction and segmentation, and the cross entropy loss function can be calculated to optimize the model.

Benefits of technology

It improves the efficiency and accuracy of thunderstorm cloud recognition, can identify thunderstorm clouds under night low light conditions, expands usage scenarios, improves the spatial resolution and accuracy of recognition, and can identify richer detailed information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119087542A_ABST
    Figure CN119087542A_ABST
Patent Text Reader

Abstract

The invention discloses a high-resolution thunderstorm cloud recognition method based on deep learning. The high-resolution thunderstorm cloud recognition method comprises the following steps: acquiring real-time satellite low-light data, corresponding infrared data and GNSS-R data; radiation calibration, brightness temperature inversion and wind speed inversion are carried out on the real-time satellite low-light data, the infrared data and the GNSS-R data respectively, and data to be identified are obtained; and inputting the to-be-identified data into the trained thunderstorm cloud identification model for identification to obtain thunderstorm cloud annotation data. According to the invention, the trained neural network model is used to identify the thunderstorm cloud area, the identification efficiency and the identification precision are improved, and the thunderstorm cloud can be identified under the condition of low light at night. The invention further discloses a device for implementing the method, electronic equipment and a computer readable storage medium.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of meteorological identification technology, and in particular to a high-resolution thunderstorm cloud identification method, device, electronic device and storage medium based on deep learning. Background Art

[0002] Thunderstorms are characterized by strong suddenness, high intensity and short duration, and are accompanied by lightning, strong winds and precipitation, which are highly destructive. If the arrival of thunderstorms cannot be predicted in time, huge losses may occur, and people's production and life will also be greatly threatened. The resolution of satellite cloud images is constantly increasing, the interval time is constantly shortening, and the physical boundaries of thunderstorm clouds can be obtained, which is suitable for identifying thunderstorm clouds. However, most of the data for satellite identification of thunderstorm clouds are meteorological satellites imaged during the day, and the thunderstorm cloud area is obtained through complex physical calculations using visible light and thermal infrared data. There is a lack of visible light channels at night, and algorithms suitable for daytime cannot be directly applied to low-light remote sensing data. In addition, at this stage, the identification process of clouds in satellite cloud images requires very rich professional knowledge. It is difficult to fully discover valuable details or subtle differences in the data by relying on statistics or subjective extraction of characteristic physical quantities and threshold ranges, and it is impossible to accurately locate the location of thunderstorm clouds. Summary of the invention

[0003] In order to solve the above problems existing in the prior art, the present invention provides a high-resolution thunderstorm cloud recognition method, device, electronic device and storage medium based on deep learning. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0004] A first aspect of an embodiment of the present invention provides a high-resolution thunderstorm cloud recognition method based on deep learning, comprising the following steps:

[0005] Obtain real-time satellite low-light data, corresponding infrared data and GNSS-R data;

[0006] Performing radiation calibration, brightness temperature inversion and wind speed inversion on the real-time satellite low-light data, the infrared data and the GNSS-R data respectively to obtain data to be identified;

[0007] The data to be identified is input into the trained thunderstorm cloud identification model for identification to obtain thunderstorm cloud labeled data; wherein the trained thunderstorm cloud identification model includes: multiple depth-separable convolutional layers, pooling layers, pooling operation layers, convolutional layers, upsampling stacking layers and segmentation layers; the depth-separable convolutional layers include: DepthwiseConv layer, the first BN layer, the first ReLU layer, the PointwiseConv layer, the second BN layer and the second ReLU layer; the pooling operation layer includes: four parallel pooling windows of different sizes; the pooling window of the pooling layer is smaller than the minimum pooling window of the pooling operation layer; the convolutional layer includes: a Conv layer, a third BN layer and a third ReLU layer.

[0008] In one embodiment of the present invention, the thunderstorm cloud recognition model before training includes: a plurality of initial depth-separable convolutional layers, an initial pooling layer, an initial pooling operation layer, an initial convolutional layer, an initial upsampling stacking layer, and an initial segmentation layer;

[0009] The training method of the thunderstorm cloud recognition model includes:

[0010] Constructing a training data set; wherein the training data set includes: training samples constructed based on sample satellite low-light data, corresponding sample infrared data and sample GNSS-R data, and real labels of thunderstorm drop areas corresponding to the training samples constructed based on sample convection primary drop point data, sample precipitation data and sample meteorological radar reflectivity data;

[0011] Input the training sample into an initial depth-separable convolutional layer for feature extraction, and output a feature map;

[0012] Inputting the feature map into the initial pooling layer to extract the regional information of the feature map to obtain the first pooling data;

[0013] Inputting the first pooled data into the initial pooling operation layer to extract the region information of the first pooled data to obtain the second pooled data;

[0014] Input the second pooled data into the initial convolutional layer for feature extraction, and output a key feature map;

[0015] Upsampling the key feature map and outputting global feature data;

[0016] Inputting the global feature data into the initial segmentation layer for segmentation to obtain thunderstorm cloud training annotation data;

[0017] Calculate the value of the cross entropy loss function according to the real label of the thunderstorm area;

[0018] The training method of the thunderstorm cloud recognition model is repeatedly executed, and the learning rate of each training is optimized by using stochastic gradient descent and the weight of the thunderstorm cloud recognition model is updated until a preset condition is met to obtain a trained thunderstorm cloud recognition model; wherein the preset condition includes: the learning rate of the thunderstorm cloud recognition model is less than a threshold, or the cross entropy loss function is minimized or a preset number of iterations is reached.

[0019] In one embodiment of the present invention, constructing a training data set includes:

[0020] Obtain sample satellite low-light data, corresponding sample infrared data and sample GNSS-R data;

[0021] Performing radiation calibration on the sample satellite low-light data to obtain sample radiation calibration data;

[0022] Performing brightness temperature inversion on the sample infrared data to obtain sample brightness temperature inversion data;

[0023] Performing wind speed inversion on the sample GNSS-R data to obtain sample wind speed inversion data; wherein the sample radiation calibration data, the sample brightness temperature inversion data and the sample wind speed inversion data constitute the training sample;

[0024] Obtain sample convection initial point data, sample precipitation data and sample meteorological radar reflectivity data;

[0025] A sample thunderstorm location is determined according to the sample convection primary location data, the sample precipitation data and the sample meteorological radar reflectivity data; wherein the sample thunderstorm location is used as a true label of the thunderstorm location.

[0026] In one embodiment of the present invention, determining the sample thunderstorm drop zone according to the sample convection primary drop point data, the sample precipitation data and the sample meteorological radar reflectivity data includes:

[0027] Determine the primary convection area and its value according to the sample convection primary landing point data;

[0028] Determine the precipitation area and the corresponding precipitation amount according to the sample precipitation data;

[0029] Determining, based on the sample meteorological radar reflectivity data, a potential thunderstorm area where the radar reflectivity factor is greater than a reflectivity threshold;

[0030] Determine whether the value of the primary convection area is 1;

[0031] If it is 1, it is determined whether there is a first overlapping area between the precipitation area with a precipitation amount greater than the precipitation threshold and the primary convection area with a value of 1;

[0032] If there is a first overlapping area, determining whether there is a second overlapping area between the first overlapping area and the potential thunderstorm area;

[0033] If there is a second overlap, the second overlap area is the sample thunderstorm drop zone.

[0034] In one embodiment of the present invention, the sample radiation calibration data is radiance;

[0035] The calculation formula of the radiance is:

[0036] R = DN * gain + bias

[0037] Where R is the radiance (unit: W / (m 2 ·sr·μm)), gain and bias are the calibration coefficients obtained from the metadata of the sample satellite low-light data, and DN is the relative size of the ground spectral reflectance represented by each pixel in the sample satellite low-light data.

[0038] In one embodiment of the present invention, the sample brightness temperature inversion data is a brightness temperature value;

[0039] The brightness temperature value calculation formula is:

[0040] TBB=(c2 / (wavel*log(c1 / (wavel 5 *R)+1)))

[0041] c1=2*H*c 2

[0042] c2=H*c / kb

[0043] Where TBB is the brightness temperature in K, wavel is the central wavelength of the satellite channel in m, c is the speed of light, which is 2.99792458×108m / s, and H is Planck's constant, which is 6.62606876×10 -34 Js, kb is the Boltzmann constant, and its value is 1.3806503×10 -23 J / K, R is the radiance.

[0044] In one embodiment of the present invention, the sizes of the four parallel pooling windows of the initial pooling operation layer are 3×3, 5×5, 7×7, and 10×10 respectively; the size of the pooling window of the initial pooling layer is 2×2.

[0045] A second aspect of an embodiment of the present invention provides a high-resolution thunderstorm cloud identification device based on deep learning, comprising:

[0046] Acquisition module, used to acquire real-time satellite low-light data, corresponding infrared data and GNSS-R data;

[0047] A data preprocessing module, used to perform radiation calibration, brightness temperature inversion and wind speed inversion on the real-time satellite low-light data, the infrared data and the GNSS-R data, respectively, to obtain data to be identified;

[0048] A trained thunderstorm cloud recognition model is used to input the data to be recognized into the trained thunderstorm cloud recognition model to obtain thunderstorm cloud annotation data; wherein the trained thunderstorm cloud recognition model includes: multiple depth-separable convolutional layers, pooling layers, pooling operation layers, convolutional layers, upsampling stacking layers and segmentation layers; the depth-separable convolutional layers include: DepthwiseConv layer, first BN layer, first ReLU layer, PointwiseConv layer, second BN layer and second ReLU layer; the pooling operation layer includes: four parallel pooling windows of different sizes; the pooling window of the pooling layer is smaller than the minimum pooling window of the pooling operation layer; the convolutional layer includes: Conv layer, third BN layer and third ReLU layer.

[0049] A third aspect of an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a high-resolution thunderstorm cloud identification method based on deep learning provided by the first aspect of an embodiment of the present invention is implemented.

[0050] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a high-resolution thunderstorm cloud identification method based on deep learning provided by the first aspect of an embodiment of the present invention is implemented.

[0051] Beneficial effects of the present invention:

[0052] The present invention improves the recognition efficiency and recognition accuracy by using a trained neural network model to identify thunderstorm cloud areas. At the same time, it can meet the needs of identifying thunderstorm clouds under low-light conditions at night, expands the usage scenarios, and improves the recognition spatial resolution. It can identify richer detailed information, further improving the accuracy of recognition under low-light conditions at night.

[0053] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0054] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0056] Figure 1 A schematic diagram of a process flow of a high-resolution thunderstorm cloud identification method based on deep learning provided by an embodiment of the present invention;

[0057] Figure 2 A schematic diagram of the structure of a depth-wise separable convolutional layer provided in an embodiment of the present invention;

[0058] Figure 3a A schematic diagram of a residual structure provided by an embodiment of the present invention;

[0059] Figure 3b A schematic diagram of the structure of a thunderstorm cloud recognition model provided by an embodiment of the present invention;

[0060] Figure 4 The thunderstorm cloud annotation data at a certain time provided by the embodiment of the present invention generates a thunderstorm cloud identification visual diagram on a map;

[0061] Figure 5 for Figure 4 The corresponding cloud map of real-time satellite low-light data;

[0062] Figure 6 The thunderstorm cloud recognition result in North China at a certain time provided by the embodiment of the present invention;

[0063] Figure 7 for Figure 6 Schematic diagram of combined reflectivity observed by ground radar in North China at corresponding time;

[0064] Figure 8 A block diagram of a high-resolution thunderstorm cloud identification device based on deep learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0066] like Figure 1 As shown, a first aspect of an embodiment of the present invention provides a high-resolution thunderstorm cloud recognition method based on deep learning, comprising the following steps:

[0067] Step 11, obtaining real-time satellite low-light data, corresponding infrared data and GNSS-R data.

[0068] Step 12, respectively perform radiation calibration, brightness temperature inversion and wind speed inversion on the real-time satellite low-light data, infrared data and GNSS-R data to obtain the data to be identified;

[0069] Step 13: input the data to be identified into the trained thunderstorm cloud identification model for identification, and obtain thunderstorm cloud labeled data.

[0070] Among them, the trained thunderstorm cloud recognition model includes: depth-separable convolution layer, pooling layer, pooling operation layer, convolution layer, upsampling stacking layer and segmentation layer.

[0071] Depthwise separable convolutional layers, including: DepthwiseConv layer, first BN layer, first ReLU layer, PointwiseConv layer, second BN layer, and second ReLU layer;

[0072] The pooling operation layer includes: four parallel pooling windows of different sizes; the pooling window of the pooling layer is smaller than the minimum pooling window of the pooling operation layer;

[0073] Convolutional layers include: Conv layer, the third BN layer, and the third ReLU layer.

[0074] In this embodiment, the trained neural network model is used to identify the thunderstorm cloud area, which improves the recognition efficiency and recognition accuracy. At the same time, it can meet the requirements of identifying thunderstorm clouds under dim light conditions at night, expand the usage scenarios, and improve the recognition spatial resolution, which can identify richer detailed information and further improve the recognition accuracy.

[0075] Before the steps of the first aspect of the embodiment of the present invention, a process of pre-training a thunderstorm cloud recognition model is also included. The second aspect of the embodiment of the present invention provides a method for training a thunderstorm cloud recognition model, including the following steps:

[0076] Step 21: Build a thunderstorm cloud recognition model:

[0077] The thunderstorm cloud recognition model before training includes: initial depthwise separable convolution layer, initial pooling layer, initial pooling operation layer, initial convolution layer, initial upsampling stacking layer and initial segmentation layer.

[0078] The initial depth-separable convolutional layer includes: an initial DepthwiseConv layer, an initial first BN layer, an initial first ReLU layer, an initial PointwiseConv layer, an initial second BN layer, and an initial second ReLU layer;

[0079] The initial pooling operation layer includes: four parallel pooling windows of different sizes; the pooling window of the initial pooling layer is smaller than the minimum pooling window of the initial pooling operation layer;

[0080] The initial convolutional layer includes: the initial Conv layer, the initial third BN layer, and the initial third ReLU layer.

[0081] Specifically, the initial depth-separable convolution layer uses different convolution kernels for different inputs to perform convolution. It decomposes the ordinary convolution operation into two processes, namely depthwise-convolution and pointwise-convolution. During the convolution process, the calculation process can be simplified by adjusting the number of feature mapping layers to improve the operation speed, such as Figure 2 shown.

[0082] The ordinary convolution process assumes that there is an input of N×H×W×C, and there are k 3×3 convolutions. If pad=1 and stride=1 are set, the output of the ordinary convolution is N×H×E×k. The depth-separable convolution process divides it into two parts. In the first process, Depthwise means dividing the input of N×H×W×C into group=C groups, and then performing 3×3 convolution on each group. This is equivalent to collecting the spatial features of each channel, namely the Depthwise features. In the second process, Pointwise means performing k ordinary 1×1 convolutions on the input of N×H×E×k. This is equivalent to collecting the features of each point, namely the Pointwise features. The final output of Depthwise+Pointwise is also N×H×E×k, which is the same as the result obtained by ordinary convolution.

[0083] According to the comparison, the ordinary convolution operation is: 3×3Conv+BN+ReLU, while Depthwise+Pointwise can be approximately regarded as a convolution layer: 3×3DepthwiseConv+BN+ReLU+1×1Pointwise+BN+ReLU.

[0084] The depthwise separable convolution layer can effectively reduce the number of parameters and the number of multiplication and addition operations in the convolution operation process by separating two different convolution operations, thereby accelerating the operation of the neural network system. At the same time, the depthwise separable convolution considers both channel and region changes in the previous ordinary convolution operation (convolution first considers only the region, and then the channel), thus realizing the separation of channels and regions.

[0085] The implementation code example of the depth-wise separable convolutional layer is as follows:

[0086] class DepthwiseSeparableConv(nn.Module):

[0087]

[0088] The use of deep separable convolution can reduce the amount of computation in the neural network system to a certain extent, but to ensure the accuracy of the recognition results, the number of network layers should be appropriately deepened. However, as the depth of the network continues to increase, the convergence problem of the network will gradually emerge. During the network training process, as the network depth increases, the optimization efficiency of the parameters in the network during the back propagation process will continue to decrease, and the problem of gradient vanishing and gradient exploding may be faced.

[0089] In order to improve the accuracy of network training, the best solution is to deepen the neural network system and appropriately increase the number of parameters that need to be trained in the network. This can significantly improve the training accuracy, but deepening the network also means extending the network training time. The two are mutually restrictive. In order to reduce the impact and ensure that the network does not have the problem of gradient disappearance during the deepening process, residual blocks are introduced in appropriate places for network training to improve the convergence performance of the network. The residual block mainly has two layers, such as Figure 3a As shown, σ is used to represent the activation function, as shown in the formula: F(x) = σ(W1x); then the output y is obtained through the second ReLU: y = σ(W2σ(W1x)+xidentity).

[0090] After the convolution operation of the multi-layer network, the feature layer contains richer information. Using a single pooling layer often loses some information in the feature map. In order to further reduce the loss of contextual information between different sub-regions, a hierarchical global prior structure is used to contain information between different scales and sub-regions. A multi-pooling structure is used to fully extract the information in the feature layer. The multi-pooling structure of the initial pooling operation layer uses the idea of ​​dilated convolution and uses pooling windows of different sizes for pooling to fully extract the detailed information in the image. The pooling results are linked and then upsampled to restore them to the same size as the original image, thereby marking the data.

[0091] In summary, the model is mainly composed of convolutional neural network feature map extraction and multi-pooling structure image segmentation. Some features in the image are extracted through the deep separable convolution layer and mapped to the feature layer. The multi-pooling structure is then used to extract the context information of different regions in the feature layer. Then, the key features are extracted through the ordinary convolution layer. The channels extracted by different pooling window sizes are upsampled to the same size as the original image and then stacked. This is used as the final global feature of the network, and the global feature is segmented to obtain the labeled image. The network architecture is as follows: Figure 3b shown.

[0092] Step 22: construct a training data set. The training set construction first processes the calibration correction of low-light data, infrared inversion of cloud top brightness temperature, resolution resampling, etc.

[0093] Among them, the training data set includes: training samples constructed based on sample satellite low-light data, corresponding sample infrared data and sample GNSS-R data, and the real labels of thunderstorm drop areas corresponding to the training samples constructed based on sample convection primary drop point data, sample precipitation data and sample meteorological radar reflectivity data.

[0094] The specific steps of step 22 include step 221 to step 226:

[0095] Step 221, obtain sample satellite low-light data, corresponding sample infrared data and sample GNSS-R data. Here, these sample data can be historical data or real-time data.

[0096] Step 222, perform radiometric calibration on the sample satellite low-light data to obtain sample radiometric calibration data.

[0097] Each pixel in the raw data of low-light remote sensing images represents the relative size of the ground spectral reflectance, which is called the DN value. If it is to be used for subsequent processing or to generate some reflectance products, radiation calibration must be performed.

[0098] For low-light satellite remote sensing images, radiation calibration generally outputs radiance, and the calculation formula is:

[0099] R = DN * gain + bias

[0100] Where R is the radiance (unit: W / (m 2 ·sr·μm)), gain and bias are the calibration coefficients obtained from the metadata of the sample satellite low-light data, and DN is the relative size of the ground spectral reflectance represented by each pixel in the sample satellite low-light data.

[0101] Step 223, performing brightness temperature inversion on the sample infrared data to obtain sample brightness temperature inversion data.

[0102] The brightness temperature value calculation formula is:

[0103] TBB=(c2 / (wavel*log(c1 / (wavel 5 *R)+1)))

[0104] c1=2*H*c 2

[0105] c2=H*c / kb

[0106] Where TBB is the brightness temperature in K, wavel is the central wavelength of the satellite channel in m, c is the speed of light, which is 2.99792458×108m / s, and H is Planck's constant, which is 6.62606876×10 -34 Js, kb is the Boltzmann constant, and its value is 1.3806503×10 -23 J / K, R is the radiance.

[0107] Step 224 , performing wind speed inversion on the sample GNSS-R data to obtain sample wind speed inversion data.

[0108] Among them, sample radiation calibration data, sample brightness temperature inversion data and sample wind speed inversion data constitute the training samples.

[0109] Step 225, obtaining sample convection primary landing point data, sample precipitation data and sample meteorological radar reflectivity data.

[0110] Step 226, determining the sample thunderstorm fall area according to the sample convection primary fall point data, the sample precipitation data and the sample meteorological radar reflectivity data.

[0111] Among them, the sample thunderstorm area is used as the real label of the thunderstorm area.

[0112] The specific steps of step 226 include steps A1-A

[0113] Step A1, determining the primary convection area and its value according to the sample convection primary landing point data.

[0114] Step A2: determine the precipitation area and the corresponding precipitation amount according to the sample precipitation data.

[0115] Step A3: Determine potential thunderstorm areas where the radar reflectivity factor is greater than the reflectivity threshold according to the sample weather radar reflectivity data. For example, areas where dBz is greater than 35 are potential thunderstorm areas.

[0116] Step A4, determining whether the value of the primary convection area is 1.

[0117] Step A5, if the value is 1, determine whether there is an overlapping area between the precipitation area with a precipitation amount greater than the precipitation threshold and the primary convection area with a value of 1.

[0118] If it is not 1, it is a non-thunderstorm area.

[0119] Step A6: If yes, the area is the first overlapping area, and then it is determined whether there is an overlapping area between the first overlapping area and the potential thunderstorm area.

[0120] If there is no overlapping area, it is a non-thunderstorm area.

[0121] Step A7: If it exists, the area is the second overlap, and the second overlap area is the sample thunderstorm drop area.

[0122] If there is no overlapping area, it is a non-thunderstorm area.

[0123] For example, the training dataset structure is:

[0124] The feature set of training samples is in the form of:

[0125] {[radiance, brightness temperature, wind speed],

[0126] ....................,

[0127] [Radiance, brightness temperature, wind speed]}

[0128] The actual label set form:

[0129] {[0],[1],......,[1]}

[0130] 0 represents the sample thunderstorm area, and 1 represents not the sample thunderstorm area.

[0131] Step 23: Input the training sample into the initial depth-separable convolutional layer for feature extraction and output the feature map.

[0132] Step 24, input the feature map into the initial pooling layer to extract the regional information of the feature map to obtain the first pooling data.

[0133] Step 25, inputting the first pooled data into the initial pooling operation layer to extract the region information of the first pooled data to obtain the second pooled data.

[0134] Step 26: Input the second pooled data into the initial convolutional layer for feature extraction and output the key feature map.

[0135] Step 27, upsample the key feature map and output global feature data.

[0136] Step 28, input the global feature data into the initial segmentation layer for segmentation to obtain thunderstorm cloud training annotation data.

[0137] Step 29, calculate the value of the cross entropy loss function according to the true label of the thunderstorm area.

[0138] Step 30, repeating steps 23 to 29, using stochastic gradient descent to optimize the learning rate of each training and update the weight of the thunderstorm cloud recognition model until the preset conditions are met to obtain a trained thunderstorm cloud recognition model.

[0139] Among them, the preset conditions include: the learning rate of the thunderstorm cloud recognition model is less than a threshold, or the cross entropy loss function is minimized or reaches a preset number of iterations.

[0140] In this embodiment, the input of the thunderstorm cloud recognition model is w×h×d, where w and h represent the size of the input image, and d represents the dimension of the input image. In order to fully extract the features in the original image, the convolution kernel size in each convolution layer in the structure is not the same. The pooling in the network selects Maxpooling, the ordinary pooling window is 2×2, and the sizes of the four parallel pooling windows are 3×3, 5×5, 7×7, and 10×10 respectively. The regional information of the feature map is fully extracted through pooling layers of different sizes, and then the key features are extracted through ordinary convolution layers. After that, the feature maps of different sizes are upsampled to the same size as the original image, and the convolution layer is used again for segmentation, and finally the image segmentation result is obtained. Annotated image. In addition, the dropout optimization strategy used in the network is to avoid network overfitting, and the drop rate is selected as 0.5. Each convolution layer is activated by a ReLU nonlinear layer. The default learning rate is optimized using the cross entropy loss function and stochastic gradient descent.

[0141] For example, the code examples for steps 23 to 30 are as follows:

[0142]

[0143]

[0144]

[0145]

[0146] The trained thunderstorm cloud recognition model includes: multiple depth-separable convolutional layers, pooling layers, pooling operation layers, convolutional layers, upsampling stacking layers and segmentation layers; depth-separable convolutional layers, including: DepthwiseConv layer, first BN layer, first ReLU layer, PointwiseConv layer, second BN layer and second ReLU layer; pooling operation layer, including: four parallel pooling windows of different sizes; the pooling window of the pooling layer is smaller than the smallest pooling window of the pooling operation layer; convolutional layers, including: Conv layer, third BN layer and third ReLU layer. The structure of the model remains unchanged before and after training, but the parameters change.

[0147] After the training is completed, the real-time satellite low-light data, corresponding infrared data and GNSS-R data are obtained, and then the real-time satellite low-light data, infrared data and GNSS-R data are calibrated by radiation, brightness temperature inversion and wind speed inversion respectively to obtain the data to be identified. The data to be identified is input into the trained thunderstorm cloud recognition model to obtain thunderstorm cloud annotation data. After obtaining the thunderstorm cloud annotation data, a thunderstorm cloud identification visual map is generated on the map, such as Figure 4 As shown, Figure 5 This is a cloud map of real-time satellite low-light data.

[0148] Here, the process of processing the real-time data to obtain the data to be identified is the same as the specific process of the above-mentioned step 22, which will not be repeated here.

[0149] according to Figure 6 and Figure 7 By comparison, Figure 6 The area surrounded by the red solid line is the identified thunderstorm cloud area. Therefore, it can be seen that the thunderstorm cloud identified by the embodiment of the present invention is more accurate.

[0150] like Figure 8 As shown, a third aspect of an embodiment of the present invention provides a high-resolution thunderstorm cloud recognition device based on deep learning, comprising:

[0151] An acquisition module 41 is used to acquire real-time satellite low-light data, corresponding infrared data and GNSS-R data;

[0152] The data preprocessing module 42 is used to perform radiation calibration, brightness temperature inversion and wind speed inversion on the real-time satellite low-light data, infrared data and GNSS-R data to obtain data to be identified;

[0153] The trained thunderstorm cloud recognition model 43 is used to input the data to be recognized into the trained thunderstorm cloud recognition model to obtain thunderstorm cloud annotation data; wherein, the trained thunderstorm cloud recognition model includes: multiple depth-separable convolutional layers, pooling layers, pooling operation layers, convolutional layers, upsampling stacking layers and segmentation layers; the depth-separable convolutional layers include: DepthwiseConv layer, the first BN layer, the first ReLU layer, the PointwiseConv layer, the second BN layer and the second ReLU layer; the pooling operation layer includes: four parallel pooling windows of different sizes; the pooling window of the pooling layer is smaller than the minimum pooling window of the pooling operation layer; the convolutional layer includes: Conv layer, the third BN layer and the third ReLU layer.

[0154] In one embodiment of the present invention, the thunderstorm cloud recognition model before training includes: a plurality of initial depth-separable convolutional layers, an initial pooling layer, an initial pooling operation layer, an initial convolutional layer, an initial upsampling stacking layer, and an initial segmentation layer;

[0155] The training method of the thunderstorm cloud recognition model includes:

[0156] Constructing a training data set; wherein the training data set includes: training samples constructed based on sample satellite low-light data, corresponding sample infrared data and sample GNSS-R data, and real labels of thunderstorm drop areas corresponding to training samples constructed based on sample convection primary drop point data, sample precipitation data and sample meteorological radar reflectivity data;

[0157] Input the training samples into the initial depth-separable convolutional layer for feature extraction and output feature maps;

[0158] Input the feature map into the initial pooling layer to extract the regional information of the feature map to obtain the first pooling data;

[0159] Inputting the first pooled data into the initial pooling operation layer to extract the region information of the first pooled data, thereby obtaining the second pooled data;

[0160] Input the second pooled data into the initial convolutional layer for feature extraction and output the key feature map;

[0161] Upsample the key feature map and output global feature data;

[0162] The global feature data is input into the initial segmentation layer for segmentation to obtain thunderstorm cloud training annotation data;

[0163] Calculate the value of the cross entropy loss function based on the true label of the thunderstorm area;

[0164] The training method of the thunderstorm cloud recognition model is repeatedly executed, and the learning rate of each training is optimized by using stochastic gradient descent and the weight of the thunderstorm cloud recognition model is updated until the preset conditions are met to obtain a trained thunderstorm cloud recognition model; wherein the preset conditions include: the learning rate of the thunderstorm cloud recognition model is less than a threshold, or the cross entropy loss function is minimized or a preset number of iterations is reached.

[0165] In one embodiment of the present invention, constructing a training data set includes:

[0166] Obtain sample satellite low-light data, corresponding sample infrared data and sample GNSS-R data;

[0167] Perform radiometric calibration on sample satellite low-light data to obtain sample radiometric calibration data;

[0168] Perform brightness temperature inversion on sample infrared data to obtain sample brightness temperature inversion data;

[0169] Perform wind speed inversion on sample GNSS-R data to obtain sample wind speed inversion data; wherein the sample radiation calibration data, the sample brightness temperature inversion data and the sample wind speed inversion data constitute training samples;

[0170] Obtain sample convection initial point data, sample precipitation data and sample meteorological radar reflectivity data;

[0171] The sample thunderstorm drop zone is determined according to the sample convection primary drop point data, the sample precipitation data and the sample meteorological radar reflectivity data; wherein the sample thunderstorm drop zone is used as the true label of the thunderstorm drop zone.

[0172] In one embodiment of the present invention, determining a sample thunderstorm drop zone according to sample convective primary drop point data, sample precipitation data, and sample meteorological radar reflectivity data includes:

[0173] According to the sample convection initial landing point data, the initial convection area and its value are determined;

[0174] According to the sample precipitation data, determine the precipitation area and the corresponding precipitation amount;

[0175] Based on sample meteorological radar reflectivity data, determine potential thunderstorm areas where the radar reflectivity factor is greater than a reflectivity threshold;

[0176] Determine whether the value of the primary convection area is 1;

[0177] If it is 1, it is determined whether there is a first overlapping area between the precipitation area with a precipitation amount greater than the precipitation threshold and the primary convection area with a value of 1;

[0178] If there is a first overlapping area, determining whether there is a second overlapping area between the first overlapping area and the potential thunderstorm area;

[0179] If there is a second overlap, the second overlap area is the sample thunderstorm drop zone.

[0180] In one embodiment of the present invention, the sample radiation calibration data is radiance;

[0181] The calculation formula of radiance is:

[0182] R = DN * gain + bias

[0183] Where R is the radiance (unit: W / (m 2 ·sr·μm)), gain and bias are the calibration coefficients obtained from the metadata of the sample satellite low-light data, and DN is the relative size of the ground spectral reflectance represented by each pixel in the sample satellite low-light data.

[0184] In one embodiment of the present invention, the sample brightness temperature inversion data is a brightness temperature value;

[0185] The brightness temperature value calculation formula is:

[0186] TBB=(c2 / (wavel*log(c1 / (wavel 5 *R)+1)))

[0187] c1=2*H*c 2

[0188] c2=H*c / kb

[0189] Where TBB is the brightness temperature in K, wavel is the central wavelength of the satellite channel in m, c is the speed of light, which is 2.99792458×108m / s, and H is Planck's constant, which is 6.62606876×10 -34 Js, kb is the Boltzmann constant, and its value is 1.3806503×10 -23 J / K, R is the radiance.

[0190] In one embodiment of the present invention, the sizes of the four parallel pooling windows of the initial pooling operation layer are 3×3, 5×5, 7×7, and 10×10 respectively; the size of the pooling window of the initial pooling layer is 2×2.

[0191] A fourth aspect of an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a high-resolution thunderstorm cloud identification method based on deep learning provided in the above-mentioned embodiment of the present invention is implemented.

[0192] A fifth aspect of an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of a high-resolution thunderstorm cloud identification method based on deep learning provided in the above embodiment of the present invention are implemented.

[0193] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0194] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware systems.

[0195] The method provided in the embodiment of the present invention can be applied to electronic devices. Specifically, the electronic device can be: a desktop computer, a portable computer, an intelligent mobile terminal, a server, etc. This is not limited here, and any electronic device that can implement the present invention belongs to the protection scope of the present invention.

[0196] As for the device / electronic device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0197] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0198] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0199] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0200] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A high-resolution thunderstorm cloud recognition method based on deep learning, characterized in that: The following steps are involved: Obtain real-time satellite low-light data, corresponding infrared data and GNSS-R data; Performing radiation calibration, brightness temperature inversion and wind speed inversion on the real-time satellite low-light data, the infrared data and the GNSS-R data respectively to obtain data to be identified; The data to be identified is input into the trained thunderstorm cloud identification model for identification to obtain thunderstorm cloud labeled data; wherein the trained thunderstorm cloud identification model includes: multiple depth-separable convolutional layers, pooling layers, pooling operation layers, convolutional layers, upsampling stacking layers and segmentation layers; the depth-separable convolutional layers include: DepthwiseConv layer, the first BN layer, the first ReLU layer, the PointwiseConv layer, the second BN layer and the second ReLU layer; the pooling operation layer includes: four parallel pooling windows of different sizes; the pooling window of the pooling layer is smaller than the minimum pooling window of the pooling operation layer; the convolutional layer includes: a Conv layer, a third BN layer and a third ReLU layer.

2. The method according to claim 1, characterized in that The pre-trained thunderstorm cloud recognition model includes: multiple initial depth-wise separable convolutional layers, initial pooling layers, initial pooling operation layers, initial convolutional layers, initial upsampling stacking layers, and initial segmentation layers; The training method of the thunderstorm cloud recognition model includes: Constructing a training data set; wherein the training data set includes: training samples constructed based on sample satellite low-light data, corresponding sample infrared data and sample GNSS-R data, and real labels of thunderstorm drop areas corresponding to the training samples constructed based on sample convection primary drop point data, sample precipitation data and sample meteorological radar reflectivity data; Input the training sample into an initial depth-separable convolutional layer for feature extraction, and output a feature map; Inputting the feature map into the initial pooling layer to extract the regional information of the feature map to obtain the first pooling data; Inputting the first pooled data into the initial pooling operation layer to extract the region information of the first pooled data to obtain the second pooled data; Input the second pooled data into the initial convolutional layer for feature extraction, and output a key feature map; Upsampling the key feature map and outputting global feature data; Inputting the global feature data into the initial segmentation layer for segmentation to obtain thunderstorm cloud training annotation data; Calculate the value of the cross entropy loss function according to the real label of the thunderstorm area; The training method of the thunderstorm cloud recognition model is repeatedly executed, and the learning rate of each training is optimized by using stochastic gradient descent and the weight of the thunderstorm cloud recognition model is updated until a preset condition is met to obtain a trained thunderstorm cloud recognition model; wherein the preset condition includes: the learning rate of the thunderstorm cloud recognition model is less than a threshold, or the cross entropy loss function is minimized or a preset number of iterations is reached.

3. The method according to claim 2, characterized in that The constructing of the training data set comprises: Obtain sample satellite low-light data, corresponding sample infrared data and sample GNSS-R data; Performing radiation calibration on the sample satellite low-light data to obtain sample radiation calibration data; Performing brightness temperature inversion on the sample infrared data to obtain sample brightness temperature inversion data; Performing wind speed inversion on the sample GNSS-R data to obtain sample wind speed inversion data; wherein the sample radiation calibration data, the sample brightness temperature inversion data and the sample wind speed inversion data constitute the training sample; Obtain sample convection initial point data, sample precipitation data and sample meteorological radar reflectivity data; A sample thunderstorm location is determined according to the sample convection primary location data, the sample precipitation data and the sample meteorological radar reflectivity data; wherein the sample thunderstorm location is used as a true label of the thunderstorm location.

4. The method according to claim 3, characterized in that Determining the sample thunderstorm fall area according to the sample convection primary fall point data, the sample precipitation data and the sample meteorological radar reflectivity data includes: Determine the primary convection area and its value according to the sample convection primary landing point data; Determine the precipitation area and the corresponding precipitation amount according to the sample precipitation data; Determining, based on the sample meteorological radar reflectivity data, a potential thunderstorm area where the radar reflectivity factor is greater than a reflectivity threshold; Determine whether the value of the primary convection area is 1; If it is 1, it is determined whether there is a first overlapping area between the precipitation area with a precipitation amount greater than the precipitation threshold and the primary convection area with a value of 1; If there is a first overlapping area, determining whether there is a second overlapping area between the first overlapping area and the potential thunderstorm area; If there is a second overlap, the second overlap area is the sample thunderstorm drop zone.

5. The method according to claim 3, characterized in that The sample radiation calibration data is radiance; The calculation formula of the radiance is: R = DN * gain + bias Where R is the radiant brightness (unit: W / (m 2 ·sr·μm)), gain and bias are the calibration coefficients obtained from the metadata of the sample satellite low-light data, and DN is the relative size of the ground spectral reflectance represented by each pixel in the sample satellite low-light data.

6. The method according to claim 5, characterized in that The sample brightness temperature inversion data is a brightness temperature value; The brightness temperature value calculation formula is: TBB=(c2 / (wavel*log(c1 / (wavel 5 *R)+1))) c1=2*H*c 2 c2=H*c / kb Where TBB is the brightness temperature in K, wavel is the central wavelength of the satellite channel in m, c is the speed of light, which is 2.99792458×108m / s, and H is Planck's constant, which is 6.62606876×10 -34 Js, kb is the Boltzmann constant, and its value is 1.3806503×10 -23 J / K, R is the radiance.

7. The method according to claim 3, characterized in that The sizes of the four parallel pooling windows of the initial pooling operation layer are 3×3, 5×5, 7×7, and 10×10 respectively; the size of the pooling window of the initial pooling layer is 2×2.

8. A high-resolution thunderstorm cloud recognition device based on deep learning, characterized in that: include: Acquisition module, used to acquire real-time satellite low-light data, corresponding infrared data and GNSS-R data; A data preprocessing module, used to perform radiation calibration, brightness temperature inversion and wind speed inversion on the real-time satellite low-light data, the infrared data and the GNSS-R data, respectively, to obtain data to be identified; A trained thunderstorm cloud recognition model is used to input the data to be recognized into the trained thunderstorm cloud recognition model to obtain thunderstorm cloud annotation data; wherein the trained thunderstorm cloud recognition model includes: multiple depth-separable convolutional layers, pooling layers, pooling operation layers, convolutional layers, upsampling stacking layers and segmentation layers; the depth-separable convolutional layers include: DepthwiseConv layer, first BN layer, first ReLU layer, PointwiseConv layer, second BN layer and second ReLU layer; the pooling operation layer includes: four parallel pooling windows of different sizes; the pooling window of the pooling layer is smaller than the minimum pooling window of the pooling operation layer; the convolutional layer includes: Conv layer, third BN layer and third ReLU layer.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the high-resolution thunderstorm cloud identification method based on deep learning as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the high-resolution thunderstorm cloud identification method based on deep learning described in any one of claims 1 to 7 is implemented.