A method for locating and determining the center of a tropical cyclone based on maskrcnn deep learning

By combining the multi-channel infrared image of stationary satellites and the multi-source satellite sea surface wind field image, the Mask R-CNN deep learning method is used to achieve simultaneous prediction of tropical cyclone center positioning and fixed strength, solving the low accuracy of tropical cyclone recognition in the prior art and The difficulty of eyeless tropical cyclone positioning improves the accuracy and efficiency of identification throughout the development stage.

CN116030359BActive Publication Date: 2024-11-01ZHONGKEXING TUWEI TIANXIN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310029720.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-01-09
Publication Date
2024-11-01
Estimated Expiration
2043-01-09

AI Technical Summary

Technical Problem

The prior art is difficult to identify the entire development stage of tropical cyclones with high accuracy, especially the central positioning and fixed strength of eyeless tropical cyclones. Traditional methods usually use a single deep learning structure and stationary satellite infrared images, and cannot effectively utilize multi-source satellite data.

Method used

Using a deep learning method based on Mask R-CNN, combining the multi-channel infrared image of stationary satellites and multi-source satellite sea surface wind field image, the simultaneous prediction of tropical cyclone center positioning and fixed strength through image preprocessing, sample label production and model training. .

Benefits of technology

It improves the central positioning and fixed strength accuracy of the tropical cyclone cloud system at different development stages, improves the time efficiency and physical significance of the model, and is suitable for identification of generation, maturity and demise.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116030359B_ABST
    Figure CN116030359B_ABST
Patent Text Reader

Abstract

The embodiment of the present disclosure provides a method for locating and determining the center of a tropical cyclone based on maskrcnn deep learning, which is applied to the field of satellite remote sensing monitoring disaster technology. The method includes preprocessing the collected geostationary satellite atmospheric window channel remote sensing images, water vapor absorption band channel remote sensing images, and multi-source satellite fused sea surface wind field images to produce training sample labels; inputting training sample labels, adjusting parameters, iterative training until the model converges and there is no overfitting, and constructing a training model based on maskrcnn deep learning; tropical cyclone center positioning and intensity prediction; correcting the prediction results of multiple tropical cyclone cloud systems according to the historical path of the tropical cyclone at the previous moment, or eliminating the misidentified non-tropical cyclone cloud systems, and performing accuracy evaluation. In this way, the identification of tropical cyclone cloud systems, the prediction of center positioning, and the prediction of tropical cyclone intensity can be achieved at one time, and the accuracy of center positioning and intensity determination of eyeless tropical cyclone cloud systems in the entire development stage can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of satellite remote sensing disaster monitoring, and in particular to the technical field of a tropical cyclone center positioning and intensity determination method based on maskrcnn deep learning. Background Art

[0002] A tropical cyclone (TC) is a cyclonic circulation with a huge volume, ranging from 300-400km in diameter to 1000-2000km in diameter. It often occurs on tropical and subtropical seas, including the generation and development period, maturity period and extinction period. It exhibits different texture and shape characteristics at different development stages. Due to the limitations of traditional image segmentation technology, it is difficult to accurately identify tropical cyclones in the entire development stage based on the recognition method of a single feature of the cloud system, and different positioning methods are required for the center positioning of tropical cyclones of different forms. Most of the existing technologies for deep learning to achieve tropical cyclone center positioning and strength determination use CNN (deep convolutional neural network) structure and Faster rcnn structure, and rarely use Mask rcnn structure. In addition, most methods only perform center positioning and strength determination for typhoons with obvious eye. At present, the tropical cyclone center positioning method based on maskrcnn (Region Convolutional Neural Network) deep learning is mainly based on the single-channel infrared image of the geostationary satellite, but the geostationary satellite infrared channel cannot obtain the low-level TC cloud system information, and the tropical cyclone center positioning and intensity determination are processed separately based on different deep learning methods. Summary of the invention

[0003] The present disclosure provides a method for locating and determining the center of a tropical cyclone based on maskrcnn deep learning. The method comprises:

[0004] Satellite image processing and training sample labeling: Preprocess the collected geostationary satellite atmospheric window channel remote sensing images, water vapor absorption band channel remote sensing images, and multi-source satellite fusion sea surface wind field images to produce tropical cyclone center location and intensity determination training sample labels;

[0005] Deep learning training model construction: input training sample labels, adjust parameters, iterate training until the model converges and there is no overfitting, and build a tropical cyclone center positioning and intensity determination training model based on maskrcnn deep learning;

[0006] Tropical cyclone center location and intensity prediction: multi-channel satellite remote sensing images to be predicted are input into a pre-trained tropical cyclone center location and intensity prediction model to generate tropical cyclone center location and intensity prediction results;

[0007] Correction of forecast results and accuracy evaluation: Correct the forecast results with multiple tropical cyclone cloud systems according to the historical paths of tropical cyclones at the previous moment, or eliminate misidentified non-tropical cyclone cloud systems, and conduct accuracy evaluation.

[0008] Furthermore, the satellite image processing and training sample labeling process includes:

[0009] Collecting historical images, the historical images include atmospheric window channel remote sensing images, water vapor absorption band channel remote sensing images and sea surface wind field images;

[0010] Performing matching processing on the historical images;

[0011] According to the tropical cyclone center positioning point and the preset cropping size rule, the matched image is cropped to generate a tropical cyclone sample set; the center positioning point is determined according to the tropical cyclone case at the corresponding time of the tropical cyclone best path data set;

[0012] Automatically annotate the tropical cyclone center and different levels of tropical cyclone intensity on the cropped images to generate a tropical cyclone label set;

[0013] Data segmentation is performed to generate a training data set for the tropical cyclone center location and intensity determination model.

[0014] Furthermore, the automatic labeling of the tropical cyclone center and different levels of tropical cyclone intensities on the cropped image includes:

[0015] Automatic annotation and label generation are achieved by modifying the annotation box and image properties.

[0016] Furthermore, the method of cropping the matched image according to the tropical cyclone center positioning point and the preset cropping size rule to generate the tropical cyclone sample set also includes:

[0017] The cropped image is processed by changing brightness, adding noise, adding random points, blurring, translating, rotating, and mirroring.

[0018] Furthermore, the deep learning training model construction process includes:

[0019] Pre-set the initialization parameters of the convolutional network structure for tropical cyclone center positioning and intensity target detection;

[0020] During the model training process, the parameters of the convolutional network structure for tropical cyclone center location and intensity determination are adjusted;

[0021] Modify and optimize the maskrcnn network structure of tropical cyclone center location and intensity determination target detection; input the training sample labels into the tropical cyclone center location and intensity determination convolutional network structure for model training and verification; calculate the focal loss between the prediction results of the training set and the true label to obtain the fitting error, and use the Adam optimizer to minimize the error;

[0022] The above process is repeated iteratively until the model training and verification errors converge, the loss values ​​of the training set and the verification set tend to be stable, and the accuracy of the training set and the verification set is greater than or equal to the preset threshold. The sample learning of the tropical cyclone center positioning and intensity determination model is completed, and a tropical cyclone center positioning and intensity determination training model based on maskrcnn deep learning is constructed.

[0023] Furthermore, the process of model training and verification of the tropical cyclone center location and intensity determination convolutional network structure includes:

[0024] The Adam optimizer is used to minimize the error. Combined with the batch normalization layer, the neural network weights are iteratively updated through the training set until convergence. All the minimum batches on the training set are considered as one training round. The Adam optimization calculation formula and parameter settings are as follows:

[0025]

[0026] m t =β1m t-1 +(1-β1)g t

[0027]

[0028]

[0029]

[0030]

[0031] in, represents the derivative of the loss function, i.e., the gradient; t represents the number of minimum batches in the training set; the hyperparameter α represents the learning rate, which decays with the number of training rounds (epoch); β1 represents the exponential decay rate of the first-order moment estimate, and β2 represents the exponential decay rate of the second-order moment estimate; the ε parameter represents a very small value added when the divisor is zero during training; f t (θ) represents the random objective function; the other parameters are the initialization parameter vector θ, the first-order moment vector m t-1 , second-order moment vector v t-1 ;

[0032] g t Indicates the gradient of the random objective function, mt represents the first-order moment estimate (mean) of the updated gradient, v t represents the second-order raw moment estimate (with biased variance) of the updated gradient, represents the first-order moment estimate for the calculation of bias correction, represents the second-order raw moment estimate for the calculation bias correction, θ t It means that the parameter vector θ is updated iteratively with the above value until θ converges;

[0033] The focal loss function is used to balance the samples. The focal loss function in multi-classification target detection is defined as follows:

[0034] FL(p t )=-α t (1-p t ) γ log(p t )

[0035] p t It is the probability that the sample in the output result of the Softmax activation function belongs to the true category t, which indicates the degree of difficulty in distinguishing the sample classification. The modulation parameter γ is introduced to indicate the degree of cross entropy attenuation for the t-th class sample, and the modulation parameter α t , represents the weight of the t-th class sample.

[0036] Furthermore, the parameter adjustment process of the convolutional network structure for tropical cyclone center location and intensity determination includes:

[0037] Select a suitable deep residual network structure to extract the feature map of the original image;

[0038] Set the size and aspect ratio of the anchor box;

[0039] Set the ratio of positive and negative samples;

[0040] Adjust the corresponding parameters of the generated region candidate box;

[0041] Adjust the corresponding parameters for removing candidate boxes in overlapping areas.

[0042] Furthermore, the prediction result correction and accuracy evaluation include:

[0043] Calculate the time difference between the predicted moment of a tropical cyclone and the previous moment and the distance difference between the tropical cyclone center location;

[0044] Based on the above calculation results, according to the spacing distances between the centers of different tropical cyclone cloud systems and the moving speed of the same tropical cyclone, we jointly judge and classify them, and correct the prediction results of multiple tropical cyclone cloud systems, or eliminate the misjudged non-tropical cyclone cloud systems.

[0045] The present invention provides a method for locating and strengthening the center of a tropical cyclone based on maskrcnn deep learning. Based on the physical quantities of multi-source satellite sea surface wind field images and combined with multi-channel infrared cloud images of geostationary satellites, maskrcnn target detection deep learning is used to realize the positioning and strengthening of the center of a tropical cyclone. In this way, the recognition of tropical cyclone cloud systems and the prediction of center positioning and strengthening can be realized at one time, and the accuracy of center positioning and strengthening of eyeless tropical cyclone cloud systems in the entire development stages of generation and development, maturity and extinction can be improved.

[0046] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0048] Figure 1 A flow chart of a method for locating and determining the intensity of a tropical cyclone according to an embodiment of the present disclosure is shown;

[0049] Figure 2 A block diagram of a tropical cyclone positioning and intensity determination method according to yet another embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0051] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0052] Figure 1 A flowchart of a method 100 for locating and determining the center of a tropical cyclone based on maskrcnn deep learning according to an embodiment of the present disclosure is shown. The method 100 includes:

[0053] Step 110, satellite image processing and training sample label production: pre-process the collected geostationary satellite atmospheric window channel remote sensing images, water vapor absorption band channel remote sensing images and multi-source satellite fused sea surface wind field images to produce tropical cyclone center positioning and intensity determination training sample labels.

[0054] Step 120, deep learning training model construction: input training sample labels, adjust parameters, iterate training until the model converges and there is no overfitting, and build a tropical cyclone center positioning and intensity determination training model based on maskrcnn deep learning.

[0055] Step 130, tropical cyclone center location and intensity prediction: input the multi-channel satellite remote sensing images to be predicted into a pre-trained tropical cyclone center location and intensity determination model to generate a tropical cyclone center location and intensity determination result.

[0056] Step 140, correction of forecast results and accuracy evaluation: correction of forecast results with multiple tropical cyclone cloud systems according to the tropical cyclone historical path at the previous moment, or elimination of misidentified non-tropical cyclone cloud systems, and accuracy evaluation.

[0057] In some embodiments, the method can locate and strengthen tropical cyclone centers at different development stages and intensities, which not only improves the accuracy of tropical cyclone center location, but also improves the time efficiency of tropical cyclone location and strength model training and prediction. Figure 2 As shown, the specific steps are as follows:

[0058] (1) Since the multi-source satellite microwave scatterometer data can observe the vortex structure characteristics of the sea surface wind field, and there is a low wind speed area in the center of the vortex, by searching for the local minimum, the accuracy of eyeless tropical cyclone positioning and intensity determination can be corrected and improved. Therefore, the multi-source satellite wind field data with physical structure meaning is used as a training sample to substitute into the deep learning model, so that the trained tropical cyclone positioning and intensity determination model has a certain physical basis, and the constructed training model has the physical meaning of tropical cyclone formation. The atmospheric window channel and water vapor absorption band channel of the Himawari-8 satellite AHI payload, as well as the multi-source satellite sea surface wind field fusion image, are selected for time and space matching processing.

[0059] (2) The diameter of a tropical cyclone is generally 600-1000 km, and the largest can reach 2000 km. In order to build a maskrcnn convolutional neural network model suitable for the characteristics of tropical cyclone clouds, the satellite images of the central longitude and latitude of each tropical cyclone case at each moment in the tropical cyclone best path dataset are cropped to a size of 512x512 as the tropical cyclone sample set at that moment.

[0060] (3) Define the center of the annotation box as the tropical cyclone center. Different levels of tropical cyclone intensities correspond to different types of annotation boxes. The size of the annotation box is 100x100. By modifying the annotation box and the image attribute "imageData", automatic annotation is achieved to generate labels and create a tropical cyclone label set.

[0061] (4) Enhance samples and expand the number of sample labels, including changing brightness, adding noise, adding random points, blurring, translating, rotating, and mirroring images.

[0062] (5) Perform data segmentation and divide the data into training data set, verification data set and test data set.

[0063] (6) Set the initialization parameters of the maskrcnn convolutional network structure.

[0064] (7) Deep network structures are prone to gradient vanishing and gradient exploding. After experiments, the Resnet50 deep residual network structure was selected as the backbone feature extraction network to extract the feature map of the original image. It can better identify tropical cyclone clouds and help model convergence. It can not only accelerate the training of ultra-deep neural networks, but also greatly improve the accuracy of deep networks. To a large extent, it avoids the problem of gradient vanishing or gradient exploding caused by the increase in the number of network layers.

[0065] (8)General networks directly use the feature maps of the last layer. Although the feature maps of the last layer have strong semantics, their location and resolution are relatively low. Multi-scale detection is becoming more and more important in object detection. Feature pyramid networks (FPN) fuse feature maps from the bottom layer to the top layer to obtain a new fused feature map that can make full use of features at different scales.

[0066] (9) Each anchor point in the feature map generates different anchor boxes according to the scale and aspect ratio, and maps them to the original image. Improper selection of anchor box size and ratio will lead to model degradation. It is necessary to match the size of the tropical cyclone target to improve model performance.

[0067] (10) The Region Proposal Network (RPN) is trained by stochastic gradient descent and back propagation. The Adam optimizer is used to replace the traditional stochastic gradient descent process to minimize the error. Combined with the batch normalization (BN) layer, the neural network weights are iteratively updated through the training set until convergence. The optimized network structure for tropical cyclone center location and intensity determination based on deep learning is constructed, which can effectively solve the problems of model training convergence difficulties and lack of generalization ability. It is the key to improving the accuracy and time efficiency of the model prediction results of tropical cyclone center location and intensity determination.

[0068] 1) Use momentum and adaptive learning rate to accelerate the convergence of tropical cyclone center location and intensity determination training models;

[0069] 2) It has high computational efficiency and low memory requirements;

[0070] 3) Hyperparameters are intuitive to interpret and require minimal tuning.

[0071] Traversing all the minimum batches on one round of training sets is regarded as one training round (epoch). The Adam optimization calculation formula and parameter settings are as follows:

[0072]

[0073] m t =β1m t-1 +(1-β1)g t

[0074]

[0075]

[0076]

[0077]

[0078] in, represents the derivative of the loss function, i.e., the gradient; t represents the number of minimum batches in the training set; the hyperparameter α represents the learning rate, with a default value of 0.001, which decays with the number of training rounds (epochs); β1 represents the exponential decay rate of the first-order moment estimate, with a default value of 0.9; β2 represents the exponential decay rate of the second-order moment estimate, with a default value of 0.999; the ε parameter is to prevent division by zero during training, and its value is very small, with a value of 10 -8 .f t (θ) represents the random objective function; initialization parameter vector θ, first-order moment vector m t-1 , second-order moment vector v t-1 and epoch, the value is 0. gt Indicates the gradient of the random objective function, m t represents the first-order moment estimate (mean) of the updated gradient, v t represents the second-order raw moment estimate (with biased variance) of the updated gradient, represents the first-order moment estimate for the calculation of bias correction, represents the second-order raw moment estimate for the calculation bias correction, θ t It means that the parameter vector θ is updated iteratively with the above values ​​until θ converges.

[0079] (11) During the model training process, since there are usually only one or a few tropical cyclone targets in the image, and the targets occupy a small proportion of the image, most of the generated candidate regions are negative samples, and there is a serious imbalance between positive (foreground) and negative (background) samples, which will lead to low model accuracy. Therefore, by setting the positive and negative sample ratio, the imbalance of positive and negative samples can be balanced to a certain extent. In addition, it is difficult to distinguish the eyeless tropical cyclone samples in the development and extinction periods of tropical cyclones, while the number of negative samples is large and easy to classify. The negative sample loss value loss dominates the entire gradient descent direction of the loss function. After accumulation, it can drown out the positive samples that account for a small proportion but are difficult to distinguish, causing the model to focus on learning negative samples. When there is an imbalance in the number of positive and negative samples and the difficulty and easy samples, in order to make the model focus on training the positive samples that account for a small proportion and learning the samples that are difficult to learn, the focal loss loss function is introduced and improved to solve the problems of category imbalance and difficulty and easy sample imbalance in multi-classification target detection. The focal loss loss function in multi-classification target detection is defined as follows:

[0080] FL(p t )=-α t (1-p t ) γ log(p t )

[0081] p t It is the probability that the sample in the output result of the Softmax activation function belongs to the true category t, indicating the degree of difficulty in distinguishing the sample. The modulation parameter γ is introduced, and the default value is 2. For samples with a large probability and easy to distinguish, (1-p t ) γ It approaches 0, which reduces the weight of easy-to-classify samples and their contribution to the loss value. For difficult samples with relatively small probabilities, the weight of difficult-to-classify samples is also reduced, reducing their contribution to the loss value. However, compared with difficult-to-classify samples, easy-to-classify samples are attenuated more. Therefore, the contribution of difficult samples to the gradient can be relatively increased by reducing the loss value of easy-to-classify samples.

[0082] At the same time, increasing the modulation parameter α t , the default value is 0.25, which means the weight of the t-th class sample, which will increase the proportion of the loss value (Loss) of the positive sample judged as negative with a small proportion in the total loss value (Loss).

[0083] (12) Take the fused feature map and the ground truth box (GT) as input. For each anchor box, a four-dimensional vector (x, y, w, h) is used to represent it, which represents the center coordinates and width and height of the anchor box respectively. Use the Softmax function to determine the probability of each anchor box being the foreground or background, and obtain the confidence scores for the foreground and background. Calculate the bounding box (BB) deviation between the foreground and the true target, and use the Smooth L1 function to adjust the position of the bounding box regression. The regression offset includes Δx, Δy, Δw, and Δh, so that the predicted box is closer to the true box, thereby generating region proposals. The Smooth L1 function is defined as follows:

[0084]

[0085] Where x represents the coordinate difference between the real box and the predicted box.

[0086] The loss function is defined as follows:

[0087]

[0088] In the formula, p i represents the probability of foreground or background, It means that when the anchor point is a positive sample, Negative samples are 0. i Represents the offset of the predicted box relative to the anchor box, Indicates the offset of the real box relative to the anchor box, L cls represents the classification error, L reg represents the regression error, N cls Represents the number of classification samples, N reg represents the number of regression samples, and λ represents the weight coefficient value.

[0089] Since anchor boxes often overlap, region candidate boxes will also overlap on the same object. In order to solve the problem of repeated region candidate boxes, non-maximum suppression (NMS) is used. Non-maximum suppression sorts the region candidate boxes according to the classification score, selects the highest score and the corresponding bounding box, and traverses the remaining boxes. If the IOU (Intersection over Union) value between it and the current highest-scoring bounding box is greater than a predefined threshold, the region candidate box is removed.

[0090] Iterate the above process and calculate the foreground anchors and bounding box regression offsets to obtain the final proposals.

[0091] (13) Randomly extract region candidate boxes, and map region candidate boxes of different sizes on the feature map to fixed size feature maps through the region of interest alignment (ROI Align) algorithm. ROI Align maps the original image to the feature map. Floating point numbers are retained in the process of region candidate box mapping and generating fixed size feature maps. No quantization and rounding operations are used to eliminate mapping deviations. The basic unit of a pixel is 1. Through the bilinear interpolation method, the pixel values ​​corresponding to the virtual floating point position points in the fixed size feature map are obtained using the four real pixel values ​​around the virtual point in the original image.

[0092] (14) A fully convolutional network (FCN) is used to classify and accurately regress feature maps of a specific size. However, unlike FCN, a sigmoid function is used to calculate the probability of each pixel belonging to each category and the output generates a mask. When calculating the loss, the classification branch predicts which category the pixel belongs to and which category's mask contributes to the loss, and participates in the calculation of the binary cross entropy loss. Compared with the softmax function for multi-category classification and calculation of cross entropy loss, it can avoid inter-class competition and improve performance. The loss function is defined as:

[0093] L=L cls +L box +L mask

[0094] Where, L cls Expressed as classification error, L box Expressed as the bounding box detection error, L mask It is represented as segmentation error.

[0095] (15) Input the geostationary satellite infrared cloud image at any time and the multi-source satellite sea surface wind field fusion image matching time and space, and use the maskrcnn training model to predict whether there is a tropical cyclone target in the image. If it exists, return its spatial position and coverage. The longitude and latitude of the center of the identification box are taken as the result of the tropical cyclone center positioning, and the category of the identification box is taken as the intensity level of the tropical cyclone.

[0096] (16) The distance between the centers of different tropical cyclone cloud systems is not less than 100 km. The average moving speed of a tropical cyclone is 10-20 km / h, and the fastest can reach 30-40 km / h. The distance difference between the predicted tropical cyclone center positioning result at this moment and the tropical cyclone center positioning result at the previous moment is calculated to determine which one is the positioning result of the same tropical cyclone. Therefore, the prediction results with multiple tropical cyclone cloud systems can be corrected according to the historical path of the tropical cyclone at the previous moment, or the non-tropical cyclone cloud systems that are misidentified can be eliminated.

[0097] (17) The accuracy indicators for evaluating tropical cyclone center positioning and intensity determination algorithms include accuracy, precision, recall and F1-score.

[0098] The present invention discloses a method for locating and determining the center of a tropical cyclone based on maskrcnn deep learning. The method includes satellite image processing and preparation of training sample labels: preprocessing the collected geostationary satellite atmospheric window channel remote sensing images, water vapor absorption band channel remote sensing images and multi-source satellite fused sea surface wind field images to prepare training sample labels for locating and determining the center of a tropical cyclone; deep learning training model construction: inputting the training sample labels, adjusting the parameters, iteratively training until the model converges without overfitting, and constructing a training model for locating and determining the center of a tropical cyclone based on maskrcnn deep learning; tropical cyclone center positioning and intensity prediction: inputting the multi-channel satellite remote sensing images to be predicted into the pre-trained tropical cyclone center positioning and intensity model to generate a tropical cyclone positioning and intensity result; prediction result correction and accuracy evaluation: correcting the prediction result with multiple tropical cyclone cloud systems according to the historical path of the tropical cyclone at the previous moment, eliminating the misidentified non-tropical cyclone cloud systems and performing accuracy evaluation. In this way, the identification of tropical cyclone cloud systems and the prediction of center positioning and intensity can be achieved at one time, and the accuracy of center positioning and intensity determination of eyeless tropical cyclone cloud systems in the entire development stage of generation and development, maturity and extinction can be improved.

[0099] Based on the above implementation, a method for locating and determining the center of a tropical cyclone based on maskrcnn deep learning in another implementation provided in the present disclosure specifically includes:

[0100] (1) Obtaining remote sensing image information of the atmospheric window channel, water vapor absorption band channel, and sea surface wind field image information of the target area: Select infrared remote sensing image information of the atmospheric window channel and water vapor absorption band channel of the geosynchronous geostationary orbit satellite, and sea surface wind field image information obtained by multi-source satellite microwave scatterometer. Combine the multi-source satellite sea surface wind field image with the geostationary satellite multi-channel infrared cloud image to improve the accuracy of tropical cyclone positioning and intensity determination.

[0101] (2) performing temporal and spatial information matching processing on the atmospheric window channel remote sensing image information, the water vapor absorption band channel remote sensing image information and the sea surface wind field image information to generate image information to be processed.

[0102] (3) Inputting the image information to be processed into a pre-trained tropical cyclone location and intensity determination model, and outputting the corresponding tropical cyclone location information and cyclone intensity grade result: Inputting the obtained fused image information into a pre-trained tropical cyclone location and intensity determination model, and obtaining the corresponding tropical cyclone center location information and the corresponding cyclone intensity grade result. The cyclone intensity grades include: tropical depression, tropical storm, severe tropical storm, typhoon, severe typhoon, and super typhoon.

[0103] (4) Determine the moving path of the tropical cyclone in the target area based on the output positioning information, and issue a disaster warning based on the output cyclone intensity level result: According to the positioning information output by the tropical cyclone positioning and intensity determination model, calculate the difference between the positioning information of the tropical cyclone at the current moment and the positioning information at the previous moment, that is, the moving distance of the tropical cyclone from the previous moment to the current moment. Eliminate the positioning information at the current moment corresponding to the calculation result greater than the second preset threshold. Generate the moving path of the tropical cyclone based on the eliminated positioning information and the corresponding time information. The interval between the cloud centers of different tropical cyclones is not less than 100km. According to the calculated moving distance of the tropical cyclone from the previous moment to the current moment, it can be determined whether the tropical cyclone at the current moment and the corresponding tropical cyclone at the previous moment belong to the same tropical cyclone, that is, whether the moving distance is greater than 100km. Eliminate the positioning information that does not belong to the same tropical cyclone, so as to facilitate a more accurate analysis of the moving path of the tropical cyclone. Then, match the positioning information belonging to the same tropical cyclone with all the corresponding previous moment information to obtain the moving path of the tropical cyclone. Different levels of warnings are issued based on the obtained tropical cyclone's moving path and the output cyclone intensity level results. In addition, based on the output positioning information, the difference between the tropical cyclone's current positioning information and the previous positioning information can be calculated; the moving speed of the tropical cyclone can be calculated based on the difference, and the tropical cyclone can be classified according to the calculated moving speed; the moving path of the tropical cyclone at the next moment can be predicted based on the classification results; and disaster warnings can be issued based on the predicted moving path and the corresponding cyclone intensity level results.

[0104] Wherein, the pre-trained tropical cyclone positioning and intensity determination model includes obtaining infrared remote sensing image information of the atmospheric window channel and the water vapor absorption band channel of the geosynchronous geostationary orbit satellite in history, and the sea surface wind field image information obtained by the multi-source satellite microwave scatterometer. The historical atmospheric window channel remote sensing image information, the water vapor absorption band channel remote sensing image information and the sea surface wind field image information are matched according to the time and space data, that is, the image information with the same time and space data is matched as the same set of image information. The image information after matching is cropped according to the center positioning point and the preset cropping size rule; the center positioning point is determined according to the tropical cyclone optimal path data set at the corresponding time. The cyclone center and cyclone intensity level identification processing are performed on the cropped multiple image fragments to generate the training samples of the tropical cyclone positioning and intensity determination model. The training samples of the tropical cyclone positioning and intensity determination model are input into the maskrcnn model for deep learning, and the cyclone center and cyclone intensity level results of the samples of the tropical cyclone positioning and intensity determination model are output. When the difference between the output result and the identification is greater than the first preset threshold, the parameters of the maskrcnn model are corrected. Repeat the above process until the difference between the output result and the identification is less than the first preset threshold, and the learning of the samples of the tropical cyclone positioning and intensity determination model is completed. For example, after 300 iterations, the accuracy of the training set and the accuracy of the verification set gradually stabilize, and eventually reach more than 92%. The loss value of the training set and the loss value of the verification set tend to be stable, and finally drop to 0.15, indicating that the model has good convergence performance.

[0105] In some embodiments, the diameter of a tropical cyclone is generally 600-1000km, and the largest can reach 2000km. The center point is located based on the central longitude and latitude of the tropical cyclone at each moment in the tropical cyclone best path data set, and the image information after matching processing is cropped according to a size of 512x512. The center point of the annotation box is defined as the center point of the tropical cyclone, and the six level types of the cyclone intensity level are used as identification information to identify the multiple image fragments obtained by cropping. Among them, the size of the annotation box can be set to 100x100. Thereby forming a training sample of a tropical cyclone positioning and intensity determination model with identification information.

[0106] In some embodiments, the cropped multiple image segments are subjected to brightness change, noise addition, random point addition, image blurring, translation, rotation, and mirroring processing to enrich the training sample set, so that the trained tropical cyclone location and intensity determination model is more accurate and more flexible.

[0107] In some embodiments, the training samples of the tropical cyclone positioning and intensity determination model are input into the maskrcnn model for deep learning, and the Resnet50 deep residual network structure extraction network is selected to extract the feature map of the original image, which can better identify the tropical cyclone cloud system and help the model convergence. It can not only accelerate training, but also greatly improve the accuracy of the deep network, and to a large extent avoid the problem of gradient vanishing or gradient exploding caused by the increase in the number of network layers.

[0108] In some embodiments, multi-scale detection is used to fuse the feature maps to obtain a fused new feature map, which can make full use of features of different scales. Different anchor frames are generated according to the scale and aspect ratio, and mapped to the original image, and the target size that fits the tropical cyclone is selected to improve the model performance and model convergence efficiency. The network is trained by the stochastic gradient descent method and the back propagation method to take the fused feature map and the true frame as input. For each anchor frame, a four-dimensional vector (x, y, w, h) is used to represent it, which respectively represents the center point coordinates and width and height of the anchor frame. The Softmax function is used to judge the probability of the foreground or background of each anchor frame, and two confidences for the foreground and background are obtained. The Smooth L1 function is used to adjust the position of the bounding box regression, and the bounding box deviation value between the foreground or background and the true target is calculated. The regression offset includes Δx, Δy, Δw, and Δh, so that the bounding box is closer to the true box to generate a candidate area box. The Smooth L1 function is defined as follows:

[0109]

[0110] Where x represents the coordinate difference between the real box and the candidate box.

[0111] The loss function is defined as follows:

[0112]

[0113] In the formula, p i represents the probability of foreground or background, It means that when the anchor point is a positive sample, When the anchor point is a negative sample, t i Represents the offset of the bounding box relative to the anchor box, Indicates the offset of the real box relative to the anchor box, L cls represents the classification error, L reg represents the regression error, N cls Represents the number of classification samples, N reg represents the number of regression samples, and λ represents the weight coefficient value.

[0114] In some embodiments, the candidate region boxes are also repeatedly processed. Non-maximum suppression is used to sort the candidate region boxes according to the classification scores, the highest score and the corresponding bounding box are selected, and the remaining boxes are traversed. If the IOU value between it and the current highest score bounding box is greater than a predefined threshold, the candidate region box is eliminated, and the above process is iterated to obtain the final selected region box by combining the foreground tracing box and the bounding box offset.

[0115] In some embodiments, it also includes balanced processing of positive and negative samples. Since a tropical cyclone usually has only one target in an image, and the target accounts for a small proportion of the image, most of the generated candidate areas will be negative samples, and there will be a serious imbalance between positive and negative samples, which will result in a low model accuracy. Therefore, the region of interest alignment algorithm is used to randomly extract candidate area frames, and the positive and negative sample ratios are set, which can balance the imbalance of positive and negative samples to a certain extent. The region of interest alignment algorithm is used to map candidate area frames of different sizes on the feature map to feature maps of specific sizes. Floating point numbers are retained in the process of mapping the candidate area feature map and generating fixed-size feature maps. No quantization operation is used. The bilinear interpolation method is used to use the four real pixel values ​​around the virtual point in the original image to obtain the pixel value corresponding to the virtual floating point position point in the fixed-size feature map.

[0116] In some embodiments, the feature map of a specific size is classified and accurately regressed through a fully convolutional network structure to determine which category a pixel belongs to, and the binary cross entropy loss of the category is calculated, which can avoid inter-class competition and improve performance. The loss function is defined as:

[0117] L=L cls +L box +L mask

[0118] Where, L cls Expressed as classification error, L box Expressed as the bounding box detection error, L mask It is represented as segmentation error.

[0119] The present invention provides a method for locating and strengthening the center of a tropical cyclone based on maskrcnn deep learning. Based on the physical quantities of multi-source satellite sea surface wind field images and combined with multi-channel infrared cloud images of geostationary satellites, maskrcnn target detection deep learning is used to realize the positioning and strengthening of the center of a tropical cyclone. In this way, the recognition of tropical cyclone cloud systems and the prediction of center positioning and strengthening can be realized at one time, and the accuracy of center positioning and strengthening of eyeless tropical cyclone cloud systems in the entire development stages of generation and development, maturity and extinction can be improved.

[0120] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

Claims

1. A method for locating and determining the center of a tropical cyclone based on maskrcnn deep learning. It is characterized in that include: Satellite image processing and training sample labeling: Preprocess the collected geostationary satellite atmospheric window channel remote sensing images, water vapor absorption band channel remote sensing images, and multi-source satellite fusion sea surface wind field images to produce tropical cyclone center location and intensity determination training sample labels; Deep learning training model construction: input training sample labels, adjust parameters, iterate training until the model converges and there is no overfitting, and construct a tropical cyclone center positioning and intensity determination training model based on maskrcnn deep learning; the deep learning training model construction process includes: pre-setting the initialization parameters of the tropical cyclone center positioning and intensity determination target detection convolutional network structure; during the model training process, adjusting the parameters of the tropical cyclone center positioning and intensity determination convolutional network structure; transforming and optimizing the tropical cyclone center positioning and intensity determination target detection maskrcnn network structure; inputting the training sample labels into the tropical cyclone center positioning and intensity determination convolutional network structure for model training and verification; calculating the focal loss between the prediction results of the training set and the true label to obtain the fitting error, and minimizing the error using the Adam optimizer; iteratively repeating the above process until the model training and verification errors converge, the loss values ​​of the training set and the verification set tend to be stable, and when the accuracy of the training set and the verification set is greater than or equal to the preset threshold, the sample learning of the tropical cyclone center positioning and intensity determination model is completed, and a tropical cyclone center positioning and intensity determination training model based on maskrcnn deep learning is constructed; The process of model training and verification of the convolutional network structure for tropical cyclone center location and intensity determination includes: using the adaptive moment estimation Adam optimizer to minimize the error, combined with the batch normalization layer, traversing the training set to iteratively update the neural network weights until convergence, traversing all the minimum batches on one round of training set as one training round, and the Adam optimization calculation formula and parameter settings are as follows: m t =b 1 m t-1 +(1-β 1 )g t in, represents the derivative of the loss function, i.e., the gradient; t represents the number of minimum batches in the training set; the hyperparameter α represents the learning rate, which decays with the number of training rounds; β 1 represents the exponential decay rate of the first-order moment estimate, β 2 represents the exponential decay rate of the second-order moment estimate; the ε parameter represents a very small value added when the divisor is zero during training; f t (θ) represents the random objective function; the other parameters are the initialization parameter vector θ, the first-order moment vector m t-1 , second-order moment vector v t-1 ; g t Indicates the gradient of the random objective function, m t represents the first-order moment estimate of the updated gradient, v t represents the second-order raw moment estimate of the updated gradient, represents the first-order moment estimate for the calculation of bias correction, represents the second-order raw moment estimate for the calculation bias correction, θ t It means that the parameter vector θ is updated iteratively with the above value until θ converges; The focal loss function is used to balance the samples. The focal loss function in multi-classification target detection is defined as follows: FL(p t )=-a t (1-p t ) γ log(p t ) p t is the probability that the sample in the output result of the Softmax activation function belongs to the true category t, indicating the degree of difficulty in distinguishing the classification of samples, the modulation parameter γ, indicating the degree of cross entropy attenuation for the t-th class sample, and the modulation parameter α t , represents the weight of the t-th class sample; The parameter adjustment process of the convolutional network structure for tropical cyclone center positioning and strength determination includes: selecting a suitable deep residual network structure to extract the feature map of the original image; setting the size and aspect ratio of the anchor frame; setting the ratio of positive and negative samples; adjusting the corresponding parameters of the candidate frame of the generated area; adjusting the corresponding parameters of the candidate frame of the excluded overlapping area; Tropical cyclone center location and intensity prediction: multi-channel satellite remote sensing images to be predicted are input into a pre-trained tropical cyclone center location and intensity prediction model to generate tropical cyclone center location and intensity prediction results; Correction of forecast results and accuracy evaluation: Correct the forecast results with multiple tropical cyclone cloud systems according to the historical paths of tropical cyclones at the previous moment, or eliminate misidentified non-tropical cyclone cloud systems, and conduct accuracy evaluation.

2. The method according to claim 1, It is characterized in that The satellite image processing and training sample labeling process includes: Collecting historical images, the historical images include atmospheric window channel remote sensing images, water vapor absorption band channel remote sensing images and sea surface wind field images; Performing matching processing on the historical images; According to the tropical cyclone center positioning point and the preset cropping size rule, the matched image is cropped to generate a tropical cyclone sample set; the center positioning point is determined according to the tropical cyclone case at the corresponding time of the tropical cyclone best path data set; Automatically annotate the tropical cyclone center and different levels of tropical cyclone intensity on the cropped images to generate a tropical cyclone label set; Data segmentation is performed to generate a training data set for the tropical cyclone center location and intensity determination model.

3. The method according to claim 2, It is characterized in that The automatic labeling of the tropical cyclone center and different levels of tropical cyclone intensities on the cropped image includes: Automatic annotation and label generation are achieved by modifying the annotation box and image properties.

4. The method according to claim 2, It is characterized in that The method further includes: cropping the image after matching processing according to the tropical cyclone center positioning point and the preset cropping size rule to generate a tropical cyclone sample set; The cropped image is processed by changing brightness, adding noise, adding random points, blurring, translating, rotating, and mirroring.

5. The method according to claim 1, It is characterized in that The prediction result correction and accuracy evaluation include: Calculate the time difference between the predicted moment of a tropical cyclone and the previous moment and the distance difference between the tropical cyclone center location; Based on the above calculation results, according to the spacing distances between the centers of different tropical cyclone cloud systems and the moving speed of the same tropical cyclone, we jointly judge and classify them, and correct the prediction results of multiple tropical cyclone cloud systems, or eliminate the misjudged non-tropical cyclone cloud systems.