Space-time two-dimensional low-altitude thunderstorm prediction method based on incremental learning

Abstract

The invention discloses a space-time two-dimensional low-altitude thunderstorm prediction method based on incremental learning, and relates to the technical field of artificial intelligence and meteorology crossing, and the method comprises the steps: collecting electromagnetic wave image data, forming T electromagnetic wave data sets for low-altitude thunderstorm prediction, and obtaining T electromagnetic wave data sets for low-altitude thunderstorm prediction when each increment batch arrives; a new increment module and a current increment batch prediction model are established, Transform and a CNN feature extractor are adopted to carry out space-time two-dimensional feature extraction on electromagnetic wave image data, an increment learning method is utilized to carry out prediction model iteration training on multiple batches of electromagnetic wave image data, and on the premise that the model does not lose learned knowledge, the prediction efficiency of the current increment batch prediction model is improved. According to the method, a new category of electromagnetic waveform data is gradually learned, so that the identification and prediction capability of the finally obtained prediction model which is trained in the invention is ensured, the defect of fixed model parameters is avoided, and the classification precision of the method is improved.

Description

Technical Field

[0001] The present invention relates to the field of intersectional technology of artificial intelligence and meteorology, and in particular to a method for predicting low-altitude thunderstorms in time and space dual dimensions based on incremental learning. Background Art

[0002] Low-altitude thunderstorms are a typical severe convective weather phenomenon, accompanied by strong winds, lightning, short-term heavy rainfall and other severe weather conditions, posing a potential threat to aviation safety, public transportation and many other fields. In today's highly developed urbanization and industrialization, the frequent and sudden occurrence of low-altitude thunderstorms makes their prediction an important task to protect the safety of life and property and maintain the stable operation of key infrastructure. During the occurrence of low-altitude thunderstorms, the charges inside and outside the thunderstorm cloud layer continue to accumulate, transfer and release, and the electric field intensity will fluctuate significantly. This complex electric field change will produce a series of electromagnetic waveforms with specific characteristics, such as IC (in-cloud flash), NBE (narrow bipolar event), PB (precursor breakdown) and RS (return stroke). The appearance of these waveforms often indicates the formation of low-altitude thunderstorms. Therefore, accurate identification of electromagnetic waveforms can provide valuable early warning or alarm information for thunderstorm prediction.

[0003] Although traditional deep learning algorithms can also achieve accurate classification of different types of waveform data, they are limited by the shortcomings of fixed models and parameters. When the classification model encounters new categories, the traditional deep learning algorithm has the problem of low training efficiency or catastrophic forgetting. Taking lightning waveform classification as an example, assuming that the basic classification model is trained on two types of data, IC and NBE, the basic model can effectively classify these two types of data. If the detection station later captures a batch of PB and RS types of data that need to be classified, there are two technical routes for traditional deep learning. The first is to discard the trained basic classification model and put the old and new categories of data together to retrain a model. The trained model has high accuracy, but does not make full use of the basic model. In addition, retraining the model will consume a lot of computing resources and time when the data volume is large and the training time is long, so the training efficiency is low. The second is to use the new category data to fine-tune the basic model. This reduces the training cost to a certain extent, but in the fine-tuning process, the model often tends to favor the new category of data, resulting in a decrease in the classification ability of the original category, that is, the model is prone to catastrophic forgetting of the old category knowledge, and the classification accuracy is low. Summary of the invention

[0004] In order to overcome the defects of low classification accuracy of newly added electromagnetic wave image categories in the above-mentioned prior art and fixed models and parameters of traditional deep learning algorithms, the present invention proposes a low-altitude thunderstorm prediction method in dual dimensions of time and space based on incremental learning.

[0005] To achieve the above object, the present invention adopts the following technical solution: a low-altitude thunderstorm prediction method based on time and space dual dimensions of incremental learning, comprising: S1: Collect electromagnetic wave images to form T electromagnetic wave datasets {D0, D1, ..., D t ,...,D T-1}; where D t Indicates The electromagnetic wave dataset of the new category in the incremental batch, T-1 is the total number of incremental batches, when t=0, it represents the initial electromagnetic wave dataset; S2: When the tth incremental batch arrives, a new incremental module is established , and based on the total electromagnetic wave data set D after the tth incremental batch, i.e. the current incremental batch t model and the previous incremental batch prediction model F t-1 (x) is trained to obtain the current incremental batch prediction model F t (x); where D t model ={D0∪D1...D t}, x represents the electromagnetic wave image; According to step S2, after T-1 incremental batches, the final prediction model F is obtained. T-1 (x); S3: Input the electromagnetic wave image to be measured into the final prediction model F T-1 (x) to obtain the predicted category.

[0006] Preferably, the previous incremental batch prediction model F t-1 (x) consists of a feature extractor and a linear classifier, expressed as follows: ; Among them, Ф t-1 (·) is the feature extractor of the previous incremental batch, and , D is the original feature dimension size of the electromagnetic wave image x, d is the feature dimension size after being extracted by the feature extractor, W t-1 is the linear classifier parameter of the previous incremental batch, and , Indicates the total number of categories in the t-1th incremental batch stage, with a superscript T Indicates transpose.

[0007] Preferably, the current incremental batch prediction model F t (x) is the prediction model F of the previous incremental batch t-1 (x) and new incremental modules Combined, it is expressed as follows: ; Among them, x is the electromagnetic wave image, y is the true value of the category of the electromagnetic wave image, Indicates the use of cross entropy loss function for the new incremental module Conduct training, D t train is the total electromagnetic wave dataset D after the current incremental batch t model The training set obtained in , argmin(.) represents the function of finding the optimal parameters, and E is the expected function.

[0008] Preferably, the new incremental module It can be expressed as: ; Among them, φ t (·) is the feature extractor of the new incremental module, which is composed of the CNN spatial feature extractor φ S t (·) and the transformer temporal feature extractor φ t TE (·) composition, and ; ;x s 、x TE are the normalized matrix and waveform time matrix of the electromagnetic wave image after spatial dimension and time dimension data preprocessing respectively; d s ,d TE They are respectively passed through the CNN spatial feature extractor φ t S (·) and the transformer temporal feature extractor φ t TE (·) Feature dimension size after extraction; are the linear classifier parameters of the new incremental module, and , , represents the total number of categories in the tth incremental batch stage, are the linear classifier parameters corresponding to the old categories in the new incremental module, ; are the linear classifier parameters corresponding to the new categories in the new incremental module, .

[0009] Preferably, the predicted category The calculation formula is: ; Among them, 0 is the matrix For zero matrices of the same size, S(.) is softmax, which is a normalized exponential function.

[0010] Preferably, the new incremental module Through continuous training, the optimal parameters are obtained , the optimal parameters The expression is: ; Among them, Dis(.) is Distance(.), which means the function of finding the distance; Indicates a parameter.

[0011] Preferably, the current incremental batch prediction model F t (x) is expressed as: ; Among them, Ф t (·) is the current incremental batch feature extractor; W t Predict model F for the current incremental batch t (x) linear classifier parameters; and are the linear classifier parameters corresponding to the old and new categories in the incremental module respectively; φ t (·) is the feature extractor of the new incremental module; γ1, γ2 are scaling factors, and 0<γ1<1, 0<γ2<1, γ is the diagonal matrix composed of γ1 and γ2; the diagonal matrix composed of γ1 and γ2 can be expressed as: ; Among them, S old is the number of old category samples in the total electromagnetic wave dataset after the current incremental batch, S new It is the number of new category samples in the total electromagnetic wave dataset after the current incremental batch.

[0012] Preferably, the current incremental batch prediction model F t The loss function of (x) The calculation formula is: ; Among them, α1, α2, α3 are hyperparameters, and α1+α2+α3=1, is the first loss function, is the second loss function, is the distillation loss function, y is the true value of the electromagnetic wave category, is the cross entropy loss function, S(.) is the softmax, i.e. the normalized exponential function, W t fe is a new linear classifier parameter.

[0013] Preferably, the acquisition of the current incremental batch prediction model includes: d1: The current incremental batch prediction model F t (x) is set as the teacher model, and the student model is set at the tth incremental batch stage ; Wherein, the feature extractor of the student model is also composed of a CNN spatial feature extractor and a transformer temporal feature extractor; d2: Using the second distillation loss function Model for students Conduct training; d3: Use the trained student model as the final current incremental batch prediction model; The student model It can be expressed as: ; in, The student model for the tth incremental batch stage Linear classifier parameters, , Student model for the current incremental batch stage The feature extractor of They are respectively the student models of the current incremental batch stage CNN spatial feature extractor and transformer temporal feature extractor, , , And d=d s +d TE , x s 、x TE are the normalized matrix and waveform time matrix of the electromagnetic wave image after spatial dimension and time dimension data preprocessing respectively; d s ,d TE After CNN spatial feature extractor and transformer temporal feature extractor The size of the feature dimension after extraction; Represents the total number of categories in the tth incremental batch stage.

[0014] Preferably, the processing process of the current incremental batch prediction model includes: A1: Preprocess the electromagnetic wave image in spatial dimension and time dimension respectively to obtain the image preprocessing matrix, which includes a normalization matrix and a waveform time matrix; A2: Input the image preprocessing matrix into the spatiotemporal feature extractor in the current incremental batch prediction model to obtain a temporal feature matrix and a spatial feature matrix; the spatiotemporal feature extractor includes a CNN spatial feature extractor and a Transformer temporal feature extractor; A3: Input the temporal feature matrix and the spatial feature matrix into the feature splicing module for feature splicing to obtain the temporal and spatial feature matrix; A4: Input the spatiotemporal feature matrix into the linear classifier to obtain the predicted category.

[0015] The advantages of the present invention are: (1) The present invention uses Transformer and CNN feature extractors to extract features of electromagnetic wave images in both time and space dimensions, and uses an incremental learning method to iteratively train prediction models for multiple batches of electromagnetic wave image data using waveform data of different categories. Under the premise that the model does not lose the learned knowledge, new categories of waveform data are gradually learned, thereby ensuring the recognition and prediction capabilities of the current incremental batch prediction model in the present invention, avoiding the defect of fixed model parameters, and improving the classification accuracy of the method of the present invention.

[0016] (2) The present invention first preprocesses the electromagnetic wave image in terms of spatial and temporal dimensions to obtain a normalized matrix and a waveform-time matrix, and then extracts features in terms of both spatial and temporal dimensions of the electromagnetic wave image, thereby reducing unnecessary channel information and computational complexity, and improving the prediction speed of the method of the present invention.

[0017] (3) The present invention utilizes feature enhancement technology to extract key features of electromagnetic wave images in different incremental batches, thereby avoiding the strong classification bias of the model caused by the imbalance of new and old category waveform data, and improving the accuracy of the model in electromagnetic waveform classification tasks.

[0018] (4) The present invention utilizes feature compression technology to reduce redundant information of electromagnetic wave image features, thereby improving the computational efficiency and prediction accuracy of the prediction model in the method of the present invention.

[0019] (5) The present invention utilizes the first loss function, the second loss function and the distillation loss function, and adds hyperparameters between the three to form a loss function , using the loss function The current incremental batch prediction model is trained and the model parameters are gradually optimized to avoid overfitting of the model to old category waveform data with smaller data volume, thereby improving the accuracy of the model in electromagnetic waveform classification tasks.

[0020] (6) The present invention sets the current incremental batch prediction model as the teacher model and sets the student model at the tth incremental batch stage. Based on the teacher model, the student model is trained using the second distillation loss function. The trained student model is used as the final current incremental batch prediction model, thereby avoiding the rapid growth of the number of parameters and feature dimensions of the model when the new incremental modules are gradually added. This makes the model suitable for incremental learning tasks with multiple incremental batch data for low-altitude thunderstorm prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a model architecture diagram of the present invention; Figure 2 is a flow chart of the prediction method of the present invention; Figure 3 This is a flow chart of the model architecture method of the present invention; Figure 4 is the test accuracy of the method of the present invention when the incremental batches are 0, 1, and 2; Figure 5 This is a comparison chart of the test accuracy of the method of the present invention and the fine-tuning method (finetune) when the incremental batch is 2. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Example 1 like Figure 1-2 As shown, the present invention proposes a low-altitude thunderstorm prediction method based on time and space dual dimensions of incremental learning, including: S1: Collect electromagnetic wave images generated by changes in electric field intensity, perform manual selection and labeling, and form T electromagnetic wave data sets {D0, D1, ..., D t ,...,D T-1}, where D t Indicates The electromagnetic wave dataset of the new category in the incremental batch, T-1 is the total number of incremental batches, when t=0, it represents the initial electromagnetic wave dataset; Each electromagnetic wave dataset contains electromagnetic wave images and corresponding categories, and the categories in each electromagnetic wave dataset do not overlap; S2: When each incremental batch arrives, establish the current incremental batch prediction model F t(x), using the electromagnetic wave dataset D after the current incremental batch t model Perform training to obtain the trained current incremental batch prediction model F t (x); The current incremental batch refers to the tth incremental batch, and the total electromagnetic wave data set D after the tth incremental batch is t model It is a mixture of old category data and new category data, that is, D t model ={D0∪D1...D t}; Current incremental batch prediction model F t (x) is the prediction model F in the previous incremental batch t-1 (x), combined with the training of the new category electromagnetic wave data set in the t-th incremental batch, the previous incremental batch is the t-1th incremental batch; D t model Divide into training set D in a ratio of 8:2 t train With the test set D t test Used for model training and testing respectively; Finally, the final prediction model F is obtained after training. T-1 (x); S3: Input the electromagnetic wave image to be measured into the final prediction model F T-1 (x) to obtain the predicted category.

[0024] The current incremental batch prediction model F t (x) is trained in the tth incremental batch stage. In the tth incremental batch stage, the prediction model F of the t-1th incremental batch (the previous incremental batch) is saved. t-1 (x), the t-1th incremental batch prediction model F t-1 (x) consists of a feature extractor and a linear classifier, expressed as follows: ;

[0025] Among them, Ф t-1 (·) is the feature extractor of the previous incremental batch, and , D is the original feature dimension size of the electromagnetic wave image data sample x, W t-1 is the linear classifier parameter of the previous incremental batch, and , d represents the feature dimension size after being extracted by the feature extractor, Indicates the total number of categories in the t-1th incremental batch stage, with a superscript T Indicates transpose.

[0026] During the low-altitude thunderstorm forecasting process, when a new type of electromagnetic wave image arrives, fine-tuning or simply freezing F t-1 (·) will reduce the model's ability to process old data or lose its plasticity to new data. Inspired by the model accumulation, the present invention trains a new incremental module in the tth incremental batch stage Make incremental predictions, expressed as: ;

[0027] Among them, φ t (·) is the feature extractor of the new incremental module, and ; are the linear classifier parameters of the new incremental module, and ; Represents the new incremental module linear classifier parameters, whose size is the old category linear classifier parameters Size and new batches arrive, new type of data linear classifier parameters Size splicing, that is The parameter corresponding to the size of the linear classifier parameters of the old category is used to learn to re-mine the old category features after the new incremental batch arrives; is the linear classifier parameter for classifying new types of data after the new batch arrives; Represents the total number of categories in the tth incremental batch stage.

[0028] Because electromagnetic wave image data has strong spatial and temporal features, the present invention adopts the space-time dual-dimensional classification technology. The pre-processed electromagnetic wave image data is input into the CNN spatial feature extractor and the transformer temporal feature extractor to obtain the spatial feature matrix and temporal feature matrix of the electromagnetic wave image respectively. The two are spliced ​​to form the space-time feature matrix of the electromagnetic wave image data. Therefore, it can be further known that the new incremental module The feature extractor φ in t (x) is extracted by CNN spatial feature extractor φ t S (x s ) and the transformer temporal feature extractor φ t TE (x TE ) together, so the new incremental module It can be expressed as: ;

[0029] in, , , x s 、x TEare the normalized matrix and waveform time matrix of the electromagnetic wave image after spatial dimension and time dimension data preprocessing respectively; d s ,d TE They are respectively passed through the CNN spatial feature extractor φ t S (·) and the transformer temporal feature extractor φ t TE (·) Feature dimension size after extraction.

[0030] Prediction model F based on the previous incremental batch (t-1 incremental batch) t-1 (x) and new incremental modules , get the current incremental batch prediction model F t (x), which is represented as follows: ;

[0031] Among them, x is the electromagnetic wave image, y is the true value of the electromagnetic wave category, Indicates the use of cross entropy loss function for the new incremental module For training, argmin(.) represents the function for finding the optimal parameters, and E is the expected function.

[0032] The final prediction category of the present invention The calculation formula is: = ;

[0033] Among them, 0 is The same size 0 matrix is ​​used to unify the linear classifier parameter dimensions. S(.) is the softmax, that is, the normalized index, and W T t-1 With Ф t-1 (·) are the previous incremental batch prediction model F t-1 (x) Frozen parameters (at the tth incremental batch stage, they will not change during training), the trainable modules are , , .

[0034] New incremental modules The optimization goal is to make the true value consistent with the current incremental batch prediction model F t The gap between the class prediction values ​​of (x) is the smallest, and the new incremental module Through continuous training, the optimal parameters are obtained , the optimal parameters The expression is: ; Among them, Dis(.) is Distance(.), which means the function of finding the distance; Indicates a parameter.

[0035] Therefore, during training, the new incremental module The optimization process can be described as finding an initial loss function that is continuously reduced The process of initial loss function The calculation formula is: .

[0036] Example 2 Based on Example 1, Example 2 is a second implementation of the present invention. The current incremental batch prediction model F t (x), can be expressed as: .

[0037] In the present invention, because the electromagnetic wave data set is large in size, in order to save memory capacity, when mixing the new and old electromagnetic wave image data, only part of the old electromagnetic wave image data is used. Incremental batch stage, category space , set the memory data capacity to M, set C t for The number of waveform categories in the image is S, that is, after the t-1 incremental batch is completed, the number of samples of electromagnetic wave images saved in each category is S t =M / C t , forming the old data set ,in, This is the old electromagnetic wave dataset after deletion.

[0038] Therefore In the incremental batch stage, as new categories of electromagnetic wave images increase, the model will obtain a dataset with an unbalanced ratio of new and old categories. , D t model Divide into training set D in a ratio of 8:2 t train With the test set D t test They are used for model training and testing respectively. Similarly, D t train The old electromagnetic wave images are also far less than the new electromagnetic wave images. t train The imbalance will lead to a strong classification bias in the model of the present invention, that is, the model will tend to ignore the smaller number of old electromagnetic wave image data. Therefore, in order to strengthen the learning of old class sample data and alleviate the classification bias, the present invention uses the current incremental batch prediction model of A scaling factor is added to the output, resulting in the new Output, therefore, the current incremental batch prediction model F t (x) can be expressed as: ;

[0039] Among them, 0<γ1<1, 0<γ2<1, γ is the diagonal matrix composed of γ1 and γ2. γ1 and γ2 are obtained according to the number of samples contained in the new and old classes of the current electromagnetic wave dataset. The diagonal matrix composed of γ1 and γ2 can be expressed as: ;

[0040] Among them, S old is the number of old category samples in the total electromagnetic wave dataset after the current incremental batch, that is, S old =M,S new is the number of new category samples in the total electromagnetic wave dataset after the current incremental batch. Because the new category data in the electromagnetic wave dataset is much larger than the old category data, there is γ 1< γ2. Through this scaling strategy, the absolute value of the logit value of the old waveform category decreases, and the absolute value of the logit value of the new waveform category increases, thereby forcing the current incremental batch prediction model F t (x) produces larger old category logit values ​​and smaller new category logit values ​​to alleviate classification bias. Therefore, in the basic loss function After adding the scaling factor, we get the first loss function : .

[0041] Simply let the new incremental module Fit F t and F t-1 The residuals of F are sometimes not enough, for example, in extreme cases, t and F t-1 The residual is 0. Therefore, the new incremental module should be further encouraged to learn the new category battery wave image data and initialize a new linear classifier W t fe Learning new features, second loss function for: .

[0042] Simply using the true value of the category for training in an unbalanced dataset may lead to overfitting of the old waveform data with a small amount of data. The mitigation method is to use knowledge distillation to encourage F t On the old wave category with F t-1 Keeping similar outputs, distillation loss function for: .

[0043] In summary, the total loss function is for: ;

[0044] Among them, α1, α2, and α3 are hyperparameters, and α1+α2+α3=1. The larger the value, the more important the corresponding loss function is. In the incremental batch phase, the entire training process is repeated in multiple rounds of iterations, through multiple forward propagations and the total loss function The calculation, back propagation and parameter update cycle operation gradually optimizes the current incremental batch prediction model parameters so that the total loss function Gradually decrease, thereby gradually improving the accuracy of the model in the waveform classification task. After the iteration, the prediction model F of the tth incremental batch stage is obtained. t (x).

[0045] In this embodiment, α1=α2=0.3, α3=0.4, which means that the optimization process pays more attention to the output similarity of the new and old models on the old type waveform data. This setting can effectively slow down catastrophic forgetting in the incremental process.

[0046] Example 3 As a third implementation manner of the present invention, the acquisition of the current incremental batch prediction model includes: d1: The current incremental batch prediction model F in the second embodiment t (x) is set as the teacher model, and the student model is set at the tth incremental batch stage ; Wherein, the feature extractor of the student model is also composed of a CNN spatial feature extractor and a transformer temporal feature extractor; d2: Based on the teacher model, using the second distillation loss function Model for students Conduct training; d3: Use the trained student model as the final current incremental batch prediction model.

[0047] By using this embodiment 3 to predict low-altitude thunderstorms, the model avoids a rapid increase in the number of parameters and feature dimensions when gradually adding new incremental modules, making it suitable for incremental learning tasks when applying multiple incremental batches of data for low-altitude thunderstorm prediction.

[0048] The student model It can be expressed as: ;

[0049] in, The student model for the tth incremental batch stage Linear classifier parameters, The student model for the tth incremental batch stage The feature extractor of , are the student models in the tth incremental batch stage CNN spatial feature extractor and transformer temporal feature extractor, d=d s +d TE , x s 、x TE They are respectively the normalized matrix and waveform time matrix of the electromagnetic wave image after the spatial dimension and time dimension data preprocessing; The second distillation loss function The calculation formula is: .

[0050] Example 4 Based on the above-mentioned embodiment 1, 2 or 3, the current incremental prediction model processing process includes: A1: Preprocess the electromagnetic wave images in the electromagnetic wave data set in terms of spatial dimension and time dimension respectively to obtain an image preprocessing matrix, which includes a normalization matrix and a waveform time matrix; The preprocessing of the spatial dimension includes: In order to reduce unnecessary channel information and computational complexity, the electromagnetic wave image is grayed and converted into a grayscale waveform image; all grayscale waveform images are adjusted to a uniform size to ensure that the model receives the same input size; then the grayscale waveform image is converted into a tensor form to meet the input requirements of the deep learning framework; the mean and standard deviation std of the grayscale waveform image are calculated for normalization to obtain a normalized matrix containing waveform features.

[0051] In an embodiment of the present invention, the uniform size is set to 32×32, so the number of pixel blocks p=32×32, and the batch size is set to 128, so after preprocessing of the spatial dimension, the input of the spatiotemporal feature extractor in the incremental prediction model is of size (128, 1, 32, 32).

[0052] Batch size: The advantage of deep learning is that it is good at processing large-scale data and can process a batch of data at one time. Taking image data as an example, in one classification, the model can process or classify 128 images at a time, so its batch size is 128.

[0053] The preprocessing of the time dimension includes: After the preprocessing of the spatial dimension is completed, the normalized matrix is ​​sliced ​​along the time series (horizontal axis of the image) and divided into S segments of the same size. Assuming that the image size is ( ), the size of each fragment is , the pixel information in each segment is regarded as a feature vector of a time step. Then each segment is flattened and stacked together to obtain a waveform time matrix of S×P / S. In this way, a waveform image is converted into a waveform feature matrix containing a sequence of waveform time features, which is used as the input of the subsequent transformer time feature extractor.

[0054] In the embodiment of the present invention, S=16 is set, that is, each waveform image is cut into 16 32×2 feature vectors, which are flattened to obtain 16 1×64 feature vectors, and these 16 feature vectors are stacked to obtain a 16×64 waveform feature matrix. The input size of the transformer temporal feature extractor is (128, 16, 64).

[0055] A2: Input the image preprocessing matrix into the spatiotemporal feature extractor in the incremental prediction model to obtain a temporal feature matrix and a spatial feature matrix; wherein the spatiotemporal feature extractor includes a CNN spatial feature extractor and a Transformer temporal feature extractor; A3: Input the temporal feature matrix and the spatial feature matrix into the feature splicing module for feature splicing to obtain the temporal and spatial feature matrix; A4: Input the spatiotemporal feature matrix into the linear classifier to obtain the predicted category.

[0056] The following specific experiments are combined to verify the effectiveness and superiority of the above-mentioned low-altitude thunderstorm prediction method in both time and space dimensions based on incremental learning.

[0057] There are four types of electromagnetic wave images collected by the present invention: IC, NBE, PB, RS During the experiment, the four waveform categories were randomly sorted, and the electromagnetic wave image data of the first two categories after sorting were selected as the initial data set D0, and the electromagnetic wave images of the last two waveform categories were set as D1 and D2 respectively. IC: 0, NBE: 1, PB: 2, RS: 3 were set. After random sorting, the list [1, 2, 3, 0] was obtained, that is, the electromagnetic wave images of NBE and PB types were used as the initial data set to train the basic classification prediction model, and RS and IC were the new categories that came later.

[0058] In the experiment, the memory data capacity M is set to 1200, the learning rate is 0.1, the weight decay is 0.0005, the training round epoch is 30, and the resnet32 (spatial) and vit-base (temporal) feature extractors are used to extract spatiotemporal features of electromagnetic wave images.

[0059] The batch size is set to 128. During the preprocessing process, the image is resized to 32*32, so the input size of the CNN spatial feature extractor is 128*1*32*32 (batch size*number of channels*width*length). The electromagnetic wave image is sliced ​​from left to right, and the number of slices is set to 16, that is, each waveform image is cut into 16 32*2 feature vectors, which are flattened to obtain 16 1*64 feature vectors. These 16 feature vectors are stacked to obtain a 16*64 feature matrix. The input size of the Transformer temporal feature extractor is (128,16,64), and the loss function is The hyperparameters α1, α2, α3 are α1=α2=0.3, α3=0.4.

[0060] The experimental results of incremental test prediction accuracy are as follows Figure 4 As shown, incremental Test Accuracy is the incremental test accuracy, Epoch is the time, and incremental_batch is the number of incremental batches. With the arrival of new class batches, the model prediction accuracy will decrease to a certain extent. When the incremental batches are 0, 1, and 2, the average prediction accuracy is 97.44%, 95.81%, and 89.56%, respectively.

[0061] In this experiment, the proposed method of the present invention is also compared with the finetune method. The comparison results when the incremental batch is 2 are as follows: Figure 5 As shown. According to the experimental results, the prediction effect of the method in this embodiment is obviously better than the fine-tuning method finetune. Therefore, it can be seen from the experimental results that the method of the present invention performs well on the electromagnetic wave data set for low-altitude thunderstorm data prediction.

[0062] Of course, it is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, but also includes the same or similar structures that can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any reference numerals in the claims should not be regarded as limiting the claims involved.

[0063] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0064] The techniques, shapes, and structural parts not described in detail in the present invention are all well-known techniques.

Claims

1. A low-altitude thunderstorm prediction method based on time and space dual dimensions of incremental learning, characterized in that: include: S1: Collect electromagnetic wave images to form T electromagnetic wave datasets {D0, D1, ..., D t ,...,D T-1 }; where D t Indicates The electromagnetic wave dataset of the new category in the incremental batch, T-1 is the total number of incremental batches, when t=0, it represents the initial electromagnetic wave dataset; S2: When the tth incremental batch arrives, a new incremental module is established , and based on the total electromagnetic wave data set D after the tth incremental batch, i.e. the current incremental batch t model and the previous incremental batch prediction model F t-1 (x) is trained to obtain the current incremental batch prediction model F t (x); where D t model ={D0∪D1...D t }, x represents the electromagnetic wave image; According to step S2, after T-1 incremental batches, the final prediction model F is obtained. T-1 (x); S3: Input the electromagnetic wave image to be measured into the final prediction model F T-1 (x) to obtain the predicted category.

2. A time-space dual-dimensional low-altitude thunderstorm prediction method based on incremental learning as claimed in claim 1, characterized in that: The previous incremental batch prediction model F t-1 (x) consists of a feature extractor and a linear classifier, expressed as follows: ； Among them, Ф t-1 (·) is the feature extractor of the previous incremental batch, and , D is the original feature dimension size of the electromagnetic wave image x, d is the feature dimension size after being extracted by the feature extractor, W t-1 is the linear classifier parameter of the previous incremental batch, and , Indicates the total number of categories in the t-1th incremental batch stage, with a superscript T Indicates transpose.

3. A time-space dual-dimensional low-altitude thunderstorm prediction method based on incremental learning as claimed in claim 2, characterized in that: The current incremental batch prediction model F t (x) is the prediction model F of the previous incremental batch t-1 (x) and new incremental modules Combined, it is expressed as follows: ； Among them, x is the electromagnetic wave image, y is the true value of the category of the electromagnetic wave image, Indicates the use of cross entropy loss function for the new incremental module Conduct training, D t train is the total electromagnetic wave dataset D after the current incremental batch t model The training set obtained in , argmin(.) represents the function of finding the optimal parameters, and E is the expected function.

4. A time-space dual-dimensional low-altitude thunderstorm prediction method based on incremental learning as claimed in claim 3, characterized in that: The new incremental module It can be expressed as: ； Among them, φ t (·) is the feature extractor of the new incremental module, which is composed of the CNN spatial feature extractor φ S t (·) and the transformer temporal feature extractor φ t TE (·) composition, and ; ;x s 、x TE are the normalized matrix and waveform time matrix of the electromagnetic wave image after spatial dimension and time dimension data preprocessing respectively; d s ,d TE They are respectively passed through the CNN spatial feature extractor φ t S (·) and the transformer temporal feature extractor φ t TE (·) Feature dimension size after extraction; are the linear classifier parameters of the new incremental module, and , , represents the total number of categories in the tth incremental batch stage, are the linear classifier parameters corresponding to the old categories in the new incremental module, ; are the linear classifier parameters corresponding to the new categories in the new incremental module, .

5. A time-space dual-dimensional low-altitude thunderstorm prediction method based on incremental learning as claimed in claim 4, characterized in that: Prediction Category The calculation formula is: ； Among them, 0 is the matrix For zero matrices of the same size, S(.) is softmax, which is a normalized exponential function.

6. A time-space dual-dimensional low-altitude thunderstorm prediction method based on incremental learning as claimed in claim 5, characterized in that: The new incremental module Through continuous training, the optimal parameters are obtained , the optimal parameters The expression is: ； Among them, Dis(.) is Distance(.), which means the function of finding the distance; Indicates a parameter.

7. A time-space dual-dimensional low-altitude thunderstorm prediction method based on incremental learning as claimed in claim 2, characterized in that: Current incremental batch prediction model F t (x) is expressed as: ； Among them, Ф t (·) is the current incremental batch feature extractor; W t Predict model F for the current incremental batch t (x) linear classifier parameters; and are the linear classifier parameters corresponding to the old and new categories in the incremental module respectively; φ t (·) is the feature extractor of the new incremental module; γ1, γ2 are scaling factors, and 0<γ1<1, 0<γ2<1, γ is the diagonal matrix composed of γ1 and γ2; the diagonal matrix composed of γ1 and γ2 can be expressed as: ； Among them, S old is the number of old category samples in the total electromagnetic wave dataset after the current incremental batch, S new It is the number of new category samples in the total electromagnetic wave dataset after the current incremental batch.

8. A time-space dual-dimensional low-altitude thunderstorm prediction method based on incremental learning as claimed in claim 7, characterized in that: The current incremental batch prediction model F t The loss function of (x) The calculation formula is: ； Among them, α1, α2, α3 are hyperparameters, and α1+α2+α3=1, is the first loss function, is the second loss function, is the distillation loss function, y is the true value of the electromagnetic wave category, is the cross entropy loss function, S(.) is the softmax, i.e. the normalized exponential function, W t fe is a new linear classifier parameter.

9. A time-space dual-dimensional low-altitude thunderstorm prediction method based on incremental learning as claimed in claim 8, characterized in that: The acquisition of the current incremental batch prediction model includes: d1: The current incremental batch prediction model F t (x) is set as the teacher model, and the student model is set at the tth incremental batch stage ; Wherein, the feature extractor of the student model is also composed of a CNN spatial feature extractor and a transformer temporal feature extractor; d2: Using the second distillation loss function Model for students Conduct training; d3: Use the trained student model as the final current incremental batch prediction model; The student model It can be expressed as: ； in, The student model for the tth incremental batch stage Linear classifier parameters, , Student model for the current incremental batch stage The feature extractor of They are respectively the student models of the current incremental batch stage CNN spatial feature extractor and transformer temporal feature extractor, , , And d=d s +d TE , x s 、x TE are the normalized matrix and waveform time matrix of the electromagnetic wave image after spatial dimension and time dimension data preprocessing respectively; d s ,d TE After CNN spatial feature extractor and transformer temporal feature extractor The size of the feature dimension after extraction; Represents the total number of categories in the tth incremental batch stage.

10. A method for predicting low-altitude thunderstorms in time and space based on incremental learning according to any one of claims 1 to 9, characterized in that: The current processing of the incremental batch prediction model includes: A1: Preprocess the electromagnetic wave image in spatial dimension and time dimension respectively to obtain the image preprocessing matrix, which includes a normalization matrix and a waveform time matrix; A2: Input the image preprocessing matrix into the spatiotemporal feature extractor in the current incremental batch prediction model to obtain a temporal feature matrix and a spatial feature matrix; the spatiotemporal feature extractor includes a CNN spatial feature extractor and a Transformer temporal feature extractor; A3: Input the temporal feature matrix and the spatial feature matrix into the feature splicing module for feature splicing to obtain the temporal and spatial feature matrix; A4: Input the spatiotemporal feature matrix into the linear classifier to obtain the predicted category.

Images