Rainfall image sequence similarity search method based on twin neural network

Abstract

The invention discloses a rainfall image sequence similarity search method based on a twin neural network. The rainfall image sequence similarity search method comprises the steps of obtaining a target rainfall image sequence and a target rainfall image library; constructing a rainfall image similarity model; based on the target rainfall image sequence and the target rainfall image library, performing training optimization on the constructed rainfall image similarity model to obtain a target rainfall image similarity model; and based on the target rainfall image sequence and the feature vector of the corresponding frame in the target rainfall image sequence extracted by the target rainfall image similarity model, carrying out rainfall image sequence similarity search on the target rainfall image library to obtain a rainfall image sequence similar to the target rainfall image sequence. According to the method, deep learning and a meteorological mechanism are fused, efficient similarity search of a rainfall image sequence is realized by constructing a rainfall image sequence similarity search model of a feature extraction network and twin neural network architecture, and three bottlenecks of feature robustness, time sequence modeling and calculation efficiency are broken through.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of hydro-meteorological image processing, in particular to a rainfall image sequence similarity search method based on a twin neural network. BACKGROUND

[0002] Rainfall prediction is the core of modern meteorological services, directly affecting key areas such as flood control and disaster reduction, water resource management, and agricultural production. Rainfall systems are essentially complex atmospheric dynamic processes, with significant nonlinear characteristics and spatiotemporal variability. Although traditional numerical weather prediction models (such as WRF and ECMWF) have made progress in macro trend prediction, their ability to accurately predict extreme weather such as local strong convection and sudden rainstorms is still insufficient, especially at the 0-6 hour short-term prediction scale, with prediction accuracy generally below 70%. This bottleneck is mainly due to atmospheric initial value uncertainty and the cumulative effect of model physical parameterization errors.

[0003] Under this background, the "Model Output Statistics" (MOS) method based on historical similarity analysis shows unique value - by searching the historical radar library for highly similar evolution sequences to the current rainfall process, the weather system evolution rules in historical cases can be effectively extracted, providing reliable decision-making reference for forecasters. Especially in flood warning scenarios, matching similar rainfall image sequences can extend the prediction period of small and medium river floods from the conventional 1-3 hours to 6-12 hours, significantly improving emergency response capabilities. With the increasing density of meteorological observation networks, establishing an efficient rainfall image sequence intelligent search system has become an urgent need to improve the quality of meteorological services.

[0004] However, in actual use, it still has some disadvantages. The current rainfall image sequence matching technology faces three core challenges: first, in terms of feature expression, traditional methods rely on manually designed low-level visual features such as histogram statistics and texture analysis, which are extremely sensitive to changes in imaging conditions. When the radar beam attenuation reaches 15 dB, the feature stability decreases by more than 40%, and it cannot effectively distinguish between convective precipitation, local strong echoes, and the meteorological nature of large-scale uniform echoes of stratiform cloud precipitation, resulting in a high mismatch rate of up to 35%.

[0005] Second, in terms of time series modeling, the mainstream solution has serious defects: single-frame independent search ignores the continuity of cloud movement, with a 10-frame sequence cumulative offset of over 50 kilometers; the Dynamic Time Warping (DTW) algorithm can handle time-varying sequences, but its O(N 2 L) computational complexity makes the search of a billion-level database take more than 30 minutes; and deep models such as 3D-CNN are difficult to handle long sequences of more than 15 frames due to excessive memory consumption. Third, in terms of business efficiency, existing systems cannot meet real-time requirements, and data reading accounts for more than 70% of processing time in traditional architectures.

[0006] Therefore, there is an urgent need to propose and develop a similarity search method for rainfall image sequences based on Siamese neural networks to solve the aforementioned technical problems. Summary of the Invention

[0007] This invention provides a similarity search method for rainfall image sequences based on Siamese neural networks. It integrates deep learning with meteorological mechanisms and constructs a similarity search model for rainfall image sequences using a feature extraction network and a Siamese neural network architecture. This method achieves efficient similarity search for rainfall image sequences and overcomes the triple bottlenecks of feature robustness, temporal modeling, and computational efficiency.

[0008] The technical solution adopted in this invention is:

[0009] A method for searching the similarity of rainfall image sequences based on Siamese neural networks includes the following steps:

[0010] S1. Obtain the target rainfall image sequence and target rainfall image library;

[0011] S2. Construct a rainfall image similarity model, which includes a feature extraction network and a Siamese neural network:

[0012] S21. Construct a feature extraction network. The feature extraction network adopts a multi-level convolutional architecture. Each level includes a convolutional layer, a ReLU activation function, a batch normalization layer, and a max pooling layer to extract feature vectors from the target rainfall image sequence and the corresponding frames in the target rainfall image library. At the same time, L2 regularization constraints and Dropout constraints are introduced to suppress overfitting of the feature extraction network.

[0013] S22. Construct a Siamese neural network, which includes two sub-networks sharing weights and parameters. The two sub-networks respectively receive the feature vectors of corresponding frames in the target rainfall image sequence and the feature vectors of corresponding frames in the target rainfall image library, and map them into a feature space. By comparing the feature vectors of corresponding frames in the target rainfall image sequence and the feature vectors of corresponding frames in the target rainfall image library, the distance between the feature vectors of corresponding frames in the target rainfall image sequence and the feature vectors of corresponding frames in the target rainfall image library in the feature space is output, which is used to measure the similarity between the feature vectors of corresponding frames in the target rainfall image sequence and the feature vectors of corresponding frames in the target rainfall image library.

[0014] S3. Based on the target rainfall image sequence and the target rainfall image library, the constructed rainfall image similarity model is trained and optimized to obtain the target rainfall image similarity model;

[0015] S4. Based on the feature vectors of the corresponding frames in the target rainfall image sequence extracted by the target rainfall image sequence and the target rainfall image similarity model, perform a rainfall image sequence similarity search on the target rainfall image library to obtain rainfall image sequences similar to the target rainfall image sequence, which are denoted as similar rainfall image sequences;

[0016] S5. Display the target rainfall image sequence and similar rainfall image sequences side by side in a layout matrix. The target rainfall image sequence is placed in the first row of the layout matrix, and the similar rainfall image sequences are placed in the remaining rows in descending order of similarity. Then, output a high-resolution image for visualization.

[0017] Further, step S1 includes:

[0018] S11. Obtain historical rainfall maps;

[0019] S12. Select rainfall images of a set sequence length from the historical rainfall image database to obtain a rainfall image sequence, wherein the rainfall images are daily rainfall maps;

[0020] S13. Preprocess the images in the rainfall image sequence and the historical rainfall image library respectively to obtain the target rainfall image sequence and the target rainfall image library.

[0021] Furthermore, the preprocessing in step S13 includes:

[0022] 1) Resize each image in the rainfall image sequence to a uniform size of 128×128 to obtain the first image;

[0023] 2) The pixel values ​​of the first image are mapped to the range [0, 1] through normalization to obtain the second image;

[0024] 3) Perform channel adaptive processing on the second image to obtain an input format that matches the feature extraction network;

[0025] 4) After processing, the target rainfall image sequence is obtained;

[0026] 5) Perform steps 1) to 3) on the images in the historical rainfall map library to obtain the target rainfall map library;

[0027] The normalization formula is as follows:

[0028]

[0029] In the formula, I norm The normalized image pixel matrix; I originalDivide the image pixel matrix of the historical rainfall image library or the target rainfall image sequence by 255.0 to normalize the image pixel values ​​from [0,255] to the range of [0,1].

[0030] The channel adaptive processing formula is:

[0031] I adapted =reshape(I norm ,(128,128,C))

[0032] In the formula, I adapted The image pixel matrix for channel adaptive processing; I norm is the normalized image pixel matrix; C is the number of automatically detected channels.

[0033] Furthermore, the workflow of the feature extraction network in step S21 includes:

[0034] 1) Convolutional layer operations, the calculation formula is as follows:

[0035] X (l) =ReLU(W (l) *X (l-1) +b (l) )

[0036] In the formula, * represents convolution operation; represents the output feature map of the l-th layer; and ReLU activation function max(0,x);

[0037] W (l) X represents the weights of the convolutional kernel in layer l; (l-1) b is the input feature map for the (l-1)th layer; (l) This is the bias vector for the l-th layer;

[0038] 2) Batch normalization operation, the calculation formula is as follows:

[0039]

[0040] In the formula, The normalized feature map is x; the original input feature map is μ. B This is the average value of the current batch; The variance of the current batch; ∈ is the numerical stability constant 10. -5 ; y is the final output feature map; γ is the scaling parameter; β is the translation parameter;

[0041] 3) L2 regularization constraint, the calculation formula is as follows:

[0042]

[0043] In the formula, λ is the regularization penalty term; the regularization coefficient λ = 0.01; w is the model weight parameter; θ is the model parameter set;

[0044] 4) Dropout constraint, the calculation formula is as follows:

[0045]

[0046] In the formula, p is the retention probability of the neuron; x is the input vector.

[0047] Furthermore, in step S22, the distance is calculated using the cosine distance formula:

[0048] Let feature vector A be the feature vector of the corresponding frame in the target rainfall image sequence, and feature vector B be the feature vector of the corresponding frame in the target rainfall image library. The calculation formula is as follows:

[0049]

[0050] In the formula, sim(A,B) is the similarity value between feature vector A and feature vector B; A·B is the dot product operation between vector A and feature vector B; ||A|| is the L2 norm of feature vector A; ||B|| is the L2 norm of feature vector B; the output range is [0,2], and the smaller the value of sim(A,B), the more similar feature vector A and feature vector B are; d is the cosine distance between feature vector A and feature vector B.

[0051] Further, step S3 includes:

[0052] S31. Based on the sample pair generation strategy, construct positive and negative sample pairs to distinguish between similar and dissimilar rainfall images. The expression is as follows:

[0053]

[0054] In the formula, The set of positive sample pairs; For the set of negative sample pairs; I i ,I j The images are rainfall samples; class(I) is the class label of image I; the ratio of positive to negative samples is maintained at a 1:1 balance during learning.

[0055] 2) Construct a contrastive loss function, and evaluate the similarity of sample pairs based on the contrastive loss function. The calculation formula is as follows:

[0056]

[0057] In the formula, is the loss value for the sample pair; y is the sample pair label (y∈{0,1}, 1 indicates a similar pair, 0 indicates a dissimilar pair); d is the feature distance (d=1-sim(A,B)); margin is the boundary threshold (set to 1.0); max(·,0) is the ReLU function to ensure non-negative output;

[0058] 3) Use an exponentially decaying learning rate to train the Siamese neural network to improve the model's training performance and generalization ability. The formula is as follows:

[0059] lr(t) = lr0 × Decay t / decay

[0060] In the formula, lr(t) is the learning rate at step t; lr0 is the initial learning rate of 0.001; Decay is the decay rate of 0.9; decay is the decay step size of 1000; and t is the current training step number.

[0061] 4) Based on the above training and optimization, a similarity model for the target rainfall image is obtained.

[0062] Further, step S4 includes:

[0063] S41. Feature vectors of corresponding frames in the target rainfall image sequence extracted based on the target rainfall image similarity model;

[0064] Specifically, the calculation formula is as follows:

[0065] E t =f θ (I t ), t=1,2,…,T

[0066] In the formula, E t f is the feature vector of the t-th frame of the target rainfall image sequence; θ For the trained feature extraction network; I t Let t be the rainfall image in the target rainfall image sequence; T is the total length of the target rainfall image sequence.

[0067] S42. Based on the sliding window algorithm, extract a rainfall image sequence of the same length as the target rainfall image sequence from the target rainfall image library and use it as a candidate rainfall image sequence;

[0068] S43. Extract features from the candidate rainfall image sequence to obtain the feature vector of the corresponding frame in the candidate rainfall image sequence;

[0069] S44. Compare the feature vectors of corresponding frames in the target rainfall image sequence with the feature vectors of corresponding frames in the candidate rainfall image sequence, and calculate the similarity between them using Euclidean distance;

[0070] S45. Using the Top-K queue update rule, during the search process, the queue dynamically retains the top K candidate rainfall image sequences that are similar to the feature sequence of the target rainfall image. The calculation formula is as follows:

[0071]

[0072] In the formula, A Top-K similar rainfall image sequence queue; S seq The similarity score for the current candidate rainfall image sequence; k is the current number of elements in the queue; k is the queue capacity. The score is the maximum similarity score in the queue;

[0073] Furthermore, the formula for calculating the Euclidean distance in step S44 is as follows:

[0074]

[0075] In the formula, S seq The similarity score is calculated between the feature vectors of corresponding frames in the target rainfall image sequence and the feature vectors of corresponding frames in the candidate rainfall image sequence. Let be the feature vector of the t-th frame of the rainfall image in the target rainfall image sequence; Let be the feature vector of the t-th frame of the candidate rainfall image sequence; ||·|2 is the Euclidean distance; where the smaller the Euclidean distance value, the more similar the candidate rainfall image sequence is to the target rainfall image sequence.

[0076] Further, step S5 includes:

[0077] 1) The target rainfall image sequence and the searched similar rainfall image sequences are displayed in the form of a layout matrix, as follows:

[0078]

[0079] In the formula, M is the visualization layout matrix; The target rainfall image sequence is the t-th frame of the rainfall image sequence; Let be the t-th frame of the i-th similar rainfall image sequence; k+1 is the number of rows in the matrix, i.e., the target rainfall image sequence + k similar rainfall image sequences; the number of columns in the matrix is ​​the sequence length T;

[0080] 2) The parameter expression for high-resolution image output is as follows:

[0081]

[0082] In the formula, width is the width of the high-resolution image output; height is the height of the high-resolution image output; DPI is dots per inch; and k is the number of similar rainfall image sequences.

[0083] Compared with the prior art, the present invention has the following advantages:

[0084] 1) In this invention, target rainfall image sequences and candidate rainfall image sequences are obtained and preprocessed to obtain unified, standardized and structured samples, thereby improving the quality and adaptability of input samples for the feature extraction network;

[0085] 2) The feature extraction network uses convolutional layers, batch normalization, and pooling layers to encode the features of the rainfall images, mapping the rainfall images to a low-dimensional feature space, and extracting the feature vectors of the corresponding frames in the target rainfall image sequence and the feature vectors of the corresponding frames in the candidate rainfall image sequence; the feature extraction network extracts feature vectors and reduces feature dimensionality, which are then used as input to the Siamese neural network.

[0086] 3) Siamese neural networks calculate the cosine similarity between the feature vectors of corresponding frames in the target rainfall image sequence and the feature vectors of corresponding frames in the candidate rainfall image sequence. The Siamese neural network architecture can directly process rainfall images, avoiding the complex feature engineering in traditional methods, reducing the dependence on prior knowledge, and improving the generalization ability of the model. Combined with cosine similarity, it can effectively measure the similarity between rainfall images and has high computational efficiency, making it suitable for search tasks on large-scale datasets.

[0087] 4) During the training and optimization phase, the Siamese neural network generates positive and negative sample pairs and optimizes the network parameters using a contrastive loss function, enabling the network to learn the ability to distinguish between similar and dissimilar rainfall images. Through the optimization of the contrastive loss function, the network can learn more discriminative feature representations, making similar rainfall images closer in the feature space and dissimilar rainfall images farther apart, thereby improving the accuracy of similarity search.

[0088] 5) Candidate rainfall image sequences are obtained by traversing the historical rainfall image database using a sliding window method. The similarity between the target rainfall image sequence and the candidate rainfall image sequences is calculated based on the Euclidean distance formula, and several rainfall image sequences similar to the target rainfall image sequence are selected. Among them, the sliding window algorithm is used to traverse the rainfall image sequences in the historical rainfall image database, which can flexibly handle rainfall image sequences of different lengths and has strong adaptability.

[0089] 6) Experimental results of this invention show that, based on the target rainfall image sequence similarity model and the target rainfall image sequence, it is possible to accurately search for rainfall image sequences similar to the target rainfall image sequence from a large-scale target rainfall image library, providing strong technical support for rainfall pattern analysis and meteorological forecasting. Attached Figure Description

[0090] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0091] Figure 1 This is a flowchart illustrating a rainfall image sequence similarity search method based on a Siamese neural network according to the present invention.

[0092] Figure 2 This is a schematic diagram of a historical rainfall map database for a certain watershed in this invention;

[0093] Figure 3 This is a real-time rainfall map of a certain watershed from July 3 to 8, 2017, as described in this invention.

[0094] Figure 4 This is a screenshot showing the similarity search results for rainfall image sequences from July 3 to 8, 2017, in a certain watershed, based on a twin neural network-based rainfall image sequence similarity search according to the present invention. Detailed Implementation

[0095] Example

[0096] like Figure 1 As shown, a method for searching the similarity of rainfall image sequences based on Siamese neural networks includes the following steps:

[0097] S1. Obtain the target rainfall image sequence and target rainfall image library:

[0098] Specifically, S11, obtain historical rainfall maps;

[0099] S12. Select rainfall images of a set sequence length from the historical rainfall image database to obtain a rainfall image sequence, wherein the rainfall images are daily rainfall maps;

[0100] S13. Preprocess the rainfall image sequence and the images in the historical rainfall image database to obtain the target rainfall image sequence and the target rainfall image database, specifically including:

[0101] 1) Resize each image in the rainfall image sequence to a uniform size of 128×128 to obtain the first image;

[0102] 2) The pixel values ​​of the first image are mapped to the range [0, 1] through normalization to obtain the second image;

[0103] 3) Perform channel adaptive processing on the second image to obtain an input format that matches the feature extraction network;

[0104] 4) After processing, the target rainfall image sequence is obtained;

[0105] 5) Perform steps 1) to 3) on the images in the historical rainfall map library to obtain the target rainfall map library;

[0106] The normalization formula is as follows:

[0107]

[0108] In the formula, I norm The normalized image pixel matrix; I original Divide the image pixel matrix of the historical rainfall image library or the target rainfall image sequence by 255.0 to normalize the image pixel values ​​from [0,255] to the range of [0,1].

[0109] The channel adaptive processing formula is:

[0110] I adapted =reshape(I norm ,(128,128,C))

[0111] In the formula, I adapted The image pixel matrix for channel adaptive processing; I norm is the normalized image pixel matrix; C is the number of automatically detected channels;

[0112] S2. Construct a rainfall image similarity model, which includes a feature extraction network and a Siamese neural network:

[0113] S21. Construct a feature extraction network. The feature extraction network adopts a multi-level convolutional architecture, with each level containing a convolutional layer, a ReLU activation function, a batch normalization layer, and a max pooling layer, used to extract feature vectors from the target rainfall image sequence and corresponding frames in the target rainfall image library. Simultaneously, L2 regularization and Dropout constraints are introduced to suppress overfitting of the feature extraction network.

[0114] Specifically, the workflow of a feature extraction network includes:

[0115] 1) Convolutional layer operations, the calculation formula is as follows:

[0116] X (l) =ReLU(W (l) *X (l-1) +b (l) )

[0117] In the formula, * represents convolution operation; represents the output feature map of the l-th layer; and ReLU activation function max(0,x);

[0118] W (l) X represents the weights of the convolutional kernel in layer l; (l-1) b is the input feature map for the (l-1)th layer; (l) This is the bias vector for the l-th layer;

[0119] 2) Batch normalization operation, the calculation formula is as follows:

[0120]

[0121]

[0122] In the formula, The normalized feature map is x; the original input feature map is μ. B This represents the average value of the current batch (batch statistics). The variance of the current batch (batch statistic); ∈ is a numerically stable constant 10. -5 ; y is the final output feature map; γ is the scaling parameter (learnable parameter); β is the translation parameter (learnable parameter);

[0123] 3) L2 regularization constraint, the calculation formula is as follows:

[0124]

[0125] In the formula, λ is the regularization penalty term; the regularization coefficient λ = 0.01; w is the model weight parameter; θ is the model parameter set;

[0126] 4) Dropout constraint, the calculation formula is as follows:

[0127]

[0128] In the formula, p is the retention probability of the neuron; x is the input vector.

[0129] Explanation: Convolution extracts local features from an image through local connectivity and weight sharing. This operation uses a learnable convolution kernel to slide across the input feature map, calculating the dot product of the local region and the kernel to capture spatial features. Batch Normalization addresses the internal covariate shift problem in deep neural network training by standardizing the input distribution of the neural network. It stabilizes the mean and variance of the inputs at each layer, thereby accelerating training convergence and improving the model's generalization ability. L2 regularization prevents overfitting by adding a weight sum of squares penalty term to the loss function. It constrains the weight magnitude, avoids extreme weights, improves the model's generalization ability, and enhances the numerical stability of the optimization process. Dropout is a simple and effective regularization technique that significantly improves the generalization ability of neural networks and prevents overfitting by randomly discarding the outputs of neurons.

[0130] S22. Construct a Siamese neural network, comprising two sub-networks sharing weights and parameters. Each sub-network receives feature vectors of corresponding frames in the target rainfall image sequence and feature vectors of corresponding frames in the target rainfall image library, respectively, and maps them into a feature space. By comparing the feature vectors of corresponding frames in the target rainfall image sequence and the target rainfall image library, the distance between the feature vectors of corresponding frames in the target rainfall image sequence and the target rainfall image library in the feature space is output, which is used to measure the similarity between the feature vectors of corresponding frames in the target rainfall image sequence and the target rainfall image library.

[0131] Specifically, in step S22, the distance is calculated using the cosine distance formula:

[0132] Let feature vector A be the feature vector of the corresponding frame in the target rainfall image sequence, and feature vector B be the feature vector of the corresponding frame in the target rainfall image library. The calculation formula is as follows:

[0133]

[0134] In the formula, sim(A,B) is the similarity value between feature vector A and feature vector B; A·B is the dot product operation between vector A and feature vector B; ||A|| is the L2 norm (modulus) of feature vector A; ||B|| is the L2 norm (modulus) of feature vector B; the output range is [0,2], the smaller the value of sim(A,B), the more similar feature vector A and feature vector B are; d is the cosine distance between feature vector A and feature vector B.

[0135] Note: When comparing the feature spaces of feature vectors of corresponding frames in two input samples in a Siamese neural network, this implementation first calculates the similarity using the cosine similarity formula, which is a simple and effective similarity measurement method. It measures the directional similarity by calculating the cosine of the angle between the feature vectors of corresponding frames in two samples. Then, the distance between the feature vectors of corresponding frames in two samples is calculated using the cosine distance formula, i.e., (1-cosine similarity).

[0136] S3. Based on the target rainfall image sequence and the target rainfall image library, the constructed rainfall image similarity model is trained and optimized to obtain the target rainfall image similarity model:

[0137] Specifically, S31, construct positive and negative sample pairs based on the sample pair generation strategy to distinguish between similar and dissimilar rainfall images, as expressed below:

[0138]

[0139] In the formula, The set of positive sample pairs; For the set of negative sample pairs; I i ,I j The images are rainfall samples; class(I) is the class label of image I; the ratio of positive to negative samples is maintained at a 1:1 balance during learning.

[0140] 2) Construct a contrastive loss function, and evaluate the similarity of sample pairs based on the contrastive loss function. The calculation formula is as follows:

[0141]

[0142] In the formula, is the loss value for the sample pair; y is the sample pair label (y∈{0,1}, 1 indicates a similar pair, 0 indicates a dissimilar pair); d is the feature distance (d=1-sim(A,B)); margin is the boundary threshold (set to 1.0); max(·,0) is the ReLU function to ensure non-negative output;

[0143] 3) Use an exponentially decaying learning rate to train the Siamese neural network to improve the model's training performance and generalization ability. The formula is as follows:

[0144] lr(t) = lr0 × Decay t / decay

[0145] In the formula, lr(t) is the learning rate at step t; lr0 is the initial learning rate of 0.001; Decay is the decay rate of 0.9; decay is the decay step size of 1000; and t is the current training step number.

[0146] 4) Based on the above training and optimization, a similarity model for the target rainfall image is obtained;

[0147] Explanation: The core purpose of the sample pair generation strategy is to help the model learn how to distinguish between similar and dissimilar samples by constructing positive and negative sample pairs; the basic idea of ​​the contrastive loss function is to learn an effective feature embedding space by minimizing the distance between similar sample pairs and maximizing the distance between dissimilar sample pairs; exponential decay learning rate: is an effective learning rate adjustment strategy. It gradually decreases the learning rate in the form of an exponential function, enabling the model to converge quickly in the early stage of training and finely adjust the parameters in the later stage of training, thereby improving the training effect and generalization ability of the model;

[0148] S4. Based on the feature vectors of corresponding frames in the target rainfall image sequence extracted by the target rainfall image similarity model, perform a rainfall image sequence similarity search on the target rainfall image library to obtain rainfall image sequences similar to the target rainfall image sequence, denoted as similar rainfall image sequences:

[0149] Specifically, S41, the feature vector of the corresponding frame in the target rainfall image sequence extracted based on the target rainfall image similarity model is calculated using the following formula:

[0150] E t =f θ (I t ), t=1,2,…,T

[0151] In the formula, E t f is the feature vector of the t-th frame of the target rainfall image sequence; θ For the trained feature extraction network; I t Let t be the rainfall image in the target rainfall image sequence; T is the total length (total number of frames) of the target rainfall image sequence;

[0152] S42. Based on the sliding window algorithm, extract a rainfall image sequence of the same length as the target rainfall image sequence from the target rainfall image library and use it as a candidate rainfall image sequence;

[0153] S43. Extract features from the candidate rainfall image sequence to obtain the feature vector of the corresponding frame in the candidate rainfall image sequence;

[0154] S44. Compare the feature vectors of corresponding frames in the target rainfall image sequence with the feature vectors of corresponding frames in the candidate rainfall image sequence, and calculate the similarity between them using Euclidean distance;

[0155] The formula for calculating Euclidean distance is as follows:

[0156]

[0157] In the formula, S seq The similarity score is calculated between the feature vectors of corresponding frames in the target rainfall image sequence and the feature vectors of corresponding frames in the candidate rainfall image sequence. Let be the feature vector of the t-th frame of the rainfall image in the target rainfall image sequence; Let be the feature vector of the t-th frame of the candidate rainfall image sequence; ||·|2 is the Euclidean distance; where, the smaller the Euclidean distance value, the more similar the candidate rainfall image sequence is to the target rainfall image sequence;

[0158] S45. Using the Top-K queue update rule, during the search process, the queue dynamically retains the top K candidate rainfall image sequences that are similar to the feature sequence of the target rainfall image. The calculation formula is as follows:

[0159]

[0160] In the formula, A Top-K similar rainfall image sequence queue; S seq The similarity score for the current candidate rainfall image sequence; k is the current number of elements in the queue; k is the queue capacity. The score is the maximum similarity score in the queue;

[0161] Explanation: The Top-K queue update rule is a mechanism for maintaining a queue of the most similar rainfall image sequences. Its purpose is to dynamically retain candidate rainfall image sequences that are most similar to the target rainfall image sequence during the search process, thereby improving search efficiency and accuracy.

[0162] S5. Display the target rainfall image sequence and similar rainfall image sequences side-by-side in a layout matrix. The target rainfall image sequence is placed in the first row of the layout matrix, and the similar rainfall image sequences are placed in the remaining rows in descending order of similarity. Then, output a high-resolution image for visualization.

[0163] Among them, 1) the target rainfall image sequence and the searched similar rainfall image sequences are displayed in the form of a layout matrix, as follows:

[0164]

[0165] In the formula, M is the visualization layout matrix; The target rainfall image sequence is the t-th frame of the rainfall image sequence;

[0166] Let be the t-th frame of the i-th similar rainfall image sequence; k+1 is the number of rows in the matrix, i.e., the target rainfall image sequence + k similar rainfall image sequences; the number of columns in the matrix is ​​the sequence length T;

[0167] 2) The parameter expression for high-resolution image output is as follows:

[0168]

[0169] In the formula, width is the width of the high-resolution image output; height is the height of the high-resolution image output; DPI is dots per inch (Dots Per Inch); and k is the number of similar rainfall image sequences.

[0170] Explanation: A layout matrix is ​​a structure used to organize and visualize multiple rainfall image sequences. It displays frames of rainfall images of the target rainfall image sequence and similar rainfall image sequences in a matrix format. The target image sequence is placed in the first row. By arranging these frames of rainfall images according to the matrix structure, a visual layout matrix can be generated, which facilitates the comparison of the target rainfall image sequence and similar rainfall image sequences in the image.

[0171] In this embodiment, the parameter configuration of the feature extraction network in the target rainfall image similarity model is shown in Table 1:

[0172] Table 1

[0173]

[0174]

[0175] 2) The parameter configuration of the Siamese neural network in the target rainfall image similarity model is shown in Table 2:

[0176] Table 2

[0177]

[0178] The method of this invention is used to perform a similarity search on rainfall image sequences in a certain watershed, as detailed below:

[0179] like Figure 2 As shown, a historical rainfall map database for a certain watershed from 2017 to 2025 is obtained; then, a rainfall image sequence is randomly selected from the historical rainfall map database for the same watershed from 2017 to 2025, as shown. Figure 3 As shown, rainfall images from July 3rd to 8th, 2017, were selected from the historical rainfall image database as the target rainfall image sequence. After inputting the target rainfall image sequence and the historical rainfall image database into a rainfall image sequence similarity model for similarity search, four rainfall image sequences similar to the target rainfall image sequence were obtained, which are then visualized as follows: Figure 4As shown in (a) to (d), the first row is the target rainfall image sequence, arranged in descending order of similarity. The second row consists of rainfall image sequences from June 29 to July 5, 2018, from August 10 to 15, 2020, from June 14 to 19, 2021, and from August 5 to 11, 2021. Among them, the rainfall image sequence from June 29 to July 5, 2018 has the highest similarity to the target rainfall image sequence.

[0180] The experimental results of this invention show that, based on the target rainfall image sequence similarity model and the target rainfall image sequence, it is possible to accurately search for rainfall image sequences similar to the target rainfall image sequence from a large-scale target rainfall image library, providing strong technical support for rainfall pattern analysis and meteorological forecasting.

[0181] This invention presents a rainfall image similarity model constructed based on feature extraction networks and Siamese neural networks. Through training and optimization, a target rainfall image similarity model is obtained. The distance between the feature vectors of the target rainfall image sequence and a target rainfall image library is obtained by inputting these elements. This distance is used to measure the similarity between single-frame images in the sample. Then, based on the similarity between single-frame images in the sample, the similarity between rainfall image sequences is obtained, thereby improving the accuracy of rainfall image sequence similarity search, enhancing the matching effect between the target rainfall image sequence and similar rainfall image sequences, and reducing computational load.

[0182] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the principles and essence of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for searching the similarity of rainfall image sequences based on Siamese neural networks, characterized in that, Includes the following steps: S1. Obtain the target rainfall image sequence and target rainfall image library; S2. Construct a rainfall image similarity model, which includes a feature extraction network and a Siamese neural network: S21. Construct a feature extraction network. The feature extraction network adopts a multi-level convolutional architecture. Each level includes a convolutional layer, a ReLU activation function, a batch normalization layer, and a max pooling layer to extract feature vectors from the target rainfall image sequence and the corresponding frames in the target rainfall image library. At the same time, L2 regularization constraints and Dropout constraints are introduced to suppress overfitting of the feature extraction network. S22. Construct a Siamese neural network, which includes two sub-networks sharing weights and parameters. The two sub-networks respectively receive the feature vectors of corresponding frames in the target rainfall image sequence and the feature vectors of corresponding frames in the target rainfall image library, and map them into a feature space. By comparing the feature vectors of corresponding frames in the target rainfall image sequence and the feature vectors of corresponding frames in the target rainfall image library, the distance between the feature vectors of corresponding frames in the target rainfall image sequence and the feature vectors of corresponding frames in the target rainfall image library in the feature space is output, which is used to measure the similarity between the feature vectors of corresponding frames in the target rainfall image sequence and the feature vectors of corresponding frames in the target rainfall image library. S3. Based on the target rainfall image sequence and the target rainfall image library, the constructed rainfall image similarity model is trained and optimized to obtain the target rainfall image similarity model; S4. Based on the feature vectors of the corresponding frames in the target rainfall image sequence extracted by the target rainfall image sequence and the target rainfall image similarity model, perform a rainfall image sequence similarity search on the target rainfall image library to obtain rainfall image sequences similar to the target rainfall image sequence, which are denoted as similar rainfall image sequences; S5. Display the target rainfall image sequence and similar rainfall image sequences side by side in a layout matrix. The target rainfall image sequence is placed in the first row of the layout matrix, and the similar rainfall image sequences are placed in the remaining rows in descending order of similarity. Then, output a high-resolution image for visualization.

2. The method for searching the similarity of rainfall image sequences based on Siamese neural networks according to claim 1, characterized in that, Step S1 includes: S11. Obtain historical rainfall maps; S12. Select rainfall images of a set sequence length from the historical rainfall image database to obtain a rainfall image sequence, wherein the rainfall images are daily rainfall maps; S13. Preprocess the images in the rainfall image sequence and the historical rainfall image library respectively to obtain the target rainfall image sequence and the target rainfall image library.

3. The method for searching the similarity of rainfall image sequences based on Siamese neural networks according to claim 2, characterized in that, The preprocessing in step S13 includes: 1) Resize each image in the rainfall image sequence to a uniform size of 128×128 to obtain the first image; 2) The pixel values ​​of the first image are mapped to the range [0, 1] through normalization to obtain the second image; 3) Perform channel adaptive processing on the second image to obtain an input format that matches the feature extraction network; 4) After processing, the target rainfall image sequence is obtained; 5) Perform steps 1) to 3) on the images in the historical rainfall map library to obtain the target rainfall map library; The normalization formula is as follows: In the formula, I norm The normalized image pixel matrix; I original Divide the image pixel matrix of the historical rainfall image library or the target rainfall image sequence by 255.0 to normalize the image pixel values ​​from [0,255] to the range of [0,1]. The channel adaptive processing formula is: I adapted =reshape(I norm ,(128,128,C)) In the formula, I adapted The image pixel matrix for channel adaptive processing; I norm is the normalized image pixel matrix; C is the number of automatically detected channels.

4. The method for searching the similarity of rainfall image sequences based on Siamese neural networks according to claim 1, characterized in that, The workflow of the feature extraction network in step S21 includes: 1) Convolutional layer operations, the calculation formula is as follows: X (l) =ReLU(W (l) *X (l-1) +b (l) ) In the formula, * represents convolution operation; represents the output feature map of the l-th layer; represents the ReLU activation function max(0,x); W (l) X represents the weights of the convolutional kernel in layer l; (l-1) b is the input feature map for the (l-1)th layer; (l) This is the bias vector for the l-th layer; 2) Batch normalization operation, the calculation formula is as follows: In the formula, The normalized feature map is x; the original input feature map is μ. B This is the average value of the current batch; The variance of the current batch; ∈ is the numerical stability constant 10. -5 ; y is the final output feature map; γ is the scaling parameter; β is the translation parameter; 3) L2 regularization constraint, the calculation formula is as follows: In the formula, λ is the regularization penalty term; the regularization coefficient λ = 0.01; w is the model weight parameter; θ is the model parameter set; 4) Dropout constraint, the calculation formula is as follows: In the formula, p is the retention probability of the neuron; x is the input vector.

5. The method for searching the similarity of rainfall image sequences based on Siamese neural networks according to claim 1, characterized in that, In step S22, the distance is calculated using the cosine distance formula: Let feature vector A be the feature vector of the corresponding frame in the target rainfall image sequence, and feature vector B be the feature vector of the corresponding frame in the target rainfall image library. The calculation formula is as follows: In the formula, sim(A,B) is the similarity value between feature vector A and feature vector B; A·B is the dot product operation between vector A and feature vector B; ||A|| is the L2 norm of feature vector A; ||B|| is the L2 norm of feature vector B; the output range is [0,2], and the smaller the value of sim(A,B), the more similar feature vector A and feature vector B are; d is the cosine distance between feature vector A and feature vector B.

6. The method for searching the similarity of rainfall image sequences based on Siamese neural networks according to claim 1, characterized in that, Step S3 includes: S31. Based on the sample pair generation strategy, construct positive and negative sample pairs to distinguish between similar and dissimilar rainfall images. The expression is as follows: In the formula, The set of positive sample pairs; For the set of negative sample pairs; I i ,I j The images are rainfall samples; class(I) is the class label of image I; the ratio of positive to negative samples is maintained at a 1:1 balance during learning. 2) Construct a contrastive loss function, and evaluate the similarity of sample pairs based on the contrastive loss function. The calculation formula is as follows: In the formula, is the loss value for the sample pair; y is the sample pair label (y∈{0,1}, 1 indicates a similar pair, 0 indicates a dissimilar pair); d is the feature distance (d=1-sim(A,B)); margin is the boundary threshold (set to 1.0); max(·,0) is the ReLU function to ensure non-negative output; 3) Use an exponentially decaying learning rate to train the Siamese neural network to improve the model's training performance and generalization ability. The formula is as follows: lr(t)=lr0×Decay t / decay In the formula, lr(t) is the learning rate at step t; lr0 is the initial learning rate of 0.001; Decay is the decay rate of 0.9; decay is the decay step size of 1000; and t is the current training step number. 4) Based on the above training and optimization, a similarity model for the target rainfall image is obtained.

7. The method for searching the similarity of rainfall image sequences based on Siamese neural networks according to claim 1, characterized in that, Step S4 includes: S41. Feature vectors of corresponding frames in the target rainfall image sequence extracted based on the target rainfall image similarity model; Specifically, the calculation formula is as follows: E t =f θ (I t ),t=1,2,…,T In the formula, E t f is the feature vector of the t-th frame of the target rainfall image sequence; θ For the trained feature extraction network; I t Let t be the rainfall image in the target rainfall image sequence; T is the total length of the target rainfall image sequence. S42. Based on the sliding window algorithm, extract a rainfall image sequence of the same length as the target rainfall image sequence from the target rainfall image library and use it as a candidate rainfall image sequence; S43. Extract features from the candidate rainfall image sequence to obtain the feature vector of the corresponding frame in the candidate rainfall image sequence; S44. Compare the feature vectors of corresponding frames in the target rainfall image sequence with the feature vectors of corresponding frames in the candidate rainfall image sequence, and calculate the similarity between them using Euclidean distance; S45. Using the Top-K queue update rule, during the search process, the queue dynamically retains the top K candidate rainfall image sequences that are similar to the feature sequence of the target rainfall image. The calculation formula is as follows: In the formula, A Top-K similar rainfall image sequence queue; S seq The similarity score for the current candidate rainfall image sequence; k is the current number of elements in the queue; k is the queue capacity. The score is the maximum similarity score in the queue.

8. The method for searching the similarity of rainfall image sequences based on Siamese neural networks according to claim 7, characterized in that, The formula for calculating the Euclidean distance in step S44 is as follows: In the formula, S seq The similarity score is calculated between the feature vectors of corresponding frames in the target rainfall image sequence and the feature vectors of corresponding frames in the candidate rainfall image sequence. Let be the feature vector of the t-th frame of the rainfall image in the target rainfall image sequence; Let be the feature vector of the t-th frame of the candidate rainfall image sequence; ||·|2 is the Euclidean distance; where the smaller the Euclidean distance value, the more similar the candidate rainfall image sequence is to the target rainfall image sequence.

9. The method for searching the similarity of rainfall image sequences based on Siamese neural networks according to claim 1, characterized in that, Step S5 includes: 1) The target rainfall image sequence and the searched similar rainfall image sequences are displayed in the form of a layout matrix, as follows: In the formula, M is the visualization layout matrix; The target rainfall image sequence is the t-th frame of the rainfall image sequence; Let be the t-th frame of the i-th similar rainfall image sequence; k+1 is the number of rows in the matrix, i.e., the target rainfall image sequence + k similar rainfall image sequences; the number of columns in the matrix is ​​the sequence length T; 2) The parameter expression for high-resolution image output is as follows: In the formula, width is the width of the high-resolution image output; height is the height of the high-resolution image output; DPI is dots per inch; and k is the number of similar rainfall image sequences.

Images