Visibility prediction method and system

Abstract

The invention discloses a visibility prediction method and system, and the method employs a dynamic window division and abnormal value processing technology based on data variability, solves a problem that meteorological data abnormal values are difficult to accurately recognize, achieves the precise recognition and processing of the abnormal values, and improves the data quality. By adopting the missing value interpolation method based on the weighted distance, the problem that the missing value of the meteorological data is difficult to fill is solved, the accurate interpolation of the missing value is realized, and the integrity and the reliability of the data are enhanced. According to the method, a dynamic normalization processing method is adopted, the problems of different scales of meteorological factor data and local characteristic differences are solved, adaptive normalization of the meteorological factor data is achieved, important fluctuation information is reserved, and the sensitivity of a model to data details is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of meteorological forecasting, and in particular relates to a visibility forecasting method and system. Background Art

[0002] Visibility is an important indicator of atmospheric transparency and is of great significance to transportation and public safety. Traditional visibility prediction methods mainly rely on linear extrapolation of meteorological data and numerical weather forecasting technology. However, due to the high nonlinearity of weather systems and the complexity of local weather changes, traditional methods have limitations in predicting visibility with high temporal resolution and high accuracy. With the continuous development of deep learning technology, existing visibility prediction methods based on deep learning models include: recurrent neural network (RNN), convolutional neural network (CNN), long short-term memory network (LSTM), etc. The visibility prediction method based on recurrent neural network (RNN) processes time series data through a cyclic structure, so that the output of each time step depends not only on the current input, but also on the output of the previous time step, trying to capture the impact of historical meteorological data (such as temperature, humidity, wind speed, etc.) on visibility. However, changes in visibility are affected by long-term and short-term factors of complex meteorological conditions, including seasonal changes, sudden weather events, and environmental pollution. When processing long time series, RNN is prone to gradient vanishing and gradient explosion problems, which makes it difficult to effectively capture the long-term trend of visibility. In addition, there are often a lot of noise and outliers in meteorological data, such as sensor errors, missing data, and data anomalies caused by extreme weather. RNN has poor robustness to these noisy data, resulting in a decrease in the accuracy of visibility prediction. The visibility prediction method based on convolutional neural network (CNN) extracts local features from historical meteorological data through convolution operations, such as the change pattern of temperature and humidity in the short term, for visibility prediction. However, visibility is affected by multiple meteorological factors in time and space, and has global and dynamic characteristics. Although CNN is good at extracting local features, it has limitations in capturing the global temporal dependency of visibility changes and cannot fully reflect the dynamic changes of visibility at different time scales, resulting in limited prediction performance under complex meteorological conditions. For example, CNN may not be able to effectively capture the visibility changes under the influence of large-scale weather systems. Long short-term memory network (LSTM) is an improved recurrent neural network that aims to better capture the long-term dependency of historical meteorological data on visibility by introducing input gates, forget gates, and output gates. However, the noise and outliers in visibility data, such as extreme weather events such as sudden haze, sandstorms, and heavy rainfall, will interfere with model training and prediction. When LSTM processes meteorological data containing a lot of noise, its prediction performance will still be affected. In addition, the factors affecting visibility vary under different weather conditions and environments. The generalization ability of the LSTM model in different regions and seasons may be insufficient, making it difficult to predict rapid changes in visibility in a timely and accurate manner.

[0003] In summary, the existing visibility prediction methods have shortcomings in dealing with noise and outliers in the data and capturing the long-term dependencies of visibility changes. Summary of the invention

[0004] In order to solve the problems existing in the prior art, the present invention proposes a visibility prediction method and system.

[0005] The technical solution of the present invention is as follows:

[0006] A visibility prediction method, comprising:

[0007] Acquire meteorological factor data, wherein the meteorological factors include environmental factors and visibility; preprocess the meteorological factor data, wherein the preprocessing includes outlier processing and missing value processing considering the variability of the meteorological factor data and dynamic normalization processing considering the fluctuation information of the meteorological factor data; perform modal decomposition on the preprocessed visibility to form a number of intrinsic modal function components with different central frequencies; construct a meteorological data set based on the environmental factors in the preprocessed meteorological factor data and the several intrinsic modal function components; construct and train a dynamic convolution adaptive recursive visibility prediction model corresponding to each intrinsic modal function component based on the meteorological data set; input the current environmental factors into the dynamic convolution adaptive recursive visibility prediction models of different intrinsic modal function components respectively, obtain different intrinsic modal function components corresponding to the current environmental factors, and fuse the different intrinsic modal function components corresponding to the current environmental factors to obtain a visibility prediction result.

[0008] Furthermore, the environmental factors include temperature, dew point temperature, air pressure, wind speed and precipitation.

[0009] Furthermore, the specific steps of the outlier processing include: sorting the meteorological factors to obtain the data sequence of each meteorological factor; for each meteorological factor, using a fixed initial window size W 0 Slide on the data sequence and calculate the mean μ of the data in each window i and standard deviation σ i , and then calculate the coefficient of variation of the data in each window Calculate the average value of the coefficient of variation of each meteorological factor window For the i-th window, determine the dynamic window size W i , expressed as: Where W min is the minimum value of the dynamic window size; W max is the maximum value of the dynamic window size; use the calculated dynamic window size W i , starting from the beginning of the data sequence, divide each window in turn, each window contains W idata points; in each dynamic window, calculate the upper quartile Q3 and lower quartile Q1 of each meteorological factor, the upper quartile is the position of 75% of the data points in the sorted data, and the lower quartile Q1 is the position of 25% of the data points in the sorted data, and calculate the interquartile range of each meteorological factor in each dynamic window, which is expressed by the formula: IQR = Q3-Q1; where IQR represents the interquartile range, Q3 represents the upper quartile, and Q1 represents the lower quartile; calculate the lower and upper limits of the abnormal values ​​of each meteorological factor in each window, which is expressed by the formula: Line1 = Q1-1.5×IQR; Line2 = Q3+1.5×IQR; where Line1 represents the lower limit of the abnormal value, Line2 represents the upper limit of the abnormal value, and IQR represents the interquartile range; find out the abnormal value according to the lower and upper limits of the abnormal value calculated in each window and delete it, and the range of the abnormal value is expressed by the formula: Abnormal value = {x outer ∣x outer <Line1,x outer >Line2}; where x outer Represents an outlier, Line1 represents the lower limit of the outlier, and Line2 represents the upper limit of the outlier; mark the location of the outlier.

[0010] Furthermore, the specific steps of the missing value processing include: using the data sequence as a sample to form an original data set T, T = {(x i ,t i )|i=1,2,…,N}; at the same time, the outlier mark position is obtained in the data sequence and used as the missing value position, and the data sequence without outlier mark is used as a sample, and some data are randomly selected from it as the interpolation verification set T'. The interpolation verification set T' is expressed as: T'={(x i ′,t i ′)|i=1,2,…,N′}; where x i ′ represents the observed value of a meteorological factor in the i-th sample in the interpolation validation set, t i ′ represents the time label corresponding to the i-th sample in the interpolation validation set, and N′ represents the total number of meteorological data samples in the interpolation validation set. For each sample in the interpolation validation set, the weighted distance d(i, j) between it and other samples is calculated to measure the similarity between the two samples, which is expressed as: In the formula, d(i,j) represents the weighted distance between sample i and sample j, and w factor and w time are the weight factors of meteorological factor value and time difference, x i ′ and x j ′ are the meteorological factor observation values ​​of sample i and sample j respectively, t i ′ and t j′ are the time labels of sample i and sample j respectively; the weight factor w of the difference in the numerical value of the initial meteorological factor is factor The weight factor w of the time difference between the meteorological factor time The value of Define the mean square error as the objective function and define its convergence threshold μ, which can be expressed as: Where M is the number of samples with non-missing observations in the interpolation validation set, x i ′ represents the true observed value of the i-th known sample in the interpolation validation set, represents the interpolated value calculated using the current weight factor, μ represents the threshold set to make the objective function converge, and σ represents the standard deviation of the data before simulating the missing value of each meteorological factor in the interpolation verification set; the weight factor is automatically adjusted and optimized using the gradient descent method until the objective function converges to the set threshold μ, and the optimized weight factor w is obtained. factor and w time , expressed as: In the formula, η is the learning rate used by the gradient descent method, k is the number of iterations of the gradient descent method, and are the partial derivatives of the objective function with respect to the weight factor; after optimization, the weight factor w factor and w time For each sample with missing values ​​in the original data set T, calculate its weighted distance d(i, j) to all other samples, sort them from small to large according to the distance, select the K samples with the closest distance to form the nearest neighbor set, which can be expressed as: Where N K (i) represents the set of nearest neighbor samples of sample i, is the meteorological factor observation value of the kth sample closest to sample i; the missing meteorological factor value is interpolated using the weighted average of the nearest neighbor samples, which is expressed as: In the formula, represents the interpolation estimate of the missing sample i, x j is the meteorological factor observation value of the nearest neighbor sample j, w j is the weight of sample j, ∈ is a small constant to prevent division by zero.

[0011] Furthermore, the specific steps of the dynamic normalization process considering the fluctuation information of meteorological factor data include: converting the original data set T after outlier processing and missing value processing into {(x i ,t i)|i=1,2,…,N} is divided into training set and validation set according to a certain ratio; each meteorological factor is normalized in each dynamic window corresponding to each meteorological factor; each meteorological factor is traversed and an array with the same shape as its original data is initialized for the kth meteorological factor to store the normalized data; starting from the starting position of the data sequence, according to the dynamic window size W i,k , divide each window in turn, determine the start and end index of the i-th window, and express it as: In the formula, s i Indicates the starting index of the i-th window, s i-1 represents the starting index of the i-1th window, e i represents the end index of the i-th window, and if e i >N, then e i =N; extract the data in the window and calculate the maximum and minimum values ​​in the window. The sample set in the i-th window is X i,k ={x j ∣j∈[s i ,e i ]}, x j is the observed value of the kth meteorological factor in the jth sample, and the maximum and minimum values ​​are calculated using the formula: In the formula, represents the maximum value of the kth meteorological factor in the i-th window, represents the minimum value of the kth meteorological factor in the i-th window; the data in the window are normalized, and for each sample x in the window j ∈X i,k , the normalized value is expressed as: In the formula, is the normalized value of the kth meteorological factor in the jth sample. When To avoid division by zero errors; the normalized value Stored in the corresponding position; update the window index, expressed as: s i+1 =e i +1; if s i+1 >N, the processing of the remaining data ends; the dynamic normalization process is repeated for all meteorological factors until all meteorological factors have completed dynamic normalization.

[0012] Furthermore, the specific steps of performing modal decomposition on the pre-processed visibility to form a plurality of intrinsic modal function components with different center frequencies include: obtaining an analysis signal by Hilbert transform calculation for each modal classification, and calculating a single-sided spectrum. The center band of the visibility data is transferred to the corresponding baseband region by aliasing an index corresponding to the center frequency of the modal component: In the formula, u k (t) represents the kth intrinsic mode function of visibility data, ω k (t) represents the center frequency of the kth intrinsic mode function of the visibility data, δ(t) represents the Dirac function, and the value range of k is [1, MAX]. MAX is the total number of intrinsic mode function components of the visibility data after decomposition; the square norm L2 of the demodulation gradient is calculated to estimate the bandwidth of each modal component, and the constraints are solved as follows: In the formula, u k = {u 1 ,u 2 ,…,u k} represents the intrinsic mode function component of visibility data, ω k ={ω 1 ,ω 2 ,…,ω k} represents the central frequency of the intrinsic mode function component of the visibility data, and f(t) represents the visibility data signal. The Lagrange multiplier λ(t) and the second-order penalty factor α are introduced to transform the constrained variation problem into an unconstrained variation problem:

[0013]

[0014] In the formula, α is the second-order penalty factor, λ(t) is the Lagrange multiplier;

[0015] By continuously updating the components and the center frequency using the alternating direction multiplier method, the saddle point of the unconstrained model is finally obtained: Where, f(ω), They represent f(t) and u respectively. k (t), λ(t), n represents the number of iterations; for u k 1 ,ω k 1 , 1 Initialize, set the number of iterations to 1, execute cycle n = n + 1, update u k ,ω k , and double-boost λ, which can be expressed as: When the following equation is satisfied, the iteration stops. Where γ represents the step size parameter, which is used to control the update rate of the Lagrange multiplier; τ represents the noise tolerance; ε represents the convergence threshold, which is used to determine whether the iteration converges.

[0016] Furthermore, the dynamic convolution adaptive recursive visibility prediction model includes a bidirectional convolution layer, an adaptive recursive unit and a Transformer encoder-decoder; the construction steps of the dynamic convolution adaptive recursive visibility prediction model are as follows: constructing a bidirectional convolution layer, the bidirectional convolution layer is used to process the convolution operation of the meteorological factor data sequence; forward convolution, using one-dimensional convolution to input the meteorological factor data sequence data T = {(x i ,t i )|i=1,2,…,N} to perform convolution operation and extract the forward features of the meteorological factor sequence, which can be expressed as: F fwd (i) = Conv1D fwd (x i );where x i is the meteorological factor feature vector at the i-th time step, which contains the observed values ​​of all meteorological factors. fwd (i) is the output of the forward convolution, Conv1D fwd Represents the forward one-dimensional convolution operation; backward convolution flips the input data to obtain the reverse sequence x rev (i) = x N-i+1 , and then use one-dimensional convolution to perform convolution operation. The convolution kernel size and filling method are the same as the forward convolution. The backward features of the meteorological factor sequence are extracted, which can be expressed as: F bwd (i) = Conv1D bwd (x rev (i)); where F bwd (i) is the output of the backward convolution, Conv1D bwd Represents the backward one-dimensional convolution operation; output fusion, the output of the forward convolution and the backward convolution is averaged as the output of this layer, which is expressed as: In the formula, F conv (i) is the output feature of the bidirectional convolutional layer; the Dropout layer is used to prevent overfitting and improve the generalization ability of the model; the adaptive recursive unit is used to process the recursive feature extraction of time series data; the steps of constructing the adaptive recursive unit are as follows: the output F of the bidirectional convolutional layer is converted into conv (i) Input to the gated recurrent unit, expressed as: In the formula, represents the hidden state of GRU at the i-th time step, GRU represents the recursive calculation function of the GRU unit; a bidirectional long short-term memory network is used to process the input data, and the bidirectional output is merged, which is expressed as: In the formula, and are the hidden states of the forward and backward LSTM, respectively. is the output of the bidirectional LSTM, LSTM forward and LSTM backward Represent the recursive calculation functions of the forward LSTM unit and the backward LSTM unit respectively;

[0017] Use the fully connected layer to generate the gating signal, and adaptively select the outputs of GRU and BiLSTM through the gating signal; calculate the gating signal through the fully connected layer, and perform weighted summation on the outputs of GRU and BiLSTM to achieve adaptive selection, which can be expressed as: α(i) = σ(W α ·F conv (i)+b α ); Where W α is the weight matrix of the fully connected layer, b α is the bias vector of the fully connected layer, σ is the Sigmoid activation function, α(i) represents the gated signal vector of the i-th time step, which determines the weights of the GRU and BiLSTM outputs. is the final output of the adaptive recurrent unit at the i-th time step, and ⊙ represents the element-wise product.

[0018] The Dropout layer is used to prevent overfitting and improve the generalization ability of the model. The Transformer encoder-decoder is used to further process the output of the adaptive recurrent unit, extract global features, and realize the prediction of visibility. The specific implementation steps of the Transformer encoder-decoder are as follows: Use sine and cosine functions to generate a position encoding matrix, where each position encoding vector corresponds to a feature dimension, and add the position encoding to the input data, which is expressed as: Where, t i is the time label of the ith time step, d model is the feature dimension of the model, that is, the dimension of the input and output feature vectors, PE (i,2j) represents the sinusoidal position encoding value of the ith time step in the 2jth dimension, PE (i,2j+1) represents the cosine position encoding value of the ith time step in the 2j+1 dimension, PE i Represents the position encoding vector of the i-th time step, containing the encoding values ​​of all dimensions, F input(i) represents the input feature vector after adding position encoding; the multi-head attention mechanism is used to perform attention calculation on each position in the input sequence. The input data is divided into multiple subspaces, each subspace corresponds to a head, and each head performs attention calculation separately. The output of the multi-head attention mechanism is the connection result of each head, which can be expressed as: Q = F input W Q ,K=F input W K ,V=F input W V ; MultiHead(Q,K,V)=Concat(head 1 ,…,head H )W O ; In the formula, Q, K, and V represent the query matrix, key matrix, and value matrix respectively. h , K h 、V h represents the query, key, and value matrix of the h-th head, Attention(Q h ,K h ,V h ) represents the attention calculation function of the hth head, head h represents the output matrix of the hth head, H represents the number of attention heads, and W Q , W K , W V is the learnable weight matrix, W O is the output transformation matrix, MultiHead(Q,K,V) is the final output of the multi-head attention mechanism, softmax() means normalizing the attention weights of each time step, Concat() means concatenating the outputs of all heads in the feature dimension, h = 1, 2, ..., H; the results of the attention calculation are further processed using a feedforward neural network, which consists of two fully connected layers; the first layer performs a nonlinear transformation on the output of the multi-head attention mechanism, and the second layer maps the transformed result to the output space; the feedforward neural network is expressed as: FFN(x) = ReLU(xW 1 +b 1 )W 2 +b 2 Where W 1 , W 2 is the weight matrix, b 1 , b 2is the bias vector, ReLU is the activation function, and FFN(x) represents the output of the feedforward neural network; construct encoder and decoder layers, stack multiple encoder and decoder layers containing multi-head attention mechanisms and feedforward neural networks to build a complete Transformer model; the encoder is used to process the input sequence and extract features; the decoder generates a visibility prediction result based on the output of the encoder.

[0019] A visibility prediction system includes a meteorological factor data acquisition module, a preprocessing module, a modal decomposition module, a data set construction module, a model construction module and a prediction reorganization module; the meteorological factor data acquisition module is used to acquire meteorological factor data, and meteorological factors include environmental factors and visibility; the preprocessing module is used to preprocess the meteorological factor data, and the preprocessing includes abnormal value processing and missing value processing considering the variability of meteorological factor data and dynamic normalization processing considering the fluctuation information of meteorological factor data; the modal decomposition module is used to perform modal decomposition on the visibility after preprocessing to obtain key features; the data set construction module is used to construct a meteorological data set with environmental factors and key features in the meteorological factor data; the model construction module is used to train a pre-constructed dynamic convolution adaptive recursive visibility prediction model based on the meteorological data set; the prediction reorganization module is used to reorganize the key features output by the trained dynamic convolution adaptive recursive visibility prediction model to complete visibility prediction.

[0020] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is used to call and run the computer program stored in the memory to execute any of the methods described above.

[0021] A computer-readable storage medium stores a computer program, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] The present invention proposes a visibility prediction method and system. The method adopts dynamic window division and outlier processing technology based on data variability, solves the problem that outliers in meteorological data are difficult to accurately identify, realizes accurate identification and processing of outliers, and improves data quality.

[0024] The present invention adopts a missing value interpolation method based on weighted distance, solves the problem of difficulty in filling missing values ​​in meteorological data, realizes accurate interpolation of missing values, and enhances the integrity and reliability of data.

[0025] The present invention adopts a dynamic normalization processing method to solve the problems of inconsistent scales and local characteristic differences of meteorological factor data, realizes adaptive normalization of each meteorological factor data, retains important fluctuation information, and improves the model's sensitivity to data details.

[0026] The present invention uses signal decomposition technology to perform modal decomposition on visibility data, which solves the problem that visibility data is complex and noisy and difficult to directly model, realizes the extraction of key features, reduces the complexity of data, and provides a better quality input for the model.

[0027] The present invention constructs a dynamic convolution adaptive recursive Transformer model, which solves the problem of insufficient feature extraction and prediction accuracy of complex time series data, achieves high-precision prediction of visibility, and improves the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a flow chart of a visibility prediction method in an embodiment;

[0029] Figure 2 This is a result diagram of visibility signal decomposition data in the application embodiment;

[0030] Figure 3 This is a diagram of the training set prediction results in the application example;

[0031] Figure 4 This is a diagram of the prediction results of the validation set in the application example. DETAILED DESCRIPTION

[0032] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent modifications of the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0033] Embodiment 1:

[0034] A visibility prediction method of the present invention, as Figure 1 As shown, including:

[0035] Obtain meteorological factor data, including environmental factors and visibility;

[0036] Preprocessing the meteorological factor data includes outlier processing and missing value processing considering the variability of meteorological factor data and dynamic normalization processing considering the fluctuation information of meteorological factor data;

[0037] Perform modal decomposition on the preprocessed visibility to form several intrinsic modal function components with different center frequencies;

[0038] The meteorological data set is constructed by using the environmental factors and several intrinsic mode function components in the preprocessed meteorological factor data;

[0039] Based on the meteorological dataset, a dynamic convolution adaptive recursive visibility prediction model corresponding to each intrinsic mode function component is constructed and trained;

[0040] The current environmental factors are respectively input into the dynamic convolution adaptive recursive visibility prediction model of different intrinsic modal function components, and the different intrinsic modal function components corresponding to the current environmental factors are obtained. The different intrinsic modal function components corresponding to the current environmental factors are fused to get the visibility prediction result.

[0041] Embodiment 2:

[0042] This embodiment is further designed on the basis of the first embodiment in that the environmental factors in this embodiment include temperature, dew point temperature, air pressure, wind speed and precipitation.

[0043] Embodiment three:

[0044] This embodiment is further designed on the basis of the first embodiment in that the specific steps of outlier processing in this embodiment include: Step 1001-1.0: sorting meteorological factors to obtain a data sequence of each meteorological factor;

[0045] Step 1001-1.1: Considering the variability and time correlation of meteorological data, the meteorological factor dataset is dynamically windowed based on data variability. For each meteorological factor, a fixed initial window size W is used. 0 Slide on the data sequence and calculate the mean μ of the data in each window i and standard deviation σ i , and then calculate the coefficient of variation CV of the data in each window i :

[0046] Step 1001-1.2: Calculate the average value of the coefficient of variation of each meteorological factor window

[0047] Step 1001-1.3: For the i-th window, determine the dynamic window size W i At the same time, considering that when the window is too small, the short-term fluctuation of a single meteorological element may be over-amplified, causing greater noise, and when the window is too large, because visibility in meteorology is usually affected by events such as short-term heavy precipitation and sudden haze, using an overly large window will cover up such local sudden changes, making it difficult to identify abnormal values ​​in time, so the minimum value of the dynamic window size W is set min and the maximum value W max , expressed as:

[0048]

[0049] Where W min is the minimum value of the dynamic window size; W max is the maximum value of the dynamic window size;

[0050] Step 1001-1.4: Use the calculated dynamic window size W i , starting from the beginning of the data sequence, divide each window in turn, each window contains W i data points;

[0051] Step 1001-1.5: In each dynamic window, calculate the upper quartile (Q3) and lower quartile (Q1) of each meteorological factor. The upper quartile is the location of 75% of the data points in the sorted data, and the lower quartile (Q1) is the location of 25% of the data points in the sorted data. Calculate the interquartile range of each meteorological factor in each dynamic window, expressed as: IQR = Q3-Q1

[0052] In the formula, IQR represents the interquartile range, Q3 represents the upper quartile, and Q1 represents the lower quartile;

[0053] Step 1001-1.6: Calculate the lower and upper limits of the abnormal values ​​of each meteorological factor in each window, expressed as follows:

[0054] Line1=Q1-1.5×IQR

[0055] Line2=Q3+1.5×IQR

[0056] In the formula, Line1 represents the lower limit of the outlier, Line2 represents the upper limit of the outlier, and IQR represents the interquartile range;

[0057] Step 1001-1.7: Find outliers based on the lower and upper limits of outliers calculated for each window and delete them. The range of outliers is expressed as:

[0058] Outliers = {x outer ∣x outer <Line1,x outer >Line2}

[0059] In the formula, x outer Represents an outlier, Line1 represents the lower limit of the outlier, and Line2 represents the upper limit of the outlier.

[0060] Mark the location of the outlier, usually marked as NAN. The outliers identified in the window are marked as missing values ​​and retained in the data sequence so that they can be uniformly processed in the subsequent missing value processing of each meteorological factor. This processing method can avoid the data integrity problem caused by directly deleting outliers, and improve the accuracy and reliability of the data through subsequent interpolation methods. Improve data quality.

[0061] Embodiment 4:

[0062] This embodiment is further designed on the basis of the third embodiment in that the specific steps of missing value processing in this embodiment include:

[0063] Step 1001-2.1: Use the data sequence as a sample to form an original data set T, T = {(x i ,t i )|i=1,2,…,N}; at the same time, the outlier mark position is obtained in the data sequence and used as the missing value position. In order to optimize the following weight factors, the data sequence without outlier mark is used as a sample, and some data are randomly selected from it as the interpolation verification set T'. These data are regarded as missing values ​​for simulation. The interpolation verification set T' is expressed as: T'={(x i ′,t i ′)|i=1,2,…,N′}

[0064] In the formula, x i ′ represents the observed value of a meteorological factor in the i-th sample in the interpolation validation set, t i ′ represents the time label corresponding to the i-th sample in the interpolation validation set, and N′ represents the total number of samples of meteorological data in the interpolation validation set;

[0065] Step 1001-2.2: For each sample in the original data set, calculate the weighted distance d(i, j) between it and other samples to measure the similarity between two samples, so as to find other samples that are most similar to the missing value sample, and introduce the weight factor w factor and w time To adjust the impact of meteorological factor numerical differences and time differences on similarity, the formula is:

[0066] In the formula, d(i,j) represents the weighted distance between sample i and sample j, and w factor and w time are the weight factors of meteorological factor value and time difference, respectively, i ′ and x j ′ are the meteorological factor observation values ​​of sample i and sample j respectively, t i ′ and t j ′ are the time labels of sample i and sample j respectively;

[0067] Step 1001-2.3: Initialize the weight factor w of the difference in meteorological factor values factor The weight factor w of the time difference between the meteorological factor time The value of

[0068] Step 1001-2.4: Define the mean square error (MSE) as the objective function and define its convergence threshold μ, which is used to measure the difference between the interpolation estimate and the true value, expressed as:

[0069]

[0070] μ=0.01×σ 2

[0071] Where M is the number of samples with non-missing observations, x i ′ represents the true observed value of the i-th known sample in the interpolation validation set, represents the interpolated value calculated using the current weight factor, μ represents the threshold set to make the objective function converge, and σ represents the standard deviation of the data before simulating the missing value of each meteorological factor in the interpolation validation set;

[0072] Step 1001-2.5: Use the gradient descent method to automatically adjust and optimize the weight factor until the objective function converges to the set threshold μ, which can be expressed as:

[0073]

[0074]

[0075] In the formula, η is the learning rate used by the gradient descent method, k is the number of iterations of the gradient descent method, and are the partial derivatives of the objective function with respect to the weight factor;

[0076] Step 1001-2.6: After optimization, the weight factor w factor and w time For each sample with missing values ​​in the original data set T, calculate its weighted distance d(i, j) to all other samples, sort them from small to large according to the distance, select the K samples with the closest distance to form the nearest neighbor set, which can be expressed as:

[0077] Where N K (i) represents the set of nearest neighbor samples of sample i, is the meteorological factor observation value of the kth sample closest to sample i;

[0078] Step 1001-2.7: Interpolate the missing meteorological factor values ​​using the weighted average of the nearest neighbor samples, expressed as:

[0079] In the formula, represents the interpolation estimate of the missing sample i, x j is the meteorological factor observation value of the nearest neighbor sample j, w j is the weight of sample j, ∈ is a small constant to prevent division by zero, and its value is 10 -6 .

[0080] Embodiment five:

[0081] This embodiment is further designed on the basis of the fourth embodiment in that the specific steps of dynamic normalization processing considering the fluctuation information of meteorological factor data in this embodiment include:

[0082] Step 1002-1: The original data set T after outlier and missing value processing is processed. i ,t i )|i=1,2,…,N} is divided into training set and validation set according to a certain ratio;

[0083] Preferably, the training set accounts for 80% of the dataset and the validation set accounts for 20% of the dataset;

[0084] Step 1002-2: In step 1001, for each dynamic window size corresponding to each meteorological factor, such as the size W of the ith dynamic window of the kth meteorological factor, i,k , each meteorological factor is normalized so that the normalization process adapts to the local characteristics of the data and retains important fluctuation information;

[0085] Step 1002-3.1: traverse each meteorological factor, and initialize an array with the same shape as its original data for the kth meteorological factor to store the normalized data;

[0086] Step 1002-3.2: Starting from the start position of the data sequence, according to the dynamic window size W i,k , divide each window in turn, determine the start and end index of the i-th window, and express it as:

[0087]

[0088] e i =s i +W i,k -1

[0089] In the formula, s i Indicates the starting index of the i-th window, s i-1represents the starting index of the i-1th window, e i represents the end index of the i-th window, and if e i >N, then e i =N.

[0090] Step 1002-3.3: Extract the data in the window and calculate the maximum and minimum values ​​in the window. The sample set in the i-th window is X i,k ={x j ∣j∈[s i ,e i ]}, x j is the observed value of the kth meteorological factor in the jth sample, and the maximum and minimum values ​​are calculated using the formula:

[0091]

[0092]

[0093] In the formula, represents the maximum value of the kth meteorological factor in the i-th window, Represents the minimum value of the kth meteorological factor in the ith window.

[0094] Step 1002-3.4: Normalize the data in the window. For each sample x in the window j ∈X i,k , the normalized value is expressed as:

[0095]

[0096] In the formula, is the normalized value of the kth meteorological factor in the jth sample. When to avoid division by zero errors.

[0097] Step 1002-3.5: Normalize the value Stored in the corresponding location.

[0098] Step 1002-3.6: Update the window index, which is expressed as:

[0099] s i+1 =e i +1

[0100] If s i+1 >N, the processing of the remaining data ends.

[0101] Preferably, if the last window fails to completely cover the remaining data, the remaining data is regarded as a window and normalized according to steps 1002-3.1 to 1002-3.6.

[0102] Step 1002-4: Repeat the dynamic normalization process from step 1002-3.1 to step 1002-3.6 for all meteorological factors until all meteorological factors have completed dynamic normalization.

[0103] Embodiment six:

[0104] This embodiment is further designed on the basis of the fifth embodiment in that, in this embodiment, modal decomposition is performed on the pre-processed visibility to form a plurality of intrinsic modal function components with different center frequencies, and the specific steps include:

[0105] Step 1003-1: For each modal classification, the analysis signal is obtained by Hilbert transform calculation, and the one-sided spectrum is calculated. The center band of the visibility data is transferred to the corresponding baseband area by aliasing an index of the center frequency corresponding to the modal component:

[0106]

[0107] In the formula, u k (t) represents the kth intrinsic mode function of visibility data, ω k (t) represents the center frequency of the kth intrinsic mode function component of the visibility data, δ(t) represents the Dirac function, and the value range of k is [1, MAX]. MAX is the total number of intrinsic mode function components of the visibility data after decomposition;

[0108] Step 1003-2: Calculate the square norm L2 of the demodulation gradient to estimate the bandwidth of each modal component, and solve the constraints as follows:

[0109]

[0110] In the formula, u k = {u 1 ,u 2 ,…,u k} represents the intrinsic mode function component of visibility data, ω k ={ω 1 ,ω 2 ,…,ω k} represents the center frequency of the intrinsic mode function component of the visibility data, and f(t) represents the visibility data signal;

[0111] Step 1003-3: Introduce the Lagrange multiplier λ(t) and the second-order penalty factor α to transform the constrained variational problem into an unconstrained variational problem:

[0112]

[0113] In the formula, α is the second-order penalty factor, λ(t) is the Lagrange multiplier;

[0114] Step 1003-4: By continuously updating each component and the center frequency using the alternating direction multiplier method, the saddle point of the unconstrained model is finally obtained:

[0115]

[0116] Where, f(ω), They represent f(t) and u respectively. k (t), the Fourier transform of λ(t), and n represents the number of iterations.

[0117] Step 1003-5: k 1 ,ω k 1 , 1 Initialize, set the number of iterations to 1, execute cycle n = n + 1, and update u according to step 1003-4 k ,ω k , and double-boost λ, which can be expressed as:

[0118]

[0119] When the following equation is satisfied, the iteration stops.

[0120]

[0121] Where γ represents the step size parameter, which is used to control the update rate of the Lagrange multiplier; τ represents the noise tolerance. When τ = 0 is set, a good denoising effect can be achieved; ε represents the convergence threshold, which is 10-7 and is used to determine whether the iteration has converged.

[0122] Embodiment seven:

[0123] This embodiment is further designed on the basis of the sixth embodiment in that the dynamic convolution adaptive recursive visibility prediction model (i.e., dynamic convolution adaptive recursive Transformer model) in this embodiment includes a bidirectional convolution layer, an adaptive recursive unit, and a Transformer encoder-decoder. Each component works together to fully exploit the spatiotemporal characteristics of the meteorological factor sequence data and improve the accuracy of visibility prediction.

[0124] The construction steps of the dynamic convolution adaptive recursive visibility prediction model are as follows:

[0125] Step 1004-1: construct a bidirectional convolution layer, which is used to process the convolution operation of the meteorological factor data sequence. This layer realizes bidirectional feature extraction of the data by combining forward convolution and backward convolution. The specific implementation steps of the bidirectional convolution layer construction are as follows:

[0126] Step 1004-1.1: forward convolution, using one-dimensional convolution to input the meteorological factor data sequence data T = {(x i ,t i )|i=1,2,…,N} to perform convolution operation and extract the forward features of the meteorological factor sequence to help the model understand the impact of past meteorological conditions on current visibility, which can be expressed as:

[0127] F fwd (i) = Conv1D fwd (x i )

[0128] In the formula, x i is the meteorological factor feature vector at the i-th time step, which contains the observed values ​​of all meteorological factors. fwd (i) is the output of the forward convolution, Conv1D fwd Represents a forward one-dimensional convolution operation;

[0129] Step 1004-1.2: Backward convolution, flip the input data to obtain the reverse sequence x rev (i) = x N-i+1 , and then use one-dimensional convolution for convolution operation. The convolution kernel size and filling method are the same as the forward convolution. The backward features of the meteorological factor sequence are extracted to help the model consider the potential impact of future meteorological conditions on visibility, which is conducive to capturing the bidirectional dependency of the time series. The formula is expressed as:

[0130] F bwd (i) = Conv1D bwd (x rev (i))

[0131] In the formula, F bwd (i) is the output of the backward convolution, Conv1D bwd Represents a backward one-dimensional convolution operation;

[0132] Step 1004-1.3: Output fusion, taking the average of the outputs of the forward convolution and the backward convolution as the output of this layer, so that the model can pay attention to the past and future information of the meteorological factor sequence at the same time, and enhance the ability to capture the trend of visibility changes, which can be expressed as:

[0133]

[0134] In the formula, Fconv (i) is the output feature of the bidirectional convolutional layer;

[0135] Step 1004-1.4: Use the Dropout layer to prevent overfitting and improve the generalization ability of the model;

[0136] Step 1004-2: The adaptive recursive unit is used to process the recursive feature extraction of time series data; this unit combines the advantages of GRU and bidirectional LSTM, and selects the optimal recursive unit output through an adaptive gating mechanism. The steps for constructing the adaptive recursive unit are as follows:

[0137] Step 1004-2.1: The output F of the bidirectional convolutional layer conv (i) Input into the gated recurrent unit (GRU) to capture the short-term dependencies of the meteorological factor sequence and help the model respond quickly to rapid changes in visibility, which can be expressed as:

[0138]

[0139] In the formula, represents the hidden state of GRU at the i-th time step, and GRU represents the recursive calculation function of the GRU unit;

[0140] Step 1004-2.2: Use a bidirectional long short-term memory network (BiLSTM) to process the input data and merge the bidirectional outputs to capture the long-term dependencies of the meteorological factor sequence, including forward and backward information, which can be expressed as:

[0141]

[0142]

[0143]

[0144] In the formula, and are the hidden states of the forward and backward LSTM, respectively. is the output of the bidirectional LSTM, LSTM forward and LSTM backward Represent the recursive calculation functions of the forward LSTM unit and the backward LSTM unit respectively;

[0145] Step 1004-2.3: Generate a gating signal using a fully connected layer (Dense), and adaptively select the outputs of GRU and BiLSTM through the gating signal; calculate the gating signal through the fully connected layer, perform weighted summation on the outputs of GRU and BiLSTM, and achieve adaptive selection, so that the model is more flexible in capturing short-term fluctuations and long-term trends of visibility, which can be expressed as:

[0146] α(i)=σ(W α ·F conv (i)+b α )

[0147]

[0148] Where W α is the weight matrix of the fully connected layer, b α is the bias vector of the fully connected layer, σ is the Sigmoid activation function, α(i) represents the gated signal vector of the i-th time step, which determines the weights of the GRU and BiLSTM outputs. is the final output of the adaptive recurrent unit at the i-th time step, and ⊙ represents the element-wise product.

[0149] Step 1004-2.4: Use the Dropout layer to prevent overfitting and improve the generalization ability of the model.

[0150] Step 1004-3: The Transformer encoder-decoder is used to further process the output of the adaptive recurrent unit, extract global features, and predict visibility. Through the multi-head attention mechanism and position encoding, the model can capture the complex temporal dependencies and the interactive relationship between meteorological factors. The specific implementation steps of the Transformer encoder-decoder are as follows:

[0151] Step 1004-3.1: Use sine and cosine functions to generate a position encoding matrix, where each position encoding vector corresponds to a feature dimension, and add the position encoding to the input data so that the Transformer can perceive the position information in the time series data, which can be expressed as:

[0152]

[0153]

[0154]

[0155] Where, t i is the time label of the ith time step, d model is the feature dimension of the model, i.e. the dimension of the input and output feature vectors, j = 0, 1, ..., PE (i,2j) represents the sinusoidal position encoding value of the ith time step in the 2jth dimension, PE (i,2j+1) represents the cosine position encoding value of the ith time step in the 2j+1 dimension, PE i Represents the position encoding vector of the i-th time step, containing the encoding values ​​of all dimensions, F input(i) represents the input feature vector after adding position encoding;

[0156] Step 1004-3.2: Use the multi-head attention mechanism to perform attention calculation on each position in the input sequence. The input data is divided into multiple subspaces, each subspace corresponds to a head, and each head performs attention calculation separately to capture the dependencies between different parts of the input sequence. The output of the multi-head attention mechanism is the connection result of each head, which is expressed as:

[0157] Q=F input W Q ,K=F input W K ,V=F input W V

[0158]

[0159] MultiHead(Q,K,V)=Concat(head 1 ,…,head H )W O

[0160] In the formula, Q, K, and V represent the query matrix, key matrix, and value matrix respectively. h , K h 、V h represents the query, key, and value matrix of the h-th head, Attention(Q h ,K h ,V h ) represents the attention calculation function of the hth head, head h represents the output matrix of the hth head, H represents the number of attention heads, and W Q , W K , W V is the learnable weight matrix, W O is the output transformation matrix, MultiHead(Q,K,V) is the final output of the multi-head attention mechanism, softmax() means normalizing the attention weights of each time step, Concat() means concatenating the outputs of all heads in the feature dimension, h = 1, 2, ..., H;

[0161] Step 1004-3.3: Use a feedforward neural network to further process the results of the attention calculation. The feedforward neural network consists of two fully connected layers. The first layer performs a nonlinear transformation on the output of the multi-head attention mechanism, and the second layer maps the transformed result to the output space. The feedforward neural network can further process the results of the attention calculation and extract more advanced features, which can be expressed as follows:

[0162] FFN(x)=ReLU(xW 1 +b 1 )W 2 +b 2

[0163] Where W 1 , W 2 is the weight matrix, b 1 , b 2 is the bias vector, ReLU is the activation function, and FFN(x) represents the output of the feedforward neural network;

[0164] Step 1004-3.4: Construct encoder and decoder layers, stack multiple encoder and decoder layers containing multi-head attention mechanisms and feedforward neural networks, and construct a complete Transformer model; the encoder is used to process the input sequence and extract features; the decoder generates visibility prediction results based on the output of the encoder.

[0165] Preferably, in step 1006, the trained model is verified for accuracy using a validation set, and the error evaluation index is expressed as:

[0166]

[0167]

[0168]

[0169]

[0170] Where N val is the total number of samples in the validation set, X i represents the true value of the i-th visibility data, represents the predicted value of the i-th visibility data, Represents the average of the true values ​​of the visibility data.

[0171] Preferably, in step 1007, before outputting the predicted value, the predicted result needs to be denormalized. Since each meteorological factor is normalized based on a dynamic window in step 1002, the denormalization process needs to use the maximum and minimum values ​​of the corresponding dynamic window. The denormalization formula is expressed as:

[0172]

[0173] In the formula, is the visibility prediction value of the jth sample after denormalization, is the normalized visibility prediction value of the jth sample, and are the dynamic window W to which the jth sample belongs respectively j The maximum and minimum values ​​of the internal visibility data, W j Indicates the index of the dynamic window where the jth sample is located

[0174] Embodiment eight:

[0175] A visibility prediction system of the present invention comprises a meteorological factor data acquisition module, a preprocessing module, a modal decomposition module, a data set construction module, a model construction module and a prediction reorganization module;

[0176] The meteorological factor data acquisition module is used to obtain meteorological factor data, which includes environmental factors and visibility; the preprocessing module is used to preprocess the meteorological factor data, which includes outlier processing and missing value processing considering the variability of meteorological factor data and dynamic normalization processing considering the fluctuation information of meteorological factor data; the modal decomposition module is used to perform modal decomposition on the visibility after preprocessing to obtain key features; the data set construction module is used to construct a meteorological data set based on the environmental factors and key features in the meteorological factor data; the model construction module is used to train the pre-constructed dynamic convolution adaptive recursive visibility prediction model based on the meteorological data set; the prediction reorganization module is used to reorganize the key features output by the trained dynamic convolution adaptive recursive visibility prediction model to complete the visibility prediction.

[0177] Embodiment nine:

[0178] An electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is used to call and run the computer program stored in the memory to execute any of the methods described above.

[0179] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0180] Application examples:

[0181] This example simulates the implementation of the visibility prediction method of the present invention. The basic data selected is the hourly meteorological data of a province from 2000 to 2023. Some parameters set when simulating the implementation of the visibility prediction method of the present invention are as follows: Initialize w factor and w timeare all set to 0.5; μ of temperature variable is set to 3.79, μ of dew point temperature variable is set to 4.74, μ of air pressure variable is set to 1.05, μ of wind speed variable is set to 0.04, μ of precipitation variable is set to 0.01, and μ of visibility variable is set to 0.12; the value of K is 5, the value of α is 2000, the value of τ is 0, and the value of ε is 10-6; the number of input channels of forward convolution, that is, the number of input features, is set to 6, the number of output channels, that is, the hidden layer size is set to 128, and the convolution kernel size is set to 3; the Dropout of the reverse convolution layer is set to 0.1; the Dropout of the adaptive recurrent unit is set to 0.1; the GRU and Bi of the adaptive recurrent unit are The number of input features of LSTM and Dense models is set to 6, and the hidden layer size is set to 1; the input feature dimension of position encoding, that is, the hidden layer size is set to 128, Dropout is set to 0.1, and the maximum sequence length is set to 5000; the number of input features of Transformer encoder-decoder is set to 6, the hidden layer size is 128, the number of output features is set to 1, the number of multi-head attention mechanism heads is set to 6, the number of encoder and decoder layers is set to 4, the hidden layer size of the feedforward neural network is set to 256, and Dropout is set to 0.1; the model training batch size is set to 64, the number of training rounds is set to 200, and the learning rate is set to 0.0001;

[0182] The visibility signal decomposition data results are as follows: Figure 2 As shown in the figure, the original visibility signal was modally decomposed into five intrinsic mode function components (IMF1-IMF5), among which IMF1 has a lower frequency and a larger amplitude variation, reflecting the long-term trend of the visibility signal and can capture the overall change characteristics of visibility. The frequencies of IMF2-IMF5 components gradually increase and the amplitudes are lower, mainly reflecting the short-term fluctuations contained in the visibility signal, and the high-frequency components can reveal the rapid changes in local meteorological conditions.

[0183] The accuracy verification results of the training set and the test set are shown in Table 1. The prediction results of the training set and the verification set are shown in Figure 3 and Figure 4 As shown,

[0184] Table 1 Model accuracy verification results

[0185] RMSE MAE MAPE <![CDATA[R 2 ]]> Training set 0.13518 0.10479 1.117％ 0.99689 Test Set 0.21036 0.15932 1.9325％ 0.99568

[0186] The RMSE and MAE of the model on the training set are both kept at a low level, with MAPE being only 1.117%, indicating that the deviation between the predicted value and the actual value is small. 2It reaches 0.99689, indicating that the model can explain most of the changes in visibility data and has a good fitting effect. On the test set, RMSE and MAE increase slightly to 0.21036 and 0.15932 respectively, but are still at a low level. MAPE is 1.9325%, indicating that the model still maintains a high prediction accuracy on unseen data. 2 The value is 0.99568, showing good generalization ability. Overall, the performance of the model on the training set and the test set proves its effectiveness and robustness in the visibility prediction task.

[0187] Figure 3 and Figure 4 The comparison between the prediction results of the training set model and the test set model is revealed respectively. It can be seen that the model shows good prediction ability in both the training set and the test set, and can accurately reflect the fluctuation trend of visibility.

[0188] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a technician familiar with the technical field within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A visibility prediction method, characterized in that: include: Acquiring meteorological factor data, wherein the meteorological factors include environmental factors and visibility; Preprocessing the meteorological factor data includes outlier processing and missing value processing considering the variability of the meteorological factor data and dynamic normalization processing considering the fluctuation information of the meteorological factor data; Perform modal decomposition on the preprocessed visibility to form several intrinsic modal function components with different center frequencies; Constructing a meteorological data set using the environmental factors in the preprocessed meteorological factor data and the plurality of intrinsic mode function components; Based on the meteorological data set, construct and train a dynamic convolution adaptive recursive visibility prediction model corresponding to each intrinsic mode function component; The current environmental factors are respectively input into the dynamic convolution adaptive recursive visibility prediction model of different intrinsic modal function components, and the different intrinsic modal function components corresponding to the current environmental factors are obtained. The different intrinsic modal function components corresponding to the current environmental factors are fused to get the visibility prediction result.

2. The visibility prediction method according to claim 1, characterized in that: The environmental factors include temperature, dew point temperature, air pressure, wind speed and precipitation.

3. The visibility prediction method according to claim 1, characterized in that: The specific steps of the outlier processing include: Arrange the meteorological factors and obtain the data sequence of each meteorological factor; For each meteorological factor, a fixed initial window size W0 is used to slide over the data sequence and the mean value μ of the data in each window is calculated. i and standard deviation σ i , and then calculate the coefficient of variation CV of the data in each window i : Calculate the average value of the coefficient of variation of each meteorological factor window For the i-th window, determine the dynamic window size W i , expressed as: Where W min is the minimum value of the dynamic window size; W max is the maximum value of the dynamic window size; Use the calculated dynamic window size W i , starting from the beginning of the data sequence, divide each window in turn, each window contains W i data points; In each dynamic window, the upper quartile Q3 and lower quartile Q1 of each meteorological factor are calculated. The upper quartile is the location of 75% of the data points in the sorted data, and the lower quartile Q1 is the location of 25% of the data points in the sorted data. The interquartile range of each meteorological factor in each dynamic window is calculated, which is expressed as follows: IQR = Q3-Q1; where IQR represents the interquartile range, Q3 represents the upper quartile, and Q1 represents the lower quartile; Calculate the lower and upper limits of the abnormal values ​​of each meteorological factor in each window, expressed as follows: Line1=Q1-1.5×IQR Line2=Q3+1.5×IQR In the formula, Line1 represents the lower limit of the outlier, Line2 represents the upper limit of the outlier, and IQR represents the interquartile range; The outliers are found and deleted according to the lower and upper limits of the outliers calculated for each window. The range of outliers is expressed as follows: Outliers = {x outer ∣x outer <Line1,x outer >Line2} In the formula, x outer Indicates outliers, Line1 indicates the lower limit of outliers, and Line2 indicates the upper limit of outliers; Mark the locations of outliers.

4. The visibility prediction method according to claim 3, characterized in that: The specific steps of the missing value processing include: The data sequence is used as a sample to form the original data set T, T = {(x i ,t i )|i=1,2,…,N}; at the same time, the outlier mark position is obtained in the data sequence and used as the missing value position, and the data sequence without outlier mark is used as a sample, and some data are randomly selected from it as the interpolation verification set T'. The interpolation verification set T' is expressed as: T′={(x i ′,t i ′)∣i=1,2,…,N′} In the formula, x i ′ represents the observed value of a meteorological factor in the i-th sample in the interpolation validation set, t i ′ represents the time label corresponding to the i-th sample in the interpolation validation set, and N′ represents the total number of samples of meteorological data in the interpolation validation set; For each sample in the interpolation validation set, the weighted distance d(i,j) between it and other samples is calculated to measure the similarity between the two samples, which is expressed as: In the formula, d(i,j) represents the weighted distance between sample i and sample j, and w factor and w time are the weight factors of meteorological factor value and time difference, respectively, i ′ and x j ′ are the meteorological factor observation values ​​of sample i and sample j respectively, t i ′ and t j ′ are the time labels of sample i and sample j respectively; Initialize the weight factor w of the difference in meteorological factor values factor The weight factor w of the time difference between the meteorological factor time The value of Define the mean square error as the objective function and define its convergence threshold μ, which can be expressed as: μ=0.01×σ 2 Where M is the number of samples with non-missing observations in the interpolation validation set, x i ′ represents the true observed value of the i-th known sample in the interpolation validation set, represents the interpolated value calculated using the current weight factor, μ represents the threshold set to make the objective function converge, and σ represents the standard deviation of the data before simulating the missing value of each meteorological factor in the interpolation validation set; Use the gradient descent method to automatically adjust and optimize the weight factor until the objective function converges to the set threshold μ, and obtain the optimized weight factor w factor and w time , expressed as: In the formula, η is the learning rate used by the gradient descent method, k is the number of iterations of the gradient descent method, and are the partial derivatives of the objective function with respect to the weight factor; After optimization, the weight factor w factor and w time For each sample with missing values ​​in the original data set T, calculate its weighted distance d(i, j) to all other samples, sort them from small to large according to the distance, select the K samples with the closest distance to form the nearest neighbor set, which can be expressed as: N K (i)={x j1 ,x j2 ,…,x jK } Where N K (i) represents the nearest neighbor sample set of sample i, x jK is the meteorological factor observation value of the kth sample closest to sample i; The missing meteorological factor values ​​are interpolated using the weighted average of the nearest neighbor samples, expressed as: In the formula, represents the interpolation estimate of the missing sample i, x j is the meteorological factor observation value of the nearest neighbor sample j, w j is the weight of sample j, ∈ is a small constant to prevent division by zero.

5. The visibility prediction method according to claim 4, characterized in that: The specific steps of the dynamic normalization process considering the fluctuation information of meteorological factor data include: The original data set T after outlier and missing value processing is i ,t i )|i=1,2,…,N} is divided into training set and validation set according to a certain ratio; In each dynamic window corresponding to each meteorological factor, each meteorological factor is normalized; Traverse each meteorological factor and initialize an array with the same shape as its original data for the kth meteorological factor to store the normalized data; Starting from the beginning of the data sequence, according to the dynamic window size W i,k , divide each window in turn, determine the start and end index of the i-th window, and express it as: e i =s i +W i,k -1 In the formula, s i Indicates the starting index of the i-th window, s i-1 represents the starting index of the i-1th window, e i represents the end index of the i-th window, and if e i >N, then e i =N; Extract the data in the window and calculate the maximum and minimum values ​​in the window. The sample set in the i-th window is X i,k ={x j ∣j∈[s i ,e i ]}, x j is the observed value of the kth meteorological factor in the jth sample, and the maximum and minimum values ​​are calculated using the formula: In the formula, represents the maximum value of the kth meteorological factor in the i-th window, represents the minimum value of the kth meteorological factor in the i-th window; Normalize the data in the window. For each sample x in the window j ∈X i,k , the normalized value is expressed as: In the formula, is the normalized value of the kth meteorological factor in the jth sample. When To avoid division by zero errors; The normalized value Stored in the corresponding location; Update the window index, expressed as: s i+1 =e i +1 If s i+1 >N, the processing of the remaining data ends; Repeat the dynamic normalization process for all meteorological factors until all meteorological factors have completed dynamic normalization.

6. The visibility prediction method according to claim 5, characterized in that: The specific steps of performing modal decomposition on the preprocessed visibility to form a plurality of intrinsic modal function components with different center frequencies include: For each modal classification, the analysis signal is calculated by Hilbert transform, and the one-sided spectrum is calculated; the central band of the visibility data is transferred to the corresponding baseband area by aliasing an index corresponding to the center frequency of the modal component: In the formula, u k (t) represents the kth intrinsic mode function of visibility data, ω k (t) represents the center frequency of the kth intrinsic mode function of the visibility data, δ(t) represents the Dirac function, and the value range of k is [1, MAX]; MAX is the total number of intrinsic mode function components of the visibility data after decomposition; Calculate the square norm L2 of the demodulation gradient to estimate the bandwidth of each modal component and solve the following constraints: In the formula, u k ={u1,u2,…,u k } represents the intrinsic mode function component of visibility data, ω k ={ω1,ω2,…,ω k } represents the center frequency of the intrinsic mode function component of the visibility data, and f(t) represents the visibility data signal; The Lagrange multiplier λ(t) and the second-order penalty factor α are introduced to transform the constrained variational problem into an unconstrained variational problem: In the formula, α is the second-order penalty factor, λ(t) is the Lagrange multiplier; By continuously updating the components and the center frequency using the alternating direction multiplier method, the saddle point of the unconstrained model is finally obtained: Where, f(ω), They represent f(t) and u respectively. k (t), Fourier transform of λ(t), n represents the number of iterations; To u k 1 ,ω k 1 , 1 Initialize, set the number of iterations to 1, execute cycle n = n + 1, update u k ,ω k , and double-boost λ, which can be expressed as: When the following equation is satisfied, the iteration stops Where γ represents the step size parameter, which is used to control the update rate of the Lagrange multiplier; τ represents the noise tolerance; ε represents the convergence threshold, which is used to determine whether the iteration converges.

7. The visibility prediction method according to claim 6, characterized in that: The dynamic convolution adaptive recursive visibility prediction model includes a bidirectional convolution layer, an adaptive recursive unit and a Transformer encoder-decoder; The steps of constructing the dynamic convolution adaptive recursive visibility prediction model are as follows: Construct a bidirectional convolutional layer, which is used to process the convolution operation of the meteorological factor data sequence; Forward convolution uses one-dimensional convolution to input the meteorological factor data sequence data T = {(x i ,t i )|i=1,2,…,N} to perform convolution operation and extract the forward features of the meteorological factor sequence, which can be expressed as: F fwd (i) = Conv1D fwd (x i ) In the formula, x i is the meteorological factor feature vector at the i-th time step, which contains the observed values ​​of all meteorological factors. fwd (i) is the output of the forward convolution, Conv1D fwd Represents a forward one-dimensional convolution operation; Backward convolution, flip the input data to get the reverse sequence x rev (i) = x N-i+1 , and then use one-dimensional convolution to perform convolution operation. The convolution kernel size and filling method are the same as the forward convolution. The backward features of the meteorological factor sequence are extracted, which can be expressed as: F bwd (i) = Conv1D bwd (x rev (i)) In the formula, F bwd (i) is the output of the backward convolution, Conv1D bwd Represents a backward one-dimensional convolution operation; Output fusion, taking the average of the outputs of the forward convolution and the backward convolution as the output of this layer, is expressed as: In the formula, F conv (i) is the output feature of the bidirectional convolutional layer; Use the Dropout layer to prevent overfitting and improve the generalization ability of the model; Adaptive recursive unit is used to process recursive feature extraction of time series data; The steps to construct an adaptive recursive unit are as follows: The output F of the bidirectional convolutional layer conv (i) Input to the gated recurrent unit, expressed as: In the formula, represents the hidden state of GRU at the i-th time step, and GRU represents the recursive calculation function of the GRU unit; Use a bidirectional long short-term memory network to process the input data and merge the bidirectional outputs, which can be expressed as: In the formula, and are the hidden states of the forward and backward LSTM, respectively. is the output of the bidirectional LSTM, LSTM forward and LSTM backward Represent the recursive calculation functions of the forward LSTM unit and the backward LSTM unit respectively; Use the fully connected layer to generate the gating signal, and adaptively select the outputs of GRU and BiLSTM through the gating signal; calculate the gating signal through the fully connected layer, and perform weighted summation on the outputs of GRU and BiLSTM to achieve adaptive selection, which can be expressed as: α(i)=σ(W α ·F cinv (i)+b α ) Where W α is the weight matrix of the fully connected layer, b α is the bias vector of the fully connected layer, σ is the Sigmoid activation function, α(i) represents the gated signal vector of the i-th time step, which determines the weights of the GRU and BiLSTM outputs. is the final output of the adaptive recurrent unit at the i-th time step, ⊙ represents the element-by-element product; Use the Dropout layer to prevent overfitting and improve the generalization ability of the model; The Transformer encoder-decoder is used to further process the output of the adaptive recursive unit, extract global features, and predict visibility. The specific implementation steps of the Transformer encoder-decoder are as follows: Use sine and cosine functions to generate a position encoding matrix, where each position encoding vector corresponds to a feature dimension, and add the position encoding to the input data, expressed as: Where, t i is the time label of the ith time step, d midel is the feature dimension of the model, that is, the dimension of the input and output feature vectors, PE (i,2j) represents the sinusoidal position encoding value of the ith time step in the 2jth dimension, PE (i,2j+1) represents the cosine position encoding value of the ith time step in the 2j+1 dimension, PE i Represents the position encoding vector of the i-th time step, containing the encoding values ​​of all dimensions, F input (i) represents the input feature vector after adding position encoding; The multi-head attention mechanism is used to perform attention calculation on each position in the input sequence. The input data is divided into multiple subspaces, each subspace corresponds to a head, and each head performs attention calculation separately. The output of the multi-head attention mechanism is the connection result of each head, which is expressed as: Q=F input W Q ,K=F input W K ,V=F inpuh W V MultiHead(Q,K,V)=Concat(head1,…,head H )W O In the formula, Q, K, and V represent the query matrix, key matrix, and value matrix respectively. h , K h 、V h represents the query, key, and value matrix of the h-th head, Attention(Q h ,K h ,V h ) represents the attention calculation function of the hth head, head h represents the output matrix of the hth head, H represents the number of attention heads, and W Q , W K , W V is the learnable weight matrix, W O is the output transformation matrix, MultiHead(Q,K,V) is the final output of the multi-head attention mechanism, softmax() means normalizing the attention weights of each time step, Concat() means concatenating the outputs of all heads in the feature dimension, h = 1, 2, ..., H; The results of the attention calculation are further processed using a feedforward neural network, which consists of two fully connected layers; the first layer performs a nonlinear transformation on the output of the multi-head attention mechanism, and the second layer maps the transformed result to the output space; the feedforward neural network is expressed as: FFN(x)=ReLU(xW1+b1)W2+b2 Where W1 and W2 are weight matrices, b1 and b2 are bias vectors, ReLU is the activation function, and FFN(x) represents the output of the feedforward neural network; Construct encoder and decoder layers, stack multiple encoder and decoder layers containing multi-head attention mechanism and feedforward neural network to build a complete Transformer model; the encoder is used to process the input sequence and extract features; The decoder generates visibility predictions based on the encoder output.

8. A visibility prediction system, characterized in that: It includes meteorological factor data acquisition module, preprocessing module, modal decomposition module, data set construction module, model construction module and prediction reorganization module; The meteorological factor data acquisition module is used to acquire meteorological factor data, where meteorological factors include environmental factors and visibility; The preprocessing module is used to preprocess the meteorological factor data, and the preprocessing includes abnormal value processing and missing value processing considering the variability of the meteorological factor data and dynamic normalization processing considering the fluctuation information of the meteorological factor data; The modal decomposition module is used to perform modal decomposition on the preprocessed visibility to obtain key features; The data set construction module is used to construct a meteorological data set based on environmental factors and key features in meteorological factor data; The model building module is used to train a pre-built dynamic convolution adaptive recursive visibility prediction model based on a meteorological data set; The prediction reorganization module is used to reorganize the key features output by the trained dynamic convolution adaptive recursive visibility prediction model to complete visibility prediction.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

Images