Meteorological data processing method and device based on edge protection gateway algorithm

Abstract

The embodiment of the invention provides a meteorological data processing method and device based on an edge protection gateway algorithm. The method is applied to the technical field of data processing, and comprises the following steps: acquiring meteorological data by utilizing an edge computing node, and transmitting, verifying and filtering through a mesh network to obtain an initial data set; establishing a protection mechanism for encrypted storage, and generating a hash value in combination with the block chain; establishing a standardized model for noise filtering and data restoration; constructing a fusion model to extract features; establishing a federated learning framework for prediction; and establishing a verification mechanism, and generating decision suggestions. According to the invention, by deploying a data processing and security protection mechanism at the edge node, efficient processing and security protection of meteorological data are realized, and the real-time performance and reliability of data processing are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a meteorological data processing method and device based on an edge protection gateway algorithm. Background Art

[0002] With the continuous expansion of meteorological observation networks and the increasing richness of observation methods, meteorological data presents the characteristics of multi-source, heterogeneous, and large-scale. Traditional meteorological data processing methods mainly rely on centralized data processing architectures to uniformly transmit decentralized observation data to central servers for processing and analysis. This method deploys professional meteorological observation equipment to collect data on various meteorological elements such as temperature, humidity, and air pressure, and transmits the data to the processing center via wired or wireless networks. In the data processing process, standardized processing, quality control, data assimilation and other technical means are used to extract meteorological characteristics and build prediction models to provide support for meteorological forecasting and decision-making.

[0003] However, the traditional centralized processing method has many shortcomings: first, the remote transmission of a large amount of raw data is prone to cause network congestion and increase the delay of data processing; second, centralized storage and processing face greater data security risks and it is difficult to protect the privacy of sensitive data; third, the format and quality of meteorological data from different sources vary greatly, and the data fusion effect is not ideal; finally, the training of the prediction model depends on centralized computing resources, and it is difficult to fully utilize the computing power of distributed nodes. These problems seriously restrict the efficiency and accuracy of meteorological data processing and prediction. Summary of the invention

[0004] The present invention provides a meteorological data processing method and device based on an edge protection gateway algorithm, which solves the technical problems of low data transmission efficiency, poor security, and unsatisfactory fusion effect under the traditional centralized processing mode. By deploying data processing and security protection mechanisms at edge nodes, efficient processing and security protection of meteorological data are achieved, and the real-time and reliability of data processing are improved.

[0005] According to a first aspect of the present disclosure, a meteorological data processing method based on an edge protection gateway algorithm is provided, comprising:

[0006] Use edge computing nodes to collect meteorological observation data in a distributed manner, transmit the data through the mesh network, and filter it through the verification module to obtain the initial meteorological data set;

[0007] Establish a protection mechanism based on the initial meteorological data set, perform encrypted storage through dynamic key management, build a baseline model using a deep learning algorithm, generate a hash value in combination with blockchain technology, and obtain encrypted meteorological data;

[0008] A standardized processing model is established based on the encrypted meteorological data, noise is filtered through sliding windows and wavelet transform, and anomaly identification and data repair are performed using an adaptive algorithm to obtain a preprocessed data set;

[0009] A weight fusion model is constructed based on the preprocessed data set, spatiotemporal features are extracted by tensor decomposition, and feature learning and screening are performed using a deep learning network to obtain fused feature data;

[0010] Establishing a federated learning framework based on the fused feature data, building a prediction model through an attention mechanism and a gated recurrent unit, and performing prediction using an integrated learning method to obtain a prediction result;

[0011] A verification mechanism is established based on the prediction results, decision rules are constructed through knowledge graphs, adaptive algorithms are used to optimize decision strategies, and plans are generated in combination with early warning mechanisms to obtain decision recommendations.

[0012] According to a second aspect of the present disclosure, a meteorological data processing device based on an edge protection gateway algorithm is provided, comprising:

[0013] The acquisition module is used to collect meteorological observation data in a distributed manner using edge computing nodes, transmit the data through the mesh network, and filter it through the verification module to obtain the initial meteorological data set;

[0014] Establish a module for establishing a protection mechanism based on the initial meteorological data set, encrypting and storing the data through dynamic key management, building a baseline model using a deep learning algorithm, and generating a hash value in combination with blockchain technology to obtain encrypted meteorological data;

[0015] A filtering module is used to establish a standardized processing model based on the encrypted meteorological data, perform noise filtering through a sliding window and wavelet transform, and perform anomaly identification and data repair using an adaptive algorithm to obtain a preprocessed data set;

[0016] A screening module is used to construct a weight fusion model based on the preprocessed data set, extract spatiotemporal features through tensor decomposition, and use a deep learning network to perform feature learning and screening to obtain fused feature data;

[0017] A prediction module is used to establish a federated learning framework based on the fused feature data, construct a prediction model through an attention mechanism and a gated recurrent unit, and perform prediction using an integrated learning method to obtain a prediction result;

[0018] The generation module is used to establish a verification mechanism based on the prediction results, build decision rules through the knowledge graph, optimize the decision strategy using an adaptive algorithm, generate a plan in combination with the early warning mechanism, and obtain decision recommendations.

[0019] The present disclosure realizes the local collection and preprocessing of meteorological data through the distributed collection mechanism of edge computing nodes, reduces the data transmission load, and improves the real-time performance of data collection; adopts the multi-path transmission mode of mesh network to enhance the reliability of data transmission and the fault tolerance of the network; and filters the data in combination with the verification module to ensure the validity of the data. In terms of data security, the protection mechanism constructed by dynamic key management and deep learning algorithm realizes the secure storage and access control of data, and the introduction of blockchain technology ensures the immutability and traceability of data. The sliding window and wavelet transform technology used in the standardized processing model effectively removes the noise interference in the data, and the application of adaptive algorithm improves the accuracy of anomaly identification and data repair. The weight fusion model realizes the effective extraction and screening of spatiotemporal features through the combination of tensor decomposition and deep learning network, and enhances the richness of feature expression. The introduction of the federated learning framework solves the problem of data islands, the application of attention mechanism and gated recurrent unit enhances the model's ability to capture key features, and the integrated learning method improves the stability and reliability of the prediction results. Finally, the decision support mechanism based on knowledge graph and adaptive algorithm realizes the intelligent optimization and dynamic adjustment of decision rules, and the combination of early warning mechanism ensures the timeliness and practicality of decision suggestions. The organic combination of these technical effects comprehensively improves the efficiency, security and accuracy of meteorological data processing.

[0020] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0022] Figure 1 A flow chart of a meteorological data processing method based on an edge protection gateway algorithm according to an embodiment of the present disclosure is shown;

[0023] Figure 2 A block diagram of a meteorological data processing device based on an edge protection gateway algorithm according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0025] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0026] Figure 1 A schematic flow chart of a meteorological data processing method 100 based on an edge protection gateway algorithm in an embodiment of the present disclosure is shown, as shown in FIG. Figure 1 As shown, the method 100 includes:

[0027] S110: Use edge computing nodes to collect meteorological observation data in a distributed manner, transmit the data through a mesh network, and filter the data through a verification module to obtain an initial meteorological data set;

[0028] Optionally, temperature, humidity, air pressure, wind speed and precipitation data are collected through multi-dimensional sensors, and meteorological observation data is formed through data integration; spatial stratification is performed based on the meteorological observation data, real-time collection is performed through the bottom layer, data is temporarily stored in the middle layer, and the top layer is responsible for communication scheduling to obtain layered processed data; a validity threshold range is set based on the layered processed data, anomalies are marked through numerical comparison, and data exceeding the threshold range is eliminated to obtain a valid data set; a mesh network topology is constructed based on the valid data set, multiple transmission paths are calculated through a routing algorithm, and the optimal transmission path is selected according to the network load to obtain a transmission data stream; data packets are segmented using the transmission data stream, data integrity is ensured through checksum calculation, data is encapsulated according to the transmission protocol, and a data packet to be verified is obtained; data validity is checked based on the data packet to be verified, data timing is ensured through timestamp verification, and data consistency is verified based on spatial correlation to obtain an initial meteorological data set.

[0029] Among them, the edge computing node is a data processing device distributed at the edge of the meteorological observation network. By deploying multiple edge computing nodes at different locations in the observation network, distributed collection of meteorological data is realized. Multidimensional sensors include temperature sensors, humidity sensors, air pressure sensors, wind speed sensors, and precipitation sensors. Each sensor collects corresponding meteorological element data. The temperature sensor collects ambient temperature in degrees Celsius; the humidity sensor collects relative humidity of the air, and the data is expressed in percentage; the air pressure sensor collects atmospheric pressure in hectopascals; the wind speed sensor collects wind speed in meters per second; and the precipitation sensor collects precipitation in millimeters. During the data integration process, the raw data collected by each sensor is first time-aligned to ensure that the data has the same sampling timestamp, and then the aligned data is organized according to a unified data structure to form meteorological observation data containing multiple meteorological elements.

[0030] Spatial stratification is to divide the data processing function into three levels: bottom layer, middle layer and top layer. The bottom layer is responsible for real-time data collection and interacts directly with sensors. The collection frequency is set according to the changing characteristics of different meteorological elements. Temperature and humidity are collected once a minute, air pressure is collected once every 5 minutes, wind speed is collected once every 30 seconds, and precipitation is collected once an hour. The middle layer is responsible for data temporary storage and uses a cache mechanism to store short-term data. When the data volume reaches the preset threshold or the storage time reaches the set value, the data is transmitted to the top layer. The top layer is responsible for communication scheduling, managing data transmission tasks, and determining the priority and routing method of data transmission. The validity threshold range is set based on the physical characteristics and historical statistical characteristics of meteorological elements. Abnormal data is identified by numerical comparison, and the data is marked when it exceeds the threshold range. The data elimination process considers the temporal and spatial correlation of the data to avoid accidental deletion of valid data. The eliminated data forms a valid data set to ensure the reliability of the data.

[0031] The mesh network topology is a distributed network architecture in which each node can be used as both a data source and a data relay. The routing algorithm calculates multiple possible transmission paths based on the network topology and dynamically selects the optimal transmission path based on the network load. The network load includes indicators such as link bandwidth utilization, node processing capacity, and transmission delay. The packet segmentation process divides a large data stream into fixed-size packets, each of which contains header information and data. The checksum calculation uses a cyclic redundancy check method to ensure the integrity of data transmission. Data encapsulation adds necessary control information according to the requirements of the transmission protocol to form a standard packet format.

[0032] Data validity verification includes timestamp verification and spatial correlation verification. Timestamp verification ensures the temporal sequence of data and checks the continuity and validity of data timestamps. Spatial correlation verification verifies the spatial consistency of data by comparing data from adjacent observation sites.

[0033] For example: In an observation network, the edge computing node collects data of temperature 23.5℃, relative humidity 65%, air pressure 1013.2 hPa, wind speed 4.2 m / s, and precipitation 2.1 mm through multi-dimensional sensors within a sampling period. These data are time-aligned to form meteorological observation data. The data flows in a three-layer architecture. The bottom layer completes real-time collection, and the middle layer temporarily stores the data for 30 minutes. During this period, 1800 records are accumulated. When the preset threshold of 1000 records is reached, data transmission to the top layer is triggered. The data is verified for validity. The temperature is within the effective range of -40℃ to 50℃, the relative humidity is within the range of 0-100%, and the air pressure is within the range of 800-1100 hPa. The routing algorithm in the mesh network finds three possible transmission paths. Through calculation, it is found that the network load of path 1 is 85%, path 2 is 60%, and path 3 is 75%. Path 2 with the smallest load is selected for data transmission. The data is divided into multiple 256-byte data packets, and a 32-bit cyclic redundancy check code is calculated for each data packet. Finally, timestamp verification is used to confirm that the data is transmitted in the order in which it was collected, and the spatial consistency of the data is verified by comparing the observations of adjacent stations to form a reliable initial meteorological data set.

[0034] S120: Establish a protection mechanism based on the initial meteorological data set, encrypt and store it through dynamic key management, build a baseline model using a deep learning algorithm, generate a hash value in combination with blockchain technology, and obtain encrypted meteorological data;

[0035] Optionally, identity features are extracted based on the initial meteorological data set, access requests are graded through zero-trust authentication, authentication tokens are assigned based on access levels, and authentication data packets are obtained; dynamic keys are generated using the authentication data packets, keys are distributed through an asymmetric encryption algorithm, and keys are updated based on time windows to obtain encryption key sequences; data are encrypted in blocks based on the encryption key sequence, a secure communication tunnel is established through a channel encryption protocol, and data is graded for protection based on encryption strategies to obtain encrypted data streams; a behavioral feature sequence is established based on the encrypted data stream, data transmission patterns are extracted through a deep learning algorithm, baseline judgment rules are constructed based on normal behavioral features to obtain a baseline model; data streams are monitored in real time using the baseline model, anomalies are calculated through feature matching, security responses are performed based on threat levels, and security status data are obtained; blocks are generated based on the security status data and encrypted data streams, data integrity is verified through a consensus mechanism, a data traceability mechanism is established based on timestamps and hash links, and encrypted meteorological data are obtained.

[0036] Among them, the identity feature extraction process is based on the access features in the initial meteorological data set, including access source, access time, access frequency and other information. The zero-trust authentication mechanism strictly verifies each access request and does not preset a trust relationship. Permission classification divides permissions into different levels according to access sensitivity, such as viewing permissions, modification permissions and management permissions. The access level is bound to the authentication token, which contains attributes such as visitor identity information, permission level and validity period. Dynamic key generation is based on the information in the authentication data packet and uses key dispersion technology to generate independent encryption keys. Asymmetric encryption algorithms use public keys and private keys to distribute keys to ensure the security of key transmission. The time window sets the key update cycle and regularly updates the key to prevent the key from being cracked. The encryption key sequence contains multiple sets of time-limited keys to achieve dynamic replacement of keys.

[0037] Data block encryption divides data into fixed-size blocks, each of which is encrypted using a different key. The channel encryption protocol establishes a secure communication tunnel during data transmission to prevent data from being stolen or tampered with. Data classification protection uses encryption algorithms of different strengths according to the sensitivity of the data to form a multi-level encrypted data stream. The behavior feature sequence records the behavior features during data transmission, including information such as transmission time, transmission volume, and transmission path. The deep learning algorithm extracts data transmission patterns through multi-layer neural networks and identifies the features of normal transmission behavior. The baseline judgment rule is established based on normal behavior features and is used to identify abnormal behavior.

[0038] The feature matching process compares the real-time data stream with the baseline model and calculates the abnormality of the behavior feature. When the abnormality exceeds the threshold, a security response is triggered. The response measures include blocking suspicious connections, switching to backup channels, etc. The security status data records the security status of the system, including abnormal events, response measures, and processing results. The block generation process organizes the security status data and encrypted data streams into a block structure. Each block contains data content, timestamp, and hash value of the previous block. The consensus mechanism ensures the integrity of block data through multi-node verification. The timestamp and hash link form a complete blockchain structure to support data traceability.

[0039] For example, an access request carries user identification information and access purpose. After passing zero-trust authentication, it is granted read-only permission, and the system generates an authentication token containing permission information. A 512-bit dynamic key is generated based on the token information, and the RSA algorithm is used for key distribution, with a key update cycle of 4 hours. Data is encrypted in blocks of 1MB in size, and a TLS secure channel is established to transmit encrypted data. The deep learning model analyzes historical transmission records to establish a baseline model that includes features such as transmission time distribution and data volume changes. Real-time monitoring found that the data transmission volume suddenly increased by 10 times, and the abnormality calculation result exceeded the warning threshold, so the system automatically started data transmission flow limiting measures. Finally, the security status data and encrypted data of the entire process are packaged into blocks, and the data consistency is ensured through a consensus mechanism to form a traceable blockchain structure.

[0040] S130: Establish a standardized processing model based on the encrypted meteorological data, filter noise through sliding windows and wavelet transform, use adaptive algorithms to identify anomalies and repair data, and obtain a preprocessed data set;

[0041] Optionally, a unified data format specification is established based on the encrypted meteorological data, the format of multi-source data is unified through unit conversion, and the data is aligned according to the time mark to obtain standard format data; a sliding window sequence is constructed using the standard format data, the data is segmented by adaptively adjusting the window size, and the optimal window parameters are calculated according to the data change rate to obtain windowed data; wavelet basis functions are selected based on the windowed data, data features are extracted through multi-scale decomposition, and thresholds are determined based on energy distribution for denoising to obtain filtered data; spatiotemporal correlation indicators are calculated based on the filtered data, anomaly judgment criteria are constructed through probability density estimation, and abnormal samples are identified based on the multidimensional feature space to obtain data to be repaired; data to be repaired are classified according to the missing type, short-term missing data are filled by a time series interpolation algorithm, and long-term missing data are reconstructed based on spatial correlation to obtain repaired data; quality assessment is performed using the repaired data, data availability is calculated through integrity indicators, and data is screened based on accuracy standards to obtain a preprocessed data set.

[0042] Among them, unified data format specifications are the basis for establishing standardized processing, and the formats of different data sources in encrypted meteorological data are unified. The unit conversion process converts different units of measurement into standard units, such as temperature is unified as degrees Celsius and wind speed is unified as meters per second. Time stamp alignment ensures that the sampling time of different data sources is consistent. The timestamp is used as the alignment standard, and the data time points are aligned through interpolation or downsampling. The construction of the sliding window sequence adopts an overlapping sliding method, and the window size is dynamically adjusted according to the characteristics of the data. The adaptive adjustment of the window size is based on the data change rate. When the change rate is large, the window size is reduced, and when the change rate is small, the window size is increased. The data change rate is obtained by calculating the difference between adjacent data points, and the window parameter optimization considers data stability and computational efficiency.

[0043] The selection of wavelet basis functions takes into account the time-frequency characteristics of the data. Commonly used wavelet basis functions include Haar wavelet, db wavelet, and sym wavelet. Multiscale decomposition decomposes the data into different frequency components and extracts the characteristics of the data at different scales. Energy distribution analysis is based on the amplitude of the wavelet coefficients. It removes noise by setting the energy threshold and retains the main signal components. The spatiotemporal correlation indicators include the temporal autocorrelation coefficient and the spatial cross-correlation coefficient, which are used to describe the spatiotemporal correlation characteristics of the data. The probability density estimation uses the kernel density estimation method to construct a data distribution model. The anomaly judgment criterion is based on the statistical characteristics of the data and identifies abnormal samples that deviate from the normal distribution in the multidimensional feature space.

[0044] Missing data classification is based on the duration of missing data, and missing data are divided into short-term missing data and long-term missing data. Short-term missing data are filled using time series interpolation algorithms, such as linear interpolation or spline interpolation. Long-term missing data are reconstructed through spatial correlation, and missing values ​​are estimated using data from adjacent observation points. Quality assessment uses a multi-dimensional indicator system, where completeness indicators reflect the missingness of data and accuracy indicators measure the reliability of data. Data availability calculation is based on the comprehensive score of quality indicators, and high-quality data is screened by setting thresholds.

[0045] For example, for the temperature data series, first unify the units and convert Fahrenheit to Celsius. Use a 60-minute sliding window with a window overlap rate of 50%, and adaptively adjust the window size between 30 minutes and 120 minutes according to the temperature change rate. Select the db4 wavelet basis function for 4-layer decomposition, analyze the energy distribution of the wavelet coefficients, and set the energy threshold to remove high-frequency noise. Calculate the temporal autocorrelation coefficient of the temperature series to identify abnormal temperature values ​​that do not conform to historical rules. For short-term missing values ​​within 10 minutes, use cubic spline interpolation to fill; for long-term missing values ​​of more than 2 hours, use the temperature data and spatial correlation of surrounding observation stations for reconstruction. Finally, by calculating the data integrity and accuracy indicators, screen out data with a quality score greater than 0.8 and retain them in the preprocessing data set.

[0046] S140: construct a weight fusion model based on the preprocessed data set, extract spatiotemporal features through tensor decomposition, use a deep learning network to learn and filter features, and obtain fused feature data;

[0047] Optionally, the reliability of the data source is calculated according to the preprocessed data set, the initial weight is determined by evaluating the timeliness of the data, and the weight is dynamically updated according to the quality index to obtain the weight coefficient; the multi-source data is weightedly combined using the weight coefficient, the time difference is eliminated by data alignment, and the data association is established based on the spatial mapping to obtain the fused data set; a three-dimensional tensor structure is constructed based on the fused data set, the time series pattern is extracted by time dimension decomposition, and the regional features are obtained by space dimension decomposition to obtain the feature tensor; the feature tensor is subjected to dimensionality reduction processing, the key features are extracted by principal component analysis, the feature dimension is determined according to the variance contribution rate, and the dimensionality reduction feature is obtained; a feature correlation matrix is ​​established according to the dimensionality reduction feature, the feature importance is evaluated by mutual information calculation, and the features are sorted according to the importance threshold to obtain the candidate features; the feature combination is constructed using the candidate features, the feature stability is evaluated by cross-validation, and the features are screened according to the prediction effect to obtain the fused feature data.

[0048] Among them, the calculation of data source reliability is based on multiple evaluation dimensions, including data collection accuracy, transmission stability and historical performance. Data timeliness evaluation takes into account the update frequency and delay of data, with newly collected data given a higher weight and outdated data given a lower weight. Quality indicators include data integrity, consistency and accuracy, which are used to dynamically adjust weight coefficients. The weight coefficient is updated using the exponential weighted average method to ensure that the weight can reflect changes in data quality in a timely manner. The weighted combination process merges data from different data sources according to the weight coefficient. Data alignment uses a time window mechanism to unify data with different sampling frequencies to the same time scale. Spatial mapping establishes the correspondence between data at different spatial locations, taking into account the influence of spatial distance and geographical features. Data association analysis reveals the interactions and dependencies between different data sources.

[0049] The three-dimensional tensor structure organizes data into three-order tensors of time dimension, space dimension and feature dimension. The time dimension decomposition uses tensor decomposition algorithm to extract time series change patterns and identify periodic, trend and mutation characteristics. The spatial dimension decomposition focuses on the spatial distribution characteristics of the data, capturing regional change laws and spatial correlations. The feature tensor contains the temporal and spatial evolution characteristics of the data. Dimensionality reduction processing aims to reduce the data dimension and improve the representativeness of the features. Principal component analysis maps high-dimensional features to low-dimensional space through orthogonal transformation, retaining the main variation information. The variance contribution rate reflects the importance of the feature, and the feature dimension is determined when the cumulative variance contribution rate reaches the preset threshold.

[0050] The feature correlation matrix describes the strength of association between features, and the mutual information calculation quantifies the nonlinear dependency between features. The feature importance evaluation is based on the mutual information value to sort the features by importance. The importance threshold is set taking into account the distinguishing ability and redundancy of the features.

[0051] The construction of feature combinations adopts feature combination strategies to evaluate the prediction performance of different feature combinations. Cross-validation uses the k-fold cross-validation method to evaluate the stability and generalization ability of features. The evaluation of prediction effects uses multiple performance indicators, such as mean square error and correlation coefficient.

[0052] For example, for the data of the meteorological observation station network, the reliability of each station is first calculated based on the data quality indicators. For example, the data integrity of station A is 0.95, the accuracy is 0.92, and the weight coefficient is calculated to be 0.93. During the data alignment process, the temperature data sampled for 5 minutes and the precipitation data sampled for 1 hour are unified to 15-minute intervals. The constructed three-dimensional tensor contains 24 time points, 100 spatial locations, and 10 meteorological elements. The daily variation pattern and spatial distribution characteristics are extracted through tensor decomposition. The principal component analysis reduces the 10-dimensional features to 4 dimensions, retaining 85% of the variance information. Through the mutual information calculation, it is found that the mutual information values ​​of temperature and humidity are high, and the feature combination is optimized. Finally, the feature stability is evaluated through 5-fold cross validation, and the best performing feature combination is selected to form fused feature data.

[0053] S150: Establish a federated learning framework based on the fused feature data, build a prediction model through the attention mechanism and gated recurrent unit, and use the ensemble learning method to make predictions to obtain prediction results;

[0054] Optionally, data is sharded according to the fused feature data, data is distributed among nodes through encrypted transmission, training rules are determined according to the privacy protection strategy, and a distributed data set is obtained; local gradients are calculated using the distributed data set, parameters are aggregated through gradient encryption, and the model is iteratively optimized according to the global update strategy to obtain a local model; an attention weight matrix is ​​constructed based on the local model, feature importance is calculated through temporal correlation, and features are weighted according to the attention score to obtain the focused feature; a gated state sequence is established for the focused feature, redundant information is filtered through a forget gate, and feature combinations are adjusted according to input gates and update gates to obtain a state vector; multiple base learners are generated according to the state vector, the prediction performance is evaluated through cross-validation, and the optimal combination is selected according to the model difference to obtain an integrated model; multiple rounds of predictions are performed on new data using the integrated model, the prediction results are fused through a voting mechanism, and the results are screened according to the confidence level to obtain the prediction results.

[0055] Among them, the data sharding process divides the fused feature data according to time series or spatial regions. Encrypted transmission uses homomorphic encryption technology to ensure the security of data transmission between nodes. The privacy protection strategy sets data access rights and usage scope, and the training rules include parameters such as the number of iterations, learning rate, and batch size. Distributed data sets form data islands between nodes to protect data privacy. Local gradient calculations are performed independently on each node, and the gradient values ​​of model parameters are calculated based on the local data of the node. Gradient encryption uses homomorphic encryption technology to encrypt gradients to ensure the security of the gradient aggregation process. The global update strategy uses the federated average algorithm to update the global model based on the gradient information of each node. The local model is obtained through multiple rounds of iterative optimization.

[0056] The attention weight matrix describes the importance of different features at different time points. The temporal correlation calculation is based on the correlation analysis of the feature sequence to evaluate the influence of the feature on the prediction target. The attention score is normalized by the softmax function to weight the importance of the features and highlight the role of key features.

[0057] The gated state sequence adopts the LSTM (Long Short-Term Memory) structure, which includes a forget gate, an input gate, and an update gate. The forget gate is responsible for filtering irrelevant information, the input gate controls the input of new information, and the update gate adjusts the proportion of state updates. The state vector contains a comprehensive representation of historical information and current input. The base learner contains multiple prediction models with different structures or parameters, such as decision trees, neural networks, and support vector machines. Cross-validation evaluates the prediction performance of each base learner, and the model difference measures the differences between base learners. The optimal combination selects base learners with strong complementarity through an integration strategy.

[0058] For example: In the weather forecasting task, the data is first divided into multiple data shards according to geographical regions, and each node is responsible for data training in one region. After node A calculates the local gradient, it uses homomorphic encryption to protect the gradient information and securely aggregates the gradient with other nodes. The attention mechanism analysis found that the temperature series has a higher weight in the prediction, and a higher attention score is given to this feature. During the gated recurrent unit processing, the forget gate filters out noise interference and retains valid temperature change information. The integrated model includes random forest, GBDT and XGB oo st Three base learners, temperature prediction results obtained by model voting.

[0059] S160: Establish a verification mechanism based on the prediction results, build decision rules through the knowledge graph, use adaptive algorithms to optimize decision strategies, combine early warning mechanisms to generate solutions, and obtain decision recommendations.

[0060] Optionally, perform historical data comparison based on the prediction results, calculate the prediction accuracy through error analysis, perform rationality check based on physical constraint rules, and obtain verification indicators; construct knowledge entity relationships based on verification indicators, establish semantic connections through entity attribute extraction, form reasoning rules based on historical cases, and obtain a knowledge graph; use the knowledge graph to extract rules, establish a decision chain through causal relationship analysis, set rule weights based on expert experience, and obtain a decision rule base; dynamically update the decision rule base, evaluate the rule effect through feedback data, adjust the optimization direction based on target deviation, and obtain an optimization strategy; calculate the warning level threshold based on the optimization strategy, determine the risk level through disaster impact assessment, divide the warning area according to the time and space scale, and obtain a warning plan; generate multiple sets of candidate decisions based on the warning plan, evaluate the plan through cost-benefit analysis, select the optimal plan based on feasibility indicators, and obtain decision recommendations.

[0061] Among them, the historical data comparison process compares the prediction results with the historical real observation values. Error analysis uses multiple statistical indicators, including mean absolute error, root mean square error and relative error. Physical constraint rules are based on the physical characteristics of meteorological elements, such as temperature change range, air pressure change rate, etc., to verify the rationality of the prediction results. The verification indicators comprehensively reflect the reliability of the prediction results.

[0062] The construction of knowledge entity relationships includes entity types such as meteorological elements, weather phenomena, and warning levels. Entity attribute extraction includes the basic characteristics and associated attributes of entities. Semantic connections describe the relationship types between entities, such as "cause", "influence", etc. The reasoning rules of historical cases are based on case analysis, summarizing the development laws and handling experience of events. The rule extraction process extracts decision rules from the knowledge graph. Causal analysis reveals the causal chain between events and forms a complete decision reasoning process. Expert experience is used to adjust the rule weights to ensure the practicality and reliability of the rules. The decision rule library contains a multi-level decision rule system.

[0063] The dynamic update of decision rules is based on the actual application effect. Feedback data includes decision execution results and actual impact assessment. Target deviation analyzes the gap between decision results and expected goals, and guides the optimization direction of rules. Optimization strategies ensure continuous improvement of decision rules. Warning level threshold setting takes into account the degree of harm of meteorological elements. Disaster impact assessment is based on historical disaster data and vulnerability analysis. Warning area division takes into account geographical characteristics and disaster prevention capabilities to ensure the accuracy and effectiveness of warnings. Warning plans include response measures at different levels.

[0064] The generation of candidate decisions considers multiple possible response options. Cost-benefit analysis evaluates the economic and feasibility of the options. Feasibility indicators include technical feasibility, resource feasibility, and time feasibility. The optimal option selection balances multiple decision-making objectives.

[0065] For example, for the prediction results of heavy rainfall, we first compare them with historical rainfall records, calculate the prediction accuracy to be 85%, and verify the rationality of the prediction through the change law of rainfall intensity. In the knowledge graph, we establish the association relationship of "heavy rainfall-inland flooding risk-traffic impact", and extract the decision rule of "inland flooding is prone to occur when the hourly rainfall exceeds 50 mm". According to the actual response effect, adjust the warning activation threshold and response measures. Combined with the terrain characteristics and drainage capacity, the warning area is divided into key prevention areas and general prevention areas. The generated decision recommendations include regional flood control plans and traffic control plans.

[0066] The above is an introduction to the method embodiment. The following is a further explanation of the disclosed solution through an apparatus embodiment.

[0067] Figure 2 FIG. 2 shows a block diagram of a meteorological data processing device 200 based on an edge protection gateway algorithm according to an embodiment of the present disclosure. Figure 2 As shown, the device 200 includes:

[0068] The acquisition module 210 is used to collect meteorological observation data in a distributed manner using edge computing nodes, transmit the data through a mesh network, and filter the data through a verification module to obtain an initial meteorological data set;

[0069] Establishing module 220, for establishing a protection mechanism according to the initial meteorological data set, performing encrypted storage through dynamic key management, building a baseline model using a deep learning algorithm, generating a hash value in combination with blockchain technology, and obtaining encrypted meteorological data;

[0070] A filtering module 230 is used to establish a standardized processing model based on the encrypted meteorological data, perform noise filtering through a sliding window and wavelet transform, and perform anomaly identification and data repair using an adaptive algorithm to obtain a preprocessed data set;

[0071] A screening module 240 is used to construct a weight fusion model based on the preprocessed data set, extract spatiotemporal features through tensor decomposition, and use a deep learning network to perform feature learning and screening to obtain fused feature data;

[0072] A prediction module 250 is used to establish a federated learning framework according to the fused feature data, construct a prediction model through an attention mechanism and a gated recurrent unit, and perform prediction using an integrated learning method to obtain a prediction result;

[0073] The generation module 260 is used to establish a verification mechanism based on the prediction results, construct decision rules through the knowledge graph, optimize the decision strategy using an adaptive algorithm, generate a plan in combination with the early warning mechanism, and obtain decision recommendations.

[0074] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0075] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A meteorological data processing method based on edge protection gateway algorithm, characterized in that: include: Use edge computing nodes to collect meteorological observation data in a distributed manner, transmit the data through the mesh network, and filter it through the verification module to obtain the initial meteorological data set; Establish a protection mechanism based on the initial meteorological data set, perform encrypted storage through dynamic key management, build a baseline model using a deep learning algorithm, generate a hash value in combination with blockchain technology, and obtain encrypted meteorological data; A standardized processing model is established based on the encrypted meteorological data, noise is filtered through sliding windows and wavelet transform, and anomaly identification and data repair are performed using an adaptive algorithm to obtain a preprocessed data set; A weight fusion model is constructed based on the preprocessed data set, spatiotemporal features are extracted by tensor decomposition, and feature learning and screening are performed using a deep learning network to obtain fused feature data; Establishing a federated learning framework based on the fused feature data, building a prediction model through an attention mechanism and a gated recurrent unit, and performing prediction using an integrated learning method to obtain a prediction result; A verification mechanism is established based on the prediction results, decision rules are constructed through knowledge graphs, adaptive algorithms are used to optimize decision strategies, and plans are generated in combination with early warning mechanisms to obtain decision recommendations.

2. The meteorological data processing method based on the edge protection gateway algorithm according to claim 1 is characterized in that: The distributed collection of meteorological observation data using edge computing nodes, transmission of data through a mesh network, and filtering through a verification module to obtain an initial meteorological data set include: Temperature, humidity, air pressure, wind speed and precipitation data are collected through multi-dimensional sensors, and meteorological observation data is formed through data integration; According to the meteorological observation data, spatial stratification is performed, real-time collection is performed through the bottom layer, data is temporarily stored by the middle layer, and communication scheduling is performed by the top layer to obtain stratified processing data; Based on the hierarchical processing data, a validity threshold range is set, abnormal marking is performed through numerical comparison, and data exceeding the threshold range is eliminated to obtain a valid data set; A mesh network topology is constructed according to the valid data set, multiple transmission paths are calculated by a routing algorithm, and an optimal transmission path is selected according to the network load to obtain a transmission data stream; The transmission data stream is used to segment data packets, data integrity is ensured by checksum calculation, and data is encapsulated according to the transmission protocol to obtain a data packet to be verified; The data validity is verified based on the data packet to be verified, the data temporality is ensured by timestamp verification, and the data consistency is verified based on spatial correlation to obtain the initial meteorological data set.

3. The meteorological data processing method based on the edge protection gateway algorithm according to claim 1 is characterized in that: The protection mechanism is established according to the initial meteorological data set, encrypted storage is performed through dynamic key management, a baseline model is constructed using a deep learning algorithm, and a hash value is generated in combination with blockchain technology to obtain encrypted meteorological data, including: Extract identity features based on the initial meteorological data set, classify access requests through zero-trust authentication, assign authentication tokens based on access levels, and obtain authentication data packets; Generate a dynamic key using the authentication data packet, distribute the key using an asymmetric encryption algorithm, update the key according to a time window, and obtain an encryption key sequence; Encrypt the data in blocks based on the encryption key sequence, establish a secure communication tunnel through a channel encryption protocol, perform hierarchical data protection according to the encryption strategy, and obtain an encrypted data stream; Establishing a behavior feature sequence according to the encrypted data stream, extracting the data transmission mode through a deep learning algorithm, building a baseline determination rule according to normal behavior features, and obtaining a baseline model; The baseline model is used to monitor the data flow in real time, the abnormality is calculated through feature matching, and a security response is performed according to the threat level to obtain security status data; Blocks are generated based on the security status data and encrypted data streams, data integrity is verified through a consensus mechanism, and a data traceability mechanism is established based on timestamps and hash links to obtain encrypted meteorological data.

4. The meteorological data processing method based on the edge protection gateway algorithm according to claim 1 is characterized in that: The method of establishing a standardized processing model based on the encrypted meteorological data, filtering noise by sliding window and wavelet transform, and using an adaptive algorithm to perform anomaly identification and data repair to obtain a preprocessed data set includes: Establish a unified data format specification based on the encrypted meteorological data, unify the format of multi-source data through unit conversion, align the data according to time tags, and obtain standard format data; Using the standard format data to construct a sliding window sequence, segmenting the data by adaptively adjusting the window size, calculating the optimal window parameters according to the data change rate, and obtaining windowed data; Selecting a wavelet basis function based on the windowed data, extracting data features through multi-scale decomposition, and performing denoising based on a threshold value determined according to energy distribution to obtain filtered data; Calculating the spatiotemporal correlation index based on the filtered data, constructing anomaly determination criteria through probability density estimation, identifying abnormal samples based on multidimensional feature space, and obtaining data to be repaired; Classify the missing data to be repaired according to their missing types, fill short-term missing data with a time series interpolation algorithm, and reconstruct long-term missing data based on spatial correlation to obtain repaired data; The repaired data is used to perform quality assessment, data availability is calculated using integrity indicators, and data is screened based on accuracy standards to obtain a preprocessed data set.

5. The meteorological data processing method based on the edge protection gateway algorithm according to claim 1 is characterized in that: The weight fusion model is constructed based on the preprocessed data set, spatiotemporal features are extracted by tensor decomposition, and feature learning and screening are performed using a deep learning network to obtain fused feature data, including: Calculate the reliability of the data source according to the preprocessed data set, determine the initial weight by evaluating the timeliness of the data, and dynamically update the weight according to the quality index to obtain the weight coefficient; The multi-source data are weightedly combined by using the weight coefficients, time differences are eliminated by data alignment, data association is established according to spatial mapping, and a fused data set is obtained; Based on the fused data set, a three-dimensional tensor structure is constructed, time series patterns are extracted by time dimension decomposition, and regional features are obtained by space dimension decomposition to obtain feature tensors; Performing dimensionality reduction processing on the feature tensor, extracting key features through principal component analysis, determining feature dimensions according to variance contribution rates, and obtaining dimensionality reduction features; Establishing a feature correlation matrix based on the dimensionality reduction features, evaluating feature importance through mutual information calculation, and sorting features according to importance thresholds to obtain candidate features; The candidate features are used to construct feature combinations, feature stability is evaluated through cross-validation, and feature screening is performed based on the prediction effect to obtain fused feature data.

6. The meteorological data processing method based on the edge protection gateway algorithm according to claim 1 is characterized in that: The method of establishing a federated learning framework based on the fused feature data, constructing a prediction model through an attention mechanism and a gated recurrent unit, and using an integrated learning method to perform prediction to obtain a prediction result includes: Slice the data according to the fused feature data, distribute the data among nodes through encrypted transmission, determine the training rules according to the privacy protection strategy, and obtain a distributed data set; Calculate local gradients using the distributed data set, aggregate parameters through gradient encryption, iteratively optimize the model according to the global update strategy, and obtain a local model; An attention weight matrix is ​​constructed based on the local model, feature importance is calculated by temporal correlation, and feature weighting is performed according to the attention score to obtain attention features; Establish a gated state sequence for the feature of interest, filter redundant information through a forget gate, adjust the feature combination according to an input gate and an update gate, and obtain a state vector; Generate multiple base learners according to the state vector, evaluate the prediction performance through cross-validation, select the optimal combination according to the model difference, and obtain the integrated model; The integrated model is used to perform multiple rounds of predictions on new data, the prediction results are integrated through a voting mechanism, and the results are screened according to the confidence level to obtain the prediction results.

7. The meteorological data processing method based on the edge protection gateway algorithm according to claim 1 is characterized in that: The verification mechanism is established based on the prediction results, decision rules are constructed through knowledge graphs, decision strategies are optimized using adaptive algorithms, and plans are generated in combination with early warning mechanisms to obtain decision suggestions, including: Based on the prediction results, historical data comparison is performed, the prediction accuracy is calculated through error analysis, and rationality is tested according to physical constraint rules to obtain verification indicators; Construct knowledge entity relationships based on the verification indicators, establish semantic connections through entity attribute extraction, form reasoning rules based on historical cases, and obtain a knowledge graph; The knowledge graph is used to extract rules, a decision chain is established through causal analysis, and rule weights are set based on expert experience to obtain a decision rule library; Dynamically update the decision rule base, evaluate the rule effect through feedback data, adjust the optimization direction according to the target deviation, and obtain the optimization strategy; Calculate the warning level threshold based on the optimization strategy, determine the risk level through disaster impact assessment, divide the warning area according to the time and space scale, and obtain the warning plan; According to the early warning scheme, multiple sets of candidate decisions are generated, the schemes are evaluated through cost-benefit analysis, the best scheme is selected based on feasibility indicators, and decision recommendations are obtained.

8. A meteorological data processing device based on an edge protection gateway algorithm, used to implement the meteorological data processing method based on an edge protection gateway algorithm as described in any one of claims 1 to 7, characterized in that: The meteorological data processing device based on the edge protection gateway algorithm includes: The acquisition module is used to collect meteorological observation data in a distributed manner using edge computing nodes, transmit the data through the mesh network, and filter it through the verification module to obtain the initial meteorological data set; Establish a module for establishing a protection mechanism based on the initial meteorological data set, encrypting and storing the data through dynamic key management, building a baseline model using a deep learning algorithm, and generating a hash value in combination with blockchain technology to obtain encrypted meteorological data; A filtering module is used to establish a standardized processing model based on the encrypted meteorological data, perform noise filtering through a sliding window and wavelet transform, and perform anomaly identification and data repair using an adaptive algorithm to obtain a preprocessed data set; A screening module is used to construct a weight fusion model based on the preprocessed data set, extract spatiotemporal features through tensor decomposition, and use a deep learning network to perform feature learning and screening to obtain fused feature data; A prediction module is used to establish a federated learning framework based on the fused feature data, construct a prediction model through an attention mechanism and a gated recurrent unit, and perform prediction using an integrated learning method to obtain a prediction result; The generation module is used to establish a verification mechanism based on the prediction results, build decision rules through the knowledge graph, optimize the decision strategy using an adaptive algorithm, generate a plan in combination with the early warning mechanism, and obtain decision recommendations.

Images