Geographic information space-time big data real-time aggregation and cleaning method based on edge computing

Abstract

The invention relates to the technical field of geographic information big data processing, and discloses a geographic information space-time big data real-time aggregation and cleaning method based on edge computing, and the method comprises the following specific steps: S1, deploying edge computing nodes in a geographic information collection terminal coverage area, constructing an edge computing network, an acquisition terminal pushes original geographic information spatio-temporal data including geographic coordinates (x, y, z), an acquisition timestamp t and attribute information to an edge node through a protocol, and the edge node establishes a data buffer queue and preliminarily classifies and stores the data according to a spatio-temporal dimension. According to the geographic information space-time big data real-time aggregation and cleaning method based on edge computing, efficient and low-delay transmission of geographic information space-time big data is achieved through edge computing node deployment and a multi-algorithm fusion aggregation cleaning strategy, and the bandwidth and delay problems of a traditional centralized architecture are solved; multi-dimensional collaborative cleaning (geographic constraint, statistics and machine learning) is combined with an intelligent algorithm, and abnormal data are accurately recognized and processed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of geographic information big data processing, in particular to a geographic information spatio-temporal big data real-time aggregation and cleaning method based on edge computing. BACKGROUND

[0002] Geographic information spatio-temporal big data contains rich geographic spatial position, time series and associated attribute information, and is widely used in the fields of smart city construction, environmental monitoring, traffic planning, etc.; however, the existing geographic information spatio-temporal big data processing method has significant defects:

[0003] 1. When processing geographic information spatio-temporal big data by using traditional centralized cloud computing architecture, massive data (such as geographic coordinates collected by multiple sources of sensors, time series monitoring data, etc.) needs to be transmitted to the cloud, which faces the problems of network bandwidth limitation and transmission delay; especially in low-latency demand scenarios such as traffic flow monitoring and real-time geological disaster warning, the time-consuming data transmission easily leads to decision lag, which cannot meet the real-time requirement;

[0004] 2. The geographic information data has the characteristics of multi-source heterogeneity (satellite remote sensing images, ground station collected data, crowd-sourced geographic information, etc. with large differences in format and precision), and the existing aggregation algorithm does not fully consider the spatio-temporal correlation, which easily causes data redundancy or loss of key spatio-temporal features; the cleaning link has low identification accuracy for spatio-temporal abnormal data (such as incorrect geographic coordinates caused by GPS drift and abnormal time series values caused by sensor failure), relies on manual rules or simple statistical models, and is difficult to adapt to complex and variable geographic scenes, which affects the accuracy of subsequent geographic information analysis and application;

[0005] 3. Although the current geographic information big data processing attempts to introduce edge computing, it only simply deploys edge nodes for data caching, and does not deeply explore the potential of edge computing in real-time processing of geographic information spatio-temporal big data in the "end-edge-cloud" collaboration; the edge node lacks intelligent aggregation and cleaning logic for the spatio-temporal characteristics of geographic information, and cannot efficiently preprocess at the data source, which leads to the cloud still needing to process a large amount of "dirty data", increasing the cloud computing power burden and overall processing cost;

[0006] Therefore, a geographic information spatio-temporal big data real-time aggregation and cleaning method based on edge computing is proposed to solve the above-mentioned problems. SUMMARY

[0007] (I) Technical problems solved

[0008] In view of the deficiencies of the prior art, the application provides a geographic information space-time big data real-time aggregation and cleaning method based on edge computing, which has the advantages of efficient and low-delay data transmission, solves the transmission delay problem caused by network bandwidth limitation under the traditional centralized cloud computing architecture, meets the demand of high real-time requirement scenes such as traffic flow monitoring and geological disaster warning, and avoids the problem of decision lag caused by time-consuming data transmission.

[0009] (II) Technical scheme

[0010] In order to achieve the above-mentioned purpose of efficient and low-delay data transmission, the application provides the following technical scheme: a geographic information space-time big data real-time aggregation and cleaning method based on edge computing, comprising the following specific steps:

[0011] S1: deploying edge computing nodes in the coverage area of a geographic information collection terminal, constructing an edge computing network, and pushing the original geographic information space-time data containing geographic coordinates (x, y, z), collection timestamp t and attribute information to the edge node through a protocol by the collection terminal, and the edge node establishes a data buffer queue and stores it in a preliminary classification according to the space-time dimension;

[0012] S2: the edge node extracts the space-time features, constructs a space-time cube model, takes the geographic grid and time window as the aggregation unit, respectively constructs the spatial relationship network through Delaunay triangulation and combines the spatial interpolation algorithm for spatial aggregation, adopts the sliding window LSTM model for time aggregation, and constructs the space-time association rule library for space-time joint aggregation;

[0013] S3: the edge node respectively performs the following operations on the aggregated data: geographic constraint verification according to the geographic knowledge graph and through the road network matching algorithm, statistical feature cleaning combining the historical statistical characteristics by calculating the attribute statistics, machine learning cleaning by training the space-time anomaly detection model and inputting the space-time feature vector;

[0014] S4: the cleaned data is evaluated by the edge node, the high-quality aggregated data is uploaded to the cloud, the cloud feeds back the rule library update and model parameter optimization suggestion based on global analysis to the edge node, and the closed-loop iteration is realized.

[0015] Preferably, the edge computing nodes deployed in the S1 adopt a distributed cluster architecture, each node realizes dynamic load balancing through an SDN network, and the coverage radius of the edge computing network is dynamically adjusted according to the signal strength of the collection terminal and the data transmission delay threshold.

[0016] Preferably, in the S1, the original geographic information space-time data pushing adopts the QUIC protocol based on UDP, a multi-path transmission channel is established between the collection terminal and the edge node, the data transmission rate is dynamically adjusted through the BBR congestion control algorithm, and the data transmission delay is reduced. Preferably, in the S1, the original geographic information space-time data pushing adopts the QUIC protocol based on UDP, a multi-path transmission channel is established between the collection terminal and the edge node, the data transmission rate is dynamically adjusted through the BBR congestion control algorithm, and the data transmission delay is reduced.

[0017] Preferably, the spatio-temporal cube model constructed in S2 adopts a quadtree index structure, the geographical grid granularity is adaptively adjusted based on the complexity of regional geographical features, and the time window width is dynamically stretched according to the frequency of data time sequence changes.

[0018] Preferably, in S2, a spatio-temporal association rule library is constructed by spatio-temporal joint aggregation, a geographical information spatio-temporal frequent item set is mined based on an Apriori algorithm, rules are clustered by combining a cosine similarity algorithm, the rule library is divided into sub-libraries according to geographical regions and is deployed on corresponding edge nodes, and local rapid calling of rules is realized.

[0019] Preferably, in S3, a multi-level spatial index mechanism is adopted for geographical constraint verification, a road network matching algorithm is combined with Monte Carlo path sampling to optimize matching accuracy, a quantile regression model of a sliding time window is adopted for statistical feature cleaning, and an attention mechanism is introduced into a spatio-temporal anomaly detection model of machine learning cleaning to enhance the time sequence feature extraction capability.

[0020] Preferably, in S3, during the statistical feature cleaning process, a standardized statistical template library is established for different types of attribute information, the template library contains attribute statistical feature threshold standards in the fields of meteorology, traffic, geology and the like, and corresponding templates are called for abnormal value determination through attribute label matching.

[0021] Preferably, in S4, the quality evaluation adopts an analytic hierarchy process to construct an evaluation index system, including three-level indexes of data integrity, spatio-temporal consistency and semantic accuracy; the rule library update fed back by the cloud adopts a federated learning framework, and the edge node trains rule parameters based on local data and uploads gradient information.

[0022] Preferably, in S4, the data transmission between the edge node and the cloud adopts a lightweight geographical information coding format OGCAPI-Features, and a Zstandard compression algorithm for spatio-temporal data is adopted for data compression, so as to reduce the data transmission amount and reduce the storage pressure of the cloud.

[0023] Preferably, the data processing flow of S1-S4 is realized by adopting a micro-service architecture, asynchronous communication is performed between processing modules through a message queue, and resource scheduling of the edge node follows an energy consumption optimization strategy under QoS constraints.

[0024] (Three) beneficial effects

[0025] Compared with the prior art, the present application provides a geographical information spatio-temporal big data real-time aggregation and cleaning method based on edge computing, and has the following beneficial effects:

[0026] 1. The method for real-time aggregation and cleaning of geographic information spatiotemporal big data based on edge computing, through the deployment of edge computing nodes, the aggregation and cleaning strategy of multi-algorithm fusion, realizes the efficient and low-delay transmission of geographic information spatiotemporal big data, solves the bandwidth and delay problem of traditional centralized architecture; multi-dimensional collaborative cleaning (geographical constraints, statistics, machine learning) combined with intelligent algorithm, accurately identifies and processes abnormal data, improves data quality, meets the real-time and accuracy requirements of smart city, environmental monitoring and other scenarios, avoids decision lag.

[0027] 2. The method for real-time aggregation and cleaning of geographic information spatiotemporal big data based on edge computing, by constructing an "end-edge-cloud" collaborative iteration mechanism, using federated learning, genetic algorithm and other methods to realize dynamic optimization of rule base and model parameters; integrate encryption, blockchain and other technologies to ensure data transmission and storage security and processing flow traceability and tamper resistance, tap the potential of edge computing, reduce the burden of cloud computing power, and enhance the overall performance and credibility of the system. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 The flowchart of the method for real-time aggregation and cleaning of geographic information spatiotemporal big data based on edge computing of the present application is shown.

[0029] Figure 2 The flowchart of the spatiotemporal data aggregation and cleaning process of the present application is shown.

[0030] Figure 3 The logic diagram of the dynamic resource allocation strategy of the present application is shown.

[0031] Figure 4 The schematic diagram of the comparison effect before and after data cleaning of the present application is shown. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0033] Embodiment one

[0034] Please refer to Figures 1-4 A method for real-time aggregation and cleaning of geographic information spatiotemporal big data based on edge computing, comprising the following specific steps:

[0035] (S1) S1: Edge computing node deployment and data access

[0036] Node deployment logic: In the coverage area of geographic information collection terminals (such as vehicle-mounted GPS, unmanned aerial vehicles, and ground monitoring stations), plan the layout of edge computing nodes; use a distributed cluster architecture and dynamically manage nodes using SDN (Software Defined Network) technology; adjust the node coverage radius in real time based on the signal strength (such as RSSI index) and data transmission delay threshold (configured on demand, such as ≤50ms for intelligent transportation scenarios) of the collection terminal:

[0037] Weak signal / terminal dense areas (such as urban high-rise areas): reduce coverage radius (such as deploying 2 nodes within 1km), improve access capacity;

[0038] Strong signal / terminal sparse areas (such as suburban areas): expand coverage radius (such as deploying 1 node within 5km), reduce hardware cost;

[0039] Data transmission and storage: Collection terminals establish multi-path transmission channels through QUIC protocol based on UDP, and push raw data containing geographic coordinates (x, y, z), time stamp t, and attributes (such as temperature, traffic flow) to edge nodes. During transmission, BBR congestion control algorithm dynamically adjusts the rate:

[0040] Network congestion (delay > threshold): reduce transmission rate, prioritize data integrity;

[0041] Network idle (delay < threshold): increase transmission rate, speed up data transmission;

[0042] After the edge node receives the data, it constructs a spatio-temporal buffer queue (classified by latitude and longitude grid + time window), which is preprocessed for subsequent aggregation and cleaning.

[0043] (2) S2: Spatio-temporal data aggregation processing

[0044] Construction of spatio-temporal cube model: The edge node extracts the spatio-temporal features (spatial position, time trend, attribute association) of the data and constructs a spatio-temporal cube model with a quadtree index;

[0045] Dynamic adaptation of model parameters:

[0046] Geographical grid granularity: adjust according to regional complexity (refine to 100m x 100m in mountainous / city areas, coarsen to 1km x 1km in plains);

[0047] Time window width: adjust according to data fluctuation frequency (set traffic flow to 10s window, set meteorological data to 1h window);

[0048] Spatial aggregation (Delaunay triangulation + spatial interpolation)

[0049] With geographical grid as the unit, Delaunay triangulation connects discrete data points to form continuous spatial topology; combined with Kriging interpolation algorithm, fill in the blank area of triangulation (such as mountain terrain monitoring data), remove redundant spatial points (keep key features such as terrain mutation, flow extreme value), compress spatial dimension data volume;

[0050] Temporal aggregation (sliding window LSTM model)

[0051] With time window as the unit, LSTM model learns time series rules (such as morning / evening peak pattern of traffic flow); through sliding window traversal of time series data, similar fragments are aggregated (such as continuous smooth flow merging), features are extracted (peak start and end time, fluctuation amplitude), and time dimension redundancy is compressed;

[0052] Spacetime joint aggregation (Apriori+cosine similarity clustering)

[0053] Based on Apriori algorithm to mine spatiotemporal frequent itemsets (such as "summer afternoon + precipitation + traffic congestion" association), through cosine similarity clustering rules, divide the rule base according to geographical area (administrative area, landform area), and deploy to the corresponding edge node; for example, city A node stores local "precipitation-road waterlogging-control" rules, which can be directly called when processing data, without the need for cross-region transmission.

[0054] (Three) S3: multi-dimensional data cleaning

[0055] Geographical constraint verification (multi-level spatial index + Monte Carlo sampling)

[0056] Based on geographical knowledge graph (including road topology, geographical boundary), multi-level spatial index (R-tree + grid index) is constructed; for abnormal coordinates (such as vehicle position deviating from the road), through Monte Carlo path sampling, reasonable driving path is simulated, and coordinates are corrected to road network, to ensure compliance with geographical constraints (reference Figure 4 Coordinate correction process);

[0057] Statistical feature cleaning (quantile regression + standardized template library)

[0058] Calculate the statistics (mean, variance, quantile) of data attributes, and match with historical statistical template library (thresholds in meteorology, traffic, geology, etc.):

[0059] Temperature data: marked as abnormal if it exceeds the 95% quantile of the same period in history;

[0060] Traffic flow data: marked as abnormal if it is lower than the 5% quantile of the same period in history;

[0061] Repair abnormal values through spatiotemporal interpolation (fuse surrounding station data), or directly exclude invalid data (reference Figure 4Statistical cleaning process)

[0062] Machine learning cleaning (attention mechanism + spatio-temporal anomaly detection model)

[0063] Train spatio-temporal anomaly detection model, introduce attention mechanism to focus on key time sequence segments (such as data mutations caused by geological disasters); input spatio-temporal feature vector (coordinate transformation, time difference), identify complex anomaly patterns (such as earthquake precursor data anomalies), output cleaning suggestions (repair / delete), execute after threshold determination (reference Figure 4 Machine learning cleaning process).

[0064] (Four) S4: Quality Assessment and End-Cloud Collaborative Iteration

[0065] Quality Assessment (Analytic Hierarchy Process)

[0066] Construct three-level evaluation indicators:

[0067] Data integrity: check field missing rate (such as completeness of coordinates, timestamps);

[0068] Spatio-temporal consistency: check the matching degree of coordinates and timestamps (such as whether the vehicle trajectory is continuous);

[0069] Semantic accuracy: verify attribute information compliance (such as whether the temperature value is within a reasonable range);

[0070] Calculate weights through the analytic hierarchy process, and upload high-quality data to the cloud;

[0071] End-Cloud Collaborative Iteration

[0072] Edge nodes upload high-quality data to the cloud through OGCAPI-Features lightweight encoding + Zstandard compression; the cloud is based on global data (cross-regional traffic / weather correlation), and feedback is given:

[0073] Rule base update: add regional rules such as "typhoon path-coastal control";

[0074] Model parameter optimization: adjust LSTM time window, attention mechanism weight;

[0075] Edge nodes train rule parameters based on local data through federated learning framework (only upload gradient, protect privacy), realize end-cloud closed-loop iteration (reference Figure 3 Resource scheduling logic).

[0076] In this embodiment, the edge computing-based geographic information spatio-temporal big data real-time aggregation and cleaning method deploys edge computing nodes in the data collection terminal coverage area, constructs a distributed cluster network, uses the QUIC protocol and multi-path transmission to reduce data transmission delay, and realizes fast data processing from the source; using spatio-temporal cube model, Delaunay triangulation, LSTM model and other technologies, combined with the spatio-temporal association rule base, accurate spatial, temporal and joint aggregation of data is realized, effectively solving the problems of data redundancy and feature loss; through multi-dimensional cleaning strategies such as geographic constraint verification, statistical feature cleaning and machine learning cleaning, according to the geographic knowledge graph and standardized template library, combined with intelligent algorithms, the accuracy of abnormal data identification is greatly improved; finally, with the help of the closed-loop iteration mechanism of edge node quality evaluation and cloud feedback, the dynamic optimization of rule base and model parameters is realized, and lightweight coding and compression algorithms are used to reduce the cloud storage pressure.

[0077] The overall scheme deeply integrates the needs of edge computing and geographic information big data processing, significantly improves the data processing efficiency and quality, reduces the cloud computing cost, and effectively meets the strict requirements of real-time and accuracy of geographic information data in the fields of smart city, environmental monitoring, etc.

[0078] The original geographic information spatio-temporal data uses an end-to-end encryption mechanism based on the SM4 algorithm in the transmission and storage process, and the edge node generates an encryption key management table when establishing a spatio-temporal index for encrypted data, realizing data privacy protection and security traceability; the model parameter optimization suggestion fed back by the cloud uses an incremental learning framework, and the edge node updates the spatio-temporal anomaly detection model based on new local data through a gradient accumulation strategy, and uses knowledge distillation technology to compress the model parameter amount during the update process, adapting to the power limit of the edge node.

[0079] In this embodiment, the SM4 algorithm is used to encrypt the original geographic information spatio-temporal data and generate an encryption key management table, which can ensure the security of data transmission and storage while realizing privacy protection and security traceability;

[0080] The model parameter optimization fed back by the cloud uses an incremental learning framework, a gradient accumulation strategy, and knowledge distillation technology, so that the edge node can update the spatio-temporal anomaly detection model based on new data and compress the parameter amount under the power limit.

[0081] The edge computing node supports cross-platform deployment of heterogeneous hardware architecture and multiple operating systems, encapsulates data processing modules through containerization technology (Docker), realizes protocol adaptation and driver compatibility of different manufacturers' collection terminals (such as Beidou / GPS hybrid positioning equipment), and embeds a real-time monitoring module in the micro-service architecture. Through Prometheus, the CPU / memory / network traffic indicators of the edge node are collected, and Grafana is combined to build a space-time data processing heat map. When the data processing delay exceeds the threshold, an alarm is automatically triggered and the elastic scaling mechanism is started. The machine learning cleaning link introduces a transfer learning strategy. For unannotated new geographic scenes (such as sudden geological disaster areas), a pre-trained cross-domain spatio-temporal feature extractor is used to initialize model parameters, reducing the dependence on large-scale labeled data. In the collaborative feedback between the edge node and the cloud, a blockchain storage mechanism is added to hash and chain the rule base update records and model parameter iteration history, ensuring the traceability and tamper resistance of the geographic information data processing flow.

[0082] In this embodiment, the containerization technology is used to realize the cross-platform deployment of the edge computing node and the compatibility of multi-manufacturer devices, solve the problem of hardware heterogeneity and protocol adaptation, and improve the flexibility of system deployment.

[0083] The real-time monitoring and elastic scaling mechanism is embedded to dynamically perceive the load state of the edge node and automatically optimize resource allocation, ensuring the stability of data processing delay.

[0084] The transfer learning strategy is introduced to optimize the machine learning cleaning link, and the pre-trained model is used to reduce the dependence of new geographic scenes on large-scale labeled data and enhance the model generalization ability.

[0085] The blockchain storage mechanism ensures the traceability and tamper resistance of the rule and model iteration records in the edge-cloud collaborative process, and strengthens the credibility of the geographic information data processing flow.

[0086] Embodiment Two

[0087] 1. Dynamic transmission routing based on reinforcement learning includes the following:

[0088] In the original geographic information space-time data transmission stage, a reinforcement learning model based on the QUIC protocol is constructed. The node state (bandwidth, load, delay, etc.) in the edge computing network is taken as the environment state S, the data transmission path selection is taken as the action A, and the transmission delay and packet loss rate are taken as the reward function R.

[0089] State space S={s1, s2,..., s n}, where s i represents the state vector of the i-th node, including bandwidth bw i , load load i, delay 11111delay i s i = [bw i , load i , delay i ].

[0090] Action space A = {a1, a2,..., a m} represents the next hop nodes that can be selected from the current node.

[0091] 2. Adaptive partitioning of spatiotemporal cubes

[0092] For the construction of the spatiotemporal cube model, an adaptive partitioning algorithm based on density clustering and entropy calculation is designed.

[0093] The DBSCAN density clustering algorithm is used to perform spatial clustering on geographic data to determine the density distribution of geographic regions.

[0094] The time series change complexity of each region data is calculated by combining information entropy. For time series data X = {x1, x2,..., x T}, the information entropy H(X) is calculated as follows:

[0095]

[0096] Where p(x i ) is the probability of data point x i .

[0097] The density and entropy value are integrated to dynamically adjust the geographic grid granularity g and the time window width w.

[0098] The grid granularity adjustment formula is:

[0099]

[0100] Where,

[0101] g0 is the initial grid granularity, density avg and H avg are the global average density and average information entropy, density i and H i are the density and information entropy of the current region.

[0102] The time window width adjustment formula is similar, and the spatiotemporal cube model can more accurately respond to the complexity of different regional geographic features and the frequency of data time series changes in a general way.

[0103] 3. Dynamic updating of spatiotemporal association rules includes the following:

[0104] In the maintenance of spatio-temporal association rules library, a dynamic updating algorithm based on time decay factor and confidence drift detection is introduced.

[0105] A time decay factor γ∈[0, 1] is set for each spatio-temporal association rule r, and the weight w of the rule gradually decreases over time, with the calculation formula being: r

[0106] w r (t)=w r (t-1)·γ;

[0107] Where t represents the time step.

[0108] The confidence of the rule is monitored in real time, and the confidence conf(r) is calculated as:

[0109]

[0110] When the confidence drift exceeds the set threshold δ, the rule is updated or eliminated by using an online learning algorithm combined with newly collected geographic information data, ensuring that the rule library always remains effective and accurate.

[0111] 4. Multi-dimensional cleaning fusion decision includes the following contents:

[0112] In the data cleaning link, a multi-dimensional cleaning fusion decision algorithm is designed; the results of geographic constraint verification, statistical feature cleaning, and machine learning cleaning are taken as inputs to construct a decision model based on evidence theory.

[0113] Different trust degrees m1, m2, and m3 are assigned to the results of each cleaning dimension, corresponding to geographic constraint verification, statistical feature cleaning, and machine learning cleaning, respectively.

[0114] The comprehensive trust degree m is calculated through evidence synthesis rules:

[0115]

[0116] Where k is the conflict coefficient, and A represents the decision result of whether the data is abnormal data.

[0117] In this way, the data is comprehensively judged to be abnormal data, avoiding the limitations of single cleaning dimension and improving the accuracy and reliability of abnormal data identification.

[0118] 5. Edge-cloud collaborative evolution includes the following contents:

[0119] In the collaborative feedback process between edge nodes and the cloud, a collaborative evolution algorithm based on genetic algorithm is adopted.

[0120] ​​The local data features of the edge node, the model parameters and the global knowledge of the cloud are taken as the individual genes G = {g1, g2,..., gN} of the genetic algorithm. n};

[0121] The data processing accuracy P, the efficiency E and the energy consumption C are taken as the fitness function F(G):

[0122] F(G) = a P + b E - g C.

[0123] Wherein, a, b, g are weight coefficients.

[0124] Through selection, crossover, mutation and other operations, the rule base and model parameters of the edge node are continuously optimized, and at the same time more valuable local information is provided for the cloud, realizing the collaborative evolution of edge-cloud, and improving the performance of the whole system.

[0125] In summary, the real-time aggregation and cleaning method of geographic information spatio-temporal big data based on edge computing realizes the efficient and low-delay transmission of geographic information spatio-temporal big data through the deployment of edge computing nodes and the aggregation and cleaning strategy of multi-algorithm fusion, solves the bandwidth and delay problem of traditional centralized architecture; multi-dimensional collaborative cleaning (geographical constraints, statistics, machine learning) combined with intelligent algorithms accurately identifies and processes abnormal data, improves data quality, meets the demand for real-time and accuracy in smart city, environmental monitoring and other scenarios, and avoids decision lag.

[0126] Moreover, by constructing an "end-edge-cloud" collaborative iteration mechanism, the rule base and model parameters are dynamically optimized by means of federated learning and genetic algorithm; by integrating encryption, blockchain and other technologies, the data transmission and storage security is guaranteed, and the processing flow is traceable and tamper-proof, the potential of edge computing is tapped, the cloud computing burden is reduced, and the overall performance and credibility of the system are enhanced.

[0127] It should be noted that in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or equipment including the element.

[0128] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary of the principles and application of the present application. Numerous modifications and adaptions can be effected without departing from the spirit and scope of the present application, which is not limited to the exact construction and arrangement described. It is intended, therefore, to cover all modifications and adaptions that fall within the scope of the claims and their equivalents.

Claims

1.A method for real-time aggregation and cleaning of geospatial big data based on edge computing, characterized in that: Comprise the following specific steps: S1: Deploy edge computing nodes in the coverage area of the geographic information collection terminal, build an edge computing network, and push the original geographic information spatio-temporal data containing geographic coordinates (x, y, z), collection timestamp t, and attribute information to the edge node through the protocol. The edge node establishes a data buffer queue and stores it in the spatio-temporal dimension; S2: The edge node extracts the data spatio-temporal features, constructs a spatio-temporal cube model, and aggregates the data in the geographical grid and time window as the aggregation unit. The spatial relationship network is constructed through Delaunay triangulation, combined with spatial interpolation algorithm for spatial aggregation, sliding window LSTM model for time aggregation, and spatio-temporal association rule library for spatio-temporal joint aggregation; S3: The edge node performs geographical constraint verification based on the geographical knowledge graph and through the road network matching algorithm, calculates the attribute statistics and combines the historical statistical characteristics for statistical feature cleaning, trains the spatio-temporal anomaly detection model and inputs the spatio-temporal feature vector for machine learning cleaning; S4: The cleaned data is evaluated by the edge node, and the high-quality aggregated data is uploaded to the cloud. The cloud feeds back rule library updates and model parameter optimization suggestions based on global analysis to the edge node, realizing a closed-loop iteration. 2.The method of claim 1, wherein the method is characterized in that: The edge computing nodes deployed in S1 adopt a distributed cluster architecture, and each node realizes dynamic load balancing through an SDN network. The coverage radius of the edge computing network is dynamically adjusted according to the signal strength of the collection terminal and the data transmission delay threshold. 3.The method of claim 1, wherein the method further comprises: In S1, the original geographic information spatio-temporal data pushing adopts the QUIC protocol based on UDP, and a multi-path transmission channel is established between the collection terminal and the edge node. The data transmission rate is dynamically adjusted through the BBR congestion control algorithm to reduce the data transmission delay. 4.The method of claim 1, wherein the method further comprises: The spatio-temporal cube model constructed in S2 adopts a quadtree index structure, and the geographical grid granularity is adaptively adjusted based on the complexity of regional geographical features. The time window width is dynamically stretched according to the data time sequence change frequency. 5.The method of claim 1, wherein the method further comprises: In S2, the spatio-temporal association rule library constructed by spatio-temporal joint aggregation mines the geographic information spatio-temporal frequent item set based on the Apriori algorithm, clusters the rules using the cosine similarity algorithm, and divides the rule library into sub-libraries according to geographical regions and deploys them on corresponding edge nodes to realize localized and rapid rule invocation. 6.The method of claim 1, wherein the method further comprises: In S3, the geographical constraint verification adopts a multi-level spatial index mechanism, and the road network matching algorithm optimizes the matching accuracy by combining Monte Carlo path sampling; The statistical feature cleaning uses a quantile regression model with a sliding time window, and the spatio-temporal anomaly detection model for machine learning cleaning introduces an attention mechanism to enhance the time series feature extraction capability. 7.The method of claim 1, wherein the method further comprises: In S3, during the statistical feature cleaning process, a standardized statistical template library is established for different types of attribute information. The template library contains attribute statistical feature threshold standards in meteorology, transportation, geology, and other fields. The corresponding template is called for abnormal value judgment through attribute label matching. 8.The method of claim 1, wherein the method further comprises: In the S4, the quality evaluation adopts the analytic hierarchy process to construct the evaluation index system, including three-level indexes of data integrity, spatio-temporal consistency and semantic accuracy; the rule base update of cloud feedback adopts the federated learning framework, and the edge node trains the rule parameters based on local data and uploads gradient information. 9.The method of claim 1, wherein the method further comprises: In the S4, the data transmission between the edge node and the cloud adopts the lightweight geographic information coding format OGCAPI-Features, and the data compression adopts the Zstandard compression algorithm for spatio-temporal data, thereby reducing the data transmission amount and the storage pressure of the cloud. 10.The method of claim 1, wherein the method is based on edge computing. The data processing flow of the S1-S4 adopts the micro-service architecture to realize asynchronous communication of each processing module through a message queue, and the edge node resource scheduling follows the energy consumption optimization strategy under the QoS constraint.

Images