Observation station end data preprocessing system based on edge computing

Abstract

This invention relates to the field of edge computing technology, specifically to a data preprocessing system for observation stations based on edge computing. The system includes: acquiring multi-dimensional sensor data, mobile node coordinates, mobile node velocity, and mobile node signal strength transmitted back from the observation station collaborative system; updating spatial topology relationships based on the mobile node coordinates, velocity, and signal strength to obtain dynamic graph connection weights. In this invention, physical consistency calculation is used to evaluate the physical residual values ​​generated by the spatiotemporal continuous observation map. Anomalies are identified based on these physical residual values, and data isolation and discarding operations are performed. This overcomes the limitations of traditional anomaly identification relying solely on statistical thresholds, endowing the edge side with the ability to distinguish between underlying physical extreme events and sensor hardware failures, avoiding the erroneous deletion of high-value extreme disaster data, and improving the reliability of edge-end data pre-filtering.

Description

Technical Field

[0001] This invention relates to the field of edge computing technology, and in particular to a data preprocessing system for observation stations based on edge computing. Background Technology

[0002] Edge computing is a distributed computing architecture that physically moves computing resources, data storage, and application processing power from the cloud at the network center to the network edge, closer to the data source or end user.

[0003] Existing edge distributed computing architectures primarily rely on fixed, static network topology configurations to receive data sources aggregated from downstream nodes. Since edge resources only possess basic data relay and preset numerical filtering capabilities, when the terminal network includes highly mobile devices, the fixed communication routes cannot adapt to drastic changes in node spatial locations. This easily leads to data acquisition gaps in communication obstruction areas. Furthermore, edge data cleaning often relies solely on the mean or variance of numerical fluctuations to remove anomalies. The system lacks deep-level logic for discerning objective physical facts when facing sudden numerical changes. For example, in hydrogeological environmental monitoring scenarios, once a severe rainstorm triggers a flood peak, the water level and flow velocity sensor values ​​will instantly far exceed the statistical average standards of daily monitoring. Edge gateways, following conventional fixed threshold filtering principles, directly classify abrupt changes containing real disaster evolution information as sudden outlier noise and discard them. This not only results in the permanent loss of core, high-value data assets at the source but also delays the response time for high-risk disaster alarms and emergency rescue. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing an edge computing-based data preprocessing system for observation stations.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: An edge computing-based observation station data preprocessing system includes:

[0006] Acquire multidimensional sensor data, mobile node coordinates, mobile node speed, and mobile node signal strength transmitted back by the observation station collaborative system;

[0007] Spatial topology relationships are updated based on the mobile node coordinates, the mobile node speed, and the mobile node signal strength to obtain dynamic graph connection weights;

[0008] The attention features of the multidimensional sensing data and the connection weights of the dynamic graph are extracted by a dynamic graph attention network to obtain hidden layer feature data.

[0009] Determine whether the hidden layer feature data has missing data, and obtain the missing data determination result;

[0010] Based on the missing data judgment result, the hidden layer feature data is interpolated to generate a spatiotemporal continuous observation map.

[0011] The fundamental physical differential equations of the observed objects are used as regularization terms in the loss function of the neural network embedded with physical knowledge.

[0012] The physical consistency of the spatiotemporal continuous observation map is calculated by the embedded physical knowledge neural network to obtain the physical residual value;

[0013] Perform an abnormal data isolation and discard operation based on the physical residual value.

[0014] Preferably, the steps for acquiring multidimensional sensor data, mobile node coordinates, mobile node velocity, and mobile node signal strength transmitted back by the observation station cooperative system specifically include:

[0015] Establish communication connections between edge observation stations and multiple mobile nodes within their jurisdiction;

[0016] Collect multi-source environmental information transmitted back from the multiple mobile nodes;

[0017] The multi-source environmental information is timestamped to obtain synchronized environmental information;

[0018] The monitoring messages in the synchronization environment information are analyzed to obtain the multidimensional sensing data;

[0019] The location frame data in the synchronization environment information is extracted to obtain the coordinates of the mobile node;

[0020] The speed of the mobile node is obtained by separating the navigation message data from the synchronization environment information;

[0021] The channel state of the multiple mobile nodes at the receiving end is measured to obtain the signal strength of the mobile nodes.

[0022] Preferably, the step of updating the spatial topology based on the mobile node coordinates, the mobile node speed, and the mobile node signal strength to obtain the dynamic graph connection weights specifically includes:

[0023] Based on the coordinates and velocity of the mobile nodes, calculate the distance change and motion direction vector of each mobile node between adjacent time steps;

[0024] The distance change and the motion direction vector are fused together to obtain the first... The mobile node and the first The dynamic feature vectors of the _ ... and ;

[0025] By combining the dynamic feature vector and the signal strength of the mobile node, the dynamic correlation degree between nodes is calculated. The formula is:

[0026] ;

[0027] in, The squared Euclidean norm of the difference in the dynamic eigenvectors is represented by... This represents the bandwidth parameter of the Gaussian kernel function. Indicates the first The mobile node and the first Joint signal strength among mobile nodes;

[0028] Construct a topological adjacency matrix based on the dynamic correlation between the nodes;

[0029] The topological adjacency matrix is ​​row-normalized to obtain the dynamic graph connection weights.

[0030] Preferably, the step of extracting attention features from the multidimensional sensing data and the dynamic graph connection weights using a dynamic graph attention network to obtain hidden layer feature data specifically includes:

[0031] The multidimensional sensing data is mapped to the initial feature space to obtain the initial feature representation of each mobile node;

[0032] Based on the dynamic graph connection weights, attention coefficients are calculated on the initial feature representations of the nodes to obtain the elements of the dynamic attention score matrix;

[0033] The initial feature representation of the node is weighted and summed according to the dynamic attention score matrix to obtain the first-level feature vector.

[0034] The first-level feature vector is input into a fully connected feedforward layer for nonlinear feature mapping to obtain the hidden layer feature data.

[0035] Preferably, the step of determining whether the hidden layer feature data has missing data and obtaining the missing data determination result specifically includes:

[0036] Traverse the data dimension feature information of the hidden layer feature data within a preset time window;

[0037] The number of missing values ​​in the data dimension feature information is counted to obtain the feature loss statistic.

[0038] Compare the feature loss statistics with a preset missing threshold, and extract the abnormal node analysis data where the feature loss statistics are greater than the preset missing threshold;

[0039] Determine that a signal interruption has occurred at the current time node corresponding to the abnormal node analysis data, and generate missing status flag record information;

[0040] Collect the missing status flag records of all nodes and perform resource integration and encoding processing;

[0041] The system information after resource integration and encoding is summarized into a network-wide node status monitoring vector, and the network-wide node status monitoring vector is used as the missing judgment result.

[0042] Preferably, the step of generating a spatiotemporally continuous observation map by interpolating the hidden layer feature data based on the missing data determination result specifically includes:

[0043] Extract the target node identity index representing the missing data from the missing data determination result;

[0044] The missing feature sequence vector is located in the hidden layer feature data according to the target node identity index;

[0045] Extract the time step parameters and spatial coordinate values ​​of the non-missing nodes surrounding the identity index of the target node;

[0046] The time step parameters and spatial coordinate values ​​of the non-missing nodes are input into the spatiotemporal kriging interpolation algorithm to obtain the spatiotemporal distribution variation model.

[0047] The Kriging interpolation weighting coefficients are calculated based on the spatiotemporal distribution variation model.

[0048] The feature data of the non-missing nodes are weighted linearly combined using the Kriging interpolation weight coefficients to generate a spatially compensated feature vector.

[0049] The spatial compensation feature vector is filled into the missing feature sequence vector to reconstruct the global observation feature matrix;

[0050] The global observation feature matrix is ​​subjected to spatiotemporal smoothing filtering to obtain the spatiotemporal continuous observation map.

[0051] Preferably, the step of using the fundamental physical differential equations of the observed object as regularization terms of the loss function of the neural network embedded with physical knowledge specifically includes:

[0052] The underlying geological and hydrodynamic mechanisms of the environment in which the observed object is located are analyzed, and the hydrodynamic dynamic equations are extracted.

[0053] The fluid dynamics equations are continuously discretized to obtain the fundamental physical differential equations.

[0054] A deep network architecture incorporating a multilayer perceptron is constructed as the embedded physical knowledge neural network.

[0055] The partial derivatives of the predicted output data of the embedded physical knowledge neural network with respect to the input spatiotemporal coordinate variables are extracted using automatic differentiation techniques.

[0056] Substituting the partial derivatives into the fundamental physical differential equations for calculation yields the physical constraint error term;

[0057] Calculate the mean square error between the predicted output data of the embedded physical knowledge neural network and the real sensing data, and establish a data-driven error configuration item;

[0058] The physical constraint error term is used as the regularization term, and the regularization term is linearly weighted and fused with the data-driven error configuration term to obtain the target combined loss function. The target combined loss function is used as the loss function of the embedded physical knowledge neural network.

[0059] Preferably, the step of calculating the physical consistency of the spatiotemporal continuous observation map using the embedded physical knowledge neural network to obtain the physical residual value specifically includes:

[0060] The spatiotemporal continuous observation map is input into the trained embedded physical knowledge neural network.

[0061] The spatiotemporal continuous observation map is forward propagated in the embedded physical knowledge neural network to obtain the predicted environmental state value;

[0062] Automatic differentiation is used to extract the derivative tensors of the predicted environmental state values ​​in the time and space dimensions.

[0063] Substitute the derivative tensors of each order into the fundamental physical differential equation, calculate the unsquared physical equation residuals that deviate from the physical conservation laws, and extract the absolute values ​​to obtain the absolute values ​​of the residuals.

[0064] The absolute value of the residual is subjected to a time-smoothing window filter to eliminate noise data interference in the preset frequency band and obtain a smooth output residual value.

[0065] The smoothed output residual value is transformed and mapped using a one-dimensional scalarization to obtain the physical residual value.

[0066] Preferably, the step of performing anomaly data isolation and discarding operation based on the physical residual value specifically includes:

[0067] Collect the historical physical residual mean and standard deviation of the observed object under normal baseline operating conditions;

[0068] The upper limit safety threshold for anomaly detection is calculated based on the mean of the historical physical residuals and the standard deviation of the historical physical residuals.

[0069] The physical residual value at the current time step is compared with the upper safety threshold for anomaly detection, and abnormal data fluctuation segments with physical residual values ​​greater than the upper safety threshold for anomaly detection are selected.

[0070] Extract the anomalous data fluctuation segment into the spatiotemporal continuous observation map and measure the magnitude of the data mutation.

[0071] Compare the data mutation magnitude scalar with a preset magnitude tolerance limit, and filter out the illegal data nodes whose data mutation magnitude scalar exceeds the preset magnitude tolerance limit;

[0072] The violation data node was determined to have caused a sensor failure event that violated physical constraints;

[0073] Trigger the edge security isolation mechanism to disconnect the data communication transmission channel between the edge computing gateway and the violating data node;

[0074] Clear the invalid and redundant record data corresponding to the non-compliant data nodes in the edge computing memory cache area to complete the abnormal data isolation and discarding operation.

[0075] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0076] In this invention, multidimensional sensor data, coordinate velocity, and signal strength of mobile nodes are acquired. Spatial topology relationships are updated and dynamic graph connection weights are calculated by combining spatial location and signal changes. Hidden layer feature data is extracted using a dynamic graph attention network combined with these weights. This not only achieves adaptive fusion of heterogeneous node features but also captures the dynamic evolution trajectory of the observation network in spatial layout in real time at the edge, improving the accuracy of multi-source data feature extraction. The invention judges feature missing states and performs data interpolation based on the missing state judgment to construct a spatiotemporally continuous observation map, filling the spatiotemporal data blind spots caused by node occlusion or communication interruptions. This ensures a complete and continuous observation view output to the superstructure. The fundamental physical differential equations of the observed objects are transformed into regularization terms of the loss function of an embedded physical knowledge neural network. Physical consistency is used to evaluate the physical residual values ​​produced by the spatiotemporally continuous observation map. Anomalies are identified based on the physical residual values, and data isolation and discarding operations are performed. This breaks away from the limitations of traditional anomaly identification relying solely on statistical thresholds, endowing the edge side with the cognitive ability to distinguish between underlying physical extreme events and sensor hardware failures, avoiding the erroneous deletion of high-value extreme disaster data, and improving the reliability of edge-side data pre-filtering. Attached Figure Description

[0077] Figure 1This is a schematic diagram of the principle of the present invention. Detailed Implementation

[0078] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0079] Please see Figure 1 The present invention provides a technical solution: a data preprocessing system for observation stations based on edge computing, comprising:

[0080] Acquire multidimensional sensor data, mobile node coordinates, mobile node speed, and mobile node signal strength transmitted back by the observation station collaborative system;

[0081] The spatial topology is updated based on the coordinates, speed, and signal strength of the mobile nodes to obtain the dynamic graph connection weights.

[0082] The attention features of multidimensional sensor data and dynamic graph connection weights are extracted by using a dynamic graph attention network to obtain hidden layer feature data.

[0083] Determine whether there are missing data in the hidden layer feature data, and obtain the missing data determination result;

[0084] Based on the missing data determination results, the hidden layer feature data is interpolated to generate a spatiotemporal continuous observation map;

[0085] The fundamental physical differential equations of the observed objects are used as regularization terms in the loss function of the neural network embedded with physical knowledge.

[0086] Physical consistency calculations are performed on the spatiotemporal continuous observation map using an embedded physical knowledge neural network to obtain physical residual values;

[0087] Perform abnormal data isolation and discard operations based on the physical residual value.

[0088] In this embodiment, the steps of acquiring multidimensional sensor data, mobile node coordinates, mobile node speed, and mobile node signal strength transmitted back by the observation station collaborative system specifically include: establishing communication connections between the edge observation station and multiple mobile nodes within its jurisdiction; collecting multi-source environmental information transmitted back by multiple mobile nodes; performing timestamp alignment processing on the multi-source environmental information to obtain synchronization environmental information; parsing the monitoring messages in the synchronization environmental information to obtain multidimensional sensor data; extracting positioning frame data from the synchronization environmental information to obtain mobile node coordinates; separating navigation message data from the synchronization environmental information to obtain mobile node speed; and measuring the channel state of multiple mobile nodes at the receiving end to obtain mobile node signal strength.

[0089] Specifically, a communication connection is established between the edge observation station and multiple mobile nodes within its jurisdiction. This involves assigning each mobile node an independent communication topic via the encrypted MQTT protocol. The edge observation station acts as a proxy server, subscribing to all nodes' topics and collecting multi-source environmental information transmitted back from multiple mobile nodes. Each mobile node publishes a data packet containing sensor readings, GPS coordinates, velocity vectors, and timestamps to its corresponding topic five times per second. The multi-source environmental information includes scalar data such as temperature, humidity, air pressure, and PM2.5 concentration. The multi-source environmental information undergoes timestamp alignment. Specifically, the edge observation station records the local reception time of each data packet and corrects the timestamps on the data packets based on a pre-determined and dynamically updated average network transmission delay from the node to the station. The correction formula is as follows: ,in This is the corrected timestamp. The timestamp is included with the data packet. To average network transmission latency, synchronization environment information is obtained. Monitoring messages within this information are parsed; these messages are encapsulated in JSON format. Values ​​from each sensor are extracted using key-value pairs to obtain multi-dimensional sensor data. Positioning frame data, conforming to the NMEA 0183 standard, is extracted from the GPRMC statement to obtain the mobile node coordinates. Navigation message data within the synchronization environment information is separated; east-west and north-south velocity components are decoded from a custom binary message containing speed and heading to obtain the mobile node speed. The channel status of multiple mobile nodes at the receiving end is measured. Specifically, at the data receiving hardware level, for each data packet from a source IP address, the corresponding received signal strength indicator (RSI) and signal-to-noise ratio (SNR) values ​​are read. The normalized RSI and SNR values ​​are then weighted and summed, with weights set empirically to 0.6 and 0.4, to obtain the mobile node signal strength.

[0090] In this embodiment, the step of updating the spatial topology based on the coordinates, velocity, and signal strength of mobile nodes to obtain the dynamic graph connection weights specifically includes: calculating the distance change and motion direction vector of each mobile node between adjacent time steps based on the coordinates and velocity of the mobile nodes; and fusing the distance change and motion direction vector to obtain the first... The mobile node and the first The dynamic feature vectors of the _ ... and By combining dynamic feature vectors and mobile node signal strength, the dynamic correlation between nodes is calculated. The formula is: ,in, The squared Euclidean norm of the difference between dynamic eigenvectors. This represents the bandwidth parameter of the Gaussian kernel function. Indicates the first The mobile node and the first The joint signal strength among mobile nodes; the topological adjacency matrix is ​​constructed based on the dynamic correlation between nodes; the topological adjacency matrix is ​​row normalized to obtain the dynamic graph connection weights.

[0091] Specifically, spatial topology updates are performed based on the coordinates, velocity, and signal strength of mobile nodes. First, based on the coordinates and velocity of each mobile node, the distance change and motion direction vector between adjacent time steps are calculated. The distance change is obtained by calculating the Euclidean distance between the node coordinates at two consecutive timestamps, while the motion direction vector is directly taken as the unit vector of the mobile node velocity. The distance change and motion direction vector are then fused together. Specifically, the node's current coordinates, velocity vector, and distance change are concatenated into a high-dimensional vector to obtain the... The mobile node and the first The dynamic feature vectors of the _ ... and By combining dynamic feature vectors and mobile node signal strength, the dynamic correlation between nodes is calculated. The calculation formula is as follows: ,in, The squared Euclidean norm of the difference between dynamic eigenvectors. The bandwidth parameter of the Gaussian kernel function is dynamically set to the median of the Euclidean distances between all node pairs at the current time step. For example, if the calculated median distance is 65 meters, then... Set to 65. Indicates the first The mobile node and the first The joint signal strength among mobile nodes is calculated as the geometric mean of the individual mobile node signal strengths of each node. An N×N topological adjacency matrix is ​​constructed based on the dynamic correlation between nodes, where N is the total number of mobile nodes. The element at position (i, j) in the matrix is... The topological adjacency matrix is ​​row normalized by dividing each row of the matrix by the sum of all elements in that row to ensure that the sum of the elements in each row is 1, thus obtaining the dynamic graph connection weights.

[0092] In this embodiment, the step of extracting attention features from multidimensional sensing data and dynamic graph connection weights using a dynamic graph attention network to obtain hidden layer feature data specifically includes: mapping the multidimensional sensing data to an initial feature space to obtain the initial feature representation of each mobile node; and calculating the attention coefficients of the initial node feature representations in conjunction with the dynamic graph connection weights to obtain the elements of the dynamic attention score matrix. The formula is: ,in, and These represent the corresponding numbers in the connection weights of the dynamic graph. The mobile nodes respectively interact with the first The mobile node, the Prior weight values ​​between each mobile node This represents the learnable weight parameter matrix. , and They represent the first The mobile node, the The mobile node and the first Initial node feature representation of each mobile node Indicates feature concatenation operator, This represents the bias vector. Indicates the first The set of neighboring nodes of a mobile node. This represents the activation function of the linear unit with leakage correction. The natural exponential function is represented; the initial feature representation of the node is weighted and summed according to the dynamic attention score matrix to obtain the first-level feature vector; the first-level feature vector is input into the fully connected feedforward layer for nonlinear feature mapping to obtain the hidden layer feature data.

[0093] Specifically, attention features are extracted from multidimensional sensor data and dynamic graph connection weights using a dynamic graph attention network. First, the multidimensional sensor data is mapped to a 128-dimensional initial feature space through a fully connected linear layer, obtaining the initial feature representations of each mobile node. A dynamic graph attention network consisting of two stacked graph attention layers is then constructed, with each layer followed by an exponential linear unit activation function. The network input consists of the initial feature representations of all nodes and the dynamic graph connection weights. In conjunction with the dynamic graph connection weights, attention coefficients are calculated for the initial node feature representations. The dynamic graph connection weights are incorporated as prior knowledge into the calculation of the attention score, resulting in the elements of the dynamic attention score matrix. The calculation formula is as follows: ,in, and These are the scalar values ​​in the dynamic graph connection weight matrix representing the connection strength between node i and node j, and between node i and node k, respectively. It is a learnable weight parameter matrix with dimensions (256, 1), used to map the concatenated 256-dimensional features into a single attention scalar. , and It is a 128-dimensional initial feature representation of the nodes. This represents the operation of concatenating two 128-dimensional vectors into a single 256-dimensional vector. The bias vector is set to 0 here and is not used. Indicates the relationship with the first A mobile node connects to a set of neighboring nodes whose weights are not zero. The negative slope is set to 0.2. The initial feature representation of the node is weighted and summed according to the dynamic attention score matrix to obtain a 128-dimensional first-level feature vector after the first layer of attention aggregation. The first-level feature vector is input into a second graph attention layer with the same structure for processing. The output 128-dimensional feature is then input into a feedforward network consisting of two fully connected layers for nonlinear feature mapping. The first layer of the feedforward network has 256 neurons, the second layer has 128 neurons, and a modified linear unit activation function is used in between to obtain the hidden layer feature data.

[0094] In this embodiment, the step of determining whether there is missing data in the hidden layer feature data and obtaining the missing data determination result specifically includes: traversing the data dimension feature information of the hidden layer feature data within a preset time window; counting the number of missing values ​​in the data dimension feature information to obtain the feature loss statistic; comparing the feature loss statistic with a preset missing data determination threshold, and extracting abnormal node analysis data where the feature loss statistic is greater than the preset missing data determination threshold; determining that a signal interruption has occurred at the current time node corresponding to the abnormal node analysis data, and generating missing status flag record information; collecting the missing status flag record information of all nodes and performing resource integration and encoding processing; summarizing the system information after resource integration and encoding processing into a network-wide node status monitoring vector, and using the network-wide node status monitoring vector as the missing data determination result.

[0095] Specifically, to determine if there is missing data in the hidden layer feature data, firstly, iterate through the data dimensional feature information of the hidden layer feature data within a sliding time window of a fixed size of 20 consecutive time steps, counting the number of missing values ​​in the statistical dimensional feature information. Specifically, check whether the 128-dimensional hidden layer feature vector of each node within the 20 time steps is a preset null placeholder, such as an all-zero vector. Count the number of times each node appears as a null placeholder within this window to obtain the feature loss statistic. Set a preset missing value judgment threshold, which is empirically set to 10. That is, within a 20-time-step window, if the feature loss statistic of a node is greater than 10, the node is considered to have severe data loss. Compare the feature loss statistic with the preset missing value judgment threshold, and extract all... The identity identifiers of nodes with a feature loss statistic greater than 10 and their corresponding data within the time window are used as abnormal node analysis data. It is determined that the current time node corresponding to the abnormal node analysis data has experienced a signal interruption. A missing status flag record containing the node ID and the current timestamp is generated for each identified node. At the end of each time step, the missing status flag record information of all nodes is collected and processed by resource integration encoding. Specifically, a binary vector with a length equal to the total number of mobile nodes is created. If the i-th node is determined to have missing data in this time step, the i-th position of the vector is 1, otherwise it is 0. The system information after resource integration encoding is summarized into a network-wide node status monitoring vector, and the network-wide node status monitoring vector is used as the missing judgment result.

[0096] In this embodiment, the step of generating a spatiotemporal continuous observation map by interpolating the hidden layer feature data based on the missing data determination result specifically includes: extracting the target node identity index representing the missing data from the missing data determination result; locating the missing feature sequence vector in the hidden layer feature data based on the target node identity index; extracting the time step parameters and spatial coordinate values ​​of the non-missing nodes surrounding the target node identity index; inputting the time step parameters and spatial coordinate values ​​of the non-missing nodes into a spatiotemporal kriging interpolation algorithm to fit a spatiotemporal distribution variation model; and calculating the kriging interpolation weight coefficients based on the spatiotemporal distribution variation model. The formula is: ,in, This represents the total number of non-missing nodes. Represents the spatiotemporal semivariogram matrix The inverse matrix of the first Line number The elements of the column, the spatiotemporal semi-variogram matrix The elements are calculated from the temporal and spatial distances between non-missing nodes using a spatiotemporal distribution variation model. This represents the target spatiotemporal semivariogram calculated based on the spatiotemporal distribution variability model. Indicates the first The time step parameters for each non-missing node. Indicates the first Spatial coordinates of non-missing nodes Indicates the target interpolation time step parameter. The target interpolation spatial location is represented; the feature data of non-missing nodes are weighted linearly combined using the Kriging interpolation weight coefficients to generate a spatially compensated feature vector; the spatially compensated feature vector is filled into the missing feature sequence vector to reconstruct the global observation feature matrix; the global observation feature matrix is ​​smoothed by spatiotemporal dimension filtering to obtain a spatiotemporally continuous observation map.

[0097] Specifically, based on the missing data determination results, data interpolation is performed on the hidden layer feature data. First, the identity index of the target node representing the missing data is extracted from the missing data determination results. Based on the target node identity index, the specific missing feature sequence vector is located in the time series of the hidden layer feature data. The time step parameters and spatial coordinate values ​​of the non-missing nodes surrounding the target node identity index are extracted. Specifically, the 30 nearest neighbors with valid data to the target missing point are searched in the spatiotemporal dimension. The time step parameters and spatial coordinate values ​​of these 30 non-missing nodes are input into the spatiotemporal kriging interpolation algorithm. By calculating the spatiotemporal distance and feature value differences between each pair of these 30 nodes, an empirical semi-variogram is constructed. A spherical spatiotemporal distribution variation model is then fitted to this graph using the weighted least squares method, thereby obtaining the specific parameters of the spatiotemporal distribution variation model, including the spatiotemporal nugget value, sill value, and range. Based on the spatiotemporal distribution variation model, the kriging interpolation weight coefficients are calculated. The calculation formula is as follows: ,in It includes all weight coefficients The vector, It is a 30x30 spatiotemporal semi-variogram matrix, whose elements The spatiotemporal distance between the m-th and n-th non-missing nodes is calculated using a spatiotemporal distribution variation model. It is a 30x1 target spatiotemporal semi-variogram vector, whose elements are calculated by the spatiotemporal distance between each non-missing node and the target interpolation point through a spatiotemporal distribution variogram model. The calculated Kriging interpolation weight coefficients are used to perform a weighted linear combination calculation on the 128-dimensional feature data of the 30 non-missing nodes to generate a 128-dimensional spatial compensation feature vector. The spatial compensation feature vector is filled into the missing feature sequence vector to reconstruct the global observation feature matrix. The global observation feature matrix is ​​then subjected to spatiotemporal smoothing filtering, specifically using a 3x3x3 spatiotemporal Gaussian filter to perform convolution operations on the interpolation point and its neighborhood to obtain a spatiotemporal continuous observation map.

[0098] In this embodiment, the step of using the fundamental physical differential equation of the observed object as the regularization term of the loss function of the embedded physical knowledge neural network specifically includes: analyzing the underlying geological fluid dynamics mechanism of the environment in which the observed object is located, and extracting the fluid dynamics dynamic equation; performing continuous discretization transformation on the fluid dynamics dynamic equation to obtain the fundamental physical differential equation; constructing a deep network architecture including a multilayer perceptron as the embedded physical knowledge neural network; extracting the partial derivatives of the predicted output data of the embedded physical knowledge neural network with respect to the input spatiotemporal coordinate variables using automatic differentiation technology; and substituting the partial derivatives into the fundamental physical differential equation for calculation to obtain the physical constraint error term. The formula is: ,in, This indicates the total number of monitoring points for the sampled data. The neural network with embedded physical knowledge is in the first... Predicted output data at each sampling data monitoring point Represents time-independent variables. Represents spatially independent variables. This indicates the fluid dynamic viscosity adjustment parameter. This represents the partial derivative operator; it calculates the mean square error between the predicted output data of the embedded physical knowledge neural network and the real sensing data, and establishes a data-driven error configuration term; it uses the physical constraint error term as a regularization term, and performs linear weighted fusion calculation between the regularization term and the data-driven error configuration term to obtain the target combined loss function, and uses the target combined loss function as the loss function of the embedded physical knowledge neural network.

[0099] Specifically, the fundamental physical differential equation of the observed object is used as the regularization term of the loss function of the embedded physical knowledge neural network. First, the underlying geological fluid dynamics mechanism of the observed object's environment is analyzed, and the one-dimensional inviscid Burgers equation is extracted as the fluid dynamics equation. This equation does not require discretization transformation; its continuous form is the fundamental physical differential equation. A multilayer perceptron with 8 hidden layers and 20 neurons per layer is constructed as the deep network architecture of the embedded physical knowledge neural network. The network input layer has 2 neurons, receiving time and spatial coordinates respectively, and the output layer has 1 neuron, outputting the predicted flow field velocity. All hidden layers use the hyperbolic tangent activation function. The partial derivatives of the predicted output data of the embedded physical knowledge neural network with respect to the input spatiotemporal coordinate variables are extracted using automatic differentiation technology. The partial derivatives are substituted into the fundamental physical differential equation for calculation to obtain the physical constraint error term. The calculation formula is as follows: ,in, This refers to the total number of collocation points used to calculate the physical loss, for example, 10,000 points uniformly sampled within the training domain. For the network in the first Prediction speed at each collocation point and For time and space coordinates, fluid dynamic viscosity adjustment parameters Based on prior knowledge, it is set to 0.01. The partial derivative is calculated by automatic differentiation. The mean square error between the predicted output data of the embedded physical knowledge neural network and the real sensing data is calculated. A data-driven error configuration term is established, and the physical constraint error term is used as a regularization term. The regularization term and the data-driven error configuration term are linearly weighted and fused. The weight of the data-driven error term is set to 1.0 and the weight of the physical constraint error term is set to 0.1 to obtain the target combined loss function. The target combined loss function is used as the loss function of the embedded physical knowledge neural network.

[0100] In this embodiment, the steps of calculating the physical consistency of a spatiotemporal continuous observation map using an embedded physical knowledge neural network to obtain physical residual values ​​specifically include: inputting the spatiotemporal continuous observation map into a trained embedded physical knowledge neural network; performing forward propagation of the spatiotemporal continuous observation map in the embedded physical knowledge neural network to obtain environmental state prediction values; extracting the derivative tensors of the environmental state prediction values ​​in the time and space dimensions using automatic differentiation; substituting the derivative tensors into the fundamental physical differential equations to calculate the unsquared physical equation residuals that deviate from the physical conservation laws, and extracting the absolute values ​​to obtain the absolute values ​​of the residuals; performing a time-smoothing window filtering operation on the absolute values ​​of the residuals to eliminate noise data interference in a preset frequency band, obtaining smoothed output residual values; and performing a one-dimensional scalarization transformation mapping on the smoothed output residual values ​​to obtain physical residual values.

[0101] Specifically, a physical consistency calculation is performed on the spatiotemporal continuous observation map using an embedded physical knowledge neural network. First, the time and spatial coordinates of each grid point in the spatiotemporal continuous observation map are input into the pre-trained embedded physical knowledge neural network. The network then performs forward propagation on each input spatiotemporal coordinate point to obtain the predicted environmental state value for that point. Automatic differentiation techniques are used to extract the derivative tensors of the predicted environmental state value in the time and spatial dimensions. Specifically, the first-order time derivative, first-order spatial derivative, and second-order spatial derivative of the network output relative to the input are calculated. These derivative tensors are then substituted into the fundamental physical differential equations, namely the Burgers equations. The unsquared physical equation residuals of each grid point deviating from the physical conservation law are calculated, and their absolute values ​​are extracted to obtain the absolute residual values. For each spatial location point, the absolute residual values ​​formed on the time series are input into a moving average filter with a window size of 5 for time smoothing window filtering to eliminate high-frequency fluctuation interference introduced by data noise or numerical calculation, and obtain smooth output residual values. For each time step, the smooth output residual values ​​of all spatial grid points at that time are averaged, and this average value is used as the physical inconsistency measure of the system as a whole at that time step. A one-dimensional scalarization transformation mapping is performed to obtain the physical residual values.

[0102] In this embodiment, the steps for performing abnormal data isolation and discarding based on physical residual values ​​specifically include: collecting the historical mean and standard deviation of the physical residuals of the observed object under normal baseline operating conditions; calculating the upper limit safety threshold for anomaly judgment based on the historical mean and standard deviation of the physical residuals; comparing the physical residual value of the current time step with the upper limit safety threshold for anomaly judgment, and filtering out abnormal data fluctuation segments whose physical residual values ​​are greater than the upper limit safety threshold for anomaly judgment; extracting the data mutation amplitude scalar of the abnormal data fluctuation segment in the spatiotemporal continuous observation map; comparing the data mutation amplitude scalar with the preset amplitude tolerance upper limit, and filtering out the violation data nodes whose data mutation amplitude scalar exceeds the preset amplitude tolerance upper limit; determining that the violation data node has generated a sensor failure event that violates physical constraints; triggering the edge security isolation mechanism to disconnect the data communication transmission channel between the edge computing gateway and the violation data node; clearing the invalid redundant record data corresponding to the violation data node in the edge computing memory cache area, and completing the abnormal data isolation and discarding operation.

[0103] Specifically, based on the physical residual values, anomaly data isolation and discarding operations are performed. First, historical physical residual values ​​of the observed object under normal baseline operating conditions for 24 consecutive hours are collected. The mean and standard deviation of the historical physical residuals are calculated. For example, if the mean is 0.04 and the standard deviation is 0.015, the upper limit safety threshold for anomaly judgment is calculated using the 3-sigma criterion based on the historical physical residual mean and standard deviation. The calculation method is: Threshold = Historical Physical Residual Mean + 3 × Historical Physical Residual Standard Deviation, i.e., 0.04 + 3 × 0.015 = The physical residual value of the current time step is compared with the anomaly judgment upper limit safety threshold of 0.085. If the current physical residual value, for example, 0.12, is greater than 0.085, the spatiotemporal data segment corresponding to this time step is selected as an abnormal data fluctuation segment. The data mutation amplitude scalar of the abnormal data fluctuation segment in the spatiotemporal continuous observation spectrum is extracted, that is, the maximum absolute value of all sensor readings in the segment is found. At the same time, the data mutation amplitude scalar is compared with the preset amplitude tolerance upper limit. This upper limit is set according to the sensor range. For example, for a temperature sensor with a range of -20 to 50 degrees Celsius, its tolerance upper limit is set to 60 degrees Celsius. Violating data nodes with a maximum absolute value of reading exceeding 60 degrees Celsius are selected. It is determined that the violating data node has generated a sensor failure event that violates the physical constraints. The edge security isolation mechanism is triggered. The network identifier of the node is added to the denial-of-service list of the edge computing gateway, the data communication transmission channel between the edge computing gateway and the violating data node is disconnected, and all invalid redundant record data corresponding to the violating data node in the edge computing memory cache area in the past minute is cleared, completing the abnormal data isolation and discard operation.

Claims

1. A data preprocessing system for observation stations based on edge computing, characterized in that, The system includes: Acquire multidimensional sensor data, mobile node coordinates, mobile node speed, and mobile node signal strength transmitted back by the observation station collaborative system; Spatial topology relationships are updated based on the mobile node coordinates, the mobile node speed, and the mobile node signal strength to obtain dynamic graph connection weights; The attention features of the multidimensional sensing data and the connection weights of the dynamic graph are extracted by a dynamic graph attention network to obtain hidden layer feature data. Determine whether the hidden layer feature data has missing data, and obtain the missing data determination result; Based on the missing data judgment result, the hidden layer feature data is interpolated to generate a spatiotemporal continuous observation map. The fundamental physical differential equations of the observed objects are used as regularization terms in the loss function of the neural network embedded with physical knowledge. The physical consistency of the spatiotemporal continuous observation map is calculated by the embedded physical knowledge neural network to obtain the physical residual value; Perform an abnormal data isolation and discard operation based on the physical residual value.

2. The edge computing-based observation station data preprocessing system according to claim 1, characterized in that, The steps for acquiring multidimensional sensor data, mobile node coordinates, mobile node velocity, and mobile node signal strength transmitted back by the observation station collaborative system specifically include: Establish communication connections between edge observation stations and multiple mobile nodes within their jurisdiction; Collect multi-source environmental information transmitted back from the multiple mobile nodes; The multi-source environmental information is timestamped to obtain synchronized environmental information; The monitoring messages in the synchronization environment information are analyzed to obtain the multidimensional sensing data; The location frame data in the synchronization environment information is extracted to obtain the coordinates of the mobile node; The speed of the mobile node is obtained by separating the navigation message data from the synchronization environment information; The channel state of the multiple mobile nodes at the receiving end is measured to obtain the signal strength of the mobile nodes.

3. The edge computing-based observation station data preprocessing system according to claim 1, characterized in that, The step of updating the spatial topology based on the mobile node coordinates, the mobile node speed, and the mobile node signal strength to obtain the dynamic graph connection weights specifically includes: Based on the coordinates and velocity of the mobile nodes, calculate the distance change and motion direction vector of each mobile node between adjacent time steps; The distance change and the motion direction vector are fused together to obtain the first... The mobile node and the first The dynamic feature vectors of the _ ... and ; By combining the dynamic feature vector and the signal strength of the mobile node, the dynamic correlation degree between nodes is calculated. The formula is: ； in, The squared Euclidean norm of the difference in the dynamic eigenvectors is represented by... This represents the bandwidth parameter of the Gaussian kernel function. Indicates the first The mobile node and the first Joint signal strength among mobile nodes; Construct a topological adjacency matrix based on the dynamic correlation between the nodes; The topological adjacency matrix is ​​row-normalized to obtain the dynamic graph connection weights.

4. The edge computing-based observation station data preprocessing system according to claim 1, characterized in that, The step of extracting hidden layer feature data by using a dynamic graph attention network to perform attention feature extraction on the multidimensional sensing data and the dynamic graph connection weights to obtain hidden layer feature data specifically includes: The multidimensional sensing data is mapped to the initial feature space to obtain the initial feature representation of each mobile node; Based on the dynamic graph connection weights, attention coefficients are calculated on the initial feature representations of the nodes to obtain the elements of the dynamic attention score matrix; The initial feature representation of the node is weighted and summed according to the dynamic attention score matrix to obtain the first-level feature vector. The first-level feature vector is input into a fully connected feedforward layer for nonlinear feature mapping to obtain the hidden layer feature data.

5. The edge computing-based observation station data preprocessing system according to claim 1, characterized in that, The steps for determining whether the hidden layer feature data has missing data and obtaining the missing data determination result specifically include: Traverse the data dimension feature information of the hidden layer feature data within a preset time window; The number of missing values ​​in the data dimension feature information is counted to obtain the feature loss statistic. Compare the feature loss statistics with a preset missing threshold, and extract the abnormal node analysis data where the feature loss statistics are greater than the preset missing threshold; Determine that a signal interruption has occurred at the current time node corresponding to the abnormal node analysis data, and generate missing status flag record information; Collect the missing status flag records of all nodes and perform resource integration and encoding processing; The system information after resource integration and encoding is summarized into a network-wide node status monitoring vector, and the network-wide node status monitoring vector is used as the missing judgment result.

6. The edge computing-based observation station data preprocessing system according to claim 1, characterized in that, The step of generating a spatiotemporal continuous observation map by interpolating the hidden layer feature data based on the missing data determination result specifically includes: Extract the target node identity index representing the missing data from the missing data determination result; The missing feature sequence vector is located in the hidden layer feature data according to the target node identity index; Extract the time step parameters and spatial coordinate values ​​of the non-missing nodes surrounding the identity index of the target node; The time step parameters and spatial coordinate values ​​of the non-missing nodes are input into the spatiotemporal kriging interpolation algorithm to obtain the spatiotemporal distribution variation model. The Kriging interpolation weighting coefficients are calculated based on the spatiotemporal distribution variation model. The feature data of the non-missing nodes are weighted linearly combined using the Kriging interpolation weight coefficients to generate a spatially compensated feature vector. The spatial compensation feature vector is filled into the missing feature sequence vector to reconstruct the global observation feature matrix; The global observation feature matrix is ​​subjected to spatiotemporal smoothing filtering to obtain the spatiotemporal continuous observation map.

7. The edge computing-based observation station data preprocessing system according to claim 1, characterized in that, The steps of using the fundamental physical differential equations of the observed object as regularization terms in the loss function of an embedded physical knowledge neural network specifically include: The underlying geological and hydrodynamic mechanisms of the environment in which the observed object is located are analyzed, and the hydrodynamic dynamic equations are extracted. The fluid dynamics equations are continuously discretized to obtain the fundamental physical differential equations. A deep network architecture incorporating a multilayer perceptron is constructed as the embedded physical knowledge neural network. The partial derivatives of the predicted output data of the embedded physical knowledge neural network with respect to the input spatiotemporal coordinate variables are extracted using automatic differentiation techniques. Substituting the partial derivatives into the fundamental physical differential equations for calculation yields the physical constraint error term; Calculate the mean square error between the predicted output data of the embedded physical knowledge neural network and the real sensing data, and establish a data-driven error configuration item; The physical constraint error term is used as the regularization term, and the regularization term is linearly weighted and fused with the data-driven error configuration term to obtain the target combined loss function. The target combined loss function is used as the loss function of the embedded physical knowledge neural network.

8. The edge computing-based observation station data preprocessing system according to claim 1, characterized in that, The step of calculating the physical consistency of the spatiotemporal continuous observation map using the embedded physical knowledge neural network to obtain the physical residual values ​​specifically includes: The spatiotemporal continuous observation map is input into the trained embedded physical knowledge neural network. The spatiotemporal continuous observation map is forward propagated in the embedded physical knowledge neural network to obtain the predicted environmental state value; Automatic differentiation is used to extract the derivative tensors of the predicted environmental state values ​​in the time and space dimensions. Substitute the derivative tensors of each order into the fundamental physical differential equation, calculate the unsquared physical equation residuals that deviate from the physical conservation laws, and extract the absolute values ​​to obtain the absolute values ​​of the residuals. The absolute value of the residual is subjected to a time-smoothing window filter to eliminate noise data interference in the preset frequency band and obtain a smooth output residual value. The smoothed output residual value is transformed and mapped using a one-dimensional scalarization to obtain the physical residual value.

9. The edge computing-based observation station data preprocessing system according to claim 1, characterized in that, The steps for performing abnormal data isolation and discarding operations based on the physical residual value specifically include: Collect the historical physical residual mean and standard deviation of the observed object under normal baseline operating conditions; The upper limit safety threshold for anomaly detection is calculated based on the mean of the historical physical residuals and the standard deviation of the historical physical residuals. The physical residual value at the current time step is compared with the upper safety threshold for anomaly detection, and abnormal data fluctuation segments with physical residual values ​​greater than the upper safety threshold for anomaly detection are selected. Extract the anomalous data fluctuation segment into the spatiotemporal continuous observation map and measure the magnitude of the data mutation. Compare the data mutation magnitude scalar with a preset magnitude tolerance limit, and filter out the illegal data nodes whose data mutation magnitude scalar exceeds the preset magnitude tolerance limit; The violation data node was determined to have caused a sensor failure event that violated physical constraints; Trigger the edge security isolation mechanism to disconnect the data communication transmission channel between the edge computing gateway and the violating data node; Clear the invalid and redundant record data corresponding to the non-compliant data node in the edge computing memory cache area to complete the abnormal data isolation and discarding operation.

Images