Fusion method and system based on interaction iteration of sensor data and satellite data

Abstract

The present application relates to a fusion method and system based on interactive iteration of sensor data and satellite data. The method obtains sensor data of one or more sensor observation points and satellite data of satellite observation points in a long period, to form a sample data with short period data and long period data in the same long period, to construct a neural network regression model with short period data as input value and long period data as output value, to obtain a prediction model by using the actual obtained long period data as expected output value and adopting a back propagation algorithm to iteratively train the model. The system comprises a satellite remote sensing system, a plurality of ground sensors and a neural network system / platform, and the neural network system / platform is used to implement the method. The present application can construct a prediction model based on fusion of sensor data and satellite data, and can be used for modeling, prediction / analysis of land subsidence and other suitable occasions / purposes.

Description

Technical Field

[0001] This invention relates to a fusion method and system based on the interactive iteration of sensor data and satellite data, which can be used in applications such as ground subsidence model research and prediction. Background Technology

[0002] Currently, ground subsidence analysis is typically conducted using interferometry techniques from synthetic aperture radar (SAR) on remote sensing satellites. SAR interferometry fully utilizes acquired radar phase information to extract three-dimensional ground information. The differential interferometry (InSAR) technology used analyzes phase changes to detect surface variations at the centimeter or even millimeter level, generating ground subsidence data. However, due to the long revisit cycle of satellites at the same location and angle, ground sensors are usually required to increase the monitoring frequency for areas with significant subsidence variations. Ground sensors are expensive, and the monitored area is limited to the sensor's location (ground observation point), while remote sensing satellite differential interferometry covers a large area (typically hundreds of square kilometers per scene). Therefore, there is currently a lack of technology that effectively utilizes and integrates these two types of data to gain advantages from both.

[0003] On the other hand, existing forecasting techniques use machine learning, neural networks, and nested hybrid models to predict data over a single period, meaning they only use short-term data to predict data at future times (or points in time). This approach presents problems such as: the limitation on data types, preventing the fusion of multiple data types; and the limitation on forecasting models, preventing the fusion of multiple models / data sets. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, the present invention provides a fusion method based on the interactive iteration of sensor data and satellite data, and a fusion system based on the interactive iteration of sensor data and satellite data for implementing this method, so as to build a prediction model based on the fusion of sensor data and satellite data.

[0005] The technical solution of this invention is as follows: a fusion method based on the interactive iteration of sensor data and satellite data, which acquires sensor data from one or more long-term sensor observation points and satellite data from satellite observation points, wherein the sensor data is short-term data based on ground sensors, the satellite data is long-term data based on remote sensing satellites, and the satellite observation points are adjacent satellite observation points of related sensor observation points. A sample data is composed of short-term data and long-term data of the same long period, and a neural network regression model is constructed with short-term data as input value and long-term data as output value. The actual long-term data is used as the expected output value, and the model is iteratively trained using the backpropagation algorithm to obtain a trained model that meets the error requirements. The trained model is then used as the prediction model for the satellite observation point.

[0006] Preferably, the long period is a multiple of the short period.

[0007] Preferably, the sensor data and satellite data are ground subsidence data, and the so-called prediction model is a model used for ground subsidence analysis and / or prediction.

[0008] In calculations involving predictive models, for any point, it is possible and appropriate to consider only its planar coordinates.

[0009] Ground settlement data can be obtained solely from the vertical displacement data of the ground surface.

[0010] The observation area can be divided into several closely spaced zones, with the edges of each zone forming convex polygons. Each zone contains exactly one sensor observation point, and the vertices of each zone are satellite observation points. The so-called observation area should be a part of the satellite observation area; however, when appropriate, it can also refer to the entire satellite observation area.

[0011] The number and distribution of ground sensors can be determined based on zoning requirements and prediction accuracy requirements.

[0012] For any vertex in any partition, a prediction model for that vertex can be established using long-period data of that vertex and short-period data of sensor observation points in any partition in which that vertex is located.

[0013] Predictive data for the corresponding vertex can be calculated based on the prediction model, which may include predictive data at the same time point as the short-cycle data and / or predictive data at the same time point as the long-cycle data.

[0014] Preferably, the partition is divided into several closely spaced triangular regions, and the vertices of the triangles in each triangular region of the same partition are the vertices of the convex polygons of that partition. Based on the predicted data of the triangle vertices, the predicted data of the points in the triangular region other than the vertices can be calculated by interpolation or other suitable methods.

[0015] The fusion system based on the interactive iteration of sensor data and satellite data includes a satellite remote sensing system and several ground sensors, as well as a neural network system / platform. The satellite remote sensing system is used to generate long-term data on ground subsidence involving the satellite observation area. The ground sensors are set in the satellite observation area and are used to generate short-term data on ground subsidence involving the observation points of their respective sensors. The neural network system / platform is used to implement any of the fusion methods based on the interactive iteration of sensor data and satellite data disclosed in this invention. The satellite observation points are points in the satellite observation area.

[0016] The neural network system / platform can be connected to an operating terminal for human-computer interaction via a communication network or other communication methods, accepting manual input from the operating terminal and sending the analysis / prediction results to the operating terminal for display.

[0017] The beneficial effects of this invention are: through the interactive iteration of sensor data and satellite remote sensing data, the effective fusion of the two data analyses is achieved, and the established ground subsidence model can be analyzed and predicted from the time and space dimensions. The number of ground sensors required is small, and in conjunction with other existing data processing methods, it can perform ground subsidence analysis and prediction over a large area, and can also be used in other suitable occasions / purposes. Attached Figure Description

[0018] Figure 1 This is an example of the distribution of short-period observation data based on ground sensors and long-period observation data based on remote sensing satellites;

[0019] Figure 2 It is obtained by using the method of the present invention and Figure 1 Example of the regression curve;

[0020] Figure 3 This is an example of the distribution of ground sensor observation points and their related (nearby) remote sensing satellite observation points;

[0021] Figure 4 Based on Figure 3 The example shown is a convex polygon partition of ground sensor observation points and related (nearby) remote sensing satellite observation points;

[0022] Figure 5 Based on Figure 4 Example of a Delaunay triangulation for a convex polygon shown;

[0023] Figure 6 Based on Figure 5 The example shown illustrates the interpolation operation and data changes of the Delaunay triangle (the color depth represents the numerical changes). Detailed Implementation

[0024] See Figures 1-3 In the observation area, several ground sensors are set up to obtain relevant data on ground subsidence. The ground sensors use a short-period sampling method, for example, collecting data once every 10 minutes. Such observation points that can collect short-period data can be called short-period observation points, or, in the scenario of this invention, sensor observation points or ground observation points. Figure 3 In the example, the sensor observation point is labeled as point O.

[0025] Ground sensors can employ any suitable existing technology capable of collecting ground settlement data. Based on their respective working principles / methods, ground settlement data at the sensor observation point can be obtained through data processing by the sensor itself or by supporting facilities. This data is ground settlement data based on ground sensors, and in the context of this invention, it can also be referred to as short-period data or sensor data.

[0026] Based on actual needs, several locations on the ground can be selected within the coverage area of ​​remote sensing satellites as observation (analysis) points for remote sensing satellite signals. Suitable ground landmarks can be used as identification markers for the corresponding observation points. In the current technological context, the data involved in remote sensing satellites is long-period data, for example, once every 10 days. Based on existing technology, ground subsidence data for each observation point can be obtained through synthetic aperture radar signals or other remote sensing signals from remote sensing satellites. This type of data obtained based on long-period sampling can be called long-period data, and the observation points capable of acquiring long-period data can be called long-period observation points. In the scenario involved in this invention, this type of long-period data can also be called satellite data, or remote sensing data, or satellite remote sensing data, and these long-period observation points can also be called remote sensing satellite observation points, or satellite observation points. Figure 3 The satellite observation points in the example are labeled as points A, B, C, D, and E, respectively.

[0027] For any sensor observation point O and any satellite observation point around that sensor observation point (e.g., Figure 3 In the example, any point A, B, C, D, or E may or may not include satellite observation points that coincide with the sensor observation point. Based on short-period data (e.g., data every 10 minutes) from the sensor observation point and long-period data (e.g., data every 10 days) from the satellite observation point, modeling and prediction can be performed using methods such as neural networks to learn / train a ground subsidence model (regression model) that fuses short-period and long-period data. Since ground subsidence data is time-series data, both the short-period and long-period data involved in the sample / model can include ground subsidence amount and time (sampling time point, or simply sampling point).

[0028] The method utilizes ground-based sensors to acquire actual values ​​of ground subsidence (or subsidence changes) at the sensor observation point, which is the short-period data of that observation point. Then, it obtains actual values ​​of ground subsidence (or subsidence changes) at relevant satellite observation points from satellite remote sensing data, which is the (measured) long-period data of that observation point. Based on the short-period data, methods such as neural networks are used to predict the long-period data. The error is calculated based on the measured long-period data as the desired output value. According to the overall loss, the model is trained using a backpropagation algorithm to learn the data patterns of the long-period observation point at various short-period sampling points (time points). This allows for the calculation of the data that the long-period observation point should have under short-period conditions and the prediction of future data (including data from both short-period and long-period sampling points).

[0029] For example, for any sensor observation point (e.g., point O) and any adjacent (related) satellite observation point (e.g., point A), acquire k (k≥1) long-period (time period) short-period data and long-period data to form k samples. The short-period data is the sensor data for that sensor observation point, which may or may not include sensor data at the corresponding long-period start (start time) or end (end time) (if any). For example, based on prediction requirements, if a long-period sampling point (which can be considered the long-period end) is also a short-period sampling point (a certain short-period end), the sample may include sensor data at the corresponding long-period start (if any) but exclude sensor data at the corresponding long-period end. The long-period data is the satellite remote sensing data for that satellite observation point at that long-period end.

[0030] Based on the above samples / data, a neural network regression model (or regression model, or model) is constructed with short-period data as input and long-period data as output. The actual long-period data is used as the target value (the desired output value, or the true value, or the label value). The model is iteratively trained using the backpropagation algorithm to obtain a trained model that meets the error requirements. The error, or loss function, can be set according to existing technology, such as mean squared error. When using multiple samples, the errors corresponding to each sample should be comprehensively considered, for example, the average or weighted average of the errors corresponding to each sample. The weight of later samples (errors) is assigned higher than that of earlier samples (errors). Based on the trained model, a regression curve of the ground subsidence (subsidence amount) at the satellite observation point over time can be obtained (where the data of each short-period sampling point on the regression curve is included) (see...). Figure 2 (Example), and the model can be used to predict ground subsidence at the satellite observation point.

[0031] The sample size k can be chosen based on the actual situation; for example, k = 5. When k > 1, it is preferable to use multiple consecutive long periods (see...). Figure 1 Example).

[0032] exist Figures 1-3 In the example shown: For Figure 1 The example uses data from 5 long periods (time periods) (including short-period data and long-period data). In the first k (e.g., k=5) long periods, the long-period data is used as a feedback signal. Through the backpropagation algorithm, the pattern of short-period data at the long-period observation point is learned, and the corresponding short-period data is predicted.

[0033] Specifically, using short-cycle data before the first long cycle as known values ​​and the first long cycle data as predicted values, the same applies to short-cycle data before the second long cycle, and so on. A regression model (using methods such as neural networks or SVR) is constructed based on all data from the first k long cycle points (including both long-cycle and short-cycle data). This regression model forms a regression curve that passes through all measured long-cycle data points. Based on this regression curve, all values ​​at the short-cycle time points can be obtained (values ​​calculated based on the regression model, such as...). Figure 2 Show).

[0034] Since the displacement of observation points involved in short-period and long-period data can be decomposed into two components, vertical displacement and horizontal displacement, in the scenario involved in this invention, only the vertical displacement can be retained as the ground subsidence, and these data can be processed into total data (dataset) according to the time series.

[0035] Typically, the long period is a multiple of the short period, or the duration of the short period and the sampling time (time point) of the short period data are set accordingly, so that the sampling time (end of the long period) of any long period data is also the sampling time (end of a certain short period) of the short period data.

[0036] For ease of analysis or prediction, the entire observation area can be divided into several closely spaced convex polygonal partitions (or sub-regions, or polygons) using existing technologies (e.g., Voronoi diagrams). These partitions are interconnected, without overlap or gaps, and each partition contains only one sensor observation point. The vertices of each partition (convex polygon) are satellite observation points (see [reference]). Figure 4 If there are vertices in the initially divided convex polygon that are not suitable for use as satellite observation points, their positions can be adjusted to make them usable as satellite observation points.

[0037] The number and distribution of ground sensors should be adapted to the zoning requirements and the accuracy requirements of regression models / prediction.

[0038] For computational convenience, polygon partitions can be treated as planar figures, that is, only their planar coordinates are considered.

[0039] Regression models for each partition vertex can be established (constructed and trained) based on short-period data from sensor observation points within the partition and long-period data from partition vertices (polygon vertices).

[0040] Since a vertex of one partition may simultaneously be a vertex of multiple partitions, a regression model for that partition's vertices can be built using sensor observation data from only one partition, based on predefined rules or random selection, without needing to perform repeated calculations on common vertices of multiple polygons. For example, starting from the lower left corner of the entire observation area (the lower left corner of the area image in the planar coordinate system), all sensor observation points are traversed upwards and to the right. For any given sensor observation point, a regression model is built for all vertices in its partition that have not yet been modeled, based on the short-period data of that sensor observation point.

[0041] See Figure 5 For any partition, a Delaunay triangulation can be constructed using existing techniques (e.g., Lawson algorithm or Bowyer-Watson algorithm), or in other words, the convex polygon can be divided into several tessellated triangles (Delaunay triangles, or triangular regions), the vertices of which are all vertices of the polygon.

[0042] For any point P in any triangle at any time point (e.g., a future short-period sampling point or a long-period sampling point), the predicted value (calculated value) of point P is calculated by weighted interpolation based on the predicted values ​​of each vertex of the triangle (calculated values ​​based on the corresponding regression model).

[0043] See Figure 6 Without loss of generality, let the vertices of the triangle be ( Figure 6 The coordinates of the three points (A, B, and C) are (x, y). a ,y a ), (x b ,y b ), (x c ,y c The coordinates of any point P in the triangle are (x, y). p ,y p ),

[0044] Find the weights (or weight coefficients) w a ,w b ,w c , so that:

[0045] x p =w a ×x a +w b ×x b +w c×x c ,

[0046] y p =w a ×y a +w b ×y b +w c ×y c ,

[0047] and

[0048] w a +w b +w c =1,

[0049] Solution:

[0050]

[0051]

[0052] w c =1-w a -w b ,

[0053] If v a ,v b ,v c Let A, B, and C be the predicted values ​​of the three vertices of the triangle (values ​​calculated based on their respective regression models). Then, the predicted value for a point P inside the triangle is:

[0054] v p =w a ×v a +w b ×v b +w c ×v c ,

[0055] Using the above method, for all time points (or sampling points) t of the short period i With i = 1, 2, 3, ..., N, the predicted values ​​of all points within each triangular region can be obtained, and then, in the time dimension, the predicted values / predicted value surfaces for the entire observation region or any part thereof in the short period can be given.

[0056] For points on the common side (shared edge) of multiple (two) triangles, the data of that point can be calculated using only the vertex data of one triangle according to the set rules or by random selection, without repeating the calculation.

[0057] As time accumulates, at any data point corresponding to a long period (the time point where the long-period data is located can be called a long-period sampling point) t k(k=1,2,3,…,K), the long-period data of long-period observation points A, B, C, D, and E (forming the vertices of the corresponding convex polygons) constitute time series data. Point A at t k The predicted value for the next time step (the next long-period sampling point) can be obtained from the short-period observation point O (located within the corresponding convex polygon) at time t. k The observations at time t1 and the observations at points B, C, D, and E (the remaining vertices of the convex polygon) (long-period data for each point, e.g., in the long period t1-t2). K Using long-period data as features, predictions are made using methods such as LSTM, and the prediction results are compared with t. K+1 The mean squared error (MSE) of the observed values ​​is used as the prediction evaluation index, thus enabling long-term predictions. Similarly, t... K Using time point O and the values ​​of points A, B, C, D, and E as features, short-period data for point O (predicted values ​​at short-period sampling points) can be predicted, thus constituting an interactive iterative method. This interactive iterative method allows the model to maximize the accuracy of predictions during the prediction process.

[0058] Unless otherwise specified, the preferred and optional technical means disclosed in this invention can be arbitrarily combined to form several different specific embodiments when one preferred or optional technical means is a further limitation of another technical means.

Claims

1. A fusion method based on the interactive iteration of sensor data and satellite data, characterized in that... The process involves acquiring sensor data from one or more long-term sensor observation points and satellite data from satellite observation points. The sensor data consists of short-term data from ground sensors, while the satellite data consists of long-term data from remote sensing satellites. The satellite observation points are adjacent satellite observation points of the relevant sensor observation points. A sample dataset is formed by combining short-term and long-term data within the same long-term period. A neural network regression model is constructed with short-term data as input and long-term data as output. The actual long-term data is used as the desired output value. The model is iteratively trained using the backpropagation algorithm to obtain a trained model that meets the error requirements. This trained model is then used as the prediction model for the satellite observation point. The sensor data and satellite data represent ground subsidence data.

2. The fusion method based on the interactive iteration of sensor data and satellite data as described in claim 1, characterized in that... Longer periods are integer multiples of shorter periods.

3. The fusion method based on the interactive iteration of sensor data and satellite data as described in claim 1, characterized in that... The so-called prediction model is a model used for ground subsidence analysis and / or prediction.

4. The fusion method based on the interactive iteration of sensor data and satellite data as described in claim 3, characterized in that... Only the vertical displacement data of the ground surface is used as the ground settlement data.

5. The fusion method based on the interactive iteration of sensor data and satellite data as described in any one of claims 1-4, characterized in that... The observation area is divided into several closely spaced zones with convex polygonal edges. Each zone contains only one sensor observation point, and the vertices of each zone are satellite observation points.

6. The fusion method based on the interactive iteration of sensor data and satellite data as described in claim 5, characterized in that... The number and distribution of ground sensors are determined based on zoning requirements and prediction accuracy requirements.

7. The fusion method based on the interactive iteration of sensor data and satellite data as described in claim 5, characterized in that... For any vertex in any partition, a prediction model for the vertex is established using long-period data of the vertex and short-period data of sensor observation points in any partition in which the vertex is located. Based on the prediction model, the prediction data for the corresponding vertex is calculated.

8. The fusion method based on the interactive iteration of sensor data and satellite data as described in claim 7, characterized in that... The partition is divided into several closely spaced triangular regions. The vertices of the triangles in each region of the same partition are the vertices of the convex polygons of that partition. Based on the predicted data of the triangle vertices, the predicted data of the points in the triangular regions other than the vertices are calculated by interpolation.

9. A fusion system based on the interactive iteration of sensor data and satellite data, comprising a satellite remote sensing system and several ground sensors, wherein the satellite remote sensing system is used to generate long-period data on ground subsidence involving the satellite observation area, and the ground sensors are set in the satellite observation area to generate short-period data on ground subsidence involving their respective sensor observation points, characterized in that... It also includes a neural network system / platform for implementing the fusion method based on the interactive iteration of sensor data and satellite data as described in any one of claims 1-8, wherein the satellite observation point is a point in the satellite observation area.

10. The fusion system based on the interactive iteration of sensor data and satellite data as described in claim 9, characterized in that... The neural network system / platform is connected to an operating terminal for human-computer interaction, accepts manual input from the operating terminal, and sends the analysis / prediction results to the operating terminal for display.

Images