A multi-sensing information fusion energy station digital twin space-time registration method and system

Abstract

The application discloses a kind of multi-sensing information fusion's energy station digital twin space-time registration method, comprising: obtaining multi-modal sensing information, the multi-modal sensing information is preprocessed, and the multi-modal sensing information is synchronized in time using time registration algorithm;Using frame insertion method to predict time series data, obtain predicted time data points, to accelerate the evolution reasoning of digital virtual space;Fusion space coordinates and information, coordinate transformation is carried out, and the multi-modal sensing information is synchronized in space using three-position registration method;According to the multi-modal sensing information after time and space registration, physical modeling is carried out, and the predicted time data points are used to construct prediction model, obtain the digital twin model of energy station, applied to energy station operation and maintenance, realize the efficient energy station management and control.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data fusion, in particular to a multi-sensor information fusion energy station digital twin space-time registration method and system. BACKGROUND

[0002] Digital twin is a technology that simulates and reflects real objects or systems in real time through digital models. It combines physical entities with their digital representations to achieve real-time monitoring, prediction and optimization of physical entities, helping people better understand and manage the behavior of entities or systems. Through digital twin, virtual testing, optimization and prediction can be performed to reduce risks and costs in actual operations and improve work efficiency and decision-making quality, providing a powerful tool and support for the management and decision-making of entities and systems. In urban planning, digital twin can simulate urban traffic flow, energy consumption, etc. to support decision-making for urban planning and sustainable development. In the field of energy management, digital twin can help energy stations with unit control, decision support, operation and maintenance, etc.

[0003] Energy station operation and maintenance need to monitor and analyze multi-modal sensor measurement information, but these sensors usually provide different information levels and space-time granularities, which is not conducive to data integrity, information analysis, fault diagnosis and model accuracy. Therefore, it is necessary to perform space-time registration on these multi-modal sensor information to eliminate data inconsistency, analyze comprehensively in the same space-time framework, and detect data anomalies and predict faults in advance. A multi-sensor information fusion energy station digital twin space-time registration method is proposed to achieve high-precision space-time registration of multi-modal sensors, and to construct a highly simulated digital twin model based on the fusion processing. Excellent model performance can provide better support and decision-making basis for energy station operation and management. The method uses efficient time registration and space registration algorithms to improve the accuracy of space-time synchronization, and uses deep neural networks with excellent matching performance to capture complex correlations between data and achieve multi-sensor information fusion. SUMMARY

[0004] In view of the above problems, the present application is proposed.

[0005] Therefore, the technical problem solved by the present application is that the present application solves the problem that energy station operation and maintenance need to monitor and analyze multi-modal sensor measurement information, but these sensors usually provide different information levels and space-time granularities, which destroys data integrity, information analysis, fault diagnosis and model accuracy.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a multi-sensor information fusion energy station digital twin space-time registration method, comprising:

[0008] acquiring multi-modal sensing information, pre-processing the multi-modal sensing information, synchronizing the multi-modal sensing information in time by using a time registration algorithm;

[0009] predicting the time series data by using an interpolation method to obtain predicted time data points, so as to accelerate the evolution reasoning of the digital virtual space;

[0010] fusing the spatial coordinates and information, performing coordinate transformation, and synchronizing the multi-modal sensing information in space by using a three-dimensional registration method;

[0011] performing physical modeling according to the multi-modal sensing information after time and space registration, and constructing a prediction model through the predicted time data points to obtain a digital twin model of the energy station.

[0012] As a preferred scheme of the multi-sensing information fusion energy station digital twin space-time registration method, the time registration algorithm includes,

[0013] a time registration algorithm combining the Lagrange interpolation method and the Kalman filter, initializing the Kalman filter, and calculating a one-step prediction matrix of the state and the covariance according to the high-frequency sensor measurement value;

[0014] the state estimation at the kth moment is the state transition matrix at the kth moment multiplied by the state estimation at the (k-1)th moment plus the control input matrix at the kth moment multiplied by the control input at the kth moment;

[0015] the state covariance matrix at the kth moment is the state transition matrix at the kth moment multiplied by the state covariance matrix at the (k-1)th moment multiplied by the transpose of the state transition matrix plus the process noise covariance matrix.

[0016] As a preferred scheme of the multi-sensing information fusion energy station digital twin space-time registration method, the time registration algorithm includes,

[0017] calculating a gain matrix of the filter, updating the state equation and the covariance equation to obtain a filtered value and time data, and being expressed as,

[0018]

[0019]

[0020] P k =(I-K k ·H k )·P k

[0021] wherein, Kk is a Kalman gain, H k is a measurement matrix, R k is a measurement noise covariance matrix, z k is a measurement value at the kth moment, and I is an identity matrix.

[0022] As a preferred scheme of the multi-sensor information fusion energy station digital twin space-time registration method, the method further comprises the following steps:

[0023] According to the sampling frequency of the low-frequency sensor, the moment that needs to be registered is selected, and three corresponding measurement moments t k-1 ,t k ,t k+1 are selected from the measurement moments of the high-frequency sensor.

[0024] The filter estimation values corresponding to the selected three measurement moments are substituted into the Lagrange interpolation formula for registration, and the time registration of the sensor data of different frequencies is completed until the time-consistent sensor measurement information is obtained.

[0025] As a preferred scheme of the multi-sensor information fusion energy station digital twin space-time registration method, the method further comprises the following steps:

[0026] The registered sensor data is input into the network model input layer for data prediction, and a sliding window is used to convert the time series data into a two-dimensional input form. The original N-step one-dimensional time series can obtain M input samples similar to images through the sliding window. For the i-th sliding window sample, the corresponding data is represented as X i =[x (i-1)S+1 ,x (i -1)S+2,...,x (i-1)S+T ], wherein S is the step size of the sliding window, and T is the size of the sliding window.

[0027] A two-dimensional convolution operation is performed, which is represented as wherein Y[i] is the output after convolution, X is the input data, and K is the convolution kernel.

[0028] A ReLU nonlinear activation function is applied to the result after convolution to introduce a nonlinear transformation.

[0029] As a preferred scheme of the multi-sensor information fusion energy station digital twin space-time registration method, the method further comprises the following steps:

[0030] After the nonlinear transformation of the result after convolution, a max-pooling operation is performed, which is represented as ​

[0031] Y[i,j] = max m,n (X[i x s + m, j x s + n])

[0032] Wherein Y is the value at (i,j) in the output matrix, X is the input matrix, S is the size of the pooling window, m and n are the indices within the window;

[0033] The feature map is flattened and input into a fully connected layer for prediction, the output of the fully connected layer is the input multiplied by a weight matrix plus a bias term, and the prediction result is output in the output layer;

[0034] The Adam optimizer is used for parameter updating, the mean square error is used as the loss function, the difference between the prediction result and the actual value is compared, and the model is trained to perform accelerated evolution reasoning of the sensor information in the virtual space.

[0035] As a preferred scheme of the multi-sensor information fusion energy station digital twin space-time registration method, wherein: the space coordinates and information are fused, coordinate transformation is performed, and the three-dimensional registration method is used to synchronize the multi-modal sensor information in space,

[0036] The three-dimensional registration method includes the combination of generalized least squares and Kalman filtering;

[0037] Let A=(X a ,Y a ,Z a ), B=(X b ,Y b ,Z b ) be the three-dimensional rectangular coordinates of sensors a, b respectively, a ′ =(x ′ a ,y a ′ ,z ′ a ), b ′ =(x ′ b ,y b ′ ,z b ′ ) be the error data of the local coordinate system of sensors a, b respectively, let the same target coordinates measured by the two sensors be equal, and the equation is represented as,

[0038] A+T a ·a ′ =B+T b ·b ′

[0039] Wherein T a , T bRotation matrix corresponding to sensor a, b respectively;

[0040] First order expansion of the equation and expressed in matrix form, expressed as,

[0041] X k = H k × β k + F k × ω k

[0042] X k = A-B

[0043] Wherein, H k System error partial derivative matrix, beta k System error, F k Random error partial derivative matrix, omega k Random error;

[0044] The covariance matrix is initialized by using the least square method, the system error is estimated by applying the linear Kalman filter, and the registered sensor information on the spatial scale is obtained.

[0045] In a second aspect, the present application provides a multi-sensor information fusion energy station digital twin space-time registration system, comprising:

[0046] A time synchronization module is configured to obtain multi-modal sensor information, pre-process the multi-modal sensor information, and synchronize the multi-modal sensor information in time by using a time registration algorithm.

[0047] A data prediction module is configured to realize time series data prediction by using an interpolation method to obtain predicted time data points, so as to accelerate the evolution reasoning of the digital virtual space.

[0048] A space synchronization module is configured to fuse spatial coordinates and information, perform coordinate transformation, and synchronize the multi-modal sensor information in space by using a three-position registration method.

[0049] A model construction module is configured to construct a digital twin model of the energy station according to the multi-modal sensor information registered in time and space.

[0050] In a third aspect, the present application provides a computing device, comprising:

[0051] A memory and a processor;

[0052] The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, so as to realize the steps of the multi-sensor information fusion energy station digital twin space-time registration method.

[0053] In a fourth aspect, the present application provides a computer readable storage medium storing computer executable instructions, which, when executed by a processor, implement the steps of the multi-sensor information fusion energy station digital twin space-time registration method.

[0054] The multi-sensor information fusion energy station digital twin space-time registration method can greatly improve the time consistency and spatial consistency of multi-sensor information, and through the establishment of a mathematical dynamic model, the establishment and training of a prediction model, and other means, a high-precision, space-time synchronous digital twin model is constructed and applied to energy station operation and maintenance, achieving efficient energy station management and control. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Among them:

[0056] Figure 1 The present application provides a multi-sensor information fusion energy station digital twin space-time registration method. DETAILED DESCRIPTION

[0057] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0058] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the scope of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0059] Secondly, the "one embodiment" or "embodiment" referred to herein can include specific features, structures or characteristics contained in at least one implementation of the present application. In this specification, "in one embodiment" appearing in different places does not mean the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0060] The application is described in detail in combination with the schematic diagram, and in the detailed description of the embodiments of the application, the cross-sectional view of the device structure is partially enlarged without the general proportion for the convenience of illustration, and the schematic diagram is only an example, which should not limit the scope of protection of the application herein. In addition, the three-dimensional spatial dimensions of length, width and depth should be included in actual production.

[0061] Meanwhile, in the description of the application, it should be noted that the terms "upper, lower, inner and outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application. In addition, the terms "first, second or third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0062] Unless otherwise specifically defined and limited in the application, the terms "mounting, connecting, connecting" should be broadly understood, for example: it can be fixedly connected, detachably connected or integrally connected; it can also be mechanically connected, electrically connected or directly connected; it can also be indirectly connected through an intermediate medium; it can also be the communication between the two elements. For those skilled in the art, the specific meaning of the above terms in the application can be understood according to the specific circumstances.

[0063] Embodiment 1

[0064] Reference Figure 1 For an embodiment of the application, a multi-sensor information fusion energy station digital twin space-time registration method is provided, comprising:

[0065] S100: acquiring multi-modal sensor information, preprocessing the multi-modal sensor information, and synchronizing the multi-modal sensor information in time using a time registration algorithm;

[0066] Further, a time registration algorithm combining Lagrange interpolation method and Kalman filter is adopted, the Kalman filter is initialized, and the one-step prediction matrix of state and covariance is calculated according to the high-frequency sensor measurement value;

[0067] The state estimation at the kth moment is the state transition matrix at the kth moment multiplied by the state estimation at the k-1th moment plus the control input matrix at the kth moment multiplied by the control input at the kth moment;

[0068] The state covariance matrix at the kth moment is the state transition matrix at the kth moment multiplied by the state covariance matrix at the k-1th moment multiplied by the transpose of the state transition matrix plus the process noise covariance matrix.

[0069] The gain matrix of the filter is calculated, the state equation and the covariance equation are updated, and the filtered value and the time data are obtained, represented as

[0070]

[0071]

[0072] P k =(IK k ·H k )·P k

[0073] Among them, K k For Kalman gain, H k For the measurement matrix, R k To measure the noise covariance matrix, z k Let I be the measurement value at time k, and let I be the identity matrix.

[0074] Based on the sampling frequency of the low-frequency sensor, select the time to be registered, and choose the corresponding three measurement times t from the measurement times of the high-frequency sensor. k-1 ,t k ,t k+1 ;

[0075] The filtered estimates corresponding to the three selected measurement times are... The data is then substituted into the Lagrange interpolation formula for registration until the time registration of sensor data at different frequencies is completed, thus obtaining sensor measurement information with consistent time.

[0076] It should be noted that, compared to other algorithms, Lagrange interpolation has advantages such as estimating other location data using only known data points, traversing all known data points, being unaffected by data point spacing, and producing smooth interpolation results. However, the registration accuracy of this method is significantly affected by measurement noise, exhibiting rapid performance degradation under conditions of high measurement noise. Considering the good performance of Kalman filters in filtering random errors in practical engineering, this method adopts a time registration algorithm combining Lagrange interpolation and Kalman filters, employing a three-point Lagrange interpolation method and a three-point start-up Kalman filter, assuming that sensor data with a high sampling frequency is registered to sensor data with a low sampling frequency.

[0077] The initialization of the three-point filter and error covariance matrix, the state prediction equation and covariance prediction equation are expressed as follows:

[0078]

[0079]

[0080] in, For the state estimation at time k, F k Let B be the state transition matrix. ku is a control input matrix k P is a control input k Q is a state covariance matrix k is a process noise covariance matrix.

[0081] On this basis, the gain matrix of the filter is calculated, and the selected three-point measurement time corresponding to the filter estimation value is brought into the Lagrange formula for registration, and the Lagrange three-point interpolation formula is expressed as,

[0082]

[0083] Where, t k-1 ,t k ,t k+1 is the selected corresponding three-point measurement time, and P(t) is the estimated function value of any point between the three points.

[0084] Specifically, multi-modal sensing information is collected and preprocessed by data cleaning, which includes two parts. The first part is data filtering, that is, some redundant and abnormal information attributes are removed, and only relevant attributes with research value for subsequent processing and analysis are left. In this part, a Gaussian filter is used to weight average the data using a Gaussian function to smooth the data and remove noise. The second part is digitalization of measurement information, that is, general attribute information is converted into mathematical model information for operation. In this part, the measurement information is converted into a mathematical model through an analog-to-digital converter and a digital filter to perform registration and other operations.

[0085] It should also be noted that a large number of sensors are arranged in the energy station to collect necessary information, which involves various types of sensors such as pressure sensors, temperature sensors, flow sensors, etc. The data collected by these sensors may have problems such as noise, abnormality, redundancy, missing, etc. The above problems will damage the readability and uniformity of the data, and may also increase the cost and complexity of subsequent data processing and calculation, thereby affecting the analysis and application of the data, and increasing the modeling error. The data cleaning operation on the sensor measurement information can improve the usability and reliability of the data, and ensure the quality of the energy station operation data and the performance of the digital system.

[0086] S200: predicting the time series data by using the interpolation method to obtain the predicted time data points, so as to accelerate the evolution reasoning of the digital virtual space;

[0087] Further, the registered sensor data is input into the network model input layer for data prediction, and a sliding window is used to convert the time series data into a two-dimensional input form. The original N-step one-dimensional time series can obtain M input samples similar to images through the sliding window. For the i-th sliding window sample, the corresponding data is expressed as, i X(i-1)S+1 x (i-1)S+2 ,...,x (i-1)S+T ], where S is the step size of the sliding window and T is the size of the sliding window.

[0088] The two-dimensional convolution operation is represented as, where Y[i] is the output after convolution, X is the input data, and K is the convolution kernel.

[0089] The ReLU nonlinear activation function is applied to the result of convolution to introduce a nonlinear transformation.

[0090] After the nonlinear transformation of the result of convolution, the max-pooling operation is performed, represented as,

[0091] Y[i,j] = max m,n (X[i×s+m,j×s+n])

[0092] where Y is the value at (i,j) in the output matrix, X is the input matrix, S is the size of the pooling window, and m and n are the indices within the window.

[0093] The feature map is flattened and input into the fully connected layer for prediction. The output of the fully connected layer is the input multiplied by the weight matrix plus the bias term, and the prediction result is output in the output layer.

[0094] The Adam optimizer is used to update the parameters, and the mean square error is used as the loss function to compare the difference between the prediction result and the actual value. Through training the model, the acceleration evolution reasoning of the sensor information in the virtual space is performed.

[0095] It should be noted that interpolation refers to estimating the intermediate values between two known data points to obtain a smoother curve or denser data points. In time series analysis, interpolation is used to fill in missing data points or to smooth the data. In time series prediction, interpolation can make the time series continuous and easier to model and predict. Through interpolation technology, future time data points can be inferred from known time series data points for use in subsequent prediction models.

[0096] It should also be noted that the convolutional neural network (CNN) can learn the spatial and temporal patterns in the data through multiple convolutional layers and pooling layers, gradually extract the abstract feature representation of the data, and capture the important evolution features in the data. CNN captures complex relationships in time series data by introducing non-linear relationships in the activation function, and better handles non-linear and non-stationary time series data. Because CNN has translational invariance, that is, it is not affected by the specific position when identifying data patterns, it can effectively identify the same type of situation at different time points on the time series data, and be used to predict the value or trend at future time.

[0097] The obtained registered sensor data is input into the network model input layer for data prediction. The CNN used in this scheme consists of an input layer, three convolutional layers, three maximum pooling layers, a fully connected layer, and an output layer. The convolutional layer and the fully connected layer have many neurons, which learn the nonlinear feature representation of the input data by passing information between layers.

[0098] The time series data is converted into a two-dimensional input form using the sliding window method, then a two-dimensional convolution operation is performed, and a ReLU nonlinear activation function is applied to the convolution result to introduce a nonlinear transformation, represented as ReLU(x)=max(0,x); Then perform a maximum pooling operation to reduce the data volume and retain important features. After three convolution and maximum pooling operations, respectively, the feature map is flattened and input into the fully connected layer for final prediction, represented as Y=XW+b, where Y is the output of the fully connected layer, X is the input, W is the weight matrix, and b is the bias term.

[0099] Finally, a one-dimensional prediction result is output in the output layer, and the Adam (Adaptive Moment Estimation) optimizer is used. Adam adjusts the parameter learning rate by calculating the first and second moment estimates of the gradient, so that the neural network more effectively updates the parameters during the training process. The parameter update is represented as

[0100]

[0101] where θ is the parameter value, η is the learning rate, is a constant, is the second moment estimate correction bias, is the first moment estimate correction bias;

[0102] The mean square error is used as the loss function to measure the difference between the predicted value and the actual value, represented as

[0103]

[0104] where n is the number of samples, y i is the actual value of the i-th sample, is the predicted value of the i-th sample.

[0105] Model other parameter settings: batch size is 32, the number of training iterations is 50, the initial learning rate is 0.001. Through the trained CNN, the future data trend can be predicted from the known time series data, and the acceleration evolution reasoning of the sensor information in the virtual space is realized.

[0106] S300: fuse the space coordinates and information, perform coordinate transformation, and use a three-position registration method to synchronize the multi-modal sensor information in space;

[0107] Further, the three-position registration method includes a combination of generalized least squares and Kalman filtering.

[0108] Let A = (X a ,Y a ,Z a ), B = (X b ,Y b ,Z b ) be the three-dimensional rectangular coordinates of sensors a, b, respectively, a ′ = (x ′ a ,y a ′ ,z ′ a ), b ′ = (x ′ b ,y b ′ ,z b ′ ) be the error data of the local coordinate system of sensors a, b, respectively, let the same target coordinates measured by the two sensors be equal, and the equation is represented as,

[0109] A + T a ·a ′ = B + T b ·b ′

[0110] where T a , T b are the rotation matrices corresponding to sensors a, b, respectively.

[0111] First-order expansion of the equation and represented in matrix form, represented as,

[0112] X k = H k ×β k +F k ×ωk

[0113] X k = A - B

[0114] where H k is the system error partial derivative matrix, β k is the system bias, F k is the random error partial derivative matrix, ω k is the random error;

[0115] The least square method is used to initialize the covariance matrix, the linear Kalman filter is applied to estimate the system error, and the registered sensor information on the spatial scale is obtained.

[0116] It should be noted that the fusion of digital virtual space coordinates, real space coordinates, BIM information and GIS information may have differences in spatial position due to different sensors, devices or data sources. Through coordinate transformation, these information can be integrated into a unified coordinate system to achieve data consistency and integrity, thereby better information fusion and analysis. Therefore, before fusing multiple spatial information, coordinate transformation is needed to unify the coordinate system. In BIM, the geographic coordinate system plays an important role, mainly used to connect the building model with GIS data, to realize the accurate positioning and display of the building model in geographic space.

[0117] The conversion relationship between geographic coordinates and three-dimensional rectangular coordinates is as follows. Assuming that the earth is an ellipsoid with a major axis of a and a minor axis of b. Given a geographic coordinate (φ, λ, h), where φ is the latitude, λ is the longitude, and h is the altitude. The geographic coordinate is converted to the coordinate (X, Y, Z) in the three-dimensional rectangular coordinate system by the formula, where X is the east-west direction, Y is the south-north direction, and Z is the altitude, represented as,

[0118]

[0119] X = (N + h) · cos(φ) · cos(λ)

[0120] Y = (N + h) · cos(φ) · sin(λ)

[0121]

[0122] where N is the radius of curvature of the prime vertical circle, e 2 is the square of the first eccentricity of the ellipsoid;

[0123] The local coordinates and three-dimensional rectangular coordinates of the target measured by the sensor are converted by the formula. The conversion relationship between the local coordinates and the three-dimensional rectangular coordinates is as follows. Assuming that the measurement of the sensor in the local coordinate system is (r, θ, η), where r is the slant range, θ is the azimuth angle, and η is the elevation angle, represented as,

[0124] x = r*sin(theta)*cos(phi)

[0125] y = r*cos(theta)*cos(phi)

[0126] z = r*sin(phi)

[0127] (X,Y,Z) = (x,y,z)*T

[0128] where (x,y,z) is local coordinate, (X,Y,Z) is three-dimensional rectangular coordinate, and T is rotation matrix;

[0129] Three-dimensional registration using least squares and Kalman filtering, assuming that the system error is constant, the random error is Gaussian white noise, using least squares to initialize the covariance matrix, applying linear Kalman filter to estimate the system error, expressed as,

[0130]

[0131] P k = P k-1

[0132]

[0133]

[0134] where, is the system error prediction, P k is the prediction covariance, K k is the filter gain, R k is the noise covariance;

[0135] The space registration process based on non-cooperative targets is realized, and the registered sensing information on the space scale is obtained, and then the key node and device coordinates can be obtained.

[0136] S400: According to the multi-modal sensing information after time and space registration, physical modeling is carried out, and a prediction model is constructed through the predicted time data points to obtain a digital twin model of the energy station;

[0137] It should be noted that in the process of physical modeling, the water source heat pump energy station is taken as an example. For the physical process of the water source heat pump energy station, a mathematical model based on physical principles is established, including thermodynamic equations and energy balance equations; considering the energy and mass transfer processes between various components in the system, such as centrifugal refrigeration main machine, electrode boiler, circulating water pump, and pipe network equipment, and the influence of external environmental conditions on the system; according to the structure and parameters of the system, a dynamic model is established to describe the time-varying behavior of the system.

[0138] In the machine learning modeling process, based on the data features, select the appropriate machine learning algorithm to establish the prediction model, such as regression model, neural network, support vector machine; use the historical data to train the model, so that it can learn the complex relationship and dynamic characteristics of the system.

[0139] Finally, combine the physical model and the machine learning model to form a digital twin model. By inputting the actual running data and the time and space registered data into the digital twin model, real-time monitoring, prediction and optimization of the system state and performance are realized, and through the secondary registration of real-time measurement information and the digital twin model, the acceleration evolution of the digital model on the time scale is realized, which assists the early warning decision; the accurate mapping of virtual and real spaces on the spatial scale is realized, which assists the operation and maintenance.

[0140] In an implementable manner, step S400 can be placed before step S200.

[0141] The above is a schematic scheme of the multi-sensor information fusion energy station digital twin space-time registration method of the embodiment. It should be noted that the technical scheme of the multi-sensor information fusion energy station digital twin space-time registration device belongs to the same concept as the technical scheme of the multi-sensor information fusion energy station digital twin space-time registration method described above. The technical scheme of the multi-sensor information fusion energy station digital twin space-time registration device in this embodiment is not described in detail, and can be referred to the description of the technical scheme of the multi-sensor information fusion energy station digital twin space-time registration method.

[0142] The multi-sensor information fusion energy station digital twin space-time registration device in this embodiment comprises:

[0143] The time synchronization module is configured to obtain multi-modal sensor information, pre-process the multi-modal sensor information, and synchronize the multi-modal sensor information in time using a time registration algorithm.

[0144] The data prediction module is configured to use the interpolation method to realize time series data prediction and obtain predicted time data points to accelerate the evolution reasoning of the digital virtual space.

[0145] The space synchronization module is configured to fuse spatial coordinates and information, perform coordinate transformation, and synchronize the multi-modal sensor information in space using a three-dimensional registration method.

[0146] The model construction module is configured to construct a digital twin model of the energy station according to the multi-modal sensor information registered in time and space.

[0147] The embodiment also provides a computing device suitable for multi-sensor information fusion energy station digital twin space-time registration, comprising:

[0148] The memory is configured to store computer-executable instructions, and the processor is configured to execute the computer-executable instructions to implement the energy station digital twin space-time registration method for multi-sensor information fusion as proposed in the above embodiments.

[0149] The embodiment also provides a storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the energy station digital twin space-time registration method for multi-sensor information fusion as proposed in the above embodiments.

[0150] The storage medium proposed in the embodiment belongs to the same inventive concept as the energy station digital twin space-time registration method for multi-sensor information fusion proposed in the above embodiments, and the technical details not described in the embodiment can be referred to the above embodiments, and the embodiment has the same beneficial effects as the above embodiments.

[0151] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and necessary universal hardware, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH memory, a hard disk, or an optical disc, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods of various embodiments of the present application.

[0152] Embodiment 2

[0153] Referring to Table 1, for an embodiment of the present application, an energy station digital twin space-time registration method for multi-sensor information fusion is provided, which is used to verify and illustrate the technical effects in the method.

[0154] Taking a water source heat pump energy station as a specific scene, the Monte Carlo method is used, that is, the statistical sampling theory is used to approximately solve the actual problem,

[0155] The sampling periods of the high and low frequency sensors in time registration are 4s and 7s respectively, the noise is Gaussian white noise, the total sampling time is 200s, there are three types of temperature, water pressure and flow, and the simulation times are 30. In data prediction, the first 160s of data is the training set, the last 20s of data is the test set, and the prediction accuracy is evaluated. In spatial registration, the radial system deviation, azimuth system deviation and pitch system deviation of the two sensors are the same, the sampling period is 1s, the sampling times are 4000, the simulation times are 30, and the geographical coordinates of the two sensors are different. The overall registration accuracy is obtained by weighting the simulation results.

[0156] Table 1 Comparison of performance effects of traditional method and method of the application

[0157]

[0158] From the above table, the root mean square error (RMSE) of the temperature sensor data of the method of the application is 0.23℃, while the RMSE of the traditional method is 0.98℃. The method of the application performs more accurately in the registration of temperature sensor data, with lower error.

[0159] The method of the application also achieves significant improvement in the registration of water pressure sensor data and flow sensor data. The RMSE of the water pressure sensor data is reduced from 8720N / m 3 to 2037N / m 3 , and the RMSE of the flow sensor data is reduced from 795m 3 / s to 201m 3 / s. This means that the method of the application can more accurately estimate and predict sensor data for time registration tasks.

[0160] The method of the application also performs well in data prediction, with the prediction accuracy improved from 92.3% of the traditional method to 95.1%, indicating that the method of the application can more accurately predict future data values, improving the reliability of data prediction.

[0161] The method of the application also achieves significant improvement in azimuth system error estimation and pitch system error estimation. The number of convergence sampling points is reduced from 1550 and 900 of the traditional method to 600 and 300 respectively, indicating that the method of the application can more efficiently perform spatial registration, reducing the required number of sampling points.

[0162] In overall registration, the method of the application achieves higher accuracy. The accuracy is improved from 83% of the traditional method to 94%, meaning that the method of the application can more accurately integrate and match data at different registration levels, improving the quality of overall registration.

[0163] The proposed manner exhibits significant advantages in different registration levels, including lower data error, more accurate data prediction, higher efficient spatial registration, and higher overall registration accuracy, which can help improve sensor data processing and registration tasks, and improve the accuracy and reliability of data.

[0164] It should be noted that the above examples are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the present application, and they should be covered in the scope of the claims of the present application.

Claims

1. A multi-sensing information fusion energy station digital twin space-time registration method, characterized in that, The method comprises the following steps: acquiring multi-modal sensing information, preprocessing the multi-modal sensing information, and synchronizing the multi-modal sensing information in time by using a time registration algorithm; predicting time series data by using an interpolation method to obtain predicted time data points to accelerate the evolution reasoning of the digital virtual space; fusing spatial coordinates and information, performing coordinate transformation, and synchronizing the multi-modal sensing information in space by using a three-dimensional registration method; performing physical modeling according to the multi-modal sensing information registered in time and space, and constructing a prediction model through the predicted time data points to obtain a digital twin model of the energy station; synchronizing the multi-modal sensing information in time by using a time registration algorithm comprises: a time registration algorithm combining Lagrange interpolation and Kalman filtering is used to initialize a Kalman filter, and a one-step prediction matrix of state and covariance is calculated according to the high-frequency sensor measurement value; the state estimation at the kth moment is the state transition matrix at the kth moment multiplied by the state estimation at the (k-1)th moment plus the control input matrix at the kth moment multiplied by the control input at the kth moment; the state covariance matrix at the kth moment is the state transition matrix at the kth moment multiplied by the state covariance matrix at the (k-1)th moment multiplied by the transpose of the state transition matrix plus the process noise covariance matrix; synchronizing the multi-modal sensing information in time by using a time registration algorithm comprises: a gain matrix of filtering is calculated, and a state equation and a covariance equation are updated to obtain a filtering value and time data, which are represented as: P k = (I - K k · H k ) · P k where K k is the Kalman gain, H k is the measurement matrix, R k is the measurement noise covariance matrix, z k is the measurement value at the kth time, and I is the identity matrix. further comprising: According to the sampling frequency of the low-frequency sensor, select the time point that needs to be registered, and select the corresponding three measurement time points t from the measurement time points of the high-frequency sensor k-1 ,t k ,t k+1 ; The selected three-point measurement time corresponding to the filter estimation value Substitute the Lagrange interpolation formula for registration until the sensor data of different frequencies are time-registered, and the time-consistent sensor measurement information is obtained.

2. The method of claim 1, wherein the method further comprises: time series data prediction is realized by using an interpolation method to obtain predicted time data points to accelerate the evolution reasoning of the digital virtual space, which comprises: The registered sensor data is input to the network model input layer for data prediction, a sliding window is used to convert the time series data into a two-dimensional input form, and the original N-step one-dimensional time series is obtained through the sliding window to obtain M input samples similar to images. For the i-th sliding window sample, the corresponding data is represented as X i =[x (i-1)S+1 ,x (i-1)S+2 ,...,x (i-1)S+T ], wherein S is the step size of the sliding window, and T is the size of the sliding window; A two-dimensional convolution operation is performed, denoted as, where Y[i] is the output after convolution, X is the input data, and K is the convolution kernel. a ReLU nonlinear activation function is applied to the convolution result to introduce a nonlinear transformation.

3. The method of claim 2, wherein the method further comprises: further comprising: after the nonlinear transformation of the convolution result, a max-pooling operation is performed, which is represented as: Y[i,j] = max m,n (X[i x s + m, j x s + n]) where Y is the value at (i,j) in the output matrix, X is the input matrix, S is the size of the pooling window, and m and n are the indices within the window; the feature map is flattened and input into a fully connected layer for prediction, and the output of the fully connected layer is the input multiplied by a weight matrix plus a bias term, and the prediction result is output in the output layer; the Adam optimizer is used for parameter updating, the mean square error is used as a loss function, the difference between the prediction result and the actual value is compared, the model is trained, and the accelerated evolution reasoning of the sensing information in the virtual space is performed.

4. The method of claim 3, wherein, fusing spatial coordinates and information, performing coordinate transformation, and synchronizing the multi-modal sensing information in space by using a three-dimensional registration method comprises: the three-dimensional registration method comprises a combination of generalized least squares and Kalman filtering; Let A = (X a ,Y a ,Z a ), B = (X b ,Y b ,Z b ) be the three-dimensional rectangular coordinates of sensors a, b, respectively, a ′ = (x ′ a ,y a ′ ,z ′ a ), b ′ = (x ′ b ,y b ′ ,z b ′ ) be the erroneous data of the local coordinate systems of sensors a, b, respectively, let the same target coordinate measured by the two sensors be equal, the equation is expressed as, A + T a • a ′ = B + T b • b ′ where T a , T b are the rotation matrices corresponding to the sensors a, b respectively; the first-order expansion of the equation is performed and represented in matrix form, which is represented as: X k = H k x β k + F k x ω k X k = A - B where H k the system error partial derivative matrix, β k is the system bias, F k is the random error partial derivative matrix, ω k is the random error; the covariance matrix is initialized by using the least squares method, the linear Kalman filter is used to estimate the system error, and the sensing information registered in the spatial scale is obtained.

5. A system of multi-sensor information fusion energy station digital twin space-time registration method, applying a multi-sensor information fusion energy station digital twin space-time registration method according to any one of claims 1-4, characterized in that, The method comprises the following steps: a time synchronization module is configured to acquire multi-modal sensing information, preprocess the multi-modal sensing information, and synchronize the multi-modal sensing information in time by using a time registration algorithm; a data prediction module is configured to realize time series data prediction by using an interpolation method to obtain predicted time data points to accelerate the evolution reasoning of the digital virtual space; The space synchronization module fuses the space coordinates and information, performs coordinate transformation, and synchronizes the multi-modal sensing information in space by using a three-position registration method. The model construction module is configured to construct a digital twin model of the energy station according to the multi-modal sensing information registered in time and space.

6. An electronic device, comprising: The device comprises: a processor; a memory for storing processor-executable instructions; the processor is configured to invoke the instructions stored in the memory to perform the method of any one of claims 1-4.

7. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by the processor, implement the method of any one of claims 1-4.

Images