An internet-of-things-oriented cross-scale time-series data fusion method and system

Abstract

The application provides a cross-scale time series data fusion method and system for the Internet of Things. The method comprises: collecting cross-scale time series data from multiple subsystems and multiple types of sensors; performing time window sliding processing on the cross-scale time series data, and the processed results include two types of data, i.e., equidistant multivariate time series and multivariate correlation relationship; performing modelized feature fusion on the two types of data, and synchronously, dynamically loading parameters and running according to a transaction processing cycle to obtain fused implicit feature data; and applying the fused implicit feature data to safety monitoring, system situation assessment and subsystem interlinkage operation decision-making after processing according to specific steps. The application can effectively solve the problems that a large amount of sensor data and subsystem data in some environment cannot be uniformly managed, fused, shared and logically linked and controlled between subsystems.

Description

TECHNICAL FIELD

[0001] The application relates to an Internet of Things-oriented cross-scale time series data fusion method and system, and belongs to the electronic information technology field. BACKGROUND

[0002] In recent years, with the development of semiconductor technology and Internet of Things technology, intelligent sensors are widely used in large buildings, transportation, power grids, petrochemical industries and other fields. These widely existing terminal information carriers are interconnected and interconnected through Ethernet, twisted pair, optical fiber, 4G, 5G and other communication methods.

[0003] At present, there are various intelligent sensors in the Internet of Things, and the communication protocols and interface standards adopted by the intelligent sensors are different. The cross-scale time series data from each sensor lacks unified storage and management, and the analysis and application of these time series data are limited to the internal independent (autonomous) subsystem. There is no data sharing between subsystems, so the entire Internet of Things system cannot realize collaborative operation.

[0004] Taking a large building as an example, there are a large number of subsystems inside the entire building, such as lighting systems, elevator systems, water supply systems, parking management systems, intrusion alarm systems, access control systems, power monitoring systems, and automatic fire alarm systems. There is no information interconnection and intercommunication between each subsystem, and there is no data fusion and sharing, so it is difficult to realize linkage and cooperation between systems. SUMMARY

[0005] The main purpose of the present application is to provide an Internet of Things-oriented cross-scale time series data fusion method and system, which can effectively solve the problem that a large number of sensor data and subsystem data in some environment cannot be uniformly managed, fused and shared, and the logical linkage control between subsystems cannot be realized.

[0006] In order to solve the above problems, the technical scheme adopted by the present application is as follows:

[0007] An Internet of Things-oriented cross-scale time series data fusion method, comprising the following steps:

[0008] Data acquisition: collecting cross-scale time series data from multiple subsystems and multiple types of sensors;

[0009] Sensor data fusion: performing time window sliding processing on the cross-scale time series data, and the processed results include two types of data, i.e., equidistant multivariate time series and multivariate correlation relationship;

[0010] Modeling feature fusion: performing modeling feature fusion on the two types of data, i.e., equidistant multivariate time series and multivariate correlation relationship, and synchronously, dynamically loading parameters and running with the transaction processing period to obtain fused implicit feature data;

[0011] Transaction processing: the fused implicit feature data is processed according to specific steps and applied to security monitoring, system situation assessment and inter-subsystem linkage operation decision.

[0012] A cross-scale time series data fusion system for the Internet of Things, which realizes all functions from data acquisition to transaction processing, mainly includes the following modules:

[0013] 1) Data acquisition module: used for collecting cross-scale time series data from various subsystems and various types of sensors, and sending the collected cross-scale time series data to the sensor data fusion module after denoising and filtering;

[0014] 2) Sensor data fusion module: used for performing time window sliding processing on the cross-scale time series data received from the data acquisition module, and providing the modelized feature fusion module with the processed results containing two types of data, i.e., equidistant multivariate time series and multivariate correlation relationship, through an internal interface;

[0015] 3) Modelized feature fusion module: used for modelizing the two types of data, i.e., equidistant multivariate time series and multivariate correlation relationship, processed by the sensor data fusion module, and synchronously, the modelized feature fusion module dynamically loads parameters and runs according to the transaction processing period, and provides the transaction processing module with fused implicit feature data through an internal interface;

[0016] 4) Transaction processing module: used for processing the implicit feature data provided by the modelized feature fusion module according to specific steps, and applying the processed data to security monitoring, system situation assessment and inter-subsystem linkage operation decision.

[0017] Further, the cross-scale time series data involved in the data acquisition module of the present application includes data of different time granularities (if millisecond, second, minute), different communication modes (such as Ethernet, twisted pair, wifi, 5G), different communication protocols (such as KNX, BACnet, OPC, ModBus, HTTP, 104, DDE, etc.), and different data types (floating point type, Boolean type, long integer type, string).

[0018] Further, the multivariate correlation relationship involved in the present application refers to the causal relationship between cross-scale time series data, and other relationships (including but not limited to spatial relationship, device topology relationship, etc.) between cross-scale time series data established by using a graph model.

[0019] Further, the time window sliding processing involved in the sensing data fusion module adopts time window to realize sensing data fusion in combination with the following two aspects: a) converting the cross-scale time series data into equal-interval time series data, i.e., equal-interval multivariate time series, by means of algebraic interpolation, machine learning fitting, spline function or numerical integration equivalent method; b) dynamically extracting the multivariate correlation between data in the set time window during system operation.

[0020] Further, the modeling feature fusion involved in the modeling feature fusion module includes the following three steps:

[0021] a) taking the historical data of the equal-interval multivariate time series received from the sensing data fusion module as the reference sample, and taking the current and subsequent time series data as the learning target, to establish a multivariate time series prediction model;

[0022] b) taking the attributes of the measured entities in the Internet of Things as nodes and the multivariate correlation as edges, to establish a multivariate feature graph model and mine potential correlation or correlation change;

[0023] c) saving the model parameters of the multivariate time series prediction model and the multivariate feature graph model as readable data files by means of a programmed automatic optimization solver and updating them regularly.

[0024] Further, the dynamic loading parameters involved in the modeling feature fusion module include the following four types of parameters:

[0025] a) the time attribute of the data sampling time in the period set for the time series signal, according to the characteristics of the application scene, the phase in the period or the following (time) position coding formula expression:

[0026]

[0027] where i represents the sampling time, d is a coefficient set in the model, and T represents the period set for the time series signal.

[0028] The dynamic updating method of the parameter is to update the proportion i / T of the sampling time i relative to the starting position of the period division with the system running.

[0029] b) the loading data of the modeling feature fusion focusing on the time series aspect, including the sampling data X i, of each sensor i at each time j, because the physical data dimensions represented by each type of sensor are different, parameter normalization processing is often required:

[0030]

[0031] The parameter is dynamically updated as follows: when an equally spaced update is triggered, each data item loads a value that is shifted one position to the left of the historical record along the time sequence, i.e., X. i, ←X i, , j = 2, 3, ..., t.

[0032] c) Model-based feature fusion focuses on the raw data regarding relationships. ij ∈{0,1} with time window [t0,t1,...,t N The data update generated by the sliding motion. For the dynamic update method of determining the stability of the relationship between entity nodes (i,j) and the parameterized discrimination of the relationship heterogeneity, please refer to the determination method in sub-step 1) and the encoding transformation in sub-step 2) of the multivariate feature graph model construction.

[0033] d) Variable parameters related to model structure and model training (i.e., parameter learning) in the model-based feature fusion module include linear mapping parameters in vector transformations in multivariate time-series prediction models and multivariate feature map models, model parameters in nonlinear units (Transformer, ReLU / GELU, MLP) within neural networks, and hyperparameters set for model training (such as learning rate, training data batch, and number of training epochs). The parameters are dynamically updated as follows: 1) As the system runs, it gathers time-series data and data relationships within a new time window, thereby periodically updating the training set data in batches; 2) If the total number of data batches used for model training remains constant, historical training data and newly gathered training data are mixed with different weight ratios. For example, under the condition that the total number of model training batches remains constant, the epochs of the new training set are increased, the epochs of the historical training set are decreased, or some historical data are randomly discarded; 3) Multivariate time-series prediction models and multivariate feature map models use their own optimization solvers, where the learning rate of neural network training can be adaptively and dynamically adjusted in the optimizer; the currently used model parameters are used as initial values ​​for iterative optimization, and the optimized model parameter values ​​are obtained after training on the new dataset.

[0034] Furthermore, the multivariate time series prediction model (MTS model, referred to as MTS in the following text and figures) involved in the modeling feature fusion module of this invention consists of an integrated model composed of three modules: single-step prediction target fitting (MTS-R1 module in the following text and figures), cross-period prediction target fitting (MTS-R2 module in the following text and figures), and sequence-to-sequence model (MTS-S2S module in the following text and figures). The multivariate time series prediction model is as follows... Figure 3 As shown, it follows the steps below to learn the system situation prediction task:

[0035] 1) Data Acquisition

[0036] Receive the multi-element time series formed by the integration of multi-class Internet of Things data collected over time from the sensor data fusion module, represented as follows:

[0037]

[0038] Where X i,j represents the sampling data of the ith sensor at the jth time; the processing time window length of the sampling data is t, and the total number of Internet of Things sensors included in the model is M. The cross-scale nature of the data X is that the dimensions and ranges of X i,j may be different for different sensors i; the time sequence nature of the data X is that the sampling values of X i,j may be different for different sampling times j, showing dynamic characteristics.

[0039] 2) Data dimension reduction

[0040] To overcome the interference of irrelevant or secondary information in the massive data of the Internet of Things, the feature importance coefficients are ranked through two modules of single-step prediction target fitting and cross-period prediction target fitting, as follows:

[0041] The single-step prediction target fitting model receives the multi-element time series data X as input, and the following k prediction targets at the next step (t+1 time) as the model learning task:

[0042]

[0043] When k = 1, learn a single-target prediction task, such as predicting only the electricity consumption in the next period of the system; when k > 1, learn a multi-target prediction task, such as predicting the deviation of the system electricity consumption and environmental monitoring indicators from the ideal state at the same time.

[0044] After the single-step prediction target fitting model is trained, the importance coefficients of the features corresponding to each dimension of the sensor are output:

[0045]

[0046] The cross-period prediction target fitting model focuses on system signals with periodic fluctuations, such as days, weeks, and quarters; receives the multi-element time series data X as input, and the following k prediction targets at multiple periods (t+s time) as the model learning task:

[0047]

[0048] When k = 1, learn a single-target prediction task, and when k > 1, learn a multi-target prediction task.

[0049] After the cross-period prediction target fitting model is trained, the importance coefficients of the features corresponding to each dimension of the sensor are output:

[0050]

[0051] The feature importance of the first to Mth sensor is converted to:

[0052]

[0053] To facilitate quantitative comparison of the standard, further normalization is performed to obtain the importance coefficient of each dimension feature:

[0054]

[0055] v (1) ~v (M) In descending order, according to the system characteristics and actual demand, directly select top m-dimensional features according to the order or screen the features by threshold value (such as retaining all corresponding features with importance coefficient > 0.05). Record the final retained feature dimension m, that is, retain the data records of m sensors from M sensors, where m << M.

[0056] 3) Sequence to sequence modeling

[0057] The m x t two-dimensional data retained after data dimension reduction

[0058]

[0059] As a sequence composed of column vectors

[0060]

[0061] Wherein i = 1, 2,..., t.

[0062] The sequence of As the t input of the sequence to sequence model, obtain As the r output target of the sequence to sequence model, according to Select different starting time points to build a large number of samples, and learn the model parameters in the neural network structure of the sequence to sequence model through training.

[0063] 4) System situation prediction

[0064] There are two cases: (A) if the prediction target is completely contained in the Output of the sequence to sequence model (that is, the prediction target is the direct quantity of the sensor data in the future r periods), then the prediction value is directly obtained from the target value of the corresponding sensor at the corresponding time in the matrix composed of (B) Otherwise, the prediction target comes from The indirect integration of these known information (such as the estimation system comprehensively keeps the current mode running duration), at this time, the output layer of the sequence to sequence model structure is taken out, denoted as The decoding model with the Transformer mechanism is added downstream, specifically expressed as:

[0065] The three key parameters Q, K, and V of the Transformer are calculated by the following formula:

[0066]

[0067] Wherein is the model parameter; d k is the dimension of the transformed vector of the multi-head attention mechanism of the Transformer, and the relationship between H and d k = m. The multi-head attention calculation method is:

[0068]

[0069] According to the prediction target property, the nonlinear unit RELU or GELU is selected, and the final system situation prediction output is:

[0070]

[0071] Or

[0072] Wherein W Y is the model parameter of the output layer.

[0073] Optimization target (loss function): for each sample model output and each sample true "prediction" value derived from historical data The construction of the objective function is:

[0074] Further, the multi-attribute graph model (Multi-Attribute Graph model, hereinafter and in the drawings referred to as MAG) involved in the model feature fusion module of the present application is used to fuse the correlation knowledge between measurement points and implement more accurate anomaly detection, or automatically identify data linkage relationships that have not been manually marked. The fixed window time series data collected by each sensor is used as a multi-attribute input, the existing correlation graph composed of measurement points is used for relationship learning, and a feature representation model of each node in the correlation graph is constructed. The multi-attribute graph model is as shown in Figure 4 The following steps are followed to complete the identification task of the multi-attribute graph model:

[0075] 1) Entity correlation stability determination

[0076] In contrast to the Internet, where links change frequently, the Internet of Things identifies entities in the physical world, and the rules governing the network are usually more stable, such as the production and manufacturing flow in process industries, device topology connection in smart buildings, and sensor spatial relationships in geographic information systems. To ensure comprehensiveness, first check whether there are any entity links or deletions within the sampling time window [t0, t1,...,t N ) in the Internet of Things, and verify whether there are any link additions or deletions in the key-value pair / relational database / graph database records according to the actual storage form.

[0077] A. If there are no link additions or deletions within the time window [t0, t1,...,t N ), it means that the Internet of Things is stable, and we can directly jump to step 2);

[0078] B. If there are link additions or deletions within the time window [t0, t1,...,t N ), take the changed link e ij as an example. If the time occupied by e ij = 0(1) is k times that of e ij = 1(0), and k is large (e.g., k > 2), then e ij = 0(1) is considered to be the normal association within [t0, t1,...,t N ), and we jump to step 2). Moreover, e ij = 1(0) corresponds to an abnormal sample that can be identified by the model.

[0079] C. If the above two cases do not meet the requirements, it means that the characteristics of the Internet of Things do not conform to the requirements of this patent, and we need to reselect the data sampling period t i -t i-1 , or discard all model training samples within the time window [t0, t1,...,t N ).

[0080] 2) Node feature encoding that fuses stable association information (hereinafter referred to as MAG-Encode)

[0081] After step 1) processing, the Internet of Things data is organized into a static structure heterogeneous network within any sliding time window, and each node in the network contains the above time-varying features.

[0082] To fuse the existing association information and enrich the feature representation of the nodes, the TransH or TransR algorithm suitable for knowledge graphs is applied to the heterogeneous network in this window period. Considering that the entities contained in the Internet of Things also have heterogeneity (such as different device types), the one-hot encoding representing the type is concatenated with the collected time series data as the original multi-element encoding of node i:

[0083]

[0084] wherein denotes that node i belongs to the kth type of device, denotes that node i is in the time window [t0, t1,..., t N ) collected. With this as the original encoding, the node i feature encoding after fusing the association information can be obtained using TransH or TransR:

[0085]

[0086] wherein denotes the fused node i feature encoding, which is valid within the sampled time window [t0, t1,..., t N ). The adoption of TransH or TransR is based on the algorithm that has a smaller measured error in identifying the known links in the heterogeneous network.

[0087] 3) Multivariate feature map model structure

[0088] For the sake of generality, let denote the feature vector of the ith entity obtained through the projection / rotation of TransH / TransR algorithm in the corresponding semantic space. In order to expand the relationship determination in the TransH or TransR algorithm, facing the anomaly detection of the heterogeneous association between entity i and entity j in the Internet of Things, the model input is defined as:

[0089]

[0090] The model structure is designed to output a K-dimensional multilayer perceptron (hereinafter and in the drawings, referred to as MAG-MLP):

[0091]

[0092] wherein K denotes the total number of association types. The corresponding rth element Y ij (r)→1 indicates that entity i and entity j are extremely likely to have the rth type of linkage relationship within the time window [t0, t1,..., t N ). On the contrary, Y ij (r)→0 indicates that there is no such linkage relationship.

[0093] In actual model construction, in order to enhance the robustness of the model, a regularization term is additionally added to the model parameters directly associated with in the input dimension, preventing overfitting caused by excessive use of multi-dimensional time series information within the window period.

[0094] 4) Training and application

[0095] The training samples of the model come from the relationship samples that are not sampled by the TransH or TransR algorithm in the stable relationship in sub-step A of step 1) and the samples that are not involved in the operation of step 2) because of the relationship change as abnormal association in sub-step B of step 1), which are regarded as negative sampling in the model training process when they are recognized as abnormal association changes by the model.

[0096] The training sample format is Each dimension element Y ij (r)∈{0,1,Based on such samples, the neural network training is performed on the multi-layer perception MAG-MLP. The actual training process adopts the random gradient descent and batch gradient descent method to optimize the model parameters; and new samples are collected and generated during the operation of the system, so that the model is usually regularly trained and updated.

[0097] Application: After the model is put into application, according to the collected multivariate data According to the steps described in this section, the input format required by the model is processed, and the judgment of the K-dimensional relationship generated by the model For r=1,2,...,K, when Y ij (r) is inconsistent with the linkage rule set in the window period [t0,t1,...,t N When the system operation between entities i and j occurs abnormal association in this time period.

[0098] Further, the implicit feature data involved in the modelization feature fusion module is a feature vector obtained by splicing two vectors extracted from the output layers of the multivariate time series prediction model MTS-S2S module and the multivariate feature map model MAG-MLP module, and the fusion processing result does not have explicit physical meaning.

[0099] Further, the specific steps involved in the transaction processing module include the following three steps:

[0100] a) different relationship associations are performed on the data received from the modelization feature fusion module or the data of the sensor data fusion module (data after time window sliding processing), and the results of another variable are derived by inputting different values of a single or multiple variables, to form a fuzzy rule;

[0101] b) a data classifier is established for the implicit features received from the modelization feature fusion module; the data collected by the data collection module is mapped to the data classifier output through the two-layer fusion module (i.e. the modelization feature fusion module and the sensor data fusion module) to form a link, which is used as a rule formed by data mining;

[0102] c) on the basis of the mining rules of the dynamic verification of the operation data and the expert approval, a transaction-based ECA rule, i.e. an ECA system cascade action rule, is created.

[0103] Further, the transaction processing module of the present application establishes the transaction relationship between the subsystems based on the new ECA (Event-Condition-Action) rule of the transaction, forms the linkage control logical relationship between the subsystems, and is used for the safety monitoring, system situation assessment and linkage operation decision between the subsystems.

[0104] The present application has the following beneficial effects:

[0105] The cross-scale time series data fusion method and system for the Internet of Things of the present application complete the data collection from each subsystem and sensor, filter and uniformly store the data, extract the correlation or dependency from a large amount of cross-scale data through sensor data fusion, model feature fusion and establishment of ECA transaction processing rules, and realize the data fusion sharing and logical linkage between the subsystems. BRIEF DESCRIPTION OF DRAWINGS

[0106] Figure 1 is a cross-scale time series data fusion system architecture diagram for the Internet of Things proposed by the present application;

[0107] Figure 2 is a cross-scale data collection schematic diagram for the Internet of Things proposed by the present application;

[0108] Figure 3 is a multivariate time series prediction model (MTS) proposed by the present application;

[0109] Figure 4 is a multivariate feature map model (MAG) proposed by the present application;

[0110] Figure 5 is a subsystem linkage architecture diagram proposed by the present application;

[0111] Figure 6 is a model feature fusion specific implementation flowchart for the large intelligent building application scenario proposed by the present application. DETAILED DESCRIPTION

[0112] The present application will be further described in detail below in combination with the drawings and specific implementation cases. It can be understood that the described implementation cases are only a part of the present application, but not all the implementation cases. Based on the implementation cases in the present application, those skilled in the art can make corresponding changes and modifications according to the present application, and these corresponding changes and modifications should all belong to the protection scope of the claims attached to the present application.

[0113] The application will be described below in combination with the application in a large intelligent building.

[0114] Firstly refer to Figure 1 The embodiment realizes the four modules as shown in Figure 1 , completes the cross-scale data collection of the large intelligent building subsystem, the time window sliding processing of the data, the modelized feature fusion technology, the real-time rule inference transaction processing, and the system operation decision.

[0115] The cross-scale time series data fusion method of the application is as follows:

[0116] Step 1, cross-scale time series data collection. As shown in Figure 2 , in the present case, the data of more than 30 subsystems including the intelligent lighting system, the power monitoring system, the fire alarm system, the pneumatic window system, etc. are collected, and the protocol types include KNX, OPC, Modbus, BACnet, etc. The data collection program analyzes the data transmission protocols of the terminal devices of these subsystems, unifies the cross-scale time series data, and completes the cross-node collection of the cross-scale data. Meanwhile, the data collection program sends the collected data to the Agilent real-time database after denoising and filtering, uniformly manages all the subsystem data, and forms a data management platform.

[0117] Step 2, sensor data fusion. In the present case, the fusion of the sensor data is realized by preprocessing the data collected in step 1, that is, by converting the cross-scale time series data into equidistant time series data through algebraic interpolation and machine learning fitting methods; by integrating the graph-level associated data of the Internet of Things devices by extracting the data in the set time window in real time; by integrating the data nodes through ECA rules, completing the first data fusion of the collected more than 30 subsystem data, and outputting the multivariate time series data and multivariate graph data to the modelized feature fusion module.

[0118] Step 3, modelized feature fusion. In the present case, in view of the inference decision demand suitable for the intelligent operation and regulation of large buildings, multivariate time series prediction models (as shown in Figure 3 ), multivariate feature graph models (as shown in Figure 4 ), etc. are used to study the modelized feature fusion technology, and high-quality knowledge representation and inference rules for the operation and regulation in buildings are generated.

[0119] In view of the time-space correlation coupling characteristics of the operation dynamics of multiple systems and multiple devices in large intelligent buildings, the modelized feature fusion implementation scheme (as shown in Figure 6As shown in Fig. 1, i.e. MTS and MAG retain their respective complete mechanisms in this case, but instead of being applied individually, they are reused to extract intermediate features. Overall, Figure 6 The five branches 1.1-2.2 cover the sum of the MTS and MAG modeling feature extraction mechanisms. The implementation of MTS, MAG, and directional extraction tasks in the modeling feature fusion is described in the following stages.

[0120] 1) The first stage is the multi-channel signal feature extraction step dominated by MTS, which in this case seeks new feature combination methods that may exist. To overcome the interference of irrelevant or secondary information in the massive data of the Internet of Things, two modules, single-step prediction target fitting and cross-cycle prediction target fitting, are used to rank the importance of features by feature importance coefficients Figure 6 Branch 1.1), the specific steps of which are described in the multivariate time series prediction module MTS-R1 and MTS-R2 as described above. In addition, through visual analysis of real data, it is found that a large number of device measurement time series signals in the system have multi-scale periodic approximations (for example, air conditioning systems highly correlated with outdoor weather present a mixed fluctuation of daily, seasonal, and even potential annual cycles, units in automatic control mode present reciprocating signal oscillation between high and low set thresholds, intelligent lighting systems present daily cycle data fluctuations, etc.). Based on this phenomenon, this case designs information combination in the time domain channel and the frequency domain channel to extract and transform system characteristics with rich information.

[0121] Specifically, in this case, the time domain feature extraction channel Figure 6 Branch 1.2), on the basis of preserving the time series signals of each dimension of the system, the performance improvement brought by the convolution unit in the convolutional neural network for downstream recognition tasks is referenced, and a one-dimensional window convolution (also known as 1-Dimensional Convolution, or 1 x D Conv) unit is added to the data model in this case, which functions to extract time domain features within a local time segment; according to different convolution parameters set in practice, the potential functions also include local time domain signal smoothing (to overcome local high-frequency interference) and the function of measuring the degree of fit with a reference signal segment (Shapelet).

[0122] Specifically, in this case, the frequency domain feature extraction channel Figure 6Branch 1.3) addresses the issue that time-series signals monitored by equipment within large buildings do not exhibit a single-scale periodicity. Therefore, to address the multi-scale frequency domain characteristics of multi-dimensional data, it is necessary to retain data features over a wide spectral range. Based on this requirement, this case study designs a branch channel for frequency domain feature extraction. The time-series data within the window period is transformed into the frequency domain using Discrete Fourier Transform. Except for filtering out anomalies in each single-dimensional signal band that significantly exceeds the threshold, the signal quantity within the remaining window spectrum is retained. Furthermore, mimicking the design of the time-domain channel, a 1×D convolutional unit is added to identify local frequency domain signal patterns.

[0123] The final output features of the first stage are composed of automatically evaluated m-dimensional important original features, temporal convolutional features, and frequency domain convolutional features.

[0124] 2) The second stage involves the parallel implementation of the multivariate feature map model association extraction process and the MTS-S2S built-in temporal coding process. The association feature coding process is led by the multivariate feature map model (MAG). Figure 6 In Branch 2.1), the content for determining the stability of entity associations for specific building application cases includes: element associations that can be extracted programmatically from Building Information Modeling (BIM) or equipment associations that can be manually organized / defined by rules. This includes topological associations between devices or collection points in the building (such as pipeline connections, heat exchange channels, power monitoring master-slave meters, etc.), location associations located in adjacent or the same space, and logical associations in the dynamic operation of the system. The latter fully reflects the necessity of dynamically determining the stability of entity associations in MAG over time windows. For example, the automatic control of indoor lighting during non-working hours transitions from a strong association with outdoor weather conditions to a strong association with human body detection sensors in the office area. The dynamic switching of association characteristics transmitted by the various modes of operation of the central air conditioning unit's cold and heat source, such as the control of indoor CO2 content → fresh air switch, and so on.

[0125] Specifically, Figure 6 Branch 2.1 The specific process of MAG-Encode, which is dominated by the multivariate feature map model, in this case receives the original multivariate encoded input.

[0126]

[0127] Each of them The method is adjusted to integrate the m-dimensional important original features, temporal convolutional features, and frequency domain convolutional features extracted by MTS in branches 1.1 to 1.3. Thus, MAG utilizes system dynamic features from the time-frequency domain in addition to the model-independent mechanism. The MAG-Encode module converts the above original encoding into node encoding. Subsequently, in order to extract core features from the encoding for subsequent directional tasks (such as multi-dimensional time series prediction, abnormal data correction), a module using a graph attention mechanism (GAT) is concatenated in branch 2.1, which automatically quantifies the importance of the correlation between nodes and adaptively improves the ability to transmit important information to the next stage.

[0128] In parallel with this, Figure 6 The process shown in branch 2.2 focuses on integrating and encoding time series information. This process is dominated by MTS, which arranges the original encoding received from the first stage in the time direction The equal-interval multivariate time series is encoded into a low-dimensional embedding vector by the encoding module built into MTS-S2S In this case, according to the different characteristics of the directional tasks in the next stage, the encoding module built into MTS-S2S selects an encoding module with a temporal self-attention mechanism, which adaptively transmits important information in the time direction by capturing the importance of the time relationship within the time window.

[0129] Based on the above implementation steps, the output features of this stage are transformed from the high-dimensional mixed information of the first stage into low-dimensional and dense vector features, while containing important information in both the multivariate correlation and the time correlation.

[0130] 3) Third stage, model feature fusion for directional tasks. In this case, the multi-dimensional data in the large building data management platform has both time correlation and significant inter-feature correlation (such as the influence of outdoor fresh air temperature on the adjustment of the air conditioning system in the building, the correlation between the meteorological station system and the operation of the air window system, etc.), two types of directional tasks are designed as evaluation benchmarks for the joint training of MTS and MAG models - abnormal data correction tasks and multi-dimensional time series prediction tasks (such as Figure 5 The third stage shown in the third stage); In this stage, a gating unit is placed on each task branch to avoid coupling interference between feature components, and the model is trained and optimized according to the loss function jointly formulated for the two types of tasks, to avoid the model feature fusion process falling into a local optimal solution.

[0131] In the two types of directional tasks, the abnormal data error correction includes both restoring the correct distribution of the collected data and identifying the abnormal feature correlation between the data, which can be regarded as a pre-training basic task related to system anomaly detection in subsequent transaction processing. The multi-dimensional time series data prediction task is divided into short-term and long-term data prediction according to the application requirements of building operation and management. In step 3, the fusion results of MTS-R1 and MTS-R2 modules can be effectively echoed, and it can also be regarded as a pre-training basic task related to system situation analysis and prediction in subsequent transaction processing. Therefore, the modelized feature fusion case using the embodiment can provide strong support for the transaction processing (step 4) of the system.

[0132] Step 4, transaction processing. As shown in ​ the case uses a dynamic linkage scheme iteration, a rule-based reasoning engine technology, and an interactive customization scenario to achieve intelligent diagnosis in different scenarios such as regular operation, emergency situations, and outdoor disasters, and to complete explainable intelligent decision-making. For example, transaction processing between a meteorological system, a fresh air system, and an air conditioning system: real-time acquisition of air quality values, indoor temperature values, fresh air system parameters, and air conditioning system parameters in the meteorological system; when the air quality value is found to be substandard, the fresh air system is started, and the operation mode of the fresh air system is automatically adjusted according to different pollution programs; at the same time, according to the monitored indoor temperature, it is judged whether to start the air conditioning system. When the air quality is up to standard, the fresh air system is automatically stopped, and the air conditioning system is stopped when the temperature reaches the set temperature.

[0133] Another embodiment of the present application provides an Internet of Things-oriented cross-scale time series data fusion system, which comprises:

[0134] A data acquisition module is used to collect cross-scale time series data from multiple subsystems and multiple types of sensors, and to send the collected cross-scale time series data to a sensor data fusion module after denoising and filtering;

[0135] A sensor data fusion module is used to perform time window sliding processing on the cross-scale time series data received from the data acquisition module. The processed results include two types of data: equidistant multivariate time series and multivariate correlation relationship. The processed results are provided to a modelized feature fusion module through an internal interface;

[0136] A modelized feature fusion module is used to perform modelized feature fusion on the two types of data: equidistant multivariate time series and multivariate correlation relationship, which are processed by the sensor data fusion module. Simultaneously, the modelized feature fusion module dynamically loads parameters and runs according to the transaction processing period, and provides the fused implicit feature data to a transaction processing module through an internal interface;

[0137] The transaction processing module is used for applying the implicit feature data provided by the modeling feature fusion module to security monitoring, system situation assessment and inter-subsystem linkage operation decision after processing according to specific steps.

[0138] Another embodiment of the present application provides a computer device (computer, server, smart phone, etc.), which comprises a memory and a processor, the memory stores a computer program configured to be executed by the processor, the computer program comprises instructions for executing steps in the method of the present application.

[0139] Another embodiment of the present application provides a computer readable storage medium (such as ROM / RAM, magnetic disk, optical disk), which stores a computer program, the computer program is executed by a computer to realize each step of the method of the present application.

Claims

1. A method for cross-scale time series data fusion for Internet of Things, characterized in that, The method comprises the following steps: Collecting cross-scale time series data from multiple subsystems and multiple types of sensors; Sliding time window processing of the cross-scale time series data, the processed results including two types of data: equidistant multivariate time series and multivariate correlation; Modeling feature fusion of the two types of data, and synchronously loading parameters and running with transaction processing period to obtain fused implicit feature data; Applying the fused implicit feature data to security monitoring, system situation assessment and inter-subsystem linkage operation decision-making after processing according to specific steps; The multivariate correlation includes causal relationships between cross-scale time series data, and other relationships between cross-scale time series data established by using a graph model, the other relationships including spatial relationships and device topology relationships; The modeling feature fusion includes: a) using historical data of equidistant multivariate time series as a reference sample, and using current and subsequent time series data as a learning target to establish a multivariate time series prediction model; b) using the attributes of measured entities in the Internet of Things as nodes, and using multivariate correlation as edges to establish a multivariate feature graph model to mine potential correlation or correlation changes; c) saving model parameters of the multivariate time series prediction model and the multivariate feature graph model as readable data files by using a programmed automatic optimization solver and updating them regularly.

2. The method of claim 1, wherein, The cross-scale time series data includes one or more of the following data: data of different time granularities, data of different communication modes, data of different communication protocols, and data of different data types.

3. The method of claim 1, wherein, The time window sliding processing realizes sensor data fusion by using a time window and the following two aspects: a) converting the cross-scale time series data into equidistant multivariate time series by using methods such as algebraic interpolation, machine learning fitting, spline function or numerical integration; b) dynamically extracting multivariate correlation between data in a set time window during system operation.

4. The method of claim 1, wherein, The multivariate time series prediction model is an integrated model composed of a single-step prediction target fitting model, a cross-period prediction target fitting model and a sequence-to-sequence model, and learns the prediction task of system situation according to the following steps: 1) Data acquisition: Receiving multivariate time series formed by collecting multiple types of Internet of Things data over time, which is represented by the following two-dimensional data: wherein X i,j represents the sampling data of the ith sensor at the jth time; the processing time window length of the sampling data is t, the total number of the Internet of Things sensors included in the model is M, the cross-scale nature of the data X is that the dimensions and ranges of X i,j may be different for different sensors i; the time sequence nature of the data X is that the sampling values of X i,j may be different for different sampling times j, presenting dynamic characteristics; 2) Data dimension reduction: Ranking the importance of features by using the single-step prediction target fitting model and the cross-period prediction target fitting model, the steps are as follows: The single-step prediction target fitting model receives multivariate time series data X as input, and the following k prediction targets at "t+1 time" are the model learning tasks: When k = 1, a single target prediction task is learned; when k > 1, a multi-target prediction task is learned; The single-step prediction target fitting model outputs the importance coefficients of the features corresponding to each dimension of the sensor after training: The cross-period prediction target fitting model focuses on system signals with periodic fluctuations, receives multivariate time series data X as input, and the following k prediction targets at "t+s time" are the model learning tasks: Wherein when k=1, a single-target prediction task is learned, and when k>1, a multi-target prediction task is learned; The cross-cycle prediction target fitting model outputs the importance coefficients of the features corresponding to each dimension sensor after training: The evaluation results of the single-step prediction target fitting model and the cross-cycle prediction target fitting model are integrated, and the feature importance corresponding to the first M sensors is converted into: Normalization is performed to obtain the importance coefficients of the final features of each dimension: v (1) ~v (M) In descending order, according to the system characteristics and actual needs directly by sorting to select top m-dimensional feature or threshold screening features, record the final reserved feature dimension m, that is, from M sensors to retain m sensor data records, wherein m << M; 3) Sequence-to-sequence modeling: The m*t two-dimensional data retained after data dimensionality reduction is: Considered as a sequence composed of column vectors wherein serializing obtaining, as t inputs of the sequence-to-sequence model, from historical data samples obtaining, as r sequentially output targets of the sequence-to-sequence model, according to selecting different starting time points to construct a large number of samples, and learning model parameters in the neural network structure of the sequence-to-sequence model through end-to-end training; 4) System situation prediction: There are two cases: (A) if the prediction target is completely contained in the output of the sequence-to-sequence model, the prediction value is directly obtained from the corresponding time point and sensor value in the matrix composed of the output of the sequence-to-sequence model; (B) otherwise, the prediction target comes from the indirect integration of known information, at this time, the hidden vector of the output layer in the sequence-to-sequence model structure is taken out, denoted as In the downstream, a decoding model with a Transformer mechanism is added, which is specifically expressed as: In the downstream, a decoding model with a Transformer mechanism is added, which is specifically expressed as:​​ The three groups of key parameters Q, K, and V of the Transformer are calculated by the following formula: wherein is a model parameter, d k is the dimension of the transformed backward vector of the multi-head attention mechanism of the Transformer, and the relationship between the dimension and the number of multi-head attention H is: Hd k = m; the multi-head attention calculation method is: According to the nature of the prediction target, select the nonlinear unit RELU or GELU, and finally output the system situation prediction: or where W Y is the model parameter of the output layer, Optimization objective: model output for each sample true "predicted" value for each sample derived from historical data The objective function is constructed as:

5. The method of claim 1, wherein, The multi-element feature graph model is used to fuse the correlation knowledge between measurement points and implement more accurate anomaly detection, or automatically identify the data linkage relationship that has not been manually marked, and uses the existing correlation graph composed of measurement points for relationship learning, and constructs a feature representation model of each node in the correlation graph; The following steps are followed to complete the identification task of the multi-element feature graph model: 1) Entity correlation stability determination: First check the sampling time window [t0, t1,..., t N ) whether there is an entity link or deletion inside the Internet of Things, and the verification method is to check whether there is a link addition or deletion in the "key-value pair / relational database / graph database" record according to the actual storage form; A. If there is no link addition or deletion in the time window [t0, t1,..., t N , it means that the operation in the Internet of Things is stable, and directly jump to step 2)； B. If the time window [t0, t1,..., tk] is divided into several sub-windows, the link e N ) is added or deleted to change the link e ij ) is added or deleted to change the link e ij ) is added or deleted to change the link e ij ) is added or deleted to change the link e ij ) is added or deleted to change the link e N ) is added or deleted to change the link e ij ) is added or deleted to change the link e C. Otherwise, the above two cases are not met, indicating that it does not meet the characteristics of the Internet of Things that this patent is facing, and the data sampling period t needs to be reselected i -t i-1 , or give up all model training samples within the time window [t0, t1,..., t N ) 2) Node feature coding that fuses stable correlation information: Considering that the entities contained in the Internet of Things also have heterogeneity, the one-hot coding of the feature type is spliced with the collected time series data as the original multi-element coding of node i: wherein denotes that node i belongs to the kth type of device, denotes that node i is in the window period [t0, t1,..., t N ) collected; taking this as the original code, the node i feature code after fusing the associated information can be obtained by using TransH or TransR: wherein represents the fused node i feature encoding, valid within the sampled time window [t0, t1,..., t N ). TransH or TransR is used according to the algorithm that has less identification error in the known link identification of the heterogeneous network; 3) Multi-element feature graph model structure: In represents the feature vector of the ith entity obtained by the projection / rotation of the TransH / TransR algorithm on the corresponding semantic space; in order to extend the relationship determination in the TransH or TransR algorithm, facing the anomaly detection of the heterogeneous association between the entity i and the entity j in the Internet of Things, the model input is defined as: The model structure is designed as a multi-layer perceptron with K dimensions as output: wherein K represents the total number of association types; the corresponding rth-dimensional element Y ij (r)→1 means that entity i and entity j are very likely to have the rth type of linkage relationship within the time window [t0, t1,..., t N ) and vice versa Y ij (r)→0 means that there is no such linkage relationship. For input dimensions and Add regularization terms to directly related model parameters to prevent overfitting caused by excessive use of multidimensional time series information within the window period; 4) Training and application: The training samples of the model come from the relationship samples that are not sampled by the TransH or TransR algorithm in the stable relationship in A of step 1) and the samples that are not involved in step 2) because of relationship changes in B of step 1) as abnormal correlation changes, which are regarded as negative sampling in the model training process when they are identified as abnormal correlation changes by the model, The training sample format is Each dimension element Y ij (r) e {0,1}, based on such samples, a multi-layer perceptron is trained for neural network, and the actual training process adopts a random gradient descent and a batch gradient descent method to optimize the model parameters; new samples are generated as the system runs, so the model is usually regularly trained and updated; After the model is put into application, the collected multi-element data The input format required for processing the model, the determination of K-dimensional relationship generated by the model For r = 1, 2, …, K, when Y ij (r) and the actual system do not match the linkage rule set in the window period [t0, t1, …, t N If they do not match, it is considered that the system operation has an association anomaly between entities i and j in the time period.

6. The method of claim 1, wherein, The specific steps include: a) Different relationship correlations are performed on the data after feature fusion or the data after time window sliding processing, different values of a single or multiple variables are input, and the result of another variable is derived, forming a fuzzy rule; b) A data classifier is established for the implicit features after feature fusion, and the link formed by the output of the data classifier is mapped by the two fusion modules, which is regarded as a rule formed by data mining; c) On the basis of the mining rules verified by running data and expert approval, transaction-based ECA rules are created.

7. The method of claim 6, wherein, Transaction relationships between subsystems are established by using the transaction-based ECA rules, and a linkage control logic relationship is formed between the subsystems, thereby being used for security monitoring, system situation assessment, and linkage operation decision-making between subsystems.

8. An Internet of Things oriented cross-scale time series data fusion system, characterized in that, It includes: The data acquisition module is configured to collect cross-scale time series data from various subsystems and various types of sensors, and send the collected cross-scale time series data to the sensor data fusion module after denoising and filtering. The sensor data fusion module is configured to perform time window sliding processing on the cross-scale time series data received from the data acquisition module, and provide the processed results, including equally-spaced multivariate time series and multivariate correlation relationship data, to the modeling feature fusion module through an internal interface. The modeling feature fusion module is configured to perform modeling feature fusion on the equally-spaced multivariate time series and multivariate correlation relationship data processed by the sensor data fusion module. Simultaneously, the modeling feature fusion module dynamically loads parameters and runs according to a transaction processing cycle, and provides fused implicit feature data to the transaction processing module through an internal interface. The transaction processing module is configured to apply the implicit feature data provided by the modeling feature fusion module to security monitoring, system situation assessment, and inter-subsystem linkage operation decision-making after processing according to specific steps. The multivariate correlation relationship includes causal relationships between cross-scale time series data, and other relationships between cross-scale time series data established by using a graph model. The other relationships include spatial relationships and device topology relationships. The modeling feature fusion includes: a) using historical data of equally-spaced multivariate time series as a reference sample, and using current and subsequent time series data as a learning target to establish a multivariate time series prediction model; b) using properties of measured entities in the Internet of Things as nodes, and using multivariate correlation relationships as edges to establish a multivariate feature graph model, and mining potential correlation relationships or changes in correlation relationships; c) using a programmed automatic optimization solver to save model parameters of the multivariate time series prediction model and the multivariate feature graph model as readable data files and update them regularly.

Images