Multi-sensing data time sequence registration method and system based on iterative update graph neural network

Abstract

The invention relates to the technical field of industrial data time sequence registration, and discloses a multi-sensing data time sequence registration method and system based on an iterative update graph neural network, and the method comprises the steps: building a domain knowledge graph based on the mechanism knowledge of an industrial process and expert experience, and building a multi-sensing data time sequence registration algorithm based on a network topology structure of the knowledge graph. According to the method, the graph neural network model is guided to be constructed, the coupling relation among the multiple sensing data is initialized, the analysis of the model on the relation among the multiple sensing data is enhanced, the time sequence registration process of the multiple sensing data is guided, and the accuracy of the time sequence registration result is improved.

Description

Technical Field

[0001] This application relates to the technical field of industrial data time series registration, and in particular, to a multi-sensor data time series registration method and system based on an iterative update graph neural network. Background Technique

[0002] Multi-sensor data, as the time series variables collected by sensors deployed in each link of the production line, reflects the operation information of the industrial production process in various states, providing an important basis for production control and optimization. In the industrial production process, due to the differences in material transfer time and the spatial and temporal distributions of process reaction units, the sensor detection data such as temperature, pressure, flow rate, speed, and composition in the production process has obvious time-delay characteristics, which affects the true causal relationship between data and increases the difficulty of online monitoring of operating states and online modeling of product quality. Therefore, effectively estimating the time-delay information between multi-sensor data in the industrial process and performing time series registration on the data is an important prerequisite for subsequent process modeling, optimization control, and performance evaluation.

[0003] Existing time series registration methods can be divided into statistical learning methods, signal processing methods, machine learning methods, and deep learning methods, etc. Statistical learning methods, such as methods based on correlation analysis, mutual information, and association matrix analysis, mainly rely on calculating the correlation between time series, finding the sequence displacement corresponding to the maximum correlation, and thus determining the time delay between variables. However, this type of method is relatively sensitive to noise, and in a non-linear and complex environment, its estimation accuracy will be limited. Signal processing methods represented by methods such as wavelet transform and autoregressive model focus on extracting the time-frequency information and periodic characteristics of signals, and then calculating the time delay. However, this type of method is only applicable to processing short-term, linear, and stationary time series registration problems. In recent years, methods based on machine learning and deep learning have been widely used due to their powerful learning ability, such as support vector machines, random forests, convolutional neural networks, long short-term memory networks, etc. These methods show significant advantages in processing non-linear and complex process time series registration by learning the complex mapping relationship between multi-sensor data from a large number of samples.

[0004] Through a comprehensive comparative analysis of the advantages and limitations of the above various time series registration methods, it can be known that the deep learning-based method has the widest applicable range and the best model effect. However, when the current deep learning method is applied to time series registration, it often ignores the coupling relationship and interaction between variables, resulting in low accuracy of the time series registration result. At the same time, considering that the purpose of the time series registration process is to serve subsequent downstream tasks, such as regression prediction of key process parameters, identification and classification of process states, etc., but the existing time series registration process is often separated from subsequent downstream tasks, making the applicability of the time series registration result poor.

[0005] It can be seen that the time series registration methods in the prior art have problems of poor accuracy and applicability. Summary of the Invention

[0006] The present application provides a multi-sensor data time series registration method and system based on an iterative update graph neural network to solve the problems of poor accuracy and applicability of the time series registration methods in the prior art.

[0007] To achieve the above object, the present application is implemented through the following technical solutions:

[0008] In a first aspect, the present application provides a multi-sensor data time series registration method based on an iterative update graph neural network, including:

[0009] S1: Construct a domain knowledge graph with each sensor as a node and the relationship between sensors as an edge;

[0010] S2: Initialize the graph neural network topology structure based on the graph topology structure of the domain knowledge graph; wherein, each node in the graph neural network topology structure corresponds to each entity in the domain knowledge graph, each edge in the graph neural network topology structure corresponds to the relationship between nodes in the domain knowledge graph, and the probability that there are multiple directed edges between two nodes is greater than 0;

[0011] S3: Determine a time series registration algorithm based on the graph neural network topology structure, and register the data set based on the time series registration algorithm to obtain a registered data set;

[0012] S4: Construct a downstream task based on the registered data set, and construct a model loss function based on the downstream task;

[0013] S5: Design an iterative update mechanism for the graph neural network topology structure;

[0014] S6: Perform iterative update training based on the model loss function and the iterative update mechanism until the set conditions are met to obtain the final graph neural network structure;

[0015] S7: Input the multi-sensor data collected in real time into the final graph neural network structure for time series registration.

[0016] In a second aspect, the present application provides a multi-sensor data time series registration system based on an iterative update graph neural network, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the method described in the first aspect are implemented.

[0017] Advantageous Effects:

[0018] The multi-sensor data temporal registration method based on iterative update graph neural network provided by this application creates a domain knowledge graph based on the mechanism knowledge and expert experience of industrial processes, uses the domain knowledge graph of industrial processes to guide the temporal registration process of multi-sensor data, and constructs a graph neural network model based on the network topology structure of the knowledge graph, initializes the coupling relationship between multi-sensor data, strengthens the analysis of the relationship between multi-sensor data by the model, and is beneficial to improving the accuracy of temporal registration results.

[0019] In a further technical solution, this application proposes a temporal registration algorithm based on the topology structure of a graph neural network. By calculating the weighted hop distance, the zero-delay reference column for each delay calculation process and the initial delay interval of each variable are determined, fully considering the physical meaning and delay differences of each variable in the specific process, making the temporal registration result more matched and suitable for downstream tasks.

[0020] In a further technical solution, this application designs a model loss function driven by downstream tasks and guided by a knowledge graph. This function consists of three parts: the model performance of the downstream task, the smoothness constraint of the graph neural network structure, and the relationship constraint and transitivity constraint of the knowledge graph. The former guides the optimization of the parameters of the graph neural network, and the latter two parts play a regularization role, jointly providing feedback guidance for the parameter update of the nodes and edges of the graph neural network, making the network information more applicable to specific downstream tasks.

[0021] In a further technical solution, this application designs an iterative update mechanism for the graph neural network structure based on the adjacency matrix and node similarity matrix of the graph neural network, adaptively deletes or generates edges, plays a role in screening key variables applicable to downstream tasks, gradually obtains the key variables and their associated relationships matching specific downstream tasks, realizes the optimization of the network topology structure applicable to downstream tasks, and is beneficial to strengthening the applicability of temporal registration results to downstream tasks. Description of the Drawings

[0022] Figure 1 One of the flowcharts of a multi-sensor data temporal registration method based on iterative update graph neural network for a preferred embodiment of this application;

[0023] Figure 2 Another flowchart of a multi-sensor data temporal registration method based on iterative update graph neural network for a preferred embodiment of this application;

[0024] Figure 3 Schematic diagram of the blast furnace ironmaking knowledge graph for a preferred embodiment of this application;

[0025] Figure 4 One of the partial enlarged views of the blast furnace ironmaking knowledge graph for a preferred embodiment of this application;

[0026] Figure 5 It is the second partial enlarged view of the blast furnace ironmaking knowledge graph of the preferred embodiment of the present application;

[0027] Figure 6 It is the curve graph of the experimental test effect of the preferred embodiment of the present application;

[0028] Figure 7 It is the scatter plot of the experimental test effect of the preferred embodiment of the present application. Detailed implementation manners

[0029] The technical solutions of the present application will be described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the scope of protection of the present application.

[0030] Unless otherwise defined, the technical terms or scientific terms used in the present application shall have the ordinary meanings understood by those of ordinary skill in the art to which the present application belongs. The "first", "second" and similar terms used in the present application do not denote any order, quantity or importance, but are only used to distinguish different components. Similarly, the terms such as "a" or "one" do not denote a quantity limitation, but mean that there is at least one. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left" and "right" are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship also changes accordingly.

[0031] First of all, it is worth explaining that a knowledge graph is a data structure used to represent entities and the relationships between them. It organizes data in the form of a graph, where nodes represent entities and edges represent the relationships between entities. A domain knowledge graph is a knowledge graph specifically constructed for the knowledge of a certain domain, and it aims to integrate the mechanism knowledge, expert experience and data of the domain. Based on the mechanism knowledge and expert experience of the industrial process, the present invention creates a domain knowledge graph and guides the construction of a graph neural network to provide theoretical guidance for time series registration and the relationship information between multi-sensor data.

[0032] It should be understood that a multi-sensor data time series registration method based on an iteratively updated graph neural network provided by this application can be applied to various industrial scenarios, and each industrial scenario corresponds to different multi-sensor data types. For example, in the blast furnace ironmaking industry, the multi-sensor data types include top pressure, hot blast temperature, cold air flow rate, oxygen enrichment rate, resistance coefficient, total pressure difference, etc. In the petrochemical industry, the multi-sensor data types include natural gas flow rate, hydrogen sulfide concentration, pipeline pressure, reactor temperature, etc. In the field of transportation, the multi-sensor data types include vehicle speed, vehicle acceleration, tire pressure, vehicle flow rate, etc.

[0033] Please refer to Figures 1 - 2 , a multi-sensor data time series registration method based on an iteratively updated graph neural network provided by this application includes:

[0034] S1: Construct a domain knowledge graph with each sensor as a node and the relationship between sensors as an edge;

[0035] S2: Initialize the graph neural network topology structure based on the graph topology structure of the domain knowledge graph; where each node in the graph neural network topology structure corresponds to each entity in the domain knowledge graph, each edge in the graph neural network topology structure corresponds to the relationship between nodes in the domain knowledge graph, and the probability that there are multiple directed edges between two nodes is greater than 0;

[0036] S3: Determine a time series registration algorithm based on the graph neural network topology structure, and perform registration on the data set based on the time series registration algorithm to obtain a registered data set;

[0037] S4: Construct a downstream task based on the registered data set, and construct a model loss function based on the downstream task;

[0038] S5: Design an iterative update mechanism for the graph neural network topology structure;

[0039] S6: Perform iterative update training based on the model loss function and the iterative update mechanism until the set conditions are met to obtain the final graph neural network structure;

[0040] S7: Input the multi-sensor data collected in real time into the final graph neural network structure for time series registration.

[0041] The above multi-sensor data time series registration method based on an iteratively updated graph neural network creates a domain knowledge graph based on the mechanism knowledge and expert experience of the industrial process, guides the time series registration process of multi-sensor data by using the domain knowledge graph of the industrial process, and guides the construction of a graph neural network model based on the network topology structure of the knowledge graph, initializes the coupling relationship between multi-sensor data, strengthens the analysis of the relationship between multi-sensor data by the model, and is conducive to improving the accuracy of the time series registration result.

[0042] Next, a complete example is used to describe the specific implementation solution of the above multi-sensor data time series registration method based on iterative update graph neural network in detail as follows:

[0043] (1) Construct a domain knowledge graph

[0044] Step1: In this embodiment, the present application collects and collates relevant mechanism knowledge and expert experience for a specific industrial process, and develops a domain knowledge graph suitable for the specific process. The nodes in it are all multi-sensor data that need to perform time series registration, and the edges are the relationships between multi-sensor data. Moreover, the relationship data information is represented by triples. The representation method of the triple Triplet shown in Equation (1) is as follows:

[0045] Triplet = (p, r, q), Triplet ∈ KG (1)

[0046] Among them, p and q are the head node and the tail node respectively, r is the relationship type, and KG is the domain knowledge graph of this industrial process. This formula indicates that the relationship between node p and node q is r, and this relationship is directional. At the same time, there can be multiple relationships between two nodes.

[0047] The present application designs the following six relationship types by sorting out common relationship types:

[0048] Table 1 Introduction to relationship types

[0049]

[0050] Step2: After constructing the domain knowledge graph containing the variables to be estimated and their relationships, the present application initializes the graph neural network based on the graph topology structure of the knowledge graph, that is, each node in the graph neural network corresponds to each entity in the knowledge graph, and each edge in the graph neural network corresponds to the relationship between nodes in the knowledge graph, and the probability that there are multiple directed edges between two nodes is greater than 0.

[0051] Let the time series be Containing N samples and M variables, the data contained in node p and node q are respectively and Let the graph structure of the graph neural network be G = (V, E), E represents the edge set, and e p,q ∈ E represents the edge vector from node p to node q. For edges corresponding to the same relationship type, their initial sizes are the same. For edges corresponding to different relationship types, their initial sizes are different, and e p,q Is set as a trainable parameter. At the same time, e p,qIt can contain multiple numerical values, representing multiple edges between two nodes. V represents the set of nodes, and the embedding vector of node p is z p , and its calculation formula is as follows:

[0052]

[0053] In the formula, and are respectively trainable weight vectors. N(p) represents the first-order neighbor nodes of node p, which provide relationship information with node p. q ∈ N(p) means that node q is a first-order neighbor node of node p. x p and x q are respectively the time-series data of node p and node q, and e p,q represents the edge vector from node p to node q. Let the adjacency matrix of the graph neural network be represents the influence degree of node p on node q, |V| represents the number of elements in the node set, and the elements corresponding to unconnected node pairs in matrix A are 0. Therefore, matrix A is an asymmetric matrix. The symbol represents element-wise multiplication, and ReLU is the activation function, and its calculation formula is:

[0054] ReLU(x) = max(0, x) (3).

[0055] In summary, based on the domain knowledge of the industrial process, the present invention creates a domain knowledge graph and initializes the graph neural network for time-series registration, creating a prerequisite for subsequent time-series registration.

[0056] (2) Construct a time-series registration algorithm based on the topological structure of the graph neural network

[0057] Step1: Calculate the node similarity matrix of the graph neural network Let the similarity between node p and node q be S p,q , and its calculation formula is as follows:

[0058]

[0059] Among them, and respectively represent the cosine similarity and neighbor similarity between node p and node q, is used to measure the similarity of the numerical values of two nodes, and its calculation formula is:

[0060]

[0061] In the formula, cos is the function for calculating the cosine similarity, z p and z q are respectively the node embedding vectors of node p and node q, and the symbol ||zp || and ||z q || respectively represent taking the modulus operation on the vector z p and z q for modulus operation.

[0062] is an index to measure the tightness of the connection between nodes through common neighbors, usually calculated based on the number of common neighbors, and its calculation formula is:

[0063]

[0064] Among them, {N(p) ∪ {p}} represents the set of node p and its first-order neighbor nodes, {N(q) ∪ {q}} represents the set of node q and its first-order neighbor nodes, the symbol ∩ represents finding the intersection of elements, the symbol ∪ represents finding the union of elements, and the symbol || represents counting the number of elements in the set. Due to the symmetry of the calculation process, the matrix S is a symmetric matrix.

[0065] Furthermore, based on the above graph neural network that includes all the time delay variables to be estimated, the present application proposes a time series registration algorithm based on the topological structure of the graph neural network. By calculating the weighted hop distance, the zero time delay reference column and the initial time delay interval of each variable in each iteration process are determined, which specifically includes the following steps:

[0066] Step2: Based on the adjacency matrix A and the node similarity matrix S of the graph structure G, define a weight matrix whose calculation formula is:

[0067]

[0068] Among them, the symbol represents element-wise multiplication, and the element W in W p,q represents the weight from node p to node q, which is based on the similarity between node p and node q and the direct connection between them. Since the matrix A is an asymmetric matrix and the matrix S is a symmetric matrix, the matrix W is an asymmetric matrix.

[0069] Step3: Initialize a weighted hop distance matrix The element D in D p,q represents the weighted hop distance from node p to node q. It is worth explaining that the hop distance is a concept describing the distance between two nodes in a graph network. It refers to the minimum number of edges required to reach one node from another node, that is, the number of edges on the shortest path between the two nodes.

[0070] Based on the weight matrix W, if there is a shortest path R between two nodes p,q , then calculate R p,qThe sum of the weights of all the edges above; if there is no connected path between two nodes, the hop distance between these two nodes can be considered to be infinite. The calculation formula is:

[0071]

[0072] where the symbols and represent the existence and non-existence of the shortest path respectively, and \(e_{ij}\in R\) represents the edge from node \(i\) to node \(j\) on the shortest path \(R\). It can be analyzed that the matrix \(D\) is asymmetric. i,j ∈R p,q represents the edge from node \(i\) to node \(j\) on the shortest path \(R\). It can be analyzed that the matrix \(D\) is asymmetric. p,q Step4: Then, according to the weighted hop distance matrix \(D\), calculate the average weighted hop distance from each node to all the other nodes, and regard the node corresponding to the minimum average weighted hop distance as the most "central" point in the graph structure \(G\), and regard the variable represented by this point as the zero-delay variable in the time series registration process.

[0073] Step4: Then, according to the weighted hop distance matrix \(D\), calculate the average weighted hop distance from each node to all the other nodes, and regard the node corresponding to the minimum average weighted hop distance as the most "central" point in the graph structure \(G\), and regard the variable represented by this point as the zero-delay variable in the time series registration process.

[0074] where the initial time delay interval is the search interval for the subsequent time series registration algorithm to search for the optimal time delay size. Based on the interval where the average weighted hop distance of each node is located, set a symmetric initial time delay interval for each variable. For example, the interval \([-T,T]\) means that the time delay size of this variable relative to the zero-delay variable is a certain value in the interval \([-T,T]\). The larger the average weighted hop distance, the longer the width of the interval. Among them, for the variable corresponding to the node with an average weighted hop distance of \(+\infty\), its initial time delay interval is the time interval with the longest width.

[0075] Step5: Finally, according to the particle swarm optimization algorithm, search for the optimal time delay size of each variable, and the optimization function is the grey correlation coefficient, and the time delay size of each variable in each iteration process can be obtained.

[0076] Furthermore, perform time series registration on the data set according to the time series registration algorithm based on the graph neural network topology structure proposed in this application, and obtain the time series registered data set for constructing the downstream task.

[0077] To sum up, the time series registration algorithm based on the graph neural network topology structure proposed in this application determines the zero-delay reference column in each calculation process and the initial time delay interval of each variable by innovatively calculating the weighted hop distance, accurately reflects the time delay set that can be positive or negative for each variable at the same physical moment, fully considers the physical meaning and time delay difference of each variable in the specific process, accurately calculates the time delay size of each time series, and makes the time series registration result more matching and suitable for the downstream task.

[0078] (3) Design the model loss function driven by the downstream task and guided by the knowledge graph

[0079] To combine the temporal registration process with downstream tasks, this application constructs downstream tasks based on the registered dataset and designs a model loss function driven by downstream tasks and guided by knowledge graphs to provide feedback guidance for the parameter update of graph neural network nodes and edges, thereby guiding the temporal registration process based on the topological structure of the graph neural network and making the temporal registration results more applicable to specific downstream tasks.

[0080] In this example, the model loss function LF designed in this application is as follows:

[0081] LF = L D + L Sm + L KG (9)

[0082] Among them, L D represents the performance of the downstream task. If the downstream task is a regression task, then L D is:

[0083]

[0084] In the formula, N is the number of samples, y i is the true value of the i-th sample, is the predicted value of the i-th sample. If the downstream task is a classification task, then L D is:

[0085]

[0086] In the formula, K is the number of sample categories, y i is the true probability distribution of the i-th category, taking values of 0 or 1, is the probability distribution when the model predicts the i-th category.

[0087] L Sm in the model loss function LF is the smoothness constraint of the graph structure. Based on the graph regularization theory, a good graph structure tends to be low-rank, sparse, and smooth. And according to the graph homogeneity hypothesis, similar nodes tend to build edges, and the changes between similar nodes are smooth. Therefore, a smoothness constraint L Sm is imposed, and its calculation formula is:

[0088]

[0089] In the formula, α is the regularization parameter, which is used to control the strength of the regularization term.

[0090] L KG in the model loss function LF is the constraint condition based on knowledge graph information, which includes the relationship constraint L Rel of triples and the transitivity constraint L TrTwo parts, and the calculation formulas are as follows:

[0091]

[0092] Among them, Equation (13) indicates that there is a relationship constraint for the triple (p, r, q) in the knowledge graph KG, that is, the relationship from node p to node q is r. Equation (14) represents the transitivity constraint of the same relationship type r in the knowledge graph KG, that is, the relationship from node p to node q is r, and the relationship from node q to node u is r, then the relationship from node p to node u is r. When and only when r is "negatively correlated", the relationship remains "negatively correlated (r)" after an odd number of transmissions, and after an even number of transmissions, the relationship will become "positively correlated (-r)" (see Equation (15)).

[0093]

[0094] In summary, the present application designs a model loss function driven by downstream tasks and guided by a knowledge graph. This function provides feedback guidance for the parameter update of the nodes and edges of the graph neural network, and then guides the temporal registration process based on the topological structure of the graph neural network, making the temporal registration result more in line with specific downstream tasks.

[0095] (4) Design an iterative update mechanism for the graph neural network structure

[0096] Based on the adjacency matrix and node similarity matrix of the graph neural network, the present invention designs an iterative update mechanism for the graph neural network structure, adaptively deletes or generates edges, plays a role in screening key variables, reduces redundant variables with coupling situations, and gradually obtains key variables and their associated relationships that match specific downstream tasks, improving the applicability of the temporal registration result to downstream tasks. The specific steps are as follows:

[0097] Step1: First, initialize the adjacency matrix A of the graph structure G = A 0 , A 0 The elements in represent the influence degree of node p on node q, and its calculation formula is:

[0098]

[0099] Step2: Then, design an iterative update mechanism for the graph neural network structure, including two parts: edge deletion and generation. Based on the node similarity matrix S, for the existing edges, judge the similarity S p,q between two nodes and the threshold σ 1 for deleting edges. The specific rules are as follows:

[0100]

[0101] If De is 1, keep this edge; if De is 0, delete this edge. For two nodes without edge connection, judge the similarity S between them p,q and the threshold σ for generating an edge 2 relation Ge, and the specific rules are as follows:

[0102]

[0103] If Ge is 1, generate a new directed edge between these two nodes; if Ge is 0, keep it unchanged.

[0104] Step3: After updating the edges, the topological structure of the graph structure G has changed, and the edge representation vector e p,q and the node embedding vectors z p and z q have all been updated.

[0105] Finally, update the adjacency matrix A of the graph structure G, and the update formula is:

[0106] A ep = λ·A ep-1 +(1 - λ)·S ep-1 (19)

[0107] where λ is the proportionality coefficient, ep is the index of the iteration process, A ep is the updated adjacency matrix, A ep-1 and S ep-1 are the adjacency matrix and the node similarity matrix of the previous iteration process respectively. When updating the adjacency matrix in Equation (19), the adjacency matrix and the similarity matrix of the previous iteration process are weighted and fused, which not only retains the topological information of the initial graph structure but also considers the similarity of nodes in the hidden space, thus better capturing the relationship between nodes.

[0108] When the model performance of the downstream task reaches the set threshold, the optimal graph neural network structure, latency estimation value, and the dataset after time series registration can be output, so as to better serve subsequent process modeling, optimization control, and performance evaluation, etc.

[0109] In summary, the iterative update mechanism of the graph neural network structure designed by the present invention can adaptively delete or generate edges, realizes the optimization of the network topological structure suitable for downstream tasks, provides a reliable network structure for the time series registration process, and improves the applicability of the time series registration result to downstream tasks.

[0110] In summary, the present application proposes a multi-sensor data temporal registration method based on an iteratively updated graph neural network. Aiming at the problem that the existing temporal registration methods do not consider the coupling relationship between multi-sensor data, resulting in inaccurate temporal registration, a domain knowledge graph is created based on the mechanism knowledge and expert experience of the industrial process, providing theoretical guidance and relationship information between multi-sensor data for temporal registration, and guiding the construction of the graph neural network. Aiming at the problems that the existing temporal registration methods assume that the zero-delay reference column is fixed, the initial delay intervals of all variables are the same, and the delay results are all positive, the present invention proposes a temporal registration algorithm based on the topological structure of the graph neural network, and determines the zero-delay reference column and the initial delay intervals of all variables in each calculation process by calculating the weighted hop distance. To solve the problem of only using the correlation of time series as the optimization index for temporal registration, the present invention designs a model loss function driven by a downstream task and guided by a knowledge graph, providing feedback guidance for the parameter update of the graph neural network nodes and edges, making the network information more suitable for specific downstream tasks. To establish the connection between temporal registration and downstream tasks, the present invention designs an iterative update mechanism of the graph neural network structure based on the adjacency matrix and node similarity matrix of the graph neural network, adaptively deleting or generating edges, providing a reliable network structure for temporal registration, and improving the applicability of the temporal registration result to downstream tasks. In summary, the method proposed by the present invention can accurately perform temporal registration on multi-sensor data, has the advantages of high credibility and high accuracy, and also provides an innovative idea for the temporal registration of multi-sensor data in the industrial process.

[0111] Next, a specific example is used to verify the effect of the above multi-sensor data temporal registration method based on an iteratively updated graph neural network as follows:

[0112] This implementation is carried out based on a 2650m 3 blast furnace of a domestic steel plant. First, based on the technological process, mechanism knowledge and expert experience of the blast furnace ironmaking process, a blast furnace ironmaking domain knowledge graph as Figure 3 shown is established, and the node information and relationship information related to the variables to be temporally registered are extracted, and a graph neural network G is established, where Figure 3 The partial enlarged view of Figures 4 - 5 is shown as

[0113] In this embodiment, the downstream task is the regression prediction of key process variables in blast furnace ironmaking. After the multi-sensor data set X is registered in time series, a scientific and reliable new data set X′ is obtained. The present invention trains a regression model based on X′ and predicts the silicon content parameter. Through strict experimental verification, good experimental results are achieved. Figure 6 , Figure 7 respectively show the curve graph and scatter plot between the predicted value and the true value. In terms of the performance evaluation indexes of the model, the root mean square error is 0.0123, the mean absolute percentage error is 1.4770, the fitting coefficient is 0.9998, the proportion of the error of the predicted value within 5% of the true value is 94.80%, and the proportion within 10% is 99.86%. The experimental results fully prove that the method proposed by the present invention performs well in the specific regression task, can meet the requirements of on-site work, and provides scientific and reliable decision-making support for on-site staff.

[0114] The embodiment of the present application also provides a multi-sensor data time series registration system based on an iterative update graph neural network, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0115] The multi-sensor data time series registration system based on the iterative update graph neural network can implement each embodiment of the above method and can achieve the same beneficial effects. Here, it will not be elaborated.

[0116] The above has described in detail the preferred specific embodiments of the present application. It should be understood that those of ordinary skill in the art can make many modifications and variations according to the concept of the present application without creative work. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field of the present application based on the concept of the present application through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the protection scope determined by the claims.

Claims

1. A multi-sensor data temporal registration method based on iterative updating graph neural network, characterized in that: include: S1: Build a domain knowledge graph with each sensor as a node and the relationship between sensors as an edge; S2: Initialize the graph neural network topology structure based on the graph topology structure of the domain knowledge graph; wherein each node in the graph neural network topology structure corresponds to each entity in the domain knowledge graph, each edge in the graph neural network topology structure corresponds to the relationship between nodes in the domain knowledge graph, and the probability that there are multiple directed edges between two nodes is greater than 0; S3: Determine a temporal registration algorithm based on the graph neural network topology structure, and register the data set based on the temporal registration algorithm to obtain a registered data set; S4: constructing a downstream task based on the registered data set, and constructing a model loss function based on the downstream task; S5: Design an iterative update mechanism for the topology of a graph neural network; S6: Perform iterative update training based on the model loss function and iterative update mechanism until the set conditions are met to obtain the final trained graph neural network structure; S7: Input the multi-sensor data collected in real time into the finally trained graph neural network structure for time series alignment.

2. The multi-sensor data temporal registration method based on iterative updating graph neural network according to claim 1 is characterized in that: The S2 includes: Assume the time series is Contains N samples and M variables. The data contained in nodes p and q are and Suppose the graph structure of the graph neural network is G = (V, E), where E represents the edge set, e p,q ∈E represents the edge vector from node p to node q. The edges corresponding to the same relationship type have the same initialization size, while the edges corresponding to different relationship types have different initialization sizes, and e p,q Set to trainable parameters, and e p,q The probability of containing multiple values ​​is greater than 0, indicating multiple edges between two nodes. V represents the node set, and the embedding vector of node p is z p , and its calculation formula satisfies the following relationship: In the formula, and are trainable weight vectors, N(p) represents the first-order neighbor nodes of node p, which provide relationship information with node p, q∈N(p) represents that node q is the first-order neighbor node of node p, x p and x q are the time series data of node p and node q respectively, p,q Represents the edge vector from node p to node q. Let the adjacency matrix of the graph neural network be represents the influence of node p on node q, |V| represents the number of elements in the node set, and the elements in matrix A corresponding to unconnected node pairs are 0, so matrix A is an asymmetric matrix, and the symbol It represents element-by-element multiplication and ReLU is the activation function.

3. The multi-sensor data temporal registration method based on iterative updating graph neural network according to claim 1 is characterized in that: The S3 includes: S31: Calculate the node similarity matrix of the graph neural network topology; S32: Calculate a weight matrix based on the graph structure of the graph neural network topology structure and the node similarity matrix, and determine a weighted hop distance matrix based on the weight matrix; S33: Determine the delay size of each variable in each iteration process of the time series registration process based on the weighted hop distance matrix, and perform time series registration on the data set based on the delay size of each variable and the initial delay interval of each variable to obtain a registered data set.

4. The multi-sensor data temporal registration method based on iterative updating graph neural network according to claim 3 is characterized in that: The S31 includes: Let the similarity between node p and node q be S p,q , the calculation formula satisfies the following relationship: in, and They represent the cosine similarity and neighbor similarity of node p and node q respectively.

5. The multi-sensor data temporal registration method based on iterative updating graph neural network according to claim 3 is characterized in that: The S32 includes: Step 1: Define a weight matrix based on the adjacency matrix A of the graph structure G of the graph neural network topology and the node similarity matrix S The calculation formula satisfies the following relationship: Among them, the symbol Represents element-wise multiplication; Step 2: Initialize a weighted hop distance matrix Based on the weight matrix W, if there is a shortest path R between two nodes p,q , then calculate R p,q The sum of the weights of all the edges on the node; if there is no connected path between two nodes, the hop distance between the two nodes is considered to be infinite, and the calculation formula satisfies the following relationship: Among them, the symbol and Respectively indicate the existence or non-existence of the shortest path, e i,j ∈R p,q Represents the shortest path R p,q The edge from node i to node j on .

6. The multi-sensor data temporal registration method based on iterative updating graph neural network according to claim 3 is characterized in that: The S33 comprises: The average weighted hop distance from each node to all other nodes is solved according to the weighted hop distance matrix, and the node corresponding to the minimum average weighted hop distance is regarded as the most central point in the graph structure, and the variable represented by the most central point is regarded as the zero-delay variable in the time series alignment process. The time series corresponding to the zero-delay variable is used as the zero-delay reference column, and the initial delay interval is determined; The optimal delay size of each zero-delay variable is searched according to the particle swarm algorithm, and the grey correlation coefficient is used as the optimization function to obtain the delay size of each variable and the initial delay interval of each variable in each iteration process; Based on the time delay size of each variable and the initial time delay interval of each variable, the data set is time-series aligned to obtain the aligned data set.

7. The multi-sensor data temporal registration method based on iterative updating graph neural network according to claim 1 is characterized in that: The model loss function in S4 satisfies the following relationship: LF=L D +L Sm +L KG (5) Among them, L D represents the performance of the downstream task, L Sm is the smoothness constraint of the graph structure, L KG are constraints based on knowledge graph information; If the downstream task is a regression task, then L D Satisfies the following relationship: In the formula, N is the number of samples, y i is the true value of the i-th sample, is the predicted value of the i-th sample; If the downstream task is a classification task, then L D Satisfies the following relationship: In the formula, K is the number of sample categories, y i is the true probability distribution of the i-th category, which takes the value of 0 or 1. The probability distribution when the model predicts the i-th category; The smoothness constraint L Sm Satisfies the following relationship: In the formula, α is the regularization parameter, which is used to control the strength of the regularization term. represents the influence of node p on node q, z p is the embedding vector of node p, z q is the embedding vector of node q; The constraint condition based on the knowledge graph information includes the relationship constraint L of the triple Rel and the transitive constraint L of the same relation type Tr , the calculation formulas satisfy the following relationships: Among them, formula (13) indicates that there is a relational constraint on the triple (p, r, q) in the knowledge graph KG, that is, the relationship from node p to node q is r, and formula (14) indicates the transitive constraint of the same relation type r in the knowledge graph KG, that is, the relationship from node p to node q is r, the relationship from node q to node u is r, then the relationship from node p to node u is r, if and only if r is negatively correlated, the relationship is still negatively correlated (r) after an odd number of transfers, and becomes positively correlated (-r) after an even number of transfers, as shown in the following relational formula:

8. The multi-sensor data temporal registration method based on iterative updating graph neural network according to claim 7 is characterized in that: The S5 includes: Initialize the adjacency matrix A=A of the graph structure G of the graph neural network topology 0 , A 0 Elements in It represents the influence degree of node p on node q, and its calculation formula satisfies the following relationship: Design an iterative update mechanism for the topological structure of the graph neural network, including the deletion and generation of edges. Based on the node similarity matrix S, for existing edges, determine the similarity S between two nodes. p,q The relationship between De and the threshold σ1 for deleting edges is as follows: If De is 1, keep this edge; if De is 0, delete this edge; For two nodes that do not have an edge connection, determine the similarity S between them p,q The relationship Ge with the threshold σ2 of generating edges, the specific rules are as follows: If Ge is 1, a new directed edge is generated between the two nodes; if Ge is 0, it remains unchanged; Update the adjacency matrix A of the graph structure G. The update formula satisfies the following relationship: A ep =λ·A ep-1 +(1-λ)·S ep-1 (15) Among them, λ is the proportionality coefficient, ep is the index of the iteration process, A ep is the updated adjacency matrix, A ep-1 and S ep-1 They are the adjacency matrix and node similarity matrix of the previous iteration process respectively. When updating the adjacency matrix, the adjacency matrix and similarity matrix of the previous iteration process are weightedly fused; The graph structure is updated based on the above update formula and model loss function. When the performance of the downstream task meets the set threshold, the final trained graph neural network structure is obtained.

9. A multi-sensor data temporal registration system based on iterative update graph neural network, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 8 are implemented.

Images