Industrial internet defect data estimation method based on graph signal convolutional neural network

Abstract

The invention provides an industrial internet defect data estimation method based on a graph signal convolutional neural network, and belongs to the technical field of industrial internet, and the method comprises the steps: constructing a sample structure graph reflecting the industrial internet, and carrying out the preprocessing of data, and obtaining a training set and a test set; training the optimized graph neural network model by using the training set to obtain a trained graph neural network model; and evaluating the test set by using the trained graph neural network model to obtain a relative error between a data completion value and an original value, and completing prediction completion of node features. According to the method, the problem that the multi-dimensional features cannot be effectively complemented and predicted while the prediction accuracy is ensured in the prior art is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial Internet, and in particular relates to an industrial Internet missing data estimation method based on graph signal convolutional neural network. Background Art

[0002] In the application scenarios of the Industrial Internet, certain key parameters are crucial. If the data of these parameters is missing or abnormal, the security risk will be greatly increased. If effective safety protection measures are not taken, various safety accidents may occur, resulting in serious consequences such as casualties and property losses. Therefore, ensuring the integrity and accuracy of these key parameters is the key to maintaining the security of the Industrial Internet system. In industrial scenarios, the detection process is often not limited to the monitoring of a single parameter, but adopts a comprehensive detection method with multiple dimensions and multiple points. In order to more accurately predict and analyze these complex relationships, the industry currently generally uses graph neural networks to construct and analyze the relationships between multiple nodes.

[0003] Graph neural network is a highly flexible neural network architecture that can process graph structured data. By learning the interaction patterns of nodes and edges in the graph structure, it can deeply understand the topological structure of the data and extract valuable feature information. This ability makes graph neural networks particularly suitable for processing data with non-Euclidean structures, such as complex systems such as social networks, molecular structures, and transportation networks. In these scenarios, GNNs have demonstrated their unique advantages in capturing and analyzing complex relationships, providing powerful tools for data analysis, pattern recognition, and predictive modeling. However, in industrial environments, parameter detection is usually not targeted at a single parameter, but involves a comprehensive consideration of multidimensional features. In this context, how to effectively complete and predict these multidimensional features while ensuring prediction accuracy has become an urgent problem to be solved. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides an industrial Internet missing data estimation method based on graph signal convolutional neural network, which solves the problem in the prior art that it is impossible to effectively complete and predict multi-dimensional features while ensuring prediction accuracy.

[0005] In order to achieve the above objectives, the technical solution adopted by the present invention is: a method for estimating missing data of industrial Internet based on graph signal convolutional neural network, comprising the following steps:

[0006] S1. Construct a sample structure diagram reflecting the industrial Internet and preprocess the data to obtain a training set and a test set. In the training set, some data is randomly hidden to simulate data missing, and the nodes in the sample structure diagram are processed for dimensionality reduction.

[0007] S2. Use the training set to train the optimized graph neural network model to obtain a trained graph neural network model;

[0008] S3. Use the test set to evaluate the trained graph neural network model and complete the estimation of missing data for the industrial Internet.

[0009] The beneficial effects of the present invention are as follows: the present invention constructs an improved graph neural network model, and comprehensively evaluates the improved graph neural network model by adopting appropriate evaluation indicators to verify the prediction accuracy and reliability of the model. Finally, the test set data is input into the trained graph neural network model, and the relative error between the predicted data and the original data is calculated to evaluate the accuracy of the prediction results. The present invention is applicable to a variety of industrial scenarios, and can provide an effective solution for the prediction and completion of multi-dimensional and multi-feature data, solving the problem in the prior art that it is not possible to effectively complete and predict multi-dimensional features while ensuring prediction accuracy.

[0010] Furthermore, the step S1 includes the following steps:

[0011] S101, identifying and defining nodes and edges in the sample structure graph;

[0012] S102, constructing an adjacency matrix according to the relationship between nodes and edges to represent the connection status between nodes;

[0013] S103, constructing a feature matrix for each node;

[0014] S104, integrating the adjacency matrix and the feature matrix to obtain a sample structure diagram reflecting the industrial Internet;

[0015] S105. According to the sample structure diagram, the labels corresponding to the nodes are split by using a direct learning method, and the label masks are reordered to obtain a training set and a test set, wherein some data are randomly hidden in the training set to simulate data missing, and at the same time, the nodes in the sample structure diagram are subjected to dimensionality reduction processing.

[0016] The beneficial effect of the above further solution is that the present invention integrates the relationship between nodes and edges to construct a graph that can reflect the sample structure of the industrial Internet. This graph can not only capture the complex relationship between sensors, but also provide a basis for data analysis and feature learning through its structured representation, and divide its data set into training set and test set, providing a basis for model training and evaluation.

[0017] Furthermore, step S2 includes the following steps:

[0018] S201, constructing a graph neural network model;

[0019] S202. Setting an activation function, and using the training set to train the optimized graph neural network model to obtain a trained graph neural network model, wherein gradient clipping is introduced during the training process.

[0020] The beneficial effect of the above further scheme is that the present invention optimizes the network model structure and activation function under the improved graph neural network framework, and introduces a gradient clipping mechanism to prevent gradient explosion and ensure the stability of the model training process. In addition, an additional regularization loss term is added to the model to prevent overfitting and improve the generalization ability of the model.

[0021] Furthermore, the construction of the graph neural network model is specifically as follows:

[0022] Three layers of graph convolution layers are defined, where the first layer maps the input feature space to the intermediate feature space; the second layer reduces the dimension of the intermediate feature space, and the third layer maps the intermediate features after dimension reduction back to the original feature space; batch normalization and activation functions are introduced after each layer of convolution layers;

[0023] During the back-propagation process, the optimizer is introduced to update the weights of the graph neural network model, and the loss function of the graph neural network model is set to complete the construction of the graph neural network model.

[0024] The beneficial effect of the above further scheme is that the present invention introduces batch normalization and activation function after each graph convolution layer, which not only helps to speed up the training speed of the model, but also improves the stability of the model. Batch normalization normalizes the input of the layer so that the input of each layer maintains a similar distribution, thereby reducing the problem of internal covariate shift; while the activation function introduces nonlinearity, allowing the model to learn more complex patterns. At the same time, regularization penalizes the complexity of the model, so that the model tries to maintain a low complexity while pursuing the minimization of training error, thereby improving the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0026] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0027] Example

[0028] like Figure 1 As shown, the present invention provides an industrial Internet missing data estimation method based on graph signal convolutional neural network, and its implementation method is as follows:

[0029] S1. Construct a sample structure diagram reflecting the industrial Internet, and preprocess the data to obtain a training set and a test set. In the training set, some data are randomly hidden to simulate data missing, and the nodes in the sample structure diagram are processed by dimensionality reduction. The implementation method is as follows:

[0030] S101, identifying and defining nodes and edges in the sample structure graph;

[0031] S102, constructing an adjacency matrix according to the relationship between nodes and edges to represent the connection status between nodes;

[0032] S103, constructing a feature matrix for each node;

[0033] S104, integrating the adjacency matrix and the feature matrix to obtain a sample structure diagram reflecting the industrial Internet;

[0034] S105. According to the sample structure diagram, the labels corresponding to the nodes are split by using a direct learning method, and the label masks are reordered to obtain a training set and a test set, wherein some data are randomly hidden in the training set to simulate data missing, and at the same time, the nodes in the sample structure diagram are subjected to dimensionality reduction processing.

[0035] In this embodiment, each sensor can be regarded as a node in the graph, and the relationship between sensors is represented by the edges of the graph. Each node not only represents a sensor, but may also have multiple characteristic attributes associated with it. In this example, each node covers two different characteristic values, and these nodes are distributed differently in space, and different edge relationships can be constructed according to scenarios or different communication mechanisms. By integrating the relationships between these nodes and edges, a graph that reflects the sample structure of the Industrial Internet is constructed. This graph can not only capture the complex relationships between sensors, but also provide a basis for data analysis and feature learning through its structured representation.

[0036] In this embodiment, the data is preprocessed, and the graph of the industrial Internet can be represented in the form of an adjacency matrix or an edge list, such as representing the graph corresponding to the physical scene of the water pumping station in the form of an adjacency matrix or an edge list. The adjacency matrix is ​​a two-dimensional matrix whose rows and columns correspond to the nodes in the graph, and the connection relationship between the nodes can be intuitively expressed through the elements in the matrix. The data set is processed, and the training set and the test set are divided into a ratio of 7:3. Some data are randomly hidden in the training set to simulate data missing. At the same time, the node features are processed by dimensionality reduction, so that it retains key information while removing redundancy, so that the model can process data more efficiently and reduce the risk of overfitting.

[0037] In this embodiment, in the analysis of a single sample, the structure of the graph is predefined and unchanged, which means that the relationship between nodes and the characteristics of the nodes themselves are fixed. Each sample corresponds to a specific application scenario, and the design of its graph structure and feature matrix is ​​intended to reflect the unique needs of this scenario.

[0038] In this embodiment, in order to accurately construct a graph neural network model, it is necessary to identify the correlation between nodes according to the actual application scenario. This involves an in-depth understanding of the scenario and accurate identification of potential connections between nodes. Once the correlation between nodes is determined, edge relationship data corresponding to these node numbers can be generated to construct a graph structure that can truly reflect the characteristics of the scenario. Specifically, the graph neural network model is constructed based on the relationship between nodes and edges imported according to the data structure, and the training set is divided, including the following steps:

[0039] S1.1 Construction of Figure:

[0040] S1.1.1. Node and edge identification: Identify and define the nodes and edges in the graph, ensuring that each node and edge corresponds to the actual network architecture;

[0041] S1.1.2, Adjacency Matrix Construction: Based on the relationship between nodes and edges, an adjacency matrix is ​​constructed to represent the connection status between nodes. 0 and 1 are used to represent the association between nodes: 0 means there is no direct connection between the two nodes, while 1 means there is some form of association between them;

[0042] S1.1.3, Feature Matrix Construction: Construct a feature matrix for each node, which contains the key feature information of the node;

[0043] S1.1.4. Graph initialization: Integrate the constructed adjacency matrix and feature matrix, initialize the graph model, and prepare for subsequent training and prediction.

[0044] S1.2. The division of graphs is not like files or pictures, because different nodes are connected by different edges. One of the biggest problems with direct division is information leakage. The present invention chooses a transductive learning method to ensure that the input graph is complete in all split data sets (training set, validation set and test set). This means that although the labels corresponding to the nodes are split, the topological structure of the entire graph is still retained. This method allows the model to observe the complete graph structure during training, validation and testing, so as to better learn and understand the complex relationships between nodes. At the same time, in order to minimize information leakage between different data sets, the present invention reorders the label masks. This step comprehensively considers the correlation between nodes and minimizes the direct connection between the training set, validation set and test set by rearranging the label masks. Such processing not only reduces the risk of data leakage, but also helps to improve the performance and generalization ability of the model on unknown data.

[0045] S2. Use the training set to train the optimized graph neural network model to obtain the trained graph neural network model. The implementation method is as follows:

[0046] S201, constructing a graph neural network model;

[0047] S202. Setting an activation function, and using the training set to train the optimized graph neural network model to obtain a trained graph neural network model, wherein gradient clipping is introduced during the training process.

[0048] The construction of the graph neural network model is specifically as follows:

[0049] Three layers of graph convolution layers are defined, where the first layer maps the input feature space to the intermediate feature space; the second layer reduces the dimension of the intermediate feature space, and the third layer maps the intermediate features after dimension reduction back to the original feature space; batch normalization and activation functions are introduced after each layer of convolution layers;

[0050] During the back-propagation process, the optimizer is introduced to update the weights of the graph neural network model, and the loss function of the graph neural network model is set to complete the construction of the graph neural network model.

[0051] In this embodiment, the training set is input into the optimized graph neural network model GCNs for training to obtain the trained model weights; the GCNs model is improved on the basic framework of GNN.

[0052] In this embodiment, the construction, optimization and activation function setting of the specific graph neural network model include the following steps:

[0053] S2.1. Build a graph neural network model:

[0054] S2.1.1. When initializing the graph neural network model, three graph convolution layers are first defined, which are responsible for feature conversion at different stages. The first layer maps from the input feature space to the intermediate feature space, the second layer further reduces the feature dimension, and the last layer maps the feature back to the original feature space. This design allows the model to capture and learn complex patterns of graph data at different levels. After each layer of graph convolution, the model introduces batch normalization to speed up the training process, reduce internal covariate shift, and improve the stability of model parameter updates. The LeakyReLU activation function is used to introduce nonlinearity, where the negative slope parameter is controlled by negative_slope. This helps alleviate the gradient vanishing problem and allows the model to learn more complex feature representations;

[0055] S2.1.2 During the back propagation process, the AdamW optimizer is introduced to more effectively update the weight parameters of the model. The Adam optimizer dynamically adjusts the learning rate of each parameter by calculating the exponential moving average of the first-order moment (average value) and the second-order moment (uncentered variance) of the historical gradient. This adaptive learning rate adjustment strategy can accelerate the convergence of the model during training. However, when the L2 regularization term is included in the loss function, the optimization effect of the Adam optimizer will be affected. The L2 regularization term is intended to prevent model overfitting by penalizing larger weight values, and the variable learning rate characteristics of Adam may cause the difference between the weight parameters with larger gradients and the weight parameters with smaller gradients to further expand, which is contrary to the original intention of L2 regularization. In order to overcome the limitations of Adam under L2 regularization, the present invention uses the AdamW optimizer. AdamW improves L2 regularization on the basis of Adam, separates the L2 regularization term from the loss function, and directly applies it to the update of weight parameters. In this way, the regularization term is no longer affected by the adaptive learning rate of the Adam optimizer, thereby ensuring the stability of the regularization effect. In addition, the AdamW optimizer has the advantages of hyperparameter independence and increased regularization strength, which enables the model to more effectively prevent overfitting and accelerate convergence during training. Its principle is:

[0056]

[0057] Where mt and vt are the estimates of the first and second moments of the gradient, respectively. β1 and β2 are the exponential decay rates that control the estimates of these two moments. φ is a very small number to prevent division by zero. t represents the current iteration number, which is used for bias correction. gt represents the gradient obtained by derivation of the objective function. and Indicates bias correction, that is, biasing towards 0 in the case of zero initial value. and Represent the tth power of β1 and β2 respectively. wt+1 represents the parameter to be solved, η is the learning rate, which is used to control the step size, λ represents the weight decay factor, and the coefficient β1 controls the weight distribution, especially the momentum and the current gradient, with an exponential decay rate. It is usually set close to 1, and the default value is 0.9. The coefficient β2 controls the exponential decay rate and manages the influence of the previous squared gradient. Similar to the RMSProp algorithm, it calculates the weighted average of the squared gradient, with a default value of 0.999. λ is the weight decay factor, which is usually set to 0.005. Since the initial value m0 is set to 0, mt tends to be 0 in the early stages of training. Therefore, it is necessary to correct the deviation of the mean gradient mt to reduce its influence on the early stages of training. Similarly, since v0 is initialized to 0, vt also tends to be biased towards 0 in the initial stage of training and needs to be corrected. Finally, the parameters are updated, and the default learning rate in the present invention is 0.002. In addition, in order to prevent the occurrence of gradient explosion during training, gradient clipping is paid special attention in the design of graph neural network models. This measure ensures that the gradient remains within a reasonable range, thereby avoiding model instability caused by excessive gradients and further improving the robustness and accuracy of the model.

[0058] S2.1.3. After multiple experimental verifications, the mean square error (MSE) function was selected as the loss function of the graph neural network model because it showed the best fitting effect in this case. It is the average of the squared difference between the predicted value f(x) and the target value y:

[0059]

[0060] Among them, n represents the total number of single-type data, and i represents the mask of the current data.

[0061] S2.2, activation function setting:

[0062] Considering the limitations of the ReLU function in terms of unilateral inhibition characteristics, the LeakyReLU function (a variant of ReLU) was selected as the activation function in the neural network design. This choice is due to the excellent performance of the LeakyReLU function in solving the gradient vanishing problem. Specifically, when the input is positive, the output of the LeakyReLU function remains the same as the input, while its gradient is 1, which helps to maintain the stability of the gradient during propagation. When the output is negative, the LeakyReLU function allows a certain amount of information to pass through, unlike the ReLU function which completely suppresses negative inputs, avoiding the phenomenon of neuron failure. It is achieved by setting the output to λx, and when λ is a hyperparameter greater than 0, the input is negative. In practical applications, λ is usually set to positive to ensure that information can still be properly transmitted when the input is negative. The principle is:

[0063]

[0064] Among them, x represents the input data, and λ represents the hyperparameter, which is usually set to 0.2. This can prevent the "neuron death" phenomenon in ReLU from occurring. The gradient is as follows:

[0065]

[0066] S3. Use the test set to evaluate the performance and generalization ability of the trained graph neural network model, obtain the relative error between the data completion value and the original value, and complete the estimation of the missing data of the industrial Internet.

[0067] In this embodiment, the relative error is used as an evaluation index to evaluate the model; the test set is input into the trained graph neural network model to obtain the relative error between the data completion value and the original value. A smaller relative error means that the model can provide a completion result that is closer to the original data, thereby verifying the effectiveness of the model in practical applications.

[0068] In this embodiment, the present invention uses the correlation error RE and the Pearson correlation coefficient PCC to test the prediction ability of the model, and its principle is as follows:

[0069]

[0070]

[0071] Among them, R represents the true value, P represents the predicted value, and PCC represents the correlation between data under the same node. Specifically, n represents the total number of single-type data, i represents the mask of the current data, and X i With Y i Represents the true value under the current mask, and They represent the sample means of different data types respectively.

[0072] In this embodiment, by carefully adjusting the correlation coefficient, the correlation of data in various situations can be simulated and restored, thereby ensuring that the restored data is both accurate and reliable.

[0073] In this embodiment, the training set is input into the graph neural network model for training. The experimental hardware configuration is NVIDIA GeForce GTX 4060, Intel Core i9-13950HX@2.20GHz processor, memory 32G, operating system Windows11, deep learning framework Pytorch, CUDA version 11.6, Python version 3.10. The training batch is 20, the initial learning rate is 0.002, and 1000 epochs are trained to obtain the trained model weights. When PCC is 0.5, after 1000 iterations, the loss value gradually decreases from the initial 0.6927 through the increase and decrease of the number of training times, and finally stabilizes at about 0.02378. The value is small and the training loss is low, indicating that the error between the predicted data and the actual data of the graph neural network model is small, and the generated training results are relatively accurate.

Claims

1. A method for estimating missing data of industrial Internet based on graph signal convolutional neural network, characterized in that: The following steps are involved: S1. Construct a sample structure diagram reflecting the industrial Internet and preprocess the data to obtain a training set and a test set. In the training set, some data is randomly hidden to simulate data missing, and the nodes in the sample structure diagram are processed for dimensionality reduction. S2. Use the training set to train the optimized graph neural network model to obtain a trained graph neural network model; S3. Use the test set to evaluate the trained graph neural network model, obtain the relative error between the data completion value and the original value, and complete the estimation of the missing data of the industrial Internet.

2. According to claim 1, the method for estimating industrial Internet defective data based on graph signal convolutional neural network is characterized in that: The step S1 comprises the following steps: S101, identifying and defining nodes and edges in the sample structure graph; S102, constructing an adjacency matrix according to the relationship between nodes and edges to represent the connection status between nodes; S103, constructing a feature matrix for each node; S104, integrating the adjacency matrix and the feature matrix to obtain a sample structure diagram reflecting the industrial Internet; S105. According to the sample structure diagram, the labels corresponding to the nodes are split by using a direct learning method, and the label masks are reordered to obtain a training set and a test set, wherein some data are randomly hidden in the training set to simulate data missing, and at the same time, the nodes in the sample structure diagram are subjected to dimensionality reduction processing.

3. The method for estimating industrial Internet defective data based on graph signal convolutional neural network according to claim 1 is characterized in that: The step S2 comprises the following steps: S201, constructing a graph neural network model; S202. Setting an activation function, and using the training set to train the optimized graph neural network model to obtain a trained graph neural network model, wherein gradient clipping is introduced during the training process.

4. The method for estimating industrial Internet defective data based on graph signal convolutional neural network according to claim 3 is characterized in that: The construction of the graph neural network model is specifically as follows: Three layers of graph convolution layers are defined, where the first layer maps the input feature space to the intermediate feature space; the second layer reduces the dimension of the intermediate feature space, and the third layer maps the intermediate features after dimension reduction back to the original feature space; batch normalization and activation functions are introduced after each layer of convolution layers; During the back-propagation process, the optimizer is introduced to update the weights of the graph neural network model, and the loss function of the graph neural network model is set to complete the construction of the graph neural network model.

Images