Hybrid polynomial neural network-based industrial control device security state identification method

Abstract

The embodiment of the application relates to the technical field of industrial control equipment safety state recognition, and particularly relates to an industrial control equipment safety state recognition method based on a hybrid polynomial neural network, which comprises the following steps: collecting operation monitoring data of an industrial control equipment and dividing the operation monitoring data into a training set and a test set; configuring global hyperparameters according to the training set; constructing a composite feature extraction unit and a polynomial mapping unit according to the global hyperparameters, and mixing and stacking the composite feature extraction unit and the polynomial mapping unit into a multi-layer network structure; inputting the test set into the network structure; performing feature extraction and high-order nonlinear mapping layer by layer; and outputting a safety state recognition result. The coupling strength between the composite feature extraction unit and the polynomial mapping unit is adaptively adjusted, the error of the polynomial mapping unit is fed back to the parameter updating of the composite feature extraction unit, two-stage coupled collaborative training is realized, and the recognition accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of industrial control equipment safety status identification technology, and in particular to a method for industrial control equipment safety status identification based on a hybrid multinomial neural network. Background Technology

[0002] The safety status of industrial control equipment (ICS) refers to the overall health and controllability of the equipment's physical processes, control logic, and network communication during operation. Specifically, it can be categorized into several typical states, including normal, faulty, and those subjected to various network attacks (such as data tampering, denial-of-service attacks, and malicious command injection). Given the highly heterogeneous and complexly coupled nature of industrial control systems, the operational data characteristics of ICS are often difficult to represent and identify stably and effectively using traditional machine learning models due to strong temporal correlations, strong noise interference, and the deep coupling between physical processes and the information space. Currently, ICS safety status identification has become a research hotspot in industrial internet security and cyberspace security. Based on existing research both domestically and internationally, the main challenges in ICS safety status identification lie in feature extraction and the speed and accuracy of identification.

[0003] In terms of feature extraction, industrial control equipment operation data contains two key features: long-dependent temporal features and high-order nonlinear features. However, in existing research, most methods struggle to effectively capture both types of features in tandem, typically focusing on only one, resulting in incomplete feature representation and affecting the robustness of state identification.

[0004] In terms of recognition speed and accuracy, the commonly used recognition systems have weak generalization ability and poor adaptability when facing complex and ever-changing industrial control operating environments, making it difficult to achieve the expected recognition accuracy.

[0005] Therefore, how to effectively integrate the temporal and nonlinear characteristics of industrial control equipment operation data to improve the accuracy and robustness of industrial control equipment safety status identification has become an urgent research problem to be solved. Summary of the Invention

[0006] To address this, the present invention provides a method for identifying the safety status of industrial control equipment based on a hybrid multinomial neural network, which overcomes the problem that existing methods for identifying the safety status of industrial control equipment are unable to fully extract complex temporal features and effectively integrate nonlinear mapping relationships, resulting in low identification accuracy.

[0007] To achieve the above objectives, this invention provides a method for identifying the safety status of industrial control equipment based on a hybrid multinomial neural network. It includes: Step S1: Collect the operation monitoring data of the industrial control equipment, and divide the operation monitoring data into a training set and a test set; Step S2: Based on the data size and feature dimension of the training set, and in conjunction with the preset recognition performance target, configure the global hyperparameters of the hybrid multinomial neural network. The global hyperparameters include at least the structural parameters of the composite feature extraction unit, the nonlinear mapping order and regularization coefficient of the multinomial mapping unit, and the layer size parameter of the multilayer network structure. Step S3: Based on the structural parameters in the global hyperparameters, construct a composite feature extraction unit for extracting the temporal features of the operation monitoring data; Step S4: Based on the nonlinear mapping order and regularization coefficient in the global hyperparameters, construct a polynomial mapping unit for performing high-order nonlinear mapping on the output of the composite feature extraction unit. Step S5: Based on the layer size parameter in the global hyperparameters, the composite feature extraction unit and the multinomial mapping unit are mixed and stacked into a multi-layer network structure. Step S6: Input the test set into the multi-layer network structure. The multi-layer network structure performs feature extraction and high-order nonlinear mapping on the test set layer by layer, and outputs the safety status identification result of the industrial control equipment.

[0008] Further, in step S1, the operation monitoring data is divided into a training set and a test set, including: Step S11: Obtain the operation monitoring data and its corresponding status labels. The status labels are used to identify the actual safety status category of the industrial control equipment at the time of data collection, and determine the original dimension and total sample size of the operation monitoring data. Step S12: Based on the original dimensions and total sample size, preprocess the operation monitoring data to obtain preprocessed operation monitoring data; Step S13: Based on the total sample size of the preprocessed operation monitoring data, the preprocessed operation monitoring data is randomly divided into a training set and a test set according to a preset ratio.

[0009] Further, in step S2, the global hyperparameters of the hybrid multinomial neural network are configured, including: Step S21: Based on the feature dimensions of the training set, configure the number of input variables of the composite feature extraction unit and the number of input variables of the polynomial mapping unit, and configure the nonlinear mapping order of the polynomial mapping unit. Step S22: Based on the temporal length and feature change complexity of the training set, configure the number of convolutional output channels and the number of recurrent hidden units inside the composite feature extraction unit to ensure sufficient extraction of temporal features, and configure the learning rate and dropout rate during model training. Step S23: Based on the total number of samples and feature complexity of the training set, configure the total number of layers of the multi-layer network structure, the number of neurons to be constructed in each layer, and the maximum number of candidate neurons that can be retained in each layer. Step S24: Based on the total number of samples and feature dimensions of the training set, configure the regularization strength coefficient of the polynomial mapping unit when solving its internal mapping coefficients.

[0010] Further, in step S3, a composite feature extraction unit for extracting the time-series features of the operational monitoring data is constructed, including: Step S31: Based on the number of input variables in the global hyperparameters, construct the input layer of the composite feature extraction unit, and the input layer receives the operation monitoring data; Step S32: Based on the number of convolution output channels in the global hyperparameters, a convolution module is constructed. The convolution module performs local feature extraction and dimension compression on the running monitoring data and outputs a compressed feature sequence. Step S33: Based on the output dimension of the convolutional module and the number of recurrent hidden units in the global hyperparameters, a bidirectional long short-term memory module is constructed. The bidirectional long short-term memory module takes the compressed feature sequence as input, extracts temporal dependencies along the forward and backward time axes respectively, and outputs a hidden state sequence containing forward and backward context information. Step S34: Based on the hidden state sequence output by the bidirectional long short-term memory module, an attention module is constructed. The attention module scores the importance of the hidden states at each time step and assigns corresponding weights. Step S35: The input layer, the convolution module, the bidirectional long short-term memory module, and the attention module are sequentially connected in series to form the composite feature extraction unit.

[0011] Further, in step S4, a polynomial mapping unit is constructed for performing high-order nonlinear mapping on the output of the composite feature extraction unit, including: Step S41: Based on the nonlinear mapping order in the global hyperparameters, a polynomial mapping module is constructed. The polynomial mapping module takes the output of the composite feature extraction unit as input, performs polynomial expansion on each dimension of the input features according to the nonlinear mapping order, and generates a high-dimensional feature set containing the original features and product terms of each order. Step S42: Based on the regularization strength coefficient in the global hyperparameters, the mapping coefficients of the high-dimensional feature set are solved by the least squares method with regularization constraints to obtain the mapping coefficient vector, and the initial output of the polynomial mapping module is calculated based on the mapping coefficient vector and the high-dimensional feature set. Step S43: The deviation between the initial output of the polynomial mapping module and the corresponding state label is taken as the first error, and the deviation between the output of the composite feature extraction unit and the corresponding state label is taken as the second error; based on the comparison of the change rates of the first error and the second error, the coupling strength between the parameter update of the composite feature extraction unit and the solution of the mapping coefficients of the polynomial mapping unit is adjusted, so that the parameters of the composite feature extraction unit are updated in a direction that can simultaneously reduce the first error.

[0012] Further, in step S43, adjusting the coupling strength between the parameter update of the composite feature extraction unit and the solution of the mapping coefficients of the polynomial mapping unit includes: Monitor the changing trends of the first error and the second error in multiple consecutive training rounds, and calculate the changing rates of the first error and the second error respectively; The rate of change of the first error is compared with the rate of change of the second error. If the rate of change of the first error is greater than the rate of change of the second error, the coupling strength between the parameter update of the composite feature extraction unit and the solution of the mapping coefficients of the polynomial mapping unit is enhanced. If the rate of change of the first error is less than or equal to the rate of change of the second error, the current coupling strength is weakened or maintained. Based on the adjusted coupling strength, the parameters of the composite feature extraction unit are updated so that the output of the composite feature extraction unit changes in a direction that can simultaneously reduce the first error.

[0013] Further, in step S5, the composite feature extraction unit and the polynomial mapping unit are mixed and stacked into a multi-layer network structure, including: Step S51: Using the training set as the input of the first layer, multiple hybrid neurons are constructed in parallel in the first layer. Each hybrid neuron consists of a composite feature extraction unit and a multinomial mapping unit connected in series. A gating mechanism is introduced when the composite feature extraction unit transmits information to the multinomial mapping unit. The gating mechanism dynamically adjusts the selection ratio of information transmission according to the current recognition error. Step S52: Evaluate the recognition performance of the hybrid neurons in each layer. Based on the upper limit of the number of candidate neurons in the global hyperparameters, select the output of the composite feature extraction unit that meets the preset standard as candidate nodes. Combine the selected candidate nodes as the input of the next layer. Step S53: Repeat the process of constructing hybrid neurons and selecting candidate nodes until the total number of network layers in the global hyperparameter is reached; Step S54: Stack the layers sequentially according to the data flow direction to form the multi-layer network structure.

[0014] Further, in step S6, the safety status identification result of the industrial control equipment is output, including: The final output of the multi-layer network structure is used as the safety status identification result of the industrial control equipment. The security status identification results are compared one by one with the corresponding real status labels in the test set to determine the percentage of correctly identified samples out of the total number of samples in the test set, and the percentage is output as the identification accuracy.

[0015] Furthermore, the convolution module in the composite feature extraction unit adopts a one-dimensional convolution kernel, which slides along the time axis to extract local features from the data within each time window. The length of the output feature sequence of the convolution module is less than the time step length of the input data.

[0016] Furthermore, the attention module in the composite feature extraction unit adopts an additive attention mechanism. The additive attention mechanism scores the hidden state at each time step through a learnable feedforward network, and the scoring results are normalized to obtain the attention weight at each time step. The attention weight is weighted and summed with the corresponding hidden state to obtain the weighted feature vector.

[0017] Further, in step S2, the preset recognition performance target includes at least one of recognition accuracy threshold, recall threshold, and F1 score threshold. The configuration process of the global hyperparameters is an iterative optimization process. When the recognition performance of the training set under the initially configured global hyperparameters does not reach the preset recognition performance target, the global hyperparameters are adjusted by Bayesian search until the preset recognition performance target is reached or the preset maximum number of iterations is reached.

[0018] Furthermore, in step S3, the composite feature extraction unit further includes a batch normalization module and an activation function module. The batch normalization module is placed after the convolution module and before the bidirectional long short-term memory module, and is used to normalize the feature sequence output by the convolution module. The activation function module uses a modified linear unit to perform nonlinear activation on the output of the batch normalization module, and inputs the activated feature sequence into the bidirectional long short-term memory module.

[0019] Further, in step S42, the objective function of the least squares method with regularization constraints is: to minimize the sum of the mean square error and the regularization term between the initial output of the polynomial mapping module and the corresponding state label, wherein the regularization term is the product of the square of the L2 norm of the mapping coefficient vector and the regularization strength coefficient; and to solve the objective function to obtain the closed-form analytical solution of the mapping coefficient vector.

[0020] Furthermore, the evaluation of the recognition performance of the hybrid neurons in each layer in step S52 specifically includes: calculating the recognition accuracy of each hybrid neuron on the validation set and sorting them from high to low according to the recognition accuracy; retaining a specified number of hybrid neurons at the top of the ranking as candidate nodes according to the upper limit of the number of candidate neurons in the global hyperparameters, and discarding the remaining hybrid neurons; and concatenating the outputs of the composite feature extraction units of each hybrid neuron in the retained candidate nodes to form the input feature vector of the next layer.

[0021] Furthermore, the gating mechanism is a learnable soft gating mechanism. The soft gating mechanism generates a continuous gating value between 0 and 1 based on the input features of the current layer and the recognition error of the previous layer. The gating value is multiplied by the output of the composite feature extraction unit and then passed to the polynomial mapping unit. The parameters of the soft gating mechanism are jointly optimized with the parameters of the composite feature extraction unit and the mapping coefficients of the polynomial mapping unit during the model training process.

[0022] Furthermore, the industrial control equipment includes at least one of a programmable logic controller, a distributed control system, a remote terminal unit, and an intelligent electronic device; the operation monitoring data includes at least one of the operation log data, network traffic data, process variable data, and system call sequence data of the industrial control equipment; the safety status identification results include three categories: normal status, abnormal status, and fault status.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows: the embodiments of the present invention acquire the operation monitoring data of industrial control equipment and its corresponding status labels, perform missing value filling, outlier correction and normalization preprocessing, and randomly divide the data into non-overlapping training and test sets according to a preset ratio, which ensures the data quality and the independence of the division, and provides a reliable data foundation for the subsequent training and performance evaluation of the hybrid multinomial neural network. At the same time, the specific division ratio clarifies the data allocation relationship between model learning and verification, ensuring the objectivity and credibility of the recognition performance evaluation.

[0024] Furthermore, this embodiment of the invention configures global hyperparameters based on the specific data features of the training set, so that the input dimension of the composite feature extraction unit matches the feature dimension of the training set. It ensures the full extraction of temporal features by setting the number of convolutional output channels and the number of recurrent hidden units. It controls the model's convergence speed and generalization ability by setting the learning rate and dropout rate. It balances model complexity and computational efficiency by setting the number of network layers and the number of neurons per layer. It suppresses the overfitting risk of high-order nonlinear mapping by setting regularization coefficients. Thus, it lays a unified parameter foundation for the construction and collaborative training of hybrid multinomial neural networks.

[0025] Furthermore, this embodiment of the invention uses a convolution module to extract local features and compress dimensions of the input data, reducing data dimensionality while preserving key spatial features; it uses a bidirectional long short-term memory module to extract temporal dependencies along the forward and backward time axes, fully utilizing the forward and backward contextual information in the monitoring data; it uses an attention module to automatically assign weights to each time step, strengthening the contribution of key time steps to the recognition results; and it achieves end-to-end extraction from raw monitoring data to temporal feature vectors by sequentially combining the modules to form a complete composite feature extraction unit, providing temporally rich feature inputs for the subsequent high-order nonlinear mapping of the polynomial mapping unit.

[0026] Furthermore, in this embodiment of the invention, the output of the composite feature extraction unit is mapped to a high-dimensional feature space through polynomial expansion. The mapping coefficients are solved using the least squares method with regularization constraints to suppress the overfitting risk of high-order mapping. By monitoring the comparison of the error change rates of the polynomial mapping unit and the composite feature extraction unit, the coupling strength between the two is dynamically adjusted so that the parameter update direction of the composite feature extraction unit is related to the error change of the polynomial mapping unit. This achieves two-stage coupled collaborative training, overcoming the defects of unidirectional information flow and inability to dynamically match in traditional independent training or simple stacking, thereby improving the overall recognition performance.

[0027] Furthermore, this embodiment of the invention dynamically adjusts the transmission ratio of the output of the composite feature extraction unit to the multinomial mapping unit based on the current recognition error through a gating mechanism. This preserves effective temporal features while shielding redundant interference information, thus improving the purity of information transmission within the layer. By evaluating the performance of each layer of hybrid neurons and selecting candidate nodes to pass to the next layer, the computational scale of each layer is controlled while ensuring network depth. By stacking each layer sequentially according to the data flow direction to form a multi-layer network structure, the feature representation is abstracted layer by layer from low to high level. Finally, the output of the multinomial mapping unit with the best recognition performance is used as the recognition result of the entire network, thereby improving the accuracy and robustness of safety status recognition of industrial control equipment.

[0028] Furthermore, this embodiment of the invention directly uses the final output of the multi-layer network structure as the security status identification result, realizing end-to-end identification from raw monitoring data to security status category; by comparing the identification result with the real status label one by one and calculating the identification accuracy, it provides an objective and quantifiable evaluation index for the model identification performance, which facilitates the comparison and optimization of model performance under different parameter configurations.

[0029] Furthermore, in this embodiment of the invention, local temporal features are extracted and sequence length is compressed by sliding a one-dimensional convolutional kernel along the time axis, reducing the computational overhead of the subsequent bidirectional long short-term memory module. The implicit states of each time step output by the bidirectional long short-term memory module are automatically scored and weighted by an additive attention mechanism, enabling the model to adaptively focus on key time steps that contribute more to safety status identification and suppress interference from irrelevant or redundant time steps. This improves the ability to capture key temporal information in the monitoring data of industrial control equipment operation and further enhances the accuracy of safety status identification. Attached Figure Description

[0030] Figure 1 This is a flowchart of the industrial control equipment safety status identification method based on a hybrid multinomial neural network according to an embodiment of this application; Figure 2 This is a structural diagram of the hybrid polynomial neural network in the industrial control equipment safety status identification method based on the hybrid polynomial neural network in the embodiments of this application. Detailed Implementation

[0031] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0032] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0033] Please see Figures 1-2 As shown, Figure 1 This is a flowchart of the industrial control equipment safety status identification method based on a hybrid multinomial neural network according to an embodiment of this application; Figure 2 This is a structural diagram of the hybrid polynomial neural network in the industrial control equipment safety status identification method based on the hybrid polynomial neural network in the embodiments of this application.

[0034] The technical solution provided in this application includes the following steps: Step S1: Collect the operation monitoring data of the industrial control equipment, and divide the operation monitoring data into a training set and a test set; Step S2: Based on the data size and feature dimension of the training set, and in conjunction with the preset recognition performance target, configure the global hyperparameters of the hybrid multinomial neural network. The global hyperparameters include at least the structural parameters of the composite feature extraction unit, the nonlinear mapping order and regularization coefficient of the multinomial mapping unit, and the layer size parameter of the multilayer network structure. Step S3: Based on the structural parameters in the global hyperparameters, construct a composite feature extraction unit for extracting the temporal features of the operation monitoring data; Step S4: Based on the nonlinear mapping order and regularization coefficient in the global hyperparameters, construct a polynomial mapping unit for performing high-order nonlinear mapping on the output of the composite feature extraction unit. Step S5: Based on the layer size parameter in the global hyperparameters, the composite feature extraction unit and the multinomial mapping unit are mixed and stacked into a multi-layer network structure. Step S6: Input the test set into the multi-layer network structure. The multi-layer network structure performs feature extraction and high-order nonlinear mapping on the test set layer by layer, and outputs the safety status identification result of the industrial control equipment.

[0035] Specifically, in step S1, the operation monitoring data is divided into a training set and a test set, including: Step S11: Obtain the operation monitoring data and its corresponding status labels. The status labels are used to identify the actual safety status category of the industrial control equipment at the time of data collection, and determine the original dimension and total sample size of the operation monitoring data. Step S12: Based on the original dimensions and total sample size, preprocess the operation monitoring data to obtain preprocessed operation monitoring data; Step S13: Based on the total sample size of the preprocessed operation monitoring data, the preprocessed operation monitoring data is randomly divided into a training set and a test set according to a preset ratio.

[0036] Specifically, based on the distribution ratio of each category of samples in the training set, the sample weights of each category are calculated, and the training set is augmented by combining synthetic minority class oversampling with time series truncation enhancement to obtain an amplified training set. The time series truncation enhancement strategy is used to randomly truncate, scale the time axis, and randomly mask the channels of the time series samples in the training set to generate diverse variant samples. The variant samples are then merged with the original samples to expand the sample size and distribution diversity of the training set.

[0037] In this embodiment of the invention, after dividing the training set and the test set, the number of samples in the training set for the three categories of normal state, abnormal state, and fault state is counted. If the number of abnormal state and fault state samples is found to be significantly less than the number of normal state samples, the sample weight for each category is calculated, and this weight is inversely proportional to the number of samples in that category. Subsequently, a synthetic minority class oversampling technique is used to process the time series data: in the feature space, the minority class samples and their nearest neighbor samples are linearly interpolated to generate new synthetic samples. At the same time, time series truncation enhancement is performed: a subsequence of 70% to 100% of the length is randomly truncated from each time series sample, and the time axis is scaled by a factor of 0.9 to 1.1. Then, a mask (zeroing) is applied to 10% to 20% of the randomly selected feature channels. The variant samples generated by the above enhancement operations are merged with the original samples to form the amplified training set.

[0038] In this embodiment of the invention, operational monitoring data collected during the continuous operation of an industrial control system is acquired. This data includes time-series signals from multiple monitoring channels, with each sampling time corresponding to a set of monitoring readings. Simultaneously, a safety status label marked at each sampling time is acquired. Through data reading and parsing, the original dimension of the operational monitoring data is determined to contain 44 feature variables at each sampling time, with the total sample size equal to the total number of all sampling times. After data acquisition, preprocessing operations are performed: missing values ​​in the operational monitoring data are filled using the average of adjacent times; outliers exceeding the normal range are replaced using a statistical threshold-based correction method; and to address the issue of significant differences in the dimensions of different feature variables, normalization is used to unify each feature to the same numerical range, ensuring that the preprocessed data dimension remains consistent with the original dimension.

[0039] Finally, all preprocessed sample data were randomly divided in a 7:3 ratio, with 70% of the samples used as the training set for model parameter learning and the remaining 30% used as the test set for performance evaluation after model training. The training and test sets did not overlap. This completed the preprocessing and dataset partitioning of the industrial control equipment operation monitoring data.

[0040] This invention acquires the operation monitoring data of industrial control equipment and its corresponding status labels, performs missing value filling, outlier correction and normalization preprocessing, and randomly divides it into non-overlapping training and test sets according to a preset ratio. This ensures data quality and the independence of the division, providing a reliable data foundation for the subsequent training and performance evaluation of the hybrid multinomial neural network. At the same time, the specific division ratio clarifies the data allocation relationship between model learning and verification, ensuring the objectivity and credibility of the recognition performance evaluation.

[0041] Specifically, when the operational monitoring data contains multiple data types, a corresponding modal encoder is constructed for each data type. The modal encoder includes at least one of a text encoder, a temporal encoder, and a graph encoder. The modal feature vectors output by each modal encoder are concatenated along the feature dimension to form a multimodal fusion feature vector, and the multimodal fusion feature vector is used as the input of the composite feature extraction unit. During the training process of the hybrid multinomial neural network, a cross-modal contrastive loss is introduced. The cross-modal contrastive loss makes positive sample pairs of different modalities at the same time approach each other in the feature space, while negative sample pairs at different times or of different categories move away from each other.

[0042] In this embodiment of the invention, three types of modal encoders are constructed for industrial control scenarios that simultaneously collect operation log data (text-based), process variable data (time-series), and network traffic data (graph structure-based). For operation log data, a lightweight text encoder trained by prediction is used to convert log events into a fixed-length semantic vector sequence. For process variable data, the convolutional module and bidirectional long short-term memory module in the composite feature extraction unit are used for time-series feature encoding. For network traffic data, a graph attention network is used to encode the communication topology graph composed of source address and destination address, extracting the structural features of nodes and edges. The modal feature vectors output by each encoder are aligned in the time dimension and then concatenated in the feature dimension to form a unified multimodal fusion feature vector. This vector serves as the input to the attention module in the subsequent composite feature extraction unit for cross-modal global temporal dependency modeling. During training, in addition to the original classification loss, a cross-modal contrast loss is introduced: for the same sampling time, the feature vectors output by the text encoder, temporal encoder and graph encoder are paired as positive samples to narrow the cosine distance; for samples with different time or different state labels, their modal feature vectors are paired as negative samples to widen the cosine distance.

[0043] Specifically, in step S2, the global hyperparameters of the hybrid multinomial neural network are configured, including: Step S21: Based on the feature dimensions of the training set, configure the number of input variables of the composite feature extraction unit and the number of input variables of the polynomial mapping unit, and configure the nonlinear mapping order of the polynomial mapping unit. Step S22: Based on the temporal length and feature change complexity of the training set, configure the number of convolutional output channels and the number of recurrent hidden units inside the composite feature extraction unit to ensure sufficient extraction of temporal features, and configure the learning rate and dropout rate during model training. Step S23: Based on the total number of samples and feature complexity of the training set, configure the total number of layers of the multi-layer network structure, the number of neurons to be constructed in each layer, and the maximum number of candidate neurons that can be retained in each layer. Step S24: Based on the total number of samples and feature dimensions of the training set, configure the regularization strength coefficient of the polynomial mapping unit when solving its internal mapping coefficients.

[0044] In this embodiment of the invention, the parameters of the hybrid multinomial neural network are set, and the specific steps are as follows: Set the number of input variables for CBA neurons and multinomial neurons. Set the polynomial order of the polynomial neuron. Number of output channels of CBA neurons Number of hidden units in CBA neurons Learning rate of CBA neurons The dropout value for CBA neurons is 0.01. Set the number of neurons built in each layer of the model. ; Set the number of model layers ; Set the maximum number of neurons in each layer. ; Set the L2 regularization coefficients for the method of estimating polynomial parameters (L2-LSE). .

[0045] This invention, through configuring global hyperparameters based on the specific data characteristics of the training set, matches the input dimension of the composite feature extraction unit with the feature dimension of the training set. It ensures the full extraction of temporal features by setting the number of convolutional output channels and the number of recurrent hidden units. It controls the model's convergence speed and generalization ability by setting the learning rate and dropout rate. It balances model complexity and computational efficiency by setting the number of network layers and the number of neurons per layer. It suppresses the overfitting risk of high-order nonlinear mappings by setting regularization coefficients. Thus, it lays a unified parameter foundation for the construction and collaborative training of hybrid multinomial neural networks.

[0046] Specifically, in step S3, a composite feature extraction unit for extracting the time-series features of the operational monitoring data is constructed, including: Step S31: Based on the number of input variables in the global hyperparameters, construct the input layer of the composite feature extraction unit, and the input layer receives the operation monitoring data; Step S32: Based on the number of convolution output channels in the global hyperparameters, a convolution module is constructed. The convolution module performs local feature extraction and dimension compression on the running monitoring data and outputs a compressed feature sequence. Step S33: Based on the output dimension of the convolutional module and the number of recurrent hidden units in the global hyperparameters, a bidirectional long short-term memory module is constructed. The bidirectional long short-term memory module takes the compressed feature sequence as input, extracts temporal dependencies along the forward and backward time axes respectively, and outputs a hidden state sequence containing forward and backward context information. Step S34: Based on the hidden state sequence output by the bidirectional long short-term memory module, an attention module is constructed. The attention module scores the importance of the hidden states at each time step and assigns corresponding weights. Step S35: The input layer, the convolution module, the bidirectional long short-term memory module, and the attention module are sequentially connected in series to form the composite feature extraction unit.

[0047] In this embodiment of the invention, an input layer is constructed based on the number of input variables in the global hyperparameters. This input layer receives runtime monitoring data and passes it to subsequent modules. A convolutional module is constructed based on the number of convolutional output channels in the global hyperparameters. This convolutional module uses a one-dimensional convolutional kernel to perform a sliding window operation on the input data along the time axis, extracting local features within each time window and outputting a compressed feature sequence through convolution operations, thereby reducing the data dimensionality. A bidirectional long short-term memory (LSTM) module is constructed based on the output dimension of the convolutional module and the number of recurrent hidden units in the global hyperparameters. This bidirectional LSM module takes the compressed feature sequence as input and extracts temporal dependencies along both the forward and reverse time axes, outputting a hidden state sequence containing forward and backward contextual information. An attention module is constructed based on the hidden state sequence output by the bidirectional LSM module. This attention module scores the importance of the hidden states at each time step, converts the scores into weights for each time step through normalization, and performs a weighted summation of the hidden states at each time step to obtain a summarized feature vector. Finally, the input layer, convolution module, bidirectional long short-term memory module, and attention module are sequentially connected in series according to the data flow direction to form a complete composite feature extraction unit.

[0048] This invention employs a convolution module to extract local features and compress dimensions from the input data, reducing data dimensionality while preserving key spatial features. A bidirectional long short-term memory module extracts temporal dependencies along the forward and backward time axes, fully utilizing the forward and backward contextual information in the monitoring data. An attention module automatically assigns weights to each time step, enhancing the contribution of key time steps to the recognition results. By sequentially connecting and combining these modules to form a complete composite feature extraction unit, end-to-end extraction from raw monitoring data to temporal feature vectors is achieved, providing temporally rich feature inputs for the subsequent high-order nonlinear mapping of the polynomial mapping unit.

[0049] Specifically, in step S4, a polynomial mapping unit is constructed for performing high-order nonlinear mapping on the output of the composite feature extraction unit, including: Step S41: Based on the nonlinear mapping order in the global hyperparameters, a polynomial mapping module is constructed. The polynomial mapping module takes the output of the composite feature extraction unit as input, performs polynomial expansion on each dimension of the input features according to the nonlinear mapping order, and generates a high-dimensional feature set containing the original features and product terms of each order. Step S42: Based on the regularization strength coefficient in the global hyperparameters, the mapping coefficients of the high-dimensional feature set are solved by the least squares method with regularization constraints to obtain the mapping coefficient vector, and the initial output of the polynomial mapping module is calculated based on the mapping coefficient vector and the high-dimensional feature set. Step S43: The deviation between the initial output of the polynomial mapping module and the corresponding state label is taken as the first error, and the deviation between the output of the composite feature extraction unit and the corresponding state label is taken as the second error; based on the comparison of the change rates of the first error and the second error, the coupling strength between the parameter update of the composite feature extraction unit and the solution of the mapping coefficients of the polynomial mapping unit is adjusted, so that the parameters of the composite feature extraction unit are updated in a direction that can simultaneously reduce the first error.

[0050] Specifically, in step S43, adjusting the coupling strength between the parameter update of the composite feature extraction unit and the solution of the mapping coefficients of the polynomial mapping unit includes: Monitor the changing trends of the first error and the second error in multiple consecutive training rounds, and calculate the changing rates of the first error and the second error respectively; The rate of change of the first error is compared with the rate of change of the second error. If the rate of change of the first error is greater than the rate of change of the second error, the coupling strength between the parameter update of the composite feature extraction unit and the solution of the mapping coefficients of the polynomial mapping unit is enhanced. If the rate of change of the first error is less than or equal to the rate of change of the second error, the current coupling strength is weakened or maintained. Based on the adjusted coupling strength, the parameters of the composite feature extraction unit are updated so that the output of the composite feature extraction unit changes in a direction that can simultaneously reduce the first error.

[0051] In this embodiment of the invention, a multinomial neuron is constructed, and the specific steps are as follows: A multinomial neuron is designed, which is composed of multinomial functions. These functions map the input to a high-dimensional space to describe the high-order nonlinear relationship between the input and output. Parameter estimation is performed using least squares with L2 norm regularization to estimate the parameters of the formed features, and the output of the multinomial neuron is derived from these parameters. Dual parameter estimation is then performed: after estimating the parameters of the multinomial neuron, the error information from the second stage (multinomial neuron) is fed back to the parameter update process of the first stage (CBA neuron). Specifically, a joint loss function is defined. ,in For binary cross-entropy loss, For the output error of the polynomial neuron, This is an adaptively adjustable regularization coefficient. This coefficient is dynamically adjusted based on the rate of change of the error in the two stages: when the polynomial error increases, The parameters are automatically increased, forcing the CBA neurons to adjust in the direction of compensating for the error; conversely, they are decreased. This achieves coupled optimization of the parameters in the two stages, enabling the multinomial neurons and the CBA network to train collaboratively. This overcomes the shortcomings of traditional independent training or simple stacking, which involves unidirectional information flow and the inability to dynamically match, thereby improving the overall recognition performance.

[0052] This invention maps the output of the composite feature extraction unit to a high-dimensional feature space through polynomial expansion, and uses the least squares method with regularization constraints to solve the mapping coefficients, suppressing the risk of overfitting in high-order mapping. By monitoring the rate of change of errors between the polynomial mapping unit and the composite feature extraction unit, the coupling strength between them is dynamically adjusted, so that the parameter update direction of the composite feature extraction unit is related to the error change of the polynomial mapping unit, realizing two-stage coupled collaborative training. This overcomes the defects of unidirectional information flow and inability to dynamically match in traditional independent training or simple stacking, thereby improving the overall recognition performance.

[0053] Specifically, in step S5, the composite feature extraction unit and the multinomial mapping unit are mixed and stacked into a multi-layer network structure, including: Step S51: Using the training set as the input of the first layer, multiple hybrid neurons are constructed in parallel in the first layer. Each hybrid neuron consists of a composite feature extraction unit and a multinomial mapping unit connected in series. A gating mechanism is introduced when the composite feature extraction unit transmits information to the multinomial mapping unit. The gating mechanism dynamically adjusts the selection ratio of information transmission according to the current recognition error. Step S52: Evaluate the recognition performance of the hybrid neurons in each layer. Based on the upper limit of the number of candidate neurons in the global hyperparameters, select the output of the composite feature extraction unit that meets the preset standard as candidate nodes. Combine the selected candidate nodes as the input of the next layer. Step S53: Repeat the process of constructing hybrid neurons and selecting candidate nodes until the total number of network layers in the global hyperparameter is reached; Step S54: Stack the layers sequentially according to the data flow direction to form the multi-layer network structure.

[0054] In this embodiment of the invention, a hybrid multinomial neural network is constructed, with the following specific steps: The model input is designed by using the constructed training and test sets as inputs to the hybrid multinomial neural network; the first layer of the model is constructed by determining the number of CBA neurons and multinomial neurons, along with their respective parameters, to create the first layer of hybrid neurons. When the output of the first layer is transmitted to the multinomial neurons, a gating mechanism is introduced to control the information flow. This gating mechanism includes a learnable gating vector. ,in For the output of CBA neurons, The sigmoid function. Gated vector. Element-by-element regulation The proportion flowing to the multinomial neuron enables information feedback and selective transmission between the two stages: it allows the main temporal features to enter the nonlinear mapping module, while dynamically shielding redundant or interfering information based on the current recognition error.

[0055] Design candidate nodes, sort the outputs of CBA neurons in the first layer of the model, and use parameters The steps for identifying candidate neurons are shown in Table 1. The second layer of the model is designed by using the selected candidate neurons from the first layer as input to the second layer, and utilizing... , and structure Here, we'll take the construction of the first polynomial neuron as an example: Assumption If the input is a multinomial neuron, then the final estimated output of the multinomial neuron is... It can be described as:

[0056] in, , and These are the coefficients of the polynomial function. Next, a least squares method based on L2 norm regularization is used to... The parameter estimation is performed using the L2 norm regularization expression as follows:

[0057] in, The regularization coefficient is . These are the coefficients of the polynomial function. It is the actual output. This is the output of the multinomial neuron. By differentiating it, we can obtain the expression for the analytical solution:

[0058] in, Given an input matrix containing constant terms, For the output vector, The regularization coefficient is . This is the parameter vector. Finally, the output of the multinomial neuron is calculated using the estimated parameters.

[0059] Design higher-level models and repeat until the number of model levels reaches a defined parameter. The model output is designed to use the multinomial neuron with the best recognition accuracy in the last layer of the model as the model output.

[0060] Table 1

[0061] Specifically, the multi-layer network structure is divided into an edge execution segment and a cloud execution segment. The edge execution segment contains the first few layers of the multi-layer network structure, and the cloud execution segment contains the remaining layers. During the training phase, the training set is distributed to edge nodes and cloud nodes. The edge nodes complete the forward and backward propagation of the first few layers and upload the intermediate gradients to the cloud. The cloud nodes summarize the gradients, update the global parameters, and send the updated edge layer parameters back to the edge nodes. During the inference phase, the edge nodes perform forward computation on the real-time collected operation monitoring data, compress the calculated intermediate feature vectors, and upload them to the cloud. The cloud execution segment completes the computation of the remaining layers and returns the final recognition result.

[0062] In this embodiment of the invention, the constructed multi-layer network structure is divided into layers based on the computing power difference between the edge computing nodes (such as industrial gateways) and the cloud server in the industrial control field. The dividing point is set as the third layer, that is, the first three layers (including the composite feature extraction unit and the first polynomial mapping unit) are deployed on the edge nodes, and the fourth to Nth layers are deployed on the cloud server. During the training phase, the edge nodes use locally collected real-time data to perform forward propagation of the first three layers, and package the intermediate feature vectors output by the third layer and the corresponding local gradients and upload them to the cloud; the cloud server summarizes the gradient information of all edge nodes, performs global parameter updates, and distributes the updated parameters of the first three layers to each edge node to achieve distributed collaborative training. During the inference phase, the edge nodes only need to perform lightweight calculations of the first three layers on the real-time monitoring data, obtain the intermediate feature vectors, and then upload them to the cloud after data compression (such as using lossy compression to reduce transmission bandwidth). The cloud completes the calculation of all subsequent layers and sends the safety status identification results back to the edge nodes and the industrial control monitoring platform.

[0063] This invention employs a gating mechanism to dynamically adjust the transmission ratio of the composite feature extraction unit's output to the multinomial mapping unit based on the current recognition error. This preserves effective temporal features while shielding redundant interference information, improving the purity of information transmission within layers. By evaluating the performance of each layer's hybrid neurons and selecting candidate nodes to pass to the next layer, the computational scale of each layer is controlled while maintaining network depth. By stacking layers sequentially according to the data flow direction to form a multi-layer network structure, feature representation is abstracted layer by layer from low to high level. Finally, the output of the multinomial mapping unit with the best recognition performance is used as the recognition result of the entire network, thereby improving the accuracy and robustness of safety status recognition for industrial control equipment.

[0064] Specifically, in step S6, the safety status identification result of the industrial control equipment is output, including: The final output of the multi-layer network structure is used as the safety status identification result of the industrial control equipment. The security status identification results are compared one by one with the corresponding real status labels in the test set to determine the percentage of correctly identified samples out of the total number of samples in the test set, and the percentage is output as the identification accuracy.

[0065] Specifically, while outputting the security status identification result, based on the multinomial mapping unit in the multilayer network structure, the kernelized Shapley value interpretation method is used to calculate the feature contribution of each feature dimension in the operation monitoring data to the current identification result; the feature contributions are sorted according to their absolute values, and several key features with the highest ranking are selected to generate an interpretability report containing the key feature names, contribution symbols, and contribution values, and the interpretability report is output together with the security status identification result.

[0066] In this embodiment of the invention, after the multi-layer network structure outputs the category identification for the test set samples, the interpretability analysis process is initiated. For the current test sample, it is input into the trained multinomial mapping unit. Based on the gradient of the unit's output relative to the input features (i.e., the feature vector output by the composite feature extraction unit), and combined with the kernelized Shapley value interpretation method, a Shapley value is calculated for each dimension of the input feature vector. This value quantifies the marginal contribution of that dimension's feature to the final identification result (e.g., a "fault state" determination). Subsequently, the Shapley values ​​of each feature dimension are sorted from largest to smallest by their absolute value. The top 10 original monitoring variables with the largest contributions (e.g., temperature, flow rate, specific network port communication volume, etc.) are selected, and the contribution sign (positive promotion or negative inhibition) and specific value of each variable are recorded to form a structured interpretability report. Finally, the identification result and the interpretability report are output together to the monitoring interface of the industrial control system.

[0067] Specifically, after the multi-layer network structure is deployed and running, new operational monitoring data of industrial control equipment is collected in real time, the feature distribution statistics of the new data are extracted, and the feature distribution statistics are compared with the historical feature distribution statistics of the training set to calculate the distribution offset. When the distribution offset exceeds a preset drift threshold, the model adaptive update process is triggered: the parameters of the composite feature extraction unit in the multi-layer network structure are frozen, the mapping coefficients of the polynomial mapping unit are re-estimated online only with the new data, and the mapping coefficients before and after the update are smoothly fused using a moving average strategy to obtain the updated multi-layer network structure.

[0068] In this embodiment of the invention, after the hybrid multinomial neural network is deployed to the actual industrial control environment, newly added operational monitoring data is collected every fixed time window (e.g., every hour). First, the mean and covariance matrix of the new data on the output layer of the composite feature extraction unit are calculated, and compared with the baseline feature distribution recorded during the training phase (i.e., the mean and covariance of all samples in the training set output by the composite feature extraction unit). Mahalanobis distance is used as a quantitative measure of distribution offset. When the distance exceeds a preset drift threshold (e.g., 3.0), it is determined that the current operating condition has changed significantly. At this time, an adaptive update process is triggered: all learnable parameters of the composite feature extraction unit are frozen, and only the new data is input into the unit to obtain the new feature vector. Then, the least squares method with regularization constraints is used, combined with the new feature vector and the corresponding state label, to resolve the mapping coefficients of the multinomial mapping unit. To avoid coefficient mutations leading to unstable recognition, an exponential moving average strategy is used to fuse the old and new coefficients, i.e., the updated coefficient = 0.7 × old coefficient + 0.3 × new coefficient. Thus, the model can quickly adapt to new operating conditions without retraining all parameters. It maintains the learned general temporal feature extraction capability and adapts to the drift of the operating environment through local adjustment of the polynomial mapping coefficients, which significantly improves the robustness and continuous recognition accuracy of the model in long-term deployment.

[0069] In this embodiment of the invention, the specific process for outputting the security status identification result of industrial control equipment is as follows: The trained multi-layer network structure is processed on the test set. The multi-layer network structure performs feature extraction and high-order nonlinear mapping on the test set layer by layer. Finally, the multinomial mapping unit with the best identification performance in the last layer outputs the security status identification result of the industrial control equipment. This identification result is the determination of whether the industrial control equipment is in normal operation or a specific type of network attack state at the current monitoring time. The above security status identification result is compared with the pre-labeled real state labels in the test set one by one. For each test sample, if the state category output by the model is consistent with its corresponding real state label, the sample is determined to be correctly identified; otherwise, it is determined to be incorrectly identified. The number of correctly identified samples in all test samples is counted, and the percentage of this number to the total number of test set samples is calculated. This percentage is used as the identification accuracy output of the hybrid multinomial neural network model to measure the model's performance in identifying the security status of industrial control equipment.

[0070] This invention achieves end-to-end identification of security status categories from raw monitoring data by directly using the final output of the multi-layer network structure as the security status identification result. By comparing the identification result with the real status label one by one and calculating the identification accuracy, it provides an objective and quantifiable evaluation index for the model identification performance, which facilitates the comparison and optimization of model performance under different parameter configurations.

[0071] Specifically, after the multi-layer network structure has been deployed and is running, when the cumulative number of newly added labeled samples reaches a preset batch size, an incremental learning process is initiated to calculate the importance weight of each parameter in the multi-layer network structure to the original recognition task. During the incremental learning process, the importance weight is used as an elastic constraint to apply a regularization penalty proportional to the importance to the parameter update. At the same time, the mapping coefficients of the polynomial mapping unit are given a lower importance weight to allow them to adapt to new data more flexibly. This enables the absorption of knowledge from new samples without forgetting the original recognition capabilities, thus completing the hot update of the model.

[0072] In this embodiment of the invention, after three months of deployment and operation of the hybrid multinomial neural network, maintenance personnel have collected 5,000 new operational monitoring data points with status labels. At this point, the incremental learning process is initiated. First, the trained multi-layer network structure is used to calculate the diagonal elements of the Fisher information matrix for each parameter on the original training set. These diagonal elements represent the importance weight of that parameter to the original recognition task. In the loss function of incremental learning, in addition to the recognition error of the newly added samples, an extra regularization term is added: this term is the sum of the products of the square of the current update amount of each parameter and its importance weight, i.e., the elastic weight consolidation term. This constraint ensures that when updating parameters, the parameters that are more important to the original task have their changes suppressed more significantly; and since the mapping coefficients of the multinomial mapping unit are structurally located at the end of the network, their Fisher information values ​​are usually small, thus they are given greater freedom of change during incremental updates to fully fit the new distribution patterns in the new samples. Finally, a limited number of training rounds are performed on the 5,000 newly added samples until the incremental loss converges, completing the hot update of the model.

[0073] Specifically, the convolution module in the composite feature extraction unit uses a one-dimensional convolution kernel. The one-dimensional convolution kernel slides along the time axis to extract local features from the data within each time window. The length of the output feature sequence of the convolution module is less than the time step length of the input data.

[0074] Specifically, the attention module in the composite feature extraction unit adopts an additive attention mechanism. The additive attention mechanism scores the hidden state at each time step through a learnable feedforward network. The scoring results are normalized to obtain the attention weight at each time step. The attention weight is weighted and summed with the corresponding hidden state to obtain the weighted feature vector.

[0075] In this embodiment of the invention, the convolution module in the composite feature extraction unit employs a one-dimensional convolution kernel. This one-dimensional convolution kernel slides gradually along the time axis with a preset stride. Within each sliding window, it performs local feature extraction and weighted combination on the covered time step data, generating a single output value corresponding to that window. As the convolution kernel slides continuously along the time axis, it gradually covers the entire time series, ultimately outputting a compressed feature sequence with a length smaller than the input data time step length. This achieves dimensionality reduction in the time dimension, reducing the computational burden on subsequent modules.

[0076] In this embodiment of the invention, the attention module in the composite feature extraction unit employs an additive attention mechanism. This mechanism first uses a learnable feedforward network to perform a nonlinear transformation and scoring on the hidden states of each time step output by the bidirectional long short-term memory module, obtaining an unnormalized score value for each time step. Then, the scores of all time steps are normalized using a normalized exponential function, converting the scores of each time step into an attention weight distribution with a sum of 1. This weight distribution reflects the importance of each time step to the current recognition task. Finally, the attention weights of each time step are multiplied element-wise with their corresponding hidden states and summed to obtain a feature vector after attention weighting. This feature vector strengthens the information representation of important time steps while suppressing interference from irrelevant time steps.

[0077] This invention extracts local temporal features and compresses sequence length by sliding a one-dimensional convolutional kernel along the time axis, reducing the computational overhead of the subsequent bidirectional long short-term memory module. An additive attention mechanism is used to automatically score and weight the hidden states of each time step output by the bidirectional long short-term memory module, enabling the model to adaptively focus on key time steps that contribute more to safety status identification and suppress interference from irrelevant or redundant time steps. This improves the ability to capture key temporal information in industrial control equipment operation monitoring data and further enhances the accuracy of safety status identification.

[0078] Specifically, in step S2, the preset recognition performance target includes at least one of recognition accuracy threshold, recall threshold, and F1 score threshold. The configuration process of the global hyperparameters is an iterative optimization process. When the recognition performance of the training set under the initially configured global hyperparameters does not reach the preset recognition performance target, the global hyperparameters are adjusted by Bayesian search until the preset recognition performance target is reached or the preset maximum number of iterations is reached.

[0079] In this embodiment of the invention, firstly, a multi-dimensional performance target consisting of an accuracy threshold, a recall threshold, and an F1 score threshold is set. For example, the accuracy threshold, recall threshold, and F1 score threshold are set to 95%, 92%, and 93%, respectively. The current hyperparameter configuration is deemed to meet the requirements and the optimization process is terminated only when the model simultaneously reaches or exceeds the above three indicators on the validation set. Secondly, a reasonable search space is defined for each hyperparameter to be optimized, including setting the range of the number of convolutional output channels to 16 to 128, the range of the number of recurrent hidden units to 32 to 256, the range of the regularization strength coefficient to 1e-5 to 1e-1, the range of the learning rate to 1e-4 to 1e-2 using a log-uniform distribution, setting the range of the order of the polynomial mapping to {2,3,4}, the range of the total number of network layers to {2,3,4,5}, the range of the maximum number of candidate neurons per layer to {3,5,10,20} using a discrete uniform distribution, and setting the range of the dropout rate to 0.1 to 0.5 using a uniform distribution.

[0080] Then, the Latin hypercube sampling method is used to uniformly sample N initial hyperparameter combinations from the search space. These N hyperparameter combinations are trained and validated in parallel to obtain the corresponding F1 scores as observations. A Gaussian process regression model is constructed using these N hyperparameter-performance data as samples as a surrogate model. This surrogate model can not only predict the F1 score of any hyperparameter combination, but also provide the uncertainty of the prediction. In each iteration, the expected improvement is used as the sampling function. For each candidate hyperparameter combination, the expected improvement value that it may bring relative to the current known best F1 score is measured. The hyperparameter combination that maximizes the expected improvement value is found in the entire search space and selected as the object of actual training and evaluation in the next round. The model is trained with the selected hyperparameter combination, and the true F1 score, accuracy, and recall are calculated on the validation set. The newly obtained hyperparameter-performance data are added to the historical observations and the Gaussian process surrogate model is updated.

[0081] The iterative search and evaluation process described above is repeated until termination conditions are met. Termination conditions include: the accuracy, recall, and F1 score of the current optimal hyperparameter combination simultaneously reaching preset thresholds; or the improvement in the optimal F1 score over multiple consecutive iterations being less than a preset small threshold; or the number of iterations reaching a preset maximum number of iterations. During the search process, if accuracy, recall, and F1 score cannot be simultaneously achieved, the F1 score becomes the primary optimization objective. Accuracy and recall thresholds are treated as soft constraints, and a penalty term is added to the acquisition function. When the accuracy or recall of a candidate hyperparameter combination in the surrogate model prediction falls below a preset threshold, the expected improvement value of that combination is multiplied by a penalty factor to reduce its probability of being selected, guiding the search towards regions that satisfy multiple objectives. To improve search efficiency, an early stopping mechanism and transfer initialization are introduced. For each candidate hyperparameter combination training process, if the validation set loss does not decrease within multiple consecutive training rounds, the training is terminated early to avoid wasting computational resources on low-performance combinations. If there are historical hyperparameter search results for similar industrial control equipment, the optimal combination is used as the initial prior for the Gaussian process mean function of this Bayesian search to accelerate search convergence. When the search terminates, the combination with the highest F1 score among all historically evaluated hyperparameter combinations is used as the final global hyperparameter. Under this hyperparameter configuration, the recognition accuracy, recall, and F1 score on the validation set simultaneously meet the three preset performance targets.

[0082] Specifically, in step S3, the composite feature extraction unit further includes a batch normalization module and an activation function module. The batch normalization module is placed after the convolution module and before the bidirectional long short-term memory module, and is used to normalize the feature sequence output by the convolution module. The activation function module uses a modified linear unit to perform nonlinear activation on the output of the batch normalization module, and inputs the activated feature sequence into the bidirectional long short-term memory module.

[0083] In this embodiment of the invention, the composite feature extraction unit further includes a batch normalization module and an activation function module between the convolution module and the bidirectional long short-term memory module. The specific implementation is as follows: After constructing the convolution module based on the number of convolution output channels in the global hyperparameters, the convolution module performs local feature extraction and dimensionality compression on the input runtime monitoring data, outputting a compressed feature sequence. This feature sequence then first enters the batch normalization module.

[0084] The batch normalization module normalizes the feature sequence output by the convolution module along the feature channel dimension. Specifically, for each feature channel, the batch normalization module calculates the mean and variance of all samples and all time steps in that channel within the current training batch. Then, it uses these mean and variance to normalize each feature value in that channel, adjusting the feature values ​​to a distribution range with a mean of zero and a variance of units. After normalization, the batch normalization module also introduces learnable scaling and translation parameters to perform a linear transformation on the normalized features to restore the feature representation capability required by the model. Through batch normalization, the numerical distribution of each channel in the feature sequence output by the convolution module is unified to a standardized range, thereby suppressing the adverse effects of differences in numerical magnitude between different channels on the training of subsequent modules.

[0085] The feature sequence processed by the batch normalization module is then input to the activation function module. The activation function module uses a modified linear unit (MRU) as the activation function, performing a nonlinear transformation on each feature value output by the batch normalization module. The modified linear unit sets negative values ​​to zero and keeps positive values ​​unchanged, outputting a nonlinearly activated feature sequence. This activation operation, while preserving effective feature information, introduces nonlinear expressive power into the network, enabling the composite feature extraction unit to fit more complex function mappings. The nonlinearly activated feature sequence output by the activation function module is ultimately passed to the bidirectional long short-term memory (BSSM) module as its input sequence, according to the input dimension requirements of the BSSM module. The BSSM module extracts temporal dependencies from this input sequence along the forward and backward time axes, outputting a hidden state sequence containing forward and backward contextual information.

[0086] Specifically, in step S42, the objective function of the least squares method with regularization constraints is to minimize the sum of the mean square error and the regularization term between the initial output of the polynomial mapping module and the corresponding state label, wherein the regularization term is the product of the square of the L2 norm of the mapping coefficient vector and the regularization strength coefficient; and solve the objective function to obtain the closed-form analytical solution of the mapping coefficient vector.

[0087] In this embodiment of the invention, the specific implementation of the least squares method with regularization constraints is as follows: After constructing a polynomial mapping module based on the nonlinear mapping order in the global hyperparameters, the polynomial mapping module takes the output of the composite feature extraction unit as input and performs polynomial expansion on each dimension of the input features according to the nonlinear mapping order to generate a high-dimensional feature set containing the original features and each product term. Subsequently, based on the regularization strength coefficient in the global hyperparameters, the mapping coefficients of the high-dimensional feature set are solved using the least squares method with regularization constraints. The solution objective of this method is to minimize the mean square error between the initial output of the polynomial mapping module and the corresponding state label while taking into account the size constraint of the mapping coefficient vector itself. Specifically, the objective function consists of the sum of two terms: the first term is the mean squared error between the initial output of the polynomial mapping module and the corresponding state label, which measures the fit between the model's predicted value and the true label; the second term is the regularization term, which is the product of the squared L2 norm of the mapping coefficient vector and the regularization strength coefficient, used to penalize the magnitude of the mapping coefficient vector to prevent the absolute value of the mapping coefficient from being too large, which would lead to model overfitting.

[0088] In the optimization process of the above objective function, since the objective function has the property of a convex function, the analytical expression of the mapping coefficient vector can be directly obtained by taking the derivative of the mapping coefficient vector and setting the derivative to zero, i.e., the closed-form analytical solution. This closed-form analytical solution ensures that, given a high-dimensional feature set and corresponding state labels, the mapping coefficient vector can be uniquely and accurately calculated without relying on iterative optimization algorithms for approximation. After obtaining the closed-form analytical solution of the mapping coefficient vector, matrix multiplication is performed between the mapping coefficient vector and the high-dimensional feature set to calculate the initial output of the polynomial mapping module. This initial output is the preliminary recognition result after the output of the composite feature extraction unit undergoes high-order nonlinear mapping by the polynomial mapping module.

[0089] Specifically, the evaluation of the recognition performance of hybrid neurons in each layer in step S52 includes: calculating the recognition accuracy of each hybrid neuron on the validation set and sorting them from high to low according to the recognition accuracy; retaining a specified number of hybrid neurons at the top of the ranking as candidate nodes according to the upper limit of the number of candidate neurons in the global hyperparameters, and discarding the remaining hybrid neurons; and concatenating the outputs of the composite feature extraction units of each hybrid neuron in the retained candidate nodes to form the input feature vector of the next layer.

[0090] In this embodiment of the invention, after the construction of hybrid neurons in each layer is completed, it is necessary to quantitatively evaluate the recognition performance of all hybrid neurons in the current layer. Specifically, each hybrid neuron in the current layer is subjected to forward propagation calculation on the validation set to obtain the recognition output result of each hybrid neuron for the input samples of that layer. The recognition output result is compared one by one with the corresponding real state labels in the validation set, and the proportion of the number of samples correctly recognized by each hybrid neuron to the total number of samples in the validation set is counted. This proportion is used as the recognition accuracy of the hybrid neuron. By calculating the recognition accuracy of each hybrid neuron in the current layer in the above manner, the set of accuracy values ​​of all hybrid neurons in the current layer is obtained.

[0091] After obtaining the recognition accuracy of all hybrid neurons in the current layer, they are sorted in descending order of accuracy, resulting in a sequence of hybrid neurons with the highest to lowest accuracy. Then, based on the pre-set upper limit for the number of candidate neurons in the global hyperparameters, a specified number of hybrid neurons are retained from the top of the sorted sequence; these retained hybrid neurons are the candidate nodes for the current layer. The remaining hybrid neurons in the current layer that are not retained, i.e., all hybrid neurons ranked after the upper limit for the number of candidate neurons, are discarded and will not participate in the construction of subsequent layers.

[0092] After identifying candidate nodes in the current layer, the output of the composite feature extraction unit within each hybrid neuron in these candidate nodes is extracted. Since each hybrid neuron contains a composite feature extraction unit, its output is a temporal feature vector processed sequentially by the convolution module, bidirectional long short-term memory module, and attention module. The outputs of the composite feature extraction units corresponding to all retained candidate nodes in the current layer are concatenated along the feature dimension according to the candidate node sorting order or a preset concatenation rule, forming a new feature vector. This feature vector becomes the input feature vector for the next layer. This concatenation operation merges the feature information extracted from multiple candidate nodes along the feature dimension, enabling each hybrid neuron in the next layer to simultaneously receive composite information from multiple preferred feature extraction channels in the previous layer.

[0093] Specifically, the gating mechanism is a learnable soft gating mechanism. The soft gating mechanism generates a continuous gating value between 0 and 1 based on the input features of the current layer and the recognition error of the previous layer. The gating value is multiplied by the output of the composite feature extraction unit and then passed to the polynomial mapping unit. The parameters of the soft gating mechanism are jointly optimized with the parameters of the composite feature extraction unit and the mapping coefficients of the polynomial mapping unit during model training.

[0094] In this embodiment of the invention, during the construction of each layer of hybrid neurons, the output of the composite feature extraction unit is not directly passed entirely to the multinomial mapping unit. Instead, it first undergoes information selection and regulation through a learnable soft gating mechanism. This soft gating mechanism includes a gating generation network composed of learnable parameters. The input to this gating generation network contains two parts: the first part is the original input features of the current layer, i.e., the feature vector passed to the current layer's hybrid neurons; the second part is the recognition error of the previous layer, which is the deviation between the output of the multinomial mapping unit of the previous layer and the corresponding state label, reflecting the performance of the previous layer in the recognition task. The gating generation network takes these two parts of information as input, and through internal linear transformation and nonlinear activation processing, generates a continuous gating value between zero and one for each feature dimension output by the composite feature extraction unit. The meaning of the continuous gating value is as follows: when the gating value is close to one, it means that the feature information of this dimension is allowed to pass through the gating almost completely and be passed to the subsequent polynomial mapping unit; when the gating value is close to zero, it means that the feature information of this dimension is significantly suppressed, and only a small amount of information is allowed to pass through or not pass through; when the gating value is between zero and one, it means that the feature information of this dimension is selectively passed according to the proportion indicated by the gating value.

[0095] After generating the gate value, the output feature vector of the composite feature extraction unit is multiplied element-wise with the gate value vector. Specifically, each feature dimension output by the composite feature extraction unit is multiplied with its corresponding gate value, and the product is used as the gated feature vector. This filtered feature vector is then passed to the multinomial mapping unit as its input for subsequent high-order nonlinear mapping.

[0096] During model training, the learnable parameters within the soft gating unit, the parameters of the composite feature extraction unit, and the mapping coefficients of the polynomial mapping unit are jointly optimized under the same objective function. Specifically, during the backpropagation phase, based on the final recognition error, the gradients of the parameters of the composite feature extraction unit, the mapping coefficients of the polynomial mapping unit, and the parameters within the soft gating unit are calculated simultaneously, and these three types of parameters are updated synchronously according to the direction of each gradient. Because the parameters of the soft gating unit, the parameters of the composite feature extraction unit, and the mapping coefficients of the polynomial mapping unit are coordinated and adjusted during training, the soft gating unit can gradually learn during training how to allocate the gating values ​​of each feature dimension based on the input feature characteristics of the current layer and the recognition error of the previous layer, so that the feature information ultimately passed to the polynomial mapping unit is most conducive to reducing the recognition error of the current layer.

[0097] Specifically, the industrial control equipment includes at least one of programmable logic controllers, distributed control systems, remote terminal units, and intelligent electronic devices; the operation monitoring data includes at least one of the operation log data, network traffic data, process variable data, and system call sequence data of the industrial control equipment; and the safety status identification results include three categories: normal status, abnormal status, and fault status.

[0098] In this embodiment of the invention, the applicable industrial control equipment includes core control and monitoring devices in various industrial control systems. Specifically, the industrial control equipment includes programmable logic controllers (PLCs), which are widely used for logic and sequential control in industrial production processes; distributed control systems, which are typically used for decentralized control and centralized management in large-scale process industries; remote terminal units, which are commonly used for remote data acquisition and remote control scenarios; and intelligent electronic devices, which integrate multiple functions such as protection, control, and monitoring, and are commonly found in power automation systems. In actual industrial production environments, these various types of equipment are typically deployed individually or in combination to work collaboratively, forming the overall operating architecture of the industrial control system. The technical solution of this application embodiment can be applied to the safe status identification of any of the above-mentioned devices when they are running alone, or simultaneously to the collaborative operation of multiple devices.

[0099] Regarding the collection of operational monitoring data, the monitoring data collected in this application embodiment includes multiple types. Specifically, the operational monitoring data includes operational log data of industrial control equipment, which records various operational events, system response information, and timestamps during equipment operation; it also includes network traffic data, which reflects the data packet transmission status of industrial control equipment during network communication, including source address, destination address, communication protocol type, data packet size, and communication frequency; it also includes process variable data, which consists of various analog and digital signals collected or output by industrial control equipment during control, such as temperature, pressure, flow rate, liquid level, and switch status—key parameters reflecting the state of industrial production processes; and it also includes system call sequence data, which records various system calls initiated by the operating system or control program of industrial control equipment during execution, forming a sequence according to the chronological order of the calls. These multiple data sources cover multiple dimensions of industrial control equipment operation, including internal records, external communications, process parameters, and system behavior, comprehensively reflecting the operating status of the equipment. In practical applications, any one of the above data types can be collected, or multiple types of data can be collected simultaneously and combined, depending on the specific configuration of the industrial control equipment and data acquisition conditions.

[0100] Regarding the output of security status identification results, this application embodiment divides the security status of industrial control equipment into three categories. Specifically, a normal status indicates that all operating parameters, communication behaviors, and system calls of the current industrial control equipment are within the expected range, and no abnormal activities or security risks have been found; an abnormal status indicates that the current industrial control equipment deviates from the normal baseline in terms of operating parameters, network communication, or system behavior, and there are potential security risks, such as unexpected parameter fluctuations, abnormal network connection attempts, or unauthorized system calls, but the equipment has not completely lost its basic functions; a fault status indicates that the current industrial control equipment has experienced a clear hardware failure, software crash, or has suffered a successful network attack, resulting in a substantial failure of the equipment's control functions, communication functions, or computing functions. The security status identification result ultimately output by the multi-layer network structure is one of these three categories.

[0101] The above embodiments are merely illustrative examples and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations. However, obvious variations or modifications derived therefrom are still within the scope of protection of this application.

Claims

1. A method for identifying the safety status of industrial control equipment based on a hybrid multinomial neural network, characterized in that, include: Step S1: Collect the operation monitoring data of the industrial control equipment, and divide the operation monitoring data into a training set and a test set; Step S2: Based on the data size and feature dimension of the training set, and in conjunction with the preset recognition performance target, configure the global hyperparameters of the hybrid multinomial neural network. The global hyperparameters include at least the structural parameters of the composite feature extraction unit, the nonlinear mapping order and regularization coefficient of the multinomial mapping unit, and the layer size parameter of the multilayer network structure. Step S3: Based on the structural parameters in the global hyperparameters, construct a composite feature extraction unit for extracting the temporal features of the operation monitoring data; Step S4: Based on the nonlinear mapping order and regularization coefficient in the global hyperparameters, construct a polynomial mapping unit for performing high-order nonlinear mapping on the output of the composite feature extraction unit. Step S5: Based on the layer size parameter in the global hyperparameters, the composite feature extraction unit and the multinomial mapping unit are mixed and stacked into a multi-layer network structure. Step S6: Input the test set into the multi-layer network structure. The multi-layer network structure performs feature extraction and high-order nonlinear mapping on the test set layer by layer, and outputs the safety status identification result of the industrial control equipment.

2. The method according to claim 1, characterized in that, In step S1, the operation monitoring data is divided into a training set and a test set, including: Step S11: Obtain the operation monitoring data and its corresponding status labels. The status labels are used to identify the actual safety status category of the industrial control equipment at the time of data collection, and determine the original dimension and total sample size of the operation monitoring data. Step S12: Based on the original dimensions and total sample size, preprocess the operation monitoring data to obtain preprocessed operation monitoring data; Step S13: Based on the total sample size of the preprocessed operation monitoring data, the preprocessed operation monitoring data is randomly divided into a training set and a test set according to a preset ratio.

3. The method according to claim 1, characterized in that, In step S2, the global hyperparameters of the hybrid multinomial neural network are configured, including: Step S21: Based on the feature dimensions of the training set, configure the number of input variables of the composite feature extraction unit and the number of input variables of the polynomial mapping unit, and configure the nonlinear mapping order of the polynomial mapping unit. Step S22: Based on the temporal length and feature change complexity of the training set, configure the number of convolutional output channels and the number of recurrent hidden units inside the composite feature extraction unit to ensure sufficient extraction of temporal features, and configure the learning rate and dropout rate during model training. Step S23: Based on the total number of samples and feature complexity of the training set, configure the total number of layers of the multi-layer network structure, the number of neurons to be constructed in each layer, and the maximum number of candidate neurons that can be retained in each layer. Step S24: Based on the total number of samples and feature dimensions of the training set, configure the regularization strength coefficient of the polynomial mapping unit when solving its internal mapping coefficients.

4. The method according to claim 1, characterized in that, In step S3, a composite feature extraction unit is constructed for extracting the time-series features of the operational monitoring data, including: Step S31: Based on the number of input variables in the global hyperparameters, construct the input layer of the composite feature extraction unit, and the input layer receives the operation monitoring data; Step S32: Based on the number of convolution output channels in the global hyperparameters, a convolution module is constructed. The convolution module performs local feature extraction and dimension compression on the running monitoring data and outputs a compressed feature sequence. Step S33: Based on the output dimension of the convolutional module and the number of recurrent hidden units in the global hyperparameters, a bidirectional long short-term memory module is constructed. The bidirectional long short-term memory module takes the compressed feature sequence as input, extracts temporal dependencies along the forward and backward time axes respectively, and outputs a hidden state sequence containing forward and backward context information. Step S34: Based on the hidden state sequence output by the bidirectional long short-term memory module, an attention module is constructed. The attention module scores the importance of the hidden states at each time step and assigns corresponding weights. Step S35: The input layer, the convolution module, the bidirectional long short-term memory module, and the attention module are sequentially connected in series to form the composite feature extraction unit.

5. The method according to claim 1, characterized in that, In step S4, a polynomial mapping unit is constructed for performing high-order nonlinear mapping on the output of the composite feature extraction unit, including: Step S41: Based on the nonlinear mapping order in the global hyperparameters, a polynomial mapping module is constructed. The polynomial mapping module takes the output of the composite feature extraction unit as input, performs polynomial expansion on each dimension of the input features according to the nonlinear mapping order, and generates a high-dimensional feature set containing the original features and product terms of each order. Step S42: Based on the regularization strength coefficient in the global hyperparameters, the mapping coefficients of the high-dimensional feature set are solved by the least squares method with regularization constraints to obtain the mapping coefficient vector, and the initial output of the polynomial mapping module is calculated based on the mapping coefficient vector and the high-dimensional feature set. Step S43: The deviation between the initial output of the polynomial mapping module and the corresponding state label is taken as the first error, and the deviation between the output of the composite feature extraction unit and the corresponding state label is taken as the second error; based on the comparison of the change rates of the first error and the second error, the coupling strength between the parameter update of the composite feature extraction unit and the solution of the mapping coefficients of the polynomial mapping unit is adjusted, so that the parameters of the composite feature extraction unit are updated in a direction that can simultaneously reduce the first error.

6. The method according to claim 5, characterized in that, In step S43, adjusting the coupling strength between the parameter update of the composite feature extraction unit and the solution of the mapping coefficients of the polynomial mapping unit includes: Monitor the changing trends of the first error and the second error in multiple consecutive training rounds, and calculate the changing rates of the first error and the second error respectively; The rate of change of the first error is compared with the rate of change of the second error. If the rate of change of the first error is greater than the rate of change of the second error, the coupling strength between the parameter update of the composite feature extraction unit and the solution of the mapping coefficients of the polynomial mapping unit is enhanced. If the rate of change of the first error is less than or equal to the rate of change of the second error, the current coupling strength is weakened or maintained. Based on the adjusted coupling strength, the parameters of the composite feature extraction unit are updated so that the output of the composite feature extraction unit changes in a direction that can simultaneously reduce the first error.

7. The method according to claim 1, characterized in that, In step S5, the composite feature extraction unit and the multinomial mapping unit are mixed and stacked into a multi-layer network structure, including: Step S51: Using the training set as the input of the first layer, multiple hybrid neurons are constructed in parallel in the first layer. Each hybrid neuron consists of a composite feature extraction unit and a multinomial mapping unit connected in series. A gating mechanism is introduced when the composite feature extraction unit transmits information to the multinomial mapping unit. The gating mechanism dynamically adjusts the selection ratio of information transmission according to the current recognition error. Step S52: Evaluate the recognition performance of the hybrid neurons in each layer. Based on the upper limit of the number of candidate neurons in the global hyperparameters, select the output of the composite feature extraction unit that meets the preset standard as candidate nodes. Combine the selected candidate nodes as the input of the next layer. Step S53: Repeat the process of constructing hybrid neurons and selecting candidate nodes until the total number of network layers in the global hyperparameter is reached; Step S54: Stack the layers sequentially according to the data flow direction to form the multi-layer network structure.

8. The method according to claim 1, characterized in that, In step S6, the safety status identification result of the industrial control equipment is output, including: The final output of the multi-layer network structure is used as the safety status identification result of the industrial control equipment. The security status identification results are compared one by one with the corresponding real status labels in the test set to determine the percentage of correctly identified samples out of the total number of samples in the test set, and the percentage is output as the identification accuracy.

9. The method according to claim 1, characterized in that, The convolution module in the composite feature extraction unit uses a one-dimensional convolution kernel. The one-dimensional convolution kernel slides along the time axis to extract local features from the data within each time window. The length of the output feature sequence of the convolution module is less than the time step length of the input data.

10. The method according to claim 1, characterized in that, The attention module in the composite feature extraction unit adopts an additive attention mechanism. The additive attention mechanism scores the hidden state at each time step through a learnable feedforward network. The scoring results are normalized to obtain the attention weight at each time step. The attention weight is weighted and summed with the corresponding hidden state to obtain the weighted feature vector.

11. The method according to claim 1, characterized in that, In step S2, the preset recognition performance target includes at least one of recognition accuracy threshold, recall threshold, and F1 score threshold. The configuration process of the global hyperparameters is an iterative optimization process. When the recognition performance of the training set under the initially configured global hyperparameters does not reach the preset recognition performance target, the global hyperparameters are adjusted by Bayesian search until the preset recognition performance target is reached or the preset maximum number of iterations is reached.

12. The method according to claim 4, characterized in that, In step S3, the composite feature extraction unit further includes a batch normalization module and an activation function module. The batch normalization module is placed after the convolution module and before the bidirectional long short-term memory module, and is used to normalize the feature sequence output by the convolution module. The activation function module uses a modified linear unit to perform nonlinear activation on the output of the batch normalization module, and inputs the activated feature sequence into the bidirectional long short-term memory module.

13. The method according to claim 5, characterized in that, In step S42, the objective function of the least squares method with regularization constraints is to minimize the sum of the mean square error and the regularization term between the initial output of the polynomial mapping module and the corresponding state label, wherein the regularization term is the product of the square of the L2 norm of the mapping coefficient vector and the regularization strength coefficient; the closed-form analytical solution of the mapping coefficient vector is obtained by solving the objective function.

14. The method according to claim 7, characterized in that, The evaluation of the recognition performance of hybrid neurons in each layer in step S52 specifically includes: calculating the recognition accuracy of each hybrid neuron on the validation set and sorting them from high to low according to the recognition accuracy; retaining a specified number of hybrid neurons at the top of the ranking as candidate nodes according to the upper limit of the number of candidate neurons in the global hyperparameters, and discarding the remaining hybrid neurons; and concatenating the outputs of the composite feature extraction units of each hybrid neuron in the retained candidate nodes to form the input feature vector of the next layer.

15. The method according to claim 7, characterized in that, The gating mechanism is a learnable soft gating mechanism. The soft gating mechanism generates a continuous gating value between 0 and 1 based on the input features of the current layer and the recognition error of the previous layer. The gating value is multiplied by the output of the composite feature extraction unit and then passed to the polynomial mapping unit. The parameters of the soft gating mechanism are jointly optimized with the parameters of the composite feature extraction unit and the mapping coefficients of the polynomial mapping unit during the model training process.

16. The method according to claim 1, characterized in that, The industrial control equipment includes at least one of programmable logic controllers, distributed control systems, remote terminal units, and intelligent electronic devices; the operation monitoring data includes at least one of the operation log data, network traffic data, process variable data, and system call sequence data of the industrial control equipment; the safety status identification results include three categories: normal status, abnormal status, and fault status.

Images