Network node monitoring method, system, device and medium based on industrial internet of things

Abstract

The application discloses a network node monitoring method, system, device and medium based on an industrial Internet of Things, relates to the technical field of networks, and comprises the following steps: determining a sound time sequence prediction signal based on a sound IMF prediction model and according to the sound time sequence signal, and determining an electric time sequence prediction signal based on an electric IMF prediction model and according to the electric time sequence signal; performing feature engineering on the sound time sequence signal, the electric time sequence signal, the sound time sequence prediction signal and the electric time sequence prediction signal respectively to obtain corresponding sound time sequence features, electric time sequence features, sound time sequence prediction features and electric time sequence prediction features; performing first feature interaction on the sound time sequence features and the electric time sequence features to obtain first sound-electric interaction features, and performing second feature interaction on the sound time sequence prediction features and the electric time sequence prediction features to obtain second sound-electric interaction features. The application has the effect of improving the reliability of network node monitoring.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of networks, in particular to a network node monitoring method, system, device and medium based on an industrial Internet of Things. BACKGROUND

[0002] As a product of the deep integration of industrial systems and Internet technology, the industrial Internet of Things realizes the full interconnection of people, machines and things, and builds a new manufacturing and service system covering the entire industry chain and value chain. In this system, network nodes (such as intelligent sensors, controllers, gateway devices, etc.) are the basic units that carry out data collection, transmission and processing functions, and their stability and reliability directly affect the safety and efficiency of the entire industrial Internet of Things system. Therefore, real-time and accurate monitoring and health status evaluation of network nodes are of great significance for preventing faults, ensuring production continuity and achieving predictive maintenance.

[0003] At present, the monitoring method for network nodes mainly relies on single-dimensional analysis of their operating parameters, such as analyzing electrical signal parameters such as power consumption, CPU load and memory usage of the node, or collecting physical signals such as sound and vibration generated during device operation for anomaly detection. However, this method has obvious limitations: first, the information dimension provided by a single data source (such as only electrical signals or only sound signals) is limited, making it difficult to fully and accurately reflect the complex operating state of the node, especially in the early stages of failure, where weak features are easily overwhelmed by noise, leading to false negatives or false positives. For example, electrical signals can directly reflect electrical characteristics, but are not sensitive to physical changes such as mechanical wear; while sound signals can effectively capture mechanical abnormalities, but are easily disturbed by environmental noise.

[0004] Secondly, traditional monitoring methods are mostly based on analysis of current or historical signals, lacking effective prediction ability for the evolution trend of device status, which means that the system can only alarm after an anomaly or failure has occurred, which is a passive response and cannot achieve true predictive maintenance, thereby minimizing unplanned downtime.

[0005] In summary, the current technology has the problems of single monitoring dimension and lack of prediction ability, which cannot meet the urgent needs of the industrial Internet of Things for high reliability and high availability of network node intelligent monitoring. SUMMARY

[0006] In order to improve the reliability of network node monitoring, the present application provides a network node monitoring method, system, device and medium based on an industrial Internet of Things.

[0007] In a first aspect, the present application provides a network node monitoring method based on an industrial Internet of Things, which adopts the following technical solution:

[0008] A network node monitoring method based on the Industrial Internet of Things (IIoT) is applied to an IIoT system, which includes a management platform, a sensor network platform, and an object platform connected in sequence. The method is executed by the management platform and includes:

[0009] Acquire the audio timing signal and electrical timing signal of the target network node, determine the audio timing prediction signal based on the audio IMF prediction model, and determine the electrical timing prediction signal based on the electrical IMF prediction model.

[0010] Feature engineering operations are performed on the audio timing signal, the electrical timing signal, the audio timing prediction signal, and the electrical timing prediction signal respectively to obtain the corresponding audio timing features, electrical timing features, audio timing prediction features, and electrical timing prediction features;

[0011] A first feature interaction operation is performed on the sound temporal features and the electrical temporal features to obtain a first sound-electrical interaction feature, and a second feature interaction operation is performed on the sound temporal prediction features and the electrical temporal prediction features to obtain a second sound-electrical interaction feature. The first sound-electrical interaction feature is used to represent the multimodal cooperative state of the target network node at the current moment, and the second sound-electrical interaction feature is used to represent the multimodal cooperative evolution trend of the target network node at future moments.

[0012] Based on the network node monitoring model, and by evaluating the current state of the target network node according to the first voice-electric interaction feature and the second voice-electric interaction feature, the node state evaluation result of the target network node is obtained, so as to realize the monitoring of the target network node.

[0013] By employing the above technical solution, the audio timing signal and electrical timing signal of the target network node are obtained. Based on the audio IMF prediction model, the audio timing prediction signal is determined according to the audio timing signal. Based on the electrical IMF prediction model, the electrical timing prediction signal is determined according to the electrical timing signal. Then, feature engineering operations are performed on the audio timing signal, electrical timing signal, audio timing prediction signal, and electrical timing prediction signal respectively to obtain the corresponding audio timing features, electrical timing features, audio timing prediction features, and electrical timing prediction features. Then, a first feature interaction operation is performed on the audio timing features and electrical timing features to obtain the first audio-electrical interaction feature. A second feature interaction operation is performed on the audio timing features and electrical timing prediction features to obtain the second audio-electrical interaction feature. The first audio-electrical interaction feature is used to represent the multimodal cooperative state of the target network node at the current moment, and the second audio-electrical interaction feature is used to represent the multimodal cooperative state of the target network node at the current moment. Electrical interaction features are used to represent the multimodal collaborative evolution trend of target network nodes in the future. Then, based on the network node monitoring model, the current state of the target network node is evaluated according to the first and second sound-electric interaction features to obtain the node state evaluation result of the target network node, so as to realize the monitoring of the target network node. This method comprehensively utilizes the sound signals and electrical signals of the node operation and their prediction information, and through fine feature engineering and cross-modal feature interaction, it greatly enriches the information dimension and depth of the state evaluation. This allows the method to not only reflect the current real-time state of the node, but also capture its future change trend through prediction signals, thereby realizing the transformation from passive response to active prediction, significantly improving the accuracy and foresight of the state evaluation, thus improving the reliability of network node monitoring, and helping to ensure the safe and stable operation of industrial production systems.

[0014] Optionally, the step of determining the sound time-series prediction signal based on the sound IMF prediction model and the sound time-series signal includes:

[0015] The audio timing signal is subjected to EMD processing to obtain audio timing IMF components;

[0016] Time-frequency features are extracted from the temporal IMF components of the sound to obtain the sound time-frequency features;

[0017] The sound temporal IMF component and the sound time-frequency feature are input into the sound IMF prediction model to obtain at least one sound temporal IMF component;

[0018] A predefined residual sound component is obtained, and the residual sound component is linearly superimposed based on the at least one temporal IMF component of the sound to obtain a temporal prediction signal of the sound.

[0019] By adopting the above technical solution, in order to determine the audio timing prediction signal, the audio timing signal is subjected to EMD processing to obtain audio timing IMF components. Then, time-frequency features are extracted from the audio timing IMF components to obtain audio time-frequency features. Then, the audio timing IMF components and audio time-frequency features are input into the audio IMF prediction model to obtain at least one audio timing IMF component. Then, a predefined audio residual component is obtained, and the audio timing prediction signal is obtained by linearly superimposing at least one audio timing IMF component and the audio residual component.

[0020] Optionally, the step of determining the electrical time-series prediction signal based on the electrical IMF prediction model and the electrical time-series signal includes:

[0021] The electrical timing signal is subjected to EMD processing to obtain the electrical timing IMF component;

[0022] Time-frequency features are extracted from the electrical time-series IMF components to obtain electrical time-frequency features;

[0023] The electrical time-series IMF component and the electrical time-frequency feature are input into the electrical IMF prediction model to obtain at least one electrical time-series IMF component;

[0024] A predefined electrical residual component is obtained, and the electrical residual component is linearly superimposed based on the at least one electrical time-series IMF component to obtain an electrical time-series prediction signal.

[0025] By adopting the above technical solution, in order to determine the electrical time series prediction signal, the electrical time series signal is subjected to EMD processing to obtain the electrical time series IMF component. Then, the time-frequency feature of the electrical time series IMF component is extracted to obtain the electrical time-frequency feature. Then, the electrical time series IMF component and the electrical time-frequency feature are input into the electrical IMF prediction model to obtain at least one electrical time series IMF component. Then, a predefined electrical residual component is obtained, and the electrical time series prediction signal is obtained by linear superposition of at least one electrical time series IMF component and the electrical residual component.

[0026] Optionally, the step of performing feature engineering operations on the audio timing signal, the electrical timing signal, the audio timing prediction signal, and the electrical timing prediction signal respectively to obtain corresponding audio timing features, electrical timing features, audio timing prediction features, and the electrical timing prediction features includes:

[0027] The audio timing signal, the electrical timing signal, the audio timing prediction signal, and the electrical timing prediction signal are respectively subjected to latent space mapping to obtain the corresponding audio timing latent representation, electrical timing latent representation, audio timing prediction latent representation, and electrical timing prediction latent representation;

[0028] High-level semantic features are extracted from the latent temporal representation of the sound to obtain high-level semantic features of the sound temporal sequence;

[0029] High-level semantic features are extracted from the electrical time-series latent representation to obtain high-level semantic features of the electrical time series.

[0030] High-level semantic features are extracted from the latent representation of sound temporal prediction to obtain high-level semantic features of sound temporal prediction.

[0031] High-level semantic features are extracted from the potential representation of electrical time series prediction to obtain high-level semantic features of electrical time series prediction.

[0032] By adopting the above technical solution, in order to obtain the corresponding audio timing features, electrical timing features, audio timing prediction features, and electrical timing prediction features, latent space mapping is performed on the audio timing signal, electrical timing signal, audio timing prediction signal, and electrical timing prediction signal respectively to obtain the corresponding audio timing latent representation, electrical timing latent representation, audio timing prediction latent representation, and electrical timing prediction latent representation. Then, high-level semantic features are extracted from the audio timing latent representation to obtain audio timing high-level semantic features. Then, high-level semantic features are extracted from the electrical timing latent representation to obtain electrical timing high-level semantic features. Then, high-level semantic features are extracted from the audio timing prediction latent representation to obtain audio timing prediction high-level semantic features. Finally, high-level semantic features are extracted from the electrical timing prediction latent representation to obtain electrical timing prediction high-level semantic features.

[0033] Optionally, the step of extracting high-level semantic features from the latent temporal representation of the sound to obtain high-level semantic features of the sound temporal sequence includes:

[0034] Self-attention processing is performed on the latent temporal representation of the sound to obtain the temporal attention features of the sound.

[0035] A residual connection is performed between the audio temporal attention features and the audio temporal latent representation to obtain the residual connection result;

[0036] The residual connection results are subjected to layer normalization to obtain the first intermediate feature;

[0037] The first intermediate feature is nonlinearly transformed using a feedforward neural network to obtain the second intermediate feature;

[0038] The second intermediate feature is residually connected to the first intermediate feature, and the connection result is layer normalized to obtain the high-level semantic features of the sound temporal sequence.

[0039] By adopting the above technical solution, in order to obtain the high-level semantic features of sound temporal sequence, self-attention processing is performed on the latent representation of sound temporal sequence to obtain sound temporal attention features. Then, residual connection is performed between the sound temporal attention features and the latent representation of sound temporal sequence to obtain the residual connection result. Then, layer normalization is performed on the residual connection result to obtain the first intermediate feature. Then, nonlinear transformation is performed on the first intermediate feature through a feedforward neural network to obtain the second intermediate feature. Then, residual connection is performed between the second intermediate feature and the first intermediate feature, and layer normalization is performed on the connection result to obtain the high-level semantic features of sound temporal sequence.

[0040] Optionally, the step of performing a first feature interaction operation on the sound timing features and the electrical timing features to obtain a first sound-electrical interaction feature includes:

[0041] The sound temporal features are taken as input, and a first attention weight vector is generated through a first conditional attention network. The first conditional attention network includes a first semantic matrix, which is used to transform the sound temporal features from the current semantic space to the semantic space where the electrical temporal features are located.

[0042] The sound temporal features are taken as input, and a second attention weight vector is generated through a second conditional attention network. The second conditional attention network includes a second semantic matrix, which is used to transform the sound temporal features from the current semantic space to the semantic space where the electrical temporal features are located.

[0043] The electrical timing features are weighted and multiplied according to the first attention weight vector and the second attention weight vector to obtain the first sound timing weighted feature and the second sound timing weighted feature.

[0044] The first audio temporal weighted feature, the second audio temporal weighted feature, and the electrical temporal feature are weighted and concatenated to obtain the first audio-electrical interaction feature.

[0045] By adopting the above technical solution, in order to obtain the first sound-electric interaction feature, the sound temporal feature is taken as input, and a first attention weight vector is generated through a first conditional attention network. The first conditional attention network includes a first semantic matrix, which is used to transform the sound temporal feature from the current semantic space to the semantic space where the electrical temporal feature is located. Then, the sound temporal feature is taken as input, and a second attention weight vector is generated through a second conditional attention network. The second conditional attention network includes a second semantic matrix, which is used to transform the sound temporal feature from the current semantic space to the semantic space where the electrical temporal feature is located. Then, the electrical temporal feature is weighted and multiplied according to the first attention weight vector and the second attention weight vector to obtain the first sound temporal weighted feature and the second sound temporal weighted feature. Finally, the first sound temporal weighted feature, the second sound temporal weighted feature and the electrical temporal feature are weighted and concatenated to obtain the first sound-electric interaction feature.

[0046] Optionally, the step of performing a second feature interaction operation on the sound timing prediction feature and the electrical timing prediction feature to obtain a second sound-electrical interaction feature includes:

[0047] The sound temporal prediction feature is used as input, and a third attention weight vector is generated through a third conditional attention network. The third conditional attention network includes a third semantic matrix, which is used to transform the sound temporal prediction feature from the current semantic space to the semantic space where the electrical temporal prediction feature is located.

[0048] The sound temporal prediction feature is used as input, and a fourth attention weight vector is generated through a fourth conditional attention network. The fourth conditional attention network includes a fourth semantic matrix, which is used to transform the sound temporal prediction feature from the current semantic space to the semantic space where the electrical temporal prediction feature is located.

[0049] The electrical timing prediction features are weighted and multiplied according to the third attention weight vector and the fourth attention weight vector respectively to obtain the first sound timing weighted prediction feature and the second sound timing weighted prediction feature;

[0050] The first audio time-weighted prediction feature, the second audio time-weighted prediction feature, and the electrical time-weighted prediction feature are weighted and concatenated to obtain the second audio-electrical interaction feature.

[0051] By adopting the above technical solution, in order to obtain the second sound-electric interaction feature, the sound temporal prediction feature is used as input, and a third attention weight vector is generated through a third conditional attention network. The third conditional attention network includes a third semantic matrix, which is used to transform the sound temporal prediction feature from the current semantic space to the semantic space where the electrical temporal prediction feature is located. Then, the sound temporal prediction feature is used as input, and a fourth attention weight vector is generated through a fourth conditional attention network. The fourth conditional attention network includes a fourth semantic matrix, which is used to transform the sound temporal prediction feature from the current semantic space to the semantic space where the electrical temporal prediction feature is located. Then, the electrical temporal prediction feature is weighted and multiplied according to the third attention weight vector and the fourth attention weight vector to obtain the first sound temporal weighted prediction feature and the second sound temporal weighted prediction feature. Finally, the first sound temporal weighted prediction feature, the second sound temporal weighted prediction feature, and the electrical temporal prediction feature are weighted and concatenated to obtain the second sound-electric interaction feature.

[0052] Secondly, this application also provides a network node monitoring system based on the Industrial Internet of Things, which adopts the following technical solution:

[0053] A network node monitoring system based on the Industrial Internet of Things (IIoT) includes a management platform, a sensor network platform, and an object platform that are sequentially connected in communication. The management platform is configured with:

[0054] The prediction module is used to acquire the audio timing signal and electrical timing signal of the target network node, determine the audio timing prediction signal based on the audio timing signal and the electrical timing prediction signal based on the electrical timing signal using the electrical timing signal and the electrical timing prediction signal.

[0055] The feature engineering module is used to perform feature engineering operations on the audio timing signal, the electrical timing signal, the audio timing prediction signal, and the electrical timing prediction signal respectively to obtain the corresponding audio timing features, electrical timing features, audio timing prediction features, and the electrical timing prediction features;

[0056] The feature interaction module is used to perform a first feature interaction operation on the sound temporal feature and the electrical temporal feature to obtain a first sound-electrical interaction feature, and to perform a second feature interaction operation on the sound temporal prediction feature and the electrical temporal prediction feature to obtain a second sound-electrical interaction feature. The first sound-electrical interaction feature is used to represent the multimodal cooperative state of the target network node at the current moment, and the second sound-electrical interaction feature is used to represent the multimodal cooperative evolution trend of the target network node at future moments.

[0057] The monitoring module is used to evaluate the current state of the target network node based on the network node monitoring model and according to the first voice-electric interaction feature and the second voice-electric interaction feature, so as to obtain the node state evaluation result of the target network node and realize the monitoring of the target network node.

[0058] Thirdly, this application also provides a computer device, which adopts the following technical solution:

[0059] A computer device includes a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the computer program to implement the method described in the first aspect.

[0060] Fourthly, this application also provides a computer-readable storage medium, which adopts the following technical solution:

[0061] A computer-readable storage medium storing a computer program capable of being loaded by a processor and executing the method described in the first aspect.

[0062] In summary, this application includes at least the following beneficial technical effects: acquiring the audio timing signal and electrical timing signal of the target network node; determining the audio timing prediction signal based on the audio IMF prediction model and the audio timing prediction signal based on the audio timing signal; determining the electrical timing prediction signal based on the electrical IMF prediction model and the electrical timing signal; then performing feature engineering operations on the audio timing signal, electrical timing signal, audio timing prediction signal, and electrical timing prediction signal respectively to obtain corresponding audio timing features, electrical timing features, audio timing prediction features, and electrical timing prediction features; then performing a first feature interaction operation on the audio timing features and electrical timing features to obtain a first audio-electrical interaction feature; and performing a second feature interaction operation on the audio timing prediction features and electrical timing prediction features to obtain a second audio-electrical interaction feature. The first audio-electrical interaction feature is used to represent the multimodal cooperative state of the target network node at the current moment. The second sound-electric interaction feature is used to represent the multimodal collaborative evolution trend of the target network node at future moments. Then, based on the network node monitoring model, and according to the first and second sound-electric interaction features, the current state of the target network node is evaluated to obtain the node state evaluation result of the target network node, so as to realize the monitoring of the target network node. This method comprehensively utilizes the sound signals and electrical signals of the node operation and their prediction information, and through fine feature engineering and cross-modal feature interaction, it greatly enriches the information dimension and depth of the state evaluation. This allows the method to not only reflect the current real-time state of the node, but also capture its future change trend through prediction signals, thereby realizing the transformation from passive response to active prediction, significantly improving the accuracy and foresight of the state evaluation, thus improving the reliability of network node monitoring, and helping to ensure the safe and stable operation of industrial production systems. Attached Figure Description

[0063] Figure 1 This is a schematic diagram of the overall process of an embodiment of this application.

[0064] Figure 2 This is a structural diagram of one application scenario of the system in this application embodiment.

[0065] Figure 3 This is a structural diagram of another application scenario of the system according to an embodiment of this application.

[0066] Figure 4 This is a structural block diagram of the computer device described in this application. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0068] This application discloses a network node monitoring method based on the Industrial Internet of Things.

[0069] Reference Figure 1 A network node monitoring method based on the Industrial Internet of Things (IIoT) is applied to an IIoT system, which includes a management platform, a sensor network platform, and an object platform connected in sequence. The method is executed by the management platform and includes:

[0070] Step S11: Obtain the audio timing signal and electrical timing signal of the target network node; determine the audio timing prediction signal based on the audio IMF prediction model and the electrical timing prediction signal based on the electrical IMF prediction model and the electrical timing prediction signal.

[0071] It should be noted that in step S11, the management platform first obtains the raw sound signals and current / voltage / power signals generated during the operation of the target network node from the object platform of the industrial IoT system (such as devices equipped with sensors and smart meters). These signals are presented in the form of time series. Then, the platform uses pre-trained sound IMF prediction model and electrical IMF prediction model to analyze and extrapolate the raw sound and electrical time series signals respectively, and generates predicted values ​​of sound and electrical signals for a future period of time, namely sound time series prediction signals and electrical time series prediction signals, to provide forward-looking data for subsequent state assessment.

[0072] Step S12: Perform feature engineering operations on the audio timing signal, electrical timing signal, audio timing prediction signal, and electrical timing prediction signal respectively to obtain the corresponding audio timing features, electrical timing features, audio timing prediction features, and electrical timing prediction features.

[0073] It should be noted that in step S12, the original signal and the predicted signal data are large in volume and directly contain a lot of redundancy and noise, making them unsuitable for efficient and accurate state assessment. Therefore, feature engineering operations need to be performed on these four sets of time-series signals respectively, and key features that can significantly characterize the equipment state need to be extracted. Finally, they are transformed into a set of highly condensed and information-rich feature vectors, namely, sound time-series features, electrical time-series features, sound time-series prediction features, and electrical time-series prediction features.

[0074] Step S13: Perform a first feature interaction operation on the sound timing features and electrical timing features to obtain a first sound-electrical interaction feature, and perform a second feature interaction operation on the sound timing prediction features and electrical timing prediction features to obtain a second sound-electrical interaction feature.

[0075] Among them, the first voice-electric interaction feature is used to represent the multimodal collaborative state of the target network node at the current moment, and the second voice-electric interaction feature is used to represent the multimodal collaborative evolution trend of the target network node at future moments.

[0076] It should be noted that in step S13, in order to capture the device state more comprehensively, the features of a single mode may have limitations. This step aims to fuse information from two different physical domains: sound and electricity. First, the original sound features and original electrical features obtained in step S12 at the same time point are subjected to feature interaction operations to generate a first sound-electricity interaction feature that can simultaneously reflect the current joint state of sound and electricity. Similarly, the predicted sound features and predicted electrical features are subjected to the same interaction operations to generate a second sound-electricity interaction feature that can predict the future trend of joint changes in sound and electricity. These two sets of interaction features together constitute the multi-dimensional, cross-temporal and spatial fusion information basis for evaluating the node state.

[0077] Step S14: Based on the network node monitoring model, and according to the first sound-electric interaction characteristics and the second sound-electric interaction characteristics, the current state of the target network node is evaluated to obtain the node state evaluation result of the target network node, so as to realize the monitoring of the target network node.

[0078] It should be noted that in step S14, the management platform inputs the two sets of fused features obtained in step S13 (the first interactive feature represents the current joint state, and the second interactive feature represents the future predicted joint trend) into a pre-trained network node monitoring model (which can be a classification model, such as used to determine whether a node is normal, in a warning, or in a fault; or a regression model, such as predicting the remaining lifetime). Through in-depth analysis of these fused features, the model comprehensively evaluates the current health status, performance level, or potential risks of the target network node, and finally outputs a quantitative node status evaluation result. Based on this result, the management platform can issue warnings, arrange maintenance, or make other control decisions, thereby achieving effective monitoring of the target network node.

[0079] In the above implementation, the audio timing signal and electrical timing signal of the target network node are acquired. Based on the audio IMF prediction model, an audio timing prediction signal is determined according to the audio timing signal. Based on the electrical IMF prediction model, an electrical timing prediction signal is determined according to the electrical timing signal. Then, feature engineering operations are performed on the audio timing signal, electrical timing signal, audio timing prediction signal, and electrical timing prediction signal respectively to obtain corresponding audio timing features, electrical timing features, audio timing prediction features, and electrical timing prediction features. Then, a first feature interaction operation is performed on the audio timing features and electrical timing features to obtain a first audio-electrical interaction feature. Finally, a second feature interaction operation is performed on the audio timing features and electrical timing prediction features to obtain a second audio-electrical interaction feature. The first audio-electrical interaction feature is used to represent the multimodal cooperative state of the target network node at the current moment, and the second audio-electrical interaction feature is used to represent the multimodal cooperative state of the target network node at the current moment. Interactive features are used to represent the multimodal collaborative evolution trend of target network nodes in the future. Then, based on the network node monitoring model, the current state of the target network node is evaluated according to the first and second sound-electric interaction features to obtain the node state evaluation result of the target network node, so as to realize the monitoring of the target network node. This method comprehensively utilizes the sound signals and electrical signals of the node operation and their prediction information, and through fine feature engineering and cross-modal feature interaction, it greatly enriches the information dimension and depth of the state evaluation. This allows the method to not only reflect the current real-time state of the node, but also capture its future change trend through prediction signals, thereby realizing the transformation from passive response to active prediction, significantly improving the accuracy and foresight of the state evaluation, thus improving the reliability of network node monitoring, and helping to ensure the safe and stable operation of industrial production systems.

[0080] As a further implementation of the method, the step of determining the sound time-series prediction signal based on the sound IMF prediction model and the sound time-series signal includes:

[0081] Step S21: Perform EMD processing on the audio timing signal to obtain the audio timing IMF component.

[0082] Step S22: Extract time-frequency features from the audio temporal IMF components to obtain audio time-frequency features.

[0083] Step S23: Input the sound temporal IMF component and the sound time-frequency feature into the sound IMF prediction model to obtain at least one sound temporal IMF component.

[0084] Step S24: Obtain a predefined residual sound component, and linearly superimpose the residual sound component with at least one temporal IMF component to obtain a temporal prediction signal.

[0085] In the above implementation, in order to determine the audio timing prediction signal, the audio timing signal is subjected to EMD processing to obtain audio timing IMF components. Then, time-frequency features are extracted from the audio timing IMF components to obtain audio time-frequency features. The audio timing IMF components and audio time-frequency features are then input into the audio IMF prediction model to obtain at least one audio timing IMF component. Then, a predefined audio residual component is obtained, and the audio timing prediction signal is obtained by linearly superimposing the at least one audio timing IMF component and the audio residual component.

[0086] As a further implementation of the method, the step of determining the electrical time-series prediction signal based on the electrical IMF prediction model and the electrical time-series signal includes:

[0087] Step S31: Perform EMD processing on the electrical timing signal to obtain the electrical timing IMF component.

[0088] It should be noted that in step S31, the management platform performs Empirical Mode Decomposition (EMD) on the acquired raw electrical time-series signal (which may contain noise and be non-stationary). EMD is an adaptive signal processing method that can decompose complex raw electrical signals into a series of relatively stationary intrinsic mode functions (IMF) components with frequencies ranging from high to low.

[0089] Step S32: Extract time-frequency features from the electrical time-series IMF components to obtain electrical time-frequency features.

[0090] It should be noted that in step S32, in order to fully explore the deep information contained in each electrical time-series IMF component, the platform further extracts time-frequency features for each IMF component. This process goes beyond simple time-domain or frequency-domain analysis. Through methods such as Hilbert-Huang Transform (HHT), features that can simultaneously reflect how the frequency content of the signal changes over time are obtained, such as marginal spectrum, instantaneous energy, and instantaneous frequency. The final electrical time-frequency features are a feature set containing rich time-frequency information, which can more accurately describe the non-stationary dynamic characteristics of the signal.

[0091] Step S33: Input the electrical time-series IMF component and electrical time-frequency features into the electrical IMF prediction model to obtain at least one electrical time-series IMF component.

[0092] It should be noted that in step S33, the management platform inputs the obtained electrical time-series IMF components (along with the extracted corresponding electrical time-frequency features (as supplementary context information)) into a pre-trained electrical IMF prediction model. This model is typically a complex time-series prediction network (such as LSTM, GRU, or Transformer), which predicts the future values ​​of at least one IMF component (usually all components) at the same time step by learning the complex nonlinear mapping relationship between historical IMF components and their corresponding time-frequency features. Furthermore, the construction of the electrical IMF prediction model is a supervised learning process. A large amount of historical electrical time-series signal data needs to be collected as a training set. For each training... The data undergoes EMD decomposition to obtain its historical IMF component sequences, and the time-frequency features of each component are further extracted. The model input consists of all IMF component values ​​and their corresponding time-frequency features within the historical time window. The model's training labels (target output) are the true IMF component values ​​corresponding to one or more future time steps. Subsequently, a suitable time series prediction network structure (such as Seq2Seq, TCN, etc.) is selected, and the mean squared error (MSE) or similar loss function is used as the optimization objective. The model parameters are iteratively trained using the backpropagation algorithm, enabling it to learn to accurately predict the mapping relationship of future IMF components from historical inputs. Finally, the validated model is deployed for online prediction in step S33.

[0093] Step S34: Obtain a predefined electrical residual component, and linearly superimpose the electrical time series IMF component and the electrical residual component to obtain an electrical time series prediction signal.

[0094] It should be noted that the principles of steps S21 to S24 are basically the same as those of steps S31 to S34.

[0095] In the above embodiments, in order to determine the electrical time series prediction signal, the electrical time series signal is subjected to EMD processing to obtain the electrical time series IMF component. Then, the time-frequency feature of the electrical time series IMF component is extracted to obtain the electrical time-frequency feature. Then, the electrical time series IMF component and the electrical time-frequency feature are input into the electrical IMF prediction model to obtain at least one electrical time series IMF component. Then, a predefined electrical residual component is obtained, and the electrical time series prediction signal is obtained by linear superposition of at least one electrical time series IMF component and the electrical residual component.

[0096] As a further implementation of the method, the step of performing feature engineering operations on the audio timing signal, electrical timing signal, audio timing prediction signal, and electrical timing prediction signal respectively to obtain the corresponding audio timing features, electrical timing features, audio timing prediction features, and electrical timing prediction features includes:

[0097] Step S41: Perform latent space mapping on the audio timing signal, electrical timing signal, audio timing prediction signal, and electrical timing prediction signal respectively to obtain the corresponding audio timing latent representation, electrical timing latent representation, audio timing prediction latent representation, and electrical timing prediction latent representation.

[0098] It should be noted that in step S41, the management platform inputs the four sets of timing signals (audio timing signal, electrical timing signal, audio timing prediction signal, and electrical timing prediction signal) into an encoder (usually the encoding part of a neural network-based autoencoder-AE or variational autoencoder-VAE). The function of this encoder is to map the high-dimensional, complex original signal space, which may contain redundant information, to a low-dimensional, dense "latent space" that is better able to express the essential laws of the data. The resulting "latent representation" is a highly refined code of the original signal, which captures the most critical and distinctive patterns in the signal.

[0099] Step S42: Extract high-level semantic features from the latent temporal representation of sound to obtain high-level semantic features of sound temporal sequence.

[0100] Step S43: Extract high-level semantic features from the electrical time-series latent representation to obtain high-level semantic features of the electrical time-series.

[0101] It should be noted that steps S42 and S43 are deep abstractions of the potential representation of the current state. Although the potential representation obtained in step S41 has been compressed and refined, it may still contain some details that serve reconstruction. Steps S42 and S43 aim to further mine the high-level semantic information in these potential representations. Specifically, the platform inputs the "sound temporal potential table" and the "electrical temporal potential representation" into a deeper neural network (which can be a multilayer perceptron-MLP or another deep learning model). This network learns and extracts high-level semantic features that can directly characterize the current operating state or health mode of the device through non-linear hierarchical transformations. These features are more abstract than the potential representation and are closer to the requirements of the final evaluation task, such as "stable mode of normal operation" and "vibration characteristics of slight wear".

[0102] Step S44: Extract high-level semantic features from the latent representation of sound temporal prediction to obtain high-level semantic features for sound temporal prediction.

[0103] Step S45: Extract high-level semantic features from the potential representation of electrical time series prediction to obtain high-level semantic features of electrical time series prediction.

[0104] It should be noted that steps S44 and S45 are deep abstractions of the potential representation of the future predicted state. Their purpose is similar to that of S42 and S43, but the perspective is future-oriented. The platform performs the same high-level semantic feature extraction operation on the potential representation of sound timing prediction and the potential representation of electrical timing prediction generated from the predicted signal. This process aims to infer the future state evolution trend of the equipment from the predicted signal. The extracted "predictive high-level semantic features" capture the expected future operating mode, such as "the trend of impending vibration intensification" and "the risk of increased current harmonics," thereby providing direct feature basis for achieving forward-looking predictive maintenance.

[0105] In the above embodiments, in order to obtain the corresponding audio timing features, electrical timing features, audio timing prediction features, and electrical timing prediction features, latent space mapping is performed on the audio timing signal, electrical timing signal, audio timing prediction signal, and electrical timing prediction signal respectively to obtain the corresponding audio timing latent representation, electrical timing latent representation, audio timing prediction latent representation, and electrical timing prediction latent representation. Then, high-level semantic features are extracted from the audio timing latent representation to obtain audio timing high-level semantic features. Then, high-level semantic features are extracted from the electrical timing latent representation to obtain electrical timing high-level semantic features. Then, high-level semantic features are extracted from the audio timing prediction latent representation to obtain audio timing prediction high-level semantic features. Finally, high-level semantic features are extracted from the electrical timing prediction latent representation to obtain electrical timing prediction high-level semantic features.

[0106] As a further implementation of the method, the step of extracting high-level semantic features from the latent temporal representation of sound to obtain high-level semantic features of sound temporal sequence includes:

[0107] Step S51: Perform self-attention processing on the latent temporal representation of sound to obtain the temporal attention features of sound.

[0108] Step S52: Perform residual connection between the audio temporal attention features and the audio temporal latent representation to obtain the residual connection result.

[0109] Step S53: Perform layer normalization on the residual connection results to obtain the first intermediate feature.

[0110] Step S54: The first intermediate feature is nonlinearly transformed using a feedforward neural network to obtain the second intermediate feature.

[0111] Step S55: Perform residual connection between the second intermediate feature and the first intermediate feature, and perform layer normalization on the connection result to obtain the high-level semantic features of the sound temporal sequence.

[0112] In the above implementation, in order to obtain the high-level semantic features of the audio temporal sequence, self-attention processing is performed on the latent representation of the audio temporal sequence to obtain audio temporal attention features. Then, residual connections are performed between the audio temporal attention features and the latent representation of the audio temporal sequence to obtain residual connection results. Then, layer normalization is performed on the residual connection results to obtain first intermediate features. Then, nonlinear transformation is performed on the first intermediate features through a feedforward neural network to obtain second intermediate features. Then, residual connections are performed between the second intermediate features and the first intermediate features, and layer normalization is performed on the connection results to obtain the high-level semantic features of the audio temporal sequence.

[0113] As a further implementation of the method, the step of performing a first feature interaction operation on the sound timing features and electrical timing features to obtain a first sound-electrical interaction feature includes:

[0114] Step S61: The audio temporal features are used as input to generate a first attention weight vector through a first conditional attention network. The first conditional attention network includes a first semantic matrix, which is used to transform the audio temporal features from the current semantic space to the semantic space where the audio temporal features are located.

[0115] It should be noted that step S61 aims to generate attention weights from the first perspective. Specifically, the temporal features of the sound are input into a neural network called a first conditional attention network. The core of this network contains a learnable first semantic matrix, which acts like a translator or projector, transforming (or projecting) the "temporal features of the sound" from its original semantic space (i.e., the feature space representing sound characteristics) to the semantic space of the electrical temporal features (i.e., the feature space representing electrical characteristics). Through this transformation, the system can understand the meaning of the sound features from the perspective of electrical features. Based on this transformed representation, the network ultimately outputs a "first attention weight vector," which indicates which parts of the electrical temporal features are most relevant and important to the current sound features from the first electrical semantic perspective.

[0116] Step S62: The audio temporal features are used as input to generate a second attention weight vector through a second conditional attention network. The second conditional attention network includes a second semantic matrix, which is used to transform the audio temporal features from the current semantic space to the semantic space where the electrical temporal features are located.

[0117] Step S63: The electrical timing features are weighted and multiplied according to the first attention weight vector and the second attention weight vector respectively to obtain the first sound timing weighted feature and the second sound timing weighted feature.

[0118] It should be noted that the two different attention weight vectors generated in steps S61 and S62 are weighted and multiplied with the electrical temporal features. The essence of this operation is "focusing" or "filtering." Using the first attention weight vector to weight the electrical features yields the first audio temporal weighted feature. This is equivalent to strengthening the electrical features highly correlated with the audio features and weakening irrelevant parts from the first perspective. Similarly, using the second attention weight vector to weight the features yields the second audio temporal weighted feature, which represents the important electrical features selected from the second perspective. In this way, we extract two weighted features rich in cross-modal interaction information from the same set of electrical features, selected from different perspectives.

[0119] Step S64: The first sound time-weighted feature, the second sound time-weighted feature, and the electrical time-weighted feature are weighted and concatenated to obtain the first sound-electrical interaction feature.

[0120] It should be noted that step S64 involves fusing information from multiple sources to form the final interaction feature. The two weighted features obtained in the previous step (the first audio temporal weighted feature and the second audio temporal weighted feature) are weighted and concatenated with the original electrical temporal feature. This weighting can be achieved by adjusting the importance of different feature subsets before and after concatenation using learnable weight parameters, or by assigning equal weighting opportunities to features from different sources through the concatenation operation itself. The resulting first audio-electric interaction feature is an extremely rich fusion vector, containing: the original electrical feature information (providing basic background), the electrical feature information selected from audio perspective one (providing one interaction relationship), and the electrical feature information selected from audio perspective two (providing another complementary interaction relationship).

[0121] It should be further explained that in steps S61 to S64, instead of simple feature concatenation or addition, a dual-path conditional attention mechanism is used, which allows one modality (sound) to actively "inquire" or "examine" another modality (electricity), thereby dynamically and specifically extracting the interaction information most relevant to the current context. This method greatly improves the quality and effectiveness of feature interaction and provides a powerful and accurate input for the subsequent state evaluation model.

[0122] In the above implementation, to obtain the first sound-electric interaction feature, the sound temporal feature is used as input, and a first attention weight vector is generated through a first conditional attention network. The first conditional attention network includes a first semantic matrix, which is used to transform the sound temporal feature from the current semantic space to the semantic space where the electrical temporal feature is located. Then, the sound temporal feature is used as input, and a second attention weight vector is generated through a second conditional attention network. The second conditional attention network includes a second semantic matrix, which is used to transform the sound temporal feature from the current semantic space to the semantic space where the electrical temporal feature is located. Then, the electrical temporal feature is weighted and multiplied according to the first attention weight vector and the second attention weight vector to obtain the first sound temporal weighted feature and the second sound temporal weighted feature. Finally, the first sound temporal weighted feature, the second sound temporal weighted feature, and the electrical temporal feature are weighted and concatenated to obtain the first sound-electric interaction feature.

[0123] As a further implementation of the method, the step of performing a second feature interaction operation on the sound timing prediction features and the electrical timing prediction features to obtain second sound-electrical interaction features includes:

[0124] Step S71: The sound temporal prediction features are used as input to generate a third attention weight vector through a third conditional attention network. The third conditional attention network includes a third semantic matrix, which is used to transform the sound temporal prediction features from the current semantic space to the semantic space where the electrical temporal prediction features are located.

[0125] Step S72: The sound temporal prediction features are used as input to generate a fourth attention weight vector through a fourth conditional attention network. The fourth conditional attention network includes a fourth semantic matrix, which is used to transform the sound temporal prediction features from the current semantic space to the semantic space where the electrical temporal prediction features are located.

[0126] Step S73: The electrical timing prediction features are weighted and multiplied according to the third attention weight vector and the fourth attention weight vector respectively to obtain the first sound timing weighted prediction features and the second sound timing weighted prediction features.

[0127] Step S74: The first sound time-weighted prediction feature, the second sound time-weighted prediction feature, and the electrical time-weighted prediction feature are weighted and concatenated to obtain the second sound-electric interaction feature.

[0128] It should be noted that the principles of steps S71 to S74 are basically the same as those of steps S61 to S64.

[0129] In the above implementation, to obtain the second sound-electric interaction feature, the sound temporal prediction feature is used as input, and a third attention weight vector is generated through a third conditional attention network. The third conditional attention network includes a third semantic matrix, which is used to transform the sound temporal prediction feature from the current semantic space to the semantic space where the electrical temporal prediction feature is located. Then, the sound temporal prediction feature is used as input, and a fourth attention weight vector is generated through a fourth conditional attention network. The fourth conditional attention network includes a fourth semantic matrix, which is used to transform the sound temporal prediction feature from the current semantic space to the semantic space where the electrical temporal prediction feature is located. Then, the electrical temporal prediction feature is weighted and multiplied according to the third attention weight vector and the fourth attention weight vector to obtain the first sound temporal weighted prediction feature and the second sound temporal weighted prediction feature. Finally, the first sound temporal weighted prediction feature, the second sound temporal weighted prediction feature, and the electrical temporal prediction feature are weighted and concatenated to obtain the second sound-electric interaction feature.

[0130] This application also discloses a network node monitoring system based on the Industrial Internet of Things.

[0131] refer to Figure 3 The network node monitoring system based on the Industrial Internet of Things includes a management platform, a sensor network platform, and an object platform that are connected in sequence. The management platform is configured with:

[0132] The prediction module is used to acquire the audio timing signal and electrical timing signal of the target network node, determine the audio timing prediction signal based on the audio timing signal and the electrical timing prediction signal based on the electrical timing signal using the electrical timing signal.

[0133] The feature engineering module is used to perform feature engineering operations on the audio timing signal, electrical timing signal, audio timing prediction signal, and electrical timing prediction signal respectively to obtain the corresponding audio timing features, electrical timing features, audio timing prediction features, and electrical timing prediction features;

[0134] The feature interaction module is used to perform a first feature interaction operation on the sound temporal feature and the electrical temporal feature to obtain a first sound-electrical interaction feature, and to perform a second feature interaction operation on the sound temporal prediction feature and the electrical temporal prediction feature to obtain a second sound-electrical interaction feature. The first sound-electrical interaction feature is used to represent the multimodal cooperative state of the target network node at the current moment, and the second sound-electrical interaction feature is used to represent the multimodal cooperative evolution trend of the target network node at future moments.

[0135] The monitoring module is used to evaluate the current state of the target network node based on the network node monitoring model and according to the first sound-electrical interaction characteristics and the second sound-electrical interaction characteristics, so as to obtain the node state evaluation result of the target network node and realize the monitoring of the target network node.

[0136] The overall framework of another application scenario of the network node monitoring system based on the Industrial Internet of Things in this application is as follows: Figure 3 As shown, it can include a user platform, service platform, management platform, sensor network platform, and object platform that interact sequentially, forming a five-platform architecture based on the Industrial Internet of Things. The management platform includes a prediction module, a feature engineering module, a feature interaction module, and a monitoring module. The sensor network platform includes several sensor network sub-platforms, each with its own sensor sub-database.

[0137] Specifically, in another application scenario mentioned above, the network node monitoring system based on the Industrial Internet of Things (IIoT) includes a management platform. The management platform is configured to: acquire the audio timing signal and electrical timing signal of the target network node; determine the audio timing prediction signal based on the audio IMF prediction model and the audio timing signal; and determine the electrical timing prediction signal based on the electrical IMF prediction model and the electrical timing signal; perform feature engineering operations on the audio timing signal, electrical timing signal, audio timing prediction signal, and electrical timing prediction signal respectively to obtain corresponding audio timing features, electrical timing features, audio timing prediction features, and electrical timing prediction features; and perform a first feature analysis on the audio timing features and the electrical timing features. A first voice-electric interaction feature is obtained through a feature interaction operation. Then, a second feature interaction operation is performed on the voice timing prediction feature and the electrical timing prediction feature to obtain a second voice-electric interaction feature. The first voice-electric interaction feature represents the multimodal collaborative state of the target network node at the current moment, and the second voice-electric interaction feature represents the multimodal collaborative evolution trend of the target network node at future moments. Based on a network node monitoring model, and according to the first and second voice-electric interaction features, the current state of the target network node is evaluated to obtain the node state evaluation result of the target network node, thereby achieving monitoring of the target network node.

[0138] By establishing a complete closed-loop information operation logic through the interaction between various functional platforms of the industrial IoT-based network node monitoring system based on the aforementioned three or five platforms, the orderly operation of sensing and control information is ensured, thereby realizing intelligent equipment management.

[0139] The network node monitoring system based on the Industrial Internet of Things of the present invention can implement any of the methods in the network node monitoring method based on the Industrial Internet of Things, and the specific working process of the network node monitoring system based on the Industrial Internet of Things of the present invention can refer to the corresponding process in the above-mentioned network node monitoring method based on the Industrial Internet of Things.

[0140] This application also discloses a computer device.

[0141] refer to Figure 4 A computer device includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement any of the above-described methods for monitoring network nodes based on the Industrial Internet of Things.

[0142] This application also discloses a computer-readable storage medium.

[0143] A computer-readable storage medium storing a computer program that can be loaded by a processor and executed by any of the above-described methods for monitoring network nodes based on the Industrial Internet of Things.

[0144] The computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device; the program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0145] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A network node monitoring method based on the Industrial Internet of Things, characterized in that, Applied to an industrial Internet of Things (IIoT) system, the IIoT system includes a management platform, a sensor network platform, and an object platform that are sequentially and communicatively connected. The method is executed by the management platform and includes: Acquire the audio timing signal and electrical timing signal of the target network node, determine the audio timing prediction signal based on the audio IMF prediction model, and determine the electrical timing prediction signal based on the electrical IMF prediction model. Feature engineering operations are performed on the audio timing signal, the electrical timing signal, the audio timing prediction signal, and the electrical timing prediction signal respectively to obtain the corresponding audio timing features, electrical timing features, audio timing prediction features, and electrical timing prediction features; A first feature interaction operation is performed on the sound temporal features and the electrical temporal features to obtain a first sound-electrical interaction feature, and a second feature interaction operation is performed on the sound temporal prediction features and the electrical temporal prediction features to obtain a second sound-electrical interaction feature. The first sound-electrical interaction feature is used to represent the multimodal cooperative state of the target network node at the current moment, and the second sound-electrical interaction feature is used to represent the multimodal cooperative evolution trend of the target network node at future moments. Based on the network node monitoring model, and by evaluating the current state of the target network node according to the first voice-electric interaction feature and the second voice-electric interaction feature, the node state evaluation result of the target network node is obtained, so as to realize the monitoring of the target network node. The step of determining the sound time-series prediction signal based on the sound IMF prediction model and the sound time-series signal includes: The audio timing signal is subjected to EMD processing to obtain audio timing IMF components; Time-frequency features are extracted from the temporal IMF components of the sound to obtain the sound time-frequency features; The sound temporal IMF component and the sound time-frequency feature are input into the sound IMF prediction model to obtain at least one sound temporal IMF component; Obtain a predefined residual sound component, and linearly superimpose the at least one temporal IMF component and the residual sound component to obtain a temporal prediction signal. The step of determining the electrical time-series prediction signal based on the electrical IMF prediction model and the electrical time-series signal includes: The electrical timing signal is subjected to EMD processing to obtain the electrical timing IMF component; Time-frequency features are extracted from the electrical time-series IMF components to obtain electrical time-frequency features; The electrical time-series IMF component and the electrical time-frequency feature are input into the electrical IMF prediction model to obtain at least one electrical time-series IMF component; Obtain a predefined electrical residual component, and linearly superimpose the at least one electrical time-series IMF component and the electrical residual component to obtain an electrical time-series prediction signal; The step of performing a first feature interaction operation on the sound timing features and the electrical timing features to obtain a first sound-electrical interaction feature includes: The sound temporal features are taken as input, and a first attention weight vector is generated through a first conditional attention network. The first conditional attention network includes a first semantic matrix, which is used to transform the sound temporal features from the current semantic space to the semantic space where the electrical temporal features are located. The sound temporal features are taken as input, and a second attention weight vector is generated through a second conditional attention network. The second conditional attention network includes a second semantic matrix, which is used to transform the sound temporal features from the current semantic space to the semantic space where the electrical temporal features are located. The electrical timing features are weighted and multiplied according to the first attention weight vector and the second attention weight vector to obtain the first sound timing weighted feature and the second sound timing weighted feature. The first audio time-weighted feature, the second audio time-weighted feature, and the electrical time-weighted feature are weighted and concatenated to obtain the first audio-electrical interaction feature; The step of performing a second feature interaction on the sound timing prediction feature and the electrical timing prediction feature to obtain a second sound-electrical interaction feature includes: The sound temporal prediction feature is used as input, and a third attention weight vector is generated through a third conditional attention network. The third conditional attention network includes a third semantic matrix, which is used to transform the sound temporal prediction feature from the current semantic space to the semantic space where the electrical temporal prediction feature is located. The sound temporal prediction feature is used as input, and a fourth attention weight vector is generated through a fourth conditional attention network. The fourth conditional attention network includes a fourth semantic matrix, which is used to transform the sound temporal prediction feature from the current semantic space to the semantic space where the electrical temporal prediction feature is located. The electrical timing prediction features are weighted and multiplied according to the third attention weight vector and the fourth attention weight vector respectively to obtain the first sound timing weighted prediction feature and the second sound timing weighted prediction feature; The first audio time-weighted prediction feature, the second audio time-weighted prediction feature, and the electrical time-weighted prediction feature are weighted and concatenated to obtain the second audio-electrical interaction feature.

2. The network node monitoring method based on the Industrial Internet of Things according to claim 1, characterized in that, The step of performing feature engineering operations on the audio timing signal, the electrical timing signal, the audio timing prediction signal, and the electrical timing prediction signal respectively to obtain the corresponding audio timing features, electrical timing features, audio timing prediction features, and the electrical timing prediction features includes: The audio timing signal, the electrical timing signal, the audio timing prediction signal, and the electrical timing prediction signal are respectively subjected to latent space mapping to obtain the corresponding audio timing latent representation, electrical timing latent representation, audio timing prediction latent representation, and electrical timing prediction latent representation; High-level semantic features are extracted from the latent temporal representation of the sound to obtain high-level semantic features of the sound temporal sequence; High-level semantic features are extracted from the electrical time-series latent representation to obtain high-level semantic features of the electrical time series. High-level semantic features are extracted from the latent representation of sound temporal prediction to obtain high-level semantic features of sound temporal prediction. High-level semantic features are extracted from the potential representation of electrical time series prediction to obtain high-level semantic features of electrical time series prediction.

3. The network node monitoring method based on the Industrial Internet of Things according to claim 2, characterized in that, The step of extracting high-level semantic features from the temporal latent representation of the sound to obtain high-level semantic features of the sound temporal sequence includes: Self-attention processing is performed on the latent temporal representation of the sound to obtain the temporal attention features of the sound. A residual connection is performed between the audio temporal attention features and the audio temporal latent representation to obtain the residual connection result; The residual connection results are subjected to layer normalization to obtain the first intermediate feature; The first intermediate feature is nonlinearly transformed using a feedforward neural network to obtain the second intermediate feature; The second intermediate feature is residually connected to the first intermediate feature, and the connection result is layer normalized to obtain the high-level semantic features of the sound temporal sequence.

4. A network node monitoring system based on the Industrial Internet of Things, characterized in that, It includes a management platform, a sensor network platform, and an object platform that are connected in sequence. The management platform is configured with: The prediction module is used to acquire the audio timing signal and electrical timing signal of the target network node, determine the audio timing prediction signal based on the audio timing signal and the electrical timing prediction signal based on the electrical timing signal using the electrical timing signal and the electrical timing prediction signal. The feature engineering module is used to perform feature engineering operations on the audio timing signal, the electrical timing signal, the audio timing prediction signal, and the electrical timing prediction signal respectively to obtain the corresponding audio timing features, electrical timing features, audio timing prediction features, and electrical timing prediction features; The feature interaction module is used to perform a first feature interaction operation on the sound temporal feature and the electrical temporal feature to obtain a first sound-electrical interaction feature, and to perform a second feature interaction operation on the sound temporal prediction feature and the electrical temporal prediction feature to obtain a second sound-electrical interaction feature. The first sound-electrical interaction feature is used to represent the multimodal cooperative state of the target network node at the current moment, and the second sound-electrical interaction feature is used to represent the multimodal cooperative evolution trend of the target network node at future moments. The monitoring module is used to evaluate the current state of the target network node based on the network node monitoring model and according to the first voice-electric interaction feature and the second voice-electric interaction feature, so as to obtain the node state evaluation result of the target network node and realize the monitoring of the target network node. The step of determining the sound time-series prediction signal based on the sound IMF prediction model and the sound time-series signal includes: The audio timing signal is subjected to EMD processing to obtain audio timing IMF components; Time-frequency features are extracted from the temporal IMF components of the sound to obtain the sound time-frequency features; The sound temporal IMF component and the sound time-frequency feature are input into the sound IMF prediction model to obtain at least one sound temporal IMF component; Obtain a predefined residual sound component, and linearly superimpose the at least one temporal IMF component and the residual sound component to obtain a temporal prediction signal. The step of determining the electrical time-series prediction signal based on the electrical IMF prediction model and the electrical time-series signal includes: The electrical timing signal is subjected to EMD processing to obtain the electrical timing IMF component; Time-frequency features are extracted from the electrical time-series IMF components to obtain electrical time-frequency features; The electrical time-series IMF component and the electrical time-frequency feature are input into the electrical IMF prediction model to obtain at least one electrical time-series IMF component; Obtain a predefined electrical residual component, and linearly superimpose the at least one electrical time-series IMF component and the electrical residual component to obtain an electrical time-series prediction signal; The step of performing a first feature interaction operation on the sound timing features and the electrical timing features to obtain a first sound-electrical interaction feature includes: The sound temporal features are taken as input, and a first attention weight vector is generated through a first conditional attention network. The first conditional attention network includes a first semantic matrix, which is used to transform the sound temporal features from the current semantic space to the semantic space where the electrical temporal features are located. The sound temporal features are taken as input, and a second attention weight vector is generated through a second conditional attention network. The second conditional attention network includes a second semantic matrix, which is used to transform the sound temporal features from the current semantic space to the semantic space where the electrical temporal features are located. The electrical timing features are weighted and multiplied according to the first attention weight vector and the second attention weight vector to obtain the first sound timing weighted feature and the second sound timing weighted feature. The first audio time-weighted feature, the second audio time-weighted feature, and the electrical time-weighted feature are weighted and concatenated to obtain the first audio-electrical interaction feature; The step of performing a second feature interaction on the sound timing prediction feature and the electrical timing prediction feature to obtain a second sound-electrical interaction feature includes: The sound temporal prediction feature is used as input, and a third attention weight vector is generated through a third conditional attention network. The third conditional attention network includes a third semantic matrix, which is used to transform the sound temporal prediction feature from the current semantic space to the semantic space where the electrical temporal prediction feature is located. The sound temporal prediction feature is used as input, and a fourth attention weight vector is generated through a fourth conditional attention network. The fourth conditional attention network includes a fourth semantic matrix, which is used to transform the sound temporal prediction feature from the current semantic space to the semantic space where the electrical temporal prediction feature is located. The electrical timing prediction features are weighted and multiplied according to the third attention weight vector and the fourth attention weight vector respectively to obtain the first sound timing weighted prediction feature and the second sound timing weighted prediction feature; The first audio time-weighted prediction feature, the second audio time-weighted prediction feature, and the electrical time-weighted prediction feature are weighted and concatenated to obtain the second audio-electrical interaction feature.

5. A computer device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method of any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and execute the method of any one of claims 1 to 3.

Images