A device state diagnosis method and system of a multi-channel fusion neural network, an electronic device, and a computer readable medium

Abstract

The application discloses a kind of multi-channel fusion neural network equipment state diagnosis method and system, electronic equipment, computer readable medium, method includes: the sensor data of arrangement in equipment is collected, and pre-processing is carried out, and equipment state multi-channel data is constructed;Equipment state multi-channel data is input into multi-component signal analysis model, wherein using empirical mode decomposition model obtains equipment state decomposition signal;Equipment state decomposition signal is input into non-negative matrix factorization model, and independent source equipment state signal is separated out;Independent source equipment state signal is extracted in time-frequency domain feature in combination with improved hilbert-huang transform, and time-frequency domain feature result is obtained, and equipment state time-frequency diagram is reconstructed from time-frequency domain feature result;Equipment state time-frequency diagram is input into based on multi-scale sparse time-frequency convolution network model, and equipment state diagnosis result is obtained.The equipment fault diagnosis of the application is realized by obtaining the multi-channel data of equipment, and the diagnostic accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of equipment state fault diagnosis, and particularly relates to an equipment state diagnosis method and system based on a multi-channel fusion neural network, an electronic device, and a computer readable medium. BACKGROUND

[0002] With the rapid development of technology, the number of global industrial equipment is showing an explosive growth. According to the data of market research agency Statista, the number of global industrial Internet of Things (IIoT) devices has exceeded 15 billion in 2022, and it is expected to break through 30 billion by 2030. The huge equipment scale poses unprecedented challenges to the state monitoring and management of equipment. In order to ensure the normal operation of these equipment, reduce downtime and reduce maintenance costs, equipment state diagnosis technology needs to be improved, and will gradually become an important pillar of the era of intelligent manufacturing and Industry 4.0. Equipment state diagnosis is a key technology that judges the health status of equipment and predicts potential failures by real-time monitoring and analyzing equipment operation data. According to McKinsey's research, enterprises can reduce maintenance costs by 20%-30% and reduce equipment downtime by about 50% by implementing effective equipment state diagnosis and predictive maintenance. In the modern industrial environment, the improvement of fault diagnosis technology helps enterprises save millions of dollars in costs and greatly improve production efficiency.

[0003] Traditional equipment maintenance mode usually relies on regular inspection and manual maintenance, whose limitations are obvious. For example, the direct loss caused by equipment failure in global manufacturing industry is more than 5 billion US dollars per year. A large part of these losses is due to the failure to discover potential problems of equipment in time. Therefore, how to use advanced technical means to monitor the real-time state of equipment and make accurate diagnosis has become the focus of attention of various enterprises. With the progress of sensor technology, equipment state diagnosis technology has made great progress. Today, most industrial equipment is equipped with various sensors, such as temperature sensors, pressure sensors and vibration sensors. These sensors can collect various key data in the operation of the equipment in real time, and transmit the data to the central processing system for analysis through the Internet of Things platform. Using advanced data analysis techniques such as big data analysis and machine learning algorithms, equipment state diagnosis can deeply mine and analyze massive equipment data to identify abnormal behavior and potential failure patterns of equipment. With the rapid development of deep learning technology, its application in equipment state diagnosis has attracted widespread attention. Through the construction of multi-layer neural networks, complex data can be feature extracted and pattern recognized, showing great potential in the field of equipment fault diagnosis. Many studies have shown that deep learning algorithms can automatically extract features from a large amount of equipment data and accurately classify and predict the health status of equipment. For example, convolutional neural networks (CNN) and long short-term memory networks (LSTM) have been widely used in mechanical equipment fault diagnosis, which can effectively identify bearing failure, motor abnormalities and other problems. However, the analysis of multiple sensor data in existing equipment state diagnosis is isolated, resulting in poor correlation analysis between sensor data, and the original analysis is in the time domain, making it difficult to analyze data characteristics in higher dimensions, which poses a challenge to the accuracy of equipment fault state diagnosis.

[0004] Therefore, due to the problems of multi-channel sensor data correlation of equipment fault state and effective mining of fault state in time domain in existing traditional methods and artificial intelligence methods, a new fault state diagnosis method is needed to further improve the accuracy of fault diagnosis. SUMMARY

[0005] In order to solve the above technical problems and overcome the shortcomings of the prior art, the present application provides a multi-channel fusion neural network equipment state diagnosis method and system, an electronic device and a computer readable medium. The fault diagnosis method of multi-channel fusion neural network is introduced, which can obtain better recognition effect in the field of equipment fault diagnosis and greatly improve the diagnosis performance. It can improve the accuracy of fault detection, reduce the maintenance cost, and improve the operation safety and reliability of equipment in the factory. In addition, the implementation of this method can also provide valuable data support for equipment manufacturers and maintenance service providers, helping to optimize product design and maintenance strategy.

[0006] The technical solution of the present application is as follows:

[0007] According to a first aspect, the present application provides a device state diagnosis method of a multi-channel fusion neural network, comprising:

[0008] Step S1: collecting sensor data arranged at key parts of a device and performing preprocessing, thereby constructing device state multi-channel data;

[0009] Step S2: inputting the device state multi-channel data into a multi-component signal analysis model, the multi-component signal analysis model comprising an empirical mode decomposition model and a non-negative matrix factorization model, wherein the empirical mode decomposition model is used to obtain device state decomposition signals from the device state multi-channel data; inputting the device state decomposition signals into the non-negative matrix factorization model to separate independent source device state signals; performing time-frequency domain feature extraction on the independent source device state signals by combining an improved Hilbert-Huang transform to obtain time-frequency domain feature results, and reconstructing a device state time-frequency diagram from the time-frequency domain feature results;

[0010] Step S3: inputting the device state time-frequency diagram into a multi-scale sparse time-frequency convolution network model to obtain a device state diagnosis result.

[0011] According to a second aspect, a device state diagnosis system of a multi-channel fusion neural network is provided, comprising the following modules:

[0012] A device fault multi-channel data module: collecting sensor data arranged at key parts of a device and performing preprocessing to construct device state multi-channel data;

[0013] A device state diagnosis module of a multi-channel fusion neural network: inputting the device state multi-channel data into a multi-component signal analysis model, the multi-component signal analysis model comprising an empirical mode decomposition model and a non-negative matrix factorization model, wherein the empirical mode decomposition model is used to obtain device state decomposition signals from the device state multi-channel data; inputting the device state decomposition signals into the non-negative matrix factorization model to separate independent source device state signals; performing time-frequency domain feature extraction on the independent source device state signals by combining an improved Hilbert-Huang transform to obtain time-frequency domain feature results, and reconstructing a device state time-frequency diagram from the time-frequency domain feature results; inputting the device state time-frequency diagram into a multi-scale sparse time-frequency convolution network model to obtain a device state diagnosis result.

[0014] According to a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory is used to store a computer program executable by the processor, and the processor implements the device state diagnosis method of a multi-channel fusion neural network as described above when executing the computer program.

[0015] According to a fourth aspect, the present application provides a computer readable medium, having stored thereon a computer program which, when executed, implements the method for device state diagnosis of a multi-channel fusion neural network as described above.

[0016] Compared with the prior art, the present application has the following beneficial effects:

[0017] (1) The present application provides a method or system for device state diagnosis of a multi-channel fusion neural network, which can identify device fault states. For the first time, through multi-channel fusion, it can integrate multi-dimensional information provided by multiple sensors to make up for possible missing fault features of a single signal source. This method enhances the comprehensiveness and robustness of the device fault diagnosis system, enabling the device state diagnosis to cope with more complex and diverse fault modes.

[0018] (2) The method or system for device state diagnosis of a multi-channel fusion neural network provided by the present application converts time series data into time-frequency graphs, which helps to capture changes in signals in both time and frequency dimensions. This method is more intuitive and effective than traditional time-domain or frequency-domain analysis, especially when dealing with non-stationary signals, it can more accurately identify potential fault features and improve the accuracy of device fault diagnosis.

[0019] (3) The method or system for device state diagnosis of a multi-channel fusion neural network provided by the present application can extract key features of device operating states at different time and frequency scales through the application of a multi-scale sparse time-frequency convolution network. Compared with traditional convolutional neural networks, multi-scale convolutional networks can more comprehensively capture multi-scale features of device faults, significantly improving the sensitivity and accuracy of the device fault diagnosis system.

[0020] (4) The method or system for device state diagnosis of a multi-channel fusion neural network provided by the present application makes the model more efficient in high-dimensional data processing through the design of multi-scale sparse convolution, avoiding the problem of excessively high computational complexity. At the same time, sparse connections reduce the risk of overfitting and enhance the model's generalization ability under different working conditions, thereby improving the reliability and applicability of the device fault diagnosis system in practical applications. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The flowchart of the method for device state diagnosis of a multi-channel fusion neural network in an embodiment of the present application is shown in Figure 1.

[0022] Figure 2 The flowchart of step S2 in the method for device state diagnosis of a multi-channel fusion neural network in an embodiment of the present application is shown in Figure 2.

[0023] Figure 3A flow chart of step S22 in the device state diagnosis method of the multi-channel fusion neural network in the embodiment of the present application is shown in the figure.

[0024] Figure 4 A flow chart of step S23 in the device state diagnosis method of the multi-channel fusion neural network in the embodiment of the present application is shown in the figure.

[0025] Figure 5 A flow chart of step S3 in the device state diagnosis method of the multi-channel fusion neural network in the embodiment of the present application is shown in the figure.

[0026] Figure 6 A structural block diagram of the device state diagnosis system of the multi-channel fusion neural network in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below with specific embodiments and in conjunction with the drawings.

[0028] Embodiment one

[0029] Figure 1 A flow chart of the device state diagnosis method of the multi-channel fusion neural network in the embodiment of the present application is shown in the figure. Figure 1 The method comprises the following steps.

[0030] Step S1: Collecting sensor data arranged on the device and performing preprocessing, thereby constructing device state multi-channel data.

[0031] The device state diagnosis method of the multi-channel fusion neural network provided by the present application is based on sensor measurement signal data of device faults. The installed sensors include vibration sensors, sound sensors, temperature sensors, current and voltage sensors, position sensors and load sensors; the device state data of the above-mentioned sensors are collected and pre-processing operations are used, including filter denoising, normalization, missing value supplement, to construct device state multi-channel data, which enhances the comprehensiveness and robustness of the diagnosis system.

[0032] Step S2: Inputting the device state multi-channel data into a multi-component signal analysis model, the multi-component signal analysis model comprising an empirical mode decomposition model and a non-negative matrix factorization model, wherein the empirical mode decomposition model is used to obtain device state decomposition signals from the device state multi-channel data; inputting the device state decomposition signals into the non-negative matrix factorization model to separate independent source device state signals in the device state decomposition signals; performing time-frequency domain feature extraction on the independent source device state signals by using an improved Hilbert-Huang transform to obtain time-frequency domain feature results, and reconstructing a device state time-frequency diagram from the time-frequency domain feature results;

[0033] Among them, the Empirical Mode Decomposition (EMD) model is an adaptive signal time-frequency processing method. Essentially, it's a method of decomposing a signal into a superposition of independent components, based on the time-scale characteristics of the data itself. The core of the Non-negative Matrix Factorization (NMF) model lies in decomposing a non-negative matrix into two smaller non-negative matrices. The details of these models are well-known to those skilled in the art and will not be elaborated upon here.

[0034] Step S3: Input the device status time-frequency map into the multi-scale sparse time-frequency convolutional network model to obtain the device status diagnosis result.

[0035] By inputting the device status time-frequency map into a multi-scale sparse time-frequency convolutional network model, the accuracy of device status diagnosis can be effectively improved. The design of multi-scale sparse convolution makes the model more efficient in high-dimensional data processing and avoids the problem of excessive computational complexity.

[0036] like Figure 2 As shown, in one embodiment, step S2 above involves: inputting the multi-channel device state data into a multi-component signal analysis model, which includes an empirical mode decomposition model and a non-negative matrix decomposition model, wherein the empirical mode decomposition model is used to obtain a device state decomposed signal from the multi-channel device state data; inputting the device state decomposed signal into the non-negative matrix decomposition model to separate independent source device state signals; performing time-frequency domain feature extraction on the independent source device state signals using an improved Hilbert-Huang transform to obtain time-frequency domain feature results, and reconstructing a device state time-frequency diagram from the time-frequency domain feature results, including:

[0037] Step S21: Transfer the device status multi-channel data The input is fed into the empirical mode decomposition model, and the trend term is obtained. and intrinsic mode functions ,in Furthermore, according to the following formula (1), and The device state decomposition signal is obtained by performing addition and reconstruction. ;

[0038] (1)

[0039] Step S22: Decompose the device status signal The input is fed into a nonnegative matrix factorization model to separate the independent source device state signals. ;

[0040] Step S23: Transfer the status signal of the independent source device. The improved Hilbert-Huang transform model is used to extract time-frequency domain features, and the time-frequency domain feature results are obtained. The device state time-frequency diagram is then reconstructed from the time-frequency domain feature results.

[0041] The empirical mode decomposition (EMD) model decomposes the preprocessed multi-channel equipment state data to obtain intrinsic mode functions (IMFs). The purpose of this process is to decompose complex equipment state signals into several simple, single-frequency oscillation modes. The decomposed signals obtained from the EMD model are input into a non-negative matrix factorization (NMF) model, which can separate independent source signals. The NMF model can identify and extract independent feature signals of the equipment state. Time-frequency domain features are extracted from the independent source equipment state signals separated by the NMF model using an improved Hilbert-Huang Transform (HHT). The improved HHT can accurately capture the local characteristics of the signal, improving the resolution of time-frequency analysis. The equipment state time-frequency map is reconstructed from the time-frequency domain feature results, including: reconstructing the extracted time-frequency domain feature results to generate a two-dimensional time-frequency image of the equipment state; performing pooling operations on the features of the two-dimensional time-frequency image to compress feature information and highlight the most important features; and fusing the features through a stitching operation to obtain a multi-scale fault characteristic equipment state time-frequency map, providing an intuitive visual representation and rich feature information for subsequent fault diagnosis.

[0042] like Figure 3 As shown, in one embodiment, step S22 above: inputting the device state decomposition signal into a non-negative matrix factorization model to separate and obtain independent source device state signals, includes:

[0043] Step S221: Define the problem and initialize: Decompose the device state signal using a non-negative matrix factorization model. Decomposed into two nonnegative matrices and , making ,in It is the original non-negative matrix, i.e., an approximate device state decomposition signal; two matrices are randomly initialized. and and ensure and It is a non-negative matrix;

[0044] Step S222: Iterative Update: Fixed Matrix By optimizing the matrix To minimize the loss function, the training weights are updated using the gradient descent update rule; the matrix is ​​fixed. By optimizing the matrix To minimize the loss function, the gradient descent update rule is used to update the training weights;

[0045] Step S223: Convergence judgment: After each iteration, calculate the current loss function value; when the change in the loss function is less than the preset threshold, or when the maximum number of iterations is reached, the algorithm converges and the algorithm terminates when the convergence condition is met, taking the first condition reached.

[0046] Step S224: Reconstruct the status signals of independent source devices: matrix Each column is represented as an independent source signal, through The columns in Multiply to reconstruct the state signals of the independent source devices. .

[0047] Before inputting the equipment state decomposition signal into the nonnegative matrix factorization (NMF) model, the equipment state decomposition signal needs to be preprocessed to ensure signal quality and reduce interference factors in data analysis. The intrinsic mode functions (IMFs) obtained from the decomposition are constructed into a data matrix, where each row represents an IMF and each column represents the signal value at a time point. In the NMF model, two nonnegative matrices need to be initialized, namely the basis matrix. sum coefficient matrix basis matrix Each column of the coefficient matrix represents a potential signal component. Each row represents the weight of these components in the device state decomposition signal; when the NMF model converges, from the basis matrix Independent components are extracted, representing independent source device status signals. Each independent source device status signal is a component of the original signal, reflecting the device's behavior at a specific frequency.

[0048] like Figure 4 As shown, in one embodiment, step S23 above involves: transmitting the independent source device status signal. Time-frequency domain feature extraction is performed using an improved Hilbert-Huang transform model to obtain the aforementioned time-frequency domain feature results. A device state time-frequency diagram is then reconstructed from these results, including:

[0049] Step S231: Transfer the status signal of the independent source device. Add white noise Obtain the status signal of the independent source device with added noise disturbance. ;

[0050] Step S232: The status signal of the independent source device with added noise disturbance is... Empirical Mode Decomposition (EMD) is performed to decompose the model and obtain the intrinsic mode functions. Then take the intrinsic mode functions. The average value of the set of intrinsic mode functions is obtained according to the following formula (2) :

[0051] (2)

[0052] wherein, is the intrinsic mode function after adding the nth noise . , is the total number of added noises.

[0053] Step S233: Calculate the permutation entropy of each intrinsic mode function under different time scales according to the following formula (3) , and select the intrinsic mode function with the lowest permutation entropy as the effective signal;

[0054] (3)

[0055] wherein, represents the dimension used to construct the vector of the intrinsic mode function when performing phase space reconstruction. is the time scale, is the number of all possible permutations taken from the different elements, is the probability of permutation occurrence, which represents the specific permutation corresponding to the specific ordering of the subsequence after phase space reconstruction, wherein is the specific permutation.

[0056] Step S234: Use the improved normalized Hilbert transform to calculate the normalized instantaneous phase and the instantaneous frequency , which improves the accuracy of the instantaneous frequency and reduces the constraint of the Bedrosian theorem through the normalization operation.

[0057] wherein, represents the analytic signal obtained through the Hilbert transform model at time ; represents the real part of the analytic signal , represents the modulus value of the analytic signal , represents the arccosine function for calculating the angle; represents the derivative of the normalized instantaneous phase with respect to time;

[0058] ​​​​Step S235: performing statistical analysis on the instantaneous frequency of the transient frequency , extracting time-frequency domain features by using the average operation, obtaining the time-frequency domain feature extraction result, and reconstructing the device state time-frequency feature map using the feature splicing fusion method;

[0059] Wherein, in order to improve the performance of Hilbert-Huang transform model (HHT), the application introduces the improvement measures of normalization operation, designs the improved Hilbert transform model, so as to improve the accuracy of instantaneous frequency estimation and capture the time-frequency characteristics of signal; after applying the improved HHT to each IMF, a series of instantaneous frequency and instantaneous amplitude are obtained. These time-frequency domain features can reflect the changes of device state signal at different times and frequencies. From the extracted time-frequency domain features, the most representative and most informative features are selected. The selected time-frequency domain features are used to reconstruct the time-frequency graph of the device state. This step involves mapping two-dimensional time-frequency data (time-frequency) to a graph to visually show the behavior of the device state signal at different times and frequencies. Analyzing and extracting features from the reconstructed device state time-frequency feature map can identify the change trend and potential failure mode of the device state. The training process of Hilbert-Huang transform model includes: using cross-entropy loss function, according to the following formula (4), wherein the sparse regularization term is combined to optimize the model training:

[0060] (4)

[0061] Wherein, represents the category, i.e. the number of device state categories in the classification problem (such as device normal, failure, etc.), represents the true label of the th class, represents the prediction probability of the th class, represents the regularization coefficient, represents the number of convolutional network layers, takes the logarithmic function, represents the weight matrix of the th convolution kernel of the th layer, the number of categories of the th layer.

[0062] As shown in Figure 5 , in one embodiment, the above step S3: using a multi-scale sparse time-frequency convolutional network model on the device state time-frequency graph to obtain a device state diagnosis result, specifically comprising:

[0063] Step S31: Perform a convolution operation on the device state time-frequency feature map using multi-scale convolution kernels, and obtain the device state time-frequency features according to the following formula (5);

[0064] (5)

[0065] in, Indicates the convolution output. It is the first Layer A convolutional kernel with 1 channel, It is the eigenvector of the time-frequency feature map. It is the weight vector of the convolution operation. OK, It is the weight vector of the convolution operation. OK, It is the eigenvector of the time-frequency feature map. OK, It is the eigenvector of the time-frequency feature map. List, It is an activation function. It is a bias term;

[0066] Step S32: Apply sparse regularization to the device state time-frequency feature map, and sparsify the output of the convolutional layer according to the following formula (6) to obtain the sparse convolution output result;

[0067] (6)

[0068] in, Represents the regularization coefficient. Indicates the first Layer Convolutional kernels for each channel;

[0069] Step S33: The sparse convolution output is fused using a feature concatenation operation. According to the following formula (7), the fused feature map result is obtained. ;

[0070] (7)

[0071] in, Indicates the first Feature maps at various scales;

[0072] Step S34: Introduce the feature map results into the attention mechanism, and perform weighted processing on the fused feature map according to the following formula (8) to obtain the attention-weighted feature map. ;

[0073] (8)

[0074] in, This indicates that the first [item] on the feature map... , The scoring function is based on the location characteristics. It is a feature map No. OK, It is a feature map No. List;

[0075] Step S35: Calculate the attention-weighted feature map. Max pooling is used for multi-scale downsampling, and the downsampled feature map is input into a fully connected layer. The fully connected layer flattens the feature map into a one-dimensional feature vector and outputs the probability of the fault category through the softmax activation function for fault diagnosis of equipment status.

[0076] The multi-scale convolutional network, using convolutional kernels of different sizes, extracts multi-scale features from different resolutions of the time-frequency map, capturing the state changes of the device within different time frequency ranges. Different scales of convolutional kernels can extract features at different time scales, capturing rapidly changing or long-term trending device states. To ensure feature sparsity, L1 regularization is introduced to limit the weights of the convolutional kernels, making the output feature map more sparse, thus retaining only key features related to fault or health states. Each layer of the convolutional network extracts time-frequency features at different scales. To generate a comprehensive representation of the device's state, features at different scales are fused. To further enhance the focus on key features, an attention mechanism is employed, enabling the model to better focus on important parts of the time-frequency map. The attention mechanism assigns different weights to different locations on the time-frequency map to highlight the feature regions of the device's operating state. After attention weighting, multi-scale downsampling is used to reduce the dimensionality of the feature map while retaining important features; and pooling operations compress the data volume, reduce computational complexity, and retain key information.

[0077] Example 2

[0078] like Figure 6 As shown, a device status diagnostic system using a multi-channel fused neural network includes the following modules:

[0079] A multi-channel data module 41 for equipment faults is constructed to collect sensor data deployed in key parts of the equipment and perform preprocessing to construct multi-channel data of equipment status.

[0080] The device state diagnosis module 42 of the multi-channel fusion neural network is used for inputting the device state multi-channel data into a multi-component signal analysis model, wherein a device state decomposition signal is obtained by using an empirical mode decomposition model; the device state decomposition signal is input into a non-negative matrix factorization model to separate an independent source device state signal; time-frequency domain feature extraction is performed on the independent source device state signal by using an improved Hilbert-Huang transform, time-frequency domain feature results are obtained, and a device state time-frequency diagram is reconstructed from the time-frequency domain feature results; and a multi-scale sparse time-frequency convolution network model is used for the device state time-frequency diagram to obtain a device state diagnosis result.

[0081] Optionally, the present application further provides an electronic device (computer, server, smart phone, network device, etc.), comprising a memory and a processor, wherein the memory is used for storing a computer program executable by the processor, and the processor executes the computer program to implement the above-mentioned method embodiments or device embodiments.

[0082] Optionally, the present application further provides a program product, for example, a computer readable storage medium, comprising a program, which is used for executing the above-mentioned method embodiments or device embodiments when executed by a processor.

[0083] The above embodiments are provided only for the purpose of describing the present application, and are not intended to limit the scope of the present application. The scope of the present application is defined by the appended claims. Various equivalent replacements and modifications made without departing from the spirit and principles of the present application shall be encompassed within the scope of the present application.

Claims

1. A device state diagnosis method of a multi-channel fusion neural network, characterized by, The method comprises the following steps: Step S1: collecting sensor data arranged on the equipment and preprocessing to construct equipment state multi-channel data; Step S2: inputting the equipment state multi-channel data into a multi-component signal analysis model, wherein the multi-component signal analysis model comprises an empirical mode decomposition model and a non-negative matrix factorization model, using the empirical mode decomposition model to obtain equipment state decomposition signals from the equipment state multi-channel data, inputting the equipment state decomposition signals into the non-negative matrix factorization model to separate independent source equipment state signals, combining an improved Hilbert-Huang transform to perform time-frequency domain feature extraction on the independent source equipment state signals to obtain time-frequency domain feature results, and reconstructing an equipment state time-frequency feature map from the time-frequency domain feature results; Step S3: inputting the equipment state time-frequency feature map into a multi-scale sparse time-frequency convolution network model to obtain equipment state diagnosis results, comprising: Step S31: performing convolution operation on the equipment state time-frequency feature map using a multi-scale convolution kernel to obtain equipment state time-frequency feature results according to formula (5); (5) in, Indicates the convolution output. It is the first Layer A convolutional kernel with 1 channel, It is the eigenvector of the time-frequency feature map. It is the weight vector of the convolution operation. OK, It is the weight vector of the convolution operation. List, It is the eigenvector of the time-frequency feature map. OK, It is the eigenvector of the time-frequency feature map. List, It is an activation function. It is a bias term; Step S32: performing sparse regularization on the equipment state time-frequency feature map to obtain sparse convolution output results according to formula (6); (6) wherein denotes a regularization coefficient; Step S33: the sparse convolution output result is fused by using a feature concatenation operation, and a fused feature map result is obtained according to formula (7) as follows ; (7) Step S34: obtaining the feature map result The attention mechanism operation is introduced, and the feature map after attention weighting is obtained according to formula (8) as follows ; is the attention weight (8) in, This indicates that the first [item] on the feature map... , The scoring function is based on the location characteristics. It is a feature map No. OK, It is a feature map No. List; Step S35: the attention weighted feature map The multi-scale down-sampling is performed by using the maximum pooling, and the feature map subjected to the down-sampling processing is input into a full connection layer, wherein the full connection layer flattens the feature map into a one-dimensional feature vector, and outputs the probability of the fault category through a softmax activation function, so as to perform the fault diagnosis on the device state.

2. The device state diagnosis method of a multi-channel fusion neural network according to claim 1, characterized in that, The step S2 comprises: Step S21: inputting the device state multi-channel data to an empirical mode decomposition model to obtain a trend item and an intrinsic mode function , wherein ; and adding and according to the following formula (1) to reconstruct a device state decomposition signal : (1) Step S22: decomposing the device state signal into a non-negative matrix factorization model to separate independent source device state signals ; Step S23: sending the independent source device status signal The time-frequency domain feature extraction is performed using the improved Hilbert-Huang transform model to obtain the time-frequency domain feature result, and a device status time-frequency feature map is reconstructed from the time-frequency domain feature result.

3. The method according to claim 2, wherein The step S22 comprises: Step S221 : Defining the problem and initialization: The device state decomposition signal is decomposed into two non-negative matrices and matrix such that matrix , is the original non-negative matrix; randomly initialize the two matrices and matrix and ensure that matrices and matrix are non-negative matrices; Step S222: Iterative update: fixing the matrix , minimizing the loss function by optimizing the matrix and updating the training weights using gradient descent update rule; fixing the matrix , minimizing the loss function by optimizing the matrix and updating the training weights using gradient descent update rule; Step S223: convergence judgment: calculating the loss function value of the current training after each iteration; when the change of the loss function value is less than a preset threshold value or the maximum number of iterations is reached, the algorithm is terminated when the first condition is met, and the algorithm converges to meet the convergence condition; Step S224: reconstructing independent source device status signals: the matrix is multiplied by the matrix , and the independent source device status signals are reconstructed. ​ 4. The method of claim 2, wherein, The step S23 comprises: Step S231: adding a white noise to the independent source device status signal adding white noise , obtaining an independent source device status signal with noise disturbance added ; Step S232: adding the independent source device state signal with noise disturbance Performing empirical mode decomposition to obtain a set of intrinsic mode functions Then taking the average value of each intrinsic mode function Obtaining the set average decomposition signal of according to the following formula (2) : (2) wherein is the first decomposed eigenmode function, is the first is the total number of decompositions, is the total number of eigenmode functions; Step S233: According to the following formula (3), the permutation entropy of each is calculated The intrinsic mode function corresponding to the lowest permutation entropy is selected as the effective signal ​ (3) wherein, represents the dimension used to construct the vector of the eigenmode function when performing phase space reconstruction, is the time scale, is the number of all possible orderings taken from the individual elements, is the permutation occurs, represents the probability of each permutation corresponds to a specific ordering of the subsequence after phase space reconstruction; Step S234: selecting the intrinsic mode function with the lowest permutation entropy Using the improved normalized Hilbert transform model, the normalized instantaneous phase and the instantaneous frequency are calculated in, Represents the original signal. This indicates that the value obtained from the Hilbert transform model at time [time] is [value]. Analyzed signals; Indicates analytic signal The real part, Indicates analytic signal The modulus, This represents the inverse cosine function, used to calculate angles; This represents the derivative of the normalized instantaneous phase with respect to time. Step S235: the said instantaneous frequency statistical analysis is performed, the time-frequency domain features are extracted by using the mean operation, and the time-frequency domain feature extraction result is obtained, and the device state time-frequency feature map is reconstructed.

5. A device state diagnosis system of a multi-channel fusion neural network, characterized by, The method comprises the following modules: An equipment fault multi-channel data module: collecting sensor data arranged on key parts of the equipment and preprocessing to construct equipment state multi-channel data; A multi-channel fusion neural network equipment state diagnosis module: inputting the equipment state multi-channel data into a multi-component signal analysis model, wherein the multi-component signal analysis model comprises an empirical mode decomposition model and a non-negative matrix factorization model, using the empirical mode decomposition model to obtain equipment state decomposition signals from the equipment state multi-channel data, inputting the equipment state decomposition signals into the non-negative matrix factorization model to separate independent source equipment state signals, combining an improved Hilbert-Huang transform to perform time-frequency domain feature extraction on the independent source equipment state signals to obtain time-frequency domain feature results, and reconstructing an equipment state time-frequency feature map from the time-frequency domain feature results; Inputting the equipment state time-frequency feature map into a multi-scale sparse time-frequency convolution network model to obtain equipment state diagnosis results, comprising: Step S31: performing convolution operation on the equipment state time-frequency feature map using a multi-scale convolution kernel to obtain equipment state time-frequency feature results according to formula (5); (5) in, Indicates the convolution output. It is the first Layer A convolutional kernel with 1 channel, It is the eigenvector of the time-frequency feature map. It is the weight vector of the convolution operation. OK, It is the weight vector of the convolution operation. List, It is the eigenvector of the time-frequency feature map. OK, It is the eigenvector of the time-frequency feature map. List, It is an activation function. It is a bias term; Step S32: performing sparse regularization on the equipment state time-frequency feature map to obtain sparse convolution output results according to formula (6); (6) wherein denotes a regularization coefficient; Step S33: the sparse convolution output result is fused by using a feature concatenation operation, and a fused feature map result is obtained according to formula (7) as follows ; (7) Step S34: obtaining the feature map result The attention mechanism operation is introduced, and the feature map after attention weighting is obtained according to formula (8) as follows ; is the attention weight (8) wherein, represents a score function performed on features at the , position of the feature map, is the feature map row, , is the feature map column; and row; and Step S35: the attention weighted feature map Multi-scale down-sampling is performed by using max pooling, and the down-sampled feature map is input into a fully connected layer, wherein the fully connected layer flattens the feature map into a one-dimensional feature vector, outputs the probability of the fault category through a softmax activation function, and performs fault diagnosis on the device state.

6. An electronic device, comprising: The device state diagnosis method of the multi-channel fusion neural network according to any one of claims 1 to 4 is implemented by a computer program executable by a processor, and a memory is used to store the computer program.

7. A computer readable medium characterized by The computer readable medium stores a computer program, and the computer program implements the device state diagnosis method of the multi-channel fusion neural network according to any one of claims 1 to 4 when executed.

Images