Fault monitoring method and system, electronic equipment and storage medium

Abstract

The invention provides a fault monitoring method and system, electronic equipment and a storage medium, and the method comprises the steps: obtaining a target operation signal of a to-be-detected device; inputting the target operation signal into a pre-deployed fault monitoring model to obtain a fault monitoring result, including probabilities of the to-be-detected device in a normal state and different fault modes; the fault monitoring model is obtained by carrying out cross-domain feature fusion training on a source domain data set and a target domain data set through a cloud edge collaborative architecture, and comprises a multi-source domain feature extraction network deployed at a cloud end and a cross-domain feature fusion network deployed at an edge end; the multi-source domain feature extraction network is used for extracting fault features from the source domain data set and the target domain data set, and the cross-domain feature fusion network is used for performing cross-domain fusion on the multiple fault features. Cross-domain feature fusion training is realized through a cloud edge collaborative architecture, a fault monitoring model is constructed and deployed at an edge end for fault monitoring, and the precision and efficiency of device fault monitoring in industrial automation equipment are improved.

Description

Technical Field

[0001] The present application relates to the technical field of industrial automation equipment fault diagnosis, and in particular to a fault monitoring method, system, electronic device and storage medium. Background Art

[0002] With the rapid development of industry and intelligent manufacturing, the degree of factory automation continues to increase. Automated equipment plays a vital role in manufacturing and is widely used in fields such as automotive manufacturing and food processing. However, in actual operation, various components in automated equipment may experience various failures due to factors such as long-term load, environmental changes, and mechanical wear, which in turn affect the equipment's operational stability and production efficiency. Therefore, accurate and effective monitoring of component failures in equipment to enable rapid repair is key to ensuring stable equipment operation and production efficiency.

[0003] In related technologies, deep learning technology has a significant potential in knowledge discovery and feature perception. Fault monitoring for various components in equipment is often implemented using deep learning technology to identify different faults. This involves offline learning in the cloud based on the characteristics of the component's operating signals under various fault conditions to construct a fault monitoring model. This model is then used to identify unknown component faults online, eliminating the reliance on prior knowledge in fault identification and improving fault monitoring reliability to a certain extent. However, in actual industrial scenarios, relying solely on fault monitoring models trained in the cloud for fault monitoring still lacks accuracy and efficiency. Summary of the Invention

[0004] In view of the above shortcomings, the present application discloses a fault monitoring method, system, electronic device and storage medium to solve the technical problems of low accuracy and efficiency in device fault monitoring.

[0005] In a first aspect, the present application provides a fault monitoring method, the method comprising: obtaining a target operating signal of a device under test; inputting the target operating signal into a pre-deployed fault monitoring model for fault identification to obtain a fault monitoring result, the fault monitoring result including the probability of the device under test being in a normal state and in different fault modes; the fault monitoring model is obtained by performing cross-domain feature fusion training on a source domain data set and a target domain data set through a cloud-edge collaborative architecture, the cloud-edge collaborative architecture comprising a multi-source domain feature extraction network deployed on the cloud and a cross-domain feature fusion network deployed on the edge, the multi-source domain feature extraction network being used to extract fault features from the source domain data set and the target domain data set, the cross-domain feature fusion network being used to perform cross-domain fusion on the extracted multiple fault features, the source domain data set comprising operating signals marked with fault mode labels obtained by multiple devices operating under multiple working conditions, and the target domain data set comprising operating signals without fault mode labels obtained by the device under test operating under current working conditions.

[0006] In one embodiment of the present application, the method for generating the fault monitoring model includes: using the multi-source domain shared feature extraction network to extract shared fault features from the source domain data set, and extracting the exclusive fault features of the device under test from the target domain data set, wherein the shared fault features are used to migrate the fault knowledge in the source domain data set to the exclusive fault features of different devices under test; using the cross-domain feature fusion network to perform cross-domain fusion of the shared fault features and the exclusive fault features to obtain fused fault features; performing fault classification learning based on the fused fault features, and iteratively training the multi-source domain shared feature extraction network and the cross-domain feature fusion network with the goal of minimizing classification loss until a preset iteration stop condition is reached, thereby obtaining the fault monitoring model.

[0007] In one embodiment of the present application, the extracting fault features from the source domain data set and the target domain data set includes: performing a convolution operation on each source domain data in the source domain data according to the first convolution weight matrix and the bias weight matrix in the convolutional neural subnetwork to obtain multiple source domain fault features, and performing a convolution operation on each target domain data in the target domain data set to obtain multiple target domain fault features, the multi-source domain shared feature extraction network includes the convolutional neural subnetwork, and the first convolution weight matrix and the bias weight matrix are learning parameters; constructing a shared fault feature based on the multiple source domain fault features, and constructing the exclusive fault feature of the device under test based on the multiple target domain fault features to complete the extraction of fault features.

[0008] In one embodiment of the present application, the method for extracting the multiple source domain fault features and the multiple target domain fault features also includes: performing feature analysis on each source domain data in the source domain data set to determine a first convolution weight factor for each source domain data, and performing feature analysis on each target domain data in the target domain data set to determine a second convolution weight factor for each target domain data; using the first convolution weight factor to regulate the first convolution weight matrix, and using the second convolution weight factor to regulate the first convolution weight matrix; extracting the multiple source domain fault features and the multiple target domain fault features based on the regulated first convolution weight matrix and the bias weight matrix.

[0009] In one embodiment of the present application, the cross-domain fusion of multiple fault features extracted by the multi-source domain feature extraction network includes: obtaining the shared fault features extracted by the multi-source domain feature extraction network and the exclusive fault features of the device under test; constructing a fault feature map based on the shared fault features and the exclusive fault features; performing a convolution operation on the fault feature map according to the second convolution weight matrix in the graph convolution subnetwork to complete the cross-domain fusion of the multiple fault features, the cross-domain feature fusion network includes the graph convolution subnetwork, and the second convolution weight matrix is ​​a learning parameter.

[0010] In one embodiment of the present application, constructing a fault feature map based on the shared fault features and the exclusive fault features includes: obtaining a first convolution weight factor for each source domain data in the source domain data set and a second convolution weight factor for each target domain data in the target domain data set; calculating the similarity between each first convolution weight factor and each second convolution weight factor, respectively, to construct a weight factor similarity matrix; and according to the weight factor similarity matrix, integrating multiple source domain fault features in the shared fault features and multiple target domain fault features in the exclusive fault features into the fault feature map.

[0011] In one embodiment of the present application, the convolution operation is performed on the fault feature map to complete the cross-domain fusion of the multiple fault features, and also includes: determining a cross-domain fusion factor based on the weight factor similarity matrix; and performing a convolution operation on the fault feature map based on the second convolution weight matrix and the cross-domain fusion factor to complete the cross-domain fusion of the multiple fault features.

[0012] In the second aspect, the present application provides a fault monitoring system, which includes: a signal acquisition module for obtaining a target operating signal of a device under test; a fault monitoring module for inputting the target operating signal into a pre-deployed fault monitoring model for fault identification to obtain a fault monitoring result, wherein the fault monitoring result includes the probability of the device under test being in a normal state and in different fault modes; the fault monitoring model is obtained by performing cross-domain feature fusion training on a source domain data set and a target domain data set through a cloud-edge collaborative architecture, and the cloud-edge collaborative architecture includes a multi-source domain feature extraction network deployed on the cloud and a cross-domain feature fusion network deployed on the edge, the multi-source domain feature extraction network is used to extract fault features from the source domain data set and the target domain data set, and the cross-domain feature fusion network is used to perform cross-domain fusion on the extracted multiple fault features, the source domain data set includes operating signals marked with fault mode labels obtained by multiple devices operating under multiple working conditions, and the target domain data set includes operating signals without fault mode labels obtained by the device under test operating under the current working conditions.

[0013] In a third aspect, the present application provides an electronic device comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements the fault monitoring as described in the first aspect.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor of a computer, the computer is caused to perform the fault monitoring as described in the first aspect.

[0015] As described above, the fault monitoring system, electronic device, and storage medium provided by the embodiments of the present application have the following beneficial effects:

[0016] First, the target operation signal of the device under test is obtained, and then the target operation signal is input into the pre-deployed fault monitoring model for fault identification to obtain the fault monitoring result, wherein the fault monitoring result includes the probability of the device under test being in a normal state and different fault modes. The fault monitoring model is obtained by cross-domain feature fusion training of the source domain data set and the target domain data set through the cloud-edge collaborative architecture. The cloud-edge collaborative architecture includes a multi-source domain feature extraction network deployed in the cloud and a cross-domain feature fusion network deployed at the edge. The multi-source domain feature extraction network is used to extract fault features from the source domain data set and the target domain data set, and the cross-domain feature fusion network is used to extract multiple fault features. Cross-domain fusion: The source domain dataset includes operating signals marked with fault mode labels obtained by multiple devices operating under multiple working conditions. The target domain dataset includes operating signals without fault mode labels obtained by the device under test operating under the current working conditions. Cross-domain feature fusion training is achieved through the cloud-edge collaborative architecture, which comprehensively considers the fault characteristics of multiple source domains and the fault characteristics of the target domain, improves the monitoring performance and robustness of the fault monitoring model, and deploys the constructed fault monitoring model at the edge for localized fault monitoring, avoiding the delay problem caused by data transmission when implementing fault monitoring based on the cloud, thereby improving the accuracy and efficiency of device fault monitoring in industrial automation equipment.

[0017] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, serving to explain the principles of the present application. It is obvious that the drawings described below are merely some embodiments of the present application, and a person of ordinary skill in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0019] Figure 1 is a schematic diagram of an implementation environment of a fault monitoring system shown in an exemplary embodiment of the present application;

[0020] Figure 2 is a flow chart of a fault monitoring method shown in an exemplary embodiment of the present application;

[0021] Figure 3 1 is a structural diagram of a multi-source domain feature extraction network shown in an exemplary embodiment of the present application;

[0022] Figure 4 This is a structural diagram of a cross-domain feature fusion network shown in an exemplary embodiment of the present application;

[0023] Figure 5is a block diagram of a fault monitoring system shown in an exemplary embodiment of the present application;

[0024] Figure 6 This is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will describe the embodiments of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand the other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for the purpose of illustrating the present application and are not intended to limit the scope of protection of the present application.

[0026] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the illustrations only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the shape, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0027] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0028] With the rapid development of chip computing, deep learning, as a key branch of data-driven modeling, has demonstrated strong development potential and application prospects. It has been widely applied in fields such as object detection, natural language processing, and computer vision, and has performed excellently. Due to its outstanding potential in knowledge discovery and feature perception, deep learning has achieved considerable application and development in the intelligent operation and maintenance of industrial automation equipment.

[0029] With the development of intelligent manufacturing, automated equipment plays a vital role in manufacturing and is widely used in fields such as automotive manufacturing and food processing. For example, robotics plays a crucial role in intelligent manufacturing. Dual-arm collaborative sorting robots, for example, are used in these two sectors due to their high efficiency and flexibility. Various components within automated equipment, such as motors, reducers, sensors, controllers, and actuators, are critical to its stable operation. However, in actual operation, these components can experience various failures due to factors such as long-term loads, environmental changes, and mechanical wear, which in turn affect the equipment's operational stability and production efficiency. Therefore, accurately and effectively monitoring component failures within the equipment and enabling rapid repair are crucial for ensuring stable equipment operation and production efficiency. Consequently, deep learning technology has been widely used in the field of device fault monitoring. By deeply analyzing operational signals labeled with fault modes, it learns the hidden fault features within these signals and adaptively mines the essential characteristics that reflect the patterns of device failures. This eliminates the reliance on prior knowledge in fault identification and improves the reliability of fault monitoring to a certain extent.

[0030] However, the inventors of this application have found that in actual industrial scenarios, due to factors such as limited cost of collecting model training samples, information omissions during the collection process, low value density of collected data, and the complexity of operating conditions and working conditions in real industrial scenarios, the sample data sets collected and stored in the cloud are often insufficient to cover all operating conditions of the device, resulting in poor model accuracy. For example, in actual industrial scenarios, motors are subjected to complex and changeable environmental stresses due to changes in operating conditions such as load and speed, resulting in low fault monitoring accuracy. In addition, if device fault monitoring is performed in the cloud, the delay caused by data transmission seriously affects the real-time nature of fault monitoring. Therefore, relying solely on fault monitoring models trained in the cloud for fault monitoring still has shortcomings in monitoring accuracy and efficiency.

[0031] Therefore, see Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an implementation environment of a fault monitoring system according to an exemplary embodiment of the present application. Figure 1As shown, the implementation environment includes a cloud 110 and an edge 120, wherein a fault monitoring system is embedded in the edge 120 and is used to implement device fault monitoring in the edge 120. The edge 120 can be a local controller or industrial gateway corresponding to the industrial automation equipment. The fault monitoring system implements cross-domain feature fusion training through a cloud-edge collaborative architecture, comprehensively considering the fault characteristics of multiple source domains and the fault characteristics of the target domain, improving the monitoring performance and robustness of the fault monitoring model. In addition, the constructed fault monitoring model is deployed at the edge for localized fault monitoring, avoiding the delay caused by data transmission when implementing fault monitoring based on the cloud, thereby improving the accuracy and efficiency of device fault monitoring in industrial automation equipment.

[0032] See Figure 2 , Figure 2 This is a flowchart of a fault monitoring method shown in an exemplary embodiment of the present application. This method can be applied to Figure 1 It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments, and this embodiment does not limit the implementation environment to which the method is applicable.

[0033] like Figure 2 As shown, in an exemplary embodiment, the fault monitoring method is applied to the edge end, and at least includes steps S210 to S220, which are described in detail as follows:

[0034] Step S210: Acquire a target operating signal of the device under test.

[0035] Step S220: input the target operation signal into a pre-deployed fault monitoring model to perform fault identification and obtain a fault monitoring result;

[0036] Among them, the fault monitoring results include the probability of the device under test being in a normal state and in different fault modes. The fault monitoring model is obtained by cross-domain feature fusion training of the source domain data set and the target domain data set through the cloud-edge collaborative architecture. The cloud-edge collaborative architecture includes a multi-source domain feature extraction network deployed on the cloud and a cross-domain feature fusion network deployed on the edge. The multi-source domain feature extraction network is used to extract fault features from the source domain data set and the target domain data set. The cross-domain feature fusion network is used to perform cross-domain fusion of the extracted multiple fault features. The source domain data set includes operating signals marked with fault mode labels obtained by multiple devices operating under multiple working conditions. The target domain data set includes operating signals without fault mode labels obtained by the device under test operating under the current working conditions.

[0037] In this embodiment, the device under test can be a motor, a reducer, a sensor, a controller, an actuator, etc. in industrial automation equipment; the target operation signal and the operation signal can both be current signals, voltage signals, temperature signals, vibration signals, etc. The multiple devices and the device under test are of the same type, for example, all are motors, that is, the multiple devices are multiple motors, the device under test is the motor under test, and the multiple motors do not include the motor under test. In addition, the operation signal marked with a fault mode label obtained by the device under test included in the target domain dataset when it is running under the current working condition can be the operation signal marked with a fault mode label of the same device under test in multiple industrial automation equipment running under the same working condition in the same application scenario, that is, the multiple industrial automation equipment are controlled by the same local controller or industrial gateway.

[0038] In this embodiment, for training the fault monitoring model, the cloud is responsible for collecting source domain datasets for multi-source domain feature extraction, while the edge is responsible for collecting target domain datasets for cross-domain feature fusion. This not only leverages the cloud's powerful computing power to process multi-source data, but also effectively utilizes the fault characteristics of the target domain. Furthermore, real-time fault monitoring is achieved through model deployment on the edge. In the multi-source domain feature extraction network and the cross-domain feature fusion network, diverse fault characteristics are extracted from source and target domain datasets encompassing multiple operating conditions and then fused across domains, resolving the problem of insufficient sample data and enhancing the model's adaptability to different operating conditions.

[0039] In this way, cross-domain feature fusion training is achieved through the cloud-edge collaborative architecture, which comprehensively considers the fault characteristics of multiple source domains and the fault characteristics of the target domain, improves the monitoring performance and robustness of the fault monitoring model, and deploys the constructed fault monitoring model at the edge for localized fault monitoring, avoiding the delay problem caused by data transmission when implementing fault monitoring based on the cloud, thereby improving the accuracy and efficiency of device fault monitoring in industrial automation equipment.

[0040] In addition, since the fault monitoring model is deployed at the edge, there is no need to upload the target operating signal of the device under test to the cloud when performing fault monitoring of the device under test, thus ensuring data security.

[0041] For example, the source domain dataset S = [S1, S2, ..., S n ], including n operating signals with fault mode labels collected when multiple devices are running under various working conditions, each operating signal is represented by S i =(x i ,y i ), where x i Indicates the i-th running signal, y i It represents the fault mode label corresponding to the i-th operating signal, which is a probability distribution vector and can support the target domain dataset T = (x t) Fault classification of running signals in .

[0042] For example, the device failure types include looseness, overload and wear. If the failure mode corresponding to a certain operation signal is looseness, then y i is (0, 1, 0, 0), which respectively represent the probability of normal, loose, overload and wear.

[0043] In one embodiment, a method for generating a fault monitoring model includes: utilizing a multi-source domain shared feature extraction network to extract shared fault features from a source domain data set, and extracting specific fault features of a device under test from a target domain data set, wherein the shared fault features are used to transfer fault knowledge in the source domain data set to specific fault features of different devices under test; utilizing a cross-domain feature fusion network to perform cross-domain fusion of the shared fault features and the specific fault features to obtain a fused fault feature; performing fault classification learning based on the fused fault features, and iteratively training the multi-source domain shared feature extraction network and the cross-domain feature fusion network with the goal of minimizing classification loss, until a preset iteration stop condition is reached, thereby obtaining a fault monitoring model.

[0044] The calculation of the classification loss is based on a preset classification loss function, which can be a cross-entropy loss function. The embodiments of the present application do not limit the type of loss function. The preset iteration stopping conditions can include at least one of the following: the classification loss is less than a first preset threshold, the classification loss change rate is less than a second preset threshold, the number of iterations reaches a third preset threshold, and the classification accuracy reaches a fourth preset threshold. Iterative training of the multi-source domain shared feature extraction network and the cross-domain feature fusion network refers to iteratively optimizing the learning parameters in the multi-source domain shared feature extraction network and the cross-domain feature fusion network to improve the accuracy of the fault monitoring model.

[0045] In this embodiment, the multi-source domain shared feature extraction network extracts shared fault features from the source domain data set, captures the common fault features of multiple devices operating under various working conditions, and extracts the exclusive fault features of the device under test from the target domain data set, retaining the specificity of the device under test. In this way, the multi-source domain shared feature extraction network is combined with the cross-domain feature fusion network to perform cross-domain migration of fault knowledge, realize iterative updating of the fault monitoring model, and effectively improve the accuracy of the fault monitoring model for fault identification.

[0046] In a possible embodiment, fault classification learning is performed based on the fused fault features, that is, the fused fault features are passed through a fully connected layer and then through a Softmax activation function for classification learning.

[0047] In one embodiment, fault features are extracted from a source domain data set and a target domain data set, including: performing a convolution operation on each source domain data in the source domain data according to a first convolution weight matrix and a bias weight matrix in a convolutional neural subnetwork to obtain a plurality of source domain fault features, and performing a convolution operation on each target domain data in the target domain data set to obtain a plurality of target domain fault features, wherein the multi-source domain shared feature extraction network includes a convolutional neural subnetwork, and the first convolution weight matrix and the bias weight matrix are learning parameters; constructing a shared fault feature according to the plurality of source domain fault features, and constructing a dedicated fault feature of the device under test according to the plurality of target domain fault features, thereby completing the extraction of the fault features.

[0048] Among them, fault features include shared fault features and exclusive fault features. Shared fault features are composed of multiple source domain fault features, and exclusive fault features are composed of multiple target domain fault features.

[0049] In this embodiment, when extracting features from source domain and target domain data through a convolutional neural sub-network, the first convolution weight matrix is ​​shared. This can reduce the number of model parameters, lower the risk of overfitting, and improve the generalization and robustness of the model. By constructing shared and exclusive features, effective extraction and integration of fault features in different data sets are achieved, providing reliable feature input for subsequent cross-domain fusion and fault classification.

[0050] Exemplarily, the first convolution weight matrix and the bias weight matrix are used as learning parameters and optimized and adjusted through the back-propagation algorithm during the iterative training process; the convolutional neural sub-network can adopt a one-dimensional convolution structure, including a stack of multiple convolution layers and pooling layers, to gradually extract higher-level fault features; the construction of shared fault features can be achieved by splicing multiple source domain fault features, and the construction of exclusive fault features can be achieved by splicing multiple target domain fault features.

[0051] For example, the calculation formula of the source domain fault feature is:

[0052] y i =g(x i W+b i ) Formula (1)

[0053] Among them, i=1,2,…,n; y i represents the source domain fault feature extracted from the i-th source domain data; x i represents the i-th source domain data; W represents the first convolution weight matrix of all source domain data convolution; b i represents the bias weight matrix corresponding to the convolution of the i-th source domain data; g represents the relu activation function.

[0054] For example, the target domain fault characteristics are calculated as follows:

[0055] y t =g(x t W+b t ) Formula (2)

[0056] Where, t = 1, 2, ..., m; y t represents the target domain fault feature extracted from the t-th target domain data; x t represents the t-th target domain data; W represents the first convolution weight matrix of all target domain data convolution; b t represents the bias weight matrix corresponding to the convolution of the t-th target domain data; g represents the relu activation function.

[0057] In one embodiment, the method for extracting multiple source domain fault features and multiple target domain fault features also includes: performing feature analysis on each source domain data in the source domain data set to determine a first convolution weight factor for each source domain data, and performing feature analysis on each target domain data in the target domain data set to determine a second convolution weight factor for each target domain data; using the first convolution weight factor to regulate the first convolution weight matrix, and using the second convolution weight factor to regulate the first convolution weight matrix; extracting multiple source domain fault features and multiple target domain fault features based on the regulated first convolution weight matrix and the bias weight matrix.

[0058] In this embodiment, considering that the first convolution weight matrix is ​​more inclined to extract common features between multi-source domain data, although such features are extremely important for the extraction of fault features, it is impossible to adaptively adjust the weight parameters according to the respective source domain features. Therefore, feature analysis is performed on each source domain data in the source domain data set to determine the first convolution weight factor of each source domain data, and feature analysis is performed on each target domain data in the target domain data set to determine the second convolution weight factor of each target domain data. The convolution weight factor is used to regulate the first convolution weight matrix, so that during the convolution process, the weights of the first convolution weight matrix can be adjusted according to the input of the source domain data or the target domain data, thereby generating adaptive convolution, fusing the feature expression of the own source domain while extracting the common features of the multi-source domains, and fusing the feature expression of the specific target domain data while extracting the common features of the target domain, and more comprehensively considering the features of the data in different domains.

[0059] In this way, a dynamic weight control mechanism is added on the basis of the convolutional neural sub-network. By analyzing the feature differences between the source domain data and the target domain data, the corresponding convolution weight factors are generated, and these factors are used to dynamically adjust the first convolution weight matrix. This adaptive mechanism enables the feature extraction process to be optimized according to the characteristics of different domain data, effectively solving the limitations of fixed weight matrices in feature extraction of different domain data.

[0060] For example, another calculation formula for the source domain fault feature is:

[0061] y′ i =g(x si W*W si +b i ) Formula (3)

[0062] Among them, i=1,2,…,n; y′ i represents the source domain fault feature extracted from the i-th source domain data; x si represents the i-th source domain data; W represents the first convolution weight matrix of all source domain data convolution; b i Represents the bias weight matrix corresponding to the convolution of the i-th source domain data; W si It represents the first convolution weight factor corresponding to the convolution of the i-th source domain data; g represents the relu activation function.

[0063] For example, another calculation formula for the target domain fault feature is:

[0064] y′ t =g(x t W*W t +b t ) Formula (4)

[0065] Where, t = 1, 2, ..., m; y t represents the target domain fault feature extracted from the t-th target domain data; x t represents the t-th target domain data; W represents the first convolution weight matrix of all target domain data convolution; b t W represents the bias weight matrix corresponding to the convolution of the t-th target domain data; t It represents the second convolution weight factor corresponding to the convolution of the t-th target domain data; g represents the relu activation function.

[0066] In a possible embodiment, feature analysis is performed on each source domain data in the source domain data set to determine a first convolution weight factor for each source domain data, including: extracting features from each source domain data through an MLP (Multilayer Perceptron) and then calculating the first convolution weight factor through a Sigmoid activation function.

[0067] The value range of the first convolution weight factor is 0 to 1.

[0068] As a possible embodiment, the calculation formula of the first convolution weight factor is:

[0069] W si=Sigmoid(MLP(x si )) Formula (5)

[0070] Where i = 1, 2, ..., n; W si represents the first convolution weight factor corresponding to the convolution of the i-th source domain data; x si Represents the i-th source domain data; MLP represents multi-layer perceptron processing; Sigmoid represents the activation function.

[0071] In a possible embodiment, feature analysis is performed on each target domain data in the target domain data set to determine a second convolution weight factor for each target domain data, including: extracting features from each target domain data through a multi-layer perceptron MLP and then calculating the second convolution weight factor through a Sigmoid activation function.

[0072] The value range of the second convolution weight factor is 0 to 1.

[0073] As a possible embodiment, the calculation formula of the second convolution weight factor is:

[0074] W t =Sigmoid(MLP(x t )) Formula (6)

[0075] Where, t = 1, 2, ..., m; W t represents the second convolution weight factor corresponding to the convolution of the t-th target domain data; x t Represents the t-th target domain data; MLP represents multi-layer perceptron processing; Sigmoid represents the activation function.

[0076] See Figure 3 , Figure 3 This is a structural diagram of a multi-source domain feature extraction network shown in an exemplary embodiment of the present application. Figure 3 As shown in the figure, there are n different source domain data inputs, marked as "source domain data 1", "source domain data 2", ..., "source domain data n" respectively. Each source domain data will be feature analyzed through MLP to determine the first convolution weight factor. In addition, each source domain data will be convolved through Conv1d (convolution layer, that is, convolutional neural subnetwork). During the convolution process, the first convolution weight factor and the first convolution weight matrix are combined to obtain multiple source domain fault features.

[0077] In one embodiment, a plurality of fault features extracted by a multi-source domain feature extraction network are cross-domain fused, including: obtaining shared fault features extracted by the multi-source domain feature extraction network and exclusive fault features of the device under test; constructing a fault feature map based on the shared fault features and the exclusive fault features; performing a convolution operation on the fault feature map based on the second convolution weight matrix in the graph convolution subnetwork to complete the cross-domain fusion of the plurality of fault features, wherein the cross-domain feature fusion network includes a graph convolution subnetwork, and the second convolution weight matrix is ​​a learning parameter.

[0078] Among them, shared fault features represent the common features of multiple source domains; exclusive fault features represent the personalized features of the target domain; the fault feature graph is obtained by integrating shared fault features and exclusive fault features through the graph structure; the graph convolution subnetwork realizes feature space mapping through the second convolution weight matrix.

[0079] In this embodiment, heterogeneous features are first uniformly represented as graph structure data, and then the topological perception characteristics of graph convolution are used to realize cross-domain features, that is, the similarity fusion of target domain and multi-source domain features. The graph convolution subnetwork realizes the adaptive migration of source domain knowledge to the target domain through iterative updates of the second convolution weight matrix.

[0080] In this way, by modeling feature relationships with graph structures, the limitations of traditional methods in heterogeneous feature alignment and knowledge transfer are solved, the problem of nonlinear fusion of multi-source domain fault features and target domain-specific fault features is effectively solved, and the adaptive transfer of source domain fault knowledge to the target domain is realized.

[0081] In one embodiment, a fault feature map is constructed based on shared fault features and exclusive fault features, including: obtaining a first convolution weight factor of each source domain data in a source domain data set and a second convolution weight factor of each target domain data in a target domain data set; calculating the similarity between each first convolution weight factor and each second convolution weight factor, respectively, to construct a weight factor similarity matrix; and according to the weight factor similarity matrix, integrating multiple source domain fault features in the shared fault features and multiple target domain fault features in the exclusive fault features into a fault feature map.

[0082] The weight factor similarity matrix can be calculated using cosine similarity or Euclidean distance, and the value range of the matrix elements is [0,1].

[0083] In this embodiment, based on the convolution weight factors corresponding to each domain data in the multiple source domain data and the target domain data, the similarity of the weight factors is calculated to obtain a weight factor similarity matrix, denoted as A. Based on the weight factor similarity matrix, multiple source domain fault features and multiple target domain fault features are integrated into a fault feature graph, denoted as G(X, A), where X is a set of features of each source domain and features of the target domain, and A is a weight factor similarity matrix, which represents the connection relationship between nodes, where each domain is a node, including n source domain nodes and m target domain nodes.

[0084] In this way, by constructing a weight factor similarity matrix and then performing similarity fusion of the target domain and multi-source domain features based on the graph attention network, the source domain and target domain features can establish an interpretable mapping relationship in the topological space. By directly converting the similarity measurement results into the edge weights of the graph structure, the migration ability of the source domain fault features is retained, and the differential expression of the target domain features is ensured. It effectively copes with the complexity and diversity of cross-domain data, provides structured high-dimensional input for subsequent graph convolution operations, and effectively improves the accuracy and stability of cross-domain feature fusion.

[0085] In one embodiment, a convolution operation is performed on the fault feature map to complete the cross-domain fusion of multiple fault features, and the method further includes: determining a cross-domain fusion factor based on a weight factor similarity matrix; and performing a convolution operation on the fault feature map based on a second convolution weight matrix and the cross-domain fusion factor to complete the cross-domain fusion of multiple fault features.

[0086] Among them, the weight factor similarity matrix objectively reflects the migration potential between features, and the cross-domain fusion factor converts this potential into an operational fusion coefficient. After working together with the second convolution weight matrix, high-similarity features obtain greater fusion weights.

[0087] In this embodiment, by introducing a quantifiable cross-domain fusion factor, dynamic adjustment of feature fusion strength is achieved during the graph convolution process. This adaptive fusion mechanism based on feature correlation effectively solves the feature confusion problem caused by fixed weight convolution.

[0088] Exemplarily, determining the cross-domain fusion factor according to the weight factor similarity matrix includes: normalizing the weight factor similarity matrix to obtain the cross-domain fusion factor.

[0089] For example, the calculation formula for the fusion fault feature is:

[0090]

[0091] Where H represents the fusion fault feature; W z Represents the second convolution weight matrix corresponding to the graph convolution process; A represents the weight factor similarity matrix; Represents the cross-domain fusion factor after the weight factor similarity matrix is ​​symmetrically normalized; ReLU is the activation function.

[0092] See Figure 4 , Figure 4 This is a structural diagram of a cross-domain feature fusion network shown in an exemplary embodiment of the present application. Figure 4 As shown in the figure, first, in the multi-source domain feature extraction network, multiple source domain fault features and the corresponding first convolution weight factors and one target domain fault feature and the corresponding second convolution weight factor are obtained. In the cross-domain feature fusion network, according to the cosine similarity method, the similarity between the multiple first convolution weight factors and the second convolution weight factors is calculated, and a weight factor similarity matrix is ​​constructed. Then, according to the weight factor similarity matrix, a graph structure is constructed for the multiple source domain fault features and the one target domain fault feature to obtain a fault feature graph. Finally, the fault feature graph is input into the graph convolution subnetwork for feature fusion to obtain a fused fault feature.

[0093] In one possible embodiment, the fault monitoring result includes probability distribution vectors of the device under test being in a normal state and in different fault modes. After obtaining the fault monitoring result, the further step includes determining the state or fault mode corresponding to the maximum probability item as the final result, for example, the normal state or a specific fault mode.

[0094] In a possible embodiment, after obtaining the fault monitoring result, the method further includes: formulating a fault avoidance strategy according to the probability distribution vector in the fault monitoring result.

[0095] For example, if the probability value of a certain fault in the probability distribution vector reaches a threshold, a corresponding fault avoidance strategy is formulated to avoid the fault in advance.

[0096] The above-mentioned fault monitoring method first obtains the target operation signal of the device under test, and then inputs the target operation signal into the pre-deployed fault monitoring model for fault identification to obtain the fault monitoring result, wherein the fault monitoring result includes the probability of the device under test being in a normal state and different fault modes. The fault monitoring model is obtained by cross-domain feature fusion training of the source domain data set and the target domain data set through the cloud-edge collaborative architecture. The cloud-edge collaborative architecture includes a multi-source domain feature extraction network deployed in the cloud and a cross-domain feature fusion network deployed at the edge. The multi-source domain feature extraction network is used to extract fault features from the source domain data set and the target domain data set, and the cross-domain feature fusion network is used to extract multiple fault features. The fault features are cross-domain fused. The source domain dataset includes the operating signals marked with fault mode labels obtained by multiple devices under multiple working conditions. The target domain dataset includes the operating signals marked with fault mode labels obtained by the device under test under the current working conditions. Cross-domain feature fusion training is realized through the cloud-edge collaborative architecture, which comprehensively considers the fault features of multiple source domains and the fault features of the target domain, improves the monitoring performance and robustness of the fault monitoring model, and deploys the constructed fault monitoring model at the edge for localized fault monitoring, avoiding the delay problem caused by data transmission when implementing fault monitoring based on the cloud, thereby improving the accuracy and efficiency of device fault monitoring in industrial automation equipment.

[0097] See Figure 5 , Figure 5 This is a block diagram of a fault monitoring system shown in an exemplary embodiment of the present application. The system can be applied to Figure 1 The implementation environment shown is shown. It should be understood that the system can also be applied to other exemplary implementation environments, and this embodiment does not limit the implementation environment to which the system is applicable.

[0098] like Figure 5 As shown, in an exemplary embodiment, a fault monitoring system 500 is applied to an edge end and includes at least a signal acquisition module 510 and a fault monitoring module 520, which are described in detail as follows:

[0099] The signal acquisition module 510 is used to obtain the target operating signal of the device under test;

[0100] The fault monitoring module 520 is used to input the target operation signal into the pre-deployed fault monitoring model to perform fault identification and obtain the fault monitoring result;

[0101] Among them, the fault monitoring results include the probability of the device under test being in a normal state and in different fault modes. The fault monitoring model is obtained by cross-domain feature fusion training of the source domain data set and the target domain data set through the cloud-edge collaborative architecture. The cloud-edge collaborative architecture includes a multi-source domain feature extraction network deployed on the cloud and a cross-domain feature fusion network deployed on the edge. The multi-source domain feature extraction network is used to extract fault features from the source domain data set and the target domain data set. The cross-domain feature fusion network is used to perform cross-domain fusion of the extracted multiple fault features. The source domain data set includes operating signals marked with fault mode labels obtained by multiple devices operating under multiple working conditions. The target domain data set includes operating signals marked with fault mode labels obtained by the device under test operating under the current working conditions.

[0102] It should be noted that the fault monitoring system provided in the above embodiment and the fault monitoring method provided in the above embodiment belong to the same concept, wherein the contents of the operations performed by each module have been described in detail in the method embodiment and will not be repeated here.

[0103] See Figure 6 , Figure 6 This is a structural diagram of an electronic device provided by an embodiment of the present application. Figure 6 The following is a schematic diagram showing the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application. Figure 6 The computer system 600 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0104] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 into the random access memory (RAM) 603, such as executing the method in the above embodiment. Various programs and data required for system operation are also stored in the RAM 603. The CPU 601, ROM 602 and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0105] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, and the like; an output section 607 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. Removable media 611, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 610 as needed, so that computer programs read therefrom can be installed into the storage section 608 as needed.

[0106] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from a removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the various functions defined in the system of the present application are executed.

[0107] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer processor, the computer executes the fault monitoring method described above. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist independently and not be incorporated into the electronic device.

[0108] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable computer program. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0109] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, and the above-mentioned module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. In some cases, the names of these units do not constitute limitations on the units themselves.

[0111] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, any equivalent modifications or alterations accomplished by a person of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A fault monitoring method, characterized in that: Applied to the edge, the method includes: Obtain target operating signals of the device under test; Inputting the target operation signal into a pre-deployed fault monitoring model to perform fault identification and obtain a fault monitoring result; Among them, the fault monitoring results include the probability of the device under test being in a normal state and different fault modes. The fault monitoring model is obtained by performing cross-domain feature fusion training on the source domain data set and the target domain data set through a cloud-edge collaborative architecture. The cloud-edge collaborative architecture includes a multi-source domain feature extraction network deployed on the cloud and a cross-domain feature fusion network deployed on the edge. The multi-source domain feature extraction network is used to extract fault features from the source domain data set and the target domain data set. The cross-domain feature fusion network is used to perform cross-domain fusion on the extracted multiple fault features. The source domain data set includes operating signals marked with fault mode labels obtained by multiple devices operating under multiple working conditions, and the target domain data set includes operating signals without fault mode labels obtained by the device under test operating under current working conditions.

2. The fault monitoring method according to claim 1, characterized in that: The generation method of the fault monitoring model includes: Utilizing the multi-source domain shared feature extraction network, extracting shared fault features from the source domain dataset and extracting the specific fault features of the device under test from the target domain dataset, wherein the shared fault features are used to transfer fault knowledge in the source domain dataset to the specific fault features of different devices under test; Using the cross-domain feature fusion network, cross-domain fusion is performed on the shared fault feature and the exclusive fault feature to obtain a fused fault feature; Fault classification learning is performed based on the fused fault features, and with the goal of minimizing classification loss, the multi-source domain shared feature extraction network and the cross-domain feature fusion network are iteratively trained until a preset iteration stop condition is reached, thereby obtaining the fault monitoring model.

3. The fault monitoring method according to claim 1, characterized in that: The extracting fault features from the source domain dataset and the target domain dataset includes: According to the first convolution weight matrix and the bias weight matrix in the convolutional neural sub-network, a convolution operation is performed on each source domain data in the source domain data to obtain a plurality of source domain fault features, and a convolution operation is performed on each target domain data in the target domain data set to obtain a plurality of target domain fault features, wherein the multi-source domain shared feature extraction network includes the convolutional neural sub-network, and the first convolution weight matrix and the bias weight matrix are learning parameters; A shared fault feature is constructed based on the multiple source domain fault features, and a dedicated fault feature of the device under test is constructed based on the multiple target domain fault features, thereby completing the extraction of fault features.

4. The fault monitoring method according to claim 3, characterized in that: The method for extracting the multiple source domain fault features and the multiple target domain fault features further includes: Performing feature analysis on each source domain data in the source domain data set to determine a first convolution weight factor for each source domain data, and performing feature analysis on each target domain data in the target domain data set to determine a second convolution weight factor for each target domain data; Regulating the first convolution weight matrix using the first convolution weight factor, and regulating the first convolution weight matrix using the second convolution weight factor; The plurality of source domain fault features and the plurality of target domain fault features are extracted according to the regulated first convolution weight matrix and the bias weight matrix.

5. The fault monitoring method according to any one of claims 1 to 4, characterized in that: The cross-domain fusion of the multiple fault features extracted by the multi-source domain feature extraction network includes: Obtaining shared fault features extracted by the multi-source domain feature extraction network and exclusive fault features of the device under test; Constructing a fault feature map according to the shared fault feature and the exclusive fault feature; According to the second convolution weight matrix in the graph convolution subnetwork, a convolution operation is performed on the fault feature graph to complete the cross-domain fusion of the multiple fault features. The cross-domain feature fusion network includes the graph convolution subnetwork, and the second convolution weight matrix is ​​a learning parameter.

6. The fault monitoring method according to claim 5, characterized in that: The constructing a fault feature map according to the shared fault feature and the exclusive fault feature includes: Obtaining a first convolution weight factor for each source domain data in the source domain data set and a second convolution weight factor for each target domain data in the target domain data set; Calculating the similarity between each of the first convolution weight factors and each of the second convolution weight factors, and constructing a weight factor similarity matrix; According to the weight factor similarity matrix, a plurality of source domain fault features in the shared fault features and a plurality of target domain fault features in the dedicated fault features are integrated into the fault feature graph.

7. The fault monitoring method according to claim 6, characterized in that: The performing a convolution operation on the fault feature map to complete the cross-domain fusion of the multiple fault features further includes: Determining a cross-domain fusion factor according to the weight factor similarity matrix; A convolution operation is performed on the fault feature map according to the second convolution weight matrix and the cross-domain fusion factor to complete the cross-domain fusion of the multiple fault features.

8. A fault monitoring system, characterized in that: Applied to the edge, the system includes: A signal acquisition module is used to obtain the target operating signal of the device under test; A fault monitoring module is used to input the target operation signal into a pre-deployed fault monitoring model to perform fault identification and obtain a fault monitoring result; Among them, the fault monitoring results include the probability of the device under test being in a normal state and different fault modes. The fault monitoring model is obtained by performing cross-domain feature fusion training on the source domain data set and the target domain data set through a cloud-edge collaborative architecture. The cloud-edge collaborative architecture includes a multi-source domain feature extraction network deployed on the cloud and a cross-domain feature fusion network deployed on the edge. The multi-source domain feature extraction network is used to extract fault features from the source domain data set and the target domain data set. The cross-domain feature fusion network is used to perform cross-domain fusion on the extracted multiple fault features. The source domain data set includes operating signals marked with fault mode labels obtained by multiple devices operating under multiple working conditions, and the target domain data set includes operating signals without fault mode labels obtained by the device under test operating under current working conditions.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the fault monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the fault monitoring method according to any one of claims 1 to 7.

Images