Coupler intelligent diagnosis method and system based on multi-source data fusion

Abstract

The invention discloses an intelligent coupler diagnosis method and system based on multi-source data fusion. The method comprises the steps that multi-source data of a coupler are collected in real time, feature extraction is conducted on the collected data, TimeMIL model parameters are optimized through a collaborative group optimization algorithm SSOA, and fault prediction is conducted on time series data through an optimized model. An OmniGlue-two-channel convolutional neural network PCNN is adopted to carry out collaborative feature extraction on the image data, and fault diagnosis is carried out through a structural attention mechanism StrauctSA; carrying out weighted fusion on the fault prediction result and the fault monitoring result through Bayesian model averaging, and comprehensively outputting a fused fault diagnosis result; a block chain technology is adopted, and the preprocessed time sequence data and image data and other related metadata are stored on each node of a block chain in an encrypted form. The coupling fault diagnosis precision can be effectively improved, the system robustness is improved, and the working efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent fault diagnosis of couplings, and particularly relates to an intelligent diagnosis method and system for couplings based on multi-source data fusion. Background Art

[0002] The coupling is a key component in the mechanical transmission system and is widely used in multiple fields such as automobiles and industries. However, with the continuous in-depth development of mechanization, the importance of the coupling, as an important component for connecting two shafts or a shaft and a rotating part, has become increasingly prominent. Especially in high-precision and high-speed mechanical systems, the performance and reliability issues of the coupling have become the key factors restricting the further improvement of the performance of the entire mechanical system. Due to the variable working environment, the load and stress conditions borne by the coupling are also complex and variable. For example: in an environment with high temperature, high humidity, and strong corrosion, the coupling is prone to problems such as material aging and corrosion; under heavy load and high-speed operation conditions, the coupling is prone to failures such as wear and fracture; in the working conditions of frequent start and stop, the elastic elements of the coupling are prone to fatigue failure, affecting the transmission efficiency and stability.

[0003] Traditional coupling fault diagnosis methods mainly rely on the experience judgment of operators and instrument detection. There are differences in the structure, materials, and performance of couplings of different brands. Therefore, manual diagnosis often can only conduct one-by-one inspections for specific models and fault phenomena. This method not only has low diagnosis efficiency but is also easily affected by human factors, resulting in inaccurate diagnosis results. In addition, due to the technical barriers and data non-sharing between manufacturers, many identical fault problems need to be repeatedly inspected on products of different manufacturers, wasting a large amount of time and resources. Secondly, existing coupling fault diagnosis methods mostly adopt a single mathematical model or an experience-based judgment method, lacking in-depth analysis and understanding of different fault types and fault mechanisms. These methods can often only handle some simple fault situations and are difficult to give accurate diagnosis results for complex and variable fault phenomena.

[0004] Therefore, a coupling fault diagnosis method that can handle complex working conditions and improve the efficiency and accuracy of fault diagnosis is needed. This method should be able to comprehensively consider various fault factors, utilize advanced sensor technology and data analysis methods, and achieve rapid and accurate diagnosis of coupling faults, providing a strong guarantee for the stable operation of the mechanical system. Summary of the Invention

[0005] Object of the Invention: The present invention provides an intelligent diagnosis method and system for couplings based on multi-source data fusion, realizing intelligent processing of coupling data and accurate identification of fault types.

[0006] Technical Solution: An intelligent diagnosis method for couplings based on multi-source data fusion according to the present invention includes the following steps:

[0007] (1) Collect the operation sequence data of multi-source couplings in real time in time series, and obtain the operation image data of couplings using flaw detection system, high-speed motion camera and scanner;

[0008] (2) Construct a fault prediction model for time series data; the fault prediction model uses the collaborative group optimization algorithm SSOA to optimize the multi-instance learning model TimeMIL, and the optimized TimeMIL is used as the model for time series data fault prediction; first, the time series data is feature extracted and the extracted features are spliced; then, the spliced ​​data is supplemented and normalized; then, the spliced ​​feature vectors are combined into a high-dimensional feature matrix, and principal component analysis PCA is performed; finally, the optimized TimeMIL model is used for fault prediction;

[0009] (3) Based on the image data, a fault monitoring model OmniGlue-PCNN-StructSA is constructed to perform fault diagnosis. The SuperPoint encoder and DINOv2 encoder of OmniGlue are used to extract global features of the acquired images. Then, a dual-channel convolutional neural network PCNN is used to optimize the local features of the image. Then, the structural attention mechanism StructSA is combined to monitor the coupling faults.

[0010] (4) Using the distributed ledger technology of blockchain, a blockchain network dedicated to storing coupling fault diagnosis-related data and information is established, and the pre-processed time series data and image data and other related metadata are stored in encrypted form on each node of the blockchain;

[0011] (5) The Bayesian model average (BMA) is used to weight the prediction results of the SSOA-TimeMIL model and the monitoring results of the OmniGlue-PCNN-StructSA model, and the fusion fault diagnosis results are comprehensively output;

[0012] (6) The diagnostic results and related information recorded in the blockchain are retrieved through the background supervision system and fed back to the relevant equipment maintenance personnel and management personnel; at the same time, the consensus mechanism of the blockchain is used to verify and confirm the diagnostic results by multiple nodes, making the data tamper-proof and improving the credibility of the diagnostic results.

[0013] Furthermore, the operating timing data of the multi-source coupling in step (1) includes operating data of vibration, temperature, torque and rotation speed of different brands and models.

[0014] Furthermore, the features extracted in step (2) include the root mean square and kurtosis of the vibration signal data, the rate of change and mean of the temperature signal data, and the fluctuation range of the torque signal data.

[0015] Further, the process of optimizing the multi-instance learning model TimeMIL using the cooperative swarm optimization algorithm SSOA in step (2) is as follows:

[0016] The hyperparameters of the TimeMIL model include the window covering ratio, the number of warm-up periods, the batch size, the selection of wavelet basis, and the scaling and translation parameters;

[0017] Initialize the hyperparameter group, and the formula is:

[0018] X = rand(N, Dim).*(U B -L B ) + L B

[0019]

[0020] where X is a matrix of N×Dim dimensions, N is the number of hyperparameter groups in the search space, Dim is the number of hyperparameters of the TimeMIL model, U B and L B are the upper and lower bounds of the hyperparameter group. The formula for updating the candidate hyperparameter group is as follows:

[0021] Xnew(i,j) = X(i,j) + γ(i,j)

[0022] where Xnew(i,j) is the optimized solution of the i-th candidate hyperparameter solution, X(i,j) is the current solution of the i-th candidate hyperparameter solution, and γ(i,j) is the position j of the i-th candidate hyperparameter solution; at the same time, combined with the velocity update formula, a dynamic attraction equation is introduced to guide the hyperparameter group to optimize in a more favorable direction. The formula is as follows:

[0023] γ new(i,j) = IWV + PBC + GBC + DAC + ANIC + MDC

[0024] IWV = w(t)*γ(i,j)

[0025] w(t + 1) = w(t)*(1 - exp(-k*t))

[0026] PBC = r1*(eps*rand(pbest) - X i )

[0027] GBC = r2*gbest t -X i

[0028]

[0029] ANIC = r4 * rand(bestf) - bestf i

[0030]

[0031] Wherein, IWV is the inertia weight value, w(t) is the inertia weight adaptive parameter for the t-th iteration, which is used to dynamically control the balance between different directions during optimization, t is the number of iterations, and k is a constant determining the inertia weight rate; PBC is the personal best coefficient, r1 is a randomly generated value, eps is the minute value, rand(pbest) is a group randomly selected from the existing candidate hyperparameter groups, and X i is the i-th hyperparameter group; GBC is the global best coefficient, r2 is a random integer, and gbest t is the best global hyperparameter group at the t-th iteration; DAC is the dynamic attraction coefficient, r3 is a randomly generated value, and attract i is the direction with the strongest local guiding optimization near the i-th hyperparameter group, and c1 is the additional acceleration coefficient of the dynamic guiding term; ANIC is the adaptive neighbor interaction coefficient, r4 is a randomly generated value, rand(bestf) is the random fitness value in the current fitness solution, and bestf i is the fitness value of the i-th hyperparameter group; DMC is the diversity preservation coefficient, r5 is a randomly generated value, and diversity i is the maximum diversity value of the i-th hyperparameter group in the hyperparameter group, and c2 is the additional acceleration coefficient of the diversity term.

[0032] Furthermore, the implementation process of step (3) is as follows:

[0033] (3.1) Take the collected image data as the input of the model, use the SuperPoint encoder and the DINOv2 encoder to jointly extract image features. Use the DINOv2 encoder for rough signal guidance. The SuperPoint encoder extracts the key points in the image and interpolates the feature map to obtain the DINOv2 descriptor of each key point, and each key point is consistent and associated with its local feature description; Encode the key points using position embedding, and use the multi-layer perceptron MLP layer to refine the key points;

[0034] (3.2) Guided by the DINOv2 encoder, four key-point association graphs are constructed, including two inter-graphs and two intra-graphs. The two inter-graphs are used to construct the key-point similarity between two images under the guidance of the DINOv2 encoder. The key-points with the maximum similarity between images are densely paired and connected to establish relationships. The two intra-graphs are used to construct the connectivity between key-points within the same image. The key-points within the same image propagate bidirectionally, and each key-point is closely connected to other key-points.

[0035] (3.3) Information propagation is carried out based on the key-point guidance of the DINov2 encoder. There are two attention layers in total. The first layer updates the key-point positions based on the intra-graph, and the second layer updates the key-points based on the inter-graph. The two layers perform cross-attention. Refined key-point guidance is added to separate the key-point position features and local descriptions. The specific formula is as follows:

[0036]

[0037]

[0038] In the formula, A i is the i-th key-point in image A, is the local description of the i-th key-point in image A, ← represents the update operation, is the attention update value of the local description of the i-th key-point in image A, s is the feature similarity between k key-points, including all key-points, is the position feature of the i-th key-point in image A, q i A 、k s 、v s are the query, key, and value of the attention, W q 、W k 、W v are the weights of the query, key, and value of the attention, b q 、b k 、b v are the biases of the query, key, and value of the attention;

[0039] (3.4) Build a two-channel convolutional neural network PCNN, including convolutional layers, pooling layers, and fully connected layers. The data obtained after feature extraction by OmniGlue is used as the input. Convolution operations are performed through the convolutional layers to extract local features from the input image data. The formula is as follows:

[0040]

[0041] In the formula, is the feature map, is the weight matrix, is the bias term, and f(·) is the activation function, which is generally set to ReLU;

[0042] Then, the spatial dimension of the feature signal is reduced through the pooling layer, and the number of parameters is reduced, including max pooling and average pooling. The formulas are as follows:

[0043]

[0044] In the formula, is the weight of the feature map, down(·) is the pooling function, and M l is the pooling window M l ×M l ;

[0045] (3.5) StructSA is used to fuse the features extracted by the OmniGlue and PCNN modules before and after, improve the feature extraction ability of the model, convert ordinary query-key attention to structure-aware attention, and deploy convolution on the query-key correlation. The formula is as follows:

[0046]

[0047] In the formula, is the query key K ∈ R B×C and value V ∈ R B×C 's projection matrix, Q i K T is used to calculate the correlation map, is the calculated correlation value for each map, are R convolutional kernels of size P, and each element h i of H i,J is calculated as:

[0048]

[0049] In the formula, is the local front and back feature centered on the Jth data before and after;

[0050] After obtaining the structure-aware attention, to achieve front and back aggregation and output the monitoring results, it is combined with the weight. The formula is as follows:

[0051]

[0052] In the formula, is the vector of the fractional linear combination expanded by R to the final attention weight, is the weight of the structure attention; at this time, using the front and back aggregation method, a spatial kernel is generated for each position J. The formula is as follows:

[0053]

[0054] In the formula, reprojects and reduces the dimension of the feature map generated by multi-dimensional convolution, E J is the spatial projection of each feature point after PCNN feature extraction, that is, the spatial weight around each feature point;

[0055] (3.6) Adopts a fully connected layer and a Softmax function for fault classification, and outputs the fault monitoring result.

[0056] Furthermore, the blockchain network in step (4) archives and calls the collected data, and through the hash algorithm, decentralizes the management of the data.

[0057] Furthermore, the other relevant metadata in step (4) includes the collection time, collection location, device number, and diagnosis result.

[0058] Furthermore, the implementation process of step (5) is as follows:

[0059] Construct the prior probability of the fault probability; calculate the conditional probability of the fault occurrence in the actual situation according to the actual situation and the device operation environment; obtain the posterior probability through the prior probability and the likelihood function. According to the comprehensive probability criterion, the weighted posterior mean and the likelihood function formula are as follows:

[0060] E(θ|G) = ε 1 M 1 + ε 2 M 2

[0061] L(λ) = log[ε 1 p(θ|M 1 ) + ε 2 p(θ|M 2 )]

[0062] In the formula, ε 1 and ε 2 are the weights of the SSOA-TimeMIL model and the OmniGlue-PCNN-StructSA model, M 1 and M 2 are the predictions of the SSOA-TimeMIL model and the monitoring results of the OmniGlue-PCNN-StructSA model, p(θ|M 1 ) and p(θ|M 2 ) are the posterior distributions of the prediction results of the SSOA-TimeMIL model and the monitoring results of the OmniGlue-PCNN-StructSA model, θ is the fused fault diagnosis result; ε 1 and ε 2It is obtained by using the expectation maximization log-likelihood function, and then based on the posterior mean, it is used as the final fusion fault diagnosis result, and the result is recorded in the blockchain node together with the original data.

[0063] The present invention provides a coupling intelligent diagnosis system based on multi-source data fusion, comprising:

[0064] The data acquisition module collects the operation sequence data of multi-source couplings in real time in time series; the coupling operation image data is obtained by using the flaw detection system, high-speed motion camera and scanner;

[0065] The fault prediction module uses the collaborative group optimization algorithm SSOA to optimize the multi-instance learning model TimeMIL, and uses the optimized TimeMIL as the model for time series data fault prediction. First, the time series data is feature extracted and the extracted features are spliced. Then, the spliced ​​data is supplemented and normalized. Then, the spliced ​​feature vectors are combined into a high-dimensional feature matrix and the principal component analysis PCA is performed. Finally, the optimized TimeMIL model is used for fault prediction.

[0066] The fault monitoring module uses OmniGlue's SuperPoint encoder and DINOv2 encoder to extract global features of the captured image. Then, a dual-channel convolutional neural network (PCNN) is used to optimize the local features of the image. Then, the coupling is monitored for faults in combination with the structural attention mechanism StructSA.

[0067] The blockchain network stores coupling fault diagnosis-related data and information, and stores the pre-processed time series data and image data and other related metadata in encrypted form on each node of the blockchain;

[0068] The fault output module uses the Bayesian model average (BMA) to weightedly fuse the prediction results of the SSOA-TimeMIL model and the monitoring results of the OmniGlue-PCNN-StructSA model, and comprehensively outputs the fused fault diagnosis results.

[0069] The backend supervision system retrieves the diagnostic results and related information recorded in the blockchain and feeds them back to the relevant equipment maintenance and management personnel; at the same time, the consensus mechanism of the blockchain is used to verify and confirm the diagnostic results by multiple nodes, making the data tamper-proof and improving the credibility of the diagnostic results.

[0070] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects:

[0071] By collecting multi-source coupling data, the present invention can effectively utilize the operation data of couplings of different brands and models, improve the accuracy and robustness of fault diagnosis; utilize multi-source data fusion technology to combine various types of couplings to construct a more universal fault diagnosis model, and improve the accuracy of the fault diagnosis system;

[0072] By constructing a dual-thread system for fault prediction of time-series data and fault diagnosis of image data, the present invention can comprehensively cover multiple dimensions of coupling fault diagnosis, thereby providing more comprehensive and accurate fault diagnosis results;

[0073] By using Bayesian fusion averaging to weight and fuse the two results, the present invention can effectively reduce the bias and uncertainty of a single model, thereby improving the accuracy of fault diagnosis; it can fuse the results of different types of models, make full use of the advantages of each model, automatically learn the fault types through training data, improve the diagnostic efficiency and overall diagnostic performance, and respond to real-time data to update the results in a timely manner. Brief Description of the Drawings

[0074] Figure 1 It is a flowchart of an intelligent diagnosis method for couplings based on multi-source data fusion. Detailed Embodiment

[0075] The present invention will be further described in detail below with reference to the drawings.

[0076] As Figure 1 shown, the present invention provides an intelligent diagnosis method for couplings based on multi-source data fusion. Through the dual detection of time-series data and image data, the real-time diagnosis and maintenance of the health status of the couplings can be realized, and the operation efficiency of the couplings can be effectively improved. The specific steps are as follows:

[0077] Step 1: Real-time collect the operation data of multi-source couplings in time series, including but not limited to vibration, temperature, torque, and speed. At the same time, use a flaw detection system, a high-speed motion camera, and a scanner to obtain the operation images of the couplings for in-depth diagnostic analysis.

[0078] The multi-source coupling operation data includes the operation data of couplings of different brands and models. At the same time, the operation data of vibration, temperature, torque, and speed and the image data of the couplings are collected. By fusing the operation data of different forms of multi-brand and multi-model couplings, the system can achieve comprehensive compatibility and accurate diagnosis of various types of couplings.

[0079] Step 2: For time series data, construct a fault prediction model, collect the operating data of the coupling in real time, and perform fault prediction in real time. The multi-instance learning model TimeMIL is optimized by the cooperative swarm optimization algorithm SSOA, and the optimized TimeMIL is used as the model for time series data fault prediction. First, extract features from the time series data and splice the extracted features; then, perform compensation and normalization on the spliced data; then, combine the spliced feature vectors into a high-dimensional feature matrix and perform principal component analysis PCA, which can reduce dimensions, redundant information, and computational complexity; finally, use the optimized TimeMIL model for fault prediction.

[0080] The extracted features mainly include the root mean square and kurtosis of the vibration signal data, the change rate and mean value of the temperature signal data, and the fluctuation range of the torque signal data. Then, the extracted feature data are spliced, compensated, and normalized.

[0081] Construct a fault prediction model, and use SSOA to optimize the hyperparameter group of the TimeMIL model. The hyperparameters of the TimeMIL model mainly include the window covering ratio, the number of warm-up periods, the batch size, the selection of wavelet bases, and the scaling and translation parameters. The specific optimization steps are as follows:

[0082] Initialize the hyperparameter group, and the formula is:

[0083] X = rand(N, Dim).*(U B - L B ) + L B

[0084]

[0085] In the formula, X is a matrix of N×Dim dimensions, N is the number of hyperparameter groups in the search space, Dim is the number of hyperparameters of the TimeMIL model, U B and L B are the upper and lower bounds of the hyperparameter group. The formula for updating the candidate hyperparameter group is as follows:

[0086] Xnew(i,j) = X(i,j) + γ(i,j)

[0087] In the formula, Xnew(i,j) is the optimized solution of the i-th candidate hyperparameter solution, X(i,j) is the current solution of the i-th candidate hyperparameter solution, and γ(i,j) is the position j of the i-th candidate hyperparameter solution; at the same time, combined with the velocity update formula, a dynamic attraction equation is introduced to guide the hyperparameter group to search for a more favorable direction. The formula is as follows:

[0088] γ new(i,j) = IWV + PBC + GBC + DAC + ANIC + MDC

[0089] IWV = w(t) * γ(i, j)

[0090] w(t + 1) = w(t) * (1 - exp(-k * t))

[0091] PBC = r1 * (eps * rand(pbest) - X i )

[0092] GBC = r2 * gbest t -X i

[0093]

[0094] ANIC = r4 * rand(bestf) - bestf i

[0095]

[0096] Wherein, IWV is the inertia weight value, w(t) is the inertia weight adaptive parameter of the t-th iteration, which is used to dynamically control the balance between different directions during optimization, t is the number of iterations, and k is a constant determining the inertia weight rate; PBC is the personal best coefficient, r1 is a randomly generated value, eps is the minute value, rand(pbest) is a group randomly selected from the existing candidate hyperparameter groups, and X i is the i-th hyperparameter group; GBC is the global best coefficient, r2 is a random integer, and gbest t is the best global hyperparameter group at the t-th iteration; DAC is the dynamic attraction coefficient, r3 is a randomly generated value, and attract i is the direction with the strongest local guiding optimization near the i-th hyperparameter group, and c1 is the additional acceleration coefficient of the dynamic guiding term; ANIC is the adaptive neighbor interaction coefficient, r4 is a randomly generated value, rand(bestf) is a random fitness value in the current fitness solution, and bestf i is the fitness value of the i-th hyperparameter group; DMC is the diversity preservation coefficient, r5 is a randomly generated value, and diversity i is the maximum diversity value of the i-th hyperparameter group in the hyperparameter group, and c2 is the additional acceleration coefficient of the diversity term.

[0097] Step 3: For the image data, construct a fault monitoring model, collect the image data during the operation of the coupling, and perform fault diagnosis. The fault monitoring model is constructed by OmniGlue-PCNN-StructSA. First, use the SuperPoint encoder and DINOv2 encoder of OmniGlue to extract global features from the collected images and construct four key point association graphs; then, use the dual-channel convolutional neural network PCNN to optimize the local features of the images; then, preprocess the pictures, including image grayscaling, image enhancement, and image filtering; finally, combine the structural attention mechanism StructSA to perform fault diagnosis on the coupling.

[0098] The OmniGlue-PCNN-StructSA model performs fault diagnosis on the image data of the coupling. OmniGlue is used for global feature extraction, PCNN is used for local feature extraction of the data after global feature extraction, and then StructSA is used for fault monitoring. The specific steps are as follows:

[0099] (3.1) Take the collected image data as the input of the model, use the SuperPoint encoder and DINOv2 encoder to jointly extract image features. The DINOv2 encoder is used for rough signal guidance, while the SuperPoint encoder is used to extract the key points in the image and interpolate the feature map to obtain the DINOv2 descriptor of each key point, and each key point is consistent and associated with its local feature description; at this time, the key points are encoded using position embedding, and the multi-layer perceptron MLP layer is used to refine the key points.

[0100] (3.2) Under the guidance of the DINOv2 encoder, construct key point association graphs, including two inter-graphs and two intra-graphs. The two inter-graphs are used to construct the key point similarity between two images under the guidance of the DINOv2 encoder, densely pair-connect the key points with the maximum similarity between images to establish relationships; the two intra-graphs are used to construct the connectivity between key points within the same image, and the key points within the same image propagate bidirectionally, and each key point is closely connected to other key points.

[0101] (3.3) Perform information propagation based on the key point guidance of the DINov2 encoder. There are two attention layers in total. The first is to update the key point positions based on the intra-graph, and the second is to update the key points based on the inter-graph. The two layers perform cross-attention; since the model may over-rely on the learned position-related prior knowledge during training, refined key point guidance is added to separate the key point position features and local descriptions. Taking key point A i as an example, the specific formula is as follows:

[0102]

[0103] Wherein, A i is the i-th key point in image A, is the local description of the i-th key point in image A, ← represents the update operation, is the attention update value of the local description of the i-th key point in image A, s is the feature similarity between k key points, including all key points, is the position feature of the i-th key point in image A, k s , v s are the query, key, and value of the attention, W q , W k , W v are the weights of the query, key, and value of the attention, b q , b k , b v are the biases of the query, key, and value of the attention.

[0104] (3.4) Build a two-channel convolutional neural network PCNN, which consists of a convolutional layer, a pooling layer, and a fully connected layer. Take the data after feature extraction by OmniGlue as the input, and perform convolution operations through the convolutional layer to extract local features of the input image data. The formula is as follows:

[0105]

[0106] Wherein, is the feature map, is the weight matrix, is the bias term, f(·) is the activation function, which is generally set to ReLU.

[0107] Then, reduce the spatial dimension of the feature signal through the pooling layer and reduce the number of parameters, including max pooling and average pooling. The formula is as follows:

[0108]

[0109] Wherein, is the weight of the feature map, down(·) is the pooling function, M l is the pooling window M l ×M l .

[0110] (3.5) Through StructSA, fuse the features extracted by OmniGlue and the PCNN module before and after to improve the feature extraction ability of the model, convert ordinary query-key attention to structure-aware attention, and deploy convolution on the query-key correlation. The formula is as follows:

[0111]

[0112] In the formula, is the query key \(K\in\mathbb{R}\) B×C and value \(V\in\mathbb{R}\) B×C projection matrix, \(Q\) i \(K\) T is the calculated correlation graph, is the calculated correlation value for each graph, are \(R\) convolutional kernels of size \(P\), \(H\) i each element \(h\) of i,J is calculated as:

[0113]

[0114] In the formula, are the local front and back features centered on the \(J\)-th data before and after.

[0115] After obtaining the structure-aware attention, to achieve front and back aggregation and output the diagnostic result, combine it with the weights, the formula is as follows:

[0116]

[0117] In the formula, is the vector of the fractional linear combination expanded by \(R\) to the final attention weight, is the weight of the structure attention; at this time, using the front and back aggregation method, a spatial kernel is generated for each position \(J\), and the formula is as follows:

[0118]

[0119] In the formula, is the re-projection and dimensionality reduction of the feature map generated by multi-dimensional convolution, \(E\) J is the spatial projection of each feature point after PCNN feature extraction, that is, the spatial weight around each feature point.

[0120] (3.6) Finally, the model uses a fully connected layer and the Softmax function for fault classification and outputs the fault monitoring result.

[0121] Step 4: Use the distributed ledger technology of the blockchain to establish a blockchain network specifically used to store the data and information related to the coupling fault diagnosis. Store the preprocessed time series data, image data, and other relevant metadata, including but not limited to the acquisition time, acquisition location, equipment number, and diagnostic result, in an encrypted form on each node of the blockchain.

[0122] The blockchain technology archives and retrieves the collected data. Through the hash algorithm, it conducts decentralized management of the data, stores the relevant meta-information of the data on each blockchain node, including but not limited to the collection time, location, and device number of the data, enabling users and device supervisors to pay more attention to the health status and development trend of the device; moreover, the blockchain technology has anonymity and security, which can effectively protect data security and user privacy.

[0123] Step 5: The system automatically retrieves the node data, uses the SSOA-TimeMIL model to predict faults for the time-series data, and uses the OmniGlue-PCNN-StructSA model to monitor faults for the image data; adopts Bayesian model averaging to fuse the fault prediction results and the fault monitoring results, and comprehensively outputs the fused fault diagnosis results.

[0124] Bayesian model averaging BMA combines the prediction results of the SSOA-TimeMIL model and the diagnosis results of the OmniGlue-PCNN-StructSA model with weights, and comprehensively outputs the fused fault diagnosis results.

[0125] Based on historical experience and existing knowledge, construct the prior probability of the fault probability; calculate the conditional probability of the fault occurrence in the actual situation according to the actual situation and the device operating environment; obtain the posterior probability through the prior probability and the likelihood function, and according to Bayesian model averaging, weight-average the prediction results of the SSOA-TimeMIL model and the diagnosis results of the OmniGlue-PCNN-StructSA model, so as to fuse the prediction results and the diagnosis results.

[0126] According to the comprehensive probability criterion, the weighted posterior mean and the likelihood function formula are as follows:

[0127] E(θ|G)=ε 1 M 1 +ε 2 M 2

[0128] L(λ)=log[ε 1 p(θ|M 1 )+ε 2 p(θ|M 2 )]

[0129] In the formula, ε 1 and ε 2 are the weights of the SSOA-TimeMIL model and the OmniGlue-PCNN-StructSA model, M 1 and M 2is the prediction of SSOA-TimeMIL model and the monitoring result of OmniGlue-PCNN-StructSA model, p(θ|M 1 ) and p(θ|M 2 ) is the posterior distribution of the prediction results of the SSOA-TimeMIL model and the monitoring results of the OmniGlue-PCNN-StructSA model, and θ is the fusion fault diagnosis result. The weights in the above formula are obtained by using the expectation-maximization log-likelihood function, and then according to the posterior mean, it is used as the final fusion fault diagnosis result, and the result is recorded in the blockchain node together with the original data.

[0130] Step 6: The diagnostic results and related information recorded in the blockchain are retrieved through the background supervision system, and fed back to the relevant equipment maintenance personnel and management personnel; at the same time, the consensus mechanism of the blockchain is used to verify and confirm the diagnostic results by multiple nodes, making the data tamper-proof and improving the authenticity of the diagnostic results.

[0131] The present invention also proposes a coupling intelligent diagnosis system based on multi-source data fusion, comprising:

[0132] The data acquisition module collects the operation sequence data of the multi-source coupling in real time in time series; and uses the flaw detection system, high-speed motion camera and scanner to obtain the coupling operation image data.

[0133] In the fault prediction module, the collaborative group optimization algorithm SSOA is used to optimize the multi-instance learning model TimeMIL, and the optimized TimeMIL is used as the model for time series data fault prediction. First, the time series data is feature extracted and the extracted features are spliced. Then, the spliced ​​data is supplemented and normalized. Then, the spliced ​​feature vectors are combined into a high-dimensional feature matrix, and principal component analysis PCA is performed. Finally, the optimized TimeMIL model is used for fault prediction.

[0134] The fault monitoring module uses OmniGlue's SuperPoint encoder and DINOv2 encoder to extract global features of the captured image. Then, a dual-channel convolutional neural network (PCNN) is used to optimize the local features of the image. Finally, the coupling is monitored for faults in combination with the structural attention mechanism (StructSA).

[0135] The blockchain network stores coupling fault diagnosis-related data and information, and stores the pre-processed time series data and image data and other related metadata in encrypted form on each node of the blockchain;

[0136] The fault output module uses Bayesian Model Averaging (BMA) to weightedly fuse the prediction results of the SSOA-TimeMIL model and the monitoring results of the OmniGlue-PCNN-StructSA model, and comprehensively outputs the fused fault diagnosis results.

[0137] The background supervision system retrieves the diagnosis results and relevant information recorded in the blockchain and feeds them back to the relevant equipment maintenance personnel and management personnel; at the same time, using the consensus mechanism of the blockchain, multiple nodes verify and confirm the diagnosis results, making the data tamper-proof and improving the credibility of the diagnosis results.

[0138] The above embodiments are only for illustrating the technical concept and features of the present invention, and the purpose is to enable those who are familiar with this technology to understand the content of the present invention and implement it accordingly, and it cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit and essence of the present invention should be covered within the protection scope of the present invention.

Claims

1. A coupling intelligent diagnosis method based on multi-source data fusion, characterized in that: The following steps are involved: (1) Collect the operation sequence data of multi-source couplings in real time in time series, and obtain the operation image data of couplings using flaw detection system, high-speed motion camera and scanner; (2) Construct a fault prediction model for time series data; the fault prediction model uses the collaborative group optimization algorithm SSOA to optimize the multi-instance learning model TimeMIL, and the optimized TimeMIL is used as the model for time series data fault prediction; first, the time series data is feature extracted and the extracted features are spliced; then, the spliced ​​data is supplemented and normalized; then, the spliced ​​feature vectors are combined into a high-dimensional feature matrix, and principal component analysis PCA is performed; finally, the optimized TimeMIL model is used for fault prediction; (3) Based on the image data, a fault monitoring model OmniGlue-PCNN-StructSA is constructed to perform fault diagnosis. The SuperPoint encoder and DINOv2 encoder of OmniGlue are used to extract global features of the acquired images. Then, a dual-channel convolutional neural network PCNN is used to optimize the local features of the image. Then, the structural attention mechanism StructSA is combined to monitor the coupling faults. (4) Using the distributed ledger technology of blockchain, a blockchain network dedicated to storing coupling fault diagnosis-related data and information is established, and the pre-processed time series data and image data and other related metadata are stored in encrypted form on each node of the blockchain; (5) The Bayesian model average (BMA) is used to weight the prediction results of the SSOA-TimeMIL model and the monitoring results of the OmniGlue-PCNN-StructSA model, and the fusion fault diagnosis results are comprehensively output; (6) The diagnostic results and related information recorded in the blockchain are retrieved through the background supervision system and fed back to the relevant equipment maintenance personnel and management personnel; at the same time, the consensus mechanism of the blockchain is used to verify and confirm the diagnostic results by multiple nodes, making the data tamper-proof and improving the credibility of the diagnostic results.

2. The intelligent diagnosis method for coupling based on multi-source data fusion according to claim 1 is characterized in that: The multi-source coupling operation timing data in step (1) includes operation data of vibration, temperature, torque and rotation speed of different brands and models.

3. The intelligent diagnosis method for coupling based on multi-source data fusion according to claim 1 is characterized in that: The features extracted in step (2) include the root mean square and kurtosis of the vibration signal data, the rate of change and mean of the temperature signal data, and the fluctuation range of the torque signal data.

4. The intelligent diagnosis method for coupling based on multi-source data fusion according to claim 1 is characterized in that: The optimization implementation process of the multi-instance learning model TimeMIL using the collaborative group optimization algorithm SSOA in step (2) is as follows: The hyperparameters of the TimeMIL model include window masking ratio, number of warm-up cycles, batch size, choice of wavelet basis, scaling and translation parameters; Initialize the hyperparameter group, the formula is: X=rand(N,Dim).*(U B -L B )+L B Where X is a matrix of dimension N×Dim, N is the number of hyperparameter groups in the search space, Dim is the number of hyperparameters of the TimeMIL model, and U is B and L B For the upper and lower bounds of the hyperparameter group, the formula for updating the candidate hyperparameter group is as follows: Xnew(i,j)=X(i,j)+γ(i,j) Where Xnew(i,j) is the optimized solution of the ith candidate hyperparameter solution, X(i,j) is the current solution of the ith candidate hyperparameter solution, and γ(i,j) is the position j of the ith candidate hyperparameter solution. At the same time, combined with the speed update formula, the dynamic attraction equation is introduced to guide the hyperparameter group to optimize in a more favorable direction. The formula is as follows: γ new(i,j) =IWV+PBC+GBC+DAC+ANIC+MDC IWV=w(t)*γ(i,j) w(t+1)=w(t)*(1-exp(-k*t)) PBC=r1*(eps*rand(pbest)-X i ) GBC=r2*gbest t -X i ANIC=r4*rand(bestf)-bestf i Where IWV is the inertia weight value, w(t) is the inertia weight adaptive parameter of the tth iteration, which is used to dynamically control the balance between different directions during optimization, t is the number of iterations, k is the constant that determines the inertia weight rate; PBC is the personal best coefficient, r1 is a randomly generated value, eps is the minute value, rand(pbest) is a group randomly selected from the existing candidate hyperparameter groups, and X i is the i-th hyperparameter group; GBC is the global optimal coefficient, r2 is a random integer, gbest t is the best global hyperparameter group at the tth iteration; DAC is the dynamic attraction coefficient, r3 is a randomly generated value, attract i is the direction with the strongest local guidance optimization near the ith hyperparameter group, c1 is the additional acceleration coefficient of the dynamic guidance term; ANIC is the adaptive neighbor interaction coefficient, r4 is a randomly generated value, rand(bestf) is the random fitness value in the current fitness solution, bestf i is the fitness value of the i-th hyperparameter group; DMC is the diversity preservation coefficient, r5 is a randomly generated value, and diversity i is the maximum diversity value of the i-th hyperparameter group in the hyperparameter group, and c2 is the additional acceleration coefficient of the diversity term.

5. The intelligent diagnosis method for coupling based on multi-source data fusion according to claim 1 is characterized in that: The implementation process of step (3) is as follows: (3.1) The collected image data is used as the input of the model, and the SuperPoint encoder and DINOv2 encoder are used to extract image features in collaboration. The DINOv2 encoder is used for rough signal guidance. The SuperPoint encoder extracts the key points in the image and interpolates the feature map to obtain the DINOv2 descriptor of each key point, and each key point is consistent with and associated with its local feature description; the key points are encoded using position embedding, and the multi-layer perceptron MLP layer is used to refine the key points; (3.2) Using the DINOv2 encoder as a guide, four key point association graphs are constructed, two between-image graphs and two within-image graphs; the two between-image graphs are used to construct the key point similarity between two images under the guidance of the DINOv2 encoder, densely connecting the key points with the greatest similarity between the images in pairs to establish a relationship; the two within-image graphs are used to construct the connectivity between the key points in the same image, and the key points in the same image are bidirectionally propagated, and each key point is closely connected to other key points; (3.3) Based on the key point guidance of DINov2 encoder, information propagation is performed. There are two attention layers. The first one updates the key point position based on the intra-image, and the second one updates the key point based on the inter-image. The two layers perform cross attention. Refined key point guidance is added to separate the key point position features and local descriptions. The specific formula is as follows: In the formula, A i is the i-th key point in image A, is the local description of the i-th key point in image A, ← is the update operation, is the attention update value of the local description of the i-th key point in image A, s is the feature similarity between k key points, including all key points, is the position feature of the i-th key point in image A, k s 、v s are the query, key and value of attention, W q , W k , W v are the weights of the query, key, and value of attention, b q 、b k 、b v Biases of query, key and value for attention; (3.4) Build a dual-channel convolutional neural network PCNN, including convolutional layer, pooling layer, and fully connected layer; use the OmniGlue feature extraction data as input, and perform convolution operation on the input image data through the convolution layer to extract local features. The formula is as follows: In the formula, is the feature map, is the weight matrix, is the bias term, f(·) is the activation function, which is generally set to ReLU; Then, the spatial dimension of the feature signal is reduced through the pooling layer to reduce the number of parameters, including maximum pooling and mean pooling. The formula is as follows: In the formula, is the weight of the feature map, down(·) is the pooling function, M l is the pooling window M l ×M l ; (3.5) StructSA is used to fuse the features extracted by OmniGlue and PCNN modules to improve the feature extraction capability of the model, convert ordinary query key attention into structure-aware attention, and deploy convolution on query key relevance. The formula is as follows: In the formula, For query Key K∈R B×C ,value The projection matrix, Q i K T To calculate the correlogram, Calculate the correlation value for each image, is R convolution kernels of size P, H i Each element h i,J Calculated as: In the formula, is the local front and back features centered on the Jth data; After obtaining the structure-aware attention, in order to achieve front-back aggregation, the monitoring results are output and combined with the weights. The formula is as follows: In the formula, is the vector of the final attention weights obtained by linearly combining the scores expanded by R. is the weight of structural attention; at this time, a spatial kernel is generated for each position J by using the front-back aggregation method, and the formula is as follows: In the formula, Reprojection and dimensionality reduction of feature maps generated by multi-dimensional convolution, E J It is the spatial projection of each feature point after PCNN feature extraction, that is, the spatial weight around each feature point; (3.6) Use the fully connected layer and Softmax function to perform fault classification and output the fault monitoring results.

6. The intelligent diagnosis method for couplings based on multi-source data fusion according to claim 1 is characterized in that: The blockchain network in step (4) archives and calls the collected data, and manages the data in a decentralized manner through a hash algorithm.

7. The intelligent diagnosis method for coupling based on multi-source data fusion according to claim 1 is characterized in that: Other relevant metadata described in step (4) include collection time, collection location, device number and diagnosis result.

8. The intelligent diagnosis method for coupling based on multi-source data fusion according to claim 1 is characterized in that: The implementation process of step (5) is as follows: Construct the prior probability of the fault probability; calculate the conditional probability of the fault occurrence in the actual situation according to the actual situation and the equipment operating environment; obtain the posterior probability through the prior probability and likelihood function. According to the comprehensive probability criterion, the weighted posterior mean and likelihood function formula are as follows: E(θ|G)=ε1M1+ε2M2 L(λ)=log[ε1p(θ|M1)+ε2p(θ|M2)] Where ε1 and ε2 are the weights of the SSOA-TimeMIL model and the OmniGlue-PCNN-StructSA model, M1 and M2 are the predictions of the SSOA-TimeMIL model and the monitoring results of the OmniGlue-PCNN-StructSA model, p(θ|M1) and p(θ|M2) are the posterior distributions of the prediction results of the SSOA-TimeMIL model and the monitoring results of the OmniGlue-PCNN-StructSA model, and θ is the fusion fault diagnosis result; ε1 and ε2 are obtained by using the expectation-maximization log-likelihood function, and then according to the posterior mean, they are used as the final fusion fault diagnosis results, and the results are recorded in the blockchain node together with the original data.

9. An intelligent diagnosis system for couplings based on multi-source data fusion using the method according to any one of claims 1 to 8, characterized in that: include: The data acquisition module collects the operation sequence data of the multi-source coupling in real time in time series; Use flaw detection systems, high-speed motion cameras and scanners to obtain coupling operation image data; The fault prediction module uses the collaborative group optimization algorithm SSOA to optimize the multi-instance learning model TimeMIL, and uses the optimized TimeMIL as the model for time series data fault prediction. First, the time series data is feature extracted and the extracted features are spliced. Then, the spliced ​​data is supplemented and normalized. Then, the spliced ​​feature vectors are combined into a high-dimensional feature matrix and the principal component analysis PCA is performed. Finally, the optimized TimeMIL model is used for fault prediction. The fault monitoring module uses OmniGlue's SuperPoint encoder and DINOv2 encoder to extract global features of the captured image. Then, a dual-channel convolutional neural network (PCNN) is used to optimize the local features of the image. Then, the coupling is monitored for faults in combination with the structural attention mechanism StructSA. The blockchain network stores coupling fault diagnosis-related data and information, and stores the pre-processed time series data and image data and other related metadata in encrypted form on each node of the blockchain; The fault output module uses the Bayesian model average (BMA) to weightedly fuse the prediction results of the SSOA-TimeMIL model and the monitoring results of the OmniGlue-PCNN-StructSA model, and comprehensively outputs the fused fault diagnosis results. The backend supervision system retrieves the diagnostic results and related information recorded in the blockchain and feeds them back to the relevant equipment maintenance and management personnel; at the same time, the consensus mechanism of the blockchain is used to verify and confirm the diagnostic results by multiple nodes, making the data tamper-proof and improving the credibility of the diagnostic results.

Images