Motor fault diagnosis method and system based on color image fusion symmetrical point mode

Abstract

The application discloses a motor fault diagnosis method and system based on a color image fusion symmetry point mode, and the method comprises the following steps: converting vibration signals and electromagnetic signals of a motor into symmetry point mode images respectively, and generating color signal images with fusion feature information; abstracting the color signal images with fusion feature information into nodes and edges in a high-dimensional semantic space respectively, so as to construct graph structure data; using respective capsule graph network models to diagnose and classify the graph structure data of the vibration signals and the electromagnetic signals respectively, and fusing the diagnosis and classification results of the vibration signals and the electromagnetic signals through a voting mechanism to obtain a final diagnosis and classification result. The application aims to convert multi-channel time domain signals into information-intensive and geometric-consistent image expressions, and to realize a new scheme of uniformly modeling features of the images based on a deep graph structure network with topological perception capability, so as to realize motor fault diagnosis with high diagnosis precision and strong robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of motor fault diagnosis, and in particular to a motor fault diagnosis method and system based on color image fusion of symmetric point patterns. BACKGROUND

[0002] With the rapid development of new energy vehicles, intelligent manufacturing and high-end numerical control equipment, high-performance motors such as permanent magnet synchronous motors are widely used in key scenarios such as traction drive, precision servo and high-speed machine tools. Early hidden faults such as rotor demagnetization, stator winding inter-turn short circuit, bearing wear and air gap eccentricity are prone to occur in motors under complex load, wide frequency and continuous long-time operation conditions. If not identified in time and accurately, these small abnormalities will quickly evolve into severe vibration, overheating and even system shutdown, causing huge economic losses and safety risks. Therefore, developing high-sensitivity and high-robustness motor fault diagnosis technology has become a core demand in the field of industrial operation and intelligent equipment.

[0003] The current research mainly relies on single-channel vibration signals or current signals, extracts statistical features through time-frequency analysis methods such as fast Fourier transform, wavelet packet decomposition and empirical mode decomposition, and completes fault recognition using classification models such as support vector machine and convolutional neural network. Although the above methods have achieved certain results under specific working conditions, there are still three shortcomings: first, a single signal source cannot fully represent the electrical-magnetic-mechanical coupling characteristics of the motor, resulting in limited diagnostic accuracy; second, artificially designed time-frequency features are sensitive to speed fluctuations, load changes and background noise, and have poor robustness; third, although deep networks can automatically learn features, they usually require large-scale labeled samples, and early fault samples in industrial sites are extremely scarce, making the model prone to overfitting and lacking interpretability.

[0004] To address the low utilization rate of multi-source information, some scholars have begun to try to map time-domain signals into two-dimensional images, such as symmetric point patterns or recurrence plots, and then use traditional convolutional networks to extract texture features. However, SDP images often lose cross-modal correlation when directly spliced between different channels, and ordinary CNNs are difficult to explicitly capture node-edge topological relationships, making it even more difficult to handle geometric distortion caused by rotation and scale changes. On the other hand, graph neural networks can express complex structures, but there is still a lack of unified framework for efficiently constructing graphs from original time series data and maintaining local shape invariance in the graph domain.

[0005] In summary, existing motor fault diagnosis technology has not fully fused vibration and electromagnetic multi-modal signals, and lacks a deep feature extraction method that combines scale invariance, rotation invariance and topological awareness. At the same time, in the early weak fault and working condition migration scenarios, traditional models are prone to false positives or false negatives. SUMMARY

[0006] The technical problem to be solved by this invention is to provide a motor fault diagnosis method and system based on color image fusion symmetrical point pattern in response to the above-mentioned problems in the prior art. This invention aims to transform multi-channel time-domain signals into information-dense and geometrically consistent image representations, and to perform unified feature modeling on these images based on a deep graph structure network with topological awareness capabilities, thereby achieving high diagnostic accuracy and strong robustness in motor fault diagnosis.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] A motor fault diagnosis method based on color image fusion of symmetrical point patterns includes the following steps:

[0009] 1) Acquire the vibration and electromagnetic signals of the motor;

[0010] 2) The vibration signal and electromagnetic signal are converted into symmetrical point pattern images respectively, and a color signal image with fused feature information is generated by color image fusion technology;

[0011] 3) Based on the scale-invariant feature transformation method of feature embedding, the color signal image with fused feature information is abstracted into nodes and edges in a high-dimensional semantic space, thereby constructing graph structure data composed of nodes and edges;

[0012] 4) The graph structure data of vibration signals and electromagnetic signals are respectively diagnosed and classified using their respective capsule graph network models, and the diagnostic and classification results of vibration signals and electromagnetic signals are merged through a voting mechanism to obtain the final diagnostic and classification result.

[0013] Optionally, in step 2), when converting the vibration signal and electromagnetic signal into symmetrical point pattern images respectively, this includes processing the vibration signal or electromagnetic signal... X ={ x 1, x 2, ..., x n},in x 1~ x n Vibration signal or electromagnetic signal respectively X The first to nth sampled signals, where n is the number of sampled signals, are converted into points in the symmetric point space in polar coordinates according to the following formula. :

[0014] ,

[0015] ,

[0016] ,

[0017] in, The polar radius of the transformed signal point. X max and X min Time-domain signal X The maximum and minimum values ​​in For the first s Angles of a symmetrical plane θ s =360 m / k , m The index of the mirror-symmetric plane. m =0,1,…, k -1, k , k The number of mirror-symmetric planes. Vibration signal or electromagnetic signal X The i-th sampled data in Vibration signal or electromagnetic signal X The first in Each sampled data, To delay, The angle of counterclockwise rotation of the mirror symmetry plane of the transformed signal point. The angle is the clockwise rotation of the mirror symmetry plane of the transformed signal point. For gain angle; based on the point in the symmetric space under polar coordinates The three attributes are used as color values ​​for the three RGB channels to generate a symmetrical dot pattern image.

[0018] Optionally, step 2) of generating a color signal image with fused feature information using color image fusion technology includes: firstly, calculating the signal energy entropy of each of the three RGB channels. E , cliff K and spectral peak P Construct feature vectors F k =[ E k ，K k ，P k The fusion weights of each channel signal are calculated according to the following formula:

[0019] ,

[0020] in, The fusion weight for the k-th channel. This represents the accuracy of this channel in classifying the current fault type based on historical data. β As the adjustment factor, a color signal image with fused feature information is then generated according to the following formula:

[0021] ,

[0022] in, To fuse feature information in color signal images, , and These are the fusion weights for the signals from channels 1 to 3, respectively. , and These are images of the red, green, and blue channel signals in the symmetrical point pattern image, respectively.

[0023] Optionally, step 3) includes:

[0024] 3.1) By performing Gaussian blurring on the input color signal image at multiple scales, a Gaussian pyramid is constructed from the Gaussian smoothed image layers obtained from the Gaussian blurring at multiple scales. The difference between adjacent scale Gaussian images in the Gaussian pyramid is then calculated according to the following formula to generate the Difference Gaussian Space (DoG):

[0025] ,

[0026] in, Represents the difference Gaussian space DoG with respect to Gaussian difference image layers, Indicates about Gaussian kernel function, This is a two-dimensional convolution operation. Color signal images for fusing feature information At coordinate point Pixel values; This is the scale multiplication factor, used to control the interval between adjacent scales; This serves as the baseline scale parameter, used to determine the granularity of feature detection.

[0027] 3.2) In the Difference Gaussian (DoG) space, the position and scale of candidate keypoints are corrected at the sub-pixel level by fitting a three-dimensional quadratic function or Taylor expansion. The contrast of extreme points is calculated using the Hessian matrix, and unstable keypoints with low contrast are removed according to a preset contrast threshold to obtain stable keypoints.

[0028] ,

[0029] in, This is the offset of the keypoint, including corrections for spatial coordinates and scale; This is the contrast threshold, used to filter noise-sensitive points; for The inverse matrix, It is a Gaussian difference function. To stabilize the position of key points in x coordinates, representing the contrast of extreme points;

[0030] The eigenvalue analysis of Hessian matrix is used to distinguish edge response and corner response, and the false stable key points on the edge are filtered by the principal curvature ratio to retain the features with stable geometric structure according to the following formula:

[0031] ,

[0032] wherein, is the Hessian matrix composed of the second derivative of the Gaussian difference image layer, is the trace of H , the trace of Hessian matrix represents the sum of principal curvatures; is the curvature product of H , r is the principal curvature ratio threshold, is the second derivative of the image in the x direction, representing the change rate of the change of the gray value along the horizontal direction, and are the mixed second derivatives of the image in the x and y directions, representing the joint change of the gray value in the horizontal and vertical directions, is the second derivative of the image in the y direction, representing the change rate of the change of the gray value along the vertical direction;

[0033] The neighborhood gradient magnitude and the direction gradient angle are calculated according to the surrounding area of the stable key point after filtering and correction:

[0034] ,

[0035] ,

[0036] wherein, is the neighborhood gradient magnitude of the stable key point after filtering and correction, , , and are the gray values of the positions , , and in the Gaussian smoothing image layer, is the stable key point The directional gradient angle is used to construct a multi-directional histogram based on the neighborhood gradient magnitude and the directional gradient angle. Key points in multiple directions are generated by setting the main direction as the peak value of the histogram and the secondary peaks as greater than the main peak.

[0037] 3.3) Rotate the neighborhood of the key point to the main direction, divide the 16×16 neighborhood into 4×4 sub-regions, count 8 directional buckets in each region, and concatenate them into a 128-dimensional vector according to the following formula, then normalize the vector to generate nodes in the graph structure data:

[0038] ,

[0039] in, For the first in graph structure data i The feature vector of each node For the stable key points after filtering correction The neighborhood gradient magnitude, For the first i The set of pixels in the sub-region of each node. for The pixel position in the image. For indicator functions, For the stable key points after filtering correction The direction gradient angle; Let i be the i-th directional bucket, which is one of eight angular intervals evenly divided within 360°, used to accumulate gradient energy within that angular range; the feature vectors of each node in the graph structure data are used as 128-dimensional SIFT descriptors, and the edge weights in the graph structure data are calculated according to the following formula to obtain the graph structure data:

[0040] ,

[0041] in, For nodes v i and v j Edge weights between them This is a coefficient for balancing spatial and semantic weights. For nodes v i coordinates For nodes v j coordinates σ s The standard deviation of the Gaussian kernel for spatial distance. For nodes v i 128-dimensional SIFT descriptor, For nodes v j 128-dimensional SIFT descriptor.

[0042] Optionally, in step 4), when performing diagnostic classification on the graph structure data of vibration signals and electromagnetic signals using their respective capsule graph network models, the capsule graph network model dynamically routes the input graph structure data of vibration signals or electromagnetic signals to achieve cross-modal feature fusion and output the activated capsule vector according to the following formula:

[0043] ,

[0044] ,

[0045] ,

[0046] in, Let j be the activated capsule vector output by the graph capsule network model. Let be the weighted sum of the dynamic routing coefficients of the j-th graph capsule network model and the feature vectors of the primary capsules. Let be the dynamic routing coefficient from capsule i to capsule j. Let i be the primary capsule feature vector from capsule i to capsule j. Let be the coupling coefficient from capsule i to capsule j. Let be the coupling coefficient from capsule i to capsule k.

[0047] Optionally, it also includes calculating training metrics after each round of training when training the graph capsule network model. If training metrics If the training threshold is exceeded, the graph capsule network model is considered to have completed training; otherwise, iterative training needs to continue. The training metrics are calculated as follows: include:

[0048] Adjust the initial diagnostic weights of each graph capsule network model according to the following formula:

[0049] ,

[0050] in, The initial diagnostic weights for the m-th graph capsule network model are... Let m be the activated capsule vector output by the m-th graph capsule network model. for The vector magnitude, The activated capsule vector is the output of the nth graph capsule network model. for The vector magnitude, For coefficients, The number of graph capsule network models; the entropy weight coefficients are calculated according to the following formula:

[0051] ,

[0052]

[0053] in, Let m be the entropy weight coefficient of the m-th graph capsule network model. Let m be the information entropy of the m-th graph capsule network model. The information entropy of the nth graph capsule network model is... Number of fault categories The predicted probability distribution for each fault category; the training index is calculated according to the following formula. :

[0054] ,

[0055] in, The voting score for the k types of faults in the m-th graph capsule network model.

[0056] Optionally, before step 1), a step of training graph network models for vibration signals and electromagnetic signals is included. Training the graph network models involves collecting vibration signals and electromagnetic signals of the motor under different operating conditions as fault diagnosis data. These different operating conditions include normal conditions and various fault conditions. The obtained fault diagnosis data includes the motor fault electromagnetic signal dataset shown in the following formula. :

[0057]

[0058] in, For the first i One motor fault signal, For fault labels, Electromagnetic signal dataset for motor faults The number of samples, T The number of sampling points. This is a dataset of electromagnetic fault signals for motors. This is a dataset for motor electromagnetic fault categories.

[0059] And the motor fault vibration signal dataset shown in the following formula :

[0060]

[0061] in, For the first i Vibration signal of a motor fault. For fault labels, Data set of vibration signals for motor faults The number of samples.

[0062] Furthermore, the present application also provides a motor fault diagnosis system based on color image fusion symmetric dot pattern, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the motor fault diagnosis method based on color image fusion symmetric dot pattern.

[0063] Furthermore, the present application also provides a computer readable storage medium, wherein a computer program or instructions are stored in the computer readable storage medium, and the computer program or instructions are programmed or configured to execute the motor fault diagnosis method based on color image fusion symmetric dot pattern by a processor.

[0064] Furthermore, the present application also provides a computer program product, comprising a computer program or instructions, which are programmed or configured to execute the motor fault diagnosis method based on color image fusion symmetric dot pattern by a processor.

[0065] Compared with the prior art, the present application mainly has the following beneficial effects: the method of the present application first collects vibration signals and electromagnetic time domain signals of the motor under different operating conditions to provide data for fault diagnosis. Then, the color image fusion technology is used to convert these time domain signals into symmetric dot pattern (SDP) images, generating color signal images that fuse multi-dimensional feature information. Next, the SIFT graph construction method based on feature embedding is used to abstract the image into nodes and edges in a high-dimensional semantic space, and to construct a graph structure data. These graph structure data are input into a graph capsule network, and the image features are extracted by the graph capsule layer and classified into vibration signals and electromagnetic signals. Finally, an enhanced voting strategy is used to fuse the diagnosis results of different signal sources to improve the accuracy and reliability of the diagnosis. The motor fault diagnosis method and system of the present application can effectively process multi-source signals and achieve efficient and accurate fault diagnosis by integrating multiple advanced technologies, and has a wide application prospect, especially in the field of complex and high-risk motor fault detection. By combining image processing and deep learning, the precision and robustness of fault diagnosis are improved, and the present application has strong industrial application value. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 FIG. 1 is a basic flowchart of the method of the present application.

[0067] Figure 2 FIG. 2 is a schematic diagram of the principle of the method of the present application.

[0068] Figure 3Examples of color signal images fused with feature information of embodiments of the present application, wherein (a) is a color signal image of a normal motor; (b) is a color signal image of a rotor imbalance fault; (c) is a color signal image of a rotor misalignment fault; (d) is a color signal image of a shaft bending fault; (e) is a color signal image of a bearing fault; (f) is a color signal image of a rotor bar fault; (g) is a color signal image of a stator winding fault; and (h) is a color signal image of a single-phase open circuit fault.

[0069] Figure 4 A flowchart of data preprocessing of graph structure data in embodiments of the present application.

[0070] Figure 5 A flowchart of training of a graph capsule network model in embodiments of the present application. DETAILED DESCRIPTION

[0071] In order to enable personnel in the technical field to better understand the technical solutions of the present application, the technical solutions of the present application will be further described in detail below with reference to the accompanying drawings of embodiments of the present application.

[0072] As shown in Figure 1 , the motor fault diagnosis method based on color image fusion of symmetry point patterns of the present embodiment includes the following steps:

[0073] 1) Obtain vibration signals and electromagnetic signals of the motor;

[0074] 2) Convert the vibration signals and the electromagnetic signals into symmetry point pattern images, respectively, and generate color signal images fused with feature information through color image fusion technology;

[0075] 3) Abstract the color signal images fused with feature information into nodes and edges in a high-dimensional semantic space based on a scale-invariant feature transformation method of feature embedding, respectively, so as to construct graph structure data composed of nodes and edges;

[0076] 4) Diagnose and classify the graph structure data of the vibration signals and the electromagnetic signals using respective capsule graph network models, and fuse the diagnosis and classification results of the vibration signals and the electromagnetic signals through a voting mechanism to obtain a final diagnosis and classification result.

[0077] As shown in Figure 2 , the method of the present embodiment mainly includes signal acquisition, symmetry point pattern image generation, color image fusion, graph structure data generation, diagnosis and classification of a capsule graph network model, and obtaining a final diagnosis and classification result through a voting mechanism, and further includes training of the capsule graph network model. When diagnosing and classifying the graph structure data of the vibration signals and the electromagnetic signals using respective capsule graph network models, in order to facilitate the distinction between the respective capsule graph network models, Figure 2They are named the electromagnetic-capsule graph network model and the vibration-capsule graph network model, respectively. The electromagnetic-capsule graph network model is used for the diagnostic classification of graph structure data of electromagnetic signals, and the vibration-capsule graph network model is used for the diagnostic classification of graph structure data of vibration signals.

[0078] In this embodiment, before step 1), there is a step of training graph network models for the vibration signal and electromagnetic signal respectively. The training of the graph network model includes collecting vibration signals and electromagnetic signals of the motor under different operating states as fault diagnosis data. The different operating states include normal state and various fault states (e.g., bearing inner ring fault state, bearing outer ring fault state, and bearing rolling element fault state, etc., and the fault states can be added or reduced according to actual needs). The obtained fault diagnosis data includes the motor fault electromagnetic signal dataset shown in the following formula. :

[0079]

[0080] in, For the first i One motor fault signal, For fault labels, Electromagnetic signal dataset for motor faults The number of samples, T The number of sampling points. This is a dataset of electromagnetic fault signals for motors. This is a dataset for motor electromagnetic fault categories.

[0081] And the motor fault vibration signal dataset shown in the following formula :

[0082]

[0083] in, For the first i Vibration signal of a motor fault. For fault labels, Data set of vibration signals for motor faults The number of samples. In this embodiment, during signal acquisition, an AC flux sensor and a triaxial piezoelectric accelerometer are used to collect electromagnetic signals and vibration signals of the permanent magnet synchronous motor under different operating conditions as fault diagnosis data. The vibration signals collected by the triaxial piezoelectric accelerometer include vibration signal components in the X, Y, and Z axes, and the final vibration signal is obtained by adding the vibration signal components in the X, Y, and Z axes.

[0084] Step 2) converts the obtained time-domain data into a symmetric point pattern image (SDP image) to obtain the SDP image of the vibration and electromagnetic signals, and then uses a color image fusion method to obtain a multi-source signal fused SDP color image. In this embodiment, step 2) converts the vibration signal and electromagnetic signal into symmetric point pattern images respectively, including for vibration signals or electromagnetic signals... X ={ x 1, x 2, ..., x n},in x 1~ x n Vibration signal or electromagnetic signal respectively X The first to nth sampled signals, where n is the number of sampled signals, are converted into points in the symmetric point space in polar coordinates according to the following formula. :

[0085] ,

[0086] ,

[0087] ,

[0088] in, The polar radius of the transformed signal point. X max and X min Time-domain signal X The maximum and minimum values ​​in For the first s Angles of a symmetrical plane θ s =360 m / k , m The index of the mirror-symmetric plane. m =0,1,…, k -1, k , k The number of mirror-symmetric planes. Vibration signal or electromagnetic signal X The i-th sampled data in Vibration signal or electromagnetic signal X The first in Each sampled data, To delay, The angle of counterclockwise rotation of the mirror symmetry plane of the transformed signal point. The angle is the clockwise rotation of the mirror symmetry plane of the transformed signal point. For gain angle; based on the point in the symmetric space under polar coordinates three attributes of the three channels of RGB as color values to generate a symmetric point pattern image. The shape of the symmetric point pattern image is determined jointly, and the one-dimensional electromagnetic and vibration signal is converted into a two-dimensional image to extract the subtle features thereof.

[0089] In step 2) of the embodiment, the color signal image with fused feature information is generated by a color image fusion technology, including: first, calculating the signal energy entropy E , kurtosis K and spectral peak P of each channel of RGB respectively to construct a feature vector F k =[ E k ，K k ，P k ] and calculating the fusion weight of each channel signal according to the following formula:

[0090] ,

[0091] wherein, is the fusion weight of the kth channel, is the classification accuracy of the channel to the current fault type in the historical data, β is an adjustment factor, and then superimposing the three-channel signal images according to the weight to generate a color signal image with fused feature information according to the following formula:

[0092] ,

[0093] wherein, is the color signal image with fused feature information, , and are the fusion weights of the first to third channel signals, , and are the images of the channel signals of red, green and blue in the symmetric point pattern image. The color signal image with fused feature information generated in the above manner can accurately represent various motor faults, Figure 3 is an example of the color signal image with fused feature information of the embodiment, wherein (a) is the color signal image of a normal motor; (b) is the color signal image of a rotor imbalance fault; (c) is the color signal image of a rotor misalignment fault; (d) is the color signal image of a shaft bending fault; (e) is the color signal image of a bearing fault; (f) is the color signal image of a rotor bar fault; (g) is the color signal image of a stator winding fault; and (h) is the color signal image of a single-phase open circuit fault.

[0094] Step 3) Abstract the color image into nodes and edges in a high-dimensional semantic space by a SIFT graph construction method based on feature embedding, and construct a graph structure data. Specifically, as shown in Figure 4 Step 3) of the embodiment includes:

[0095] 3.1) SIFT feature extraction, including: by performing Gaussian blur processing on the input color signal image in multiple scale spaces, constructing a Gaussian pyramid of the Gaussian smoothing image layers obtained by Gaussian blur processing in multiple scale spaces, and calculating the difference value of the Gaussian images in adjacent scales in the Gaussian pyramid to generate a difference Gaussian space DoG according to the following formula:

[0096]

[0097] wherein, represents a Gaussian difference image layer about in the difference Gaussian space DoG, represents a Gaussian kernel function about , and is a two-dimensional convolution operation, is a color signal image fused with feature information whose pixel value at the coordinate point ; is a scale multiplication factor for controlling the interval between adjacent scales; is a reference scale parameter for determining the granularity of feature detection; by generating a Gaussian pyramid through multi-scale Gaussian convolution, each group contains multiple image layers, which are down-sampled to cover different scale features. Within each group, multiple scale image layers are generated by an incremental reference scale parameter σ, which provides a basis for subsequent Gaussian difference calculation; by calculating the difference value of the Gaussian images in adjacent scales in the Gaussian pyramid, a difference Gaussian space DoG is generated, which can highlight high-frequency features (such as edges and corners) for detecting scale space extreme points;

[0098] 3.2) Key point screening, including: in the difference Gaussian space DoG, the position and scale of the candidate key points are corrected at a sub-pixel level by three-dimensional quadratic function fitting or Taylor expansion, the contrast of the extreme points is calculated by Hessian matrix, and the key points with low contrast are removed according to a preset contrast threshold to obtain stable key points:

[0099]

[0100] wherein, is the offset of the key point, which includes the correction of spatial coordinates and scale; is a contrast threshold for filtering noise-sensitive points; is the inverse matrix of , and is a Gaussian difference function,​​ To stabilize the position of the key point in the x coordinate, contrast value representing the extreme point; in the Gaussian difference pyramid, each pixel needs to be compared with the neighborhood of the same layer and the neighborhood of the upper and lower adjacent layers, if it is the maximum or minimum value, it is marked as a candidate key point, and then the stable key point is screened through the contrast threshold and the edge response, wherein the contrast threshold is a normalized threshold value, used to eliminate low-contrast points, and the contrast threshold value can be taken as needed, for example, in this embodiment, the value is 0.03;

[0101] The eigenvalue analysis of the Hessian matrix is used to distinguish the edge response and the corner response, and the pseudo-stable key points on the edge are filtered through the principal curvature ratio to retain the features with stable geometric structure according to the following formula:

[0102] ,

[0103] wherein, is the Hessian matrix composed of the second derivative of the Gaussian difference image layer, is the trace of H , the trace of the Hessian matrix represents the sum of the principal curvatures; is the curvature product of H , r is the principal curvature ratio threshold, is the second derivative of the image in the x direction, representing the change rate of the change of the gray value along the horizontal direction, and are the mixed second derivatives of the image in the x and y directions, representing the joint change of the gray value in the horizontal and vertical directions, is the second derivative of the image in the y direction, representing the change rate of the change of the gray value along the vertical direction;

[0104] The neighborhood gradient amplitude and the direction gradient angle are calculated according to the surrounding area of the stable key point after filtering and correction:

[0105] ,

[0106] ,

[0107] wherein, is the neighborhood gradient amplitude of the stable key point after filtering and correction, , , and These are the positions in the Gaussian smoothed image layer. , , and grayscale value, For the stable key points after filtering correction The directional gradient angle is used to construct a multi-directional histogram based on the neighborhood gradient magnitude and the directional gradient angle. Key points in multiple directions are generated by setting the main direction as the peak value of the histogram and the secondary peaks as greater than the main peak.

[0108] 3.3) Generate SIFT descriptors, including: rotating the keypoint neighborhood to the main direction, dividing the 16×16 neighborhood into 4×4 sub-regions, counting 8 directional buckets in each region, concatenating them into a 128-dimensional vector according to the following formula, and performing normalization to generate nodes in the graph structure data:

[0109] ,

[0110] in, For the first in graph structure data i The feature vector of each node For the stable key points after filtering correction The neighborhood gradient magnitude, For the first i The set of pixels in the sub-region of each node. for The pixel position in the image. For indicator functions, For the stable key points after filtering correction The direction gradient angle; Let i be the i-th directional bucket, which is one of eight angular intervals evenly divided within 360°, used to accumulate gradient energy within that angular range; the feature vectors of each node in the graph structure data are used as 128-dimensional SIFT descriptors, and the edge weights in the graph structure data are calculated according to the following formula to obtain the graph structure data:

[0111] ,

[0112] in, For nodes v i and v j Edge weights between them This is a coefficient for balancing spatial and semantic weights. For nodes v i coordinates For nodes v j coordinates σ sGaussian kernel standard deviation for spatial distance, for node v i 128-dimensional SIFT descriptor for node for node v j 128-dimensional SIFT descriptor for node

[0113] In step 4) of the embodiment, when the graph structure data of the vibration signal and the electromagnetic signal are diagnosed and classified by using the respective capsule graph network models, the capsule graph network model realizes cross-modal feature fusion by dynamic routing according to the following formula to output the activated capsule vector after the vibration signal or the electromagnetic signal graph structure data is input:

[0114]

[0115]

[0116]

[0117] wherein, is the activated capsule vector output by the jth graph capsule network model, is the weighted sum of the dynamic routing coefficient and the primary capsule feature vector of the jth graph capsule network model, is the dynamic routing coefficient from capsule i to capsule j, is the primary capsule feature vector from capsule i to capsule j, is the coupling coefficient from capsule i to capsule j, is the coupling coefficient from capsule i to capsule k. In step 4), the diagnostic classification results of the vibration signal and the electromagnetic signal are fused by a voting mechanism to obtain the final diagnostic classification result. For example, the final diagnostic classification result is the value with the largest or smallest diagnostic classification result. In addition, the diagnostic classification results of the vibration signal and the electromagnetic signal can be weighted and summed to obtain the final diagnostic classification result, or a classifier can be used to map the diagnostic classification results of the vibration signal and the electromagnetic signal to obtain the final diagnostic classification result, etc.

[0118] The embodiment further comprises, when training the graph capsule network model, calculating the training indicator after each round of training, and if the training indicator exceeds a preset threshold, determining that the training of the graph capsule network model is complete, otherwise the training needs to be iterated; the calculation of the training indicator includes:

[0119] The preliminary diagnosis weight of each graph capsule network model is adjusted according to the following formula:

[0120] ,​​​

[0121] wherein, is the preliminary diagnosis weight of the mth graph capsule network model, is the activated capsule vector output by the mth graph capsule network model, is the vector norm of is the activated capsule vector output by the nth graph capsule network model, is the vector norm of is the coefficient, is the number of graph capsule network models; the entropy weight coefficient is calculated according to the following formula:

[0122] ,

[0123]

[0124] wherein, is the entropy weight coefficient of the mth graph capsule network model, is the information entropy of the mth graph capsule network model, is the information entropy of the nth graph capsule network model, is the number of fault categories, is the prediction probability distribution of each fault category; the training index is obtained according to the following formula:

[0125] ,

[0126] wherein, is the kth fault voting score of the mth graph capsule network model.

[0127] The enhanced voting mechanism in the embodiment suppresses the interference of low-quality signals by weighting the diagnosis weights of the vibration and electromagnetic signals and introducing the entropy weight method to dynamically adjust the voting weights. After the training index Diagnosis ≥ 95%, it indicates that the graph capsule network model accuracy meets the use requirements and the modeling is completed. Figure 5 ​​​As shown, the training of the graph capsule network model includes: performing initialization of the graph capsule network model (the electromagnetic-capsule graph network model and the vibration-capsule graph network model); randomly extracting an I group of training sets ((x(i), y(i)) and a J group of test sets ((x(j), y(j)) as training data of the graph capsule network model, x(i) and x(j) are vibration signals / electromagnetic signals, y(i) and y(j) are labels of fault classification, performing forward propagation of the graph capsule network model (the electromagnetic-capsule graph network model and the vibration-capsule graph network model) using the training data, that is, calculating the graph capsule, including node message passing and capsule aggregation inside the graph capsule network model; dynamic routing algorithm iteratively optimizes capsule alignment to update coupling coefficients and redistributes feature weights); obtaining vibration signal output / electromagnetic signal output (from the electromagnetic-capsule graph network model and the vibration-capsule graph network model); voting the vibration signal output / electromagnetic signal output to obtain the final result, and calculating the training index using the test set after each training If the training index reaches the index, it is determined that the training of the electromagnetic-capsule graph network model and the vibration-capsule graph network model is completed, otherwise the Adam optimizer or other optimizers are used to update the parameters of the electromagnetic-capsule graph network model and the vibration-capsule graph network model, and then the forward propagation is continued to be iterated.

[0128] In order to verify the motor fault diagnosis method based on color image fusion symmetry point mode of the embodiment, the method (MCFRes-CapsGNN) of the embodiment and the existing motor fault diagnosis methods are compared, the existing motor fault diagnosis methods include long short-term memory network LSTM, multilayer perceptron MLP, symmetric equivariant convolutional neural network SECNN, capsule network CapsNet and dynamic graph convolutional network DGCN, the indexes adopted include accuracy, precision, recall and F score, and the finally obtained results are as shown in Table 1.

[0129] Table 1: Comparison of results of the method of the embodiment and the existing motor fault diagnosis methods

[0130]

[0131] As shown in Table 1, the method (MCFRes-CapsGNN) of the embodiment has achieved better results than the long short-term memory network LSTM, the multilayer perceptron MLP, the symmetric equivariant convolutional neural network SECNN, the capsule network CapsNet and the dynamic graph convolutional network DGCN in terms of the four indexes of accuracy, precision, recall and F score.

[0132] In addition, considering that the vibration signals collected by the three-axis piezoelectric acceleration sensor of the present embodiment include X, Y, and Z three-axis direction vibration signal components, and by adding the X, Y, and Z three-axis direction vibration signal components as the final vibration signal. In order to verify the effectiveness of the X, Y, and Z three-axis direction vibration signal components, an ablation experiment is performed on the X, Y, and Z three-axis direction vibration signal components and the final vibration signal (fusion) in the present embodiment, and two well-known deep learning models, LeNet and AlexNet, are used to compare the detection accuracy, and the results are shown in Table 2.

[0133] Table 2: Comparison of effectiveness of X, Y, and Z three-axis direction vibration signal components and final vibration signal (fusion)

[0134]

[0135] In Table 2, "X channel" to "Z channel" respectively represent X, Y, and Z three-axis direction vibration signal components, and "fusion" represents the final vibration signal (fusion). As can be seen from Table 2, by collecting X, Y, and Z three-axis direction vibration signal components and adding them as the final vibration signal, the detection accuracy can be greatly improved.

[0136] In summary, the method of the present embodiment first collects vibration signals and electromagnetic time domain signals of the motor under different operating conditions to provide data for fault diagnosis. Then, the color image fusion technology is used to convert these time domain signals into symmetric dot pattern images (SDP), generating color signal images that fuse multi-dimensional feature information. Next, the SIFT graph construction method based on feature embedding is used to abstract the image into nodes and edges in a high-dimensional semantic space, and to construct a graph structure data. These graph structure data are input into the graph capsule network, and the image features are extracted by the graph capsule layer and classified into vibration signals and electromagnetic signals. Finally, an enhanced voting strategy is used to fuse the diagnostic results of different signal sources to improve the accuracy and reliability of the diagnosis. The motor fault diagnosis method and system described in the present invention can effectively process multi-source signals and achieve efficient and accurate fault diagnosis by integrating multiple advanced technologies, and has a wide application prospect, especially in the field of complex and high-risk motor fault detection. By combining image processing and deep learning, the precision and robustness of fault diagnosis are improved, and it has strong industrial application value.

[0137] In addition, the embodiment further provides a motor fault diagnosis system based on a color image fusion symmetry point mode, comprising a microprocessor and a memory connected with each other, the microprocessor being programmed or configured to execute the motor fault diagnosis method based on the color image fusion symmetry point mode. In addition, the embodiment further provides a computer readable storage medium, wherein a computer program or instructions are stored, the computer program or instructions being programmed or configured to execute the motor fault diagnosis method based on the color image fusion symmetry point mode by a processor. In addition, the embodiment further provides a computer program product, comprising a computer program or instructions, the computer program or instructions being programmed or configured to execute the motor fault diagnosis method based on the color image fusion symmetry point mode by a processor.

[0138] Those skilled in the art should understand that the technical solutions provided by the present application can be in the form of a method, a system or a computer program product. Therefore, the present application can be in the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code. The present application is described with reference to flowcharts and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks Figure 1steps of the functions specified in the one or more blocks.

[0139] The above merely describes the preferred embodiments of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical scheme falling within the concept of the present application shall fall within the protection scope of the present application. It should be noted that, for ordinary skilled persons in the art, some improvements and refinements without departing from the principles of the present application shall also be considered as falling within the protection scope of the present application.

Claims

1. A method for motor fault diagnosis based on color image fusion symmetrical point patterns, characterized in that, Includes the following steps: 1) Acquire the vibration and electromagnetic signals of the motor; 2) The vibration signal and electromagnetic signal are converted into symmetrical point pattern images respectively, and a color signal image with fused feature information is generated by color image fusion technology; 3) Based on the scale-invariant feature transformation method of feature embedding, the color signal image with fused feature information is abstracted into nodes and edges in a high-dimensional semantic space, thereby constructing graph structure data composed of nodes and edges; 4) The graph structure data of vibration signals and electromagnetic signals are respectively diagnosed and classified using their respective capsule graph network models, and the diagnostic and classification results of vibration signals and electromagnetic signals are merged through a voting mechanism to obtain the final diagnostic and classification result.

2. The motor fault diagnosis method based on color image fusion symmetrical point pattern according to claim 1, characterized in that, In step 2), when converting the vibration signal and electromagnetic signal into symmetrical point pattern images respectively, this includes processing either the vibration signal or the electromagnetic signal. X ={ x 1, x 2, ..., x n },in x 1~ x n Vibration signal or electromagnetic signal respectively X The first to nth sampled signals, where n is the number of sampled signals, are converted into points in the symmetric point space in polar coordinates according to the following formula. : ， ， ， in, The polar radius of the transformed signal point. X max and X min Time-domain signal X The maximum and minimum values ​​in For the first s Angles of a symmetrical plane θ s =360 m / k , m The index of the mirror-symmetric plane. m =0,1,…, k -1, k , k The number of mirror-symmetric planes. Vibration signal or electromagnetic signal X The i-th sampled data in Vibration signal or electromagnetic signal X The first in Each sampled data, To delay, The angle of counterclockwise rotation of the mirror symmetry plane of the transformed signal point. The angle is the clockwise rotation of the mirror symmetry plane of the transformed signal point. For gain angle; based on the point in the symmetric space under polar coordinates The three attributes are used as color values ​​for the three RGB channels to generate a symmetrical dot pattern image.

3. The motor fault diagnosis method based on color image fusion symmetrical point pattern according to claim 1, characterized in that, Step 2) involves generating a color signal image with fused feature information using color image fusion technology. This includes: firstly, calculating the signal energy entropy of each of the three RGB channels. E , cliff K and spectral peak P Construct feature vectors F k =[ E k ， K k ，P k The fusion weights of each channel signal are calculated according to the following formula: ， in, The fusion weight for the k-th channel. This represents the accuracy of this channel in classifying the current fault type based on historical data. β As the adjustment factor, a color signal image with fused feature information is then generated according to the following formula: ， in, To fuse feature information in color signal images, , and These are the fusion weights for the signals from channels 1 to 3, respectively. , and These are images of the red, green, and blue channel signals in the symmetrical point pattern image, respectively.

4. The motor fault diagnosis method based on color image fusion symmetrical point pattern according to claim 1, characterized in that, Step 3) includes: 3.1) By performing Gaussian blurring on the input color signal image at multiple scales, a Gaussian pyramid is constructed from the Gaussian smoothed image layers obtained from the Gaussian blurring at multiple scales. The difference between adjacent scale Gaussian images in the Gaussian pyramid is then calculated according to the following formula to generate the Difference Gaussian Space (DoG): ， in, Represents the difference Gaussian space DoG with respect to Gaussian difference image layers, Indicates about Gaussian kernel function, This is a two-dimensional convolution operation. Color signal images for fusing feature information At coordinate point Pixel values; This is the scale multiplication factor, used to control the interval between adjacent scales; This serves as the baseline scale parameter, used to determine the granularity of feature detection; 3.2) In the Difference Gaussian (DoG) space, the position and scale of candidate keypoints are corrected at the sub-pixel level by fitting a three-dimensional quadratic function or Taylor expansion. The contrast of extreme points is calculated using the Hessian matrix, and unstable keypoints with low contrast are removed according to a preset contrast threshold to obtain stable keypoints. ， in, This is the offset of the keypoint, including corrections for spatial coordinates and scale; This is the contrast threshold, used to filter noise-sensitive points; for The inverse matrix, It is a Gaussian difference function. To stabilize the position of key points In x coordinate, Indicates the contrast at extreme points; Hessian matrix eigenvalue analysis is used to distinguish edge responses from corner responses. Based on the following formula, pseudo-stable keypoints on the edges are filtered out using the principal curvature ratio to retain features with stable geometric structures: ， in, The Hessian matrix is ​​composed of the second derivatives of the Gaussian difference image layers. for H The trace of the Hessian matrix represents the sum of the principal curvatures; for H The product of curvature, r The primary curvature ratio threshold, For the image in x The second derivative in the direction represents the rate of change of the gray value along the horizontal direction. and The images are respectively in x and y The mixed second derivative of the direction represents the joint change of grayscale value in the horizontal and vertical directions. For the image in y The second derivative of the direction represents the rate of change of the gray value along the vertical direction; Calculate the neighborhood gradient magnitude and directional gradient angle based on the surrounding region of the filtered and corrected stable keypoint: ， ， in, For the stable key points after filtering correction The neighborhood gradient magnitude, , , and These are the positions in the Gaussian smoothed image layer. , , and grayscale value, For the stable key points after filtering correction The directional gradient angle is used to construct a multi-directional histogram based on the neighborhood gradient magnitude and the directional gradient angle. Key points in multiple directions are generated by setting the main direction as the peak value of the histogram and the secondary peaks as greater than the main peak. 3.3) Rotate the neighborhood of the key point to the main direction, divide the 16×16 neighborhood into 4×4 sub-regions, count 8 directional buckets in each region, and concatenate them into a 128-dimensional vector according to the following formula, then normalize the vector to generate nodes in the graph structure data: ， in, For the first in graph structure data i The feature vector of each node For the stable key points after filtering correction The neighborhood gradient magnitude, For the first i The set of pixels in the sub-region of each node. for The pixel position in the image. For indicator functions, For the stable key points after filtering correction The direction gradient angle; Let i be the i-th directional bucket, which is one of eight angular intervals evenly divided within 360°, used to accumulate gradient energy within that angular range; the feature vectors of each node in the graph structure data are used as 128-dimensional SIFT descriptors, and the edge weights in the graph structure data are calculated according to the following formula to obtain the graph structure data: ， in, For nodes v i and v j Edge weights between them This is a coefficient for balancing spatial and semantic weights. For nodes v i coordinates For nodes v j coordinates σ s The standard deviation of the Gaussian kernel for spatial distance. For nodes v i 128-dimensional SIFT descriptor, For nodes v j 128-dimensional SIFT descriptor.

5. The motor fault diagnosis method based on color image fusion symmetrical point pattern according to claim 1, characterized in that, In step 4), when using their respective capsule graph network models to diagnose and classify the graph structure data of vibration and electromagnetic signals, the capsule graph network model dynamically routes the input graph structure data of vibration or electromagnetic signals to achieve cross-modal feature fusion and output the activated capsule vector according to the following formula: ， ， ， in, Let j be the activated capsule vector output by the graph capsule network model. Let be the weighted sum of the dynamic routing coefficients of the j-th graph capsule network model and the feature vectors of the primary capsules. Let be the dynamic routing coefficient from capsule i to capsule j. Let i be the primary capsule feature vector from capsule i to capsule j. Let be the coupling coefficient from capsule i to capsule j. Let be the coupling coefficient from capsule i to capsule k.

6. The motor fault diagnosis method based on color image fusion symmetrical point pattern according to claim 1, characterized in that, This also includes calculating the training metrics after each training round when training the graph capsule network model. If training metrics If the preset threshold is exceeded, the graph capsule network model is considered to have completed training; otherwise, iterative training needs to continue. The calculation obtains the training index. include: Adjust the initial diagnostic weights of each graph capsule network model according to the following formula: ， in, The initial diagnostic weights for the m-th graph capsule network model are... Let m be the activated capsule vector output by the m-th graph capsule network model. for The vector magnitude, The activated capsule vector is the output of the nth graph capsule network model. for The vector magnitude, For coefficients, The number of graph capsule network models; the entropy weight coefficients are calculated according to the following formula: ， in, Let m be the entropy weight coefficient of the m-th graph capsule network model. Let m be the information entropy of the m-th graph capsule network model. The information entropy of the nth graph capsule network model is... Number of fault categories The predicted probability distribution for each fault category; the training index is calculated according to the following formula. : ， in, The voting score for the k types of faults in the m-th graph capsule network model.

7. The motor fault diagnosis method based on color image fusion symmetrical point pattern according to claim 1, characterized in that, Step 1) includes training graph network models for vibration and electromagnetic signals respectively. Training the graph network models involves collecting vibration and electromagnetic signals of the motor under different operating conditions as fault diagnosis data. These different operating conditions include normal conditions and various fault conditions. The obtained fault diagnosis data includes the motor fault electromagnetic signal dataset shown in the following formula. : in, For the first i One motor fault signal, For fault labels, Electromagnetic signal dataset for motor faults The number of samples, T The number of sampling points. This is a dataset of electromagnetic fault signals for motors. This is a dataset for motor electromagnetic fault categories. And the motor fault vibration signal dataset shown in the following formula : in, For the first i Vibration signal of a motor fault. For fault labels, Data set of vibration signals for motor faults The number of samples.

8. A motor fault diagnosis system based on color image fusion symmetrical point pattern, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to execute the motor fault diagnosis method based on color image fusion symmetric point pattern as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the motor fault diagnosis method based on color image fusion symmetric point pattern as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the motor fault diagnosis method based on color image fusion symmetric point pattern as described in any one of claims 1 to 7.

Images