A dynamic asymmetric feature recursive graph construction method based on multi-modal signal fusion, a fault diagnosis method and system

Abstract

The application belongs to the technical field of intelligent operation and maintenance of electromechanical equipment and high-end equipment state monitoring, and particularly relates to a dynamic asymmetric feature recurrent graph construction method based on multi-modal signal fusion, a fault diagnosis method and a system. The fault diagnosis method comprises the following steps: extracting a sliding window and constructing a time domain and frequency domain feature vector for synchronously collected vibration and electromagnetic signals; realizing sensitivity evaluation and core feature variable screening of a primary feature vector through bidirectional disturbance and average influence value; generating a recurrent graph based on the core feature variable, extracting a single-sided triangular part of each recurrent graph, and performing space aggregation splicing to construct a dynamic asymmetric feature recurrent graph, which is input into a fault diagnosis Swin Transformer model for fault diagnosis. Through dynamic asymmetric feature recurrent graph construction and adaptive feature screening, the anti-noise robustness and diagnosis accuracy of multi-modal fusion fault diagnosis under variable speed conditions are effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent operation and maintenance of electromechanical equipment and condition monitoring technology of high-end equipment, specifically involving a method for constructing dynamic asymmetric feature recursive graphs based on multimodal signal fusion, a fault diagnosis method and system. Background Technology

[0002] Permanent magnet synchronous motors (PMSMs), with their outstanding advantages such as high power density, excellent operating efficiency, and high reliability, have become core drive components in high-end equipment fields such as new energy vehicles and industrial control. However, in actual service, PMSMs often need to operate continuously for long periods under complex conditions with frequent fluctuations in speed and load, inevitably leading to progressive fatigue damage in key components such as stators, rotors, and bearings. More challenging is the fact that these early, subtle faults are often highly concealed; if not identified and effectively intervened in time, they can easily evolve into a vicious chain of failures, causing unplanned downtime and significant economic losses, or even catastrophic safety accidents. Therefore, how to achieve high-precision diagnosis of early, fine-grained faults in PMSMs under harsh environments with variable speeds and complex operating conditions has become a key technical challenge of common concern to both academia and industry, possessing significant engineering application value and theoretical research significance.

[0003] Traditional motor condition monitoring and fault diagnosis methods often rely on the analysis of single mechanical vibration signals. However, under complex variable speed operating conditions, due to the complex mechanical transmission path, a single vibration signal is easily affected by strong background noise and vibration coupling interference from adjacent components, resulting in insufficient stability of feature extraction. More importantly, PMSMs are essentially strongly electromechanical coupled systems, and single mechanical vibration signals have low sensitivity to early electrical faults (such as stator inter-turn short circuits and rotor local demagnetization) and weak electromechanical coupling faults, making it difficult to comprehensively and accurately characterize the overall health status of the equipment. Therefore, introducing cross-modal feature fusion of electromagnetic signals and vibration signals to construct a multi-physics joint diagnostic system has become an important development trend in the field of intelligent operation and maintenance and health management of high-end equipment.

[0004] Currently, fault diagnosis technology based on multi-sensor signal fusion has made some progress, but when dealing with variable speed operating conditions, existing methods still have the following significant technical defects and limitations: (1) Lack of effective preservation of the underlying nonlinear dynamic physical meaning of multi-sensor signals. Existing fusion diagnostic methods, such as models based on conventional residual convolutional networks, usually employ simple data splicing or directly truncate one-dimensional time-series signals and rigidly reshape them into two-dimensional matrices to adapt to the standard input format of deep convolutional neural networks (CNNs). This kind of "pseudo-two-dimensionalization" processing method, which lacks physical mechanism support, severely disrupts the temporal continuity between adjacent sampling points and the nonlinear dynamic characteristics inherent in the underlying signal, resulting in the loss of a large amount of transient dynamic information with high diagnostic value under variable speed conditions.

[0005] (2) Lack of effective multi-source feature adaptive optimization mechanism. After extracting massive amounts of primary features in the time and frequency domains, existing methods usually directly input them into deep neural networks as a complete set for "black box" mapping and fusion. This not only retains a large amount of redundant information and noise components, significantly increasing computational overhead, but also easily leads to overfitting under varying working conditions and strong noise conditions, resulting in a serious decrease in the model's cross-working condition generalization ability and noise robustness.

[0006] (3) Existing two-dimensional feature images are mostly constructed using complete symmetrical recursive graphs or simple matrix stitching. Among them, the upper and lower triangular regions of the complete recursive graph contain redundant information with mirror symmetry, which is essentially a repetitive representation; while directly stacking or forcibly stitching multiple recursive graphs will not only significantly increase the image size and increase the computational cost of subsequent deep networks, but also make it difficult to achieve compactness and differentiated expression of multi-feature spatial layout while retaining the nonlinear recursive structure information of each feature variable.

[0007] In summary, how to scientifically and effectively combine and optimize the sensitive characteristics of vibration and electromagnetic signals based on nonlinear dynamic mechanisms, and accurately reconstruct and learn the deep dynamic evolution law of electromechanical fusion signals under variable speed and strong background noise conditions, so as to achieve high-precision and robust fault diagnosis of permanent magnet synchronous motors, is a key technical problem that urgently needs to be solved in this field. Summary of the Invention

[0008] To address the shortcomings of the existing technologies, this invention provides a method for constructing dynamic asymmetric feature recursive graphs based on multimodal signal fusion, a fault diagnosis method, and a system. This addresses the technical problems of existing fault diagnosis methods based on multisensor signal fusion, which often sever the underlying nonlinear dynamic features during feature fusion and struggle to effectively compress symmetric redundant information and achieve compact spatial aggregation of multiple features while preserving the nonlinear recursive structure of multimodal features.

[0009] The first aspect of this invention provides a method for constructing a dynamic asymmetric feature recursive graph based on multimodal signal fusion, comprising the following steps: S11. Using a preset step size, slide the time window, extract several feature variables within each time window, and normalize them. Then, concatenate all the extracted and normalized feature variables within the same time window to construct the primary feature vector corresponding to that time window. ,in, This represents the number of feature variables in the primary feature vector. and It is a positive integer; at the same time, the normalized feature values ​​of the same feature variable under each time window are arranged along the time axis to form a one-dimensional time series feature sequence of the feature variable; S12. Perform a bidirectional perturbation operation on the value of the first feature variable in the primary feature vector corresponding to the current time window to generate a bidirectional perturbation vector of the feature variable in the current time window, wherein the bidirectional perturbation vector includes an upward perturbation vector and a downward perturbation vector. S13. Convert the bidirectional perturbation vector of the first feature variable in the current time window into a dynamic asymmetric feature recursion graph, and obtain the dynamic asymmetric feature recursion graph corresponding to the bidirectional perturbation vector of the first feature variable in the current time window. S14. Input the dynamic asymmetric feature recursive graph corresponding to the bidirectional perturbation vector of the first feature variable in the current time window into the feature evaluation Swin Transformer model. The feature evaluation Swin Transformer model outputs the probability distribution vector corresponding to each perturbation vector and uses it as the prediction result of the bidirectional perturbation vector of the first feature variable in the current time window. The dynamic asymmetric feature recursive graph is a single-channel grayscale image. Before inputting it into the feature evaluation Swin Transformer model, the single-channel grayscale image needs to be copied and mapped to the three RGB channels to form a three-channel RGB image and used as the input of the feature evaluation Swin Transformer model. S15. Based on the prediction results of the bidirectional perturbation vector of the first feature variable in the current time window, calculate the average influence value of the feature variable. The calculation formula is as follows: , In the formula, For the first The average influence of each characteristic variable This represents the number of classes in the probability distribution vector. This represents the dimension index of the class in the probability distribution vector. For the first The predicted value of the j-th category by the down-difference vector of each feature variable in the current time window. For the first The predicted value of the j-th category by the up-adjustment perturbation vector of each feature variable in the current time window; S16. Repeat steps S12-S15 to obtain the average influence value of the remaining feature variables in the primary feature vector; S17. Based on the average influence value of each feature variable, Sort the feature variables in descending order and select the top [number] from the sorting table. These 10 characteristic variables are used as core characteristic variables to form the core feature vector, where... and It is a positive integer. , ; S18. Based on the one-dimensional time-series feature sequence of each feature variable obtained in step S11, the core feature vector is converted into the target dynamic asymmetric feature recursion graph.

[0010] Preferably, step S12 specifically includes: Keeping the values ​​of other feature variables unchanged in the current time window, the value of the first feature variable in the current time window is increased by a preset perturbation ratio to form the upward perturbation vector of the first feature variable in the current time window; Keeping the values ​​of other feature variables unchanged in the current time window, the value of the first feature variable in the current time window is reduced by a preset perturbation ratio to form the reduction perturbation vector of the first feature variable in the current time window; Step S13 is as follows: S131. Reconstruct the time series of the up-adjustment perturbation vector of the first feature variable in the current time window in the nonlinear phase space to generate a recursive graph corresponding to each feature variable in the up-adjustment perturbation vector. S132. Divide the recursive graph corresponding to each feature variable in the up-adjustment perturbation vector into two triangular regions along the main diagonal. The upper triangular region is the upper right side of the main diagonal, and the lower triangular region is the lower left side of the main diagonal. S133. According to the sorting position of each feature variable in the up-adjustment perturbation vector, extract one side of the triangular part. Specifically, extract the lower triangular part of the feature variable located in the odd position and extract the upper triangular part of the feature variable located in the even position. S134. According to the sorting position of each feature variable in the up-adjustment perturbation vector, the one-sided triangular parts extracted from two adjacent feature variables are spliced ​​together in the spatial dimension to generate several small squares. S135. Several small squares are arranged in a spatial order corresponding to their sorting positions and then aggregated and spliced ​​into a complete dynamic asymmetric feature recursion graph, which is the dynamic asymmetric feature recursion graph corresponding to the up-adjustment perturbation vector of the first feature variable in the current time window. S136. Repeat steps S131-S135 to obtain the dynamic asymmetric feature recursion graph corresponding to the down-adjustment perturbation vector of the first feature variable in the current time window. Step S18 is as follows: S181. Based on the one-dimensional time series feature sequence of each feature variable obtained in step S11, reconstruct the time series of each feature variable in the core feature vector in the nonlinear phase space to generate a recursive graph corresponding to each feature variable. S182. Divide the recursion graph corresponding to each feature variable into two triangular regions along the main diagonal. The upper right region of the main diagonal is the upper triangular part, and the lower left region of the main diagonal is the lower triangular part. S183. According to the sorting position of each feature variable in the core feature vector, extract one side of the triangular part. Specifically, extract the lower triangular part of the feature variable located in the odd-numbered position and extract the upper triangular part of the feature variable located in the even-numbered position. S184. According to the sorting position of each feature variable in the core feature vector, the one-sided triangular parts extracted from two adjacent feature variables are spliced ​​together in the spatial dimension to generate several small squares. S185. Several small squares are arranged in a spatial order corresponding to their sorting positions and then aggregated and spliced ​​together to form a complete dynamic asymmetric feature recursion graph, which is the target dynamic asymmetric feature recursion graph.

[0011] Preferably, step S131 specifically includes: S1311. Replace the value corresponding to the current time window in the one-dimensional time series feature sequence of the first feature variable with the perturbation value of the feature variable in the up-adjustment perturbation vector to form the one-dimensional time series feature sequence after the first feature variable is updated. S1312. Map and expand the one-dimensional time-series feature sequence after updating the first feature variable to a high-dimensional phase space; S1313. Calculate the Euclidean distance between any two state points in the high-dimensional phase space, and compare the Euclidean distance with a preset distance threshold. Based on the comparison result, determine whether there is a recursive relationship between the two state points in the high-dimensional phase space. Specifically: if the Euclidean distance between any two state points is less than the preset distance threshold, it is determined that there is a recursive relationship between the two state points in the high-dimensional phase space; otherwise, it is determined that there is no recursive relationship between the two state points in the high-dimensional phase space. S1314. Based on the judgment result of whether there is a recursive relationship between any two state points in the high-dimensional phase space, the recursive matrix is ​​obtained by nonlinear binarization through the Heaviside step function. Specifically, if there is a recursive relationship between any two state points in the high-dimensional phase space, the corresponding element in the recursive matrix is ​​assigned a value of 1; otherwise, the corresponding element in the recursive matrix is ​​assigned a value of 0. S1315. Convert the recursion matrix into a recursion graph to obtain the recursion graph corresponding to the first feature variable in the up-adjustment perturbation vector. S1316. Keeping the one-dimensional time-series feature sequences of the remaining feature variables in the up-adjustment perturbation vector unchanged, repeat steps S1312-S1315 to obtain the recursion graph corresponding to the remaining feature variables in the up-adjustment perturbation vector.

[0012] Preferably, the sorting position of each feature variable in the primary feature vector is a preset fixed sorting position, and the sorting position of the feature variables in each perturbation vector is consistent with the sorting position of the feature variables in the primary feature vector. The order of each feature variable in the core feature vector is determined by the magnitude of its average influence value, and the feature variables are arranged in descending order of average influence value; The recursion graph corresponding to each feature variable is a square image and is symmetrical along the main diagonal; Based on the average influence value of each feature variable, When sorting the feature variables in descending order, if there are two or more feature variables with the same average influence value, the original sorting order of these feature variables in the primary feature vector, i.e., the fixed sorting position preset in step S11, shall be used as the secondary sorting basis. The earlier the original sorting order is, the earlier the sorting order is, and the later the original sorting order is, the later the sorting order is.

[0013] Preferably, the feature evaluation Swin Transformer model is obtained by training the first Swin Transformer model offline. The first Swin Transformer model consists of an image patch embedding layer, a four-stage hierarchical feature extraction module, and a classification output layer, specifically: The image patch embedding layer consists of an image patch segmentation unit and a linear embedding unit connected in sequence. The image patch segmentation unit divides the input three-channel RGB image into non-overlapping image patches of 4×4 pixels. Each image patch contains 48 pixel values, which are flattened to form a 48-dimensional original feature vector. The linear embedding unit maps the 48-dimensional original feature vector of each image patch to a preset embedding dimension through a fully connected layer. The four-stage hierarchical feature extraction module consists of four cascaded feature extraction stages. Adjacent stages are connected by a block fusion layer to achieve downsampling. The number of attention heads in each stage is 3, 6, 12, and 24, respectively. The first, second, and fourth feature extraction stages each contain two Swing Transformer Blocks, while the third feature extraction stage contains six Swing Transformer Blocks. In each stage, the Swing Transformer Blocks are nested in pairs, alternating between window-based multi-head self-attention and shift-window-based multi-head self-attention. Two consecutive Swing Transformer Blocks form a module pair, i.e., the previous Swing Transformer Block uses window-based multi-head self-attention, and the next Swing Transformer Block uses shift-window-based multi-head self-attention. The internal structure of each Swing Transformer Block is as follows: first, it undergoes layer normalization; then, it performs multi-head self-attention calculation through either window-based or shift-window-based multi-head self-attention; after residual connection, it undergoes layer normalization again; then, it undergoes nonlinear transformation through a multilayer perceptron; and finally, it outputs through residual connection. The classification output layer consists of layer normalization, a global average pooling layer, a fully connected layer, and a Softmax activation function connected sequentially. Specifically: layer normalization standardizes the feature tensor along the channel dimension; the global average pooling layer compresses the feature tensor into a one-dimensional feature vector along the spatial dimension; the fully connected layer linearly maps the one-dimensional feature vector to the dimension of the number of fault categories, outputting a Logits vector of the corresponding dimension, where the Logits vector is the original score vector without Softmax activation normalization; and the Softmax activation function maps the Logits vector output by the fully connected layer into a probability distribution vector, where each dimension takes values ​​between 0 and 1 and sums to 1.

[0014] Preferably, the offline training process for the first Swin Transformer model is as follows: (1) Several dynamic asymmetric feature recursive maps generated by historical primary feature vectors are used as training samples and input into the first Swin Transformer model. The first Swin Transformer model outputs a probability distribution vector and uses the probability distribution vector as the prediction result. The dynamic asymmetric feature recursive map is a single-channel grayscale image. Before inputting it into the first Swin Transformer model, the single-channel grayscale image needs to be copied and mapped to the three RGB channels to form a three-channel RGB image and use it as the input of the first Swin Transformer model. (2) Calculate the loss value between the predicted result and the actual result of each historical primary feature vector based on the cross-entropy loss function. The calculation formula is as follows; , In the formula, The loss value between the actual result and the predicted result. The number of classes in the probability distribution vector; For the first The true value for each category is either 0 or 1; For the first The predicted values ​​for each category range from 0 to 1; (3) Calculate the gradient of each trainable parameter in the first Swing Transformer model with respect to the loss value using the backpropagation algorithm; (4) Update the model parameters using the Adam optimizer; (5) When a training round ends, that is, when all the dynamic asymmetric feature recursion graphs corresponding to the historical primary feature vectors have completed a complete forward and backward propagation, training stops and the feature evaluation SwinTransformer model is obtained. Furthermore, the dynamic asymmetric feature recursive map generated from the core feature vector, perturbation vector, or historical primary feature vector is a square image with a size of L×L pixels, where L is the side length of the dynamic asymmetric feature recursive map in the height or width direction. And N is a positive integer. or , or , This represents the side length of a single recursive graph within the perturbation vector or historical primary eigenvectors. The side length of a single recursive graph in the core feature vector; When the dynamic asymmetric feature recursive graph is generated from the perturbation vector or the historical primary feature vector... When the dynamic asymmetric feature recursive graph is generated from the core feature vector, When the dynamic asymmetric feature recursive graph generated from the core feature vector has the same size as the dynamic asymmetric feature recursive graph generated from the perturbation vector or historical primary feature vector, ,in, , , Pick and The least common multiple; when the dynamic asymmetric feature recursive graph generated by the core feature vector has a different size than the dynamic asymmetric feature recursive graph generated by the perturbation vector or historical primary feature vector. and .

[0015] Preferably, the dynamic asymmetric feature recursion graph generated from the core feature vector has the same size as the dynamic asymmetric feature recursion graph generated from the perturbation vector or the historical primary feature vector; The dynamic asymmetric feature recursion graph generated from the core feature vector, perturbation vector, or historical primary feature vectors has n small squares arranged in both the width and height directions, forming a total of n squares. There are 3 small squares, each with the same side length as a single recursive graph; When each feature variable in the core feature vector, perturbation vector, or historical primary feature vector is reconstructed in the nonlinear phase space, the length of the one-dimensional time-series feature sequence of each feature variable is T = 2N; after mapping and expanding the one-dimensional time-series feature sequence of each feature variable to the high-dimensional phase space, the number of state vectors obtained is... Where m is the preset embedding dimension in the high-dimensional phase space, This is a preset time delay parameter in the high-dimensional phase space; the number of state vectors in the high-dimensional phase space is equal to the side length of a single recursive graph, i.e., N = .

[0016] A second aspect of this invention provides a fault diagnosis method based on a dynamic asymmetric feature recursive graph using multimodal signal fusion, comprising the following steps: S21. Under variable speed conditions, collect the vibration signal sequence and electromagnetic signal sequence of the permanent magnet synchronous motor during operation, add timestamps to the collected data based on a unified clock reference, and store them synchronously in the database. S22. Extract the timestamped vibration signal sequence and electromagnetic signal sequence from the database, set the time window length and sliding step size, slide the time window along the time axis, and extract the vibration signal segment and electromagnetic signal segment corresponding to each time window from the vibration signal sequence and electromagnetic signal sequence respectively. Then, using the sampling time of the high sampling rate signal segment as the reference time axis, the low sampling rate signal segment is resampled using the cubic spline interpolation algorithm, so that the vibration signal segment and electromagnetic signal segment in the same time window form a sampling point correspondence on the time axis. S23. For each vibration signal segment and electromagnetic signal segment obtained in step S22 after time alignment, extract their time-domain statistical index and frequency-domain statistical index respectively, take each extracted statistical index as a feature variable, and normalize each feature variable. S24. Using the aforementioned method for constructing a dynamic asymmetric feature recursive graph based on multimodal signal fusion, a dynamic asymmetric feature recursive graph is constructed. S25. Copy and map the dynamic asymmetric feature recursive graph to the three RGB channels to form a three-channel RGB image and input it into the fault diagnosis Swing Transformer model. The fault diagnosis Swing Transformer model outputs the fault diagnosis results of the permanent magnet synchronous motor.

[0017] Preferably, the time-domain statistical indices of each vibration signal segment and electromagnetic signal segment include the mean, root mean square, standard deviation, peak-to-peak value, kurtosis, skewness, shape index, peak index, root mean square amplitude, impulse index, and margin index; and the frequency-domain statistical indices of each vibration signal segment and electromagnetic signal segment include the frequency center, mean square frequency, root mean square frequency, frequency variance, and root variance frequency. The time-domain statistical index of each vibration signal segment and electromagnetic signal segment is directly calculated from the original waveform of their respective signal segments, while the frequency-domain statistical index is calculated by performing a Fourier transform on their respective signal segments. The fault diagnosis Swin Transformer model is obtained by training the second Swin Transformer model offline. The network structure of the second Swin Transformer model is the same as that of the first Swin Transformer model, consisting of an image patch embedding layer, a four-stage hierarchical feature extraction module, and a classification output layer. The difference between the two lies in the final structure of the classification output layer. Specifically, the classification output layer of the first Swin Transformer model consists of a layer normalization layer, a global average pooling layer, a fully connected layer, and a Softmax activation function connected in sequence, stopping after outputting the probability distribution vector, and does not include an Argmax decision. The classification output layer of the second Swin Transformer model, based on the classification output layer of the first Swin Transformer model, adds an Argmax decision after the Softmax activation function. This decision is used to select the fault category corresponding to the index position with the highest probability value in the probability distribution vector as the final diagnostic output. The offline training process of the second Swin Transformer model is basically the same as that of the first Swin Transformer model. The difference is that: (1) The input sample is a dynamic asymmetric feature recursive graph generated by historical core feature vectors, rather than a dynamic asymmetric feature recursive graph generated by historical primary feature vectors; (2) The stopping condition is that the loss value converges to below the preset threshold or the training round reaches the preset upper limit, rather than stopping after completing one training round. In the actual online fault diagnosis phase, once the core feature variables are selected, they are continuously used under the given operating conditions without repeating the calculation of the average influence value of the feature variables and the selection of core feature variables. During online inference, it is only necessary to update the current value of the core feature vector based on the real-time acquired multimodal signals and construct a dynamic asymmetric feature recursion graph in real time according to the established rules. Inputting this graph into the fault diagnosis Swin Transformer model will yield the fault diagnosis result at the current moment. When the operating conditions change or the application scenario is switched, it is necessary to collect historical data of the new operating conditions to re-complete the feature evaluation Swin Transformer model training, feature variable average influence value calculation, core feature variable selection, and fault diagnosis Swin Transformer model training, so that the updated core feature vector, the spatial layout of the dynamic asymmetric feature recursion graph, and the fault diagnosis Swin Transformer model parameters are adapted to the new operating conditions.

[0018] A third aspect of the present invention provides a fault diagnosis system based on a dynamic asymmetric feature recursive graph using multimodal signal fusion, for executing a fault diagnosis method based on a dynamic asymmetric feature recursive graph using multimodal signal fusion, comprising: The multimodal signal acquisition and synchronous storage module is used to acquire vibration signal sequences and electromagnetic signal sequences of permanent magnet synchronous motors under variable speed conditions, and add timestamps to the acquired data based on a unified clock reference and synchronously store it in the database. The time alignment and preprocessing module is used to extract timestamped vibration signal sequences and electromagnetic signal sequences from the database. It sets the time window length and sliding step size, slides the time window along the time axis, and extracts the vibration signal segments and electromagnetic signal segments corresponding to each time window from the vibration signal sequence and electromagnetic signal sequence, respectively. Then, using the sampling time of the high sampling rate signal segment as the reference time axis, the low sampling rate signal segment is resampled using a cubic spline interpolation algorithm, so that the vibration signal segments and electromagnetic signal segments in the same time window form a sampling point correspondence on the time axis. The primary feature extraction module is used to extract time-domain and frequency-domain statistical indices from each vibration signal segment and electromagnetic signal segment after time alignment. Each extracted statistical index is used as a feature variable, and the feature variables are normalized. All feature variables extracted and normalized within the same time window are concatenated to construct the primary feature vector corresponding to that time window. At the same time, the normalized feature values ​​of the same feature variable in each time window are arranged along the time axis to form a one-dimensional time-series feature sequence of that feature variable. The dynamic asymmetric feature recursive graph construction module is used to construct a dynamic asymmetric feature recursive graph based on the aforementioned multimodal signal fusion-based dynamic asymmetric feature recursive graph construction method, which constructs a dynamic asymmetric feature recursive graph based on the primary feature vector and the one-dimensional temporal feature sequence of each feature variable in the primary feature vector. The fault diagnosis module is used to copy and map the dynamic asymmetric feature recursive graph to the three RGB channels to form a three-channel RGB image, which is then input into the fault diagnosis Swing Transformer model. The Swing Transformer model outputs the fault diagnosis results of the permanent magnet synchronous motor.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1. Starting from nonlinear dynamics, this invention scientifically combines and optimizes sensitive features in vibration and electromagnetic signals to accurately reconstruct and learn the deep dynamic evolution law of electromechanical fusion signals under variable speed and high noise conditions. This enables high-precision and robust fault diagnosis of permanent magnet synchronous motors, which has important theoretical value and practical significance for promoting the development of intelligent operation and maintenance technology for high-end equipment.

[0020] 2. This invention uses vibration signals and electromagnetic signals simultaneously to characterize the fault state, and integrates the dynamic response information of the mechanical side with the operating state information of the electromagnetic side. This overcomes the problems of incomplete fault information and susceptibility to noise interference under complex working conditions caused by a single signal source, making the sources of fault characteristics more comprehensive and providing a more sufficient foundation of state information for subsequent fault diagnosis.

[0021] 3. This invention extracts time-domain statistical indicators and frequency-domain statistical indicators from vibration signal segments and electromagnetic signal segments respectively, and performs contribution analysis on candidate feature variables through the average influence value evaluation method to screen core feature variables. This not only effectively reduces the impact of redundant features, weakly correlated features and noise-sensitive features on the diagnostic process, but also reduces feature dimensions and computational burden, and improves the discriminativeness of input features.

[0022] 4. This invention retains the one-sided triangular regions of the recursion graphs corresponding to different core feature variables, and concatenates two feature variables with adjacent sorting positions. This effectively reduces symmetrical redundancy in a single recursion graph and arranges the recursive structures of different feature variables in different spatial regions of the image, forming an asymmetric spatial layout. This layout helps the fault diagnosis Swin Transformer model learn the complementary differences and combination relationships between different feature regions, thereby improving the Swin Transformer model's ability to identify and generalize multimodal fusion features.

[0023] 5. This invention transforms multimodal statistical features into two-dimensional graph structure data suitable for deep learning models, enabling the originally discrete one-dimensional feature sequences to have a spatial structure representation. This not only facilitates the subsequent model in extracting local texture, regional differences, and overall structural features from the image level, but also improves the adaptability between feature utilization methods and deep vision models.

[0024] 6. The core feature variable usage strategy in this invention is dynamically adjusted according to changes in operating conditions: Under given operating conditions, this invention keeps the selected core feature variables fixed during the actual online fault diagnosis phase, only updating the current values ​​of the core feature variables based on real-time acquired multimodal signals, and generating the corresponding dynamic asymmetric feature recursion graph in real time according to predetermined rules. This avoids repeatedly performing feature contribution evaluation and core feature variable selection during online inference, reduces real-time computational overhead, and improves online diagnosis efficiency. When operating conditions change, the historical data of the new operating conditions are used to retrain the feature evaluation Swin Transformer model, feature contribution evaluation, core feature variable selection, and fault diagnosis Swin Transformer model, ensuring consistency between the updated core feature vector, the spatial layout of the dynamic asymmetric feature recursion graph, and the parameters of the fault diagnosis Swin Transformer model, thus ensuring the accuracy and generalization ability of the fault diagnosis Swin Transformer model under new operating conditions. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a recursion graph corresponding to a single feature variable in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the calculation of the average influence value in an embodiment of the present invention. Figure 3 This is a schematic diagram illustrating the construction of the target dynamic asymmetric feature recursive graph in an embodiment of the present invention; Figure 4 This is a flowchart illustrating the operation of multimodal parameter heterogeneous sensing and time synchronization storage in an embodiment of the present invention. Figure 5 This is a flowchart of the multimodal signal time alignment and FFT transform preprocessing in an embodiment of the present invention; Figure 6 This is a flowchart illustrating the initial feature vector construction process in an embodiment of the present invention. Figure 7 This is a schematic diagram illustrating the sorting of the average influence values ​​of feature variables in the primary feature vector and the extraction of core feature variables in an embodiment of the present invention. Figure 8 This is a flowchart illustrating the processing of the second Swing Transformer model in an embodiment of the present invention. Figure 9 This is a diagram illustrating the internal feature flow process of two consecutive Swing Transformer Blocks in an embodiment of the present invention. Figure 10 This is a schematic diagram of the permanent magnet synchronous motor fault diagnosis experimental platform in an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. The terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, unless otherwise explicitly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention in conjunction with the specific circumstances.

[0029] The first aspect of this invention provides a method for constructing a dynamic asymmetric feature recursive graph based on multimodal signal fusion, comprising the following steps: S11. Using a preset step size, slide the time window, extract several feature variables within each time window, and normalize them. Then, concatenate all the extracted and normalized feature variables within the same time window to construct the primary feature vector corresponding to that time window. ,in, This represents the number of feature variables in the primary feature vector. and It is a positive integer; at the same time, the normalized feature values ​​of the same feature variable under each time window are arranged along the time axis to form a one-dimensional time series feature sequence of the feature variable.

[0030] It should be noted that in this application, "sliding the time window with a preset step size" means that when the signal is segmented, the starting point of each time window moves a preset distance along the time axis relative to the starting point of the previous time window, and the preset distance is the preset step size.

[0031] In this embodiment of the application, the minimum-maximum normalization method is used for normalization processing, as shown in the following expression: , In the formula, For the first The normalized eigenvalues ​​of each feature variable within the t-th time window. For the first The original feature values ​​of each feature variable in the t-th time window. For the first The minimum value of each feature variable in the offline training samples. No. The maximum value of each feature variable in the offline training samples. A positive number set to prevent the denominator from being zero.

[0032] S12. Perform a bidirectional perturbation operation on the value of the first feature variable in the primary feature vector corresponding to the current time window to generate a bidirectional perturbation vector for that feature variable in the current time window. The bidirectional perturbation vector includes an upward perturbation vector and a downward perturbation vector, specifically: Keeping the values ​​of other feature variables unchanged in the current time window, the value of the first feature variable in the current time window is increased by a preset perturbation ratio to form the upward perturbation vector of the first feature variable in the current time window; Keeping the values ​​of other feature variables unchanged in the current time window, the value of the first feature variable in the current time window is reduced by a preset perturbation ratio to form the reduction perturbation vector of the first feature variable in the current time window.

[0033] In this embodiment, the preset perturbation ratio is 10%.

[0034] It's important to note that 10% is an empirically accepted golden ratio in neural network sensitivity analysis. When the perturbation ratio is set too small (e.g., 0.1%), the neural network's prediction results hardly change, leading to a lack of statistical validity in the calculations and making it impossible to effectively assess the contribution of each feature variable to the diagnostic results. Conversely, when the perturbation ratio is set too large (e.g., 50%), it completely destroys the original physical semantics of the feature, causing the model to make extreme erroneous judgments beyond the normal range. A 10% perturbation ratio is precisely sufficient to effectively detect the "weight elasticity" of the feature variable in the model's current decision space.

[0035] S13. Convert the bidirectional perturbation vector of the first feature variable in the current time window into a dynamic asymmetric feature recursive graph (DAFRP) to obtain the DAFRP graph corresponding to the bidirectional perturbation vector of the first feature variable in the current time window.

[0036] In this application, step S13 specifically includes: S131. Reconstruct the time series of the up-adjustment perturbation vector of the first feature variable in the current time window in the nonlinear phase space, generating a recursive graph (RP) corresponding to each feature variable in the up-adjustment perturbation vector, specifically: S1311. Replace the value corresponding to the current time window in the one-dimensional time series feature sequence of the first feature variable with the perturbation value of the feature variable in the upward perturbation vector to form the one-dimensional time series feature sequence after the first feature variable is updated.

[0037] It should be noted that, for ease of expression, in this application, "Dynamic Asymmetric Feature Recursive Graph (DAFRP)" is abbreviated as "DAFRP graph," and "Recursive Graph (RP)" is abbreviated as "RP graph." The specific meanings of these abbreviations in the context are consistent with the definitions in this paragraph and will not be repeated.

[0038] S1312. Map and expand the one-dimensional time-series feature sequence after updating the first feature variable to a high-dimensional phase space.

[0039] S1313. Calculate the Euclidean distance between any two state points in the high-dimensional phase space, and compare the Euclidean distance with a preset distance threshold. Based on the comparison result, determine whether there is a recursive relationship between the two state points in the high-dimensional phase space. Specifically: if the Euclidean distance between any two state points is less than the preset distance threshold, it is determined that there is a recursive relationship between the two state points in the high-dimensional phase space; otherwise, it is determined that there is no recursive relationship between the two state points in the high-dimensional phase space.

[0040] S1314. Based on the judgment result of whether there is a recursive relationship between any two state points in the high-dimensional phase space, the recursive matrix is ​​obtained by nonlinear binarization through the Heaviside step function. Specifically, if there is a recursive relationship between any two state points in the high-dimensional phase space, the corresponding element in the recursive matrix is ​​assigned a value of 1; otherwise, the corresponding element in the recursive matrix is ​​assigned a value of 0.

[0041] S1315. Convert the recursive matrix into an RP graph to obtain the RP graph corresponding to the first feature variable in the up-adjusted perturbation vector.

[0042] S1316. Keeping the one-dimensional time-series feature sequences of the remaining feature variables in the up-adjusted perturbation vector unchanged, repeat steps S1312-S1315 to obtain the RP map corresponding to the remaining feature variables in the up-adjusted perturbation vector.

[0043] It should be noted that in any perturbation vector, only the current window value in the one-dimensional time series feature sequence of the currently perturbed feature variable is replaced with the perturbation value, while the one-dimensional time series feature sequences of the other feature variables remain unchanged, that is, they are the same as the one-dimensional time series feature sequences of the corresponding feature variables in step S11.

[0044] In this embodiment of the application, the RP graph corresponding to a single feature variable is as follows: Figure 1 As shown.

[0045] S132. Divide the RP map corresponding to each feature variable in the up-adjustment perturbation vector into two triangular regions along the main diagonal. The upper right region of the main diagonal is the upper triangular part, and the lower left region of the main diagonal is the lower triangular part.

[0046] In this application, the RP image corresponding to each feature variable is a square image and is symmetrical along the main diagonal.

[0047] It should be noted that in this application, "main diagonal" refers to the diagonal line from the top left vertex to the bottom right vertex in the RP diagram.

[0048] S133. According to the sorting position of each feature variable in the up-adjustment perturbation vector, extract one side of the triangular part. Specifically, extract the lower triangular part of the feature variable located in the odd position and extract the upper triangular part of the feature variable located in the even position.

[0049] S134. According to the sorting position of each feature variable in the up-adjustment perturbation vector, the one-sided triangular parts extracted from two adjacent feature variables are spliced ​​together in the spatial dimension to generate several small squares.

[0050] In this embodiment of the application, step S134 specifically involves: joining the lower triangular portion of the first feature variable with the upper triangular portion of the second feature variable to form the first small square; joining the lower triangular portion of the third feature variable with the upper triangular portion of the fourth feature variable to form the second small square; joining the lower triangular portion of the fifth feature variable with the upper triangular portion of the sixth feature variable to form the third small square; and so on, in sequence, to generate a total of several small squares.

[0051] S135. Arrange several small squares in the spatial order corresponding to their sorting positions, and then aggregate and stitch them together to form a complete DAFRP map, which is the DAFRP map corresponding to the up-adjustment perturbation vector of the first feature variable in the current time window.

[0052] S136. Repeat steps S131-S135 to obtain the DAFRP plot corresponding to the down-adjustment perturbation vector of the first feature variable in the current time window.

[0053] In this application, the sorting position of each feature variable in the primary feature vector is a preset fixed sorting position, and the sorting position of the feature variables in each perturbation vector is consistent with the sorting position of the feature variables in the primary feature vector.

[0054] It should be noted that in the primary feature vector and the perturbation vector generated from it, the sorting position of each feature variable is a preset fixed sorting position, which is only used to identify the relative position of each feature variable and does not reflect the importance or sensitivity of the feature.

[0055] Taking the above-mentioned disturbance vector as an example, suppose the disturbance vector contains 8 feature variables, and each feature variable is denoted as feature variable 1 to feature variable 8 according to its ranking position in the primary feature vector. The lower triangular portion of the RP plot corresponding to the feature variables with odd ranking positions (1st, 3rd, 5th, 7th) is taken, and the upper triangular portion of the RP plot corresponding to the feature variables with even ranking positions (2nd, 4th, 6th, 8th) is taken. Then, the lower triangular portion of the first feature variable is joined with the upper triangular portion of the second feature variable to form the first small square; the lower triangular portion of the third feature variable is joined with the upper triangular portion of the fourth feature variable to form the second small square; the lower triangular portion of the fifth feature variable is joined with the upper triangular portion of the sixth feature variable to form the third small square; and the lower triangular portion of the seventh feature variable is joined with the upper triangular portion of the eighth feature variable to form the fourth small square. Finally, the four small squares are arranged in ascending order of their numbers, from left to right and from top to bottom, and then clustered together in a 2×2 grid to generate the DAFRP diagram corresponding to the up-adjustment perturbation vector.

[0056] S14. Input the DAFRP map corresponding to the bidirectional perturbation vector of the first feature variable in the current time window into the feature evaluation Swin Transformer model. The feature evaluation Swin Transformer model outputs the probability distribution vector corresponding to each perturbation vector and uses it as the prediction result of the bidirectional perturbation vector of the first feature variable in the current time window. The DAFRP map is a single-channel grayscale image. Before inputting it into the feature evaluation Swin Transformer model, the single-channel grayscale image needs to be copied and mapped to the three RGB channels to form a three-channel RGB image and used as the input of the feature evaluation Swin Transformer model.

[0057] In this application, the feature evaluation Swin Transformer model is obtained by offline training of the first Swin Transformer model. The first Swin Transformer model consists of an image patch embedding layer, a four-stage hierarchical feature extraction module, and a classification output layer, specifically: The image patch embedding layer consists of an image patch segmentation unit and a linear embedding unit connected in sequence. The image patch segmentation unit divides the input three-channel RGB image into non-overlapping image patches of 4×4 pixels. Each image patch contains 48 pixel values, which are flattened to form a 48-dimensional original feature vector. The linear embedding unit maps the 48-dimensional original feature vector of each image patch to a preset embedding dimension through a fully connected layer.

[0058] The four-stage hierarchical feature extraction module consists of four cascaded feature extraction stages. Adjacent stages are connected by a block fusion layer to achieve downsampling. The number of attention heads in each stage is 3, 6, 12, and 24, respectively. The first, second, and fourth feature extraction stages each contain two Swin Transformer Blocks, while the third feature extraction stage contains six Swin Transformer Blocks. In each stage, the Swin Transformer Blocks are nested in pairs, alternating between window-based multi-head self-attention (W-MSA) and shift-window-based multi-head self-attention (SW-MSA). Two consecutive Swin Transformer Blocks form a module pair, i.e., the previous Swin Transformer Block uses window-based multi-head self-attention, and the next Swin Transformer Block uses shift-window-based multi-head self-attention. The internal structure of each Swin Transformer Block is as follows: it first undergoes layer normalization, then multi-head self-attention calculation is performed using either window-based or shift-window-based multi-head self-attention, followed by residual connection and layer normalization again, then nonlinear transformation is performed by a multilayer perceptron, and finally output is achieved through residual connection.

[0059] The classification output layer consists of layer normalization, a global average pooling layer, a fully connected layer, and a Softmax activation function connected sequentially. Specifically: layer normalization standardizes the feature tensor along the channel dimension; the global average pooling layer compresses the feature tensor into a one-dimensional feature vector along the spatial dimension; the fully connected layer linearly maps the one-dimensional feature vector to the dimension of the number of fault categories, outputting a Logits vector with the same dimension as the number of fault categories, where the Logits vector is the original score vector without Softmax activation function normalization; and the Softmax activation function maps the Logits vector output by the fully connected layer into a probability distribution vector, where each dimension takes values ​​between 0 and 1 and sums to 1.

[0060] In this embodiment of the application, when performing multi-head self-attention calculation based on window-based multi-head self-attention or multi-head self-attention based on shifted window, a learnable relative position bias is added to the attention score. This bias is obtained from a preset learnable parameter table based on the relative position between different image blocks within the same window.

[0061] In this embodiment, the internal feature transfer process of two consecutive Swing Transformer Blocks is shown in the following formula: , , , , In the formula, For the index of the Swing Transformer Block, For window-based multi-head self-attention, For layer normalization, It is a multilayer perceptron. For multi-head self-attention based on shift window, For the first The intermediate feature tensors in each Swing Transformer Block after layer normalization and W-MSA processing For the first Intermediate feature tensors in each Swing Transformer Block after layer normalization and SW-MSA processing; , , For the first , , The output feature tensor of the first Swing Transformer Block after processing by a multilayer perceptron, i.e., the first... , , The input feature tensor of each Swing Transformer Block.

[0062] In this embodiment of the application, the preset embedding dimension in the linear embedding unit is 96.

[0063] In this application, the offline training process for the first Swin Transformer model is as follows: (1) Several DAFRP images generated from historical primary feature vectors are used as training samples and input into the first SwinTransformer model. The first SwinTransformer model outputs a probability distribution vector and uses the probability distribution vector as the prediction result. The DAFRP image is a single-channel grayscale image. Before inputting it into the first SwinTransformer model, the single-channel grayscale image needs to be copied and mapped to the three RGB channels to form a three-channel RGB image and used as the input of the first SwinTransformer model.

[0064] (2) Calculate the loss value between the predicted result and the actual result of each historical primary feature vector based on the cross-entropy loss function. The calculation formula is as follows; , In the formula, The loss value between the actual result and the predicted result. The number of classes in the probability distribution vector. This is the dimension index of the category in the probability distribution vector; For the first The true value for each category is either 0 or 1; For the first The predicted values ​​for each category range from 0 to 1.

[0065] (3) Calculate the gradient of each trainable parameter with respect to the loss value in the first Swing Transformer model using the backpropagation algorithm.

[0066] (4) Update the model parameters using the Adam optimizer.

[0067] (5) When a training round ends, that is, when all the DAFRP graphs corresponding to the historical primary feature vectors have completed a complete forward and backward propagation, training is stopped and the feature evaluation Swin Transformer model is obtained.

[0068] It should be noted that the generation of the DAFRP graph is accomplished by both vector data and time-series data, where the time-series data is the one-dimensional time-series feature sequence of each feature variable in the vector. For the sake of brevity, when referring to "DAFRP graph generated from a certain vector," the complete meaning includes both the vector itself and the one-dimensional time-series feature sequence of each feature variable in the vector.

[0069] In this embodiment, the DAFRP map generated from the historical primary feature vectors and the DAFRP map generated from the perturbation vectors have the same size. This is because the perturbation vectors are obtained by up- or down-adjusting each feature variable based on the historical primary feature vectors, and both contain the same number of feature variables. ( Since the values ​​are positive integers and the order of the feature variables remains unchanged, the DAFRP maps generated according to the same geometric splicing rules will naturally have the same size.

[0070] S15. Based on the prediction results of the bidirectional perturbation vector of the first feature variable in the current time window, calculate the average influence value of the feature variable. The calculation formula is as follows: , In the formula, For the first The average influence of each characteristic variable The number of classes in the probability distribution vector. For the first The predicted value of the j-th category by the down-difference vector of each feature variable in the current time window. For the first The predicted value of the j-th category by the up-adjustment perturbation vector of each feature variable in the current time window.

[0071] This application applies bidirectional small perturbations to each feature variable in the primary feature vector, specifically by increasing and decreasing the normalized values ​​of each feature variable by 10% and 10% respectively, to construct two sets of perturbation samples: "feature up-adjustment" and "feature down-adjustment". Subsequently, the two sets of perturbation samples are input into the feature evaluation Swin Transformer model to obtain the prediction results corresponding to each perturbation sample. The average difference between the prediction results for the same feature variable when it is up-adjusted and down-adjusted is calculated to accurately obtain the average impact value of that feature variable, thereby achieving a quantitative evaluation of the sensitivity of each feature variable in the primary feature vector. In this embodiment, the average impact value calculation process is as follows: Figure 2 As shown.

[0072] It should be noted that the magnitude of the average impact value can intuitively reflect the degree of influence of each feature variable on the diagnostic results. Specifically, the larger the average impact value, the higher the sensitivity of that feature variable to state classification, and the stronger its dominant role in decision-making.

[0073] To ensure the accuracy of sensitivity assessment for each feature variable, the first Swin Transformer model in this application is trained using a DAFRP graph generated from undisturbed primary feature vectors to establish a baseline mapping relationship between feature variables and fault categories. After one training epoch, training is stopped, and the DAFRP graph generated from the perturbation vectors is input into the trained first Swin Transformer model. This allows the feature assessment Swin Transformer model to obtain the average influence value of each feature variable, thereby eliminating the interference of model parameter updates on sensitivity assessment and ensuring that output changes originate solely from feature perturbations.

[0074] It should be noted that the first Swin Transformer model only requires one training epoch during offline training. The purpose of this epoch is to enable the model to have effective initial discrimination ability and provide a stable forward mapping benchmark for subsequent perturbation samples, rather than to pursue classification accuracy. Therefore, it does not need to be trained to convergence.

[0075] S16. Repeat steps S12-S15 to obtain the average influence value of the remaining feature variables in the primary feature vector.

[0076] S17. Based on the average influence value of each feature variable, Sort the feature variables in descending order and select the top [number] from the sorting table. These 10 characteristic variables are used as core characteristic variables to form the core feature vector, where... and It is a positive integer. , .

[0077] Preferably, based on the average influence value of each feature variable, the... When sorting the feature variables in descending order, if there are two or more feature variables with the same average influence value, the original sorting order of these feature variables in the primary feature vector, i.e., the fixed sorting position preset in step S11, shall be used as the secondary sorting basis. The earlier the original sorting order is, the earlier the sorting order is, and the later the original sorting order is, the later the sorting order is.

[0078] For example, if there are 32 feature variables in the primary feature vector, and the first 8 need to be selected as core feature variables, and if the average influence values ​​of the 3rd and 6th feature variables are equal, while the average influence values ​​of the other feature variables are different, then when sorting all feature variables in descending order, the 3rd and 6th feature variables are not placed side by side, but rather their original order in the primary feature vector is used to determine their order. That is, the 3rd feature variable is placed before the 6th feature variable, thus forming a complete descending sort table, and the first 8 feature variables are selected as core feature variables accordingly.

[0079] It should be noted that this application is based on the average influence value of each feature variable. Sort the feature variables in descending order and select the top [number] from the sorting table. The core feature vector is composed of several feature variables. This process not only realizes the transformation of the feature dimension from the initial... Towards The adaptive dynamic dimensionality reduction not only prioritizes the retention of feature variables that contribute significantly to fault diagnosis, but also effectively eliminates redundant interference and weakly related information, thus obtaining core feature vectors with stronger representation capabilities and more concise information.

[0080] S18. Based on the one-dimensional time-series feature sequences of each feature variable obtained in step S11, the core feature vector is converted into the target DAFRP map, specifically as follows: S181. Based on the one-dimensional time-series feature sequences of each feature variable obtained in step S11, the time series of each feature variable in the core feature vector is reconstructed in the nonlinear phase space to generate the RP diagram corresponding to each feature variable.

[0081] S182. Divide the RP graph corresponding to each feature variable into two triangular regions along the main diagonal. The upper right region of the main diagonal is the upper triangular part, and the lower left region of the main diagonal is the lower triangular part.

[0082] S183. According to the sorting position of each feature variable in the core feature vector, extract one side of the triangular part. Specifically, extract the lower triangular part of the feature variable located in the odd position and extract the upper triangular part of the feature variable located in the even position.

[0083] S184. According to the sorting position of each feature variable in the core feature vector, the one-sided triangular parts extracted from two adjacent feature variables are spliced ​​together in the spatial dimension to generate several small squares.

[0084] S185. Arrange several small squares in a spatial order corresponding to their sorting positions, and then combine them into a complete DAFRP diagram, which is the target DAFRP diagram.

[0085] It should be noted that in step S181 of this application, when each feature variable in the core feature vector is reconstructed in the nonlinear phase space, the one-dimensional time series feature sequence of each feature variable is the same as the one-dimensional time series feature sequence of the corresponding feature variable in step S11. Both are the original one-dimensional time series feature sequences without perturbation or replacement; the remaining operations are the same as in step S131.

[0086] In this embodiment of the application, the construction diagram of the target DAFRP graph is as follows: Figure 3 As shown.

[0087] In this application, the order of each feature variable in the core feature vector is determined according to the magnitude of its average influence value, and the feature variables are arranged in descending order of average influence value.

[0088] It should be noted that in the core feature vector, the sorting position of each feature variable is re-determined according to the magnitude of the average influence value, and arranged in descending order of the average influence value. This sorting position can reflect the sensitivity of each feature variable to the fault diagnosis result.

[0089] Preferably, the DAFRP images generated from the core feature vector, perturbation vector, or historical primary feature vector are all square images with a size of L×L pixels, where L is the side length of the DAFRP image in the height or width direction. And N is a positive integer. or , or , This represents the side length of a single RP image within the perturbation vector or historical primary feature vector. This represents the side length of a single RP image within the core feature vector.

[0090] It should be noted that the side length of a single RP image refers to the number of pixels or state points in the width or height direction of the RP image generated by a single feature variable.

[0091] When the DAFRP graph is generated from perturbation vectors or historical primary eigenvectors When the DAFRP graph is generated from the core feature vectors, When the DAFRP map generated from the core feature vectors has the same size as the DAFRP map generated from the perturbation vectors or historical primary feature vectors, ,in, , , Pick and The least common multiple; when the DAFRP map generated from the core feature vector is not the same size as the DAFRP map generated from the perturbation vector or historical primary feature vectors. and .

[0092] For example, if the perturbation vector has 32 feature variables and the core feature vector has 8 feature variables, then... , .when When the value is 1024 (which satisfies the condition of being a common multiple of both 2 and 4), , All are positive integers.

[0093] Preferably, the DAFRP map generated from the core feature vector, perturbation vector, or historical primary feature vector has n small squares arranged in both the width and height directions, forming a total of n squares. Each small square has the same side length as a single RP image.

[0094] Preferably, when each feature variable in the core feature vector, perturbation vector, or historical primary feature vector is reconstructed in the nonlinear phase space, the length of the one-dimensional time-series feature sequence of each feature variable is T=2N; after the one-dimensional time-series feature sequence of each feature variable is mapped and expanded to the high-dimensional phase space, the number of state vectors obtained is... Where m is the preset embedding dimension in the high-dimensional phase space, This is a preset time delay parameter in the high-dimensional phase space; the number of state vectors in the high-dimensional phase space is equal to the side length of a single RP graph, i.e., N = .

[0095] In this embodiment, the DAFRP map generated from the core feature vector and the DAFRP map generated from the perturbation vector or historical primary feature vector have the same size, which is L×L pixels. It should be understood that having the same size for all three is a specific setting in this embodiment, and in other embodiments, different sizes can be set according to actual needs.

[0096] It should be noted that the DAFRP image generated from the core feature vector and the DAFRP image generated from the perturbation vector or historical primary feature vectors can have the same final output image size, despite the different number of feature variables involved in the generation. This is because the final output size of the DAFRP image is determined by a preset geometric stitching rule; that is, when the number of features involved in generating the DAFRP image is... When n is a positive integer, the side length of the RP graph corresponding to each feature variable is L / n (the DAFRP graph is a square image, and L is the side length of the DAFRP graph). Zhang RP image was obtained by taking half of it and splicing it in pairs. The algorithm uses small squares, arranged in an n×n grid, to form a large image. This rule ensures that regardless of the value of n, the side length of the RP image corresponding to a single feature variable can be adjusted synchronously to ensure that the DAFRP images generated by different numbers of feature variables are of the same size.

[0097] In this application, during data collection in step S11, the data is collected and stored according to the maximum length of the one-dimensional time-series feature sequence required by the feature variables in each application scenario, to ensure that there is sufficient data length for subsequent processing in any scenario. In subsequent processing, according to the length required for the current scenario, a sequence segment of the corresponding length is extracted from the collected complete sequence for phase space reconstruction and RP graph generation. The extraction method is as follows: starting from the sampling point at the latest time (i.e., the end of the sequence), the required number of sampling points are continuously extracted along the time axis backward to ensure that the extracted sequence segment can reflect the latest state of the device operation, while maintaining the temporal continuity and consistency required for phase space reconstruction.

[0098] For example, the core feature vector contains There are characteristic variables, and the perturbation vector contains _____. The dimensions of the DAFRP map generated from the core feature vector and the DAFRP map generated from the perturbation vector are given by the following features: [Number of features]. When all values ​​are 1024, the side length of the RP graph corresponding to each feature variable in the core feature vector is... The side length of the RP graph corresponding to each feature variable in the perturbation vector. Therefore, it can be seen that when each feature variable in the core feature vector is reconstructed in the nonlinear phase space, the length of its one-dimensional time-series feature sequence is... The length of the one-dimensional time-series feature sequence of each feature variable in the perturbation vector when reconstructing the time series in the nonlinear phase space. Therefore, when collecting data in step S11, the larger of the two values, namely 1024, will be used for collection and storage; in subsequent applications, sequence segments of corresponding lengths will be extracted according to the needs of different scenarios.

[0099] A second aspect of this invention provides a fault diagnosis method based on a dynamic asymmetric feature recursive graph using multimodal signal fusion, comprising the following steps: S21. Under variable speed conditions, collect the vibration signal sequence and electromagnetic signal sequence of the permanent magnet synchronous motor during operation, add timestamps to the collected data based on a unified clock reference, and store them synchronously in the database.

[0100] In this embodiment of the application, the variable speed operating condition includes four variable speed modes: acceleration, deceleration, acceleration-deceleration, and deceleration-acceleration.

[0101] In this embodiment of the application, under variable speed conditions, the state data of the permanent magnet synchronous motor during operation are collected by independently deployed heterogeneous sensors. Specifically, the vibration signal sequence of the motor is collected by a mechanical vibration sensor, and the electromagnetic signal sequence is collected synchronously by an electromagnetic sensor.

[0102] Because asynchronous errors exist between heterogeneous sensors during data acquisition, this embodiment employs a global clock synchronization strategy to eliminate these errors. Specifically: First, both the mechanical vibration sensor and the electromagnetic sensor are equipped with GPS / BeiDou timing modules to obtain a high-precision, unified global clock reference. The electromagnetic sensor includes a receiving antenna and an electromagnetic acquisition device. The receiving antenna is used to pick up spatial electromagnetic radiation signals, and the electromagnetic acquisition device is used to acquire and extract values ​​from the signals output by the receiving antenna. Second, based on this global clock reference, nanosecond-level high-precision time is added to each frame of data for both the vibration and electromagnetic signals during the underlying hardware sampling process. Finally, the multimodal raw time-series data with absolute timestamps is stored in a pre-built MySQL database to form a raw dataset of historical and real-time PMSM operating status. The process is as follows: Figure 4 As shown.

[0103] S22. Extract the timestamped vibration signal sequence and electromagnetic signal sequence from the database, set the time window length and sliding step size, slide the time window along the time axis, and extract the vibration signal segment and electromagnetic signal segment corresponding to each time window from the vibration signal sequence and electromagnetic signal sequence respectively. Then, using the sampling time of the high sampling rate signal segment as the reference time axis, the low sampling rate signal segment is resampled using the cubic spline interpolation algorithm, so that the vibration signal segment and electromagnetic signal segment in the same time window form a sampling point correspondence on the time axis.

[0104] It should be noted that in the embodiments of this application, the sampling frequency of the same type of signal segment is the same in different time windows.

[0105] S23. For each vibration signal segment and electromagnetic signal segment obtained in step S22 after time alignment, extract their time-domain statistical index and frequency-domain statistical index respectively, take each extracted statistical index as a feature variable, and normalize each feature variable.

[0106] Multimodal signals suffer from difficulties in data fusion due to sampling rate mismatch. Therefore, this application provides a time-stamp-based temporal alignment and feature extraction method to address this problem, specifically as follows: Figure 5 , Figure 6 As shown. First, timestamped vibration and electromagnetic signals are extracted from the MySQL database. The time window length and sliding step are set, and the time window is slid along the time axis to extract vibration and electromagnetic signal segments corresponding to each time window from the vibration and electromagnetic signal sequences, respectively. Second, the sampling rates of the vibration signals are compared. With electromagnetic signal sampling rate Adaptively using the high-sampling-rate signal as a reference, cubic spline interpolation resampling compensation is performed on the low-sampling-rate signal, specifically: when > At that time, using the vibration signal as a reference, the electromagnetic signal is resampled using cubic spline interpolation; when > At that time, using the electromagnetic signal as a reference, the vibration signal is resampled using cubic spline interpolation. After the above processing, the two signals are made to form a corresponding sampling point relationship on the time axis within the same time window, thereby eliminating timing misalignment. Based on this, the vibration time-domain signal and the electromagnetic time-domain signal are output separately, and the vibration frequency-domain signal and the electromagnetic frequency-domain signal are obtained by Fast Fourier Transform (FFT). Finally, the aligned vibration and electromagnetic dual-mode time-domain-frequency-domain signals are output, providing accurate and synchronized underlying data support for subsequent multi-dimensional feature fusion analysis.

[0107] In this application, the time-domain statistical indices of each vibration signal segment and electromagnetic signal segment include the mean, root mean square, standard deviation, peak-to-peak value, kurtosis, skewness, shape index, peak index, root mean square amplitude, impulse index, and margin index. The frequency-domain statistical indices of each vibration signal segment and electromagnetic signal segment include the frequency center, mean square frequency, root mean square frequency, frequency variance, and root variance frequency.

[0108] It should be noted that time-domain analysis and frequency-domain analysis are two core methodologies in signal processing, each with its own emphasis and complementing each other. Time-domain analysis directly focuses on the change of signal amplitude over time, effectively reflecting the strength and periodicity of fault impacts. It boasts advantages such as simple calculation, high real-time performance, and sensitivity to impact-type faults. However, its limitations include susceptibility to noise interference and the inability to directly obtain frequency component information. Frequency-domain analysis, through Fourier transform, reveals the energy distribution of the signal at different frequencies, clearly identifying fault characteristic frequencies such as inner and outer rings and their harmonics, exhibiting strong noise resistance. However, it relies on the assumption of signal stationarity and completely loses dynamic information in the time dimension. In general, the time domain emphasizes waveform statistics, while the frequency domain emphasizes spectral energy; combining the two provides a comprehensive characterization of the equipment's operating status. Based on this, this application constructs a fusion diagnostic analysis system based on the multi-dimensional time-domain and frequency-domain characteristic data of permanent magnet synchronous motors.

[0109] In this embodiment, the time-domain statistical index of each vibration signal segment and electromagnetic signal segment is directly calculated from the original waveform of their respective signal segments, while the frequency-domain statistical index is calculated by performing a Fourier transform on their respective signal segments.

[0110] S24. A dynamic asymmetric feature recursive graph construction method based on multimodal signal fusion is adopted to construct the DAFRP graph.

[0111] like Figure 6 As shown in the embodiment of this application, the time-domain statistical indicators and frequency-domain statistical indicators of each vibration signal segment and electromagnetic signal segment are concatenated into vectors to form a primary feature vector containing 32 feature variables. Each signal segment includes 11 time-domain statistical indicators and 5 frequency-domain statistical indicators.

[0112] In this embodiment, when the core feature variables are reconstructed in a nonlinear phase space, a preset time delay parameter is used in the high-dimensional phase space. .

[0113] like Figure 7 As shown, in this embodiment of the application, after processing by the above-described method for constructing a dynamic asymmetric feature recursive graph through multimodal signal fusion, the 32 feature variables in the primary feature vector are sorted from largest to smallest according to their average influence value, and the top 8 feature variables are selected from the sorting table as core feature variables. When the size of the DAFRP graph is set to 1024×1024 pixels, the side length of the RP graph generated by each core feature variable is 512 pixels.

[0114] After screening in step S24, the number of core feature variables That is, the eight RP maps generated from the eight core feature variables are halved and then paired to form four small squares, which are then arranged in a 2×2 grid to form the target DAFRP map. Since each small square is formed by piecing together the triangular parts of two RP maps, the size of each small square is equal to the side length of a complete RP map. Therefore, in the embodiments of this application, when the core feature variables are reconstructed in the nonlinear phase space for time series, the preset embedding dimension in the high-dimensional phase space is determined. dimension.

[0115] S25. Copy and map the DAFRP image to the three RGB channels to form a three-channel RGB image and input it into the fault diagnosis Swing Transformer model. The fault diagnosis Swing Transformer model outputs the fault diagnosis results of the permanent magnet synchronous motor.

[0116] In this application, the fault diagnosis Swin Transformer model is obtained by offline training of the second Swin Transformer model. The network structure of the second Swin Transformer model is the same as that of the first Swin Transformer model, consisting of an image patch embedding layer, a four-stage hierarchical feature extraction module, and a classification output layer. The difference between the two lies in the final structure of the classification output layer. Specifically, the classification output layer of the first Swin Transformer model consists of a layer normalization layer, a global average pooling layer, a fully connected layer, and a Softmax activation function connected in sequence, stopping after outputting the probability distribution vector, and does not include an Argmax decision. The classification output layer of the second Swin Transformer model, based on the classification output layer of the first Swin Transformer model, adds an Argmax decision after the Softmax activation function. This decision is used to select the fault category corresponding to the index position with the highest probability value in the probability distribution vector as the final diagnostic output.

[0117] In this embodiment of the application, the processing flow of the second Swing Transformer model is as follows: Figure 8 As shown, the internal feature transfer process of two consecutive Swing Transformer Blocks is as follows: Figure 9 As shown. Figure 9 The A to I markers on the right side of the grid are used to illustrate the changes in the window boundaries before and after the cyclic shift and their impact on the reorganization of feature regions. The shift operation redivides the feature regions at the original window boundaries into adjacent new windows, thereby enabling information exchange between different windows.

[0118] In this application, the offline training process of the second Swin Transformer model is basically the same as that of the first Swin Transformer model. The difference is that: (1) the input samples are several DAFRP maps generated by historical core feature vectors, rather than several DAFRP maps generated by historical primary feature vectors; (2) the stopping condition is that the loss value converges to below the preset threshold or the training rounds reach the preset upper limit, rather than stopping after completing one training round.

[0119] It should be noted that both the feature evaluation Swin Transformer model and the fault diagnosis Swin Transformer model complete parameter optimization during the offline training phase. During the actual online diagnosis phase, the model parameters remain frozen and are no longer updated.

[0120] In this embodiment of the application, the DAFRP image is a square image with a height of Both width W and side length L are equal to the DAFRP image. This DAFRP image is generated as a single-channel grayscale image and needs to be copied and mapped to the three RGB channels to form a three-channel RGB image. The fault diagnosis Swin Transformer model inputs a three-channel RGB image in the form of a 3×W×3 (i.e., L×L×3) tensor for fault diagnosis. After inputting the three-channel RGB image into the Swin Transformer model, the image patch segmentation unit first performs image patch segmentation: dividing the three-channel RGB image into non-overlapping image patches of 4×4 pixels each. Each image patch contains 4×4×3=48 pixel values, which are then flattened to obtain a 48-dimensional original feature vector. At this point, the feature tensor size is transformed to ( (W / 4)×(W / 4)×48, which is 256×256×48. Then, a linear embedding unit maps the 48-dimensional feature vector of each image patch to a preset embedding dimension. (In the embodiments of this application) =96), the output feature tensor size is ( / 4)×(W / 4)×C, which is 256×256×96, is then input into the four-stage hierarchical feature extraction module for processing.

[0121] In the four-stage hierarchical feature extraction module, the first feature extraction stage consists of two consecutive Swin TransformerBlocks, with the output feature tensor size remaining constant at 256×256×96. After the first feature extraction stage, adjacent 2×2 image blocks are merged through the first block fusion layer, halving the spatial resolution of the feature map and doubling the number of channels, transforming the output size to ( / 8)×(W / 8)×2C, i.e., 128×128×192; the second feature extraction stage also consists of 2 SwinTransformer Blocks, and the output feature tensor size remains unchanged at 128×128×192. Then, the second block fusion layer merges adjacent 2×2 image blocks, halving the spatial resolution of the feature map and doubling the number of channels, transforming the output size to ( / 8)×(W / 8)×2C, i.e., 128×128×192; the second feature extraction stage also consists of 2 SwinTransformer Blocks, and the output feature tensor size remains unchanged at 128×128×192. / 16)×(W / 16)×4C, i.e., 64×64×384; the third feature extraction stage consists of 6 Swin TransformerBlocks, and the output feature tensor size remains unchanged at 64×64×384. Then, the third block fusion layer merges adjacent 2×2 image blocks, halving the spatial resolution of the feature map and doubling the number of channels, transforming the output size to ( / 16)×(W / 16)×4C, i.e. 64×64×384; the third feature extraction stage consists of 6 Swin TransformerBlocks, and the output feature tensor size remains unchanged at 64×64×384. / 32)×(W / 32)×8C, i.e. 32×32×768; the fourth feature extraction stage consists of two Swing Transformer Blocks, and the output feature tensor size remains unchanged at 32×32×768. Since this stage is the end of the model, no more block fusion operations are performed, so the final output feature tensor size is 32×32×768.

[0122] In the classification output layer, layer normalization standardizes the feature tensor (32×32×768) output by the four-stage hierarchical feature extraction module along the channel dimension; the global average pooling layer compresses the standardized 32×32×768 feature tensor into a 768-dimensional one-dimensional feature vector along the spatial dimension; the fully connected layer linearly maps the 768-dimensional one-dimensional feature vector to the fault category number dimension, outputting a Logits vector with the same dimension as the number of fault categories; the Softmax activation function maps the Logits vector output by the fully connected layer into a probability distribution vector, with each dimension taking values ​​between 0 and 1 and a sum of 1; the Argmax decision selects the fault category corresponding to the index position with the highest probability value in the probability distribution vector as the final diagnostic output.

[0123] In this embodiment of the application, considering that both the feature evaluation Swin Transformer model and the fault diagnosis Swin Transformer model processing include 4×4 pixel image block segmentation operations and three subsequent 2x downsampling operations, in order to prevent the size of the feature map from being a non-integer in each feature extraction stage, the side length L of the DAFRP map generated by the core feature vector, perturbation vector or historical primary feature vector must also satisfy: L / 4, L / 8, L / 16, and L / 32 are all positive integers, that is, L is an integer multiple of 32.

[0124] In this embodiment of the application, the preset fault diagnosis categories include health status, bearing inner ring fault, bearing outer ring fault, bearing ball fault, and inter-turn short circuit fault.

[0125] Preferably, in the actual online fault diagnosis stage, once the core feature variables are selected, they are used continuously under the given operating conditions, and the calculation of the average influence value of the feature variables and the screening of the core feature variables are not repeated. During online inference, it is only necessary to update the current value of the core feature vector according to the real-time acquired multimodal signals, and construct the DAFRP diagram in real time according to the given rules. By inputting the diagram into the fault diagnosis Swin Transformer model, the fault diagnosis result at the current moment can be obtained.

[0126] Preferably, when the operating conditions change or the application scenario is switched, it is necessary to collect historical data of the new operating conditions to retrain the feature evaluation Swin Transformer model, calculate the average influence value of feature variables, screen core feature variables, and train the fault diagnosis Swin Transformer model, so that the updated core feature vector, the spatial layout of the dynamic asymmetric feature recursion graph, and the parameters of the fault diagnosis Swin Transformer model are adapted to the new operating conditions.

[0127] A third aspect of the present invention provides a fault diagnosis system based on a dynamic asymmetric feature recursive graph using multimodal signal fusion, for executing a fault diagnosis method based on a dynamic asymmetric feature recursive graph using multimodal signal fusion, comprising: The multimodal signal acquisition and synchronous storage module is used to acquire vibration signal sequences and electromagnetic signal sequences of permanent magnet synchronous motors under variable speed conditions, and add timestamps to the acquired data based on a unified clock reference and synchronously store it in the database. The time alignment and preprocessing module is used to extract timestamped vibration signal sequences and electromagnetic signal sequences from the database. It sets the time window length and sliding step size, slides the time window along the time axis, and extracts the vibration signal segments and electromagnetic signal segments corresponding to each time window from the vibration signal sequence and electromagnetic signal sequence, respectively. Then, using the sampling time of the high sampling rate signal segment as the reference time axis, the low sampling rate signal segment is resampled using a cubic spline interpolation algorithm, so that the vibration signal segments and electromagnetic signal segments in the same time window form a sampling point correspondence on the time axis. The primary feature extraction module is used to extract time-domain and frequency-domain statistical indices from each vibration signal segment and electromagnetic signal segment after time alignment. Each extracted statistical index is used as a feature variable, and the feature variables are normalized. All feature variables extracted and normalized within the same time window are concatenated to construct the primary feature vector corresponding to that time window. At the same time, the normalized feature values ​​of the same feature variable in each time window are arranged along the time axis to form a one-dimensional time-series feature sequence of that feature variable. The DAFRP graph construction module is used to construct a DAFRP graph based on the primary feature vector and the one-dimensional temporal feature sequence of each feature variable in the primary feature vector using the aforementioned dynamic asymmetric feature recursive graph construction method based on multimodal signal fusion. The fault diagnosis module is used to copy and map the DAFRP image to the three RGB channels to form a three-channel RGB image, which is then input into the fault diagnosis Swing Transformer model. The Swing Transformer model outputs the fault diagnosis results of the permanent magnet synchronous motor.

[0128] To verify the effectiveness of the dynamic asymmetric feature recursive graph construction and fault diagnosis method based on multimodal signal fusion provided in this application, a permanent magnet synchronous motor fault diagnosis experimental platform was built, such as... Figure 10 As shown. The platform adopts a coaxial coupling structure and mainly consists of a motor-driven test bench, a multimodal signal acquisition system, and a host computer data processing terminal. In this experiment, the mechanical transmission part includes two CTYT-160-3000 motors with a rated speed of 3000 r / min and a rated voltage of 380V. The left side is the load motor 1, and the right side is the test object—the faulty motor 2. The two are rigidly connected by a coupling assembly and a protective cover 6 to simulate different load conditions and variable speed operation. The multimodal signal acquisition system integrates contact and non-contact sensing methods: a high-sensitivity vibration sensor 3 is directly installed on the top of the faulty motor 2 housing to synchronously acquire vibration signals; a VPMP passive monopole receiving antenna 5 with a working frequency range of DC to 300MHz is deployed next to the permanent magnet synchronous motor to non-destructively pick up spatial electromagnetic radiation signals. This signal is transmitted to an MRM080 electromagnetic acquisition instrument 4 for numerical extraction. The collected raw electromagnetic and vibration data are finally aggregated to the host computer 7 for subsequent signal preprocessing, extraction of time-domain and frequency-domain statistical indicators, construction of primary feature vectors, and generation of DAFRP diagrams.

[0129] In addition, to comprehensively evaluate the diagnostic performance of this method under variable speed conditions, this experiment sets five motor states: healthy, bearing inner ring fault, bearing outer ring fault, bearing ball fault, and inter-turn short circuit fault. It also combines four dynamic non-stationary speed modes: acceleration, deceleration, acceleration-deceleration, and deceleration-acceleration to construct 20 operating conditions. The dataset labels adopt the naming rule of "state abbreviation-speed mode" (such as "SID" representing the acceleration-deceleration condition under inter-turn short circuit fault), as shown in Table 1.

[0130]

[0131] In this experiment, vibration and electromagnetic signals under the above 20 working conditions were collected simultaneously. The time alignment, preprocessing and primary feature extraction were performed according to the dynamic asymmetric feature recursive graph construction and fault diagnosis method based on multimodal signal fusion provided in this application. Primary feature vectors and one-dimensional time-series feature sequences of each feature variable under each working condition were constructed. The corresponding DAFRP graph was then generated and input into the feature evaluation Swin Transformer model / fault diagnosis Swin Transformer model for training and testing. Finally, a complete multimodal experimental dataset for comprehensively evaluating the diagnostic performance of this method was constructed.

[0132] To test the noise robustness of this method, Gaussian white noise was superimposed on both the original vibration and electromagnetic signals to simulate noise interference environments of varying intensities. Specifically, Gaussian white noise was directly superimposed on the original clean signal, and the ratio of signal power to added noise power was controlled to generate a noisy signal with a specific signal-to-noise ratio (SNR). The SNR levels set in this experiment were -4dB, -2dB, 0dB, 2dB, and 4dB, covering typical noise scenarios from extremely low to high SNR. Subsequently, the DAFRP diagram generated from the noisy signal was input into the Swin Transformer fault diagnosis model for fault diagnosis analysis. The definition of signal-to-noise ratio (SNR) is as follows: , In the formula, For sampling point index, The total number of signal sampling points. For the first The original clean signal amplitude at each sampling point For the first The amplitude of Gaussian white noise superimposed at each sampling point.

[0133] This experiment set up a total of 5 motor states, and collected 100 samples for each motor state in 4 speed modes, that is, a total of 400 samples for each motor state, and a total of 2000 samples for the 5 motor states.

[0134] During the experiment, the number of feature variables in the primary feature vector was 32, and the number of core feature variables after adaptive dimensionality reduction was 8. The time window length was 50ms. When the feature variables in the primary feature vector and the perturbation vector were reconstructed in the nonlinear phase space, the preset embedding dimension in the high-dimensional phase space was 257, and the preset delay parameter was 1. When the feature variables in the core feature vector were reconstructed in the nonlinear phase space, the preset embedding dimension in the high-dimensional phase space was 513, and the preset delay parameter was 1. In the feature evaluation Swin Transformer model / fault diagnosis Swin Transformer model, the preset embedding dimension in the linear embedding unit was 96, the learning rate during model training was 0.0001, and the Adam optimizer was used to update the model parameters. The training epochs of the feature evaluation Swin Transformer model were 1. For each motor state, the number of training and test set samples were 320 and 80, respectively, with each speed mode corresponding to 80 training samples and 20 test samples. The maximum number of training epochs for the Transformer model is set to 200 epochs, and training is stopped early when the loss value converges to below a preset threshold.

[0135] To verify the diagnostic accuracy and noise robustness of the proposed method under variable speed conditions, this experiment selected three existing variable speed fault diagnosis methods as comparative baselines. Method 1 uses a recurrence plot (RP) to extract nonlinear features of the signal and combines it with a convolutional neural network (CNN) for fault classification; Method 2 encodes a one-dimensional signal into a two-dimensional image based on a Gramian Angular Field (GAF) and performs classification using a CNN; Method 3 uses a Markov Transition Field (MTF) combined with a ResNet for image transformation and depth feature extraction. The performance of this method was compared with the above three comparative methods on the same experimental dataset. The diagnostic results of each model under different signal-to-noise ratio conditions are shown in Table 2, and the computation time of each model is shown in Table 3.

[0136]

[0137] As shown in Table 2, in an ideal environment without artificial noise, the proposed method achieves over 98.7% in all evaluation metrics (accuracy, precision, and recall), significantly outperforming the three comparative methods. Under varying noise intensities (-4dB to 4dB), the proposed method also demonstrates superior diagnostic accuracy across the board. Particularly under -4dB strong noise conditions, the accuracy of the three comparative methods drops significantly to the 75%–81% range, while the proposed method maintains an accuracy of over 89%, fully validating its excellent noise robustness and feature extraction capabilities in complex noise environments.

[0138]

[0139] As shown in Table 3, this method demonstrates excellent computational efficiency while maintaining high diagnostic accuracy. Its single-round testing time is 11 seconds, on par with the fastest method (Method 1); the overall training time is 3.17 hours, significantly lower than Method 2 (3.73 hours) and Method 3 (5.19 hours). Although the overall training time of this method is slightly longer than that of the relatively simplified Method 1 (2.86 hours), considering the more complex dynamic asymmetric feature recursive graph construction and feature extraction mechanism it introduces, the increase in computational cost is within a reasonable range, further illustrating that this method achieves a good balance between feature learning capability and computational overhead.

[0140] In summary, this method demonstrates excellent performance in terms of diagnostic accuracy, noise robustness, and computational efficiency. Its overall performance is significantly superior to existing comparative methods, and it has promising prospects for engineering applications and potential for online real-time monitoring.

[0141] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for constructing a dynamic asymmetric feature recursive graph based on multimodal signal fusion, characterized in that, Includes the following steps: S11. Using a preset step size, slide the time window, extract several feature variables within each time window, and normalize them. Then, concatenate all the extracted and normalized feature variables within the same time window to construct the primary feature vector corresponding to that time window. ,in, This represents the number of feature variables in the primary feature vector. and It is a positive integer; at the same time, the normalized feature values ​​of the same feature variable under each time window are arranged along the time axis to form a one-dimensional time series feature sequence of the feature variable; S12. Perform a bidirectional perturbation operation on the value of the first feature variable in the primary feature vector corresponding to the current time window to generate a bidirectional perturbation vector of the feature variable in the current time window, wherein the bidirectional perturbation vector includes an upward perturbation vector and a downward perturbation vector. S13. Convert the bidirectional perturbation vector of the first feature variable in the current time window into a dynamic asymmetric feature recursion graph, and obtain the dynamic asymmetric feature recursion graph corresponding to the bidirectional perturbation vector of the first feature variable in the current time window. S14. Input the dynamic asymmetric feature recursive graph corresponding to the bidirectional perturbation vector of the first feature variable in the current time window into the feature evaluation Swin Transformer model. The feature evaluation Swin Transformer model outputs the probability distribution vector corresponding to each perturbation vector and uses it as the prediction result of the bidirectional perturbation vector of the first feature variable in the current time window. The dynamic asymmetric feature recursive graph is a single-channel grayscale image. Before inputting it into the feature evaluation Swin Transformer model, the single-channel grayscale image needs to be copied and mapped to the three RGB channels to form a three-channel RGB image and used as the input of the feature evaluation Swin Transformer model. S15. Based on the prediction results of the bidirectional perturbation vector of the first feature variable in the current time window, calculate the average influence value of the feature variable. The calculation formula is as follows: ， In the formula, For the first The average influence of each characteristic variable This represents the number of classes in the probability distribution vector. This represents the dimension index of the class in the probability distribution vector. For the first The predicted value of the j-th category by the down-difference vector of each feature variable in the current time window. For the first The predicted value of the j-th category by the up-adjustment perturbation vector of each feature variable in the current time window; S16. Repeat steps S12-S15 to obtain the average influence value of the remaining feature variables in the primary feature vector; S17. Based on the average influence value of each feature variable, Sort the feature variables in descending order and select the top [number] from the sorting table. These 10 characteristic variables are used as core characteristic variables to form the core feature vector, where... and It is a positive integer. , ; S18. Based on the one-dimensional time-series feature sequence of each feature variable obtained in step S11, the core feature vector is converted into the target dynamic asymmetric feature recursion graph.

2. The method for constructing a dynamic asymmetric feature recursive graph based on multimodal signal fusion according to claim 1, characterized in that, Step S12 is as follows: Keeping the values ​​of other feature variables unchanged in the current time window, the value of the first feature variable in the current time window is increased by a preset perturbation ratio to form the upward perturbation vector of the first feature variable in the current time window; Keeping the values ​​of other feature variables unchanged in the current time window, the value of the first feature variable in the current time window is reduced by a preset perturbation ratio to form the reduction perturbation vector of the first feature variable in the current time window; Step S13 is as follows: S131. Reconstruct the time series of the up-adjustment perturbation vector of the first feature variable in the current time window in the nonlinear phase space to generate a recursive graph corresponding to each feature variable in the up-adjustment perturbation vector. S132. Divide the recursive graph corresponding to each feature variable in the up-adjustment perturbation vector into two triangular regions along the main diagonal. The upper right region of the main diagonal is the upper triangular part, and the lower left region of the main diagonal is the lower triangular part. S133. According to the sorting position of each feature variable in the up-adjustment perturbation vector, extract one side of the triangular part. Specifically, extract the lower triangular part of the feature variable located in the odd position and extract the upper triangular part of the feature variable located in the even position. S134. According to the sorting position of each feature variable in the up-adjustment perturbation vector, the one-sided triangular parts extracted from two adjacent feature variables are spliced ​​together in the spatial dimension to generate several small squares. S135. Several small squares are arranged in a spatial order corresponding to their sorting positions and then aggregated and spliced ​​into a complete dynamic asymmetric feature recursion graph, which is the dynamic asymmetric feature recursion graph corresponding to the up-adjustment perturbation vector of the first feature variable in the current time window. S136. Repeat steps S131-S135 to obtain the dynamic asymmetric feature recursion graph corresponding to the down-adjustment perturbation vector of the first feature variable in the current time window. Step S18 is as follows: S181. Based on the one-dimensional time series feature sequence of each feature variable obtained in step S11, reconstruct the time series of each feature variable in the core feature vector in the nonlinear phase space to generate a recursive graph corresponding to each feature variable. S182. Divide the recursion graph corresponding to each feature variable into two triangular regions along the main diagonal. The upper right region of the main diagonal is the upper triangular part, and the lower left region of the main diagonal is the lower triangular part. S183. According to the sorting position of each feature variable in the core feature vector, extract one side of the triangular part. Specifically, extract the lower triangular part of the feature variable located in the odd-numbered position and extract the upper triangular part of the feature variable located in the even-numbered position. S184. According to the sorting position of each feature variable in the core feature vector, the one-sided triangular parts extracted from two adjacent feature variables are spliced ​​together in the spatial dimension to generate several small squares. S185. Several small squares are arranged in a spatial order corresponding to their sorting positions and then aggregated and spliced ​​together to form a complete dynamic asymmetric feature recursion graph, which is the target dynamic asymmetric feature recursion graph.

3. The method for constructing a dynamic asymmetric feature recursive graph based on multimodal signal fusion according to claim 2, characterized in that, Step S131 is as follows: S1311. Replace the value corresponding to the current time window in the one-dimensional time series feature sequence of the first feature variable with the perturbation value of the feature variable in the up-adjustment perturbation vector to form the one-dimensional time series feature sequence after the first feature variable is updated. S1312. Map and expand the one-dimensional time-series feature sequence after updating the first feature variable to a high-dimensional phase space; S1313. Calculate the Euclidean distance between any two state points in the high-dimensional phase space, and compare the Euclidean distance with a preset distance threshold. Based on the comparison result, determine whether there is a recursive relationship between the two state points in the high-dimensional phase space. Specifically, if the Euclidean distance between any two state points is less than the preset distance threshold, it is determined that there is a recursive relationship between the two state points in the high-dimensional phase space. Conversely, it is determined that there is no recursive relationship between these two state points in the high-dimensional phase space; S1314. Based on the judgment result of whether there is a recursive relationship between any two state points in the high-dimensional phase space, the recursive matrix is ​​obtained by nonlinear binarization through the Heaviside step function. Specifically, if there is a recursive relationship between any two state points in the high-dimensional phase space, the corresponding element in the recursive matrix is ​​assigned a value of 1; otherwise, the corresponding element in the recursive matrix is ​​assigned a value of 0. S1315. Convert the recursion matrix into a recursion graph to obtain the recursion graph corresponding to the first feature variable in the up-adjustment perturbation vector. S1316. Keeping the one-dimensional time-series feature sequences of the remaining feature variables in the up-adjustment perturbation vector unchanged, repeat steps S1312-S1315 to obtain the recursion graph corresponding to the remaining feature variables in the up-adjustment perturbation vector.

4. The method for constructing a dynamic asymmetric feature recursive graph based on multimodal signal fusion according to claim 2, characterized in that, The sorting positions of each feature variable in the primary feature vector are preset fixed sorting positions, and the sorting positions of the feature variables in each perturbation vector are consistent with the sorting positions of the feature variables in the primary feature vector. The order of each feature variable in the core feature vector is determined by the magnitude of its average influence value, and the feature variables are arranged in descending order of average influence value; The recursion graph corresponding to each feature variable is a square image and is symmetrical along the main diagonal; Based on the average influence value of each feature variable, When sorting the feature variables in descending order, if there are two or more feature variables with the same average influence value, the original sorting order of these feature variables in the primary feature vector, i.e., the fixed sorting position preset in step S11, shall be used as the secondary sorting basis. The earlier the original sorting order is, the earlier the sorting order is, and the later the original sorting order is, the later the sorting order is.

5. The method for constructing a dynamic asymmetric feature recursive graph based on multimodal signal fusion according to claim 2, characterized in that, The feature evaluation Swin Transformer model is obtained by training the first Swin Transformer model offline. The first Swin Transformer model consists of an image patch embedding layer, a four-stage hierarchical feature extraction module, and a classification output layer, specifically: The image patch embedding layer consists of an image patch segmentation unit and a linear embedding unit connected in sequence. The image patch segmentation unit divides the input three-channel RGB image into non-overlapping image patches of 4×4 pixels. Each image patch contains 48 pixel values, which are flattened to form a 48-dimensional original feature vector. The linear embedding unit maps the 48-dimensional original feature vector of each image patch to a preset embedding dimension through a fully connected layer. The four-stage hierarchical feature extraction module consists of four cascaded feature extraction stages. Adjacent stages are connected by a block fusion layer to achieve downsampling. The number of attention heads in each stage is 3, 6, 12, and 24, respectively. The first, second, and fourth feature extraction stages each contain two Swing Transformer Blocks, while the third feature extraction stage contains six Swing Transformer Blocks. In each stage, the Swing Transformer Blocks are nested in pairs, alternating between window-based multi-head self-attention and shift-window-based multi-head self-attention. Two consecutive Swing Transformer Blocks form a module pair, i.e., the previous Swing Transformer Block uses window-based multi-head self-attention, and the next Swing Transformer Block uses shift-window-based multi-head self-attention. The internal structure of each Swing Transformer Block is as follows: first, it undergoes layer normalization; then, it performs multi-head self-attention calculation through either window-based or shift-window-based multi-head self-attention; after residual connection, it undergoes layer normalization again; then, it undergoes nonlinear transformation through a multilayer perceptron; and finally, it outputs through residual connection. The classification output layer consists of layer normalization, a global average pooling layer, a fully connected layer, and a Softmax activation function connected sequentially. Specifically: layer normalization standardizes the feature tensor along the channel dimension; the global average pooling layer compresses the feature tensor into a one-dimensional feature vector along the spatial dimension; the fully connected layer linearly maps the one-dimensional feature vector to the dimension of the number of fault categories, outputting a Logits vector with the same dimension as the number of fault categories, where the Logits vector is the original score vector without Softmax activation function normalization; and the Softmax activation function maps the Logits vector output by the fully connected layer into a probability distribution vector, where each dimension takes values ​​between 0 and 1 and sums to 1.

6. The method for constructing a dynamic asymmetric feature recursive graph based on multimodal signal fusion according to claim 5, characterized in that, The offline training process for the first Swin Transformer model is as follows: (1) Several dynamic asymmetric feature recursive maps generated by historical primary feature vectors are used as training samples and input into the first Swin Transformer model. The first Swin Transformer model outputs a probability distribution vector and uses the probability distribution vector as the prediction result. The dynamic asymmetric feature recursive map is a single-channel grayscale image. Before inputting it into the first Swin Transformer model, the single-channel grayscale image needs to be copied and mapped to the three RGB channels to form a three-channel RGB image and use it as the input of the first Swin Transformer model. (2) Calculate the loss value between the predicted result and the actual result of each historical primary feature vector based on the cross-entropy loss function. The calculation formula is as follows; ， In the formula, The loss value between the actual result and the predicted result. The number of classes in the probability distribution vector; For the first The true value for each category is either 0 or 1; For the first The predicted values ​​for each category range from 0 to 1; (3) Calculate the gradient of each trainable parameter in the first Swing Transformer model with respect to the loss value using the backpropagation algorithm; (4) Update the model parameters using the Adam optimizer; (5) When a training round ends, that is, when all the dynamic asymmetric feature recursion graphs corresponding to the historical primary feature vectors have completed a complete forward and backward propagation, training stops and the feature evaluation Swin Transformer model is obtained. Furthermore, the dynamic asymmetric feature recursive map generated from the core feature vector, perturbation vector, or historical primary feature vector is a square image with a size of L×L pixels, where L is the side length of the dynamic asymmetric feature recursive map in the height or width direction. And N is a positive integer. or , or , This represents the side length of a single recursive graph within the perturbation vector or historical primary eigenvectors. The side length of a single recursive graph in the core feature vector; When the dynamic asymmetric feature recursive graph is generated from the perturbation vector or the historical primary feature vector... When the dynamic asymmetric feature recursive graph is generated from the core feature vector, When the dynamic asymmetric feature recursive graph generated from the core feature vector has the same size as the dynamic asymmetric feature recursive graph generated from the perturbation vector or historical primary feature vector, ,in, , , Pick and The least common multiple; when the dynamic asymmetric feature recursive graph generated by the core feature vector has a different size than the dynamic asymmetric feature recursive graph generated by the perturbation vector or historical primary feature vector. and .

7. The method for constructing a dynamic asymmetric feature recursive graph based on multimodal signal fusion according to claim 6, characterized in that, The dynamic asymmetric feature recursive graph generated from the core feature vector has the same size as the dynamic asymmetric feature recursive graph generated from the perturbation vector or the historical primary feature vector; The dynamic asymmetric feature recursive graph generated from the core feature vector, perturbation vector, or historical primary feature vector has n small squares arranged in both the width and height directions, forming a total of n squares. There are 3 small squares, each with the same side length as a single recursive graph; When each feature variable in the core feature vector, perturbation vector, or historical primary feature vector is reconstructed in the nonlinear phase space, the length of the one-dimensional time-series feature sequence of each feature variable is T=2N. After mapping and expanding the one-dimensional temporal feature sequence of each feature variable into a high-dimensional phase space, the number of state vectors obtained is: Where m is the preset embedding dimension in the high-dimensional phase space, This is a preset time delay parameter in the high-dimensional phase space; the number of state vectors in the high-dimensional phase space is equal to the side length of a single recursive graph, i.e., N = .

8. A fault diagnosis method based on dynamic asymmetric feature recursion graphs using multimodal signal fusion, characterized in that, Includes the following steps: S21. Under variable speed conditions, collect the vibration signal sequence and electromagnetic signal sequence of the permanent magnet synchronous motor during operation, add timestamps to the collected data based on a unified clock reference, and store them synchronously in the database. S22. Extract the timestamped vibration signal sequence and electromagnetic signal sequence from the database, set the time window length and sliding step size, slide the time window along the time axis, and extract the vibration signal segment and electromagnetic signal segment corresponding to each time window from the vibration signal sequence and electromagnetic signal sequence respectively. Then, using the sampling time of the high sampling rate signal segment as the reference time axis, the low sampling rate signal segment is resampled using the cubic spline interpolation algorithm, so that the vibration signal segment and electromagnetic signal segment in the same time window form a sampling point correspondence on the time axis. S23. For each vibration signal segment and electromagnetic signal segment obtained in step S22 after time alignment, extract their time-domain statistical index and frequency-domain statistical index respectively, take each extracted statistical index as a feature variable, and normalize each feature variable. S24. Using the dynamic asymmetric feature recursive graph construction method based on multimodal signal fusion as described in any one of claims 1-7, construct a dynamic asymmetric feature recursive graph; S25. Copy and map the dynamic asymmetric feature recursive graph to the three RGB channels to form a three-channel RGB image and input it into the fault diagnosis Swing Transformer model. The fault diagnosis Swing Transformer model outputs the fault diagnosis results of the permanent magnet synchronous motor.

9. The fault diagnosis method based on dynamic asymmetric feature recursion graph fusion according to claim 8, characterized in that, The time-domain statistical indices for each vibration signal segment and electromagnetic signal segment include the mean, root mean square, standard deviation, peak-to-peak value, kurtosis, skewness, shape index, peak index, root mean square amplitude, impulse index, and margin index. The frequency-domain statistical indices for each vibration signal segment and electromagnetic signal segment include the frequency center, mean square frequency, root mean square frequency, frequency variance, and root variance frequency. The time-domain statistical index of each vibration signal segment and electromagnetic signal segment is directly calculated from the original waveform of their respective signal segments, while the frequency-domain statistical index is calculated by performing a Fourier transform on their respective signal segments. The fault diagnosis Swin Transformer model is obtained by training the second Swin Transformer model offline. The network structure of the second Swin Transformer model is the same as that of the first Swin Transformer model, consisting of an image patch embedding layer, a four-stage hierarchical feature extraction module, and a classification output layer. The difference between the two lies in the final structure of the classification output layer. Specifically, the classification output layer of the first Swin Transformer model consists of a layer normalization layer, a global average pooling layer, a fully connected layer, and a Softmax activation function connected in sequence, stopping after outputting the probability distribution vector, and does not include an Argmax decision. The classification output layer of the second Swin Transformer model, based on the classification output layer of the first Swin Transformer model, adds an Argmax decision after the Softmax activation function. This decision is used to select the fault category corresponding to the index position with the highest probability value in the probability distribution vector as the final diagnostic output. The offline training process of the second Swin Transformer model is basically the same as that of the first Swin Transformer model. The difference is that: (1) the input sample is a dynamic asymmetric feature recursive graph generated by several historical core feature vectors, rather than a dynamic asymmetric feature recursive graph generated by several historical primary feature vectors; (2) the stopping condition is that the loss value converges to below the preset threshold or the training round reaches the preset upper limit, rather than stopping after completing one training round. In the actual online fault diagnosis phase, once the core feature variables are selected, they are continuously used under the given operating conditions without repeating the calculation of the average influence value of the feature variables and the selection of core feature variables. During online inference, it is only necessary to update the current value of the core feature vector based on the real-time acquired multimodal signals and construct a dynamic asymmetric feature recursion graph in real time according to the established rules. Inputting this graph into the fault diagnosis Swin Transformer model will yield the fault diagnosis result at the current moment. When the operating conditions change or the application scenario is switched, it is necessary to collect historical data of the new operating conditions to re-complete the feature evaluation Swin Transformer model training, feature variable average influence value calculation, core feature variable selection, and fault diagnosis Swin Transformer model training, so that the updated core feature vector, the spatial layout of the dynamic asymmetric feature recursion graph, and the fault diagnosis Swin Transformer model parameters are adapted to the new operating conditions.

10. A fault diagnosis system based on a dynamic asymmetric feature recursive graph using multimodal signal fusion, used to execute the fault diagnosis method based on a dynamic asymmetric feature recursive graph using multimodal signal fusion as described in any one of claims 8-9, characterized in that, include: The multimodal signal acquisition and synchronous storage module is used to acquire vibration signal sequences and electromagnetic signal sequences of permanent magnet synchronous motors under variable speed conditions, and add timestamps to the acquired data based on a unified clock reference and synchronously store it in the database. The time alignment and preprocessing module is used to extract timestamped vibration signal sequences and electromagnetic signal sequences from the database. It sets the time window length and sliding step size, slides the time window along the time axis, and extracts the vibration signal segments and electromagnetic signal segments corresponding to each time window from the vibration signal sequence and electromagnetic signal sequence, respectively. Then, using the sampling time of the high sampling rate signal segment as the reference time axis, the low sampling rate signal segment is resampled using a cubic spline interpolation algorithm, so that the vibration signal segments and electromagnetic signal segments in the same time window form a sampling point correspondence on the time axis. The primary feature extraction module is used to extract time-domain and frequency-domain statistical indices from each vibration signal segment and electromagnetic signal segment after time alignment. Each extracted statistical index is used as a feature variable, and the feature variables are normalized. All feature variables extracted and normalized within the same time window are concatenated to construct the primary feature vector corresponding to that time window. At the same time, the normalized feature values ​​of the same feature variable in each time window are arranged along the time axis to form a one-dimensional time-series feature sequence of that feature variable. The dynamic asymmetric feature recursive graph construction module is used to construct a dynamic asymmetric feature recursive graph based on a multimodal signal fusion method, using a primary feature vector and a one-dimensional temporal feature sequence of each feature variable in the primary feature vector. The fault diagnosis module is used to copy and map the dynamic asymmetric feature recursive graph to the three RGB channels to form a three-channel RGB image, which is then input into the fault diagnosis Swing Transformer model. The Swing Transformer model outputs the fault diagnosis results of the permanent magnet synchronous motor.

Images