Physical feature driven adaptive fusion multi-modal fault diagnosis method and system

Abstract

This invention discloses a physical feature-driven adaptive fusion multimodal fault diagnosis method and system. The method includes the following steps: acquiring leakage flux signals and vibration signals of a motor; calculating physical feature vectors; extracting leakage flux signal features, vibration signal features, and cross-modal coupling features; inputting the physical feature vectors into a modal-level gating weight generation network to obtain modal-level gating attention weights corresponding to each mode; weighting each modal feature according to the modal-level gating attention weights to obtain weighted modal features; fusing the weighted modal features with the physical feature vectors to obtain global fusion features; and outputting the fault category of the device to be diagnosed based on the global fusion features. This invention, by selecting physical features and using them as the basis for generating modal gating weights, enables the model to adaptively adjust the contribution of different modal features according to the current physical state, thereby improving the accuracy, robustness, and generalization ability under varying operating conditions of the diagnostic method and system.

Description

Technical Field

[0001] This invention belongs to the field of industrial intelligent system technology, specifically relating to a multimodal fault diagnosis method and system driven by physical features and adaptive fusion. Background Technology

[0002] As a core power device in high-end equipment and intelligent manufacturing, the intelligent condition monitoring and fault diagnosis of electric motors are important typical applications of industrial artificial intelligence systems. Utilizing artificial intelligence and machine learning technologies to intelligently analyze and diagnose faults in multi-source operating data of electric motors is a crucial means to achieve predictive maintenance and improve the operational stability of electric motor equipment.

[0003] In existing motor fault diagnosis, a multimodal sensing scheme has been proposed to overcome the shortcomings of single signal diagnosis in terms of its limited dimension and insufficient identification. This scheme achieves fault identification and judgment by simultaneously collecting multi-source operating signals such as current, vibration, leakage flux, and temperature, and combining them with feature fusion algorithms.

[0004] However, existing multimodal fusion diagnostic methods mostly employ fixed weight assignment, direct feature concatenation, or pure data-driven attention fusion mechanisms, which have the following shortcomings: different modes have different sensitivities to different faults, and fixed fusion or simple concatenation can easily lead to low-quality modes dragging down the overall diagnostic results; when a certain mode is contaminated by noise, ordinary feature-level fusion cannot automatically reduce the weight of that mode; under changes in operating conditions such as speed, load, and noise, the reliability of each mode will change, and existing fusion mechanisms cannot adaptively adjust in real time; there are serious deficiencies in adaptability to complex operating conditions and interpretability of diagnostic results. Summary of the Invention

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a physical feature-driven adaptive fusion multimodal fault diagnosis method and system. By screening physical features and using them as the basis for generating the gating weights of each mode, the model can adaptively adjust the contribution of different modal features according to the current physical state, thereby improving the accuracy, robustness, and generalization ability of the diagnosis method and system under varying operating conditions.

[0006] To achieve the above objectives, according to one aspect of the present invention, a multimodal fault diagnosis method driven by physical features and adaptive fusion is provided, comprising the following steps: Acquire the leakage flux signal and vibration signal of the motor; Obtain the physical feature vectors of the leakage flux signal and the vibration signal to characterize the motor operating conditions, fault states and modal reliability. The physical feature vectors are vectors composed of physical features of the vibration signal, physical features of the leakage flux signal, and physical features of cross-modal coupling. Based on the feature extraction network, leakage magnetic signal features, vibration signal features and cross-modal coupling features are extracted for the leakage magnetic signal and the vibration signal; The physical feature vector is input into a modal-level gated weight generation network to obtain modal-level gated attention weights corresponding to the leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature. The modal-level gated attention weights are dynamically adjusted according to the changes in the physical feature vector. The leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature are weighted according to the modal-level gated attention weights to obtain weighted modal features. The weighted modal features are fused with the physical feature vector to obtain global fused features; The fault diagnosis results of the device to be diagnosed are output based on the global fusion features.

[0007] Furthermore, the step of obtaining the physical feature vectors of the leakage flux signal and vibration signal to characterize the motor's operating conditions, fault states, and modal reliability includes the following steps: Construct a physical feature screening sample set, and construct the first candidate physical feature set; Normalize each candidate physical feature in the first candidate physical feature set; Calculate the variance of each candidate physical feature in the first candidate physical feature set on the physical feature screening sample set, and select the second candidate physical feature subset according to the variance magnitude; Based on the correlation between each candidate physical feature in the second candidate physical feature subset and the fault category, and the redundancy among the candidate physical features in the second candidate physical feature subset, a physical feature vector is obtained by filtering from the second candidate physical feature subset.

[0008] Further, the step of filtering the physical feature vector from the second candidate physical feature subset based on the correlation between each candidate physical feature in the second candidate physical feature subset and the fault category, and the redundancy among the candidate physical features in the second candidate physical feature subset, includes the following steps: S1, assuming the initially selected subset of physical features. Empty; S2: First, iterate through all candidate physical features in the second candidate physical feature subset, and directly select the candidate physical feature with the largest mutual information with the fault category as the first physical feature and store it in the physical feature subset. middle; S3, Traverse the second candidate physical feature subset and remove those already stored in the physical feature subset. The remaining candidate physical features will be used to formulate The largest candidate physical feature is selected and stored as a subset of physical features. ,in, For the second candidate physical feature subset, the first i Candidate physical features Mutual information between fault label Y, For the second candidate physical feature subset, the first i Candidate physical features With the j Candidate physical features Mutual information between them t Number of rounds of filtering; S4. Repeat step S3 until a preset number of physical features are selected.

[0009] Furthermore, the physical feature vector includes the maximum amplitude of the first sub-band after the three-layer wavelet packet decomposition of the leakage magnetic signal, the center frequency of the eighth sub-band after the three-layer wavelet packet decomposition of the vibration signal, the total harmonic distortion rate of the leakage magnetic signal, the ratio of leakage magnetic distortion rate to vibration total harmonic distortion rate, the cross-modal energy coupling fluctuation index, the DC bias of the leakage magnetic signal, the standard deviation of the sixth sub-band after the three-layer wavelet packet decomposition of the vibration signal, the amplitude of the second harmonic of the fundamental frequency of the leakage magnetic signal, the amplitude of the third harmonic of the fundamental frequency of the leakage magnetic signal, the amplitude of the sideband of the sum of the fundamental frequency and mechanical frequency of the leakage magnetic signal, the peak value of the cross power spectral density of the leakage magnetic vibration envelope, the amplitude of the fifth harmonic of the fundamental frequency of the leakage magnetic signal, the amplitude of the sideband of the difference between the fundamental frequency and mechanical frequency of the leakage magnetic signal, the cross-entropy of the leakage magnetic vibration, the maximum amplitude of the sixth sub-band after the three-layer wavelet packet decomposition of the vibration signal, and the energy of the first sub-band after the three-layer wavelet packet decomposition of the leakage magnetic signal.

[0010] Furthermore, the step of extracting leakage magnetic signal features, vibration signal features, and cross-modal coupling features for the leakage magnetic signal and vibration signal based on the feature extraction network includes the following steps: Wavelet packet decomposition is performed on the leakage magnetic signal to extract leakage magnetic signal sub-band signals of multiple frequency bands. A leakage magnetic signal recurrence graph is generated for each leakage magnetic signal sub-band signal to obtain leakage magnetic signal recurrence graphs of multiple frequency bands. The vibration signal is decomposed by wavelet packet to extract vibration signal sub-band signals of multiple frequency bands. A vibration signal recursion graph is generated for each vibration signal sub-band signal to obtain vibration signal recursion graphs of multiple frequency bands. Calculate the joint recurrence matrix of the leakage magnetic signal and the vibration signal to generate a cross-modal recurrence graph; The leakage magnetic field signal recursion map of the multiple frequency bands, the vibration signal recursion map of the multiple frequency bands, and the cross-modal recursion map are respectively input into the feature extraction network corresponding to each mode to obtain the leakage magnetic field signal features, vibration signal features and cross-modal coupling features.

[0011] Further, the step of calculating the joint recurrence matrix of the leakage magnetic signal and the vibration signal to generate a cross-modal recurrence graph includes the following steps: The thresholdless Gaussian kernel distance matrix is ​​calculated for both the leakage magnetic field signal and the vibration signal. Normalization is performed on each thresholdless Gaussian kernel distance matrix, and then the joint recursive matrix is ​​obtained by element-wise multiplication. The formula for element-wise multiplication is as follows: ,in, Represents the normalized joint recursive matrix. Line number Column elements, The thresholdless Gaussian kernel distance matrix of the leakage magnetic field signal after normalization represents the first... Line number Column elements, The thresholdless Gaussian kernel distance matrix representing the vibration signal is... Line number Column elements, This indicates element-wise multiplication, converting the joint recursive matrix into a cross-modal recursive graph.

[0012] Furthermore, before extracting leakage magnetic signal features, vibration signal features and cross-modal coupling features from the leakage magnetic signal and the vibration signal based on the feature extraction network, an equal-angle sampling grid is constructed based on the rotor position signal to resample the leakage magnetic signal and vibration signal from the time domain to the angle domain.

[0013] Furthermore, the modal-level gated weight generation network includes: A two-layer perceptron is used to first expand the dimension of the physical feature vector using a hidden layer to generate a feature tensor, and then use an output layer to compress the feature tensor to reduce the dimension to a three-dimensional vector. The function layer is used to normalize the three-dimensional vector and generate modal-level gated attention weights for the three modes. And satisfy In the formula These are the modal-level gating attention weights corresponding to the leakage magnetic signal characteristics. The modal-level gated attention weights corresponding to the vibration signal characteristics are... The modal-level gating attention weights are those corresponding to the cross-modal coupling features.

[0014] Furthermore, before fusing the weighted modal features with the physical feature vector, a high-dimensional physical feature mapping network based on a two-layer perceptron is used to convert the physical feature vector into a vector with the same dimension as the weighted modal features.

[0015] According to another aspect of the present invention, a physical feature-driven adaptive fusion multimodal fault diagnosis system is provided, comprising: The data acquisition module is used to acquire the leakage flux signal and vibration signal of the motor; The physical feature vector calculation module is used to obtain physical feature vectors of the leakage flux signal and the vibration signal to characterize the motor operating conditions, fault states and modal reliability. The physical feature vector is a vector composed of physical features of the vibration signal, physical features of the leakage flux signal and physical features of cross-modal coupling. A feature extraction network is used to extract leakage magnetic signal features, vibration signal features, and cross-modal coupling features for the leakage magnetic signal and the vibration signal. The modal-level gated attention module is used to input the physical feature vector into the modal-level gated weight generation network to obtain the modal-level gated attention weights corresponding to the leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature. The modal-level gated attention weights are dynamically adjusted according to the changes in the physical feature vector. The leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature are weighted according to the modal-level gated attention weights to obtain weighted modal features. The fusion module is used to fuse the weighted modal features with the physical feature vector to obtain global fused features; The diagnostic module is used to output the fault diagnosis results of the device to be diagnosed based on the global fusion features.

[0016] Overall, the technical solutions conceived in this invention have beneficial effects compared with the prior art: (1) This invention uses physical features that reflect the operating conditions, fault status and modal reliability of the equipment as the basis for generating gating weights. The modal-level gating weight generation network dynamically outputs the fusion weights corresponding to each modality, thereby realizing the dynamic guidance of physical features on the modal fusion process. This enables the model to adaptively adjust the contribution of different modal features according to the current physical state, avoids blind splicing of multimodal features, and effectively alleviates the problems of "all modalities are equally weighted input" or "inferior modalities directly participate in classification" in traditional feature-level fusion.

[0017] (2) Through a two-level screening mechanism of “variance, minimum redundancy and maximum correlation”, physical features with maximum fault correlation and minimum information redundancy were selected from the independently constructed full-condition domain feature subset.

[0018] (3) By using a modal-level gated weight generation network and an attention mechanism within each modality, the adaptive allocation of fault feature weights within and between modalities is achieved. Attached Figure Description

[0019] Figure 1 This is a distribution diagram of the mutual information between the physical features and fault tags selected in the embodiments of the present invention; Figure 2 This is a redundancy distribution diagram among the physical features selected in the embodiments of the present invention; Figure 3 This is a distribution diagram of the correlation coefficients between each pair of the 16 physical features selected in this embodiment of the invention; Figure 4 This is a box plot of the maximum amplitude of the first subband of the magnetic flux leakage signal after three-layer wavelet packet decomposition according to an embodiment of the present invention. Figure 5 This is a box plot of the maximum amplitude of the 6th subband of the vibration signal after wavelet packet decomposition in Embodiment 3 of the present invention. Figure 6 This is the feature preprocessing flow according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the intramodal attention mechanism structure according to an embodiment of the present invention; Figure 8 This is a block diagram of the physical feature-driven modal-level gating attention mechanism according to an embodiment of the present invention. Figure 9 This is a high-dimensional mapping of physical feature information based on a two-layer perceptron in an embodiment of the present invention; Figure 10 This is a diagram of the global feature fusion and fault classifier network structure according to an embodiment of the present invention; Figure 11 This is a schematic diagram of the network structure used in the multimodal fault diagnosis method of this invention. Figure 12 This is a flowchart illustrating the multimodal fault diagnosis method according to an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0021] In the description of the embodiments of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. "Multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0022] In this embodiment of the invention, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, apparatus, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product or device.

[0023] The naming or numbering of steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effect can be achieved.

[0024] In this invention, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this invention can be combined with other embodiments.

[0025] This invention provides a physical feature-driven adaptive fusion multimodal fault diagnosis method and system, which will be described below.

[0026] An embodiment of the present invention provides a physical feature-driven adaptive fusion multimodal fault diagnosis method, comprising: Step 1: Obtain the leakage flux signal and vibration signal of the motor; Step 2: Obtain the physical feature vectors of the leakage flux signal and the vibration signal to characterize the motor operating conditions, fault states and modal reliability. The physical feature vectors are composed of the physical features of the vibration signal, the physical features of the leakage flux signal, and the physical features of cross-modal coupling. Step 3: Extract leakage magnetic signal features, vibration signal features, and cross-modal coupling features from the leakage magnetic signal and the vibration signal based on the feature extraction network; Step four: Input the physical feature vector into the modal-level gated weight generation network to obtain the modal-level gated attention weights corresponding to the leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature. The modal-level gated attention weights are dynamically adjusted according to the changes in the physical feature vector. The leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature are weighted according to the modal-level gated attention weights to obtain the weighted modal features. Step 5: Fuse the weighted modal features with the physical feature vector to obtain the global fused features; Step 6: Output the fault diagnosis results of the device to be diagnosed based on the global fusion features.

[0027] The following sections will provide a detailed explanation of the optional implementation methods for each step.

[0028] Step 1: Obtain the leakage flux signal and vibration signal of the motor.

[0029] The fault diagnosis and identification of this invention is based on leakage magnetic signals and vibration signals.

[0030] In one embodiment, the fault diagnosis results include seven types, and the specific implementation of obtaining the leakage magnetic signal and vibration signal of the motor is as follows.

[0031] First, magnetic field sensors and vibration sensors are set up on the permanent magnet synchronous motor fault test platform, and rotor angle information is collected synchronously using a rotor position encoder.

[0032] Subsequently, samples were collected from permanent magnet synchronous motors with different fault types and under different operating conditions. By replacing the motor, the fault type was changed, and finally a multi-source dataset covering 7 different health states of motors was formed, with each health state containing 4 different operating conditions.

[0033] Considering that instantaneous speed fluctuations in permanent magnet synchronous motors can lead to inconsistent time intervals for the same fault event, resulting in recursive texture deformation that affects the final fault diagnosis, this invention proposes an angle-domain resampling technique based on rotor position. This invention first obtains the unfolded rotor position using a rotary transformer and constructs an equiangular sampling grid. Then, all collected leakage magnetic field and vibration signal data are re-interpolated and mapped from the time domain to the rotor position angle domain to obtain the resampled leakage magnetic field signal. and vibration signals The subsequent third step, extracting leakage magnetic field signal features, vibration signal features, and cross-modal coupling features, is based on the resampled leakage magnetic field signal. and vibration signals This is to reduce the impact of rotation speed fluctuations on the texture structure of the recursive graph.

[0034] Finally, the resampled samples are divided into data segments: first, stratified statistics are performed according to fault category and operating condition. In one embodiment, the training set and test set are randomly divided in a 4:1 ratio. The length of each data segment is 1280 (during resampling, angle domain resampling is performed according to 1024 data points per revolution. Considering that the motor rotates about 1.25 revolutions in 100 ms under the lowest speed condition, in order to ensure consistent data length and that the effective data period after phase space reconstruction is greater than one revolution, all sample unit sequences in this embodiment are uniformly pruned to 1280 data points after resampling). The number of training set samples for each operating condition under each fault is 400, the number of test set samples is 100, and the number of samples used for feature selection is 25. The fault dataset contains a total of 14700 sample units.

[0035] Step 2: Obtain the physical feature vectors of the leakage magnetic signal and the vibration signal to characterize the motor's operating conditions, fault state, and modal reliability. The physical feature vectors are composed of the physical features of the vibration signal, the physical features of the leakage magnetic signal, and the physical features of cross-modal coupling.

[0036] Physical feature vectors refer to features calculated based on objective physical laws such as the electromechanical principles, dynamics, electromagnetism, mechanical vibration, and circuit characteristics of motors, which are different from the features extracted based on the feature extraction network in step three.

[0037] Further, obtaining the physical feature vectors of the leakage magnetic signal and the vibration signal includes the following steps: Construct a physical feature screening sample set, and construct the first candidate physical feature set; Normalize each candidate physical feature in the first candidate physical feature set; Calculate the variance of each candidate physical feature in the first candidate physical feature set on the physical feature screening sample set, and select the second candidate physical feature subset according to the variance magnitude; Based on the correlation between each candidate physical feature in the second candidate physical feature subset and the fault category, and the redundancy among the candidate physical features in the second candidate physical feature subset, a physical feature vector is obtained by filtering from the second candidate physical feature subset.

[0038] In other words, the physical feature vectors are extracted using a two-stage screening process based on variance, minimum redundancy, and maximum correlation algorithms.

[0039] This invention recognizes that in the absence of a dynamic attention mechanism, a simply assembled multimodal fault diagnosis network can directly affect the final fault diagnosis result when a particular modality is contaminated by noise. Furthermore, different modal features exhibit varying sensitivities to different fault categories. Therefore, introducing an attention network to adaptively assign weights to different modal features can improve the robustness of the fault diagnosis model. However, traditional deep learning-based attention mechanisms generally suffer from poor interpretability. These methods heavily rely on the statistical distribution of the underlying data for optimization, and their weight assignment process lacks clear physical meaning. They also have limited generalization ability when facing unknown, complex, and variable operating conditions, making it difficult to meet the stringent requirements of modern industrial fields for the reliability and interpretability of diagnostic systems.

[0040] In view of this, embodiments of the present invention propose to utilize prior physical knowledge to guide the attention allocation of multimodal networks, aiming to overcome the limitations of purely data-driven approaches and improve the interpretability of fault diagnosis models. However, the initial physical features extracted by multimodal analysis are dimensionally large, and there is significant information redundancy among different features, such as variance and standard deviation. Therefore, selecting the most representative and reflective features from a massive dataset is a crucial step in building the fault diagnosis model proposed in these embodiments. To address this, embodiments of the present invention propose a two-level selection mechanism based on variance, minimum redundancy, and maximum correlation algorithms.

[0041] In practical industrial applications, permanent magnet synchronous motors often operate under non-stationary conditions with varying speeds and loads. Some physical quantities are far more affected by changes in operating conditions than by faults, such as the energy of vibration signals and the amplitude of leakage flux signals. If this is not controlled during the feature selection stage, the physical quantities used will become ineffective when operating conditions change. Therefore, this embodiment of the invention uses data from the entire operating domain, independent of the training and test sets, including variable speed and variable load conditions, to screen physical features in order to identify those physical quantities that consistently contribute significantly to fault diagnosis when operating conditions change.

[0042] In one embodiment, 402 candidate features were first statistically analyzed to form a first candidate physical feature set. These 402 candidate features mainly include time-domain features, frequency-domain features, higher-order statistics, and joint features of leakage magnetic field and vibration signals. 1) Temporal characteristics The main parameters include the mean, standard deviation, variance, peak value, maximum value, minimum value, root mean square, waveform factor, peak factor, impulse factor, and margin factor of leakage magnetic signals and vibration signals.

[0043] 2) Frequency domain characteristics The main parameters include the power spectral density, main frequency amplitude, subharmonic energy, frequency band energy ratio, frequency variance, and mean square frequency of leakage magnetic signals and vibration signals.

[0044] 3) Higher-order statistics Skewness, kurtosis, Shannon entropy, wavelet entropy.

[0045] 4) Combined characteristics of leakage magnetic field and vibration This mainly includes the mutual information, correlation coefficient, covariance, cross-entropy, phase synchronization index, modal amplitude ratio, and common wavelet entropy between leakage magnetic signals and vibration signals.

[0046] 5) Other physical quantities The main parameters include the energy ratio, modal amplitude ratio, phase synchronization index, amplitude ratio in different frequency bands of each layer after wavelet decomposition of leakage magnetic signals and vibration signals, joint entropy, common wavelet entropy, average recursion density, recursion entropy, and correlation entropy of the recursive image clusters and joint recursive graphs constructed from leakage magnetic signals and vibration signals.

[0047] The greater the difference between the same physical features and the data of different fault types, the more significant their impact on fault classification is usually. Therefore, after constructing the above-mentioned candidate physical information feature set, this embodiment of the invention uses the variance screening method as the first-level filtering mechanism to eliminate those static features that are not sensitive to changes in system fault type.

[0048] The data used for physical feature selection is an independent full-condition sample set, meaning the samples in the physical feature selection sample set include all diagnosable fault categories. In one embodiment, this subset contains 7 operating conditions, with 25 sample units for each fault type under each operating condition, totaling 25 × 7 × 4 = 700 sample units. These 700 sample units are not used for model training and testing. Each sample unit extracts 402 physical quantities, and the constructed first candidate feature dataset for the full operating conditions is a matrix. , express The set of real numbers, in which N s The total number of samples, in the embodiments of the present invention N s =700, M 0 represents the number of candidate physical features, which is 402 in this embodiment of the invention.

[0049] One implementation of the variance-based screening method is as follows.

[0050] Because the dimensions of leakage magnetic signals and vibration signals differ greatly, directly calculating the variance would lead to features with larger values ​​dominating the screening results. Therefore, in this embodiment of the invention, equation (1) is first used to evaluate each candidate physical feature in the first candidate physical feature set. Min-Max normalization is performed to map all features to the range [0, 1].

[0051] ; In the formula These are the normalized physical characteristics. For the first j Candidate physical features of each sample The value, and Candidate physical features The maximum and minimum values ​​in.

[0052] Next, the embodiments of the present invention will apply the normalized first... i physical characteristics Calculate its variance on the sample set. ,Right now.

[0053] ; In the formula Physical characteristics The average value.

[0054] Finally, in this embodiment of the invention, the variances of all physical features are sorted according to their numerical values. When the variance is too small, it indicates that the physical feature has low sensitivity to permanent magnet synchronous motor faults across the entire operating range and should be discarded. Accordingly, this embodiment of the invention employs a sorting and truncation strategy to ultimately retain the top few candidate physical features with large variances. In one embodiment, the top 100 candidate physical features with large variances are retained to obtain a second subset of candidate physical features, which serves as the input for the second-level screening.

[0055] By using variance screening, not only was the initial compression of feature dimensions achieved, but physical features with a certain dynamic change capability were also preserved across the entire operating range.

[0056] One implementation of the screening method based on the minimum redundancy maximum correlation algorithm is as follows.

[0057] After variance screening, 100 physical characteristics that significantly vary with the fault type and operating conditions of the permanent magnet synchronous motor were retained. However, the physical characteristics selected based on variance screening still have the following two problems: 1) Variance screening only measures its own fluctuation and does not consider the close relationship between the feature and the failure type. Some physical features with high variance may change drastically with operating conditions, but change less with failure type; 2) There is a strong coupling relationship among the retained physical features, such as variance and standard deviation, root mean square value of vibration signal and energy spectral density, etc., with high information redundancy among them. If these physical quantities are fed into the fault diagnosis model at the same time, it will not only increase the computational load of the model, but may also reduce the robustness of the model to changes in operating conditions.

[0058] Based on this, the embodiment of the present invention uses the minimum redundancy maximum correlation algorithm to perform secondary screening on these 100 physical features.

[0059] Let the physical feature matrix after variance screening be: ; In the formula This represents the number of physical features to be screened in the second round. Indicates the first i The vector of values ​​of each physical feature across all samples.

[0060] In one embodiment, the fault sample label for a permanent magnet synchronous motor is: ; For the k There are 10 samples, and their corresponding labels are 10. , .

[0061] Since the collected leakage magnetic field and vibration signal characteristics are continuous floating-point numbers, they are represented by physical feature vectors. For example, in this embodiment of the invention, a vector is first selected. The maximum and minimum values ​​in the vector are divided into 10 equally wide intervals. Each sample point in Mapped to the corresponding range index In the middle. After this step, the embodiments of the present invention will transform the original... The feature vector is transformed into a discrete interval index vector. The number of equal-width intervals is not limited to 10 and can be adjusted flexibly.

[0062] To calculate physical characteristics Based on the joint probability distribution of fault labels, this embodiment of the invention constructs a 10×7 two-dimensional list. For any element in the list... This represents the number of samples that simultaneously satisfy both of the following conditions: 1) Vector The eigenvalues ​​in the interval [0, 1] fall within the interval [0, 1]. The first in m A range; 2) Fault Label = n , .

[0063] Based on this two-dimensional list, calculate the joint probability distribution. and marginal probability , They are respectively: ; In the formula, N sIn this example, the total number of samples in the sample set is selected based on physical characteristics. N s =700, and ; Based on this, physical characteristics The mutual information between the fault label Y and the fault label Y can be represented as: ; The above analysis shows that if the data in each interval is evenly distributed across all fault types, it indicates physical characteristics. Completely insensitive to fault type, i.e. At this time, physical characteristics Mutual information with fault label Y Conversely, if physical characteristics If the fault type can be perfectly distinguished, it means that the data in each of the above intervals belong to the same fault type, and the mutual information value is relatively large at this time.

[0064] Similarly, physical characteristics can be calculated. With physical characteristics The mutual information between them can be represented as: ; In the formula: ; ; Calculate using the same method as formula (6).

[0065] If we simply use (8) as the evaluation criterion to select the physical feature with the greatest mutual information with the fault type as the final feature vector, it will lead to a high degree of information redundancy among the selected feature vectors. Therefore, in order to solve this problem, this embodiment of the invention constructs a dual evaluation function that includes maximum correlation and minimum redundancy.

[0066] set up For the selected subset of physical features, For physical feature subset The number of physical features.

[0067] Therefore, the final subset of physical features selected satisfies two conditions: 1) Feature subsets and fault types Y The mutual information between them is maximized. for: ; 2) Feature subset The mutual information between internal physical features is minimal. The mutual information between internal physical features is: ; Candidate physical features With candidate physical characteristics Mutual information between them.

[0068] This invention uses mutual information difference as the final feature evaluation operator. ,for: ; Based on this, in order to select the optimal subset satisfying equation (12) from the 100-dimensional candidate feature set, this embodiment of the invention uses a forward search algorithm to filter these 100 features one by one, specifically as follows: S1, assuming the feature subset selected at the initial time. Empty; S2: First, iterate through all candidate physical features in the second candidate physical feature subset, and directly select the candidate physical feature with the largest mutual information with the fault category as the first physical feature and store it in the physical feature subset. In the first round of screening, all elements of the 100 physical features are traversed, and the feature with the highest mutual information with the fault category is directly selected as the first screened physical feature and stored. middle; S3, Traverse the second candidate physical feature subset and remove those already stored in the physical feature subset. The remaining candidate physical features will be selected to maximize the candidate physical features in equation (13) and stored as a subset of physical features. ; S4, repeat step S3 until a preset number of physical features are selected. That is, in the second round of selection (i.e., when step S3 is executed for the first time), traverse the remaining 99 physical features and find the physical feature that maximizes the expression according to equation (13). This round of selection is the second round of selection, so the number of selection rounds is... t =2; In the third round of screening (that is, when step S3 is executed for the second time), the remaining 98 physical features are traversed, and the physical feature that makes the expression the largest is found according to equation (13). t =3, and so on, until 16 physical features are selected. The final number of physical features selected can be adjusted flexibly.

[0069] ; In formula (13) This indicates taking the maximum value. This represents the second candidate physical feature subset. This indicates that the second candidate physical feature subset excludes those already stored in the physical feature subset. The remaining candidate physical features.

[0070] The screening procedure proposed in this embodiment of the invention only needs to be executed once on the platform and dataset. After screening, in one embodiment, the above 16 physical features are first calculated and generated from the time domain sequence. Then, these 16 physical features are independently normalized according to the parameters used during feature screening. Finally, the independently normalized physical features are combined into a 16-dimensional feature vector, which is used as the fourth input branch of the fault diagnosis model. At the same time, modal weights are generated based on these 16-dimensional physical feature vectors.

[0071] Preferably, all 16 core physical features are time-domain physical features, mainly to preserve time-domain characteristics, sense changes in operating conditions, and guide subsequent multimodal fusion. For example, the leakage magnetic mode weight increases during demagnetization faults, the vibration mode weight increases during bearing faults, and the cross-modal coupling weight increases during electromagnetic-mechanical coupling faults.

[0072] Compared to variance-based screening, the minimum redundancy maximum correlation algorithm effectively eliminates a series of features that are physically repetitive despite having large variances. The features retained in the end maximize the preservation of fault classification information under all operating conditions.

[0073] Table 1 shows the 16 physical feature numbers (numbers, i.e., sorting) and their corresponding physical meanings that were finally selected from the feature pool through a two-level screening mechanism in one embodiment.

[0074] As shown in Table 1, the 16 physical features ultimately selected are not concentrated in a single physical field, but simultaneously cover leakage magnetic field single-mode features, vibration single-mode features, and leakage magnetic field vibration coupling features. The leakage magnetic field features mainly include physical quantities related to electromagnetic field harmonic distortion, low-frequency energy distribution, and frequency conversion sideband modulation information; the vibration features mainly consist of mechanical vibration intensity and impact amplitude in the high-frequency band; and the leakage magnetic field vibration coupling features focus on the synchronous coupling relationship between the electromagnetic field and the mechanical field. This indicates that the feature subset retained after two levels of screening not only has high class correlation but also good physical coverage. Furthermore, these 16 physical features, as a whole, include both low-frequency features of the leakage magnetic field signal and high-frequency features of the vibration signal. This is because the embodiment of this invention uses wavelet packet decomposition to simultaneously decompose the high-frequency and low-frequency bands of the original signal. In addition, the top 16 physical features include four coupling features of leakage magnetic field and vibration, which verifies the necessity of extracting cross-modal coupling features in the embodiment of this invention. The wavelet packet decomposition sub-band numbers are arranged according to the binary tree generation order. Layer 0: Original signal S; Layer 1: 2 sub-bands: A1 (low frequency) and D1 (high frequency); Layer 2: 4 sub-bands: low-frequency sub-band AA2 and high-frequency sub-band DA2 decomposed from A1, low-frequency sub-band AD2 and high-frequency sub-band DD2 decomposed from D1; Layer 3: 8 sub-bands, with sub-bands 1 to 8 being the low-frequency sub-band decomposed from AA2, the high-frequency sub-band decomposed from AA2, the low-frequency sub-band decomposed from DA2, the high-frequency sub-band decomposed from DA2, the low-frequency sub-band decomposed from AD2, the high-frequency sub-band decomposed from AD2, the low-frequency sub-band decomposed from DD2, and the high-frequency sub-band decomposed from DD2, respectively.

[0075] Table 5.1 Ranking of the selected physical features and their physical significance ; In one embodiment, the cross-modal energy coupling fluctuation index is calculated as follows.

[0076] ; ; ; In the formula, This is the normalized leakage magnetic signal. The normalized vibration signal This is the cross-modal coupling sequence obtained by multiplying the two points one by one. The mean of cross-modal coupled sequences, The length of a single sample sequence, in this example C s =1280, This is a cross-modal energy coupling fluctuation index, which is used to characterize the synchronous coupling fluctuation intensity between leakage magnetic field anomalies and mechanical vibration response.

[0077] To further demonstrate the execution process of the "variance-maximum correlation minimum redundancy" two-level physical feature screening mechanism proposed in this embodiment of the invention, Figure 1 and Figure 2 The mutual information values ​​between these 16 physical features and the fault labels, as well as the redundancy distribution among these 16 features, are shown respectively.

[0078] Figure 1 The mutual information between each feature and the motor fault label is shown. The higher the mutual information value, the richer the fault classification information contained in the feature itself. It can be seen that the first sub-band of the leakage magnetic signal after the 3-level wavelet packet decomposition, which ranks first, has the highest mutual information score in the entire field, at 2.01.

[0079] Figure 2 The graph illustrates the distribution of redundancy between the selected features and the features to be selected. As can be seen, the redundancy of the first selected features is 0. As the selection process progresses, feature redundancy gradually increases. This is because subsequent selected features need to strike a balance between "retaining sufficient relevance to fault labels" and "controlling the degree of information redundancy with already selected features." Although some selected features have high information redundancy with other features, they are still retained. This indicates that while these features have some similarity to existing features, they can still provide supplementary information from different harmonic orders, different frequency band statistics, or different modal coupling relationships that cannot be completely replaced.

[0080] Figure 3 The graph shows the correlation coefficient distribution between each of the 16 selected physical features. Darker colors indicate stronger correlations. Overall, the features are not entirely independent. Strong positive correlations exist between some leakage magnetic harmonics, sideband, and low-frequency subband energy features, indicating they collectively describe physical phenomena such as electromagnetic distortion and frequency modulation. However, the correlations between some vibrational subband statistics and joint leakage magnetic vibration features and these leakage magnetic features are relatively low, with some even showing weak negative correlations. This suggests they can supplement information about these leakage magnetic features from the perspectives of mechanical response or cross-modal coupling. Therefore, this... Figure 1 This demonstrates that there is indeed a natural connection between physical quantities of the same mode among the 16 retained physical features. On the other hand, it also shows that the current feature set is not concentrated in a single physical domain, but retains good feature complementarity between different physical fields.

[0081] Box plots are the best way to illustrate the distribution of a physical characteristic under different health conditions of a permanent magnet synchronous motor. In a box plot, the horizontal axis represents the fault label, and the vertical axis represents the value of the physical characteristic. The horizontal line in the middle of the box represents the most concentrated typical value of that characteristic under the current fault type. The upper and lower edges of the box represent the upper and lower quartiles of that physical characteristic. The flatter the box, the more stable the physical characteristic. Individual plus signs represent noise points that deviate from the normal range.

[0082] Figure 4 and Figure 5 Box plots are shown for the top-ranked physical feature (maximum amplitude of the first subband of the leakage magnetic field signal after 3-layer wavelet packet decomposition) and the 15th-ranked physical feature (maximum amplitude of the sixth subband of the vibration signal after 3-layer wavelet packet decomposition). It can be seen that the top-ranked feature has a significantly higher value for the LDM (Local Demagnetization Fault) tag than the other categories, and the box plots do not overlap with other categories in the vertical direction. This indicates that the peak value of the low-frequency subband of the leakage magnetic field is more sensitive to local demagnetization faults. Conversely, the peak value of the high-frequency subband of the vibration signal is more sensitive to the ITSC (Inter-turn Short Circuit Fault) tag.

[0083] In summary, the two-level physical feature screening mechanism proposed in this embodiment of the invention does not simply select the top 16 physical features most relevant to the fault label, nor does it mechanically select the 16 physical features with the lowest redundancy among the features. Instead, it seeks the optimal trade-off between the two, thus ensuring the diversity of input features while maximizing the diagnostic performance of the diagnostic model.

[0084] Step 3: Extract leakage magnetic signal features, vibration signal features and cross-modal coupling features for the leakage magnetic signal and the vibration signal based on the feature extraction network.

[0085] In one embodiment, the step of extracting leakage magnetic signal features, vibration signal features, and cross-modal coupling features from the leakage magnetic signal and the vibration signal based on the feature extraction network includes the following steps: Wavelet packet decomposition is performed on the leakage magnetic signal to extract leakage magnetic signal sub-band signals of multiple frequency bands. A leakage magnetic signal recurrence graph is generated for each leakage magnetic signal sub-band signal to obtain leakage magnetic signal recurrence graphs of multiple frequency bands. The vibration signal is decomposed by wavelet packet to extract vibration signal sub-band signals of multiple frequency bands. A vibration signal recursion graph is generated for each vibration signal sub-band signal to obtain vibration signal recursion graphs of multiple frequency bands. Calculate the joint recurrence matrix of the leakage magnetic signal and the vibration signal to generate a cross-modal recurrence graph; The leakage magnetic field signal recursion map, the vibration signal recursion map, and the cross-modal recursion map of the multiple frequency bands are respectively input into the feature extraction network corresponding to each mode to obtain the leakage magnetic field signal features, vibration signal features, and cross-modal coupling features.

[0086] The leakage magnetic signal characteristics, vibration signal characteristics, cross-modal coupling characteristics, and the physical feature vectors selected in step two constitute four input data branches.

[0087] Figure 6 This illustrates a preprocessing flow for four branches of input data in one embodiment, as follows: Input Branch 1: Recursive Image Clusters of Leakage Subband Signals After resampling and normalization, the magnetic flux leakage signal undergoes three-level wavelet packet decomposition to extract eight sub-band signals covering different frequency bands. Phase space reconstruction is then performed on the coefficients of each of these eight sub-bands, and the distance matrix is ​​calculated using a Gaussian kernel function. After image transformation to remove redundancy, eight recursive images, each 140×70 pixels, are finally generated. Therefore, input branch 1 contains eight input channels, each channel being a recursive image of the magnetic flux leakage signal for its corresponding frequency band.

[0088] Input Branch 2: Recursive Image Cluster of Sub-band Signals of Vibration Signal Completely consistent with the processing of input branch 1, the final input branch 2 also contains 8 input channels, each channel being a recursive image of the vibration signal in the corresponding frequency band, with an image pixel size of 124×62.

[0089] Input Branch 3: Joint Recursive Graph of Leakage Magnetic Vibration Cross-Modal Coupling Characteristics In order to capture common fault events in leakage magnetic signals and vibration signals, embodiments of the present invention employ a joint recursive graph.

[0090] By examining the synchronicity of the recursive characteristics of leakage flux signals and vibration signals, the joint evolution law of the electrical and mechanical characteristics of permanent magnet synchronous motors (PMSMs) is characterized. From a physical perspective, the joint recursion graph represents the PMSM fault events in which fault characteristics occur synchronously in leakage flux signals and vibration signals. Based on this, branch 3 contains one input image channel, namely the cross-modal recursion graph.

[0091] The generation of the cross-modal recurrence graph includes the following steps: calculating the thresholdless Gaussian kernel distance matrix for the leakage magnetic signal and the vibration signal respectively; normalizing each thresholdless Gaussian kernel distance matrix; and then obtaining the joint recurrence matrix through element-wise multiplication. The calculation formula is as follows: ,in, Represents the normalized joint recursive matrix. Line number Column elements, The thresholdless Gaussian kernel distance matrix of the leakage magnetic field signal after normalization represents the first... Line number Column elements, The thresholdless Gaussian kernel distance matrix representing the vibration signal is... Line number Column elements, This indicates element-wise multiplication, converting the joint recursive matrix into a cross-modal recursive graph.

[0092] Input branch 4: The filtered 16-dimensional physical feature vector This invention employs a two-level screening mechanism to select 16 physical features from a subset of physical features across the entire operating condition domain in the time domain. These 16 features are most correlated with fault characteristics while exhibiting minimal internal information redundancy. During diagnosis, these 16 physical features are calculated for each input modal signal and independently normalized using the parameters selected during screening. Based on these normalizations, they are combined into a 16-dimensional one-dimensional tensor vector. This 16-dimensional vector is used as branch 4, and this branch also serves as a physical constraint to guide the attention allocation of modal features in subsequent networks of the fault diagnosis model.

[0093] In summary, the fault diagnosis model proposed in this embodiment of the invention simultaneously receives four branches: a recursive image cluster of leakage magnetic subband signals, a recursive image cluster of vibration subband signals, a joint recursive graph of leakage magnetic vibration cross-modal coupling features, and a 16-dimensional one-dimensional physical feature tensor vector. These four branches encompass distributed multi-band images of multi-source information, cross-coupling feature images, and a 16-dimensional core physical feature tensor vector, achieving comprehensive coverage of permanent magnet synchronous motor fault states from frequency band details and multi-field coupling to global statistical features.

[0094] The recursive graphs of leakage magnetic signals, vibration signals, and cross-modal signals in multiple frequency bands are input into the feature extraction network corresponding to each mode to obtain the leakage magnetic signal features, vibration signal features, and cross-modal coupling features.

[0095] In one embodiment, a CBAM-based intramodal self-attention mechanism is used to extract features from each modality.

[0096] Both the magnetic flux leakage signal and the vibration signal were decomposed into three layers of wavelet packet decomposition, generating eight recursive images in different frequency bands, which served as the eight input channels for input branch 1 and input branch 2 of the fault diagnosis model. After multi-scale CNN feature extraction, high-dimensional feature tensors with 64 channels were generated.

[0097] Different types of permanent magnet synchronous motor faults often trigger abnormal responses in different frequency bands. This means that the contribution of each of the eight input frequency bands to the final fault diagnosis result is different. In other words, the sensitivity of each of the 64 generated high-dimensional channels to permanent magnet synchronous motor faults is different. At the same time, in the recursive image, the fault features of permanent magnet synchronous motors are often concentrated in specific local areas of the image, while the rest of the area is mostly background noise.

[0098] Based on this, in order to enable the network to adaptively complete the key frequency band selection and key texture focusing, the embodiment of the present invention adds a Convolutional Block Attention Module (CBAM) after the feature extraction of input branch 1 and input branch 2 is completed.

[0099] The feature extraction network for leakage magnetic signal modes includes: The first feature extraction module is used to extract the initial features of the multi-channel leakage magnetic signal from the recurrence map of leakage magnetic signals in multiple frequency bands. The first channel attention module is used to generate channel attention weights for each channel of the initial features of the leakage magnetic signal. Based on the channel attention weights of the leakage magnetic signal, the initial features of the leakage magnetic signal are multiplied element-wise along the channel dimension to obtain the intermediate features of the leakage magnetic signal. The first spatial attention module is used to generate spatial attention weights for intermediate features of the leakage magnetic signal, and to perform element-wise multiplication of the intermediate features of the leakage magnetic signal in spatial dimension according to the spatial attention weights of the leakage magnetic signal to obtain the leakage magnetic signal features. The feature extraction network for vibration signal modes includes: The second feature extraction module is used to extract the initial features of multi-channel vibration signals from the recurrence map of vibration signals in multiple frequency bands. The second channel attention module is used to generate channel attention weights for each channel of the initial features of the vibration signal. Based on the channel attention weights of the vibration signal, the initial features of the vibration signal are multiplied element-wise along the channel dimension to obtain the intermediate features of the vibration signal. The second spatial attention module is used to generate spatial attention weights for the intermediate features of the vibration signal. Based on the spatial attention weights of the vibration signal, the intermediate features of the vibration signal are multiplied element-wise according to the spatial dimensions to obtain the vibration signal features.

[0100] In other words, the feature extraction network for leakage magnetic signal mode has the same structure as the feature extraction network for vibration signal mode. Both adopt the CBAM attention mechanism and include a feature extraction module, a channel attention module, and a spatial attention module.

[0101] like Figure 7The structure of the CBAM attention mechanism is shown. This module consists of two parts: cascaded channel attention and spatial attention, specifically: 1) Channel Attention Module Channel attention aggregates information from the spatial dimension and calculates a weight coefficient for each of the 64 channels, thereby highlighting the frequency band characteristic channels that are most sensitive to the current permanent magnet synchronous motor faults and suppressing irrelevant channels.

[0102] For the input feature tensor , express The set of real numbers in the dimensional space (with a leakage magnetic tensor size of 64×33×16 and an oscillation of 64×29×14) is first subjected to global average pooling and global max pooling along the spatial dimension, generating two... The vectors are then fed into a two-layer perceptron with shared weights for nonlinear feature compression and recovery. Finally, channel attention weights are generated by element-wise summation and activation with a sigmoid function. ,Right now: ; in, Indicates global average pooling. Indicates global max pooling. This indicates that the data has been processed by a two-layer perceptron. This represents the Sigmoid function.

[0103] Finally, the weights With input feature tensor Element-wise multiplication along the channel dimension yields intermediate features filtered by frequency bands. ,Right now: ; 2) Spatial Attention Module Spatial attention generates a two-dimensional weight by compressing the channel dimension, guiding the fault diagnosis network to focus on the key textures in the recursive graph that characterize faults in permanent magnet synchronous motors.

[0104] The intermediate features selected through frequency band filtering mentioned above Performing average pooling and max pooling along the channel dimension yields two values ​​of size 1. The two-dimensional feature map is stitched together along the channel dimension, and then... Information is fused using standard convolutional layers, and finally, a spatial attention weight matrix is ​​generated by the sigmoid function. .

[0105] ; in, Indicates the process Information fusion is performed using standard convolutional layers.

[0106] The two-dimensional weight and Element-wise multiplication along the spatial dimension yields the final enhanced single-modal output feature. ,Right now: ; After dual self-attention extraction by CBAM, the leakage magnetic field branch and the vibration branch each output a set of 64-channel tensors highlighting fault characteristics, denoted as follows: and They are then fed into the intermodal feature fusion module.

[0107] Step four: Input the physical feature vector into the modal-level gated weight generation network to obtain the modal-level gated attention weights corresponding to the leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature. The modal-level gated attention weights are dynamically adjusted according to the changes in the physical feature vector. The leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature are weighted according to the modal-level gated attention weights to obtain weighted modal features.

[0108] In the actual operating environment of permanent magnet synchronous motors, the intensity of fault characteristics under different fault types and operating conditions varies in different signals. For example, under high-speed and light-load conditions, vibration signals are more sensitive to rotor imbalance faults, while under light-load conditions, leakage flux signals are more sensitive to local demagnetization faults. Moreover, even for the same fault, the bias of its fault characteristics in different signals will change with different speeds and loads. That is, when the operating conditions or fault types of permanent magnet synchronous motors change, the weights of input branch 1, input branch 2, and input branch 3 are constantly changing. If traditional direct dimension splicing or pure data-driven self-attention is continued for feature fusion, it will lead to problems such as decreased accuracy and weakened generalization ability of the network under changing operating conditions or noise interference.

[0109] Based on this, to achieve dynamic, adaptive, and precise allocation of feature fusion weights, this invention innovatively proposes a physical-driven modal-level gating attention mechanism. This mechanism uses the 16-dimensional physical feature vector selected in step two as a "macroscopic perceptron of global operating conditions and fault states." By constructing a two-layer fully connected gating network, it dynamically adjusts the contributions of leakage magnetic mode, vibration mode, and leakage magnetic vibration cross-coupling mode in the current state.

[0110] The input branch 4 of the fault diagnosis model in this embodiment of the invention is a 16-dimensional physical feature vector that has been independently normalized and filtered. To generate weights for different branches, this embodiment of the invention designs a nonlinear two-layer perceptron as a modal-gated weight generation network, such as... Figure 8 As shown, in order to fully extract the nonlinear high-order combination characteristics between the 16-dimensional physical features, the dual-layer perceptron of this embodiment of the invention employs... The construction strategy involves first expanding the dimensions to 32, then compressing and reducing the weights to 3 dimensions. Specifically, the hidden layer of the two-layer perceptron has 32 nodes, and the output layer has 3 nodes. The mathematical process of modality generation can be represented as: ; In the formula The feature tensor output by the hidden layer. and Here are the weights and biases of the hidden layer; and The weights and biases of the output layer; For batch normalization operations, For smooth GELU nonlinear activation function, This represents the score vector for each modality.

[0111] Next, this embodiment of the invention uses the Softmax function to refine the output score vector. After normalization, relative importance gating weights for the three modes were generated. And strictly meet ,Right now: ; In the formula , , These are the weights of the leakage magnetic mode of input branch 1, the vibration mode of input branch 2, and the leakage magnetic-vibration coupled mode of input branch 3, respectively.

[0112] The above analysis shows that, , , These represent the confidence levels of the fault diagnosis model for leakage magnetic field image features, vibration image features, and leakage magnetic field vibration coupling image features, respectively, under the current rotational speed, load, and fault type.

[0113] When the physical feature vector changes, the relative importance gating weights of the three modes generated based on the physical feature vector are adjusted. It will also change, thereby enabling the modal-level gating attention weights to be dynamically adjusted according to the changes in the physical feature vector.

[0114] In this embodiment of the invention, the fault diagnosis model, after feature extraction and weighting via a self-attention mechanism, has input branch 1 with dimensions of 64×33×16, input branch 2 with dimensions of 64×29×14, and input branch 3 with dimensions of 32×309×154. To ensure successful feature fusion, this embodiment introduces Global Average Pooling (GAP) before applying gated weights, which integrates the leakage magnetic field features refined by CBAM. Vibration characteristics and the cross-modal coupling features extracted from input branch 3 Dimensionality reduction is performed separately, compressing it into a one-dimensional vector along the channel dimension, i.e.: ; In the formula , , These are the one-dimensional feature vectors obtained by global average pooling reduction of leakage magnetic field features, vibration features, and leakage magnetic field-vibration coupling features in the channel dimension.

[0115] Subsequently, the obtained physical feature-driven gating weights are directly multiplied by the aforementioned dimensionality-reduced one-dimensional vector to obtain the weighted modal features, namely: ; In the formula , and These are the weighted one-dimensional modal feature vectors.

[0116] Compared to traditional data-driven self-attention mechanisms, the "physical feature-driven modal-level gating attention mechanism" proposed in this invention achieves deep integration of data and mechanisms at the network architecture level. For example, when a motor experiences a purely mechanical fault, the vibration-related parameters in the physical features change drastically, driving the gating network to automatically increase the vibration weight. When an electromagnetic-mechanical coupling fault occurs, the gating network will automatically shift its focus to the characteristic representing the coupling. Weight shifting. This dynamic weight adaptive allocation strategy, driven by real physical laws, alleviates the problem of decreased diagnostic accuracy caused by strong interference from a single sensor in extreme environments, enhances the model's noise resistance, and improves the physical interpretability of the fault diagnosis model.

[0117] Step 5: Fuse the weighted modal features with the physical feature vector to obtain the global fused features.

[0118] In one embodiment, before fusing the weighted modal features with the physical feature vector, a high-dimensional physical feature mapping network based on a two-layer perceptron is used to convert the physical feature vector into a vector with the same dimension as the weighted modal features.

[0119] In one embodiment, the high-dimensional transformation of physical feature information is carried out in the following manner.

[0120] In the multimodal fusion network architecture, input branches 1-3 are transformed into high-dimensional feature tensors with deep abstract semantics after multi-scale CNN feature extraction. However, these 16-dimensional vectors are only low-dimensional physical feature observations. Therefore, after the feature extraction of input branches 1-3 is completed, there is a significant dimensional mismatch and semantic gap between them and input branch 4. If the traditional direct concatenation method is used for fusion, it is easy for these 16-dimensional physical features to be submerged during backpropagation, thus losing the meaning of physical feature-driven features.

[0121] Based on this, this embodiment of the invention designs a high-dimensional physical feature mapping network based on a two-layer perceptron, aligning these numerical modes with the deep feature spaces of branches 1-3. This embodiment of the invention first expands the 16-dimensional physical features to capture the nonlinear combinations between these features, and then performs feature compression to align them with the feature dimensions of input branch 3.

[0122] The dual-layer perceptron structure used in this embodiment of the invention is as follows: Figure 9 As shown, the network mainly consists of a hidden layer with 64 neurons and an output layer with 32 neurons. Its layer-by-layer mapping process can be represented as follows: ; In the formula and The first The layer needs to learn the weight matrix and bias vector. Indicates the first The output of the layer, It can take the value 1 or 2. This is for batch normalization operations. It is a smooth GELU nonlinear activation function. This refers to the physical feature vector extracted in step two.

[0123] As can be seen, the dual-layer perceptron mapping designed in this embodiment of the invention is not a simple concatenation, but rather learns a nonlinear function from physical state to fault category through training. Simultaneously, the feature dimension is increased from... The mapping successfully transformed the original 16-dimensional physical feature vectors into a high-dimensional feature representation tensor. .

[0124] At this point, the dimensions of input branch 1 and input branch 2 are 64×1, and the dimension of input branch 3 is 32×1. The physical feature channels become 32×1 after high-dimensional mapping. In this embodiment of the invention, these processed features are directly concatenated during feature fusion, that is: ; In the formula The resulting global feature has a total dimension of 64 + 64 + 32 + 32 = 192.

[0125] Through the above-described fusion design, the physical features selected in this embodiment of the invention are not only used to generate the weights of each modality under different operating conditions and faults, but also directly participate in the final feature representation across layers. This dual-link physical guidance mechanism fundamentally avoids the semantic forgetting phenomenon that may occur in deep networks in multimodal high-dimensional spaces, and also improves the noise resistance and fault diagnosis robustness of the fault diagnosis model under extreme noise conditions.

[0126] Step 6: Output the fault category of the device to be diagnosed based on the global fusion features.

[0127] At the fault classification output level, this embodiment of the invention sets up a three-layer perceptron, such as... Figure 10 As shown, this classifier contains two hidden layers and one output layer, employing... To achieve a successive dimensionality reduction topology and prevent overfitting in deep network structures, this embodiment of the invention introduces a Dropout regularization operation with a deactivation rate of 0.5 after both hidden layers. The process can be represented as follows: ; In the formula and These are the output feature tensors of the hidden layers. , and , These are the weights and bias matrices of the two hidden layers, respectively; For batch normalization operations; This represents the smooth GELU activation function, and Dropout() represents the Dropout regularization operation.

[0128] Feature vector after three layers of perceptron The data is fed into the final fault classification output layer. Since there are 7 fault types in this embodiment of the invention, this layer has 7 nodes. The Softmax function is used to transform the logical values ​​of the output real numbers into a discrete probability distribution that sums to 1.

[0129] Figure 11 The overall network structure used in steps 2 to 6 is shown in detail. Figure 12 The overall process of the multimodal fault diagnosis method is presented in detail. From Figure 11 As can be seen, for the recursive image clusters of leakage magnetic signals and vibration subband signals in input branches 1 and 2, after multi-scale feature extraction and channel fusion in Section 3.4.2, the images are transformed into 64-channel high-dimensional features. Subsequently, the 64 channels are weighted using the CBAM attention mechanism in Section 5.4.1, and the channel dimension is compressed using GAP (Global Average Pooling) layer, ultimately compressing it into a 64-dimensional one-dimensional vector.

[0130] For the joint recursive graph of the leakage magnetic field-vibration coupling features of input branch 3, after multi-scale extraction, the output is a 32-channel deep feature, which is also transformed into a 32-dimensional one-dimensional feature vector through GAP operation.

[0131] For input branch 4, which is the selected 16-dimensional physical features, this branch has two uses: generating modal weights and serving as part of the global features. These are fed into two independent two-layer perceptrons. The first path generates the dynamic weights of input branches 1 to 3, which are modal-level gated attention weights driven by physical features. The second path maps the original physical quantities in high dimension to a 32-dimensional abstract physical tensor for the final feature fusion.

[0132] In the global feature fusion and fault classification stages, the network directly multiplies the generated gated attention weights with the feature vectors of branches 1 to 3, respectively, to adaptively weight the contribution of each modality feature. The weighted three vectors are then concatenated with the 32-dimensional physical tensor mapped by branch 4 to construct a 192-dimensional global fusion. Finally, this 192-dimensional global feature is processed through a three-layer perceptron and a Softmax classification layer to output the final fault category of the permanent magnet synchronous motor.

[0133] A physical feature-driven adaptive fusion multimodal fault diagnosis system according to an embodiment of the present invention includes: The data acquisition module is used to acquire the leakage flux signal and vibration signal of the motor; The physical feature vector calculation module is used to obtain the physical feature vectors of the leakage magnetic signal and the vibration signal. The physical feature vector is a vector composed of the physical features of the vibration signal, the physical features of the leakage magnetic signal, and the physical features of the cross-modal coupling. A feature extraction network is used to extract leakage magnetic signal features, vibration signal features, and cross-modal coupling features for the leakage magnetic signal and the vibration signal. The modal-level gated attention module is used to input the physical feature vector into the modal-level gated weight generation network to obtain the modal-level gated attention weights corresponding to the leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature. The modal-level gated attention weights are dynamically adjusted according to the changes in the physical feature vector. The leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature are weighted according to the modal-level gated attention weights to obtain weighted modal features. The fusion module is used to fuse the weighted modal features with the physical feature vector to obtain global fused features; The diagnostic module is used to output the fault category of the device to be diagnosed based on the global fusion features. Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multimodal fault diagnosis method driven by physical features and adaptive fusion, characterized in that, Including the following steps: Acquire the leakage flux signal and vibration signal of the motor; Obtain the physical feature vectors of the leakage flux signal and the vibration signal to characterize the motor operating conditions, fault states and modal reliability. The physical feature vectors are vectors composed of physical features of the vibration signal, physical features of the leakage flux signal, and physical features of cross-modal coupling. Based on the feature extraction network, leakage magnetic signal features, vibration signal features and cross-modal coupling features are extracted for the leakage magnetic signal and the vibration signal; The physical feature vector is input into a modal-level gated weight generation network to obtain modal-level gated attention weights corresponding to the leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature. The modal-level gated attention weights are dynamically adjusted according to the changes in the physical feature vector. The leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature are weighted according to the modal-level gated attention weights to obtain weighted modal features. The weighted modal features are fused with the physical feature vector to obtain global fused features; The fault diagnosis results of the device to be diagnosed are output based on the global fusion features.

2. The physical feature-driven adaptive fusion multimodal fault diagnosis method as described in claim 1, characterized in that, The steps for obtaining the physical feature vectors of the leakage flux signal and vibration signal, used to characterize the motor's operating conditions, fault states, and modal reliability, include: Construct a physical feature screening sample set, and construct the first candidate physical feature set; Normalize each candidate physical feature in the first candidate physical feature set; Calculate the variance of each candidate physical feature in the first candidate physical feature set on the physical feature screening sample set, and select the second candidate physical feature subset according to the variance magnitude; Based on the correlation between each candidate physical feature in the second candidate physical feature subset and the fault category, and the redundancy among the candidate physical features in the second candidate physical feature subset, a physical feature vector is obtained by filtering from the second candidate physical feature subset.

3. The multimodal fault diagnosis method driven by physical features and adaptive fusion as described in claim 2, characterized in that, The step of filtering physical feature vectors from the second candidate physical feature subset based on the correlation between each candidate physical feature in the second candidate physical feature subset and the fault category, and the redundancy among the candidate physical features in the second candidate physical feature subset, includes the following steps: S1, assuming the initially selected subset of physical features. Empty; S2: First, iterate through all candidate physical features in the second candidate physical feature subset, and directly select the candidate physical feature with the largest mutual information with the fault category as the first physical feature and store it in the physical feature subset. middle; S3, Traverse the second candidate physical feature subset and remove those already stored in the physical feature subset. The remaining candidate physical features will make this evaluation... The largest candidate physical feature is selected into the subset of physical features. ,in, For the second candidate physical feature subset, the first i Candidate physical features Mutual information between the fault label Y and the fault label Y For the second candidate physical feature subset, the first i Candidate physical features With the j Candidate physical features Mutual information between them t Number of rounds of filtering; S4. Repeat step S3 until a preset number of physical features are selected.

4. The multimodal fault diagnosis method driven by physical features and adaptive fusion as described in claim 1, characterized in that, The physical feature vector includes the maximum amplitude of the first sub-band after the three-layer wavelet packet decomposition of the magnetic flux leakage signal, the center frequency of the eighth sub-band after the three-layer wavelet packet decomposition of the vibration signal, the total harmonic distortion rate of the magnetic flux leakage signal, the ratio of the total harmonic distortion rate of the magnetic flux leakage signal to that of the vibration signal, the cross-modal energy coupling fluctuation index, the DC bias of the magnetic flux leakage signal, the standard deviation of the sixth sub-band after the three-layer wavelet packet decomposition of the vibration signal, the amplitude of the second harmonic of the fundamental frequency of the magnetic flux leakage signal, the amplitude of the third harmonic of the fundamental frequency of the magnetic flux leakage signal, the amplitude of the sideband of the sum of the fundamental frequency and the mechanical frequency of the magnetic flux leakage signal, the peak value of the cross power spectral density of the envelope of the magnetic flux leakage vibration, the amplitude of the fifth harmonic of the fundamental frequency of the magnetic flux leakage signal, the amplitude of the sideband of the difference between the fundamental frequency and the mechanical frequency of the magnetic flux leakage signal, the cross-entropy of the magnetic flux leakage vibration, the maximum amplitude of the sixth sub-band after the three-layer wavelet packet decomposition of the vibration signal, and the energy of the first sub-band after the three-layer wavelet packet decomposition of the magnetic flux leakage signal.

5. The multimodal fault diagnosis method driven by physical features and adaptive fusion as described in claim 1, characterized in that, The step of extracting leakage magnetic signal features and vibration signal features and cross-modal coupling features for the leakage magnetic signal and vibration signal based on the feature extraction network includes the following steps: Wavelet packet decomposition is performed on the leakage magnetic signal to extract leakage magnetic signal sub-band signals of multiple frequency bands. A leakage magnetic signal recurrence graph is generated for each leakage magnetic signal sub-band signal to obtain leakage magnetic signal recurrence graphs of multiple frequency bands. The vibration signal is decomposed by wavelet packet to extract vibration signal sub-band signals of multiple frequency bands. A vibration signal recursion graph is generated for each vibration signal sub-band signal to obtain vibration signal recursion graphs of multiple frequency bands. Calculate the joint recurrence matrix of the leakage magnetic signal and the vibration signal to generate a cross-modal recurrence graph; The leakage magnetic field signal recursion map of the multiple frequency bands, the vibration signal recursion map of the multiple frequency bands, and the cross-modal recursion map are respectively input into the feature extraction network corresponding to each mode to obtain the leakage magnetic field signal features, vibration signal features and cross-modal coupling features.

6. The physical feature-driven adaptive fusion multimodal fault diagnosis method as described in claim 5, characterized in that, The steps for calculating the joint recurrence matrix of the leakage magnetic signal and the vibration signal to generate a cross-modal recurrence graph include: The thresholdless Gaussian kernel distance matrix is ​​calculated for both the leakage magnetic field signal and the vibration signal. Normalization is performed on each thresholdless Gaussian kernel distance matrix, and then the joint recursive matrix is ​​obtained by element-wise multiplication. The formula for element-wise multiplication is as follows: ,in, Represents the normalized joint recursive matrix. Line number Column elements, The thresholdless Gaussian kernel distance matrix of the leakage magnetic field signal after normalization represents the first... Line number Column elements, The thresholdless Gaussian kernel distance matrix of the vibration signal after normalization represents the first... Line number Column elements, This indicates element-wise multiplication, converting the joint recursive matrix into a cross-modal recursive graph.

7. The physical feature-driven adaptive fusion multimodal fault diagnosis method as described in claim 5, characterized in that, Before extracting leakage magnetic signal features, vibration signal features, and cross-modal coupling features, an equal-angle sampling grid is constructed based on the rotor position signal to resample the leakage magnetic signal and vibration signal from the time domain to the angle domain.

8. The multimodal fault diagnosis method driven by physical features and adaptive fusion as described in claim 1, characterized in that, The modal-level gated weight generation network includes: A two-layer perceptron is used to first expand the dimension of the physical feature vector using a hidden layer to generate a feature tensor, and then use an output layer to compress the feature tensor to reduce the dimension to a three-dimensional vector. The function layer is used to normalize the three-dimensional vector and generate modal-level gated attention weights for the three modes. And satisfy In the formula These are the modal-level gating attention weights corresponding to the leakage magnetic signal characteristics. The modal-level gated attention weights corresponding to the vibration signal characteristics are... The modal-level gating attention weights are those corresponding to the cross-modal coupling features.

9. The multimodal fault diagnosis method driven by physical features and adaptive fusion as described in claim 1, characterized in that, Before fusing the weighted modal features with the physical feature vector, the physical feature vector is converted into a vector with the same dimension as the weighted modal features using a high-dimensional physical feature mapping network based on a two-layer perceptron.

10. A multimodal fault diagnosis system driven by physical features and adaptive fusion, characterized in that, include: The data acquisition module is used to acquire the leakage flux signal and vibration signal of the motor; The physical feature vector calculation module is used to obtain physical feature vectors of the leakage flux signal and the vibration signal to characterize the motor operating conditions, fault states and modal reliability. The physical feature vector is a vector composed of physical features of the vibration signal, physical features of the leakage flux signal and physical features of cross-modal coupling. A feature extraction network is used to extract leakage magnetic signal features, vibration signal features, and cross-modal coupling features for the leakage magnetic signal and the vibration signal. The modal-level gated attention module is used to input the physical feature vector into the modal-level gated weight generation network to obtain the modal-level gated attention weights corresponding to the leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature. The modal-level gated attention weights are dynamically adjusted according to the changes in the physical feature vector. The leakage magnetic signal feature, the vibration signal feature, and the cross-modal coupling feature are weighted according to the modal-level gated attention weights to obtain weighted modal features. The fusion module is used to fuse the weighted modal features with the physical feature vector to obtain global fused features; The diagnostic module is used to output the fault diagnosis results of the device to be diagnosed based on the global fusion features.

Images