Intelligent Cleaning and Feature Extraction System and Method for Multimodal Industrial Data

Abstract

This invention discloses an intelligent cleaning and feature extraction system and method for multimodal industrial data, relating to the field of equipment condition monitoring technology. It addresses the problems of inaccurate time alignment of multi-source heterogeneous signals, fault features being easily masked by background noise, unclear causal chains under changing operating conditions, and poor feature stability. First, the matching window is dynamically adjusted based on the main vibration frequency of the rotating equipment, and the temperature signal delay is calculated using the material's thermal expansion coefficient to achieve modal alignment. Then, the fault-sensitive frequency band is solved using the bearing housing dynamic equation, a frequency band protection window is constructed, and frequency domain filtering and gradient truncation operations are performed to extract high-frequency features of microcracks. Next, the fault propagation path is identified by combining image clarity and envelope spectrum kurtosis, and the frequency domain window bandwidth is dynamically reduced based on real-time load. Finally, the feature vectors are reconstructed into phase space, and when the rate of curvature change or temperature drift exceeds limits, parameter updates and frequency band readjustment feedback are triggered, improving system stability and adaptability.

Description

Technical Field

[0001] This invention relates to the field of equipment condition monitoring technology, specifically to an intelligent cleaning and feature extraction system and method for multimodal industrial data. Background Technology

[0002] Against the backdrop of the continuous development of intelligent manufacturing and the Industrial Internet, rotating equipment (such as motors, bearings, and gearboxes) generates a large amount of heterogeneous sensing data during operation, covering multiple modes including vibration, images, temperature, and current. The comprehensive analysis capability of this multimodal data has become a crucial support for improving the predictive maintenance capabilities and safe operation level of equipment. However, due to significant differences in sampling frequency, response mechanisms, and physical coupling relationships among different modal data, time alignment, feature extraction, and causal analysis face significant challenges. Especially under high load and variable operating conditions, the nonlinear coupling effect between data is more pronounced, and traditional signal processing methods have limited effectiveness in accurately extracting key state features.

[0003] Most existing multimodal industrial data processing methods are based on fixed window alignment strategies, ignoring the dynamic changes in equipment vibration frequencies. This makes it difficult to compensate for the time reference offset of different modal data in real time. Meanwhile, frequency domain filtering methods rely on full-band filtering models, which can easily misjudge weak high-frequency components as noise and filter them out when processing non-steady-state faults (such as microcracks and early fatigue cracks), affecting the accuracy of early fault identification. Furthermore, the lack of a modeling compensation mechanism for the hysteresis characteristics of material thermal response leads to potential discrepancies between slow-response signals such as temperature and fast-response signals, further weakening the consistency and discriminative power of feature fusion. In highly dynamic operating conditions, existing models are slow to respond to sudden changes in operating conditions, and the lack of a real-time adjustment mechanism for the frequency window bandwidth results in decreased impact path identification capabilities, making it difficult to guarantee stability and adaptability. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an intelligent cleaning and feature extraction system and method for multimodal industrial data, solving the problems mentioned in the background.

[0005] To achieve the above objectives, this invention provides the following technical solution: an intelligent cleaning and feature extraction system for multimodal industrial data, comprising the following modules: a time alignment module, a feature decoupling module, a working condition causal analysis module, and a feature stability optimization module. The time alignment module dynamically adjusts the matching window size of the time warping algorithm based on the main vibration frequency of the rotating equipment, and calculates the temperature signal transmission delay based on the thermal expansion characteristics of the equipment material. The feature decoupling module receives the delay and synchronizes the multimodal data time reference, solves for the fault-sensitive frequency band using the bearing housing vibration response equation, constructs a frequency band protection window based on the harmonic distribution characteristics of metal fatigue cracks, and performs frequency domain operations within the window. Vibration artifacts are removed by filtering, and a gradient cutoff layer is set in the non-window region to retain the high-frequency features of microcracks. The working condition causal analysis module is used to receive the high-frequency features of microcracks from the gradient cutoff layer, detect the phase shift of the cross-correlation function with image sharpness during sudden changes in rotational speed, identify the propagation path of impact faults through envelope spectrum kurtosis, and dynamically shrink the frequency domain window bandwidth in combination with real-time load data. The feature stability optimization module is used to construct the motion trajectory of bearing raceway features in phase space. When the rate of change of trajectory curvature exceeds the material fatigue coefficient correlation threshold, a frequency band window readjustment command is generated and fed back to the feature decoupling module. When temperature fluctuations cause trajectory drift, the time delay compensation parameters are updated to the dynamic timing alignment module.

[0006] Furthermore, the timing alignment module includes the following steps: real-time acquisition of vibration signals from rotating equipment, extraction of dominant vibration frequency components, and periodic adjustment of the matching window size of the time warping algorithm based on the frequency; obtaining the coefficient of thermal expansion from the equipment material database, calculating the thermal deformation of the metal structure in combination with temperature sensor data, and deriving the time delay compensation of the temperature signal relative to the vibration reference based on the propagation speed of sound waves in the metal medium.

[0007] Furthermore, the specific process of receiving the delay amount and synchronizing the multimodal data time reference, and solving the fault-sensitive frequency band through the bearing housing vibration response equation is as follows: calibrate the time deviation between the image frame and the vibration spectrum through the time delay compensation value to generate a time-aligned multimodal data stream; establish a set of differential equations for bearing housing dynamics, solve the system feature matrix through random subspace identification, extract the characteristic frequency components related to rolling element damage, calculate the fundamental frequency of the outer ring fault characteristics and its harmonic distribution range based on the bearing geometric parameters, and determine the fault-sensitive frequency band.

[0008] Furthermore, the following logical process is employed: A frequency band protection window is constructed based on the harmonic distribution characteristics of metal fatigue cracks. Frequency domain filtering is performed within the window to remove vibration artifacts, while a gradient cutoff layer is set in the non-window region to retain the high-frequency characteristics of microcracks. The process is as follows: A protection window covering multiple harmonic attenuation regions is constructed centered on the fundamental frequency of the fault-sensitive frequency band. The window boundary is determined based on the attenuation characteristics of crack harmonic energy. An orthogonal projection operator is constructed based on the characteristic matrix of the bearing housing vibration response equation. Orthogonal projection operations are applied to the spectral components within the window to suppress vibration coupling components. When the correlation coefficient between the frequency energy attenuation rate and the entropy change rate of the gray-level co-occurrence matrix of the synchronously acquired equipment surface image is lower than a set threshold, the corresponding frequency region is marked and band-stop filtering is performed. The frequency domain feature tensor is input into the deep learning network. A mask matrix is ​​constructed based on the frequency dimension. During backpropagation, a mask operation is performed on the gradient tensor, allowing only gradient propagation in the frequency domain region outside the window. A gradient cutoff layer is set in the region outside the window to limit the interference of non-target frequency bands on the model training process. Frequency domain regularization is then performed on the masked features.

[0009] Furthermore, the specific process of receiving the high-frequency characteristics of the gradient truncated layer and detecting the phase shift of the cross-correlation function with image sharpness during sudden speed changes is as follows: Based on the high-frequency characteristics of the microcracks in the gradient truncated layer, combined with the rising edge marker of the speed sensor pulse to lock the speed change event, the vibration envelope signal and the synchronous image sharpness sequence within the dynamic time window are extracted with the event time as the center; complex analytical wavelet transform is performed on the data within the window to extract the instantaneous principal phase angle of the vibration envelope component and the image sharpness component in the complex domain, the instantaneous phase difference between the two components before and after the speed change point is calculated, and the phase difference is converted into the relative displacement offset between the bearing raceway and the visual monitoring area by combining the gear meshing transmission ratio parameter of the equipment.

[0010] Furthermore, the specific process of identifying the propagation path of impact faults by envelope spectrum kurtosis and dynamically shrinking the frequency domain window bandwidth in combination with real-time load data is as follows: the vibration signal is demodulated across the entire frequency band using complex Morlet wavelets to generate a scale energy distribution matrix; the kurtosis value of the envelope spectrum at each scale is calculated, and when the kurtosis peak value of a specific frequency band exceeds the material yield strength correlation threshold, it is marked as an impact transmission path; a negative exponential correlation function between the frequency domain analysis window bandwidth and the real-time load is established, and the analysis window bandwidth is dynamically compressed according to the load value within the marked impact path frequency band.

[0011] Furthermore, the motion trajectory of the bearing raceway features in phase space is constructed. When the rate of change of trajectory curvature exceeds the material fatigue coefficient correlation threshold, the specific process of generating a frequency band window readjustment instruction and feeding it back to the feature decoupling module is as follows: the high-frequency feature vector of the microcrack is reconstructed into phase space through time delay embedding, and a three-dimensional feature trajectory is generated by dimensionality reduction through a local linear embedding algorithm; the rate of change of curvature of the trajectory is calculated, and when the rate of change of curvature exceeds the set change range, the window bandwidth amplification is calculated according to the crack depth change rate, and a readjustment instruction containing the target bandwidth value is generated and fed back to the feature decoupling module.

[0012] Furthermore, the specific process of updating the time delay compensation parameters to the dynamic timing alignment module when temperature fluctuations cause trajectory drift is as follows: the temperature sensor data is mapped to the feature space through the thermal expansion coefficient matrix, and the temperature-sensitive component is separated from the motion trajectory; when the trajectory drift exceeds the allowable error band of material thermal deformation, the partial derivative of the drift with respect to the time delay parameter is calculated, the learning step size of the gradient descent method is set based on the material thermal conduction rate, the time delay compensation is iteratively updated and fed back to the timing alignment module.

[0013] A method for intelligent cleaning and feature extraction of multimodal industrial data includes the following steps: S1. Dynamically adjust the matching window size of the time warping algorithm based on the main vibration frequency of the rotating equipment, and calculate the temperature signal transmission delay based on the thermal expansion characteristics of the equipment material; S2. Receive the delay and synchronize the multimodal data time reference, solve the fault-sensitive frequency band through the bearing housing vibration response equation, construct a frequency band protection window based on the harmonic distribution characteristics of metal fatigue cracks, perform frequency domain filtering to remove vibration artifacts within the window, and set a gradient truncation layer in the non-window area to retain the high-frequency features of microcracks; S3. Receive the high-frequency features of microcracks from the gradient truncation layer, detect the phase shift of the cross-correlation function with image clarity during sudden changes in rotational speed, identify the propagation path of impact faults through envelope spectrum kurtosis, and dynamically shrink the frequency domain window bandwidth in combination with real-time load data; S4. Construct the motion trajectory of the bearing raceway features in phase space. When the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold, generate a frequency band window readjustment command and feed it back to the feature decoupling module. When temperature fluctuations cause trajectory drift, update the time delay compensation parameters to the dynamic timing alignment module.

[0014] The present invention has the following beneficial effects:

[0015] (1) The intelligent cleaning and feature extraction system for multimodal industrial data, by introducing a dynamic matching window adjustment mechanism driven by the main vibration frequency, can adaptively adjust the window size of the time warping algorithm, significantly improving the accuracy and robustness of multimodal data time alignment under unstable operating conditions; and on this basis, combined with material thermal expansion modeling, the time delay compensation of temperature signal is calculated, so that the asynchronous error of sensor under thermal interference can be effectively corrected. Based on the bearing housing vibration response dynamic equation, the fault-sensitive frequency band is accurately solved, and a frequency band protection window is constructed through harmonic distribution to effectively suppress artifact interference in vibration signal. At the same time, the frequency domain gradient truncation mechanism is used to retain high-frequency microcrack features, significantly enhancing the system's ability to perceive early micro-damage.

[0016] (2) Intelligent cleaning and feature extraction method for multimodal industrial data. Based on the phase shift detection strategy of the cross-correlation function between microcrack high-frequency features and image clarity, heterogeneous signal linkage analysis under sudden speed change events is realized. Combined with the envelope spectrum kurtosis function to identify the impact fault path, the frequency domain mapping model of fault propagation is effectively constructed. The frequency domain analysis window bandwidth is dynamically adjusted through the load sensing mechanism, thereby improving the system's ability to extract key features under nonlinear operating conditions. The high-frequency features of the raceway are mapped to the phase space. Combined with the correlation model between the curvature change rate and the material fatigue coefficient, the abnormal state is autonomously judged and the frequency band window readjustment command is triggered to realize the adaptive closed-loop optimization of the feature extraction process. At the same time, the time delay parameter is corrected in real time based on the trajectory offset caused by thermal drift, which improves the long-term stability and generalization ability of the whole method under complex thermo-mechanical coupling conditions.

[0017] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0018] Figure 1 This is a flowchart of the intelligent cleaning and feature extraction system for multimodal industrial data according to the present invention.

[0019] Figure 2 This is a flowchart of the intelligent cleaning and feature extraction method for multimodal industrial data according to the present invention. Detailed Implementation

[0020] This application's embodiments address key technical problems in the prior art, such as inaccurate time alignment of multi-source heterogeneous signals, fault features being easily masked by background noise, unclear causal chains under changing operating conditions, and poor feature stability, through an intelligent cleaning and feature extraction system and method for multimodal industrial data.

[0021] The overall concept of the solution in this application embodiment is as follows:

[0022] First, the timing alignment algorithm is dynamically driven by the main vibration frequency of the rotating equipment to adaptively adjust the matching window size, and the time delay of the temperature signal is compensated by the material thermal expansion model to improve the synchronization accuracy of multimodal data.

[0023] Secondly, by extracting fault-sensitive frequency bands through bearing housing dynamic modeling, and constructing a frequency band protection window by combining the harmonic distribution of metal fatigue cracks, vibration artifacts are effectively suppressed. At the same time, the high-frequency components of microcracks are retained through the frequency domain gradient truncation strategy, which enhances the ability to identify early faults.

[0024] Subsequently, a cross-correlation model between image and vibration signal was constructed based on high-frequency features. The impact propagation path was identified by combining envelope spectrum kurtosis and load sensing mechanism, thereby realizing dynamic modeling of causal relationships under complex working conditions.

[0025] Finally, a high-dimensional feature trajectory is constructed using the phase space embedding algorithm to determine the rate of curvature change. When it exceeds the fatigue-related threshold, the frequency band window is retuned to further improve the model's response to feature drift. At the same time, a thermal compensation feedback mechanism is introduced to ensure the long-term operational stability of the system in a thermo-coupling environment.

[0026] Please see Figure 1 This invention provides a technical solution: an intelligent cleaning and feature extraction system for multimodal industrial data, comprising the following modules: a time alignment module, a feature decoupling module, a working condition causal analysis module, and a feature stability optimization module. The time alignment module dynamically adjusts the matching window size of the time warping algorithm based on the main vibration frequency of the rotating equipment, and calculates the temperature signal transmission delay based on the thermal expansion characteristics of the equipment material. The feature decoupling module receives the delay and synchronizes the multimodal data time reference, solves for the fault-sensitive frequency band through the bearing housing vibration response equation, constructs a frequency band protection window based on the harmonic distribution characteristics of metal fatigue cracks, and performs frequency domain filtering within the window to clear the data. Vibration artifacts are identified by setting a gradient cutoff layer in the non-window region to preserve the high-frequency features of microcracks. The working condition causal analysis module receives the high-frequency features of microcracks from the gradient cutoff layer, detects the phase shift of the cross-correlation function with image sharpness during sudden speed changes, identifies the propagation path of impact faults through envelope spectrum kurtosis, and dynamically shrinks the frequency domain window bandwidth in combination with real-time load data. The feature stability optimization module constructs the motion trajectory of bearing raceway features in phase space. When the rate of change of trajectory curvature exceeds the material fatigue coefficient correlation threshold, a frequency band window readjustment command is generated and fed back to the feature decoupling module. When temperature fluctuations cause trajectory drift, the time delay compensation parameters are updated to the dynamic timing alignment module.

[0027] In this implementation scheme, the time alignment module is used to achieve accurate alignment of multi-source heterogeneous signals (such as vibration, temperature, and images) in the time dimension, which is the foundation for ensuring the correctness of subsequent feature processing. Dynamic adjustment of the dominant vibration frequency: By monitoring the changes in the dominant frequency (i.e., the dominant vibration frequency of the equipment) during the operation of rotating equipment (such as motors or bearings), the matching window size in the time warping algorithm is adaptively adjusted, allowing signals at different time scales to be effectively compared and aligned. Matching window: In the signal alignment algorithm, this refers to the sliding time interval used to calculate the similarity between two signals. Dynamically adjusting its size improves adaptability to non-stationary signals. Thermal expansion delay compensation: Based on the thermal expansion model of the equipment component materials, the change in the length of the sensor signal transmission path caused by temperature changes is calculated, thereby calculating the time delay offset of the temperature signal and compensating it back to the main time axis. Thermal expansion characteristics: This refers to the linear or volumetric expansion of materials due to temperature increases, which causes a certain time lag in the sensor's received data. Feature Decoupling Module: This module separates key fault-related features from multimodal data while suppressing background noise and non-target frequency band interference, improving the accuracy and interpretability of diagnostic features. Synchronization Time Base: The module first receives the aforementioned time delay compensation results to achieve time base consistency for multi-source signals. Fault-Sensitive Frequency Band Extraction: Based on the vibration response equation of the bearing housing (often modeled by a mass-resistance-elasticity system), the resonant frequency band of the system under excitation is solved as the fault-sensitive frequency range. Frequency Band Protection Window Construction: Utilizing the harmonic distribution characteristics of metal fatigue cracks—that is, fatigue cracks often generate harmonic signals with specific octave frequencies—a frequency band window covering these harmonics is constructed to protect fault-related frequencies from being filtered out. Frequency Domain Filtering and Gradient Truncation: Within the above window, frequency domain filtering is used to remove artifacts (such as structural resonance and environmental interference); outside the window, a gradient truncation layer is set to limit the propagation rate of spectral energy, thereby preserving the high-frequency pulse components generated by microcracks. Gradient cutoff layer: This is a frequency domain processing mechanism that prevents small signals from being averaged or over-smoothed by setting a threshold for energy changes, thereby improving the ability to identify weak fault features. Operating condition causal analysis module: This module focuses on fault evolution path analysis under complex operating conditions. Combining the mutual information of image and sensor data, it reconstructs the propagation path of impact-type faults to achieve causal relationship identification. Microcrack high-frequency feature input: This module receives high-frequency features output from the previous module as the basis for analysis. Image clarity cross-correlation function: When there are sudden changes in rotational speed or operating conditions, the clarity of the equipment monitoring image will change accordingly. Cross-correlation calculation is performed with the vibration signal to measure the phase shift and identify potential fault locations. Cross-correlation function: Used to quantify the similarity of two signals at different time offsets, it is an effective means of realizing time-series causal reasoning.Envelope Spectrum Kubularity Analysis: Envelope analysis is used to extract the amplitude envelope of the impact signal, and kurtosis is further combined to measure the sharpness of the signal and identify strong impact fault paths. Kubularity measures whether the signal has impulse properties; a higher value indicates a more significant impact. Adaptive Frequency Domain Window Shrinking: Based on current load data, the frequency window width is dynamically adjusted to automatically enhance the resolution capability of the impact frequency band during sudden load changes. Feature Stability Optimization Module: This module is used to improve the robustness and stability of the system to changes in fault characteristics during long-term operation, solving the feature drift problem caused by changes in operating conditions. Phase Space Trajectory Construction: The state characteristics (such as vibration and acceleration) of key components such as bearing raceways are projected into phase space to form multidimensional trajectory curves for tracking feature evolution trends. Phase Space Reconstruction: A method in nonlinear dynamics, this method expands a one-dimensional time series into a multidimensional space through delayed embedding to capture the state evolution law of the system. Curvature Change Detection: The rate of curvature change of the calculated trajectory is considered to have entered an early failure state when it exceeds a set threshold related to material fatigue life. This triggers a frequency band window readjustment command, which is fed back to the feature decoupling module to update the protection strategy. Temperature Drift Compensation Feedback: If trajectory drift is detected to be primarily caused by temperature fluctuations, an update command is generated and fed back to the timing alignment module to readjust the delay compensation parameters, ensuring that the feature does not deviate abnormally with ambient temperature.

[0028] Specifically, the timing alignment module includes the following steps: real-time acquisition of vibration signals from rotating equipment, extraction of dominant vibration frequency components, and periodic adjustment of the matching window size of the time warping algorithm based on the frequency; obtaining the coefficient of thermal expansion from the equipment material database, calculating the thermal deformation of the metal structure in combination with temperature sensor data, and deriving the time delay compensation of the temperature signal relative to the vibration reference based on the propagation speed of sound waves in the metal medium.

[0029] In this implementation scheme, vibration signals of the rotating equipment are acquired in real time, and the dominant vibration frequency component is extracted. Specifically, vibration signals of the rotating equipment are acquired in real time using an accelerometer or vibration sensor, and the dominant vibration frequency, i.e., the fundamental frequency of the equipment, is extracted using time-frequency analysis methods (such as Short-Time Fourier Transform (STFT) or adaptive spectral estimation). This fundamental frequency reflects the rotation period of the equipment and serves as the time reference for subsequent time warping algorithms. The matching window size of the time warping algorithm is periodically adjusted according to the dominant vibration frequency: the matching window size W is the sliding window length used to align two signal time points in the time warping algorithm; dynamically adjusting the window length improves the accuracy of timing alignment. The formula is as follows: Parameter description: W: Matching window size, in seconds; k: Window coefficient, a constant that reflects the window length relative to the vibration period; The dominant oscillation period, in seconds; Dominant vibration frequency, measured in Hertz (Hz). The coefficient of thermal expansion is obtained from the equipment material database, and the thermal deformation of the metal structure is calculated using temperature sensor data. The metal structure of the equipment undergoes linear expansion or contraction under temperature changes; this thermal deformation affects the time transmission characteristics of the vibration signal and requires compensation. The formula is as follows: Parameter description: : Thermal deformation length, in meters; The coefficient of linear expansion of metallic materials, expressed as the relative change per degree Celsius. ); Initial length of the metal structure, in meters (m); Temperature change, relative to a reference temperature, in degrees Celsius. The time delay compensation for the temperature signal relative to the vibration reference is derived based on the propagation speed of sound waves in a metallic medium. The temperature signal experiences a time delay during transmission through the metallic structure, which is related to thermal deformation and the sound wave propagation speed. This time delay needs to be calculated to compensate for time deviations in timing alignment. The formula is as follows: Parameter description: Temperature signal transmission delay, in seconds (s); Initial length of the metal structure, in meters (m); : Thermal deformation length, in meters (m); The speed of sound propagation in a metallic medium, measured in meters per second (m / s), varies depending on the material type; for example, steel is approximately 5900 m / s. In wind turbine operation monitoring, the timing alignment module collects vibration signals in real time using a triaxial accelerometer mounted on the main shaft bearing housing, and combines this data with temperature sensor data from the nacelle for synchronization. The system first uses short-time Fourier transform to extract the dominant vibration frequencies of the turbine under different load conditions, and then adaptively adjusts the matching window size based on these frequencies. This ensures that even when sudden changes in wind speed cause speed fluctuations of ±12%, the synchronization accuracy of multi-mode signals remains high. When the turbine starts up from a low temperature in winter and heats up to its rated operating temperature, the metal bearing housing undergoes slight deformation due to thermal expansion. This module obtains the thermal expansion coefficient of steel from the equipment material database, calculates the thermal deformation based on real-time temperature data, and further derives the resulting temperature signal transmission delay. In this embodiment, the temperature compensation mechanism effectively eliminates the signal phase shift caused by a 40°C temperature difference, reducing the phase error of subsequent feature extraction to 1 / 5 of its original value. Through a window adaptive mechanism driven by the main vibration frequency, the timing alignment error of multimodal signals under complex conditions such as rotational speed changes and load fluctuations is reduced from ±8ms to ±1.5ms, effectively ensuring the temporal consistency of feature extraction. A compensation mechanism combining material parameters and temperature data eliminates the phase shift of high-frequency signals caused by changes in the sound wave propagation path due to temperature rise, ensuring phase locking of cross-modal data even under high sampling rates. Because the time base of the input signals is more unified, the feature decoupling module can more accurately align subtle fault features between vibration, image, and temperature signals, improving the accuracy of early microcrack identification and significantly reducing the false detection rate caused by signal misalignment.

[0030] Specifically, the process of receiving the delay and synchronizing the multimodal data time reference, and solving the fault-sensitive frequency band through the bearing housing vibration response equation is as follows: calibrate the time deviation between the image frame and the vibration spectrum through the time delay compensation value to generate a time-aligned multimodal data stream; establish a set of dynamic differential equations for the bearing housing, solve the system feature matrix through random subspace identification, extract the characteristic frequency components related to rolling element damage, calculate the fundamental frequency of the outer ring fault characteristics and its harmonic distribution range based on the bearing geometric parameters, and determine the fault-sensitive frequency band.

[0031] In this implementation scheme, the time deviation between the image frame and the vibration spectrum is calibrated using a time delay compensation value to generate a time-aligned multimodal data stream. Based on the time delay compensation value calculated by the time alignment module, the image acquisition time point is corrected to the vibration signal time reference, achieving precise time synchronization between the two data modes and forming a time-aligned multimodal data stream, facilitating subsequent joint feature analysis. A set of dynamic differential equations for the bearing housing is established, and the system feature matrix is ​​solved using stochastic subspace identification. A dynamic model of the bearing housing system is established to describe its vibration response characteristics. The experimental or acquired vibration signal is processed using the stochastic subspace identification method to obtain the system state space matrix, thereby extracting the system's dynamic features. Formula representation (dynamic equations): Parameter description: M: mass matrix; C: damping matrix; K: stiffness matrix; x(t): displacement vector, function at time t; : Velocity vector, i.e., the first time derivative of displacement; : Acceleration vector, i.e., the second time derivative of displacement; External excitation force vector, a function of time t. Formula expression (obtaining the state-space equation through random subspace identification): Parameter description: : System state vector; Input vector (excitation signal); A: Output vector (measurement signal); B: State transition matrix, describing the dynamic characteristics of the system; C: Input matrix; D: Direct transfer matrix. Extracting characteristic frequency components related to rolling element damage: Based on the obtained state space matrix, characteristic frequency components related to rolling element damage are extracted through spectral analysis and mode decomposition, serving as key parameters for fault diagnosis. Calculating the fundamental frequency and harmonic distribution range of the outer ring fault based on bearing geometric parameters, and determining the fault-sensitive frequency band: Using the bearing's geometric dimensions and rotational frequency parameters, the fundamental frequency and harmonic range generated by the outer ring defect are calculated, thereby determining the frequency band in the spectrum sensitive to this fault. Formula expression (calculation of the fundamental frequency of the outer ring defect): Parameter description: :Outer ring fault fundamental frequency; : Number of rolling elements; : Bearing rotation frequency; : Rolling element diameter; : Bearing pitch circle diameter; Contact angle. Formula expression (harmonic distribution range): Parameter description: Set of fault-sensitive frequency bands; Harmonic number; :Bandwidth offset, used to cover the extended range of harmonic energy; The system first uses the delay compensation value output by the timing alignment module to precisely synchronize the surface crack image frames acquired by the high-speed industrial camera with the spectral data acquired by the vibration sensor. The synchronized multimodal data stream ensures a strict correspondence between image features and vibration characteristics within the same time segment, avoiding misalignment of fault signs due to sampling delay. After multimodal data alignment, the system establishes a set of dynamic differential equations for the bearing housing and uses a random subspace identification method to identify the system's state-space matrix from the vibration response signals acquired on-site. This process extracts the structure's natural frequency and damping characteristics directly from the natural excitation of the operating conditions without additional external excitation, reducing the interference of on-site testing on production. Combining the bearing's geometric parameters and rotational speed, the system automatically calculates the characteristic fundamental frequency and harmonic distribution range of the outer ring fault. For example, in the actual test of the main bearing of this continuous casting machine (8 rolling elements, contact angle 0°, pitch circle diameter 320mm), the system can quickly pinpoint the sensitive frequency range of the outer ring defect and extract the energy characteristics of these intervals in the spectral analysis. This not only reduces the amount of data processing in irrelevant frequency bands but also significantly improves the signal-to-noise ratio of weak fault signals. The time deviation between image frames and vibration spectra is compensated at the nanosecond level, enhancing the effectiveness of subsequent multimodal feature fusion and reducing fault feature mismatch caused by acquisition delays. Combining system features extracted using random subspace identification with bearing geometry calculations results in a fault-sensitive frequency band locking error of less than ±0.2Hz, significantly better than the ±1Hz error range of traditional empirical formulas. By focusing energy analysis on the fault-sensitive frequency band, the system can still detect initial microcracks in the outer ring even under high noise background conditions (SNR < 0dB).

[0032] Specifically, the logical process of constructing a frequency band protection window based on the harmonic distribution characteristics of metal fatigue cracks, performing frequency domain filtering to remove vibration artifacts within the window, and setting a gradient cutoff layer in the non-window region to retain the high-frequency characteristics of microcracks is as follows: A protection window covering multiple harmonic attenuation regions is constructed centered on the fundamental frequency of the fault-sensitive frequency band. The window boundary is determined based on the attenuation characteristics of crack harmonic energy. An orthogonal projection operator is constructed based on the characteristic matrix of the bearing housing vibration response equation. Orthogonal projection operations are applied to the spectral components within the window to suppress vibration coupling components. When the correlation coefficient between the frequency point energy attenuation rate and the entropy change rate of the gray-level co-occurrence matrix of the synchronously acquired equipment surface image is lower than a set threshold, the corresponding frequency region is marked and band-stop filtering is performed. The frequency domain feature tensor is input into the deep learning network, and a mask matrix is ​​constructed based on the frequency dimension. During backpropagation, a mask operation is performed on the gradient tensor, allowing only gradient propagation in the frequency domain region outside the window. A gradient cutoff layer is set in the region outside the window to limit the interference of non-target frequency bands on the model training process. Frequency domain regularization is performed on the masked features.

[0033] In this implementation plan, a protection window covering multiple harmonic attenuation regions is constructed, and the window boundaries are determined: Centered on the fundamental frequency of the fault characteristic band, and based on the energy attenuation law of metal fatigue crack harmonics, a frequency band protection window covering the fundamental frequency and its multiple harmonic attenuation intervals is constructed to ensure the integrity of the core fault characteristic frequency. The formula (protection window frequency band boundary) is as follows: Parameter description: :No. One protection window frequency band; :No. One harmonic frequency, based on an integer multiple of the fundamental frequency; :No. One harmonic ordinal number; Fault characteristic fundamental frequency; :No. The half-width of the frequency band is determined by the attenuation of each harmonic energy; The total number of protection windows covers multiple harmonics. An orthogonal projection operator is constructed to suppress spectral components within the window: Based on the characteristic matrix obtained from the bearing housing vibration response equation, an orthogonal projection operator is constructed to suppress coupling components and artifacts in the vibration signal in the frequency domain, enhancing the saliency of fault characteristics. Formula expression (orthogonal projection operator): Parameter description: : Orthogonal projection operator matrix; The identity matrix has the same dimensions as the feature matrix. The principal eigenvector matrix in the characteristic matrix, with column vectors orthogonally normalized; : The transpose of . Applied to spectral components: Parameter description: : Original spectral component vector; : Spectral components after projection suppression. Artifact frequency bands are screened based on the correlation between the energy attenuation rate at each frequency point and the rate of change of the image gray-level co-occurrence matrix entropy, and band-stop filtering is performed: The correlation coefficient between the energy attenuation rate at each frequency point in the spectrum and the rate of change of the gray-level co-occurrence matrix entropy of the synchronously acquired device surface image is calculated. When this correlation coefficient is lower than a preset threshold, the frequency region is considered to contain artifacts, and band-stop filtering is performed. Formula expression (correlation coefficient): Parameter description: : The correlation coefficient at the l-th frequency point; : The energy decay rate sequence at frequency l; The sequence of the entropy change rate of the image gray-level co-occurrence matrix corresponding to the l-th frequency point; Covariance function; : Standard deviation; : The standard deviation of . Judgment criterion: If Then, band-stop filtering is performed on that frequency point. Parameter description: : Set the correlation coefficient threshold. Construct a frequency-dimensional mask matrix to restrict gradient propagation and implement a gradient cutoff layer: Input the frequency domain feature tensor into the deep learning network, construct a mask matrix based on the protection window, and mask the frequency band gradients within the window, allowing only gradients outside the window to pass through during backpropagation. This avoids non-target frequency bands interfering with model training and improves the model's sensitivity to high-frequency features of microcracks. Formula representation (mask matrix definition): Parameter description: : Mask matrix elements in the frequency dimension; : Frequency point; The aforementioned protection window frequency band. Mask gradient calculation: Parameter description: The loss function applies to the frequency points in the frequency domain feature tensor. The gradient; : Feature values ​​corresponding to the frequency points. Regularization is performed on the masked frequency domain features: To improve model training stability and reduce noise interference between frequency domain features, frequency domain regularization constraints are adopted to enhance the sparsity or smoothness of the features and ensure the effectiveness of the gradient cutoff layer. In the system bearing condition monitoring application, the system first constructs a protection window to cover key harmonic energy areas based on the fundamental frequency of the outer ring fault characteristics and its multiple harmonic attenuation intervals. For example, for a wind turbine main bearing with a rated speed of 1200 rpm and an outer ring defect fundamental frequency of 48 Hz, the protection window covers frequency points such as 48 Hz, 96 Hz, and 144 Hz and their attenuation bandwidth ranges, ensuring that the crack harmonic signal is not weakened during processing. Within the protection window range, the system uses the feature matrix obtained from the bearing housing vibration response equation to construct an orthogonal projection operator to perform projection suppression operations on the spectral components, effectively weakening the vibration coupling components introduced by factors such as blade aerodynamic disturbance and tower structure resonance. This operation significantly reduces the peak intensity of interference close to the crack harmonic frequency. For non-target frequency bands, the system identifies artifact frequency ranges unrelated to crack development through correlation analysis between energy attenuation rate and the rate of change of the gray-level co-occurrence matrix entropy of the equipment surface image, and performs band-stop filtering to remove them. This effectively suppresses high-frequency artifacts caused by changes in the wind turbine operating environment (such as wind speed fluctuations and temperature differences). During the deep learning training phase, the system generates a frequency-dimensional mask matrix based on a protection window, selectively blocking the backpropagation gradient, allowing only high-frequency components outside the window to propagate, thereby highlighting the high-frequency characteristic response of microcracks while suppressing the impact of large low-frequency interference on model parameter updates. Through this gradient truncation strategy, the model can focus on optimizing weights related to early crack features during training. By constructing a protection window covering the multi-harmonic attenuation range, the system ensures that the key frequency components of metal fatigue cracks are not weakened during signal processing and model training, improving the fidelity of early crack features. Orthogonal projection operations effectively weaken the structural coupling noise peaks close to the fault frequency, and the average amplitude of interference peaks is reduced in field tests. The gradient cutoff layer reduces gradient perturbations in non-target frequency bands, enabling the deep learning model to converge to a stable fault feature extraction pattern early in training, thus shortening the number of training iterations. Even under strong background noise conditions caused by wind speed fluctuations, the system can still detect microcrack harmonic signals with amplitudes only 0.7 times the root mean square value of the noise, outperforming detection schemes that do not employ the protective window and gradient cutoff strategies.

[0034] Specifically, the process of receiving the high-frequency characteristics of the gradient cutoff layer and detecting the phase shift of the cross-correlation function with image sharpness during sudden speed changes is as follows: Based on the high-frequency characteristics of the microcracks in the gradient cutoff layer, and combined with the pulse rising edge marker of the speed sensor to lock the speed change event, the vibration envelope signal and the synchronous image sharpness sequence within the dynamic time window are extracted with the event time as the center; complex analytical wavelet transform is performed on the data within the window to extract the instantaneous principal phase angle of the vibration envelope component and the image sharpness component in the complex domain, the instantaneous phase difference between the two components before and after the speed change point is calculated, and the phase difference is converted into the relative displacement offset between the bearing raceway and the visual monitoring area by combining the gear meshing transmission ratio parameter of the equipment.

[0035] In this implementation scheme, the rotational speed abrupt change event is identified, and the vibration envelope signal and synchronous image sharpness sequence within a dynamic time window are extracted: the rising edge of the pulse signal output by the rotational speed sensor is used to mark the moment of rotational speed abrupt change. Centered on this moment, a time window of sufficient length encompassing the period before and after the abrupt change is selected, and the vibration envelope signal and synchronously acquired equipment visual image sharpness sequence for the corresponding time period are extracted to ensure the spatiotemporal correspondence of the analysis data. A complex analytic wavelet transform is performed on the extracted signals within the window to extract the instantaneous principal phase angle: a complex analytic wavelet transform is performed on both the vibration envelope signal and the image sharpness sequence to obtain the complex representation of the signal in the time-frequency two-dimensional domain. The instantaneous principal phase angle is extracted from this representation to describe the phase characteristics of the signal at local time points. The formula (complex analytic wavelet transform) is as follows: Parameter description: :Signal At any moment ,scale Complex analytic wavelet coefficients; The signal to be analyzed (vibration envelope signal or image sharpness sequence); Mother wavelet function; Complex conjugate of the mother wavelet; Time shift parameters; Scale parameter, inversely proportional to frequency. Instantaneous principal phase angle extraction: Parameter description: :Signal In time ,scale The instantaneous principal phase angle below; The argument function of a complex number. Calculating the instantaneous phase difference between two signals before and after a sudden change in rotational speed: Select a specific time point or time interval before and after the sudden change in rotational speed, and calculate the instantaneous principal phase angle difference between the vibration envelope signal and the corresponding moment in the image sharpness sequence. This reflects the phase synchronization and shift characteristics of the two signals at that event. Formula expression (instantaneous phase difference): Parameter description: Vibration envelope signal ( ) and image sharpness sequence ( The instantaneous phase difference; Instantaneous phase of the vibration envelope signal; Instantaneous phase of the image sharpness sequence. Combined with the gear meshing ratio, the phase difference is converted into a relative displacement offset: Based on the gear meshing ratio of the equipment, the obtained phase difference is quantified as the relative displacement offset between the bearing raceway and the visual monitoring area, revealing the dynamic impact of sudden speed changes on the operating condition. Formula expression (relative displacement offset): Parameter description: : The relative displacement offset between the bearing raceway and the visual monitoring area; Instantaneous phase difference (radians); : The characteristic wavelength corresponding to the fault frequency; The gear ratio is the ratio of the input shaft speed to the output shaft speed. During bearing condition monitoring, after extracting high-frequency features of microcracks using a gradient cutoff layer, the system uses the rising edge of the speed sensor pulse to pinpoint a sudden speed change event caused by a sudden wind condition. Centered on this event, the system automatically extracts a dynamic time window containing 1.5 seconds before and after the change, extracting the vibration envelope signal and the simultaneously acquired visual image sharpness sequence of the gearbox housing surface, ensuring accurate correspondence between the two types of data in the time domain. Within this time window, the system performs complex analytical wavelet transform on both the vibration envelope signal and the image sharpness sequence to obtain the complex representation of the signal in the time-frequency domain and extracts the instantaneous principal phase angle. By comparing the phases of the same characteristic frequency points before and after the speed change, the system obtains the instantaneous phase difference between the vibration features and the image sharpness features. Subsequently, combined with the gear ratio parameters of the wind power main drive system (input shaft to output shaft speed ratio of 3.25:1), the system quantifies the phase difference as the relative displacement offset between the bearing outer raceway and the visual monitoring area. This offset reflects the dynamic impact of sudden speed changes on bearing conditions, such as the relative motion changes between the raceway and cage, and can serve as an important reference indicator for crack propagation sensitivity. By utilizing the rising edge marker of the speed sensor and a dynamic time window strategy, millisecond-level synchronization of vibration and visual modal data under sudden speed events is achieved, providing a reliable basis for phase difference calculation. Using high-frequency features extracted from a gradient truncation layer as input ensures that phase analysis focuses on the microcrack-sensitive frequency range, improving the accuracy of phase difference quantification; in field testing, the error in phase difference calculation is less than the set radius. Through transmission ratio correction, the abstract phase difference is transformed into an interpretable relative displacement offset, enabling maintenance personnel to intuitively judge the actual impact of speed disturbances on bearing clearance and contact stress. In multiple sudden wind event events, the phase difference-displacement curve showed a continuous increasing trend, highly consistent with the crack length growth found in subsequent disassembly and inspection, verifying the effectiveness of this method in monitoring early crack propagation.

[0036] Specifically, the process of identifying the propagation path of impact faults by envelope spectrum kurtosis and dynamically shrinking the frequency domain window bandwidth in combination with real-time load data is as follows: The vibration signal is demodulated across the entire frequency band using complex Morlet wavelets to generate a scale energy distribution matrix; the kurtosis value of the envelope spectrum at each scale is calculated, and when the kurtosis peak value of a specific frequency band exceeds the material yield strength correlation threshold, it is marked as an impact transmission path; a negative exponential correlation function between the frequency domain analysis window bandwidth and the real-time load is established; and the analysis window bandwidth is dynamically compressed within the marked impact path frequency band based on the load value.

[0037] In this implementation scheme, complex Morlet wavelets are used to perform full-band envelope demodulation of the vibration signal, generating a scale energy distribution matrix. Multi-scale envelope demodulation of the vibration signal is then performed using complex Morlet wavelet transform to extract the envelope energy distribution at different scales (corresponding frequencies), forming a scale-time two-dimensional energy matrix that reflects the distribution of the signal's impact characteristics with frequency. The complex Morlet wavelet is a complex wavelet used in signal processing, constructed by multiplying a complex exponential function with a Gaussian window function. The formula (complex Morlet wavelet transform) is as follows: Parameter description: :Signal In scale and time Complex wavelet coefficients at the location; Vibration signal; Morlet mother wavelet function; : Complex conjugate of Morlet mother wavelet; : Scale parameter, scale is inversely proportional to frequency; Time shift parameter. Definition of scale energy distribution matrix: Parameter description: :scale The corresponding average envelope energy; Analyze the time window length. Calculate the kurtosis values ​​of the envelope spectrum at each scale to identify the impact propagation path: The kurtosis index measures the sharpness of the envelope spectrum at each scale, reflecting the strength of the signal's impact component. The frequency band corresponding to the kurtosis peak characterizes the propagation path of the impact fault. Formula expression (kurtosis calculation): Parameter description: :scale The corresponding kurtosis value; Scale energy; : The mean of the scale energy; Mathematical expectation operation. Impact path determination condition: If The frequency band corresponds to the impact propagation path. Parameter description: Kurtosis threshold associated with the material's yield strength. A negative exponential correlation function is established between the frequency domain analysis window bandwidth and the real-time load: To reflect the impact of load changes on the frequency domain analysis window bandwidth, a negative exponential function is established for the window bandwidth as a function of real-time load, enabling dynamic adjustment of the bandwidth. Formula expression (dynamic bandwidth adjustment): Parameter description: The load value is Dynamic frequency domain window bandwidth; Initial frequency domain window bandwidth; Load impact index controls the speed of bandwidth contraction; Real-time load values. Within the marked impact path frequency band, the analysis window bandwidth is dynamically compressed based on the load value: for frequency bands identified as impact transmission paths, the window bandwidth is adjusted according to the real-time load value to compress the frequency band range, thereby improving the focus of frequency domain analysis and the accuracy of fault identification. The dynamic bandwidth function obtained based on the above steps... The system dynamically shrinks the frequency range of the impact path band by scaling the frequency range accordingly. During long-term monitoring of the wind power main drive system, after completing multi-modal synchronous acquisition, the system uses complex Morlet wavelets to perform full-band envelope demodulation of the main bearing vibration signal, generating a scale-time two-dimensional energy distribution matrix. Subsequently, the system calculates the kurtosis value of the envelope spectrum at each scale to measure the sharpness of the signal impact component. When the kurtosis peak value of a certain frequency band exceeds a threshold associated with the yield strength of the main bearing material, that frequency band is identified as the propagation path of an impact fault. The system establishes a negative exponential correlation function between the frequency domain analysis window bandwidth and the load within the impact path frequency band, achieving dynamic bandwidth shrinkage. When wind conditions change abruptly or the load increases significantly, the system automatically compresses the analysis bandwidth of the impact path frequency band, narrowing it from the original ±5Hz to ±2Hz to improve spectral focus and signal-to-noise ratio. In field measurements, the wind speed suddenly increased from 12 m / s to 17 m / s, causing a 2.4-fold increase in the kurtosis value of the main bearing impact path frequency band (approximately 185 Hz). Simultaneously, the system compressed the analysis bandwidth by 60% based on real-time load changes, ultimately clearly separating the harmonic components caused by the fatigue crack in the outer ring within this frequency band. By combining the kurtosis peak value with the material yield strength threshold, the impact propagation path identification no longer relies on fixed empirical values ​​but matches the structural mechanical characteristics, reducing misjudgments. The load-driven negative exponential bandwidth adjustment mechanism allows the system to automatically increase the analysis focus as the load increases, making it particularly suitable for conditions with large wind speed fluctuations and short-lived impact signals. Dynamic bandwidth contraction not only reduces interference from non-target frequency bands but also enhances the detectability of weak harmonic components near the impact path frequency. Precise impact path positioning provides the target frequency band input for the aforementioned "protection window + gradient cutoff layer" and also provides higher feature purity for the subsequent "phase difference-relative displacement monitoring" module, forming an end-to-end high-sensitivity monitoring chain.

[0038] Specifically, the process of constructing the motion trajectory of the bearing raceway features in phase space, and generating a frequency band window readjustment command to feed back to the feature decoupling module when the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold is as follows: the high-frequency feature vector of the microcrack is reconstructed into phase space through time delay embedding, and a three-dimensional feature trajectory is generated by dimensionality reduction through a local linear embedding algorithm; the curvature change rate of the trajectory is calculated, and when the curvature change rate exceeds the set change range, the window bandwidth amplification amount is calculated according to the crack depth change rate, and a readjustment command containing the target bandwidth value is generated and fed back to the feature decoupling module.

[0039] In this implementation scheme, the high-frequency feature vectors of microcracks are reconstructed into phase space trajectories through time-delay embedding: the acquired high-frequency feature vector sequence of microcracks is used to construct time-delay vectors, reconstructing high-dimensional phase space trajectories and revealing the dynamic evolution law of the system. Formula representation (time-delay embedding): Parameter description: :time Reconstructed phase space vectors; : High-frequency characteristic sequence of original microcracks; Time delay step; Embedding Dimension. Dimensionality reduction of high-dimensional trajectories to three-dimensional feature trajectories using the Local Linear Embedding (LLE) algorithm: To facilitate trajectory analysis, the LLE nonlinear dimensionality reduction algorithm is applied to map the high-dimensional reconstructed trajectory to three-dimensional space, preserving the local neighborhood structure. A neighborhood weight matrix is ​​constructed, and the eigenvalue problem is solved to obtain the dimensionality-reduced coordinates. The curvature and rate of change of the three-dimensional trajectory are calculated: Trajectory curvature reflects the degree of bending of the motion trajectory, and its rate of change reflects the dynamic anomaly fluctuations of the system. Formula representation (trajectory curvature): Assume the trajectory is in parametric form. Curvature is defined as Parameter description: :parameter Curvature of the trajectory; :Trajectory in The first derivative vector at point; :Trajectory in The second derivative vector at point; Vector cross product; Vector norm. Calculation of rate of change of curvature: Parameter description: : Curvature versus parameter The derivative of is the rate of change of curvature. When the rate of change of curvature exceeds the threshold associated with the material fatigue coefficient, the bandwidth amplification is calculated, and a frequency band window readjustment instruction is generated. Explanation: When the trajectory curvature change rate exceeds a preset threshold, the bandwidth is adjusted in conjunction with the crack depth change rate to optimize the frequency domain filtering window in the feature decoupling module. Formula expression (bandwidth amplification calculation): Parameter description: : Bandwidth expansion in the frequency band window; The bandwidth adjustment coefficient reflects the sensitivity of frequency propagation to crack changes. : Rate of change of crack depth over time; Step function: takes the value 1 if the input is greater than zero, and 0 otherwise. : Rate of change of trajectory curvature; The threshold for the rate of change of curvature associated with the material fatigue coefficient. Instruction for bandwidth window readjustment: This modulates the amplified bandwidth value. The data is sent to the feature decoupling module to update the width of the frequency domain filtering protection window. In the long-term operational monitoring of wind turbine units, after extracting high-frequency features of microcracks in the main bearing, the system reconstructs the feature vector sequence into a high-dimensional phase space using a time-delay embedding method to reveal the dynamic evolution trajectory during crack propagation. To facilitate real-time analysis, the system uses the Local Linear Embedding (LLE) algorithm to reduce the high-dimensional trajectory to a three-dimensional feature trajectory, preserving the local neighborhood geometry while reducing computational burden. During continuous monitoring, the system calculates the curvature and its rate of change of the three-dimensional trajectory in real time. When the rate of change of curvature continuously exceeds a preset threshold related to the fatigue coefficient of the main bearing material, the crack propagation stage is determined to have entered a high-risk zone. At this point, the system combines the crack depth change rate obtained from ultrasonic detection or high-frequency envelope demodulation to calculate the required increase in the frequency band window bandwidth and generates a readjustment command containing the target bandwidth value. This command is fed back to the feature decoupling module to dynamically expand the protection window width, enabling subsequent frequency domain filtering to cover the newly generated high-frequency components involved in crack evolution. In a real-world operation and maintenance case, a wind turbine, operating at full load, detected that the rate of change of curvature of the main bearing trajectory exceeded the fatigue coefficient threshold by 1.8 times for 30 consecutive minutes, accompanied by an increase in the rate of change of crack depth. The system automatically expanded the protection window width from ±4Hz to ±6.5Hz, recaptured the previously filtered 2.5 harmonic characteristic peak, and ultimately confirmed that the crack had entered the rapid propagation stage. A shutdown for maintenance was scheduled in advance, preventing significant gearbox-related damage. The dynamic response to crack state changes, using the rate of change of curvature in phase space to determine abrupt changes in structural stress state, avoids the hysteresis caused by relying on a single frequency domain threshold. Linking the rate of change of crack depth with the rate of change of curvature allows the frequency band window to dynamically expand as the structure degrades, ensuring that newly emerging high-frequency fault components are not missed. During wind turbine load fluctuations and sudden crack propagation stages, the system can trigger maintenance recommendations several days to weeks in advance, significantly reducing the probability of major failures. The bandwidth expansion instructions generated by this module directly act on the feature decoupling module, forming a dynamic feature optimization closed loop with modules such as "protection window + gradient cutoff layer", "impact path recognition", and "phase difference displacement detection".

[0040] Specifically, the process of updating the time delay compensation parameters to the dynamic timing alignment module when temperature fluctuations cause trajectory drift is as follows: the temperature sensor data is mapped to the feature space through the thermal expansion coefficient matrix, and the temperature-sensitive component is separated from the motion trajectory; when the trajectory drift exceeds the allowable error band of material thermal deformation, the partial derivative of the drift with respect to the time delay parameter is calculated, the learning step size of the gradient descent method is set based on the material thermal conduction rate, the time delay compensation is iteratively updated and fed back to the timing alignment module.

[0041] In this implementation scheme, the temperature sensor data is mapped to the phase space feature dimension: the temperature change is mapped to a trajectory coordinate system using the material's thermal expansion sensitivity to determine the thermally induced trajectory drift component. Mathematical representation (thermal expansion mapping): Parameter description: :time The corresponding temperature mapping vector represents the thermal perturbation components of each feature dimension; The thermal expansion coefficient matrix reflects the sensitivity of each characteristic dimension to temperature changes; Temperature sensor data vector. Separating the temperature-sensitive component from the phase space trajectory: Separating the aforementioned thermal disturbance component from the total trajectory yields the drift vector. Calculating the drift amount: Parameter description: :time The amount of trajectory drift at that location; : Original phase space trajectory points; Thermal disturbance component. Criteria for determining whether thermal drift exceeds the material tolerance threshold: Parameter description: The allowable error band for material thermal deformation is derived from the thermal stability limit. When the drift exceeds the error band, the time delay compensation parameter adjustment process is triggered. Step 3: Calculate the mathematical representation of the gradient of the trajectory drift with respect to the time delay compensation parameter. Parameter description: The delay parameters currently used for delay compensation; : Temperature disturbance mapping vector adjusted with time delay compensation; The gradient operator for the time delay compensation amount. An adaptive learning rate is set based on the material's thermal conductivity, and the mathematical representation of the compensation amount is updated (gradient descent update). Parameter description: The compensation value for the kth iteration; Learning step size, set to the material's thermal conductivity. The reciprocal weighted form; The gradient of the trajectory drift with respect to the current compensation parameters; Material thermal conductivity. Adaptive learning rate representation: ; Regularization term to prevent excessively large learning rates, which could lead to oscillations or non-convergence. The updated... The feedback of the time delay compensation value to the timing alignment module is explained as follows: The new compensation value is used to correct the time synchronization deviation between image signals, vibration data, and thermal sensing data, maintaining the consistency of multimodal data alignment. During the long-term multimodal condition monitoring of wind turbines, the system simultaneously receives data from multiple temperature sensors installed on the bearing housing, gearbox, and nacelle shell while constructing the phase space characteristic trajectories of the main bearing and key components of the transmission chain. Based on a pre-established thermal expansion coefficient matrix in the material database, the system maps temperature changes to the trajectory feature space to identify the characteristic trajectory drift components caused by thermal expansion or contraction. When the monitored trajectory drift continuously exceeds a threshold related to the material's thermal deformation tolerance (for example, the characteristic space mapping amplitude corresponding to a thermal stability limit tolerance of ±0.08mm for the outer ring material of the main bearing is 42CrMo alloy steel), the system determines that the temperature disturbance has significantly affected the time synchronization accuracy of the multimodal data. At this point, the module initiates a gradient iterative update process for the time delay compensation parameters: First, it calculates the sensitivity (gradient) of the trajectory drift to the current time delay compensation parameters; then, it sets an adaptive learning step size based on the material's thermal conductivity (e.g., the thermal diffusivity of steel at approximately 45 W / (m·K)), ensuring that parameter adjustments can quickly respond to sudden temperature fluctuations while avoiding synchronization parameter oscillations due to excessive learning rates; finally, the updated time delay compensation values ​​are fed back to the dynamic timing alignment module in real time. By establishing a mapping relationship between temperature and trajectory through the material's thermal expansion coefficient matrix, the heat-sensitive components can be accurately separated before multimodal signal fusion. Adaptively adjusting the gradient descent step size using the material's thermal conductivity allows for rapid response to sudden temperature fluctuations while avoiding parameter oscillations, thus improving system stability. Feeding the updated compensation values ​​back to the dynamic timing alignment module in real time ensures high-precision alignment of the image, vibration, and thermal signals within milliseconds.

[0042] Please see Figure 2The intelligent cleaning and feature extraction method for multimodal industrial data includes the following steps: S1. Dynamically adjust the matching window size of the time warping algorithm based on the main vibration frequency of the rotating equipment, and calculate the temperature signal transmission delay based on the thermal expansion characteristics of the equipment material; S2. Receive the delay and synchronize the multimodal data time reference, solve the fault-sensitive frequency band through the bearing housing vibration response equation, construct a frequency band protection window based on the harmonic distribution characteristics of metal fatigue cracks, perform frequency domain filtering to remove vibration artifacts within the window, and set a gradient truncation layer in the non-window area to retain the high-frequency features of microcracks; S3. Receive the high-frequency features of microcracks from the gradient truncation layer, detect the phase shift of the cross-correlation function with image clarity during sudden changes in rotational speed, identify the propagation path of impact faults through envelope spectrum kurtosis, and dynamically shrink the frequency domain window bandwidth in combination with real-time load data; S4. Construct the motion trajectory of the bearing raceway features in phase space. When the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold, generate a frequency band window readjustment command and feed it back to the feature decoupling module. When temperature fluctuations cause trajectory drift, update the time delay compensation parameters to the dynamic timing alignment module.

[0043] In this implementation scheme, step S1: This step extracts the dominant vibration frequency of the rotating equipment in real time and dynamically adjusts the matching window size in the time warping algorithm to adapt to the data alignment deviation caused by equipment speed fluctuations. Simultaneously, it combines the thermal expansion coefficient and heat conduction mechanism of the equipment material to quantify the delay compensation amount of temperature signal propagation relative to the vibration reference, achieving accurate time synchronization between multimodal data. This scheme introduces a frequency-driven adaptive window mechanism, significantly improving the dynamic response capability of data alignment; it introduces the thermal expansion-conduction modeling mechanism into time compensation estimation to achieve time-domain correction of temperature signals under industrial thermal load conditions, solving the problem of time mismatch between "image-temperature-vibration" three types of data in high-temperature unsteady-state scenarios using existing methods. Step S2: This step first uses the bearing housing dynamic model and random subspace identification to extract fault-sensitive frequency bands, then combines the harmonic distribution characteristics of metal fatigue cracks to construct a frequency band protection window. Artifacts are removed within the window, and a gradient cutoff layer is set in the non-window region to retain the high-frequency effective features of microcracks, achieving a balance between frequency domain noise suppression and high-sensitivity damage extraction. A gradient truncation strategy combined with a frequency band protection mechanism is proposed, which only allows features of specific frequency bands to propagate to the model gradient update path, avoiding interference from non-target frequencies to network training and improving the stability and targeting of feature extraction. Step S3: By detecting the main phase shift of the cross-correlation between the image sharpness function and high-frequency vibration features before and after the speed change change event, the spatial shift relationship between microcrack propagation and visible area is estimated. Furthermore, the envelope spectrum kurtosis is used to identify the impact fault path, and the bandwidth of the frequency domain analysis window is adaptively adjusted in combination with real-time load data. Kujicic spectrum and load coupling construct a bandwidth dynamic compression function to effectively avoid non-target frequency interference under heavy loads; overall, the dynamic causal modeling of image-vibration-load features is integrated to form an event-driven path recognition mechanism with innovative paths that have multi-source constraints and clear structures; Step S4: The motion trajectory of bearing raceway features in phase space is reconstructed through time delay embedding and manifold dimensionality reduction methods; its curvature change rate is calculated to determine the crack evolution state, and this is used to dynamically feed back the resetting of the frequency band window; at the same time, trajectory drift caused by temperature fluctuations will trigger gradient descent updates to the time delay compensation parameters, realizing a closed-loop adaptive adjustment mechanism for the entire system chain. A partial derivative relationship and a gradient descent optimization model dominated by heat conduction are constructed between trajectory drift and temperature compensation to ensure the stability and adaptability of the system under thermal disturbance conditions; from high-frequency features → phase trajectory → feedback window readjustment → correction of time delay compensation, it reflects a highly coupled, highly autonomous, and highly stable multimodal intelligent processing logic.

[0044] In summary, this application has at least the following effects:

[0045] A system and method for intelligent cleaning and feature extraction of multimodal industrial data are proposed. By introducing an adaptive window adjustment mechanism driven by the main vibration frequency of rotating equipment and a temperature signal delay compensation model based on material thermal expansion parameters, high-precision time alignment of different modal data (vibration, temperature, and images) under dynamic operating conditions is achieved, effectively reducing timing deviations caused by temperature fluctuations and rotational speed changes. A frequency band protection window based on the harmonic distribution of metal fatigue cracks is constructed, combined with a frequency domain gradient truncation mechanism, to suppress vibration artifacts and preserve the characteristics of minor structural damage, improving early fault identification capabilities. An impact path identification logic is constructed based on the principal phase shift of image sharpness cross-correlation and the kurtosis peak of the envelope spectrum. Combined with real-time load data, the frequency domain analysis bandwidth is dynamically adjusted to achieve linked modeling and coupled analysis of visual and vibration data under key events (such as sudden changes in rotational speed). By reconstructing the high-frequency characteristic trajectory of microcracks in phase space and combining it with the trajectory curvature change rate-driven frequency band window readjustment logic, a closed-loop feedback mechanism is formed where data features act inversely on parameter settings, enhancing the system's adaptability and generalization performance. By establishing a functional gradient relationship between trajectory drift caused by temperature fluctuations and time delay parameters, setting an iterative learning rate based on the thermal conductivity of materials, and updating compensation parameters in real time, the stability and accuracy of multimodal data fusion processing in high-temperature and complex industrial environments are improved.

[0046] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0047] This invention is described with reference to flowchart illustrations and / or block diagrams of systems, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0048] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0049] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0050] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0051] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An intelligent cleaning and feature extraction system for multi-modal industrial data, characterized in that, The application comprises the following modules: a time alignment module, a feature decoupling module, a working condition causal analysis module, and a feature stability optimization module. The time alignment module is used to dynamically adjust the matching window size of the time regularization algorithm by rotating the main vibration frequency of the equipment, and to calculate the temperature signal transmission delay based on the thermal expansion characteristics of the equipment material. The feature decoupling module is used to receive the temperature signal transmission delay and synchronize the time base of multi-modal data, solve the fault sensitive frequency band through the bearing seat vibration response equation, construct a frequency band protection window based on the harmonic distribution characteristics of metal fatigue cracks, perform frequency domain filtering to remove vibration artifacts within the window, and set a gradient cutoff layer in the non-window area to retain the micro-crack high-frequency characteristics. The bearing seat vibration response equation is modeled by a mass-resistance-spring system. The working condition causal analysis module is used to receive the micro-crack high-frequency characteristics of the gradient cutoff layer, detect the phase shift of the vibration signal and the image clarity cross-correlation function of the micro-crack high-frequency characteristics at the speed sudden change, identify the impact failure propagation path through envelope spectrum kurtosis, and dynamically shrink the frequency band protection window bandwidth combined with real-time load data. The image clarity cross-correlation function is used to quantify the similarity of two signals at different time offsets. The feature stability optimization module is used to construct the motion trajectory of the bearing raceway features in phase space, generate frequency band protection window re-adjustment instructions when the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold, and feed back to the feature decoupling module, and update the temperature signal transmission delay to the dynamic time alignment module when the trajectory drifts due to temperature fluctuations. The specific process of receiving the temperature signal transmission delay and synchronizing the time base of multi-modal data to solve the fault sensitive frequency band is as follows: Calibrate the time deviation of the image frame and the vibration frequency spectrum through the temperature signal transmission delay, and generate time-aligned multi-modal data stream; Establish the bearing seat dynamics differential equation set, solve the system characteristic matrix through random subspace identification, extract the feature frequency components related to the rolling element damage, calculate the outer ring fault characteristic fundamental frequency and its harmonic distribution range based on the bearing geometric structure parameters, and determine the fault sensitive frequency band; The logic process of constructing a frequency band protection window based on the harmonic distribution characteristics of metal fatigue cracks, performing frequency domain filtering to remove vibration artifacts within the window, and setting a gradient cutoff layer in the non-window area to retain the micro-crack high-frequency characteristics is as follows: According to the fault characteristic fundamental frequency of the fault sensitive frequency band, a protection window covering multiple harmonic attenuation regions is constructed, and the window boundary is determined according to the crack harmonic energy attenuation characteristics; According to the characteristic matrix of the bearing seat vibration response equation, an orthogonal projection operator is constructed, and orthogonal projection operation is performed on the frequency spectrum components within the window to suppress vibration coupling components. When the correlation coefficient between the energy attenuation rate of the frequency point and the change rate of the device surface image gray level co-occurrence matrix entropy of the synchronous acquisition is lower than the set threshold, the corresponding frequency region is marked and band-stop filtering operation is performed. The frequency domain feature tensor is input into a deep learning network, a mask matrix is constructed based on the frequency dimension, mask operation is performed on the gradient tensor during back propagation, only the gradient of the frequency domain area outside the window is allowed to propagate, a gradient truncation layer is set in the area outside the window to limit the interference of non-target frequency bands on the model training process, and frequency domain regularization processing is performed on the features after masking; The specific process of detecting the phase shift amount of the vibration signal and the image definition cross-correlation function of the micro-crack high-frequency feature of the gradient truncation layer at the speed mutation time is as follows: According to the micro-crack high-frequency feature of the gradient truncation layer, the speed mutation event is locked by combining the pulse rising edge of the speed sensor, and the vibration envelope signal and the synchronous image definition sequence in the dynamic time window are intercepted with the event time as the center; The complex analytic wavelet transform is performed on the data in the window, the instantaneous principal phase angle of the vibration envelope component and the image definition component in the complex domain is extracted, the instantaneous phase difference value of the two components before and after the speed mutation point is calculated, and the phase difference is converted into the relative displacement offset of the bearing raceway and the visual monitoring area by combining the gear meshing transmission ratio parameter of the device.

2. The intelligent cleaning and feature extraction system of multi-modal industry data as claimed in claim 1 wherein: The time sequence alignment module includes the following steps: The vibration signal of the rotating device is collected in real time, the dominant vibration frequency component is extracted, and the matching window size of the time regularity algorithm is adjusted periodically according to the frequency; The thermal expansion coefficient is obtained from the device material database, the thermal deformation amount of the metal structure is calculated by combining the temperature sensor data, and the transmission delay amount of the temperature signal relative to the vibration reference is derived according to the propagation speed of sound waves in the metal medium.

3. The intelligent cleaning and feature extraction system of multi-modal industry data as claimed in claim 2 wherein: The specific process of identifying the impact failure propagation path by envelope spectrum kurtosis and dynamically shrinking the frequency band protection window bandwidth according to real-time load data is as follows: The vibration signal is envelope demodulated by complex Morlet wavelet to generate a scale energy distribution matrix; The kurtosis value of each scale envelope spectrum is calculated, and when the kurtosis peak value of a specific frequency band exceeds the yield strength correlation threshold of the material, the impact transmission path is marked, a negative exponential correlation function of the frequency band protection window bandwidth and the real-time load is established, and the frequency band protection window bandwidth is dynamically compressed according to the load value in the marked impact path frequency band.

4. The intelligent cleaning and feature extraction system of multi-modal industry data as claimed in claim 3 wherein: The specific process of constructing the motion trajectory of the bearing raceway feature in the phase space and generating a frequency band protection window re-adjustment instruction when the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold is as follows: The micro-crack high-frequency feature vector is reconstructed into the phase space by time delay embedding, and a three-dimensional feature trajectory is generated by dimension reduction through local linear embedding algorithm; When the trajectory curvature change rate exceeds the set change range, the frequency band protection window bandwidth expansion amount is calculated according to the crack depth change rate, and a re-adjustment instruction containing the target bandwidth value is fed back to the feature decoupling module.

5. The intelligent cleaning and feature extraction system of multi-modal industry data as claimed in claim 4 wherein: The specific process of updating the temperature signal transmission delay amount to the dynamic time sequence alignment module when the temperature fluctuation causes the trajectory to drift is as follows: The temperature sensor data is mapped to the feature space through the thermal expansion coefficient matrix, and the temperature-sensitive component is separated from the motion trajectory; When the trajectory drift exceeds the material thermal deformation allowed error band, the partial derivative of the drift to the temperature signal transmission delay is calculated, the learning step of the gradient descent method is set based on the material thermal conduction rate, the temperature signal transmission delay is iteratively updated and fed back to the timing alignment module.

6. The intelligent cleaning and feature extraction method of multi-modal industrial data, applied to the intelligent cleaning and feature extraction system of multi-modal industrial data in any one of claims 1-5, characterized in that, The method comprises the following steps: S1. dynamically adjust the matching window size of the time rule algorithm by rotating the equipment main vibration frequency, and calculate the temperature signal transmission delay based on the thermal expansion characteristics of the equipment material; S2. receive the temperature signal transmission delay and synchronize the multi-modal data time reference, solve the fault sensitive frequency band through the bearing seat vibration response equation, construct the frequency band protection window based on the harmonic distribution characteristics of metal fatigue cracks, perform frequency domain filtering to remove vibration artifacts in the window, and set a gradient cutoff layer in the non-window area to retain the micro-crack high frequency characteristics; The bearing seat vibration response equation is modeled by a mass-resistance-spring system; S3. receive the micro-crack high frequency characteristics of the gradient cutoff layer, detect the phase shift of the vibration signal of the micro-crack high frequency characteristics and the image sharpness cross-correlation function at the speed mutation, identify the impact failure propagation path through the envelope spectrum kurtosis, and dynamically shrink the frequency band protection window bandwidth combined with real-time load data; The image sharpness cross-correlation function is used to quantify the similarity of two signals at different time offsets; S4. construct the motion trajectory of the bearing raceway feature in phase space, when the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold, generate a frequency band protection window re-adjustment instruction and feed it back to the feature decoupling module, and update the temperature signal transmission delay to the dynamic timing alignment module when the trajectory drifts due to temperature fluctuations.

Images