Intelligent cleaning and feature extraction system and method for multi-modal industrial data

Abstract

The invention discloses an intelligent cleaning and feature extraction system and method for multi-modal industrial data, and relates to the technical field of equipment state monitoring. The method is used for solving the problems that multi-source heterogeneous signal time alignment is not accurate, fault features are easily covered by background noise, a causal chain is not clear under working condition changes, and feature stability is poor. Firstly, a matching window is dynamically adjusted based on the main vibration frequency of rotating equipment, temperature signal delay is calculated in combination with a material thermal expansion coefficient, and modal alignment is achieved; then, a fault sensitive frequency band is solved through a bearing pedestal kinetic equation, a frequency band protection window is constructed, frequency domain filtering and gradient truncation operation are executed, and microcrack high-frequency features are extracted; secondly, recognizing a fault propagation path by combining image definition and envelope spectrum kurtosis, and dynamically shrinking a frequency domain window bandwidth according to a real-time load; finally, the feature vector is reconstructed to a phase space, when the curvature change rate or the temperature drift exceeds the limit, parameter updating and frequency band readjustment feedback are triggered, and the stability and adaptability of the system are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of equipment status monitoring, and specifically to a system and method for intelligent cleaning and feature extraction of multimodal industrial data. Background Art

[0002] Against the backdrop of the continued development of intelligent manufacturing and the Industrial Internet, rotating equipment (such as motors, bearings, and gearboxes) generates a large amount of heterogeneous sensory data during operation, covering multiple modalities such as vibration, image, temperature, and current. The ability to comprehensively analyze this multimodal data has become a key support for improving equipment predictive maintenance capabilities and safe operation. However, due to significant differences in sampling frequency, response mechanism, and physical coupling relationships among different modal data, time series alignment, feature extraction, and causal analysis pose significant challenges. Especially under high-load and variable operating conditions, the nonlinear coupling effect between data is more pronounced, and traditional signal processing methods are limited in their effectiveness in accurately extracting key state features.

[0003] Most existing multimodal industrial data processing methods are based on fixed window alignment strategies, which ignore the dynamic changes in equipment vibration frequency. This makes it difficult to compensate for the time reference offset of different modal data in real time. At the same time, frequency domain filtering relies on full-band filtering models. When dealing with non-steady-state faults (such as microcracks and early fatigue cracks), weak high-frequency components are easily misjudged as noise and filtered out, affecting the accuracy of early fault identification. In addition, the lack of a modeling compensation mechanism for the hysteresis characteristics of material thermal response leads to potential deviations between slow-response signals such as temperature and fast-response signals, further weakening the consistency and discriminability of feature fusion. In highly dynamic working conditions, existing models respond slowly to sudden changes in operating conditions, and the frequency window bandwidth lacks a real-time adjustment mechanism, resulting in a decrease in the ability to identify impact paths and difficulty in ensuring stability and adaptability. Summary of the Invention

[0004] In response to the deficiencies of the prior art, the present invention provides an intelligent cleaning and feature extraction system and method for multimodal industrial data, which solves the problems of the above-mentioned background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: an intelligent cleaning and feature extraction system for multimodal industrial data, including the following modules: a timing alignment module, a feature decoupling module, a working condition causal analysis module, and a feature stability optimization module; the timing alignment module is used to dynamically adjust the matching window size of the time regularization algorithm through the main vibration frequency of the rotating equipment, and calculate the temperature signal transmission delay based on the thermal expansion characteristics of the equipment material; the feature decoupling module is used to receive the delay and synchronize the multimodal data time base, solve the fault-sensitive frequency band through the bearing seat vibration response equation, construct a frequency band protection window based on the harmonic distribution characteristics of the metal fatigue crack, and perform frequency domain within the window. Filtering is used to remove vibration artifacts, and a gradient truncation layer is set in the non-window area to retain the high-frequency characteristics of microcracks. The operating condition causal analysis module is used to receive the high-frequency characteristics of microcracks in the gradient truncation layer, detect the phase offset of the cross-correlation function with the image clarity when the speed suddenly changes, identify the impact fault propagation path through the envelope spectrum kurtosis, and dynamically shrink the frequency domain window bandwidth in combination with real-time load data. The feature stability optimization module is used to construct the motion trajectory of the bearing raceway feature in the phase space. When the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold, a frequency band window readjustment instruction is generated and fed back to the feature decoupling module. When temperature fluctuations cause trajectory drift, the delay compensation parameters are updated to the dynamic timing alignment module.

[0006] Furthermore, the timing alignment module includes the following steps: real-time acquisition of the vibration signal of the rotating equipment, extraction of the dominant vibration frequency component, and periodic adjustment of the matching window size of the time warping algorithm according to the frequency; obtaining the thermal expansion coefficient from the equipment material database, calculating the thermal deformation of the metal structure in combination with the temperature sensor data, and deriving the time delay compensation amount of the temperature signal relative to the vibration reference based on the propagation speed of the sound wave in the metal medium.

[0007] Furthermore, the specific process of receiving the delay amount and synchronizing the multimodal data time base and solving the fault-sensitive frequency band through the bearing seat vibration response equation is as follows: calibrate the time deviation between the image frame and the vibration spectrum through the time delay compensation value to generate a time-aligned multimodal data stream; establish a group of bearing seat dynamic differential equations, solve the system characteristic matrix through random subspace identification, extract the characteristic frequency components related to rolling element damage, calculate the outer ring fault characteristic fundamental frequency and its harmonic distribution range based on the bearing geometric structure parameters, and determine the fault-sensitive frequency band.

[0008] Furthermore, a frequency band protection window is constructed based on the harmonic distribution characteristics of metal fatigue cracks, frequency domain filtering is performed within the window to remove vibration artifacts, and a gradient truncation layer is set in the non-window area to retain the high-frequency characteristics of microcracks. The logical process is as follows: based on the fault characteristic fundamental frequency of the fault-sensitive frequency band as the center, a protection window covering multiple harmonic attenuation areas is constructed, and the window boundary is determined according to the crack harmonic energy attenuation characteristics; an orthogonal projection operator is constructed according to the characteristic matrix of the bearing seat vibration response equation, and an orthogonal projection operation is applied to the spectral components within the window to suppress the vibration coupling components. When the correlation coefficient between the frequency point energy attenuation rate and the entropy change rate of the synchronously collected equipment surface image grayscale co-occurrence matrix is ​​lower than the set threshold, the corresponding frequency region is marked and a band-stop filtering operation is performed; the frequency domain feature tensor is input into the deep learning network, and a mask matrix is ​​constructed based on the frequency dimension. During the backpropagation process, a mask operation is performed on the gradient tensor, allowing only the gradient propagation of the frequency domain area outside the window. A gradient truncation layer is set in the area outside the window to limit the interference of non-target frequency bands on the model training process, and frequency domain regularization is performed on the masked features.

[0009] Furthermore, the specific process of receiving the high-frequency characteristics of the gradient truncation layer and detecting the phase offset of the cross-correlation function with the image clarity when the speed suddenly changes is as follows: based on the high-frequency characteristics of the microcracks in the gradient truncation layer, the speed sudden change event is locked in combination with the rising edge mark of the speed sensor pulse, and the vibration envelope signal and the synchronous image clarity sequence in the dynamic time window are intercepted with the event moment as the center; a complex analytical wavelet transform is performed on the data in the window to extract the instantaneous main phase angle of the vibration envelope component and the image clarity component in the complex domain, and the instantaneous phase difference between the two components before and after the speed sudden change point is calculated. In combination with the equipment gear meshing transmission ratio parameter, the phase difference is converted into the relative displacement offset between the bearing raceway and the visual monitoring area.

[0010] Furthermore, the impact fault propagation path is identified by the envelope spectrum kurtosis, and the specific process of dynamically shrinking the frequency domain window bandwidth in combination with real-time load data is as follows: the vibration signal is demodulated in the full-band envelope through complex Morlet wavelet to generate a scale energy distribution matrix; the kurtosis value of the envelope spectrum at each scale is calculated, and when the kurtosis peak value of a specific frequency band exceeds the material yield strength correlation threshold, it is marked as an impact transmission path. A negative exponential correlation function between the frequency domain analysis window bandwidth and the real-time load is established, and the analysis window bandwidth is dynamically compressed according to the load value within the marked impact path frequency band.

[0011] Furthermore, the motion trajectory of the bearing raceway feature in the phase space is constructed. When the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold, the specific process of generating a frequency band window readjustment instruction and feeding it back to the feature decoupling module is as follows: the high-frequency feature vector of the microcrack is reconstructed into the phase space through time delay embedding, and the dimensionality is reduced by the local linear embedding algorithm to generate a three-dimensional feature trajectory; the curvature change rate of the trajectory is calculated. When the curvature change rate exceeds the set change range, the window bandwidth expansion is calculated according to the crack depth change rate, and a readjustment instruction containing the target bandwidth value is generated and fed back to the feature decoupling module.

[0012] Furthermore, when temperature fluctuations cause trajectory drift, the specific process of updating the delay compensation parameters to the dynamic timing alignment module is as follows: the temperature sensor data is mapped to the feature space through the thermal expansion coefficient matrix, and the temperature-sensitive component is separated from the motion trajectory. When the trajectory drift exceeds the allowable error band of the material thermal deformation, the partial derivative of the drift with respect to the delay parameter is calculated, and the learning step size of the gradient descent method is set based on the material thermal conductivity rate. The delay compensation value is iteratively updated and fed back to the timing alignment module.

[0013] An intelligent cleaning and feature extraction method for multimodal industrial data includes the following steps: S1. Dynamically adjusting the matching window size of the time warping algorithm by rotating the main vibration frequency of the equipment, and calculating the temperature signal transmission delay based on the thermal expansion characteristics of the equipment material; S2. Receive the delay and synchronize the multimodal data time base, solve the fault-sensitive frequency band through the bearing seat vibration response equation, construct a frequency band protection window based on the harmonic distribution characteristics of metal fatigue cracks, perform frequency domain filtering within the window to remove vibration artifacts, and set a gradient truncation layer in the non-window area to retain the high-frequency characteristics of microcracks; S3. Receive the high-frequency characteristics of microcracks in the gradient truncation layer, detect the phase offset of the cross-correlation function with image clarity when the speed suddenly changes, identify the impact fault propagation path through the envelope spectrum kurtosis, and dynamically shrink the frequency domain window bandwidth in combination with real-time load data; S4. Construct the motion trajectory of the bearing raceway feature in phase space. When the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold, generate a frequency band window readjustment instruction and feed it back to the feature decoupling module. When temperature fluctuations cause trajectory drift, update the delay compensation parameters to the dynamic timing alignment module.

[0014] The present invention has the following beneficial effects:

[0015] (1) The intelligent cleaning and feature extraction system for multimodal industrial data, by introducing a dynamic matching window adjustment mechanism driven by the main vibration frequency, can adaptively control the window size of the time warping algorithm, significantly improving the accuracy and robustness of multimodal data timing alignment under unstable operating conditions. On this basis, combined with material thermal expansion modeling, the time delay compensation of the temperature signal is calculated, so that the sensor asynchronous error under thermal interference can be effectively corrected. Based on the dynamic equation of the bearing seat vibration response, the fault-sensitive frequency band is accurately solved, and the frequency band protection window is constructed through harmonic distribution to effectively suppress artifact interference in the vibration signal. At the same time, the frequency domain gradient truncation mechanism is used to retain the high-frequency microcrack characteristics, significantly enhancing the system's perception of early micro damage.

[0016] (2) Intelligent cleaning and feature extraction method for multimodal industrial data. Based on the phase offset detection strategy of the cross-correlation function of microcrack high-frequency features and image clarity, it realizes the linkage analysis of heterogeneous signals under speed mutation events, combines the envelope spectrum kurtosis function to identify the impact fault path, and effectively constructs a frequency domain mapping model for fault propagation. The frequency domain analysis window bandwidth is dynamically adjusted through the load sensing mechanism, thereby improving the system's ability to extract key features under nonlinear working conditions. The high-frequency features of the raceway are mapped to the phase space, and combined with the correlation model between the curvature change rate and the material fatigue coefficient, the abnormal state is autonomously judged and the frequency band window readjustment instruction is triggered to achieve adaptive closed-loop optimization of the feature extraction process. At the same time, the delay parameter is corrected in real time based on the trajectory offset caused by thermal drift, which improves the long-term stability and generalization ability of the entire method under complex thermal-mechanical coupling conditions.

[0017] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of the intelligent cleaning and feature extraction system for multimodal industrial data of the present invention.

[0019] Figure 2 This is a flow chart of the intelligent cleaning and feature extraction method for multimodal industrial data of the present invention. DETAILED DESCRIPTION

[0020] The embodiments of the present application solve the key technical problems in the prior art, such as inaccurate time alignment of multi-source heterogeneous signals, easy masking of fault features by background noise, unclear causal chains under changing working conditions, and poor feature stability, through an intelligent cleaning and feature extraction system and method for multimodal industrial data.

[0021] The overall idea of ​​the solution in the embodiments of this application is as follows:

[0022] First, the main vibration frequency of the rotating equipment is used to dynamically drive the timing alignment algorithm, adaptively adjusting the matching window size. The material thermal expansion model is then combined to compensate for the time delay of the temperature signal, thereby improving the synchronization accuracy of multimodal data.

[0023] Secondly, fault-sensitive frequency bands are extracted through bearing seat dynamics modeling. A frequency band protection window is constructed based on the harmonic distribution of metal fatigue cracks to effectively suppress vibration artifacts. A frequency domain gradient truncation strategy is used to retain the high-frequency components of microcracks, enhancing early fault identification capabilities.

[0024] Subsequently, a cross-correlation model between the image and vibration signals was constructed based on high-frequency features. The envelope spectrum kurtosis and load sensing mechanism were combined to identify the shock propagation path, thus achieving dynamic modeling of causal relationships under complex working conditions.

[0025] Finally, a phase space embedding algorithm is used to construct a high-dimensional feature trajectory to discriminate the curvature change rate. When it exceeds the fatigue-related threshold, the frequency band window is readjusted to further improve the model's responsiveness to feature drift. At the same time, a thermal compensation feedback mechanism is introduced to ensure the long-term operational stability of the system in a thermomechanical coupling environment.

[0026] See also Figure 1 , an embodiment of the present invention provides a technical solution: an intelligent cleaning and feature extraction system for multimodal industrial data, including the following modules: a timing alignment module, a feature decoupling module, a working condition causal analysis module, and a feature stability optimization module; the timing alignment module is used to dynamically adjust the matching window size of the time warping algorithm according to the main vibration frequency of the rotating equipment, and calculate the temperature signal transmission delay based on the thermal expansion characteristics of the equipment material; the feature decoupling module is used to receive the delay and synchronize the multimodal data time reference, solve the fault-sensitive frequency band through the bearing seat vibration response equation, construct a frequency band protection window based on the harmonic distribution characteristics of the metal fatigue crack, and perform frequency domain filtering and cleaning within the window. Vibration artifacts, a gradient truncation layer is set in the non-window area to retain the high-frequency characteristics of microcracks; the working condition causal analysis module is used to receive the high-frequency characteristics of microcracks in the gradient truncation layer, detect the phase offset of the cross-correlation function with the image clarity when the speed suddenly changes, identify the impact fault propagation path through the envelope spectrum kurtosis, and dynamically shrink the frequency domain window bandwidth in combination with real-time load data; the feature stability optimization module is used to construct the motion trajectory of the bearing raceway feature in the phase space. When the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold, a frequency band window readjustment instruction is generated and fed back to the feature decoupling module. When temperature fluctuations cause trajectory drift, the delay compensation parameters are updated to the dynamic timing alignment module.

[0027] In this implementation, the Timing Alignment Module is used to precisely align multi-source heterogeneous signals (such as vibration, temperature, and images) along the temporal dimension, ensuring the accuracy of subsequent feature processing. Dynamic Adjustment of the Main Vibration Frequency: By monitoring the changes in the main frequency (i.e., the dominant vibration frequency) of rotating equipment (such as motors or bearings) during operation, the matching window size in the time warping algorithm is adaptively adjusted, enabling effective comparison and alignment of signals at different time scales. The matching window is the sliding time period used in the signal alignment algorithm to calculate the similarity between two signals. Dynamically adjusting its size improves adaptability to non-stationary signals. Thermal Expansion Delay Compensation: Based on the thermal expansion model of the device component materials, the change in sensor signal transmission path length caused by temperature changes is estimated, thereby calculating the temperature signal's time delay offset and compensating it back to the main time axis. Thermal expansion characteristics refer to the linear or volumetric expansion of a material due to rising temperature, which can cause a certain time lag in sensor data reception. Feature Decoupling Module: This module is used to separate key fault-related features from multimodal data while suppressing background noise and interference from non-target frequency bands, thereby improving the accuracy and interpretability of diagnostic features. Synchronous Time Base: The module first receives the aforementioned time delay compensation results to achieve time base consistency across multiple-source signals. Fault-Sensitive Frequency Band Extraction: Based on the vibration response equation of the bearing seat (often modeled as a mass-resistance-elastic system), the resonant frequency band of the system under excitation is solved as the fault-sensitive frequency range. Frequency Band Protection Window Construction: Utilizing the harmonic distribution characteristics of metal fatigue cracks, namely that fatigue cracks often produce harmonic signals with specific frequency harmonic patterns, a frequency band window covering these harmonics is constructed to protect fault-related frequencies from being filtered out. Frequency Domain Filtering and Gradient Truncation: Within the aforementioned window, frequency domain filtering is used to remove artifacts (such as structural resonance and environmental interference). Outside the window, a gradient truncation layer is set to limit the propagation rate of the spectral energy, thereby retaining the high-frequency pulse components generated by microcracks. Gradient truncation layer: It is a frequency domain processing mechanism that prevents small signals from being averaged or over-smoothed by setting a threshold for energy changes, thereby improving the ability to identify weak fault features. Working condition causal analysis module: This module is aimed at fault evolution path analysis under complex operating conditions. It combines the mutual information of image and sensor data to reconstruct the propagation path of impact-type faults and realize causal relationship identification. Microcrack high-frequency feature input: The module receives the high-frequency features output by the previous module as the basis for analysis. Image clarity cross-correlation function: When the speed or working condition suddenly changes, the clarity of the equipment monitoring image will change accordingly. It is cross-correlated with the vibration signal to measure the phase offset and identify the potential fault occurrence point. Cross-correlation function: It is used to quantify the similarity of two signals at different time offsets. It is an effective means to achieve temporal causal reasoning.Envelope Spectrum Kurtosis Analysis: Envelope analysis (EnvelopeSpectrum) is used to extract the amplitude envelope of the impact signal. The kurtosis statistic is further combined to measure the sharpness of the signal and identify strong impact fault paths. Kurtosis: Used to measure whether the signal has pulse properties. The higher the value, the more significant the impact. Frequency Domain Window Adaptive Shrinkage: Dynamically adjusts the frequency window width based on the current load data, automatically enhancing the ability to resolve the impact frequency band when the load changes suddenly. Feature Stability Optimization Module: This module is used to improve the system's robustness and stability to changes in fault characteristics during long-term operation and address the problem of feature drift caused by changes in operating conditions. Phase Space Trajectory Construction: The state characteristics (such as vibration and acceleration) of key components such as bearing raceways are projected into phase space (PhaseSpace) to form a multidimensional trajectory curve for tracking feature evolution trends. Phase Space Reconstruction: This is a method in nonlinear dynamics that expands a one-dimensional time series into a multidimensional space through delayed embedding to capture the state evolution laws of the system. Curvature Change Identification: Calculates the trajectory's curvature change rate. When it exceeds a set threshold related to the material's fatigue life, it is considered to have entered an early failure state. A frequency band window readjustment instruction is generated and fed back to the feature decoupling module to update the protection strategy. Temperature Drift Compensation Feedback: If trajectory drift is detected to be primarily caused by temperature fluctuations, an update instruction is generated and fed back to the timing alignment module to readjust the delay compensation parameters to ensure that the feature does not shift with abnormal ambient temperature.

[0028] Specifically, the timing alignment module includes the following steps: real-time acquisition of the vibration signal of the rotating equipment, extraction of the dominant vibration frequency component, and periodic adjustment of the matching window size of the time warping algorithm according to the frequency; obtaining the thermal expansion coefficient from the equipment material database, combining the temperature sensor data to calculate the thermal deformation of the metal structure, and deriving the time delay compensation amount of the temperature signal relative to the vibration reference based on the propagation speed of the sound wave in the metal medium.

[0029] In this implementation, the vibration signal of the rotating equipment is collected in real time, and the dominant vibration frequency component is extracted. Specifically: the vibration signal of the rotating equipment is collected in real time by an acceleration sensor or a vibration sensor, and the dominant vibration frequency, that is, the fundamental frequency of the equipment, is extracted by a time-frequency analysis method (such as short-time Fourier transform STFT or adaptive spectrum estimation). The fundamental frequency reflects the rotation period of the equipment and serves as the time reference for the subsequent time warping algorithm. The matching window size of the time warping algorithm is periodically adjusted according to the dominant vibration frequency: the matching window size W is the length of the sliding window used to align the time points of the two signals in the time warping algorithm. Dynamic adjustment of the window length can improve the accuracy of timing alignment. The formula is expressed as: ; Parameter description: W: matching window size, in seconds; k: window coefficient, a constant, reflecting the multiple of the window length relative to the vibration period; : dominant vibration period, in seconds; : Dominant vibration frequency, in Hertz (Hz). Obtain the thermal expansion coefficient from the equipment material database and combine it with temperature sensor data to calculate the thermal deformation of the metal structure: The metal structure of the equipment undergoes linear expansion or contraction under temperature changes. Thermal deformation affects the time transmission characteristics of the vibration signal and requires calculation and compensation. Formula: ; Parameter description: : thermal deformation length, in meters; : Linear expansion coefficient of metal materials, expressed as relative change per degree Celsius ( ); : initial length of metal structure, in meters (m); : Temperature change, relative to the reference temperature, in degrees Celsius ( The time delay compensation for the temperature signal relative to the vibration reference is derived based on the propagation speed of sound waves in metal media. The temperature signal experiences a time delay when transmitted through metal structures, which is related to the thermal deformation and the propagation speed of sound waves. This time delay must be calculated to compensate for the time deviation in timing alignment. The formula is: ; Parameter description: : Temperature signal transmission delay, unit: seconds (s); : initial length of metal structure, in meters (m); : thermal deformation length, unit meter (m); The speed of sound waves in metal media, measured in meters per second (m / s), depends on the material type, for example, approximately 5900 m / s for steel. In a wind turbine operation monitoring scenario, the timing alignment module collects real-time vibration signals from a three-axis accelerometer mounted on the main shaft bearing seat and synchronizes them with data from the nacelle's internal temperature sensor. The system first uses a short-time Fourier transform to extract the dominant vibration frequency of the turbine under different load conditions. Based on this frequency, the matching window size is adaptively adjusted, maintaining multimodal signal synchronization accuracy even when sudden wind speed changes cause speed fluctuations of ±12%. When the turbine starts from a low temperature in winter and warms to its rated operating temperature, the metal bearing seat undergoes slight deformation due to thermal expansion. The module accesses the equipment material database to obtain the thermal expansion coefficient of the steel, calculates the thermal deformation based on real-time temperature data, and further derives the resulting temperature signal transmission delay. In this embodiment, the temperature compensation mechanism effectively eliminates the signal phase offset caused by a 40°C temperature difference, reducing the phase error in subsequent feature extraction to one-fifth of the original value. Through the window adaptation mechanism driven by the main vibration frequency, the timing alignment error of multimodal signals is reduced from ±8ms to ±1.5ms under complex working conditions such as speed changes and load fluctuations, effectively ensuring the time consistency of feature extraction. A compensation mechanism combining material parameters and temperature data is adopted to eliminate the phase offset of high-frequency signals caused by changes in the sound wave propagation path due to temperature rise, ensuring that the phase lock of cross-modal data is maintained under high sampling rate conditions. Since the time base of the input signal is more unified, the feature decoupling module can more accurately align subtle fault features between vibration, image and temperature signals, thereby improving the accuracy of early identification of microcracks and significantly reducing the false detection rate caused by signal misalignment.

[0030] Specifically, the specific process of receiving the delay amount and synchronizing the multimodal data time base, and solving the fault-sensitive frequency band through the bearing seat vibration response equation is as follows: calibrate the time deviation between the image frame and the vibration spectrum through the time delay compensation value to generate a time-aligned multimodal data stream; establish a group of bearing seat dynamic differential equations, solve the system characteristic matrix through random subspace identification, extract the characteristic frequency components related to rolling element damage, calculate the outer ring fault characteristic fundamental frequency and its harmonic distribution range based on the bearing geometric structure parameters, and determine the fault-sensitive frequency band.

[0031] In this implementation scheme, the time deviation between the image frame and the vibration spectrum is calibrated by the time delay compensation value to generate a time-aligned multimodal data stream: According to the time delay compensation value calculated by the timing alignment module, the image acquisition time point and the vibration signal time reference are corrected to achieve precise time synchronization between the two data modes, forming a time-aligned multimodal data stream, which is convenient for subsequent joint feature analysis. A group of differential equations for the dynamics of the bearing seat is established, and the system characteristic matrix is ​​solved through random subspace identification: a dynamic model of the bearing seat system is established to describe its vibration response characteristics. The random subspace identification method is used to process the experimental or collected vibration signals to obtain the system state space matrix, thereby extracting the dynamic characteristics of the system. Formula representation (dynamic equation): ; Parameter description: M: mass matrix; C: damping matrix; K: stiffness matrix; x(t): displacement vector, function at time t; : velocity vector, i.e. the time first derivative of displacement; : acceleration vector, i.e. the time second derivative of displacement; : External excitation force vector, function at time t. Formula (state space equation obtained by random subspace identification): ; Parameter description: : system state vector; : input vector (stimulus signal); : Output vector (measurement signal); A: state transfer matrix, describing the dynamic characteristics of the system; B: input matrix; C: output matrix; D: direct transfer matrix. Extract characteristic frequency components related to rolling element damage: According to the obtained state space matrix, the characteristic frequency components related to rolling element damage are extracted through spectral analysis and modal decomposition as key parameters for fault diagnosis. Calculate the characteristic fundamental frequency of the outer ring fault and its harmonic distribution range based on the bearing geometry parameters, and determine the fault-sensitive frequency band: Use the bearing geometry and rotation frequency parameters to calculate the fundamental frequency and harmonic range generated by the outer ring defect, and then determine the frequency band in the spectrum that is sensitive to the fault. Formula (outer ring defect fundamental frequency calculation): ; Parameter description: : outer race fault fundamental frequency; : Number of rolling elements; : bearing rotation frequency; : rolling element diameter; : bearing pitch diameter; : Contact angle. Formula (harmonic distribution range): ; Parameter description: : Fault-sensitive frequency band interval set; : harmonic number; : bandwidth offset, used to cover the extended range of harmonic energy; : Total number of harmonics. The system first uses the delay compensation value output by the timing alignment module to precisely time-synchronize surface crack image frames captured by a high-speed industrial camera with spectrum data collected by a vibration sensor. This synchronized multimodal data stream ensures strict correspondence between image and vibration features within the same time segment, avoiding misalignment of fault signs due to sampling delay. After multimodal data alignment, the system establishes a set of dynamic differential equations for the bearing seat and uses random subspace identification to identify the system's state-space matrix from the vibration response signals collected in situ. This process extracts the structural natural frequency and damping characteristics directly from the natural excitations of the operating conditions without the need for additional external excitation, minimizing disruption to production during on-site testing. Combining the bearing's geometric parameters and rotational speed, the system automatically calculates the characteristic fundamental frequency and harmonic distribution range of the outer ring fault. For example, in actual testing of a continuous casting machine main bearing (eight rolling elements, a contact angle of 0°, and a pitch diameter of 320 mm), the system was able to quickly identify the sensitive frequency bands of outer ring defects and focus on extracting the energy characteristics of these intervals during spectral analysis. This not only reduces the amount of data processing in irrelevant frequency bands, but also significantly improves the signal-to-noise ratio of weak fault signals. The temporal deviation between the image frame and the vibration spectrum is compensated at the nanosecond level, improving the effectiveness of subsequent multimodal feature fusion and reducing fault feature mismatches caused by acquisition delays. By combining system features extracted using random subspace identification with bearing geometry calculations, the locking error in the fault-sensitive frequency band is reduced to less than ±0.2Hz, significantly better than the traditional ±1Hz error range based on empirical formulas. By focusing on the fault-sensitive frequency band for energy analysis, the system can detect early-stage microcracks in the outer ring even in the presence of a high noise background (SNR < 0dB).

[0032] Specifically, a frequency band protection window is constructed based on the harmonic distribution characteristics of metal fatigue cracks, frequency domain filtering is performed within the window to remove vibration artifacts, and a gradient truncation layer is set in the non-window area to retain the high-frequency characteristics of microcracks. The logical process is as follows: based on the fault characteristic fundamental frequency of the fault-sensitive frequency band as the center, a protection window covering multiple harmonic attenuation areas is constructed, and the window boundary is determined according to the crack harmonic energy attenuation characteristics; an orthogonal projection operator is constructed according to the characteristic matrix of the bearing seat vibration response equation, and an orthogonal projection operation is applied to the spectral components within the window to suppress the vibration coupling components. When the correlation coefficient between the frequency point energy attenuation rate and the entropy change rate of the synchronously collected equipment surface image grayscale co-occurrence matrix is ​​lower than the set threshold, the corresponding frequency area is marked and a band-stop filtering operation is performed; the frequency domain feature tensor is input into the deep learning network, and a mask matrix is ​​constructed based on the frequency dimension. During the backpropagation process, a mask operation is performed on the gradient tensor, allowing only the gradient propagation of the frequency domain area outside the window. A gradient truncation layer is set in the area outside the window to limit the interference of non-target frequency bands on the model training process, and frequency domain regularization is performed on the masked features.

[0033] In this implementation, a protection window covering multiple harmonic attenuation regions is constructed and its boundaries are determined: Centered on the fundamental frequency of the fault characteristic in the fault-sensitive frequency band, and based on the energy attenuation law of metal fatigue crack harmonics, a frequency band protection window covering the fundamental frequency and its multiple harmonic attenuation intervals is constructed to ensure the integrity of the core fault characteristic frequency. The formula (protection window frequency band boundary) is: ; Parameter description: :No. A protection window frequency band interval; :No. harmonic frequencies, based on integer multiples of the fundamental frequency; :No. harmonic order; : Fault characteristic fundamental frequency; :No. The half-width of the frequency band is determined by the energy attenuation of each harmonic; : The total number of protection windows, covering multiple harmonics. Construct an orthogonal projection operator to suppress the spectral components within the window: Based on the characteristic matrix obtained from the bearing seat vibration response equation, construct an orthogonal projection operator to suppress the coupling components and artifacts in the vibration signal in the frequency domain and enhance the significance of the fault characteristics. Formula (orthogonal projection operator): ; Parameter description: : orthogonal projection operator matrix; : Identity matrix, dimension is the same as the feature matrix; : The main eigenvector matrix in the characteristic matrix, the column vector is orthogonal normalized; : The transpose of . Applied to spectral components: ; Parameter description: : original spectral component vector; : Spectral components after projection suppression. Artifact frequency bands are screened and band-stop filtering is performed based on the correlation between the frequency energy decay rate and the rate of change in the image gray-level co-occurrence matrix entropy. The correlation coefficient between the energy decay rate of each frequency point in the spectrum and the rate of change in the gray-level co-occurrence matrix entropy of the synchronously acquired device surface image is calculated. When the correlation coefficient is lower than the preset threshold, the frequency region is considered to contain artifacts and band-stop filtering is performed. Formula (correlation coefficient): ; Parameter description: : correlation coefficient of the lth frequency point; : the energy attenuation rate sequence of the lth frequency point; : The entropy change rate sequence of the gray-level co-occurrence matrix of the image corresponding to the lth frequency point; : covariance function; : The standard deviation of : The standard deviation of. Judgment condition: If , then perform band-stop filtering on this frequency point, parameter description: : Set the correlation coefficient threshold. Construct a mask matrix based on the frequency dimension to limit gradient propagation and implement a gradient truncation layer: Input the frequency domain feature tensor into the deep learning network, construct a mask matrix based on the protection window, mask the gradient of the frequency band within the window, and only allow the gradient of the area outside the window to pass during the back propagation process, thereby preventing non-target frequency bands from interfering with model training and improving the model's sensitivity to high-frequency characteristics of microcracks. Formula expression (mask matrix definition): ; Parameter description: : Mask matrix elements in the frequency dimension; : frequency point; : The aforementioned protection window frequency band interval. Mask gradient operation: ; Parameter description: : Loss function for frequency points in the frequency domain feature tensor gradient; : The eigenvalue of the corresponding frequency point. Regularization processing is performed on the masked frequency domain features: In order to improve the stability of model training and reduce the noise interference between frequency domain features, frequency domain regularization constraints are used to enhance the sparsity or smoothness of the features and ensure the effectiveness of the gradient truncation layer. In the system bearing condition monitoring application, the system first constructs a protection window based on the fundamental frequency of the outer ring fault feature and its multiple harmonic attenuation intervals to cover the key harmonic energy area. For example, for a wind turbine main bearing with a rated speed of 1200rpm and an outer ring defect fundamental frequency of 48Hz, the protection window covers frequency points such as 48Hz, 96Hz, and 144Hz and their attenuation bandwidth range to ensure that the crack harmonic signal is not weakened during the processing process. Within the protection window, the system uses the characteristic matrix obtained from the solution of the bearing seat vibration response equation to construct an orthogonal projection operator, and performs projection suppression operations on the spectral components, effectively weakening the vibration coupling components introduced by factors such as blade aerodynamic disturbances and tower structure resonance. This operation significantly reduces the interference peak intensity close to the crack harmonic frequency. For non-target frequency bands, the system analyzes the correlation between the energy attenuation rate and the entropy change rate of the grayscale co-occurrence matrix of the device surface image to identify artifact frequency ranges unrelated to crack development and then applies band-stop filtering to remove them. This effectively suppresses high-frequency artifacts caused by variations in the wind turbine operating environment (such as wind speed fluctuations and temperature differences). During the deep learning training phase, the system generates a frequency-dimensional mask matrix based on a protection window. This selectively blocks the backpropagation gradient, allowing only high-frequency gradient components outside the window to propagate. This emphasizes the high-frequency characteristic response of microcracks while suppressing the impact of low-frequency interference on model parameter updates. This gradient truncation strategy allows the model to focus on optimizing weights related to early crack signatures during training. By constructing a protection window covering multiple harmonic attenuation intervals, the critical frequency components of metal fatigue cracks are not weakened during signal processing and model training, improving the fidelity of early crack signatures. An orthogonal projection operation effectively attenuates structurally coupled noise peaks close to the fault frequency, reducing the average interference peak amplitude during field testing. The gradient truncation layer reduces gradient perturbations in non-target frequency bands, enabling the deep learning model to converge to a stable fault feature extraction model early in training, shortening the number of training iterations. Even under strong background noise caused by wind speed fluctuations, the system can still detect microcrack harmonic signals with an amplitude of only 0.7 times the RMS value of the noise, outperforming detection solutions that do not employ protection windows and gradient truncation strategies.

[0034] Specifically, the specific process of receiving the high-frequency characteristics of the gradient truncation layer and detecting the phase offset of the cross-correlation function with the image clarity when the speed suddenly changes is as follows: based on the high-frequency characteristics of the microcracks in the gradient truncation layer, the speed sudden change event is locked in combination with the rising edge mark of the speed sensor pulse, and the vibration envelope signal and the synchronous image clarity sequence in the dynamic time window are intercepted with the event moment as the center; a complex analytical wavelet transform is performed on the data in the window to extract the instantaneous main phase angle of the vibration envelope component and the image clarity component in the complex domain, and the instantaneous phase difference between the two components before and after the speed sudden change point is calculated. In combination with the equipment gear meshing transmission ratio parameter, the phase difference is converted into the relative displacement offset between the bearing raceway and the visual monitoring area.

[0035] In this implementation, the speed mutation event is locked, and the vibration envelope signal and the synchronous image clarity sequence within the dynamic time window are intercepted: the rising edge of the pulse signal output by the speed sensor is used to mark the moment of speed mutation. With this moment as the center, a time window of sufficient length before and after the mutation is selected, and the vibration envelope signal of the corresponding period and the synchronously collected device visual image clarity sequence are intercepted to ensure the spatiotemporal correspondence of the analyzed data. A complex analytic wavelet transform is performed on the intercepted signal within the window to extract the instantaneous main phase angle: a complex analytic wavelet transform is performed on the vibration envelope signal and the image clarity sequence respectively to obtain a complex number representation of the signal in the time-frequency two-dimensional domain, from which the instantaneous main phase angle is extracted to describe the phase characteristics of the signal at the local time point. Formula representation (complex analytic wavelet transform): ; Parameter description: :Signal At the moment ,scale Complex analytic wavelet coefficients under ; : Signal to be analyzed (vibration envelope signal or image clarity sequence); : mother wavelet function; : complex conjugate of the mother wavelet; : Time shift parameters; : Scale parameter, inversely proportional to frequency. Instantaneous main phase angle extraction: ; Parameter description: :Signal In time ,scale The instantaneous main phase angle under ; : Complex argument function. Calculate the instantaneous phase difference between the two signals before and after the speed mutation point: Select a specific time point or time interval before and after the speed mutation event, and calculate the instantaneous main phase angle difference between the vibration envelope signal and the image clarity sequence at the corresponding moment, reflecting the phase synchronization and offset characteristics of the two signals at this event. Formula expression (instantaneous phase difference): ; Parameter description: : Vibration envelope signal ( ) and image definition sequence ( )’s instantaneous phase difference; : instantaneous phase of vibration envelope signal; : The instantaneous phase of the image clarity sequence. Combined with the gear meshing transmission ratio, the phase difference is converted into a relative displacement offset: Based on the equipment gear meshing transmission ratio, the obtained phase difference is quantified as a relative displacement offset between the bearing raceway and the visual monitoring area, revealing the dynamic impact of sudden speed changes on the operating state. The formula (relative displacement offset) is: ; Parameter description: : The relative displacement between the bearing raceway and the visual monitoring area; : instantaneous phase difference (radians); : characteristic wavelength corresponding to the fault frequency; Gear ratio, defined as the ratio of input shaft speed to output shaft speed. During bearing condition monitoring, after extracting high-frequency microcrack features using a gradient truncation layer, the system uses the rising edge marker of the speed sensor pulse to identify a sudden speed change caused by a sudden wind event. Centered on the event, the system automatically captures a dynamic time window encompassing 1.5 seconds before and after the change, extracting the vibration envelope signal and the simultaneously acquired visual image clarity sequence of the gearbox housing surface, ensuring precise temporal correspondence between the two data types. Within this time window, the system performs a complex analytic wavelet transform on both the vibration envelope signal and the image clarity sequence, obtaining a complex representation of the signals in the time-frequency domain and extracting the instantaneous main phase angle. By comparing the phases of the same characteristic frequency points before and after the speed change, the system determines the instantaneous phase difference between the vibration signature and the image clarity signature. Subsequently, combining the gear ratio parameters of the wind turbine main drive system (the input-to-output shaft speed ratio is 3.25:1), the system quantifies the phase difference as the relative displacement between the bearing outer ring raceway and the visual monitoring area. This offset reflects the dynamic impact of a sudden speed change on the bearing's operating conditions, such as the relative motion change between the raceway and the cage, and can serve as an important reference indicator for crack propagation sensitivity. By leveraging the rising edge marker and dynamic time window strategy of the speed sensor, millisecond-level synchronization of vibration and visual modal data is achieved during sudden speed changes, providing a reliable basis for phase difference calculation. Using high-frequency features extracted from the gradient truncation layer as input ensures that phase analysis focuses on the microcrack-sensitive frequency range, improving the accuracy of phase difference quantification. In field tests, the error in phase difference calculation was less than the set radian. Through transmission ratio correction, the abstract phase difference is converted into an interpretable relative displacement offset, allowing maintenance personnel to intuitively determine the actual impact of speed disturbances on bearing fit clearance and contact stress. During multiple sudden wind events, the phase difference-displacement curve showed a continuous increase, which was highly consistent with the crack length growth discovered during subsequent disassembly and inspection, verifying the effectiveness of this method in monitoring early crack propagation.

[0036] Specifically, the impact fault propagation path is identified by the envelope spectrum kurtosis, and the specific process of dynamically shrinking the frequency domain window bandwidth in combination with real-time load data is as follows: the vibration signal is demodulated in the full-band envelope through complex Morlet wavelet to generate a scale energy distribution matrix; the kurtosis value of the envelope spectrum at each scale is calculated, and when the kurtosis peak value of a specific frequency band exceeds the material yield strength correlation threshold, it is marked as an impact transmission path. A negative exponential correlation function between the frequency domain analysis window bandwidth and the real-time load is established, and the analysis window bandwidth is dynamically compressed according to the load value within the marked impact path frequency band.

[0037] In this implementation, a complex Morlet wavelet is used to perform full-band envelope demodulation on the vibration signal to generate a scale energy distribution matrix. The complex Morlet wavelet transform is used to perform multi-scale envelope demodulation on the vibration signal, extracting the envelope energy distribution at different scales (corresponding to frequencies) to form a two-dimensional scale-time energy matrix that reflects the distribution of the signal's impact characteristics over frequency. The complex Morlet wavelet is a complex wavelet used for signal processing, consisting of a complex exponential function multiplied by a Gaussian window function. The formula (complex Morlet wavelet transform) is: ; Parameter description: :Signal In scale and time The complex wavelet coefficients at ; : vibration signal; : Morlet mother wavelet function; : complex conjugate of the Morlet mother wavelet; : scale parameter, the scale is inversely proportional to the frequency; : Time translation parameter. Scale energy distribution matrix definition: ; Parameter description: :scale The corresponding average envelope energy; : Analyze the time window length. Calculate the kurtosis value of the envelope spectrum at each scale and identify the impact transmission path: The kurtosis index measures the sharpness of the envelope spectrum at each scale, reflecting the strength of the signal impact component. The frequency band corresponding to the kurtosis peak represents the impact fault propagation path. Formula (kurtosis calculation): ; Parameter description: :scale The corresponding kurtosis value; : scale energy; : the mean value of scale energy; :Mathematical expectation operation. Impact path judgment condition: If , then the frequency band corresponds to the impact transmission path, parameter description: : Kurtosis threshold associated with material yield strength. Establish a negative exponential correlation function between the frequency domain analysis window bandwidth and the real-time load: To reflect the impact of load changes on the frequency domain analysis window bandwidth, establish a negative exponential function that changes the window bandwidth with the real-time load to achieve dynamic bandwidth adjustment. Formula (dynamic bandwidth adjustment): ; Parameter description: :The load value is Dynamic frequency domain window bandwidth when ; : initial frequency domain window bandwidth; : Load impact index, controls bandwidth shrinking speed; : Real-time load value. In the marked impact path frequency band, dynamically compress the analysis window bandwidth according to the load value: For the frequency band that has been determined to be the impact transmission path, adjust the window bandwidth according to the real-time load value, compress the frequency band range to improve the focus of frequency domain analysis and fault identification accuracy. The dynamic bandwidth function obtained according to the above steps , the frequency range of the impact path frequency band is scaled accordingly, thereby achieving dynamic contraction of the frequency domain window bandwidth. During the long-term operation monitoring of the wind turbine main transmission system, after completing multi-modal synchronous acquisition, the system uses complex Morlet wavelets to perform full-band envelope demodulation on the main bearing vibration signal to generate a scale-time two-dimensional energy distribution matrix. Subsequently, the system calculates the kurtosis value of the envelope spectrum at each scale to measure the sharpness of the signal impact component. When the kurtosis peak of a certain frequency band exceeds the threshold associated with the yield strength of the main bearing material, the frequency band is determined to be the propagation path of the impact fault. The system establishes a negative exponential correlation function between the frequency domain analysis window bandwidth and the load within the impact path frequency band to achieve dynamic contraction of the bandwidth. When the wind conditions suddenly change or the load increases significantly, the system automatically compresses the analysis bandwidth of the impact path frequency band from the original ±5Hz to ±2Hz to improve the spectrum focus and signal-to-noise ratio. In field measurements, a sudden increase in wind speed from 12 m / s to 17 m / s caused the kurtosis value of the main bearing impact path frequency band (approximately 185 Hz) to increase by 2.4 times. Simultaneously, the system compressed the analysis bandwidth by 60% based on real-time load fluctuations, ultimately clearly isolating the overtone components caused by outer race fatigue cracks in this frequency band. By combining the kurtosis peak with the material yield strength threshold, the impact propagation path identification no longer relies on fixed empirical values ​​but instead matches the structural mechanical properties, reducing false positives. A load-driven negative exponential bandwidth adjustment mechanism automatically increases analysis focus as the load increases, making it particularly suitable for conditions with large wind speed fluctuations and brief impact signals. Dynamic bandwidth reduction not only reduces interference from non-target frequency bands but also enhances the detectability of weak overtone components near the impact path frequency. Precise positioning of the impact path provides the target frequency input for the aforementioned "protection window + gradient truncation layer" and provides higher feature purity for the subsequent "phase difference-relative displacement monitoring" module, forming an end-to-end, highly sensitive monitoring chain.

[0038] Specifically, the motion trajectory of the bearing raceway feature in the phase space is constructed. When the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold, the frequency band window readjustment instruction is generated and fed back to the feature decoupling module. The specific process is as follows: the high-frequency feature vector of the microcrack is reconstructed into the phase space through time delay embedding, and the dimensionality is reduced by the local linear embedding algorithm to generate a three-dimensional feature trajectory; the curvature change rate of the trajectory is calculated. When the curvature change rate exceeds the set change range, the window bandwidth expansion is calculated according to the crack depth change rate, and a readjustment instruction containing the target bandwidth value is generated and fed back to the feature decoupling module.

[0039] In this implementation, high-frequency microcrack feature vectors are embedded in the phase space to reconstruct the trajectory. The collected high-frequency microcrack feature vector sequence is used to construct a time-delay vector, reconstructing the high-dimensional phase space trajectory and revealing the dynamic evolution of the system. Formula (time-delay embedding): ; Parameter description: :time The reconstructed phase space vector of ; : high-frequency characteristic sequence of original microcracks; : time delay step; : Embedding dimension. Use the local linear embedding algorithm to reduce the high-dimensional trajectory to a three-dimensional feature trajectory: To facilitate trajectory analysis, the local linear embedding (LLE) nonlinear dimensionality reduction algorithm is applied to map the high-dimensional reconstructed trajectory to three-dimensional space, maintaining the local neighborhood structure. Construct the neighborhood weight matrix, solve the eigenvalue problem, and obtain the coordinates after dimensionality reduction. Calculate the curvature of the three-dimensional trajectory and its rate of change: The trajectory curvature reflects the degree of curvature of the motion trajectory, and its rate of change reflects the abnormal dynamic fluctuations of the system. Formula (trajectory curvature): Assume that the trajectory is in parameter form , the curvature is defined as ; Parameter description: :parameter Curvature of the trajectory; :Trajectory in The first derivative vector at ; :Trajectory in The second derivative vector at ; :vector cross product; : Vector norm. Curvature change rate calculation: ; Parameter description: :Curvature pair parameter The derivative of is the curvature change rate. When the curvature change rate exceeds the threshold associated with the material fatigue coefficient, the window bandwidth expansion is calculated and the frequency band window readjustment instruction is generated. Description: When the trajectory curvature change rate exceeds the preset threshold, the frequency bandwidth is adjusted in combination with the crack depth change rate to optimize the frequency domain filter window in the feature decoupling module. Formula (bandwidth expansion calculation): ; Parameter description: : Bandwidth expansion of the frequency window; : Bandwidth adjustment coefficient, reflecting the sensitivity of frequency expansion to crack changes; : the rate of change of crack depth with time; : Step function, takes 1 when the input is greater than zero, otherwise takes 0; : trajectory curvature change rate; : The curvature change rate threshold associated with the material fatigue coefficient. Band window readjustment instruction: The expanded bandwidth value The data is sent to the feature decoupling module to update the frequency domain filtering protection window width. During long-term condition monitoring of wind turbines, after extracting high-frequency features of main bearing microcracks, the system reconstructs this feature vector sequence into a high-dimensional phase space using a time-delay embedding method to reveal the dynamic evolution trajectory of the crack during propagation. To facilitate real-time analysis, the system uses a local linear embedding (LLE) algorithm to reduce the high-dimensional trajectory to a three-dimensional feature trajectory, preserving the local neighborhood geometry while reducing the computational burden. During continuous monitoring, the system calculates the curvature and rate of change of the three-dimensional trajectory in real time. When the curvature rate of change consistently exceeds a preset threshold related to the fatigue coefficient of the main bearing material, the crack is determined to have entered a high-risk zone. At this point, the system combines the crack depth change rate estimated by ultrasonic testing or high-frequency envelope demodulation to calculate the required frequency window bandwidth expansion and generates a readjustment instruction containing the target bandwidth value. This instruction is fed back to the feature decoupling module to dynamically expand the protection window width so that subsequent frequency domain filtering can cover the newly generated high-frequency components involved in crack evolution. In one actual O&M case, a wind turbine operating at full load detected a main bearing track curvature change rate exceeding 1.8 times the fatigue factor threshold for 30 consecutive minutes, accompanied by an increase in the crack depth change rate. The system automatically expanded the protection window width from ±4Hz to ±6.5Hz, recaptured the 2.5-octave characteristic peak that had been filtered out, and ultimately confirmed that the crack had entered the rapid growth phase. It then scheduled a maintenance shutdown in advance, avoiding significant gearbox damage. The system dynamically responds to crack state changes by determining sudden changes in the structural stress state using the phase space curvature change rate, avoiding the hysteresis caused by relying on a single frequency domain threshold. By linking the crack depth change rate with the curvature change rate, the frequency band window dynamically expands as the structure degrades, ensuring that emerging high-frequency fault components are not missed. During turbine load fluctuations and sudden crack growth, the system can trigger maintenance recommendations days to weeks in advance, significantly reducing the probability of major failures. The bandwidth extension instructions generated by this module directly act on the feature decoupling module, forming a dynamic feature optimization closed loop with modules such as "protection window + gradient truncation layer", "impact path identification", and "phase difference displacement detection".

[0040] Specifically, when temperature fluctuations cause trajectory drift, the specific process of updating the delay compensation parameters to the dynamic timing alignment module is as follows: the temperature sensor data is mapped to the feature space through the thermal expansion coefficient matrix, and the temperature-sensitive component is separated from the motion trajectory; when the trajectory drift exceeds the allowable error band of the material thermal deformation, the partial derivative of the drift with respect to the delay parameter is calculated, the learning step size of the gradient descent method is set based on the material thermal conductivity rate, the delay compensation amount is iteratively updated and fed back to the timing alignment module.

[0041] In this implementation, the temperature sensor data is mapped to the phase space characteristic dimension. The temperature change is mapped to the trajectory coordinate system using the material's thermal expansion sensitivity to determine the thermally induced trajectory drift component. Mathematical representation (thermal expansion mapping): ; Parameter description: :time The corresponding temperature mapping vector represents the thermal disturbance component of each feature dimension; : Thermal expansion coefficient matrix, reflecting the sensitivity of each characteristic dimension to temperature changes; : Temperature sensor data vector. Separate the temperature sensitive component from the phase space trajectory. Description: Separate the above thermal disturbance component from the total trajectory to obtain the drift vector. Calculate the drift amount: ; Parameter description: :time The trajectory drift at ; : original phase space trajectory point; : Thermal disturbance component. Determine whether the thermal drift exceeds the material tolerance threshold. Judgment conditions: ; Parameter description: : The allowable error band for material thermal deformation is derived from the thermal stability limit. When the drift exceeds the error band, the delay compensation parameter adjustment process is triggered. Delay compensation gradient calculation and iterative update step 3: Calculate the gradient of the trajectory drift with respect to the delay compensation parameter. Mathematical representation: ; Parameter description: : The delay parameters currently used for delay compensation; : Temperature disturbance mapping vector after delay compensation adjustment; : Gradient operator for the delay compensation. Sets an adaptive learning rate based on the material's thermal conductivity and updates the compensation mathematical representation (gradient descent update): ; Parameter description: : compensation value of the kth iteration; : Learning step, set to the material heat conduction rate The reciprocal weighted form of ; : Gradient of trajectory drift with respect to current compensation parameters; : Material thermal conductivity. Learning rate adaptive representation: ; : Regularization term, to prevent the learning rate from being too large, causing oscillation or non-convergence. The feedback provided as a delay compensation value to the timing alignment module explains: The new compensation value is used to correct for time synchronization errors between image signals, vibration data, and thermal sensor data, maintaining the consistency of multimodal data alignment. During the long-term multimodal condition monitoring of wind turbines, the system constructs the phase space characteristic trajectory of the main bearing and key transmission chain components while simultaneously receiving data from multiple temperature sensors installed on the bearing seat, gearbox, and nacelle casing. Based on the pre-established thermal expansion coefficient matrix in the material database, the system maps temperature changes to the trajectory feature space to identify characteristic trajectory drift components caused by thermal expansion or contraction. When the monitored trajectory drift continuously exceeds a threshold value associated with the material's thermal deformation tolerance (for example, the feature space mapping amplitude corresponding to the thermal stability limit tolerance of ±0.08mm for the main bearing outer ring made of 42CrMo alloy steel), the system determines that the temperature disturbance has significantly affected the time synchronization accuracy of the multimodal data. At this point, the module initiates a gradient-based iterative update of the delay compensation parameters. First, the sensitivity (gradient) of trajectory drift to the current delay compensation parameters is calculated. Then, an adaptive learning step size is set based on the material's thermal conductivity (e.g., a thermal diffusion rate of approximately 45 W / (m·K) for steel). This ensures that parameter adjustments can quickly respond to sudden temperature fluctuations while avoiding synchronization parameter oscillations caused by excessive learning rates. Finally, the updated delay compensation value is fed back to the dynamic timing alignment module in real time. A mapping relationship between temperature and trajectory is established using the material's thermal expansion coefficient matrix, enabling accurate separation of thermally sensitive components before multimodal signal fusion. Adaptive adjustment of the gradient descent step size based on the material's thermal conductivity allows for rapid response to sudden temperature fluctuations while avoiding parameter oscillations and improving system stability. The updated compensation value is fed back to the dynamic timing alignment module in real time, ensuring high-precision alignment of image, vibration, and thermal signals within milliseconds.

[0042] See also Figure 2, an intelligent cleaning and feature extraction method for multimodal industrial data, comprising the following steps: S1. dynamically adjusting the matching window size of the time warping algorithm by rotating the main vibration frequency of the equipment, and calculating the temperature signal transmission delay based on the thermal expansion characteristics of the equipment material; S2. receiving the delay and synchronizing the multimodal data time base, solving the fault-sensitive frequency band through the bearing seat vibration response equation, constructing a frequency band protection window based on the harmonic distribution characteristics of metal fatigue cracks, performing frequency domain filtering within the window to remove vibration artifacts, and setting a gradient truncation layer in the non-window area to retain the high-frequency characteristics of microcracks; S3. receiving the high-frequency characteristics of microcracks in the gradient truncation layer, detecting the phase offset of the cross-correlation function with the image clarity when the speed suddenly changes, identifying the impact fault propagation path through the envelope spectrum kurtosis, and dynamically shrinking the frequency domain window bandwidth in combination with real-time load data; S4. constructing the motion trajectory of the bearing raceway feature in the phase space. When the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold, generating a frequency band window readjustment instruction and feeding it back to the feature decoupling module. When temperature fluctuations cause trajectory drift, updating the delay compensation parameters to the dynamic timing alignment module.

[0043] In this implementation, step S1: This step extracts the dominant vibration frequency of the rotating equipment in real time and dynamically adjusts the matching window size in the time warping algorithm to adapt to the data alignment deviation caused by the fluctuation of the equipment speed; at the same time, combined with the thermal expansion coefficient and heat conduction mechanism of the equipment material, quantifies the delay compensation amount of the temperature signal propagation relative to the vibration reference, and realizes accurate time synchronization between multimodal data. This solution introduces a frequency-driven adaptive window mechanism, which significantly improves the dynamic response capability of data alignment; introduces the thermal expansion-conduction modeling mechanism into the time compensation estimation, realizes the time domain correction of the temperature signal under industrial thermal load conditions, and solves the problem of "image-temperature-vibration" three-type data mismatch in high-temperature non-steady-state scenarios in existing methods; step S2: This step first uses the bearing seat dynamic model and random subspace identification to extract the fault-sensitive frequency band, and then combines the harmonic distribution characteristics of metal fatigue cracks to construct a frequency band protection window, removes artifacts within the window, and sets a gradient truncation layer in the non-window area to retain the high-frequency effective characteristics of microcracks, achieving a balance between frequency domain noise suppression and high-sensitivity damage extraction. A gradient truncation strategy combined with a frequency band protection mechanism is proposed to only allow features in specific frequency bands to propagate to the model gradient update path, preventing non-target frequencies from interfering with network training and improving the stability and pertinence of feature extraction. Step S3: By detecting the main phase offset of the cross-correlation between the image clarity function and the high-frequency vibration features before and after the speed mutation event, the spatial offset relationship between microcrack propagation and the visible area is estimated. Furthermore, the envelope spectrum kurtosis is used to identify the impact fault path, and the bandwidth of the frequency domain analysis window is adaptively adjusted in combination with real-time load data. The kurtosis spectrum is coupled with the load to construct a dynamic bandwidth compression function, effectively avoiding non-target frequency interference under heavy loads. Overall, the dynamic feature causal modeling of image, vibration, and load is integrated to form an event-driven path identification mechanism with multi-source constraints and a clearly structured innovative path. Step S4: The motion trajectory of the bearing raceway features in phase space is reconstructed through time delay embedding and manifold dimensionality reduction. The curvature change rate is calculated to determine the crack evolution state, and this is used as dynamic feedback to reset the frequency band window. Simultaneously, trajectory drift caused by temperature fluctuations triggers a gradient descent update of the delay compensation parameters, achieving a closed-loop adaptive adjustment mechanism for the entire system chain. A partial derivative relationship and a heat conduction-dominated gradient descent optimization model are constructed between trajectory drift and temperature compensation, ensuring the system's stability and adaptability under thermal disturbances. The process from high-frequency features to phase trajectory to feedback window readjustment to correct delay compensation embodies a highly coupled, autonomous, and stable multimodal intelligent processing logic.

[0044] In summary, this application has at least the following effects:

[0045] A system and method for intelligent cleaning and feature extraction of multimodal industrial data utilizes an adaptive window adjustment mechanism driven by the main vibration frequency of rotating equipment and a temperature signal delay compensation model based on material thermal expansion parameters. This achieves high-precision time alignment between different modal data (vibration, temperature, and image) under dynamic operating conditions, effectively reducing timing deviations caused by temperature fluctuations, speed variations, and other factors. A frequency band protection window based on the harmonic distribution of metal fatigue cracks, combined with a frequency domain gradient truncation mechanism, suppresses vibration artifacts while preserving the characteristics of minor structural damage, improving early fault detection capabilities. Impact path identification logic is constructed based on the main phase shift of image clarity cross-correlation and the peak of envelope spectrum kurtosis. The frequency domain analysis bandwidth is dynamically adjusted using real-time load data, enabling linked modeling and coupled analysis of visual and vibration data during critical events (such as sudden speed changes). By reconstructing the high-frequency characteristic trajectory of microcracks in phase space and combining the rate of change of trajectory curvature to drive the frequency band window readjustment logic, a closed-loop feedback mechanism is formed in which data characteristics act inversely on parameter settings, enhancing the system's adaptability and generalization performance. A functional gradient relationship is established between the trajectory drift caused by temperature fluctuations and the delay parameters, the iterative learning rate is set based on the thermal conductivity characteristics of the material, and the compensation parameters are updated in real time, thereby improving the stability and accuracy of multimodal data fusion processing in high-temperature and complex industrial environments.

[0046] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may be implemented entirely in hardware, entirely in software, or in a combination of software and hardware. Furthermore, the present invention may be implemented as a computer program product embodied in one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0047] The present invention is described with reference to flowcharts and / or block diagrams of systems, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0048] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0049] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0050] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0051] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. Intelligent cleaning and feature extraction system for multimodal industrial data, characterized by: Includes the following modules: timing alignment module, feature decoupling module, operating condition causal analysis module, feature stability optimization module; The timing alignment module is used to dynamically adjust the matching window size of the time warping algorithm by rotating the main vibration frequency of the device, and calculate the temperature signal transmission delay based on the thermal expansion characteristics of the device material; The feature decoupling module is used to receive the delay amount and synchronize the multimodal data time base, solve the fault-sensitive frequency band through the bearing seat vibration response equation, build a frequency band protection window based on the harmonic distribution characteristics of the metal fatigue crack, perform frequency domain filtering within the window to remove vibration artifacts, and set a gradient truncation layer in the non-window area to retain the high-frequency characteristics of microcracks; The operating condition causal analysis module is used to receive the high-frequency characteristics of microcracks in the gradient truncation layer, detect the phase offset of the cross-correlation function with the image clarity when the speed suddenly changes, identify the impact fault propagation path through the envelope spectrum kurtosis, and dynamically shrink the frequency domain window bandwidth in combination with real-time load data; The feature stability optimization module is used to construct the motion trajectory of the bearing raceway feature in phase space. When the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold, a frequency band window readjustment instruction is generated and fed back to the feature decoupling module. When temperature fluctuations cause trajectory drift, the delay compensation parameters are updated in the dynamic timing alignment module.

2. The intelligent cleaning and feature extraction system for multimodal industrial data according to claim 1, characterized in that: The timing alignment module includes the following steps: Collect vibration signals of rotating equipment in real time, extract the dominant vibration frequency component, and periodically adjust the matching window size of the time warping algorithm based on the frequency; The thermal expansion coefficient is obtained from the equipment material database, and the thermal deformation of the metal structure is calculated based on the temperature sensor data. The time delay compensation of the temperature signal relative to the vibration reference is derived based on the propagation speed of the sound wave in the metal medium.

3. The intelligent cleaning and feature extraction system for multimodal industrial data according to claim 2, characterized in that: The specific process of receiving the delay and synchronizing the multimodal data time base, and solving the fault-sensitive frequency band through the bearing seat vibration response equation is as follows: The time deviation between the image frame and the vibration spectrum is calibrated by the time delay compensation value to generate a time-aligned multimodal data stream; A group of differential equations for the bearing seat dynamics is established, and the system characteristic matrix is ​​solved through random subspace identification. The characteristic frequency components related to rolling element damage are extracted. The characteristic fundamental frequency of the outer ring fault and its harmonic distribution range are calculated based on the bearing geometric structure parameters, and the fault-sensitive frequency band is determined.

4. The intelligent cleaning and feature extraction system for multimodal industrial data according to claim 3, characterized in that: The logical process of constructing a frequency band protection window based on the harmonic distribution characteristics of metal fatigue cracks, performing frequency domain filtering within the window to remove vibration artifacts, and setting a gradient truncation layer in the non-window area to retain the high-frequency characteristics of microcracks is as follows: Based on the fundamental frequency of the fault characteristic in the fault-sensitive frequency band, a protection window covering multiple harmonic attenuation areas is constructed, and the window boundary is determined according to the crack harmonic energy attenuation characteristics. An orthogonal projection operator is constructed based on the characteristic matrix of the bearing seat vibration response equation. The orthogonal projection operation is applied to the spectral components within the window to suppress the vibration coupling components. When the correlation coefficient between the frequency point energy attenuation rate and the entropy change rate of the gray-level co-occurrence matrix of the synchronously collected equipment surface image is lower than the set threshold, the corresponding frequency region is marked and a band-stop filtering operation is performed. The frequency domain feature tensor is input into the deep learning network, and a mask matrix is ​​constructed based on the frequency dimension. During the back-propagation process, a mask operation is performed on the gradient tensor, allowing only the gradient propagation in the frequency domain area outside the window. A gradient truncation layer is set in the area outside the window to limit the interference of non-target frequency bands on the model training process, and frequency domain regularization is performed on the masked features.

5. The intelligent cleaning and feature extraction system for multimodal industrial data according to claim 4 is characterized by: The specific process of receiving the high-frequency features of the gradient truncation layer and detecting the phase offset of the cross-correlation function with the image clarity when the speed suddenly changes is as follows: Based on the high-frequency characteristics of micro-cracks in the gradient truncation layer, the rising edge of the speed sensor pulse is used to mark the speed mutation event, and the vibration envelope signal and the synchronous image clarity sequence within the dynamic time window are intercepted with the event moment as the center. A complex analytical wavelet transform is performed on the data in the window to extract the instantaneous main phase angles of the vibration envelope component and the image clarity component in the complex domain. The instantaneous phase difference between the two components before and after the speed mutation point is calculated. Combined with the equipment gear meshing transmission ratio parameter, the phase difference is converted into the relative displacement offset between the bearing raceway and the visual monitoring area.

6. The intelligent cleaning and feature extraction system for multimodal industrial data according to claim 5, characterized in that: The specific process of identifying the impact fault propagation path through the envelope spectrum kurtosis and dynamically shrinking the frequency domain window bandwidth in combination with real-time load data is as follows: The vibration signal is demodulated in full-band envelope by complex Morlet wavelet to generate scale energy distribution matrix; The kurtosis value of the envelope spectrum at each scale is calculated. When the kurtosis peak value in a specific frequency band exceeds the threshold associated with the material yield strength, it is marked as the impact transfer path. A negative exponential correlation function between the frequency domain analysis window bandwidth and the real-time load is established. The analysis window bandwidth is dynamically compressed according to the load value within the marked impact path frequency band.

7. The intelligent cleaning and feature extraction system for multimodal industrial data according to claim 6, characterized in that: The motion trajectory of the bearing raceway feature in phase space is constructed. When the rate of change of the trajectory curvature exceeds the threshold associated with the material fatigue coefficient, a frequency band window readjustment instruction is generated and fed back to the feature decoupling module. The specific process is as follows: The high-frequency feature vector of the microcrack is reconstructed into the phase space through time-delay embedding, and the dimensionality is reduced by the local linear embedding algorithm to generate a three-dimensional feature trajectory; The curvature change rate of the trajectory is calculated. When the curvature change rate exceeds the set change range, the window bandwidth expansion is calculated according to the crack depth change rate, and a readjustment instruction containing the target bandwidth value is generated and fed back to the feature decoupling module.

8. The intelligent cleaning and feature extraction system for multimodal industrial data according to claim 7, characterized in that: When temperature fluctuations cause trajectory drift, the specific process of updating the delay compensation parameters to the dynamic timing alignment module is as follows: The temperature sensor data is mapped to the feature space through the thermal expansion coefficient matrix to separate the temperature sensitive component from the motion trajectory; When the trajectory drift exceeds the allowable error band of the material thermal deformation, the partial derivative of the drift with respect to the delay parameter is calculated. The learning step size of the gradient descent method is set based on the material thermal conductivity rate. The delay compensation amount is iteratively updated and fed back to the timing alignment module.

9. A method for intelligent cleaning and feature extraction of multimodal industrial data, applied to the intelligent cleaning and feature extraction system for multimodal industrial data according to any one of claims 1 to 8, characterized in that: The following steps are involved: S1. Dynamically adjust the matching window size of the time warping algorithm by rotating the device's main vibration frequency and calculate the temperature signal transmission delay based on the thermal expansion characteristics of the device material; S2. Receive the delay and synchronize the multimodal data time base. Determine the fault-sensitive frequency band using the bearing seat vibration response equation. Construct a frequency band protection window based on the harmonic distribution characteristics of metal fatigue cracks. Perform frequency domain filtering within the window to remove vibration artifacts. Set a gradient truncation layer in the non-window area to preserve the high-frequency characteristics of microcracks. S3. Receive high-frequency characteristics of microcracks in the gradient truncation layer, detect the phase offset of the cross-correlation function with the image clarity when the rotational speed changes suddenly, identify the impact fault propagation path through the envelope spectrum kurtosis, and dynamically shrink the frequency domain window bandwidth based on real-time load data; S4. Construct the motion trajectory of the bearing raceway feature in the phase space. When the trajectory curvature change rate exceeds the material fatigue coefficient correlation threshold, generate a frequency band window readjustment instruction and feed it back to the feature decoupling module. When temperature fluctuations cause trajectory drift, update the delay compensation parameters to the dynamic timing alignment module.

Images