Electromechanical equipment health assessment and early warning method based on multi-mode dynamic perception

Abstract

The invention discloses an electromechanical equipment health assessment and early warning method based on multi-mode dynamic perception, and belongs to the field of intelligent operation and maintenance of electromechanical equipment. The problems that in the prior art, a single physical quantity cannot comprehensively reflect the equipment state and a traditional signal processing algorithm cannot adapt to the equipment degradation mode change are solved, a panoramic sensing system covering multiple physical fields such as vibration, temperature and noise is constructed through a multi-mode sensor network and a dynamic weight fusion algorithm, and the multi-physical-field multi-physical-field panoramic sensing method is applied to the multi-physical-field multi-physical-field panoramic sensing system. The problem of isolated island of traditional single-dimensional monitoring information is solved; through a physical-depth mixed feature extraction architecture, combining interpretable engineering features with abstract features extracted by a deep neural network to form a health assessment model with mechanism transparency and mode generalization ability; through deep integration of the digital twin platform and the RPA technology, the manual inspection frequency and workload are reduced, the fault recognition accuracy is promoted to increase year by year, and continuously optimized intelligent operation and maintenance ecology is formed.

Description

Technical Field

[0001] The present invention relates to the field of intelligent operation and maintenance of electromechanical equipment, and specifically to a health assessment and early warning method for electromechanical equipment based on multimodal dynamic perception. Background Art

[0002] In the operation and maintenance of building mechanical and electrical equipment, such as chillers, water pumps, and transformers, traditional technologies generally adopt a single-dimensional monitoring + manual inspection model, which has significant technical flaws and efficiency bottlenecks. The following details the existing technologies and their drawbacks: 1. Single sensor independent monitoring technology: Use a single type of sensor to monitor equipment status. Install a piezoelectric accelerometer to detect abnormal vibration frequencies of mechanical components such as bearings and gears through spectrum analysis. Use a PT100 thermal resistor or infrared temperature probe to set a fixed threshold to trigger an alarm.

[0003] However, a single physical quantity cannot fully reflect the status of the equipment. For example, early wear of a bearing may only manifest as an increase in vibration energy in a specific frequency band, while the temperature has not yet changed significantly, resulting in missed detection; electromagnetic noise and airflow disturbances in the computer room can easily lead to false alarms; when fault characteristics appear in a single signal, the equipment has often entered the stage of irreversible damage.

[0004] 2. Limitations of traditional signal processing algorithms: Vibration spectrum analysis using Fourier transform cannot capture the time-varying characteristics of non-stationary signals, such as transient shock caused by bearing spalling.

[0005] Although wavelet packet decomposition can extract frequency band energy, it requires manual selection of the number of decomposition layers, making it difficult to adapt to the characteristics of different devices. Dimensionality reduction is performed through principal component analysis, but the linear assumption does not match the nonlinear degradation process of the device state. Moreover, the feature extraction rules rely on expert experience and cannot adapt to changes in device degradation patterns.

[0006] Therefore, it does not meet the existing needs. We propose a health assessment and early warning method for electromechanical equipment based on multimodal dynamic perception. Summary of the Invention

[0007] The purpose of the present invention is to provide a health assessment and early warning method for electromechanical equipment based on multimodal dynamic perception. Through a multimodal sensor network and a dynamic weight fusion algorithm, a panoramic perception system covering multiple physical fields such as vibration, temperature, and noise is constructed, solving the problem of traditional single-dimensional monitoring information islands; through a physical-deep hybrid feature extraction architecture, interpretable engineering features are combined with abstract features extracted by deep neural networks to form a health assessment model with both mechanism transparency and pattern generalization capabilities; through the deep integration of the digital twin platform and RPA technology, the frequency and workload of manual inspections are reduced, the accuracy of fault identification is promoted to increase year by year, and a continuously optimized intelligent operation and maintenance ecosystem is formed, solving the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions: A method for health assessment and early warning of electromechanical equipment based on multimodal dynamic perception, the method is based on the impact of temperature, vibration, and noise on the health status of electromechanical equipment, and the method includes: Based on the vibration, temperature, and acoustic dimensions, triaxial piezoelectric accelerometers, infrared array sensors, and wideband microphone arrays are deployed in the building's computer room to obtain multimodal data from electromechanical equipment. Extract physical features and deep features from multimodal data, build a physical feature engineering and deep feature learning network architecture based on the physical features and deep features, and dynamically weight the two types of features; Build a device health quantification model based on fusion features and implement a graded early warning strategy; Based on digital twin technology, the health status of equipment and operation and maintenance decisions are visualized to achieve closed-loop management of virtual-reality interaction.

[0009] Furthermore, based on the vibration, temperature, and acoustic dimensions, a triaxial piezoelectric accelerometer, an infrared array sensor, and a wideband microphone array are deployed in the building's computer room, including: Obtain the spatial layout of electromechanical equipment in the building's computer room, the location of key components, and the dynamic characteristics of the electromechanical equipment during operation; Determine the physical quantities that need to be monitored based on the dynamic characteristics of the electromechanical equipment during operation, and determine the type of sensors that need to be monitored based on the location of key components; Determine the number and layout of sensors based on the spatial layout of electromechanical equipment, the physical quantities to be monitored, and the types of sensors to be monitored; Determine the sensor layout in the electromechanical equipment based on the number and layout of sensors, and obtain the surrounding environment characteristics of the electromechanical equipment during operation; Based on the sensor layout and surrounding environment characteristics of the electromechanical device, a simulation operation model of the electromechanical device and the sensor is constructed, and a first mutual interference between the vibration dimension and the temperature dimension, a second mutual interference between the vibration dimension and the acoustic dimension, and a third mutual interference between the temperature dimension and the acoustic dimension are obtained from the simulation operation model; According to the importance of the first mutual interference, the second mutual interference and the third mutual interference to the monitored physical quantity, the number and layout of sensors are adjusted to obtain the initial sensor deployment result; Obtaining a location range of each sensor in the initial sensor deployment result, and determining an optimal location within the location range based on installation and maintenance difficulty of the sensor within the location range; Based on the optimal position, the deployment results of the triaxial piezoelectric acceleration sensor, the infrared array sensor and the wideband microphone array are determined.

[0010] Furthermore, after obtaining the multimodal data of the electromechanical equipment, it includes: Time alignment: Using a sliding window mechanism to reduce sampling frequency differences, cubic spline interpolation is performed on the temperature and noise signals using the high-frequency timestamps of the vibration signal as a benchmark to generate synchronized data points. The calculation formula is as follows:

[0011] in, Expressed as time Temperature signal value after time alignment; Expressed as the number of spline basis functions; Expressed as the original temperature sampling time Temperature signal value; Expressed as cubic spline basis functions, Represents the original sampling time of temperature; Spatial mapping: Based on BIM coordinate binding technology, the spatial coordinates of each sensor are marked in the Revit model, and the temperature infrared array coverage area is mapped into a 3D grid to eliminate the position perception deviation of each sensor; Environmental compensation: Wavelet thresholds are used to reduce the noise of vibration signals, and an adaptive noise cancellation filter is used to suppress steady-state environmental noise. The thermal conductivity coefficient of electromechanical equipment materials is introduced to eliminate measurement deviations caused by sudden changes in ambient temperature and suppress interference coupling.

[0012] Furthermore, physical features and depth features are extracted from multimodal data, specifically: Vibration signal: Perform 16-layer wavelet packet decomposition on the vibration signal, divide the 0-10kHz frequency band into multiple sub-bands, and calculate the energy entropy of each sub-band and the effective value, crest factor, and kurtosis of the vibration signal; Temperature data: Using the trapezoidal numerical integration method, the temperature rise data after environmental compensation is discretized to obtain the degree of continuous heating of the electromechanical equipment. The horizontal and vertical gradients of the temperature distribution matrix in the infrared array sensor are calculated, and the maximum gradient value is taken as the local overheating indicator. Noise signal: The acoustic signal is pre-emphasized, framed, and windowed. After Fourier transform, it is passed through a Mel filter bank to extract the logarithmic energy. After DCT transform, the first 13 coefficients are retained. The number of times the signal crosses zero in each frame is calculated to obtain the time-domain fluctuation characteristics of the noise signal.

[0013] Furthermore, a physical feature engineering and deep feature learning network architecture is constructed based on physical features and deep features, including: Designing the network architecture: Input layer: Receives multimodal data after spatiotemporal alignment, including: Vibration signal: three-axis acceleration time series data; Temperature signal: temperature rise sequence; Noise signal: sound pressure waveform; 1D-CNN module: Convolutional layer 1: 64 filters, kernel size 64, stride 4, ReLU activation; Convolutional layer 2: 32 filters, kernel size 32, stride 4, ReLU activation; Convolutional layer 3: 16 filters, kernel size 16, stride 4, ReLU activation; Global maximum pooling layer: outputs 128-dimensional feature vector; BiLSTM module: Bidirectional LSTM layer: 128 hidden units, the sequence input is the 128-dimensional features output by CNN; Attention mechanism layer: Calculates the weight of each time step, takes weighted sum, and outputs 128-dimensional time series features.

[0014] Furthermore, the physical feature engineering and deep feature learning network architecture is constructed based on physical features and deep features, including: Data augmentation: Gaussian noise, random time shift, and frequency band attenuation are applied to normal samples; Transfer learning: Pre-train the CNN part based on the CWRU bearing dataset and freeze the parameters of the first two convolution layers; Loss function: Weighted cross entropy loss is used, and fault samples are given a 3x weight to balance the category distribution.

[0015] Furthermore, the weight vector is calculated based on the multi-layer perceptron, including: Divide the 139-dimensional input vector into three single-mode vectors according to the vibration dimension, temperature dimension, and acoustic dimension; Based on the current health assessment task, determine the key features, obtain the correlation between the key features and each dimension feature, and determine the internal correlation value between the mutual features in each single modal vector; Based on the correlation between the key feature and each dimensional feature, as well as the internal correlation value, the internal weight of each dimensional feature in each unimodal vector is calculated; Based on the external correlation value between cross-modal features between single-modal vectors and combined with the internal weight, the weight of each dimension feature is obtained; Based on the weights of the dimensional features, a weight vector corresponding to the 139-dimensional input vector is obtained.

[0016] Furthermore, the two types of features are dynamically weighted and fused, including: Normalize and standardize the output physical feature vector and depth feature vector; The normalized physical features, depth features, and real-time equipment operating parameters are concatenated into a 139-dimensional input vector, and the weight vector is calculated based on a multi-layer perceptron. Set load rate growth thresholds and temperature thresholds. If the load rate exceeds the threshold, the vibration weight automatically increases to strengthen mechanical stress monitoring; if the ambient temperature exceeds the threshold, the temperature weight decreases to suppress ambient thermal interference.

[0017] Furthermore, the dynamic weight fusion of the two types of features also includes: Input normalized physical features, depth features and dynamic weights, and perform weighted processing on the physical features and depth features respectively; The feature vectors are fused to generate a 136-dimensional comprehensive feature vector, which is used as the input of the health assessment model.

[0018] Furthermore, the output physical feature vector and depth feature vector are normalized and standardized, specifically: The output physical feature vector and depth feature vector are used as input sets and Min-Max normalization is used to map each feature in the physical feature to the interval [0, 1]. The Min-Max normalization calculation formula is as follows:

[0019] in, Expressed as Physical characteristics through - The value after normalization; Expressed as The raw value of a physical characteristic; Represents all the physical characteristics The minimum value of a feature; Represented as all the physical features The maximum value of features; Z-score normalization is used to eliminate the dimensional differences in depth features. The Z-score normalization calculation formula is as follows:

[0020] in, Expressed as The value of a deep feature after Z-score normalization; Expressed as The original value of the deep feature; Expressed as The mean of the dimensional depth feature; Expressed as The standard deviation of the dimensional depth feature.

[0021] Furthermore, a quantitative model of equipment health is constructed based on the fusion features, and a hierarchical early warning strategy is implemented, specifically: Input multimodal data of electromechanical equipment under normal operating conditions, use the K-means++ algorithm to cluster the equipment load rate, ambient temperature, and operating time, and divide it into four typical operating condition categories; For each operating condition category, the mean vector and covariance matrix of the fused features are calculated and stored as a benchmark template library for real-time health comparison. Set alert levels based on health, including normal, caution, warning, and critical, and set response actions for each level; Input the real-time fusion features of each working condition category and the benchmark template of the corresponding working condition category, and use the Mahalanobis distance to eliminate the dimensional differences and correlation effects between features; The health of each working condition sample is judged based on the health index threshold, and different levels of early warning response actions are triggered based on the health assessment results.

[0022] Furthermore, digital twin technology is used to visualize the health status of equipment and operation and maintenance decisions. Specifically: Input health assessment results and real-time operation data of electromechanical equipment, and write the equipment health data into the BIM model according to the IFC4 standard extended attribute set; Abnormal equipment is displayed as a pulsating red light in the BIM model, with real-time data curves superimposed; Input device QR code labels, real-time fusion of feature data and maintenance knowledge base, AR-based scanning of electromechanical equipment QR codes, calling ARKit / ARCore engine to identify devices; also used to display vibration thermal maps, temperature gradient vectors and maintenance instructions; Input the fault characteristics, treatment measures and verification results of each maintenance work order for later learning.

[0023] Compared with the prior art, the present invention has the following beneficial effects: 1. In this invention, a panoramic perception system covering multiple physical fields such as vibration, temperature, and noise is constructed through a multimodal sensor network and a dynamic weight fusion algorithm, solving the problem of information islands in traditional single-dimensional monitoring.

[0024] 2. In this invention, interpretable engineering features are combined with abstract features extracted by deep neural networks through a physical-deep hybrid feature extraction architecture to form a health assessment model with both mechanism transparency and pattern generalization capabilities.

[0025] 3. In this invention, through the deep integration of the digital twin platform and RPA technology, closed-loop operation and maintenance with virtual and real linkage is realized; the frequency and workload of manual inspections are reduced, and the accuracy of data collection is improved, thereby promoting the accuracy of fault identification to increase year by year, forming a continuously optimized intelligent operation and maintenance ecosystem. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of the electromechanical equipment health assessment and early warning method based on multimodal dynamic perception of the present invention. DETAILED DESCRIPTION

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0028] In order to solve the technical problems in existing technologies, where a single physical quantity cannot fully reflect the device status and traditional signal processing algorithms cannot adapt to changes in device degradation patterns, please refer to Figure 1 , this embodiment provides the following technical solutions: A method for health assessment and early warning of electromechanical equipment based on multimodal dynamic perception, the method is based on the impact of temperature, vibration, and noise on the health status of electromechanical equipment, and the method includes: Step 1: Deploy triaxial piezoelectric accelerometers, infrared array sensors, and broadband microphone arrays in the building's computer room to obtain multimodal data from electromechanical equipment based on vibration, temperature, and acoustic dimensions. In one embodiment, a triaxial piezoelectric acceleration sensor, an infrared array sensor, and a wideband microphone array are deployed in a building machine room based on vibration, temperature, and acoustic dimensions, including: Obtain the spatial layout of electromechanical equipment in the building's computer room, the location of key components, and the dynamic characteristics of the electromechanical equipment during operation; Determine the physical quantities that need to be monitored based on the dynamic characteristics of the electromechanical equipment during operation, and determine the type of sensors that need to be monitored based on the location of key components; Determine the number and layout of sensors based on the spatial layout of electromechanical equipment, the physical quantities to be monitored, and the types of sensors to be monitored; Determine the sensor layout in the electromechanical equipment based on the number and layout of sensors, and obtain the surrounding environment characteristics of the electromechanical equipment during operation; Based on the sensor layout and surrounding environment characteristics of the electromechanical device, a simulation operation model of the electromechanical device and the sensor is constructed, and a first mutual interference between the vibration dimension and the temperature dimension, a second mutual interference between the vibration dimension and the acoustic dimension, and a third mutual interference between the temperature dimension and the acoustic dimension are obtained from the simulation operation model; According to the importance of the first mutual interference, the second mutual interference and the third mutual interference to the monitored physical quantity, the number and layout of sensors are adjusted to obtain the initial sensor deployment result; Obtaining a location range of each sensor in the initial sensor deployment result, and determining an optimal location within the location range based on installation and maintenance difficulty of the sensor within the location range; Based on the optimal position, the deployment results of the triaxial piezoelectric acceleration sensor, the infrared array sensor and the wideband microphone array are determined.

[0029] In this embodiment, the dynamic characteristics of the electromechanical equipment during operation include vibration changes, temperature changes, noise changes, etc.

[0030] In this embodiment, the sensors are a three-axis piezoelectric acceleration sensor, an infrared array sensor, and a broadband microphone array.

[0031] In this embodiment, the layout of the sensors is a preliminary layout, which determines a layout range, and the specific positions still need to be further confirmed.

[0032] In this embodiment, the surrounding environment characteristics include interferences such as other noises, temperature, and sound, which should be avoided when deploying the sensor.

[0033] In this embodiment, a three-axis piezoelectric accelerometer is used to monitor the vibration characteristics of bearings, gears, etc. in electromechanical equipment, an infrared array sensor is used to monitor the temperature of motors and areas with poor heat dissipation in electromechanical equipment, and a wideband microphone array is used to monitor and capture changes in various noises during the operation of electromechanical equipment.

[0034] In this embodiment, the first mutual interference is, for example, abnormal operation of the three-axis piezoelectric acceleration sensor caused by excessive temperature, and interference with detection of the corresponding infrared array sensor caused by vibration characteristics of bearings, gears, etc.

[0035] In this embodiment, the second mutual interference is, for example, interference to the broadband microphone array caused by excessive temperature.

[0036] In this embodiment, the third mutual interference is, for example, the influence of large vibration on the microphone capturing the sound signal.

[0037] In this embodiment, the more important the monitored physical quantity is, the position of the corresponding sensor is retained, and the positions of other sensors that affect the sensor are adjusted.

[0038] In this embodiment, the initial sensor deployment result realizes the cooperative detection of the triaxial piezoelectric acceleration sensor, the infrared array sensor, and the wideband microphone array without affecting each other.

[0039] In this embodiment, the position with the lowest safety and maintenance difficulty is selected from the position range as the optimal position.

[0040] The beneficial effect of the above design scheme is: by deploying three-axis piezoelectric accelerometers, infrared array sensors and wide-band microphone arrays in the building machine room based on the vibration dimension, temperature dimension and acoustic dimension, in addition to considering the spatial layout and dynamic characteristics of the electromechanical equipment during operation during the deployment process, the mutual influence of the layout of different sensors, as well as the difficulty of sensor installation and maintenance are also considered to select the optimal deployment location, and finally determine the deployment results of the three-axis piezoelectric accelerometers, infrared array sensors and wide-band microphone arrays, providing a basis for the comprehensiveness and reliability of monitoring data collection, thereby ensuring the accuracy of the health assessment of electromechanical equipment.

[0041] In one embodiment, multi-parameter intelligent sensing terminals are installed on the surface and surrounding space of key electromechanical equipment in the building machine room, such as chillers, water pumps, and transformers. For example, the vibration sensor is installed at the connection between the bearing seat and the base with the three axes being orthogonal, and the acoustic array is configured at a 45° angle; vibration monitoring uses a three-axis piezoelectric acceleration sensor with a range covering the 0-10kHz frequency band, which is installed at the connection between the equipment bearing seat and the base; temperature monitoring uses an infrared array sensor to collect the surface temperature field distribution of the equipment with an accuracy of ±0.5°C; acoustic monitoring deploys a wideband microphone array covering the 50Hz-12kHz frequency band, and the installation angle is 45° to the axis of the equipment. Multi-sensor clock synchronization is achieved through LoRa wireless networking, and the sampling frequency is dynamically adjusted according to the operating status of the equipment: in normal mode, a data packet is collected every 2 seconds, including 12-dimensional parameters such as vibration spectrum, temperature gradient, and sound pressure level, as shown in the table below;

[0042]

[0043] Table 1: 12-dimensional parameter table This parameter is designed based on: 1. Vibration dimension selection (1-4 dimensions): VRMS and VPeak: Covering steady-state vibration intensity and transient impact detection of equipment, aiming to address the problem of traditional threshold methods being insensitive to transient anomalies; Low-frequency and characteristic frequency band energy: Monitor fundamental mechanical resonance (0–1kHz) and bearing wear characteristic frequency (2–4kHz) respectively, aiming to overcome the limitations of single-band analysis 2. Temperature dimension design (5-7 dimensions): T1 / T2 dual measurement points: distinguish between bearing friction heat and motor winding heat, aiming to solve the coupling interference problem of traditional single-point temperature measurement; Dynamic temperature gradient: Captures abnormal heating rates of equipment to compensate for the lag of fixed thresholds in responding to slowly changing faults. 3. Acoustic dimension construction (8-12 dimensions): SPL-A total sound level: quantifies the ambient noise background, aiming to solve the problem of traditional acoustic monitoring being disturbed by the reverberation of the equipment room) Frequency band energy + MFCC: Simultaneously captures low-frequency mechanical faults (50–500 Hz) and high-frequency abnormal noises (4–8 kHz), and combines the time domain zero-crossing rate to enhance abnormal voiceprint recognition capabilities.

[0044] When switching to enhanced mode, the sampling frequency is increased to 50Hz and lasts for 30 minutes; Use a hierarchical conditional trigger mechanism to switch to enhanced mode and evaluate device status in stages, balancing real-time performance and accuracy: Stage 1: Primary Trigger (Quick Decision)

[0045] When the detected vibration effective value exceeds 4.2mm / s or the temperature change rate is greater than 0.8℃ / min, dynamic threshold adjustment is introduced at the same time:

[0046]

[0047] in, Expressed as the dynamic threshold of vibration; is the real-time load rate of the equipment (%). When the load exceeds 80%, the vibration threshold increases linearly with the load, avoiding false alarms under normal working conditions with high loads. A dynamic threshold expressed as the rate of temperature change; is the ambient temperature (°C). When the ambient temperature deviates from the 25°C benchmark, the temperature change rate threshold is adaptively adjusted to compensate for environmental interference.

[0048] Phase 2: Secondary Verification (Multi-Indicator Fusion) When the primary trigger is triggered, the following verification process is started (taking < 50ms): (1) Acoustic feature verification: Abnormality was confirmed if the high-frequency noise energy (4-8 kHz) exceeded the baseline value by 30% and the MFCC0 coefficient of variation was >15%.

[0049] (2) Working condition correlation analysis: Calculate the frequency domain coherence of the vibration signal:

[0050] in, Expressed as a coherence function; Vibration signal and the equipment speed reference signal In frequency If the cross power spectrum density at the characteristic frequency band, such as 2-4kHz, has a coherence < 0.6, it is determined to be a non-load-related anomaly. Represented as a vibration signal The autopower spectral density at the frequency The distribution of the signal's own power is shown in . Indicates the equipment speed reference signal The autopower spectral density at the frequency It represents the distribution of the signal's own power.

[0051] (3) Historical trend comparison: Retrieve historical data of the same operating conditions for the equipment. If the current HI value drops by more than 10% compared to the 7-day average, the enhanced mode is triggered.

[0052] Phase 3: Enhanced Mode Activation After passing the above verification, the system executes: Sampling frequency increased: vibration signal increased from 1kHz to 5kHz, and temperature sampling interval shortened from 2 seconds to 0.5 seconds.

[0053] Enhanced feature calculation: Enables full-band wavelet packet decomposition (32 subbands) and acoustic MFCC full coefficient analysis (13 dimensions).

[0054] Edge-cloud collaboration: Cache raw data to the local NPU for real-time analysis, while uploading compressed features to the cloud for deep model inference.

[0055] Step 2: After obtaining the multimodal data of the electromechanical equipment, a unified spatiotemporal benchmark is established based on the heterogeneous characteristics of temperature, vibration, and noise signals. Specifically:

[0056] Table 2: Heterogeneity and processing logic Time alignment: Using a sliding window mechanism to reduce sampling frequency differences, we use the high-frequency timestamps of the vibration signal as a benchmark, such as t = 0.000, 0.001, and 0.002 seconds. We perform cubic spline interpolation on the temperature signal with a sampling interval of 2 seconds and the noise signal with a sampling rate of 16 kHz to generate synchronized data points. The calculation formula is as follows:

[0057] in, Expressed as time Temperature signal value after time alignment; Expressed as the number of spline basis functions; Expressed as the original temperature sampling time Temperature signal value; Expressed as cubic spline basis functions, Represents the original sampling time of temperature; Spatial mapping: Using BIM coordinate binding technology, spatial coordinates are annotated for each sensor in the Revit model. For example, the vibration sensor for the water pump's front bearing is located at X=3.2m, Y=5.7m, and Z=1.5m. The temperature infrared array's coverage area is mapped into a 3D grid with a grid size of 10cm x 10cm to eliminate positional bias among the sensors. Data association rules include: vibration data is associated only with the equipment component at the bound coordinate point, such as the bearing seat. Temperature data is associated with the component with the highest temperature within the grid, such as the motor winding. Acoustic data is located using a beamforming algorithm to locate the sound source and match it to the coordinates of the nearest equipment component.

[0058] Environmental compensation: Use wavelet threshold to reduce the noise of vibration signal. The calculation formula is as follows:

[0059] in, Represented as the vibration signal after filtering; Expressed as inverse discrete wavelet transform; Expressed as a threshold processing function; Represented as discrete wavelet transform; Represented as the original vibration signal; threshold ; Expressed as the noise standard deviation, it is used to effectively filter out the 10-50 Hz low-frequency vibration interference caused by air conditioning airflow.

[0060] The adaptive noise cancellation filter is used to suppress steady-state environmental noise. The calculation formula is as follows:

[0061] in, Represented as the filtered signal; Represented as the original sound signal; Expressed as weight; Expressed as the order or length of the filter; Represented as a background noise reference signal, i.e., from a microphone far away from the device, and the weights are updated using the LMS algorithm , suppress steady-state environmental noise, such as fan hum.

[0062] Introducing the thermal conductivity coefficient of electromechanical equipment materials, specifically: Introducing ambient temperature sensor data , calculate the actual temperature rise of the equipment, such as: stainless steel equipment k = 0.12, eliminate the measurement deviation caused by sudden changes in ambient temperature, suppress interference coupling, the calculation formula is as follows:

[0063] in, Expressed as the temperature difference between the equipment surface temperature and the ambient temperature; Expressed as the surface temperature of the equipment; Expressed as the thermal conductivity coefficient; Represented as ambient temperature sensor data; Expressed as a gradient change in ambient temperature.

[0064] Extract physical features and deep features from multimodal data, build a physical feature engineering and deep feature learning network architecture based on the physical features and deep features, and dynamically weight the two types of features. Specifically, extract physical features and deep features from multimodal data as follows: Vibration signal: Decompose the vibration signal into 16 layers of wavelet packets. Use db4 wavelet basis to divide the 0-10kHz frequency band into sub-bands, that is, in actual engineering, they are combined into 16 key frequency bands according to equipment characteristics; and the energy entropy of each sub-band is calculated using the following formula:

[0065] in, Represented as the vibration signal in time Energy entropy; Expressed as the total number of sampling points; It is expressed as the energy proportion of the k-th sampling point in the sub-band; it is used to extract the 2–4 kHz frequency band, that is, the energy entropy proportion corresponding to the bearing fault characteristic frequency as the key indicator; Expressed as The total energy of the sub-bands; Expressed as a logarithmic function.

[0066] Then calculate the effective value, crest factor and kurtosis of the vibration signal.

[0067] Temperature data: The temperature rise data after environmental compensation is discretized and calculated using the trapezoidal numerical integration method to obtain the degree of sustained heating of the electromechanical equipment. The time window length is set according to the thermal inertia of the equipment, such as 5 minutes for motor windings and 2 minutes for bearings. The transverse and longitudinal gradients of the temperature distribution matrix in the infrared array sensor are calculated, and the maximum gradient value is taken as the local overheating indicator. The calculation formula is as follows:

[0068] in, Expressed as the maximum gradient value; It is expressed as the transverse gradient; It is expressed as the longitudinal gradient; Expressed as the temperature corresponding to the maximum gradient value.

[0069] Noise signal: The acoustic signal is pre-emphasized and framed and windowed, for example, with a 25ms frame length, a 10ms frame shift, and a Hamming window. After Fourier transform, the signal is passed through a Mel filter bank and the logarithmic energy is taken. After DCT transform, the first 13 coefficients are retained and the number of zero crossings per frame is calculated to obtain the time-domain fluctuation characteristics of the noise signal.

[0070] Step 3: Build a physical feature engineering and deep feature learning network architecture based on physical features and deep features, specifically: Designing the network architecture: Input layer: Receives multimodal data after spatiotemporal alignment, including: Vibration signal: three-axis acceleration time series data, such as: length 1024 points, sampling rate 1kHz; Temperature signal: temperature rise sequence, e.g., length 512 points, sampling rate 10 Hz; Noise signal: sound pressure waveform, e.g., length 16000 points, sampling rate 16kHz; 1D-CNN module: Convolutional layer 1: 64 filters, kernel size 64, stride 4, ReLU activation; Convolutional layer 2: 32 filters, kernel size 32, stride 4, ReLU activation; Convolutional layer 3: 16 filters, kernel size 16, stride 4, ReLU activation; Global maximum pooling layer: outputs 128-dimensional feature vector; BiLSTM module: Bidirectional LSTM layer: 128 hidden units, the sequence input is the 128-dimensional features output by CNN; Attention mechanism layer: Calculates the weight of each time step, takes weighted sum, and outputs 128-dimensional time series features.

[0071] Data augmentation: Gaussian noise (SNR = 20dB), random time shift (±5%), and band attenuation (randomly filtering out 1-3 Mel bands) are applied to normal samples. Transfer learning: Pre-train the CNN part based on the CWRU bearing dataset and freeze the parameters of the first two convolution layers; Loss function: Weighted cross entropy loss is used, and fault samples are given a 3x weight to balance the category distribution.

[0072] In one embodiment, calculating the weight vector based on a multi-layer perceptron includes: Divide the 139-dimensional input vector into three single-mode vectors according to the vibration dimension, temperature dimension, and acoustic dimension; Based on the current health assessment task, determine the key features, obtain the correlation between the key features and each dimension feature, and determine the internal correlation value between the mutual features in each single modal vector; Based on the correlation between the key feature and each dimensional feature, as well as the internal correlation value, the internal weight of each dimensional feature in each unimodal vector is calculated;

[0073] in, Represents the internal weight of the current dimension feature, Indicates the number of key features. Indicates the correlation between the i-th key focus feature and the current dimension feature, Indicates the number of dimensions of the unimodal vector where the current dimension feature is located, Indicates the relationship between the current dimension feature and the first dimension feature based on the most relevant feature of the current dimension feature. Internal correlation values ​​of other dimension features; Based on the external correlation value between cross-modal features between single-modal vectors and combined with the internal weight, the weight of each dimension feature is obtained;

[0074] in, Indicates the weight of the current dimension feature, e represents a natural number, and its value is 2.72. Represents the total external correlation value between the current dimension feature and the feature in the first cross-unimodal vector, Represents the total external correlation value between the current dimension feature and the feature in the second cross-unimodal vector; Based on the weights of the dimensional features, a weight vector corresponding to the 139-dimensional input vector is obtained.

[0075] In this embodiment, the weight vector can be dynamically adjusted based on different health assessment tasks.

[0076] In this embodiment, for example, if the feature to be focused on is a vibration high-frequency signal, the three-dimensional weight of the high-frequency signal is strengthened, and the weight of the low-frequency signal is weakened.

[0077] In this embodiment, the internal correlation value indicates that the correlation between different types of high-frequency signals is strong.

[0078] In this embodiment, for example, in a health assessment task, a combination of vibration and temperature is required, and the total external correlation value of the high-frequency signal and the temperature mode is relatively large.

[0079] The beneficial effects of the above design scheme are: by dividing the 139-dimensional input vector into three single-modal vectors according to the vibration dimension, temperature dimension and acoustic dimension; based on the current health assessment task, determining the key focus features, and obtaining the correlation between the key focus features and each dimensional feature, and determining the internal correlation value between the mutual features in each single-modal vector; based on the correlation between the key focus features and each dimensional feature, and the internal correlation value, calculating the internal weight of each dimensional feature in each single-modal vector; based on the external correlation value between the cross-modal features between the single-modal vectors, combined with the internal weight, obtaining the weight of each dimensional feature; based on the weight of the dimensional feature, obtaining the weight vector corresponding to the 139-dimensional input vector, by considering the internal correlation between the single-modal vectors and the external correlation between the cross-modal based on the health assessment task, to comprehensively determine the weight of each dimensional feature, to obtain the weight vector, to realize automatic adjustment of weights according to different tasks, to improve the utilization rate of key features, and to provide a basis for the establishment of a quantitative model of equipment health.

[0080] In one embodiment, assume that steps 2 and 3 are based on a transformer winding overheating warning as an example: Step 2 output: Temperature signal: Winding hotspot temperature Thotspot = 98°C (after environmental compensation), temperature rise rate ΔT / Δt = 0.6°C / min; Vibration signal: 100Hz frequency band energy increased by 12% (after filtering out air conditioning interference); Noise signal: The MFCC0 value decreases by 8dB compared to the baseline.

[0081] Step 3: Physical characteristics: TAI = 420℃·min (threshold 350℃·min), 100Hz vibration energy entropy ratio 0.18 (normal range 0.05-0.12); Deep features: BiLSTM output shows unusual temporal patterns (attention weights are concentrated on the first 1 second of the impulse signal); Fusion decision-making: Physical features trigger primary warnings, deep features confirm abnormal patterns, and ultimately generate winding insulation aging warnings (72 hours earlier than traditional temperature rise threshold methods).

[0082] Step 4: Dynamically weight the two types of features, including: The output physical feature vector and depth feature vector are normalized and standardized as follows: The output physical feature vector and depth feature vector are used as input sets and Min-Max normalization is used to map each feature in the physical feature to the interval [0, 1]. The Min-Max normalization calculation formula is as follows:

[0083] in, Expressed as Physical characteristics through - The value after normalization; Expressed as The raw value of a physical characteristic; Represents all the physical characteristics The minimum value of a feature; Represented as all the physical features The maximum value of features; Z-score normalization is used to eliminate the dimensional differences in depth features. The Z-score normalization calculation formula is as follows:

[0084] in, Expressed as The value of a deep feature after Z-score normalization; Expressed as The original value of the deep feature; Expressed as The mean of the dimensional depth feature; Expressed as The standard deviation of the dimensional depth feature.

[0085] Specifically, by normalizing and standardizing the physical feature vectors and the depth feature vectors, the inconsistent dimensions of physical features, such as temperature in °C, and depth features, such as unitless, are solved, ensuring the fairness of fusion weight calculation.

[0086] The normalized physical features (8 dimensions), depth features (128 dimensions), and real-time equipment operating parameters (3 dimensions) are concatenated into a 139-dimensional input vector X. The weight vector is calculated based on a multi-layer perceptron. The calculation process is as follows: First fully connected layer: 64 nodes, ReLU activation;

[0087] in, Represented as the output of the first layer, that is, the activation value of hidden layer 1; Represented as the weight matrix of the first layer, used to transform the input Mapped to hidden layer 1; Represented as the bias vector of the first layer; Represented as input feature vector; It is represented as activation function.

[0088] Second fully connected layer: 32 nodes, ReLU activation;

[0089] in, Represented as the output of the second layer, that is, the activation value of hidden layer 2; Represented as the weight matrix of the second layer, used to Mapped to hidden layer 2; Represented as the bias vector of the second layer; It is represented as activation function.

[0090] Output layer: 3 nodes, corresponding to temperature, vibration, and noise modal weights, and Softmax activation;

[0091] in, Represented as an output weight vector, including temperature, vibration, and noise modal weights; Represented as the weight matrix of the output layer, used to Mapping to the output layer; Represented as the bias vector of the output layer; It is represented as activation function.

[0092] Output weight vector ,satisfy .

[0093] in, Expressed as vibration weight; Expressed as temperature weight; Expressed as noise weight; Set the load rate growth threshold and temperature threshold. If the load rate exceeds the threshold, the vibration weight will automatically increase to strengthen mechanical stress monitoring; if the ambient temperature exceeds the threshold, the temperature weight will be reduced to suppress environmental thermal interference. Dynamic adjustment example: When the load rate >85% vibration weight Automatically increase, for example, from 0.4 to 0.6, to strengthen mechanical stress monitoring; When the ambient temperature >35℃, temperature weight Reduce, for example, from 0.5 to 0.3, to suppress environmental thermal interference; thereby breaking through the limitations of traditional fixed-weight fusion and achieving feature focusing that is adaptive to working conditions.

[0094] Input normalized physical features, depth features and dynamic weights, and perform weighted processing on physical features and depth features respectively; Physical feature weighting:

[0095] in, Represented as the fused physical feature vector; 、 、 Physical feature subsets corresponding to temperature, vibration, and noise respectively; Deep feature weighting:

[0096] in, Represented as the fused deep feature vector; 、 、 Deep feature subsets corresponding to temperature, vibration, and noise respectively; Deep features are divided by modality, such as vibration-related features occupy 64 dimensions, temperature occupies 32 dimensions, and noise occupies 32 dimensions; the final fusion vector is:

[0097] in, expressed as physical characteristics; Represented as deep features.

[0098] Based on this, a 136-dimensional comprehensive feature vector (8+128) is generated as the input to the health assessment model. Weight distribution is used to suppress redundant or conflicting features, such as abnormal temperature rise caused by ambient temperature fluctuations, and to enhance the contribution of key features.

[0099] Step 5: Build a quantitative model of equipment health based on the fusion features and implement a graded early warning strategy; specifically: Input 200 hours of multimodal data of electromechanical equipment under normal operation, covering load rate 30% to 100% working conditions, and use K-means++ algorithm to analyze the load rate of equipment. , ambient temperature , running time Clustering is performed and divided into four typical working condition categories, such as: low load, medium load, high load, and peak load.

[0100] For each operating condition category, the mean vector and covariance matrix of the fused features are calculated and stored as a benchmark template library for real-time health comparison. Set alert levels based on health, including normal, caution, warning, and critical, and set response actions for each level; Input the real-time fusion features of each working condition category and the benchmark template of the corresponding working condition category, and use the Mahalanobis distance to eliminate the dimensional differences and correlation effects between features; The health of each working condition sample is judged based on the health index threshold, and different levels of early warning response actions are triggered based on the health assessment results.

[0101]

[0102] Expressed as a health index; Expressed as Mahalanobis distance value; Expressed as health index threshold; threshold Take the 95% quantile of the normal sample distance distribution of this working condition, such as: =2.8; when > When forced =0, indicating severe abnormality; Example: A water pump is under medium load condition. =1.2, =2.0, then =1−1.2 / 2.0=0.4, triggering a red alert.

[0103]

[0104] Table 3: Trigger conditions and response strategies This breaks through the limitations of traditional binary alarms and achieves progressive decision support.

[0105] Step 6: Visualize equipment health status and operation and maintenance decisions based on digital twin technology to achieve closed-loop management of virtual-reality interaction; specifically: Input health assessment results and real-time operation data of electromechanical equipment, and write the equipment health data into the BIM model according to the IFC4 standard extended property set; develop plug-ins through the Revit API to dynamically update the equipment status properties in the model; Abnormal equipment is displayed as a pulsed red light in the BIM model, with real-time data curves superimposed; for example: Trends and vibration spectrum graphs; click on a device to view historical maintenance records, spare parts inventory, and similar fault cases. This breaks through the limitations of traditional two-dimensional panel monitoring, achieves spatial expression of device status, and improves fault location accuracy to ±0.2 meters. By inputting the device's QR code label, integrating real-time feature data and a maintenance knowledge base, maintenance personnel can scan the QR code of the electromechanical device using AR, invoke the ARKit / ARCore engine to identify the device, and overlay the display on the camera image: Vibration heatmap: Renders the device surface color based on the three-axis vibration intensity (blue → green → red); Temperature gradient vector: The length and direction of the arrow indicate the trend of temperature field change; Maintenance Instructions: Animated step-by-step demonstrations of key operations (e.g., “Sequence for removing bearing cover bolts: 1→3→5→2→4”).

[0106] Remote expert collaboration: Supports real-time image sharing and annotation. Experts can mark fault points in the AR interface and push them to on-site personnel.

[0107] Input the fault characteristics, treatment measures and verification results of each maintenance work order for later learning; extract key features from the maintenance report, such as: the energy ratio of 2-4kHz vibration and the temperature rise rate at the time of the fault, and build a standardized fault feature vector. Match the new case with the historical database for similarity. If there is a difference, trigger the model fine-tuning; when the same fault occurs repeatedly and the original maintenance plan fails, automatically generate rule optimization suggestions; for example: the water pump cavitation warning threshold needs to be changed from / >0.8℃ / min was adjusted to >0.6℃ / min.

[0108] The beneficial effects achieved by the aforementioned technology include significant breakthroughs in perception capabilities, decision-making efficiency, and resource optimization compared to traditional building mechanical and electrical equipment operation and maintenance technologies. Its core advantages are reflected in three technological advancements: First, through a multimodal sensor network and a dynamic weight fusion algorithm, a comprehensive perception system covering multiple physical fields, including vibration, temperature, and noise, is constructed, addressing the information silos inherent in traditional single-dimensional monitoring. Experimental data show that in centrifugal chiller applications, the early detection rate of bearing failures increased from 68% with traditional methods to 92%, the false alarm rate decreased from 28% to 5.8%, and the fault warning window was shortened to over 72 hours. Second, a hybrid physical-deep feature extraction architecture is employed to combine interpretable engineering features, such as energy entropy and heat accumulation index in vibration frequency bands, with abstract features extracted by deep neural networks, to form a health assessment model that combines mechanistic transparency and model generalization capabilities. Application in a large-scale venue's pump room demonstrates significant improvements in adaptability to complex operating conditions. Under load factor fluctuations of ±30%, the health assessment error is reduced by 42% compared to the traditional PCA method. Furthermore, through the deep integration of the digital twin platform and RPA technology, a closed-loop operation and maintenance system with virtual-reality integration has been achieved. BIM reverse modeling technology, combined with the AABB bounding box algorithm, reduced the point cloud model capacity by 75%, improving reverse modeling efficiency by 40%. RPA robots increased inspection frequency from two to eight times per day, improving data collection accuracy by 14.5%, and reducing the workload of operation and maintenance personnel by 83%. This technological integration not only reduced unplanned downtime to one-sixth of traditional models, but also, through the self-evolution of the knowledge base, increased fault identification accuracy by 3.2% annually, forming a continuously optimized intelligent operation and maintenance ecosystem.

[0109] Working Principle: Through a multimodal dynamic fusion architecture, a deep feature learning model, and an adaptive health assessment algorithm, it achieves full-dimensional perception of equipment status, accurate early fault warning, and optimized operation and maintenance decisions. Its technical solution is significantly more advanced than existing methods.

[0110] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "including," "having," or any other variations thereof are intended to cover non-exclusive possessors, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or includes elements that are inherent to such process, method, article, or apparatus.

[0111] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions, and alterations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for health assessment and early warning of electromechanical equipment based on multimodal dynamic perception, characterized in that: The method is based on the influence of temperature, vibration and noise on the health status of electromechanical equipment, and the method includes: Based on the vibration, temperature, and acoustic dimensions, triaxial piezoelectric accelerometers, infrared array sensors, and wideband microphone arrays are deployed in the building's computer room to obtain multimodal data from electromechanical equipment. Extract physical features and deep features from multimodal data, build a physical feature engineering and deep feature learning network architecture based on the physical features and deep features, and dynamically weight the two types of features; Build a device health quantification model based on fusion features and implement a graded early warning strategy; Based on digital twin technology, the health status of equipment and operation and maintenance decisions are visualized to achieve closed-loop management of virtual-reality interaction.

2. The method for health assessment and early warning of electromechanical equipment based on multimodal dynamic perception according to claim 1 is characterized in that: Based on the vibration, temperature, and acoustic dimensions, a triaxial piezoelectric accelerometer, infrared array sensor, and wideband microphone array are deployed in the building's computer room, including: Obtain the spatial layout of electromechanical equipment in the building's computer room, the location of key components, and the dynamic characteristics of the electromechanical equipment during operation; Determine the physical quantities that need to be monitored based on the dynamic characteristics of the electromechanical equipment during operation, and determine the type of sensors that need to be monitored based on the location of key components; Determine the number and layout of sensors based on the spatial layout of electromechanical equipment, the physical quantities to be monitored, and the types of sensors to be monitored; Determine the sensor layout in the electromechanical equipment based on the number and layout of sensors, and obtain the surrounding environment characteristics of the electromechanical equipment during operation; Based on the sensor layout and surrounding environment characteristics of the electromechanical device, a simulation operation model of the electromechanical device and the sensor is constructed, and a first mutual interference between the vibration dimension and the temperature dimension, a second mutual interference between the vibration dimension and the acoustic dimension, and a third mutual interference between the temperature dimension and the acoustic dimension are obtained from the simulation operation model; According to the importance of the first mutual interference, the second mutual interference and the third mutual interference to the monitored physical quantity, the number and layout of sensors are adjusted to obtain the initial sensor deployment result; Obtaining a location range of each sensor in the initial sensor deployment result, and determining an optimal location within the location range based on installation and maintenance difficulty of the sensor within the location range; Based on the optimal position, the deployment results of the triaxial piezoelectric acceleration sensor, the infrared array sensor and the wideband microphone array are determined.

3. The method for health assessment and early warning of electromechanical equipment based on multimodal dynamic perception according to claim 1 is characterized in that: After obtaining multimodal data of electromechanical equipment, including: Time alignment: Using a sliding window mechanism to reduce sampling frequency differences, cubic spline interpolation is performed on the temperature and noise signals using the high-frequency timestamps of the vibration signal as a benchmark to generate synchronized data points. The calculation formula is as follows: ， in, Expressed as time Temperature signal value after time alignment; Expressed as the number of spline basis functions; Expressed as cubic spline basis functions; Expressed as the original temperature sampling time Temperature signal value; Represents the original sampling time of temperature; Spatial mapping: Based on BIM coordinate binding technology, the spatial coordinates of each sensor are marked in the Revit model, and the temperature infrared array coverage area is mapped into a 3D grid to eliminate the position perception deviation of each sensor; Environmental compensation: Wavelet thresholds are used to reduce the noise of vibration signals, and an adaptive noise cancellation filter is used to suppress steady-state environmental noise. The thermal conductivity coefficient of electromechanical equipment materials is introduced to eliminate measurement deviations caused by sudden changes in ambient temperature and suppress interference coupling.

4. The method for health assessment and early warning of electromechanical equipment based on multimodal dynamic perception according to claim 1 is characterized in that: Extract physical features and depth features from multimodal data, specifically: Vibration signal: Perform 16-layer wavelet packet decomposition on the vibration signal, divide the 0-10kHz frequency band into multiple sub-bands, and calculate the energy entropy of each sub-band and the effective value, crest factor, and kurtosis of the vibration signal; Temperature data: Using the trapezoidal numerical integration method, the temperature rise data after environmental compensation is discretized to obtain the degree of continuous heating of the electromechanical equipment. The horizontal and vertical gradients of the temperature distribution matrix in the infrared array sensor are calculated, and the maximum gradient value is taken as the local overheating indicator. Noise signal: The acoustic signal is pre-emphasized, framed, and windowed. After Fourier transform, it is passed through a Mel filter bank to extract the logarithmic energy. After DCT transform, the first 13 coefficients are retained. The number of times the signal crosses zero in each frame is calculated to obtain the time-domain fluctuation characteristics of the noise signal.

5. The method for health assessment and early warning of electromechanical equipment based on multimodal dynamic perception according to claim 1 is characterized in that: Construct physical feature engineering and deep feature learning network architecture based on physical features and deep features, including: Designing the network architecture: Input layer: Receives multimodal data after spatiotemporal alignment, including: Vibration signal: three-axis acceleration time series data; Temperature signal: temperature rise sequence; Noise signal: sound pressure waveform; 1D-CNN module: Convolutional layer 1: 64 filters, kernel size 64, stride 4, ReLU activation; Convolutional layer 2: 32 filters, kernel size 32, stride 4, ReLU activation; Convolutional layer 3: 16 filters, kernel size 16, stride 4, ReLU activation; Global maximum pooling layer: outputs 128-dimensional feature vector; BiLSTM module: Bidirectional LSTM layer: 128 hidden units, the sequence input is the 128-dimensional features output by CNN; Attention mechanism layer: calculates the weight of each time step, sums the weighted sum, and outputs 128-dimensional time series features; Data augmentation: Gaussian noise, random time shift, and frequency band attenuation are applied to normal samples; Transfer learning: Pre-train the CNN part based on the CWRU bearing dataset and freeze the parameters of the first two convolution layers; Loss function: Weighted cross entropy loss is used, and fault samples are given a 3x weight to balance the category distribution.

6. The method for health assessment and early warning of electromechanical equipment based on multimodal dynamic perception according to claim 1 is characterized in that: Dynamic weight fusion of two types of features, including: Normalize and standardize the output physical feature vector and depth feature vector; The normalized physical features, depth features, and real-time equipment operating parameters are concatenated into a 139-dimensional input vector, and the weight vector is calculated based on a multi-layer perceptron. Set load rate growth thresholds and temperature thresholds. If the load rate exceeds the threshold, the vibration weight will automatically increase to strengthen mechanical stress monitoring; if the ambient temperature exceeds the threshold, the temperature weight will be reduced to suppress ambient thermal interference. Input normalized physical features, depth features and dynamic weights, and perform weighted processing on the physical features and depth features respectively; The feature vectors are fused to generate a 136-dimensional comprehensive feature vector, which is used as the input of the health assessment model.

7. The method for health assessment and early warning of electromechanical equipment based on multimodal dynamic perception according to claim 6 is characterized in that: The weight vector is calculated based on the multi-layer perceptron, including: Divide the 139-dimensional input vector into three single-mode vectors according to the vibration dimension, temperature dimension, and acoustic dimension; Based on the current health assessment task, determine the key features, obtain the correlation between the key features and each dimension feature, and determine the internal correlation value between the mutual features in each single modal vector; Based on the correlation between the key feature and each dimensional feature, as well as the internal correlation value, the internal weight of each dimensional feature in each unimodal vector is calculated; Based on the external correlation value between cross-modal features between single-modal vectors and combined with the internal weight, the weight of each dimension feature is obtained; Based on the weights of the dimensional features, a weight vector corresponding to the 139-dimensional input vector is obtained.

8. The method for health assessment and early warning of electromechanical equipment based on multimodal dynamic perception according to claim 6 is characterized in that: The output physical feature vector and depth feature vector are normalized and standardized as follows: The output physical feature vector and depth feature vector are used as input sets and Min-Max normalization is used to map each feature in the physical feature to the interval [0, 1]. The Min-Max normalization calculation formula is as follows: ， in, Expressed as Physical characteristics through - The value after normalization; Expressed as The raw value of a physical characteristic; Represents all the physical characteristics The minimum value of a feature; Represented as all the physical features The maximum value of features; Z-score normalization is used to eliminate the dimensional differences in depth features. The Z-score normalization calculation formula is as follows: ， in, Expressed as The value of a deep feature after Z-score normalization; Expressed as The original value of the deep feature; Expressed as The mean of the dimensional depth feature; Expressed as The standard deviation of the dimensional depth feature.

9. The method for health assessment and early warning of electromechanical equipment based on multimodal dynamic perception according to claim 1 is characterized in that: Build a quantitative model of equipment health based on fusion features and implement a hierarchical early warning strategy, specifically: Input multimodal data of electromechanical equipment under normal operating conditions, use the K-means++ algorithm to cluster the equipment load rate, ambient temperature, and operating time, and divide it into four typical operating condition categories; For each working condition category, calculate the mean vector and covariance matrix of the fusion features; And store it as a benchmark template library for real-time health comparison; Set alert levels based on health, including normal, caution, warning, and critical, and set response actions for each level; Input the real-time fusion features of each working condition category and the benchmark template of the corresponding working condition category, and use the Mahalanobis distance to eliminate the dimensional differences and correlation effects between features; The health of each working condition sample is judged based on the health index threshold, and different levels of early warning response actions are triggered based on the health assessment results.

10. The method for health assessment and early warning of electromechanical equipment based on multimodal dynamic perception according to claim 1, characterized in that: Based on digital twin technology, the health status of equipment and operation and maintenance decisions are visualized, specifically: Input health assessment results and real-time operation data of electromechanical equipment, and write the equipment health data into the BIM model according to the IFC4 standard extended attribute set; Abnormal equipment is displayed as a pulsating red light in the BIM model, with real-time data curves superimposed; Input device QR code labels, real-time fusion of feature data and maintenance knowledge base, AR-based scanning of electromechanical equipment QR codes, calling ARKit / ARCore engine to identify devices; also used to display vibration thermal maps, temperature gradient vectors and maintenance instructions; Input the fault characteristics, treatment measures and verification results of each maintenance work order for later learning.

Images