Special equipment structure health assessment method based on multi-modal sensor fusion and deep learning

Abstract

The application relates to the technical field of special equipment safety monitoring and intelligent diagnosis, and particularly discloses a structure health assessment method based on multi-modal sensing fusion and field knowledge guidance, which comprises the following steps: synchronously collecting acoustic emission, vibration, strain, infrared and visual multi-modal data and time aligning; adaptively denoising and enhancing the features of the multi-modal signals; extracting acoustic emission event clustering, vibration harmonic ratio, fatigue damage index, thermal gradient change rate and surface crack and other deep state characteristics; constructing a graph neural network fusion model embedded with physical constraints, taking equipment subsystems as nodes and mechanical paths as connections, introducing material attenuation and load weight to simulate damage propagation; and realizing four-grade classification of the health state and remaining life prediction based on the fusion representation and combined with the Paris law correction. The application improves damage sensing accuracy, model interpretability and engineering applicability under complex working conditions, and supports real-time early warning and scientific maintenance decision-making in high-temperature and high-pressure environments.

Description

Technical Field

[0001] This invention belongs to the field of special equipment safety monitoring and intelligent diagnosis technology, specifically involving a method for structural health assessment of special equipment based on multimodal sensor fusion and deep learning. Background Technology

[0002] With the in-depth development of the Industrial Internet of Things (IIoT) and intelligent monitoring technologies, structural health assessment of special equipment has become a crucial link in ensuring the safe operation of major infrastructure. Special equipment, such as pressure vessels, lifting machinery, and large pipeline systems, typically operate in harsh environments with high temperatures, high pressures, or strong corrosion, and their structural integrity is directly related to production safety and public safety. Traditional health assessment methods mainly rely on periodic manual inspections or threshold alarm mechanisms based on single sensors, making it difficult to achieve continuous perception and early warning of the evolution of subtle damage throughout the entire equipment lifecycle. This results in delayed risk identification, high maintenance costs, and the potential for missed detections.

[0003] Among these advancements, structural health monitoring technology based on multimodal sensor fusion has garnered significant attention in recent years. This technology integrates various physical signals, including acoustic emission, vibration, strain, temperature, and visual signals, to construct a multidimensional representation of equipment condition. However, existing fusion methods often employ simple weighted averaging or rule-based concatenation strategies, failing to fully exploit the complementarity and redundancy of different modal data across time and space, thus limiting feature representation capabilities. Furthermore, traditional signal processing algorithms lack robustness in handling high-noise, non-stationary signals generated under complex operating conditions of specialized equipment, making them susceptible to environmental interference and prone to misjudgments.

[0004] Furthermore, while deep learning models have demonstrated strong potential in pattern recognition, existing solutions often directly input multi-source data into a general network architecture, lacking prior knowledge guidance on the failure mechanisms of special equipment structures. This results in poor model interpretability and weak generalization ability. Especially in real-world scenarios with small sample sizes and few fault labels, model training is prone to overfitting, making it difficult to support highly reliable health status grading and remaining life prediction. Therefore, there is an urgent need for a structural health assessment method that deeply integrates multimodal sensing information and embeds domain knowledge to achieve accurate, real-time, and adaptive identification of damage states in special equipment. Summary of the Invention

[0005] The purpose of this invention is to provide a structural health assessment method based on multimodal sensor fusion and domain knowledge guidance, which can effectively solve the technical problems mentioned in the background art, such as the difficulty of traditional monitoring methods in achieving continuous perception and early warning of the evolution process of minor damage to special equipment, the failure of existing fusion algorithms to fully explore the complementarity of multimodal data in the spatiotemporal dimension, insufficient robustness of signal processing, lack of interpretability of deep learning models, and weak generalization ability in small sample scenarios.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A structural health assessment method based on multimodal sensor fusion and domain knowledge guidance includes the following specific steps: Step 1: Synchronously collect multimodal sensing data during the operation of special equipment. Deploy acoustic emission sensors, triaxial vibration sensors, fiber optic strain gauges, infrared thermal imagers, and high-definition industrial cameras at key load-bearing parts of the equipment. Acoustic emission waveform signals are acquired synchronously at a sampling frequency of 10 kHz, vibration acceleration time-series data are acquired at a sampling frequency of 2 kHz, distributed strain response is recorded at a sampling frequency of 500 Hz, surface temperature field distribution images are acquired every 30 seconds, and visual images of the equipment's appearance are captured at a rate of one frame every 10 seconds. All data are aligned with timestamps to form a spatiotemporally synchronized original observation sequence. Step 2: Adaptive denoising and feature enhancement are performed on the original multimodal data. For acoustic emission signals, an improved wavelet threshold denoising algorithm is used. The number of decomposition layers and the threshold function type are dynamically adjusted in combination with the signal energy distribution to retain effective waveform features and suppress high-frequency noise. For vibration signals, empirical mode decomposition is performed to screen the intrinsic mode function components that match the device's natural frequency for reconstruction. Strain data is filtered by Kalman filtering to eliminate the influence of temperature drift. Infrared images are filtered by nonlocal mean filtering to suppress random noise and maintain the clarity of thermal anomaly edges. Visual images are enhanced by histogram equalization to improve the contrast of low-light areas. Step 3: Extract deep state features of each modal data, calculate rise time, ring count, energy integral and source location coordinates from the denoised acoustic emission signal, construct event spatiotemporal clustering feature vector, extract the amplitude ratio of the 3rd to 7th harmonics, kurtosis index and envelope spectrum peak frequency from the vibration reconstruction signal, generate dynamic response mode feature set, identify the time accumulation effect of stress concentration area from strain sequence, quantify fatigue damage index, segment high temperature area from infrared image and calculate its area growth rate and thermal gradient change rate, and use convolutional neural network to detect surface crack length and propagation direction from visual image; Step 4: Construct a graph neural network fusion model with embedded physical information. Divide the equipment structure into several functional subsystems as graph nodes. Establish the connection relationship between nodes according to the mechanical transmission path to form a topological graph structure. Each node is input with the multimodal feature vector of the corresponding region. The network layer design includes physical constraint regularization terms. Force the hidden layer output to meet the energy conservation condition of the basic equation of linear elasticity. In the message transmission process, the material attenuation coefficient and load transmission weight factor are introduced to realize the simulation and prediction of damage propagation path. Step 5: Perform health status grading and remaining life estimation. Based on the global representation vector output by the graph neural network, a support vector machine classifier is used to classify the equipment health status into four levels: normal, minor damage, moderate degradation, and severe deterioration. At the same time, a long short-term memory network is constructed, which uses the historical state sequence as input to predict the state transition probability distribution in the next 30 days. Combined with the Paris fatigue crack propagation law to correct the parameters, the estimated remaining life and its confidence interval are output.

[0007] Preferably, in step 1, the acoustic emission sensor uses a resonant broadband probe with a center frequency of 150 kHz, and the installation spacing does not exceed 1.2 meters to ensure a positioning accuracy better than ±5 cm. All sensors are equipped with high-temperature protective sleeves and can work stably in environments up to 350 degrees Celsius.

[0008] Preferably, the improved wavelet thresholding denoising algorithm in step 2 uses the Daubechies4 wavelet basis. The number of decomposition layers is automatically determined based on the signal-to-noise ratio. When the signal-to-noise ratio is below 10 dB, a 5-layer decomposition is enabled and soft thresholding is used. When it is above 20 dB, a 3-layer decomposition and hard thresholding combination strategy is used. The threshold calculation formula is dynamically generated based on the Stein unbiased risk estimation method.

[0009] Preferably, in step 3, the acoustic emission source localization adopts a three-dimensional spherical localization algorithm, which uses the arrival time difference received by at least 6 sensors to construct an overdetermined system of equations, and solves the spatial coordinates of the sound source by the least squares method. During the localization process, data points with residuals greater than 0.8 microseconds are removed to improve reliability.

[0010] Preferably, in step 3, the intrinsic mode function components of the vibration signal are screened based on the correlation coefficient criterion, and only the components with an absolute value of correlation coefficient with the theoretical main frequency of the equipment greater than 0.6 are retained to participate in the reconstruction, so as to avoid mode aliasing interference. At the same time, the reconstructed signal is subjected to Hilbert transform to extract the instantaneous frequency trajectory for nonlinear dynamic analysis.

[0011] Preferably, in step 4, the physical constraint regularization term of the graph neural network is defined as the degree to which the stress-strain relationship inside the node deviates from Hooke's law. The loss weight of this term is set to 0.3 and gradually decreases as the training progresses with convergence. In the message passing update rule, a decay factor inversely proportional to the square of the distance is introduced to simulate the geometric diffusion effect of energy propagation in the actual structure.

[0012] Preferably, the edge weights in the topology graph structure in step 4 consist of two parts: one is the normalized result of the static stiffness matrix obtained based on finite element simulation, and the other is the time-varying coupling strength updated online based on the measured dynamic cross-correlation coefficient. The two are weighted and fused to form a comprehensive connection weight, and the weight update cycle is once every 72 hours.

[0013] Preferably, in step 5, the support vector machine classifier uses a radial basis function kernel function, the penalty factor C is set to 100, the optimal hyperparameter combination is determined by grid search combined with cross-validation, the training sample set contains no less than 2000 sets of labeled data of no less than 5 typical fault modes, and the stability of the classification decision boundary is verified by adversarial sample perturbation test.

[0014] Preferably, in step 5, the long short-term memory network contains two hidden layers, each with 128 neurons. The input sequence is a historical state snapshot with a length of 168 hours. The activation function of the output layer is a linear function. The training objective is to minimize the mean squared error between the predicted lifetime and the actual lifetime. An L2 regularization term is added to the loss function to prevent overfitting, and the regularization coefficient is 0.001.

[0015] Preferably, it also includes: establishing a health record database for the entire life cycle of the equipment, storing the result data of each assessment, original sensor records and model parameter snapshots, the database adopts a time-series data engine to optimize query efficiency, supports retrieval by time range, equipment number or multi-dimensional condition combination, and the data retention period is not less than 15 years.

[0016] Preferably, it also includes: deploying edge computing units in the on-site monitoring system, having a built-in lightweight version of the evaluation model, an inference latency of less than 200 milliseconds, supporting local analysis tasks that can run continuously for more than 72 hours in the absence of network access, and automatically synchronizing incremental data to the cloud central server after communication is restored.

[0017] Preferably, the method is applied to the health monitoring scenario of pressure vessels in nuclear power plants, with a monitoring coverage of 98%, achieving early warning 7 days in advance in simulated crack propagation experiments, a false alarm rate of less than 2 times per year, and a single complete assessment taking no more than 90 seconds, thus meeting the requirements for real-time monitoring.

[0018] In summary, this application includes at least one of the following beneficial technical effects: This invention achieves comprehensive perception of the operating status of special equipment by constructing a spatiotemporally synchronized multimodal data acquisition system, overcoming the limitations of single-sensor information. Adaptive denoising and feature enhancement strategies significantly improve the robustness of signal processing under complex operating conditions, ensuring the accuracy of feature extraction. The proposed physical information embedded graph neural network model integrates prior knowledge of structural mechanics into a deep learning framework, enhancing the model's interpretability and generalization ability, and solving the problem of overfitting in traditional black-box models under small sample conditions. Through a joint inference mechanism of health status grading and remaining life, it provides assessment results with both diagnostic accuracy and predictive foresight, supporting the scientific formulation of preventative maintenance plans. The entire method has good engineering applicability and its long-term stability has been verified under high temperature and high pressure environments, significantly reducing the probability of sudden failure of major equipment and improving operation and maintenance efficiency and safety assurance levels. Attached Figure Description

[0019] Figure 1 This is a logical flowchart of the joint deduction of health status classification and remaining lifespan in this invention. Detailed Implementation

[0020] Example 1 To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0021] Currently, special equipment such as pressure vessels, lifting machinery, and large pipeline systems typically operate in harsh environments with high temperatures, high pressures, or strong corrosion, and their structural integrity directly impacts production and public safety. Traditional health assessment methods primarily rely on periodic manual inspections or threshold alarm mechanisms based on single sensors, making it difficult to continuously perceive and provide early warnings of the evolution of minor damage throughout the equipment's entire lifecycle. This results in delayed risk identification, high maintenance costs, and the potential for missed detections. To address these technical problems, this invention proposes a structural health assessment method based on multimodal sensor fusion and domain knowledge guidance. This method effectively solves the technical issues of traditional monitoring methods' inability to continuously perceive and provide early warnings of the evolution of minor damage in special equipment, existing fusion algorithms' failure to fully exploit the complementarity of multimodal data in the spatiotemporal dimensions, insufficient signal processing robustness, lack of interpretability in deep learning models, and weak generalization ability in small sample scenarios. This method is applied to a structural health assessment method based on multimodal sensor fusion and domain knowledge guidance.

[0022] The core process of the method described in this invention includes five major steps: synchronous acquisition of multimodal sensor data, adaptive denoising and feature enhancement, deep state feature extraction, graph neural network fusion modeling with physical information embedding, and joint extrapolation of health status grading and remaining lifespan. The entire system consists of multi-type sensor arrays deployed on-site, edge computing units, cloud analysis platforms, and visualization terminals, forming a closed-loop intelligent health assessment system.

[0023] In the above-mentioned structural health assessment method based on multimodal sensor fusion and domain knowledge guidance, step (1) involves synchronously collecting multimodal sensor data during the operation of special equipment. Specifically, acoustic emission sensors, triaxial vibration sensors, fiber optic strain gauges, infrared thermal imagers, and high-definition industrial cameras are deployed at key load-bearing parts of the equipment. These parts are predetermined based on finite element stress analysis and typically include welds, flange connections, support bases, and geometrically discontinuous areas. All sensors achieve hardware-level time synchronization through a high-precision timing module to ensure consistent sampling clock sources. Acoustic emission waveform signals are synchronously acquired at a sampling frequency of 10 kHz, which can capture transient elastic waves generated by the initiation and propagation of microcracks inside the material; vibration acceleration time series data are acquired at a sampling frequency of 2 kHz to characterize the overall dynamic response characteristics of the equipment; distributed strain response is recorded at a sampling frequency of 500 Hz to reflect local stress distribution and cumulative effects; surface temperature field distribution images are acquired every 30 seconds to monitor potential thermal anomaly areas; and visual images of the equipment's appearance are captured at a rate of one frame every 10 seconds for visual identification of macroscopic defects. All raw data streams are appended with high-precision timestamps (accuracy better than 1 microsecond) and transmitted to edge computing units via high-speed industrial buses. Finally, they are aligned to form a spatiotemporally synchronized raw observation sequence, providing a precise data foundation for subsequent multimodal fusion.

[0024] Specifically, in step (1), the acoustic emission sensor uses a resonant broadband probe with a center frequency of 150 kHz, and the installation spacing does not exceed 1.2 meters to ensure a positioning accuracy better than ±5 cm. All sensors are equipped with high-temperature protective sleeves, which can work stably in environments up to 350 degrees Celsius. The protective sleeves are filled with thermally conductive silicone grease to ensure acoustic coupling efficiency. The triaxial vibration sensor uses a piezoelectric accelerometer with a range of ±50g and a frequency response range covering 0.5 Hz to 10 kHz. It is rigidly fixed to the surface of the equipment by a magnetic base or bolts. The fiber optic strain gauge uses Brillouin optical time-domain analysis technology with a spatial resolution of 10 cm and a strain measurement accuracy of ±2 microstrain. The infrared thermal imager has a thermal sensitivity better than 50 milliklvin, an image resolution of 640×480 pixels, and is equipped with an autofocus lens. The high-definition industrial camera has a pixel count of no less than 5 million, supports global shutter mode, and is equipped with a ring LED light source to eliminate motion blur. The entire sensor network communicates with the edge computing unit via industrial Ethernet or 5G private network, and the data transmission protocol adopts the time-sensitive networking standard to ensure real-time performance.

[0025] In the above-mentioned structural health assessment method based on multimodal sensing fusion and domain knowledge guidance, step (2) involves adaptive denoising and feature enhancement of the original multimodal data. Specifically, for acoustic emission signals, an improved wavelet threshold denoising algorithm is used. This algorithm first performs multi-scale wavelet decomposition on the signal, and the number of decomposition layers is automatically determined according to the signal-to-noise ratio: when the signal-to-noise ratio is below 10 dB, a 5-layer decomposition is enabled and a soft threshold is used; when it is above 20 dB, a 3-layer decomposition and hard threshold combination strategy is used. The wavelet basis function Daubechies4 is selected because of its excellent performance in time-frequency localization. The threshold calculation formula is dynamically generated based on the Stein unbiased risk estimation method, which can adaptively balance noise suppression and signal fidelity, and effectively preserve the rising edge, peak value and attenuation characteristics of acoustic emission events. For vibration signals, empirical mode decomposition is implemented to decompose complex non-stationary signals into a series of intrinsic mode function components. The intrinsic mode function components that match the device's natural frequency are selected for reconstruction. The selection is based on the correlation coefficient criterion, retaining only components with an absolute correlation coefficient greater than 0.6 with the device's theoretical main frequency to avoid mode aliasing interference. Strain data is processed using a Kalman filter, with temperature introduced as an interference input into the state equation. The filter is calibrated online using synchronously acquired temperature data, effectively eliminating spurious strain readings caused by temperature drift. Infrared images employ a non-local mean filtering algorithm. This algorithm effectively suppresses random noise while maintaining edge clarity in thermal anomaly areas by finding similar image patches and performing weighted averaging, preventing detail loss due to over-smoothing. Visual images undergo histogram equalization, particularly in low-light areas, where adaptive gamma correction enhances local contrast, highlighting minute surface cracks and corrosion marks.

[0026] In the above-mentioned structural health assessment method based on multimodal sensor fusion and domain knowledge guidance, step (3) extracts the deep state features of each modality data. Specifically, from the denoised acoustic emission signal, the rise time, ring count, energy integral, and source location coordinates of each valid event are calculated. The rise time is defined as the time required for the signal amplitude to rise from 10% peak to 90% peak, reflecting the crack propagation rate; the ring count is the number of times the signal exceeds a preset threshold, characterizing the intensity of damage activity; the energy integral is obtained by integrating the square of the signal, quantifying the amount of energy released by the damage. The acoustic emission source location adopts a three-dimensional spherical positioning algorithm, which uses the arrival time difference received by at least 6 sensors to construct an overdetermined system of equations, and solves the spatial coordinates of the sound source by the least squares method. During the positioning process, the residuals of each solution are statistically tested, and data points with residuals greater than 0.8 microseconds are removed to improve reliability. Finally, an event clustering feature vector containing spatiotemporal information is constructed. From the vibration reconstruction signal, the amplitude ratio of the 3rd to 7th harmonics, the kurtosis index, and the peak frequency of the envelope spectrum are extracted. Harmonic amplitude ratio reflects the degree of nonlinearity, kurtosis index measures signal pulse characteristics, and envelope spectrum peak frequency indicates fault characteristic frequency; together, they generate a dynamic response mode feature set. From strain sequences, the time accumulation effect of stress concentration areas is identified through sliding window analysis, the variance of strain increment per unit time is calculated, and the fatigue damage index is quantified by combining Miner's linear cumulative damage theory. From infrared images, an adaptive threshold segmentation algorithm is used to separate high-temperature regions, and their area growth rate and thermal gradient change rate are calculated; the former reflects the heat source expansion trend, and the latter indicates abnormal heat conduction. From visual images, a pre-trained convolutional neural network is used to detect surface cracks. This network adopts the U-Net architecture and is fine-tuned on a large amount of labeled data, enabling it to accurately output pixel-level masks of cracks, thereby calculating crack length and propagation direction to form a visual feature vector.

[0027] In the above-mentioned structural health assessment method based on multimodal sensing fusion and domain knowledge guidance, step (4) involves constructing a graph neural network fusion model with embedded physical information. Specifically, the equipment structure is divided into several functional subsystems as graph nodes. The division principle is based on the physical modules, load paths, and failure modes of the equipment. For example, a pressure vessel is divided into nodes such as the cylinder, head, nozzle, and support. The connection relationship between nodes is established according to the mechanical transmission path to form a topological graph structure. The connection relationship consists of two weights: one is the normalized result of the static stiffness matrix obtained from finite element simulation, which reflects the inherent mechanical coupling strength of the structure; the other is the time-varying coupling strength updated online according to the measured dynamic cross-correlation coefficient, which reflects the dynamic interaction under actual operating conditions. The two are weighted and fused to form a comprehensive connection weight. The weight update cycle is once every 72 hours to adapt to the slow evolution of the equipment state. Each node inputs the multimodal feature vector of the corresponding region, which is the concatenation of all features extracted in step (3). Each layer of the graph neural network includes a physical constraint regularization term, defined as the degree to which the stress-strain relationship within a node deviates from Hooke's Law; that is, the mean square error between the calculated predicted strain and the theoretical strain calculated by Hooke's Law. This term's loss weight is initially set to 0.3 and gradually reduced during training as convergence progresses, ensuring the model strictly adheres to physical laws in the early stages while allowing data-driven fine-tuning in later stages. The message passing update rule introduces an attenuation factor inversely proportional to the square of the distance, simulating the geometric diffusion effect of energy propagation in actual structures. Simultaneously, a material attenuation coefficient and a load transfer weight factor are introduced; the former is preset based on the material type, while the latter is dynamically adjusted based on real-time load data, thereby achieving a physically consistent simulation and prediction of the damage propagation path within the structure. The model aggregates global information through multiple rounds of message passing, ultimately outputting a global representation vector containing overall equipment health information.

[0028] In the above-mentioned structural health assessment method based on multimodal sensing fusion and domain knowledge guidance, step (5) involves performing health status grading and remaining lifespan estimation. Specifically, based on the global representation vector output by the graph neural network, a support vector machine classifier is used to classify the equipment health status into four levels: normal, minor damage, moderate degradation, and severe deterioration. The support vector machine classifier uses a radial basis function kernel function, with a penalty factor C set to 100, and the optimal hyperparameter combination is determined through grid search combined with cross-validation. The training sample set contains no less than 2000 sets of labeled data for no less than 5 typical fault modes, covering various damage types and operating conditions. The classification decision boundary is tested for stability by adversarial sample perturbation to ensure that the classification result remains unchanged when faced with small input perturbations. At the same time, a long short-term memory network is constructed, using historical state sequences as input, to predict the state transition probability distribution within the next 30 days. This long short-term memory network contains two hidden layers, each with 128 neurons, and the input sequence is a 168-hour historical state snapshot, i.e., the state assessment results for each hour over the past 7 days. The output layer activation function is a linear function, directly outputting the estimated remaining useful life for each day of the next 30 days. The training objective is to minimize the mean squared error between the predicted and actual useful life. An L2 regularization term with a coefficient of 0.001 is added to the loss function to prevent overfitting. Furthermore, the network output is corrected using Paris's fatigue crack propagation law, which describes the relationship between crack propagation rate and the stress intensity factor range. By mapping the damage index predicted by the network to the parameter space of Paris's law, the estimated remaining useful life and its confidence interval are physically calibrated for consistency, thus providing more reliable prediction results.

[0029] To support the engineering application of the above methods, this invention also includes establishing a health record database for the entire lifecycle of equipment. This database stores the results of each assessment, original sensor records, and model parameter snapshots. It is optimized using time-series data engines such as InfluxDB or TimescaleDB, supporting efficient retrieval by time range, equipment number, or multi-dimensional condition combinations. All data is stored using compression algorithms, with a data retention period of no less than 15 years, meeting the regulatory requirements for the entire lifecycle of special equipment. Furthermore, it includes deploying an edge computing unit in the on-site monitoring system. This unit incorporates a lightweight version of the assessment model, compressing the original model to a size suitable for operation on resource-constrained devices through model pruning, quantization, and knowledge distillation techniques. The inference latency is less than 200 milliseconds, supporting continuous local analysis tasks for more than 72 hours even without network connectivity, and automatically synchronizing incremental data to the cloud central server after communication is restored, ensuring data integrity and system availability.

[0030] In a specific application instance, the method is applied to the health monitoring of pressure vessels in a nuclear power plant. The monitoring system covers the pressure vessel shell, upper and lower heads, and all nozzle areas, achieving a monitoring coverage rate of 98%. During a six-month field verification, the system successfully achieved a seven-day early warning in simulated crack propagation experiments, accurately identifying the crack location and propagation trend. Throughout the year of operation, the system's false alarm rate was less than two per year, and a single complete assessment took no more than 90 seconds, fully meeting the stringent real-time monitoring requirements of nuclear power plants. Even when the edge computing unit loses connection to the cloud, it can still independently complete local health assessments and seamlessly synchronize data after reconnection, demonstrating extremely high system robustness.

[0031] Example 2 Based on the aforementioned Embodiment 1, and considering the differences in structural characteristics and service environments of various special equipment, this invention can also adapt to new application scenarios by adjusting the topology construction rules and physical constraint terms of the graph neural network. For example, when applied to the health assessment of large lifting machinery, the equipment structure is divided into nodes such as boom, slewing bearing, hoisting mechanism, and traveling mechanism. Due to the high directionality and instantaneity of the loads on lifting machinery, the edge weight calculation in its topology graph structure needs to focus on the influence of dynamic load direction. Specifically, the static stiffness matrix is ​​obtained by multibody dynamics simulation instead of finite element simulation to more accurately reflect the kinematic constraints between mechanisms. The time-varying coupling strength is updated by analyzing the cross-correlation between torque and angular velocity at each joint. The physical constraint regularization term is correspondingly modified to conform to the form of rigid body dynamics equations, forcing the hidden layer output to satisfy the torque balance and power conservation conditions. The attenuation factor in the message passing process is not only related to distance, but also introduces the joint damping coefficient as an adjustment parameter. In the feature extraction stage, the focus of vibration signal analysis shifts from harmonic components to impact pulse characteristics to capture local faults in bearings and gears. Through this targeted adjustment, the method of the present invention can also be effectively applied to lifting machinery, enabling accurate diagnosis and life prediction of faults such as wear, fatigue and loosening of its key components, demonstrating the versatility and scalability of the method.

[0032] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0033] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment includes only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A structural health assessment method based on multimodal sensor fusion and domain knowledge guidance, characterized in that: The specific steps include the following: Step 1: Synchronously collect multimodal sensing data during the operation of special equipment. Deploy acoustic emission sensors, triaxial vibration sensors, fiber optic strain gauges, infrared thermal imagers, and high-definition industrial cameras at key load-bearing parts of the equipment. Acoustic emission waveform signals are acquired synchronously at a sampling frequency of 10 kHz, vibration acceleration time-series data are acquired at a sampling frequency of 2 kHz, distributed strain response is recorded at a sampling frequency of 500 Hz, surface temperature field distribution images are acquired every 30 seconds, and visual images of the equipment's appearance are captured at a rate of one frame every 10 seconds. All data are aligned with timestamps to form a spatiotemporally synchronized original observation sequence. Step 2: Adaptive denoising and feature enhancement are performed on the original multimodal data. For acoustic emission signals, an improved wavelet threshold denoising algorithm is used to retain effective waveform features and suppress high-frequency noise. For vibration signals, empirical mode decomposition is performed and intrinsic mode function components that match the device's natural frequency are selected for reconstruction. Strain data is filtered by Kalman filtering to eliminate the influence of temperature drift. Infrared images are filtered by nonlocal mean filtering to suppress random noise and maintain the clarity of thermal anomaly edges. Visual images are enhanced by histogram equalization to improve the contrast of low-light areas. Step 3: Extract deep state features of each modal data; calculate rise time, ring count, energy integral and source location coordinates from the denoised acoustic emission signal to construct event spatiotemporal clustering feature vector; extract the amplitude ratio of the 3rd to 7th harmonics, kurtosis index and envelope spectrum peak frequency from the vibration reconstruction signal to generate dynamic response mode feature set; identify the time accumulation effect of stress concentration area from strain sequence and quantify fatigue damage index; segment high temperature region from infrared image and calculate its area growth rate and thermal gradient change rate; detect surface crack length and propagation direction from visual image using convolutional neural network. Step 4: Construct a graph neural network fusion model with embedded physical information. Divide the equipment structure into several functional subsystems as graph nodes. Establish the connection relationship between nodes according to the mechanical transmission path to form a topological graph structure. Each node is input with the multimodal feature vector of the corresponding region. The network layer design includes physical constraint regularization terms to force the hidden layer output to meet the energy conservation conditions of the basic equation of linear elasticity. In the message transmission process, the material attenuation coefficient and load transmission weight factor are introduced to realize the simulation and prediction of the damage propagation path. Step 5: Perform health status grading and remaining life estimation. Based on the global representation vector output by the graph neural network, the support vector machine classifier is used to classify the equipment health status into four levels: normal, minor damage, moderate degradation and severe deterioration. At the same time, a long short-term memory network is constructed to predict the state transition probability distribution in the next 30 days using the historical state sequence as input. Combined with the Paris fatigue crack propagation law, the parameters are corrected to output the estimated remaining life and its confidence interval.

2. The structural health assessment method based on multimodal sensor fusion and domain knowledge guidance according to claim 1, characterized in that: The acoustic emission sensor uses a resonant broadband probe with a center frequency of 150 kHz, an installation spacing of no more than 1.2 meters, and is equipped with a high-temperature protective sleeve that can operate stably in environments up to 350 degrees Celsius.

3. The structural health assessment method based on multimodal sensor fusion and domain knowledge guidance according to claim 1, characterized in that: The improved wavelet thresholding denoising algorithm uses the Daubechies4 wavelet basis. The number of decomposition layers is automatically determined based on the signal-to-noise ratio (SNR). When the SNR is below 10 dB, a 5-layer decomposition is enabled and soft thresholding is used. When the SNR is above 20 dB, a combination strategy of 3-layer decomposition and hard thresholding is adopted. The threshold is dynamically generated based on the Stein unbiased risk estimation method.

4. The structural health assessment method based on multimodal sensor fusion and domain knowledge guidance according to claim 1, characterized in that: The acoustic emission source localization adopts a three-dimensional spherical localization algorithm, which uses the arrival time difference received by at least 6 sensors to construct an overdetermined system of equations and solves the spatial coordinates of the sound source by the least squares method. During the localization process, data points with residuals greater than 0.8 microseconds are eliminated.

5. The structural health assessment method based on multimodal sensor fusion and domain knowledge guidance according to claim 1, characterized in that: The intrinsic mode function components of the vibration signal are selected based on the correlation coefficient criterion. Only components with an absolute value of correlation coefficient greater than 0.6 with the theoretical main frequency of the device are retained for reconstruction. The instantaneous frequency trajectory is extracted by performing Hilbert transform on the reconstructed signal.

6. The structural health assessment method based on multimodal sensor fusion and domain knowledge guidance according to claim 1, characterized in that: The physical constraint regularization term of the graph neural network is defined as the degree to which the stress-strain relationship inside the node deviates from Hooke's law. The loss weight of this term is initially set to 0.3 and gradually reduced during training. A decay factor inversely proportional to the square of the distance is introduced into the message passing update rule.

7. The structural health assessment method based on multimodal sensor fusion and domain knowledge guidance according to claim 1, characterized in that: The edge weights in the topological graph structure are formed by weighted fusion of the normalized result of the static stiffness matrix obtained based on finite element simulation and the time-varying coupling strength updated online according to the measured dynamic cross-correlation coefficient. The weight update cycle is once every 72 hours.

8. The structural health assessment method based on multimodal sensor fusion and domain knowledge guidance according to claim 1, characterized in that: The support vector machine classifier uses a radial basis function kernel function, with a penalty factor C set to 100. The training sample set contains 2000 sets of labeled data for no less than 5 typical fault modes. The stability of the classification decision boundary is verified by adversarial sample perturbation test.

9. The structural health assessment method based on multimodal sensor fusion and domain knowledge guidance according to claim 1, characterized in that: The Long Short-Term Memory (LSTM) network contains two hidden layers, each with 128 neurons. The input sequence is a 168-hour historical state snapshot. The loss function includes an L2 regularization term with a regularization coefficient of 0.001, and the output layer activation function is a linear function.

Images