Aviation structure intelligent health monitoring method based on neuromorphic comparative learning and multi-modal data

Abstract

An aviation structure intelligent health monitoring method based on neuromorphic comparative learning and multi-modal data comprises the steps that firstly, time alignment and amplitude normalization preprocessing are conducted on guided wave and acoustic emission multi-modal data sets, a unified data representation space is constructed, and a multi-modal sample pair is constructed; secondly, constructing a parallel pulse convolutional neural network model, encoding the preprocessed multi-modal data into a pulse sequence, extracting damage sensitive features through parallel pulse neural network branches, fusing the damage sensitive features in a feature fusion unit to obtain comprehensive feature representation, and mapping the comprehensive feature representation into an output pulse distribution rate; a comparison learning mechanism is adopted, and a multi-level comparison loss function is designed for training, so that monotonous and robust aviation structure health indexes can be obtained; and finally, deploying the trained parallel pulse convolutional neural network model to a neuromorphic hardware system to realize real-time aviation structure health monitoring. According to the method, the dependence on supervision annotation data is remarkably reduced, and the actual requirement for real-time monitoring of an aviation structure is met.

Description

Technical Field

[0001] The present invention belongs to the technical field of aviation structure health monitoring, and in particular relates to an aviation structure intelligent health monitoring method based on neuromorphic contrast learning and multimodal data. Background Art

[0002] As modern aerospace structural design continues to evolve toward lightweighting and high integration, structural complexity has significantly increased. Aircraft development requires multiple rounds of ground testing and verification to ensure structural safety and design reliability. Because aerospace structures are prone to subtle and hidden damage evolution under complex environments and loading conditions, efficient and accurate monitoring and assessment of structural health has become a crucial means of accelerating model development and improving efficiency.

[0003] Traditional structural health monitoring (SHM) methods mostly rely on single-modal sensor signals, which have limitations in capturing the evolution of complex damage and are difficult to fully reflect the changing trends of structural performance. To solve this problem, multimodal data fusion technology has emerged, which integrates different modal data to achieve a comprehensive assessment of the structural state. However, existing multimodal data fusion methods usually rely on a large amount of accurately labeled data ([1]X.Yang,C.Fang,and P.Kundu,“A decision-level sensor fusion scheme integrating ultrasonic guided wave and vibration measurements for damage identification,”Mechanical Systems and Signal Processing,vol.219,p.111597,2024), with high training overhead and insufficient real-time performance. In addition, its high power consumption makes it difficult to deploy on resource-constrained edge devices, making it difficult to meet the application requirements of rapid model development. The contrastive learning method uses customized pseudo-labels to mine potential effective features from large-scale unlabeled data, achieving robust feature representation without a large amount of labeled data, significantly reducing the dependence on supervised data, and providing a new feasible approach to solving the above problems.

[0004] In addition, the current mainstream multimodal data fusion technology is generally based on deep learning models ([2] L. Fan, Y. Wang, and H. Zhang, “Multimodal perception and decision-making systems for complex roads based on foundation models,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 54, no. 11, pp. 6561–6569, 2024). Although it has excellent performance, it has high computational complexity and often requires the support of a high-performance computing platform. It is difficult to meet the real-time requirements of structural health monitoring in the aircraft development and flight test phases, especially in scenarios where long-term continuous online monitoring is required. High computational complexity is usually accompanied by high power consumption, which poses a great challenge to airborne real-time monitoring systems with limited power supply. This fact has not received much attention in the current literature because people usually pay more attention to computing performance in different situations rather than energy efficiency.

[0005] In recent years, neuromorphic computing, as an emerging biomimetic computing paradigm, has attracted much attention for its low energy consumption and efficient real-time processing capabilities. It simulates the event-driven mechanism of biological brain neurons, triggering calculations and consuming energy only when neurons generate pulses, thereby significantly improving the efficiency of spatiotemporal feature processing. It is particularly suitable for building low-power, high-performance intelligent real-time monitoring systems. Although neuromorphic computing has shown good application potential in fields such as condition monitoring ([3]X.Chen, X.Li, and S.Yu, “Dynamic vision enabled contactless cross-domain machine fault diagnosis with neuromorphic computing,” IEEE / CAA Journal of AutomaticaSinica, vol.11, no.3, pp.788–790, 2024; [4]Z.Xu, Y.Ma, Z.Pan, and X.Zheng, “Deepspiking residual shrinkage network for bearing fault diagnosis,” IEEE Transactions on Cybernetics, vol.54, no.3, pp.1608–1613, 2024), its energy efficiency advantage in actual artificial intelligence systems has not yet been fully utilized. Summary of the Invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide an intelligent health monitoring method for aviation structures based on neuromorphic contrastive learning and multimodal data, which solves the shortcomings of traditional methods in comprehensively capturing complex damage characteristics, real-time monitoring, and deployment of resource-constrained edge devices, and realizes low-power, high-precision intelligent health monitoring of aviation structures based on multimodal data without relying on a large amount of labeled data.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] A method for intelligent health monitoring of aviation structures based on neuromorphic contrastive learning and multimodal data is proposed. First, the collected multimodal datasets of guided waves and acoustic emission are preprocessed. Second, a parallel spike convolutional neural network model for multimodal data is established. The preprocessed multimodal data is encoded into a pulse sequence to extract high-level damage-sensitive features, which are then fused and output as the pulse emission rate. Then, a contrastive learning mechanism is adopted to design a multi-level contrastive loss function, and the spike neural network is trained to obtain monotonic and robust health indicators. Finally, the method is deployed on a neuromorphic hardware system to verify the energy consumption advantages of neuromorphic computing.

[0009] A method for intelligent health monitoring of aviation structures based on neuromorphic contrastive learning and multimodal data includes the following steps:

[0010] Step 1: Perform time alignment and amplitude normalization preprocessing on the collected multimodal data of guided waves and acoustic emission from aerospace structures to build a unified data representation space, construct multimodal sample pairs, and divide the training and test sets according to the test pieces.

[0011] Step 2: A parallel spiking convolutional neural network model is established based on the Integrate-And-Fire (IAF) neuron. The preprocessed guided wave and acoustic emission signals are converted into pulse trains through the encoding layer. Damage-sensitive features are extracted through parallel spiking neural network (SNN) branches. The damage-sensitive features are then fused through a feature fusion unit to obtain a comprehensive feature representation, which is then mapped to the output pulse firing rate.

[0012] Step 3: Using a contrastive learning mechanism, a multi-level contrastive loss function is designed to construct a monotonic and robust aviation structural health indicator, achieving accurate characterization of the health status of aviation structures.

[0013] Step 4: Input the training set obtained in step 1 into the parallel spiking convolutional neural network model constructed in step 2, calculate the output pulse rate, and optimize the multi-level contrast loss function designed in step 3. Iteratively train the model parameters until the model converges to obtain the final parallel spiking convolutional neural network model.

[0014] Step 5: Deploy the final parallel spiking convolutional neural network model obtained in step 4 to an asynchronous event-driven neuromorphic hardware system, and input the test set obtained in step 1 into the neuromorphic hardware system to achieve real-time health monitoring of aviation structures.

[0015] The step 1 is specifically as follows: performing time domain alignment on the different modal data to ensure that the different modal data are consistent in time scale; performing amplitude normalization preprocessing on the multimodal data to make it in the range [0,1]:

[0016]

[0017] Where x′ ij represents the i-th sample value of the j-th class data after normalization preprocessing, x ij is the i-th sample value of the j-th class data in the original data before normalization, min(x j ) and max(x j ) represent the minimum and maximum values of the j-th category data respectively;

[0018] The sliding window method is used to sample the normalized preprocessed multimodal signal and construct the sample pairs required for contrastive learning; the window size is fixed to W L , the sliding step size is fixed to W S , then the number of samples N obtained by sliding window sampling W for:

[0019]

[0020] Where N is the total length of the data;

[0021] The sample pairs are constructed by pairing each other, and finally N P Valid sample pairs:

[0022]

[0023] Each sample pair is represented as (x i ,y i ,x j ,y j ) i≠j , where x i and x j Represents two different randomly selected samples, y i and y j Represents x i and x j Serial number;

[0024] The sample pairs of the first test piece are used as the training set, and the sample pairs of the second test piece are used as the test set.

[0025] The step 2 is specifically: providing a diverse pulse representation based on the IntegrateAndFire (IAF) neuron with computational complexity:

[0026]

[0027] Where t represents the pulse time, τ is the time constant of the neuron, u(t) is the current membrane potential of the neuron, and u rest represents the reset potential, R is the equivalent resistance of the neuronal ion channel, I(t) represents the current of the synaptic input neuron, and w l is the synaptic weight, represents the synaptic input spike train, The sth pulse of the lth synapse is The moment of being excited; each neuron is synaptically connected to the input channel. If the sum of the weighted inputs exceeds the threshold membrane potential preset by the neuron, the neuron will generate a binary pulse output, and the membrane potential will return to the resting value u rest , and enter the refractory period;

[0028] Two parallel spike convolution branches are constructed based on IAF neurons. The parallel spike convolution branches extract multi-scale damage features through the spike convolution layer, IAF spike neuron layer and pooling layer, and realize fusion in the feature fusion unit. The comprehensive feature representation is mapped to the output spike rate through the linear spike layers FC1 and FC2. The output spike rate F of the i-th input sample is i The calculation is as follows:

[0029]

[0030] Where, represents the output binary pulse of the i-th sample at time t, u i (t) is its membrane potential, T s is the time dimension of the pulse train.

[0031] The step 3 is specifically as follows: designing a distance loss function to increase the difference between sample pairs in the feature space, ensuring that the health indicator can present different degradation states of the samples; for a set of input sample pairs (x i ,y i ,x j ,y j ) i≠j , its distance loss function L d The definition is as follows:

[0032]

[0033] Where, Fi With F j Represents the input sample x i and x j The output pulse emission rate of the corresponding linear pulse layer FC2; design the angle loss function, for a set of input sample pairs (x i ,y i ,x j ,y j ) i≠j , when y i <y j When the angle loss function L a The definition is as follows:

[0034]

[0035] V i =h i -h1,V j =h j -h1,

[0036]

[0037] Where h i Characterize the pulse firing rate characteristics of the i-th sample in the linear pulse layer FC1, is the binary pulse vector output by C neurons at time t for this sample, V i and V j Represent the eigenvector h i 、h j The relative eigenvector formed with the original state reference point h1, θ is the angle between the two.

[0038] The distance loss function amplifies the differences between samples at different degradation moments, guiding the model to generate health indicators that are more consistent with structural degradation characteristics. The angle loss function, on the other hand, encourages the feature extraction module to learn more monotonic and robust features, reducing abnormal fluctuations in health indicators. Together, the two constitute the model's multi-level contrast loss function, whose total loss is defined as follows:

[0039] L HI =λ1L d +λ2L a

[0040] Where, L HI represents the total loss function of the model, λ1 and λ2 are the weight coefficients of distance loss and angle loss respectively, and the weight coefficients are uniformly set to 1. By optimizing the above multi-level contrast loss function, the model can self-supervise and learn robust and monotonic feature representations during the process of structural health degradation, reducing the model's dependence on supervised annotation data.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] This paper proposes a method for intelligent health monitoring of aviation structures based on neuromorphic contrastive learning and multimodal data. A parallel pulse convolutional neural network model is constructed to effectively extract and fuse damage-sensitive features of data from different modalities, realizing low-power, high-precision real-time health monitoring. At the same time, a health indicator with good monotonicity and robustness is constructed through a multi-level contrast loss function, which significantly reduces the dependence on data labeling and meets the practical application needs of intelligent health monitoring of aviation structures. Finally, the parallel pulse neural network model is deployed on the neuromorphic hardware system, fully demonstrating the feasibility and application prospects of the neuromorphic computing architecture in the "always online" structural health monitoring scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Flowchart of a method according to an embodiment of the present invention.

[0044] Figure 2 Schematic diagram of neural morphological contrastive learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention is described in detail below with reference to the embodiments and accompanying drawings.

[0046] Reference Figure 1 , an intelligent health monitoring method for aviation structures based on neuromorphic contrastive learning and multimodal data, comprising the following steps:

[0047] Step 1: Perform time alignment and amplitude normalization preprocessing on the collected multimodal data of guided waves and acoustic emission from aerospace structures to build a unified data representation space, construct multimodal sample pairs, and divide the training and test sets according to the test pieces.

[0048] This embodiment strictly aligns the time domain of different modal data to ensure that the different modal data maintain a high degree of consistency in time scale; and performs amplitude normalization preprocessing on the multimodal data to keep it in the range [0,1]:

[0049]

[0050] Where x′ ij represents the i-th sample value of the j-th class data after normalization preprocessing, x ij is the i-th sample value of the j-th class data in the original data before normalization, min(x j ) and max(x j ) represent the minimum and maximum values of the j-th category data respectively;

[0051] The sliding window method is used to sample the normalized preprocessed multimodal signal and construct the sample pairs required for contrastive learning; the window size is fixed to W L , the sliding step size is fixed to W S , then the number of samples N obtained by sliding window sampling W for:

[0052]

[0053] Where N is the total length of the data;

[0054] The sample pairs are constructed by pairing each other, and finally N P Valid sample pairs:

[0055]

[0056] Each sample pair can be expressed as (x i ,y i ,x j ,y j ) i≠j , where x i and x j Represents two different randomly selected samples, y i and y j Represents x i and x j Each sample pair contains a multimodal signal sequence at different times, reflecting the continuous degradation process of the structural health state, providing high-quality data support for the subsequent construction of monotonic and robust health indicators;

[0057] The sample pairs of the first test piece are used as the training set, and the sample pairs of the second test piece are used as the test set;

[0058] Step 2: A parallel spiking convolutional neural network model is established based on the Integrate-And-Fire (IAF) neuron. The preprocessed guided wave and acoustic emission signals are converted into pulse trains through the encoding layer. Damage-sensitive features are extracted through parallel spiking neural network (SNN) branches. The damage-sensitive features are then fused through a feature fusion unit to obtain a comprehensive feature representation, which is then mapped to the output pulse firing rate.

[0059] This embodiment is based on the Integrate And Fire (IAF) neuron and can provide diverse pulse representations with relatively low computational complexity:

[0060]

[0061] Where t represents the pulse time, τ is the time constant of the neuron, u(t) is the current membrane potential of the neuron, and u rest represents the reset potential, R is the equivalent resistance of the neuronal ion channel, I(t) represents the current of the synaptic input neuron, and w l is the synaptic weight, represents the synaptic input spike train, The sth pulse of the lth synapse is The moment of being excited; each neuron is synaptically connected to the input channel. If the sum of the weighted inputs exceeds the threshold membrane potential preset by the neuron, the neuron will generate a binary pulse output, and the membrane potential will return to the resting value u rest , and enter the refractory period;

[0062] Two parallel spike convolution branches are constructed based on IAF neurons. The parallel spike convolution branches extract damage-sensitive features through the spike convolution layer, IAF spike neuron layer and pooling layer, and realize fusion in the feature fusion unit. The comprehensive feature representation is mapped to the output spike rate through the linear spike layers FC1 and FC2. The output spike rate F of the i-th input sample is i The calculation is as follows:

[0063]

[0064] Where, represents the output binary pulse of the i-th sample at time t, u i (t) is its membrane potential, T s is the time dimension of the pulse train;

[0065] Step 3: Reference Figure 2 , using contrastive learning mechanism and designing multi-level contrastive loss function to construct aviation structural health indicators with excellent monotonicity and robustness, and to achieve accurate characterization of aviation structural health status without the need for labeled data;

[0066] In order to construct an increasing health index, this embodiment designs a distance loss function to increase the difference between sample pairs in the feature space, thereby ensuring that the health index can clearly present the different degradation states of the samples; for a set of input sample pairs (x i ,y i ,x j ,y j ) i≠j , its distance loss function L d The definition is as follows:

[0067]

[0068] Where, F i With F jRepresents the input sample x i and x j The corresponding output pulse firing rate of the linear pulse layer FC2.

[0069] In order to further enhance the robustness of the health indicators, an angle loss function is designed to constrain the health indicators in the feature space from deviating from the expected trend in some areas. For a set of input sample pairs (x i ,y i ,x j ,y j ) i≠j , when y i <y j When the angle loss function L a The definition is as follows:

[0070]

[0071] V i =h i -h1,V j =h j -h1,

[0072]

[0073] Where h i Characterize the pulse firing rate characteristics of the i-th sample in the linear pulse layer FC1, is the binary pulse vector output by C neurons at time t for this sample, V i and V j Represent the eigenvector h i 、h j The relative eigenvector formed with the original state reference point h1, θ is the angle between the two.

[0074] The distance loss function amplifies the differences between samples at different degradation moments, guiding the model to generate health indicators that are more consistent with structural degradation characteristics. The angle loss function, on the other hand, encourages the feature extraction module to learn more monotonic and robust features, effectively reducing abnormal fluctuations in health indicators. Together, the two constitute the model's multi-level contrast loss function, whose total loss is defined as follows:

[0075] L HI =λ1L d +λ2L a

[0076] Where, L HIrepresents the total loss function of the model, λ1 and λ2 are the weight coefficients of distance loss and angle loss, respectively. Considering the equal importance of the two in the modeling process, the weight coefficients are uniformly set to 1. By optimizing the above multi-level contrast loss function, the model can self-supervise and learn robust and monotonic feature representations during the structural health degradation process, thereby reducing the model's dependence on supervised labeled data.

[0077] Step 4: Input the training set obtained in step 1 into the parallel spiking convolutional neural network model constructed in step 2, calculate the output pulse rate, and optimize the multi-level contrast loss function designed in step 3. Iteratively train the model parameters until the model converges to obtain the final parallel spiking convolutional neural network model.

[0078] Step 5: Deploy the final parallel spiking convolutional neural network model obtained in Step 4 to an asynchronous event-driven neuromorphic hardware system. Input the test set obtained in Step 1 into the neuromorphic hardware system to achieve real-time health monitoring of aviation structures and verify the advantages of neuromorphic computing in terms of energy consumption, latency, and accuracy.

[0079] This example is based on the fatigue tensile multi-modal experimental data of aluminum alloy plates to verify the effectiveness of the method of the present invention:

[0080] The fatigue tensile tests in this embodiment were conducted on an electro-hydraulic servo fatigue testing machine (LF5105) with a maximum rated load of 100 kN. The specimens used were 7050 aluminum alloy plates with the following dimensions: 320 mm long, 40 mm wide, and 2 mm thick, with a circular notch. The fatigue tests were conducted under sinusoidal cyclic loading conditions at a loading frequency of 6 Hz, a maximum peak load of 11.8 kN, a minimum load of 1.8 kN, and a stress ratio R of 0.1. Two specimens were used in this embodiment. The crack length of specimen 1 expanded from an initial 1 mm to a final 18.5 mm, while the crack length of specimen 2 expanded from 1 mm to 13 mm.

[0081] During the tensile test, two types of signals, acoustic emission and guided wave, were collected synchronously. The acoustic emission data acquisition system included an acoustic emission sensor and a data acquisition instrument. The acoustic emission sensor used two high-bandwidth differential acoustic emission sensors (WD) with a bandwidth range of 100-900kHz. The acoustic emission data acquisition instrument used a PCI-DSP32 channel acoustic emission acquisition device with a sampling rate of 1MHz. The guided wave signal acquisition system included a waveform generator (Keysight 33510B), a high-voltage amplifier (ATA-2021B), and an oscilloscope (Keysight The system consists of a DSOX2014A) and two piezoelectric lead zirconate titanate (PZT) sensors. The PZT sensors serve as the excitation and receiving units of the guided wave, respectively, to collect the guided wave signals propagating in the structure to extract information on the structural health status. Before crack formation, the applied load is temporarily reduced to 0 kN after every 5000 loading cycles to measure the guided wave signals. After crack initiation, measurements are taken every 500 cycles. The AE acquisition system continuously records throughout the entire loading cycle to achieve real-time monitoring of the entire process of crack initiation and propagation.

[0082] In addition, the proposed parallel spike convolutional neural network model is deployed on the asynchronous event-driven neuromorphic hardware platform Speck. Speck can meet the hardware requirements of dynamic computing and activate the processing core only when receiving input events, thereby significantly reducing the system's static power consumption and ineffective computing burden. At the same time, Speck uses its hardware circuit design to implement asynchronous event-driven distributed convolution processing of pulse sequences, meeting the stringent requirements of low latency, high throughput, and low power consumption in structural health monitoring. The software development tool chain provided by Speck provides a complete process from pulse encoding, weight mapping to model deployment, effectively simplifying the engineering implementation of SNN algorithms on embedded platforms and greatly improving the actual deployment efficiency. ([5] Yao M, Richter O, Zhao G, et al. Spike-based dynamic computing with asynchronous sensing-computing neuromorphic chip. Nature Communications, 15(1): 4464, 2024).

[0083] To comprehensively evaluate the quality of the health index constructed in this embodiment, three commonly used evaluation indicators, monotonicity, correlation, and robustness, are comprehensively considered. First, the five-point cubic smoothing technique is applied to process the pulse rate characteristic sequence to reduce the influence of noise and random fluctuations. The characteristic sequence set is denoted as h = {h(t n )|n=1,2,...,N}, the time series is T={t n|n=1,2,...,N}, the processed feature sequence set can be decomposed into the following two parts:

[0084] h(t n )=h T (t n )+h R (t n ),n=1,2,...,N

[0085] Where h T (t n ) represents the stationary trend term, h R (t n ) represents the random margin term.

[0086] Monotonicity is defined as follows:

[0087]

[0088] Where, Represents the unit step function, and the monotonicity range is between [0,1]. The closer the value is to 1, the more significant the monotonic trend of HI is.

[0089] The correlation is defined as follows:

[0090]

[0091] Where: and are the means of the time series T and the feature sequence h, respectively; the correlation value range is [0,1]. The closer the correlation coefficient is to 1, the stronger the correlation between the degradation feature and the time series.

[0092] Robustness is defined as follows:

[0093]

[0094] The value range of robustness is [0,1]. The stronger the robustness of the feature, the closer the value of this indicator is to 1.

[0095] The comprehensive evaluation indicators are defined as follows:

[0096] Cri=ω1Mon(h)+ω2Corr(h,T)+ω3Rob(h)

[0097] Where ω1, ω2, and ω3 are the weight coefficients of the three criteria, respectively. The evaluation results of different evaluation criteria vary, so the weight values need to be set according to a certain ratio. Given that the importance of the three criteria in practical applications is difficult to accurately quantify, this embodiment first normalizes the calculation results of each criterion to reduce the amplitude differences between different indicators, and then uniformly sets the weights to ω1 = ω2 = ω3 = 1 / 3.

[0098] To systematically evaluate the effectiveness of the proposed method, a comparative analysis was conducted on health indicators trained using two single modalities, acoustic emission and guided wave, as well as those trained using multimodal data fusion. The comparative results, shown in Table 1, clearly demonstrate that the overall performance of health indicators constructed using multimodal data fusion is superior to that of health indicators constructed using single modal data, both within the traditional deep neural network (DNN) architecture and within the SNN architecture. This result demonstrates the effectiveness and robustness of the multimodal data fusion approach in extracting structural degradation features and confirms its potential to improve the accuracy of health status identification and the reliability of remaining life prediction.

[0099] Table 2 compares the power consumption, energy, and latency of the proposed method on an NVIDIA GeForce GTX 3050 GPU, an Intel Core i5-12490F CPU, and a Speck2e Dev Kit neuromorphic chip. The results show that, while ensuring basic accuracy, the Speck platform consumes only 1.42mW for single-sample inference, compared to 87.57mW for the GPU platform and 1309.95mW for the CPU platform, a reduction of more than two to three orders of magnitude. Latency is also significantly superior to traditional architectures, fully demonstrating the unique advantages of neuromorphic computing combined with asynchronously driven biomimetic processors in low power consumption and low latency, making it particularly suitable for always-on structural health monitoring scenarios. Table 3 lists the specific power consumption distribution of the three power rails when the proposed method infers a single sample on the Speck platform, further revealing the energy consumption characteristics of each system component. Table 4 summarizes the memory resource usage of DNN and SNN models under the same task. The SNN model can directly process sparse input signals represented by binary pulse (1 / 0) codes, significantly reducing storage requirements. Its memory overhead is much lower than that of traditional DNNs that rely on 16-bit or 32-bit floating-point numbers for calculations, showing extremely high resource utilization.

[0100] Table 1 Quantitative score results of comprehensive performance of health indicators

[0101]

[0102] Table 2 Comparison of power, energy, and latency on different hardware systems

[0103]

[0104] Table 3 Specific power of different channels on Speck

[0105]

[0106] Table 4 Memory usage (M) of data processed by DNN and SNN models

[0107]

Claims

1. A method for intelligent health monitoring of aviation structures based on neuromorphic contrastive learning and multimodal data, characterized by: First, the collected multimodal datasets of guided waves and acoustic emission are preprocessed. Second, a parallel pulse convolutional neural network model for multimodal data is established. The preprocessed multimodal data is encoded into a pulse sequence to extract high-level damage-sensitive features, which are then fused and output as the pulse firing rate. Then, a contrastive learning mechanism is adopted, a multi-level contrastive loss function is designed, and the spiking neural network is trained to obtain monotonic and robust health indicators. Finally, it is deployed on a neuromorphic hardware system to verify the energy consumption advantages of neuromorphic computing.

2. The method according to claim 1, characterized in that The following steps are involved: Step 1: Perform time alignment and amplitude normalization preprocessing on the collected multimodal data of guided waves and acoustic emission from aerospace structures to build a unified data representation space, construct multimodal sample pairs, and divide the training and test sets according to the test pieces. Step 2: A parallel spiking convolutional neural network model is established based on the Integrate-And-Fire (IAF) neuron. The preprocessed guided wave and acoustic emission signals are converted into pulse trains through the encoding layer. Damage-sensitive features are extracted through parallel spiking neural network (SNN) branches. The damage-sensitive features are then fused through a feature fusion unit to obtain a comprehensive feature representation, which is then mapped to the output pulse firing rate. Step 3: Using a contrastive learning mechanism, a multi-level contrastive loss function is designed to construct a monotonic and robust aviation structural health indicator, achieving accurate characterization of the health status of aviation structures. Step 4: Input the training set obtained in step 1 into the parallel spiking convolutional neural network model constructed in step 2, calculate the output pulse rate, and optimize the multi-level contrast loss function designed in step 3. Iteratively train the model parameters until the model converges to obtain the final parallel spiking convolutional neural network model. Step 5: Deploy the final parallel spiking convolutional neural network model obtained in step 4 to an asynchronous event-driven neuromorphic hardware system, and input the test set obtained in step 1 into the neuromorphic hardware system to achieve real-time health monitoring of aviation structures.

3. The method according to claim 2, characterized in that The step 1 is specifically as follows: performing time domain alignment on the different modal data to ensure that the different modal data are consistent in time scale; performing amplitude normalization preprocessing on the multimodal data to make it in the range [0,1]: Where x′ ij represents the i-th sample value of the j-th class data after normalization preprocessing, x ij is the i-th sample value of the j-th class data in the original data before normalization, min(x j ) and max(x j ) represent the minimum and maximum values of the j-th category data respectively; The sliding window method is used to sample the normalized preprocessed multimodal signal and construct the sample pairs required for contrastive learning; the window size is fixed to W L , the sliding step size is fixed to W S , then the number of samples N obtained by sliding window sampling W for: Where N is the total length of the data; The sample pairs are constructed by pairing each other, and finally N P Valid sample pairs: Each sample pair is represented as (x i ,y i ,x j ,y j ) i≠j , where x i and x j Represents two different randomly selected samples, y i and y j Represents x i and x j Serial number; The sample pairs of the first test piece are used as the training set, and the sample pairs of the second test piece are used as the test set.

4. The method according to claim 2, characterized in that The step 2 is specifically as follows: providing a diverse pulse representation based on the IntegrateAndFire (IAF) neuron with computational complexity: Where t represents the pulse time, τ is the time constant of the neuron, u(t) is the current membrane potential of the neuron, and u rest represents the reset potential, R is the equivalent resistance of the neuronal ion channel, I(t) represents the current of the synaptic input neuron, and w l is the synaptic weight, represents the synaptic input spike train, The sth pulse of the lth synapse is The moment of being excited; each neuron is synaptically connected to the input channel. If the sum of the weighted inputs exceeds the threshold membrane potential preset by the neuron, the neuron will generate a binary pulse output, and the membrane potential will return to the resting value u rest , and enter the refractory period; Two parallel spike convolution branches are constructed based on IAF neurons. The parallel spike convolution branches extract multi-scale damage features through the spike convolution layer, IAF spike neuron layer and pooling layer, and realize fusion in the feature fusion unit. The comprehensive feature representation is mapped to the output spike rate through the linear spike layers FC1 and FC2. The output spike rate F of the i-th input sample is i The calculation is as follows: Where, represents the output binary pulse of the i-th sample at time t, u i (t) is its membrane potential, T s is the time dimension of the pulse train.

5. The method according to claim 2, characterized in that The step 3 is specifically as follows: designing a distance loss function to increase the difference between sample pairs in the feature space, ensuring that the health indicator can present different degradation states of the samples; for a set of input sample pairs (x i ,y i ,x j ,y j ) i≠j , its distance loss function L d The definition is as follows: Where, F i With F j Represents the input sample x i and x j The output pulse emission rate of the corresponding linear pulse layer FC2; design the angle loss function, for a set of input sample pairs (x i ,y i ,x j ,y j ) i≠j , when y i <y j When the angle loss function L a The definition is as follows: In i =h i -h1,V j =h j -h1, Where h i Characterize the pulse firing rate characteristics of the i-th sample in the linear pulse layer FC1, is the binary pulse vector output by C neurons at time t for this sample, V i and V j Represent the eigenvector h i 、h j The relative eigenvector formed with the original state reference point h1, θ is the angle between the two; The distance loss function amplifies the differences between samples at different degradation moments, guiding the model to generate health indicators that are more consistent with structural degradation characteristics. The angle loss function, on the other hand, encourages the feature extraction module to learn more monotonic and robust features, reducing abnormal fluctuations in health indicators. Together, the two constitute the model's multi-level contrast loss function, whose total loss is defined as follows: L HI =λ1L d +λ2L a Where, L HI represents the total loss function of the model, λ1 and λ2 are the weight coefficients of distance loss and angle loss respectively, and the weight coefficients are uniformly set to 1. By optimizing the above multi-level contrast loss function, the model can self-supervise and learn robust and monotonic feature representations during the process of structural health degradation, reducing the model's dependence on supervised annotation data.

Images