Friction plate damage identification and life prediction method and system based on multimodal fusion

Abstract

The present invention relates to the field of friction plate detection technology, and more specifically to a method and system for friction plate damage identification and life prediction based on multimodal fusion. The method comprises the following steps: using an infrared thermal imager, a strain sensor, and an acoustic sensor to synchronously acquire the friction plate's temperature field, strain field, and acoustic signal; analyzing the synchronized temperature and strain fields using a thermodynamic bidirectional coupling model to determine damage identification and life prediction results for the friction plate; and using a neural network to establish a synchronous mapping association between the acoustic signal and the damage identification and life prediction results, thereby forming a multimodal fusion model for friction plate damage identification and life prediction based on the acoustic signal. The present invention utilizes a neural network to migrate the friction plate detection process, which typically involves large amounts of data and complex thermodynamic coupling calculations, to a smaller, more easily monitored acoustic signal. This allows for rapid acquisition of high-precision detection results using the acoustic signal, thereby achieving lightweight friction plate detection.

Description

Technical Field

[0001] The present invention relates to the technical field of friction plate detection, and in particular to a friction plate damage identification and life prediction method and system based on multimodal fusion. Background Art

[0002] The friction plate is a friction material fixed to the rotating or brake disc, which can ensure that in emergency situations such as the lack of external power supply, the machine can achieve deceleration through the mutual friction between the brake pad and the brake disc. In order to meet this demanding application environment, the development of relatively high-precision damage defect detection technology is currently a common goal of academia and industry. There are many reasons for damage defects in high-speed rail friction plates: during the manufacturing process, improper pressure can easily lead to insufficient stability of the friction material, resulting in edge shedding or porosity on the surface of the friction plate. Insufficient stamping pressure during molding can also cause cracks in the non-friction layer parallel to the backing plate. In addition, during use, the friction material can also fall off from the base plate due to corrosion of the base pad and friction material, poor adhesion, and other reasons.

[0003] Existing technologies mostly use two detection methods to detect friction plate damage, one is contact detection and the other is non-contact detection. Contact damage detection includes strain sensing detection, and non-contact detection includes temperature field detection. These two detection methods are usually combined to obtain high-precision detection results. However, the thermodynamic coupling calculation process of the two is complex and the amount of data is large, which will cause the defect of poor timeliness of friction plate detection. Summary of the Invention

[0004] The purpose of the present invention is to provide a friction plate damage identification and life prediction method and system based on multimodal fusion, so as to solve the technical problems in the existing technology that the thermodynamic coupling calculation process is complex, the amount of data is large, and the timeliness of friction plate detection is poor.

[0005] In order to solve the above technical problems, the present invention specifically provides the following technical solutions:

[0006] A friction plate damage identification and life prediction method based on multimodal fusion includes the following steps:

[0007] Use infrared thermal imagers, strain sensors and acoustic sensors to synchronously acquire the temperature field, strain field and acoustic signals of the friction plate;

[0008] The damage identification result and life prediction result of the friction plate are analyzed based on the synchronized temperature field and strain field through the thermodynamic bidirectional coupling model;

[0009] A neural network is used to establish a synchronous mapping association between the acoustic signal and the damage identification result and the life prediction result, forming a multimodal fusion model for damage identification and life prediction of the friction plate based on the acoustic signal.

[0010] As a preferred embodiment of the present invention, the analysis method of the life prediction result includes:

[0011] Between the temperature field and the strain field, the heat conduction equation and the Chaboche damage evolution model are used for forward coupling through thermal stress calculation, and reverse coupling is performed through thermal physical property damage degradation calculation, thus obtaining a thermodynamic bidirectional coupling model.

[0012] The life prediction results of the friction plate are calculated based on the thermodynamic bidirectional coupling model using the Weibull distribution function;

[0013] The heat conduction equation is: ;

[0014] In the formula is the material density, c is the specific heat capacity, T is the temperature, t is the time, k1 is the thermal conductivity coefficient, is the frictional heat power, To dissipate heat for damage, , is the dissipation coefficient, D is the damage variable;

[0015] The Chaboche damage evolution model is: ;

[0016] Where N is the number of load cycles, is the equivalent stress amplitude obtained by inversion of the strain field, is the temperature damage threshold, is the mean stress correction function, is the mean stress, p is the stress sensitivity index, k2 is the damage acceleration index, is the activation energy parameter, T0 is the reference temperature, For Macaulay brackets, is the basic damage resistance coefficient, is the average stress sensitivity coefficient;

[0017] The thermal stress calculation formula is: ;

[0018] Where, For thermal stress, is the thermal expansion coefficient, E is the elastic modulus, m is the damage sensitivity coefficient, For local warming;

[0019] The thermal physical damage degradation model is: ;

[0020] Where, is the effective heat transfer coefficient in the damaged state, is the heat transfer coefficient under lossless state, is the thermal conductivity damage coefficient, is the effective specific heat capacity in the damaged state, is the effective specific heat capacity in the lossless state, is the heat capacity damage coefficient;

[0021] The calculation expression of the life prediction result is:

[0022] ;

[0023] Where, is the remaining life of the friction plate, is the critical damage value, is the current damage value determined in real time by the Chaboche damage evolution model, is the damage evolution rate.

[0024] As a preferred solution of the present invention, the damage identification result analysis method includes:

[0025] By introducing the weighted attention mechanism into the Weibull distribution function and based on the thermodynamic bidirectional coupling model, the damage identification results of the friction plate are calculated;

[0026] The calculation expression of the damage identification result is: ;

[0027] Where, is the failure probability of the friction plate at time t, is the median remaining life, D is the damage variable, N is the number of load cycles, is the damage evolution rate, is the Weibull shape parameter corresponding to the x term, A is the identifier of the temperature field, B is the identifier of the strain field, for The attention weight of .

[0028] As a preferred solution of the present invention, the attention weight The setting methods include:

[0029] Using Sigmoid response curve setting ,in:

[0030] When x=A, , where for The attention weight, For local warming, is the temperature rise rate of the hot spot, is the Weibull shape parameter corresponding to the temperature field;

[0031] When x=B, , where for The attention weight, is the damage evolution rate, is the Weibull shape parameter corresponding to the strain field.

[0032] As a preferred embodiment of the present invention, the method for generating training data for a neural network includes:

[0033] Combine all the temperature fields, strain fields and acoustic signals acquired by the infrared thermal imager, strain sensor and acoustic sensor into a real data set;

[0034] The GAN network is used to learn the data relationship in the real data set, and the thermodynamic bidirectional coupling model is used as a constraint term to regularize the generator in the GAN network to generate new temperature and strain fields.

[0035] The CycleGAN network is then used to learn the modal conversion relationship between the temperature field, strain field and acoustic signal in the real data set, and the new temperature field and strain field are modally converted to generate new acoustic signals;

[0036] Calculate new damage identification results and life prediction results based on the new temperature field and strain field, and combine the new damage identification results and life prediction results with the new acoustic signal to generate a data set;

[0037] All acoustic signals acquired by the acoustic sensor, as well as damage identification results and life prediction results calculated based on the temperature and strain fields acquired by the infrared thermal imager and strain sensor, are added to the generated data set to form a training data set.

[0038] The generator of the GAN network is used to generate new temperature fields and strain fields, and the discriminator is used to distinguish whether the new temperature fields and strain fields are generated data or real data;

[0039] The generator of the first GAN network in the CycleGAN network is used to convert the temperature field and the strain field into an acoustic signal, and the discriminator is used to distinguish whether the acoustic signal is generated data or real data;

[0040] The generator of the second GAN network is used to convert the acoustic signal into temperature and strain fields, and the discriminator is used to distinguish whether the temperature and strain fields are generated data or real data;

[0041] The joint training loss of the GAN network and CycleGAN network is:

[0042] ;

[0043] Where, L comb is the joint loss, G is the generator of the GAN network, z is the noise input of G, G(z) is the output of G, G c1 is the generator of the first GAN network in the CycleGAN network, G c2 is the generator of the second GAN network in the CycleGAN network, G c1 (G(z)) is G c1 The modal conversion output, G c2 (G c1 (G(z))) is G c2 The modal conversion output of is the L1 norm.

[0044] As a preferred solution of the present invention, the method for constructing the multimodal fusion model includes:

[0045] Divide the training dataset into a training set and a test set;

[0046] On the training set, the neural network is trained with acoustic signals as input and damage identification results and life prediction results as output to obtain a multimodal fusion model;

[0047] On the test set, the performance of the multimodal fusion model is evaluated;

[0048] The multimodal fusion model is:

[0049] ;

[0050] Where, is the failure probability output by the neural network, is the remaining lifespan output by the neural network, CNN is the neural network, and C is the acoustic signal.

[0051] As a preferred solution of the present invention, the loss function of the multimodal fusion model is:

[0052] ;

[0053] Where, is the loss function of the multimodal fusion model, is the failure probability output by the neural network, is the remaining lifetime of the neural network output, is the failure probability calculated by the thermodynamic bidirectional coupling model, is the remaining lifetime calculated by the thermodynamic two-way coupling model, is the L1 norm.

[0054] As a preferred solution of the present invention, after the training data set is generated, a preprocessing operation needs to be performed on the training data set.

[0055] As a preferred solution of the present invention, the multimodal fusion model performance evaluation indicators include mean square error, ROC curve, and AUC value.

[0056] As a preferred embodiment of the present invention, the present invention provides a friction plate damage identification and life prediction system based on multimodal fusion, which is applied to a friction plate damage identification and life prediction method based on multimodal fusion, including:

[0057] A data acquisition unit is used to synchronously acquire the temperature field, strain field and acoustic signal of the friction plate using an infrared thermal imager, a strain sensor and an acoustic sensor;

[0058] A physical modeling unit, configured to analyze damage identification results and life prediction results of the friction plate based on the synchronized temperature field and strain field using a thermodynamic bidirectional coupling model;

[0059] A deep learning unit is used to use a neural network to establish a synchronous mapping association between the acoustic signal and the damage identification result and the life prediction result, thereby forming a multimodal fusion model for damage identification and life prediction of the friction plate based on the acoustic signal.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] The present invention uses a neural network to migrate the friction plate detection process formed by large amounts of data and complex thermodynamic coupling calculation processes to acoustic signals with small amounts of data and easy to monitor. This can achieve rapid acquisition of high-precision detection results using acoustic signals, achieve lightweight friction plate detection, and improve timeliness. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other implementation drawings based on the provided drawings without inventive effort.

[0063] Figure 1 A flow chart of a friction plate damage identification and life prediction method based on multimodal fusion provided in an embodiment of the present invention;

[0064] Figure 2 Block diagram of the friction plate damage identification and life prediction system based on multimodal fusion provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0066] like Figure 1 As shown, the present invention provides a friction plate damage identification and life prediction method based on multimodal fusion, comprising the following steps:

[0067] Use infrared thermal imagers, strain sensors and acoustic sensors to synchronously acquire the temperature field, strain field and acoustic signals of the friction plate;

[0068] The damage identification and life prediction results of the friction plate are analyzed based on the synchronized temperature field and strain field through the thermodynamic bidirectional coupling model.

[0069] A neural network is used to establish a synchronous mapping association between acoustic signals, damage identification results, and life prediction results, forming a multimodal fusion model for damage identification and life prediction of friction plates based on acoustic signals.

[0070] This invention utilizes data from two modalities, temperature and strain fields, through bidirectional thermodynamic coupling calculations to achieve high-precision friction plate life prediction and damage identification, achieving high-precision friction plate life prediction and damage identification using multimodal data. This physical calculation process involves complex computations, and the temperature and strain fields are much larger than the small-scale acoustic emission signal data. While this physical calculation process can achieve high-precision detection, it also suffers from insufficient timeliness due to the complexity and large amount of data. In practical applications, real-time monitoring of friction plates is difficult.

[0071] The present invention uses a neural network to perform transfer learning on the detection results obtained by calculating the temperature field and strain field through the bidirectional coupling of thermodynamics, and maps and associates the acoustic signal with the detection results obtained by calculating the temperature field and strain field through the bidirectional coupling of thermodynamics, thereby achieving high-precision detection results (i.e., damage identification results and life prediction results) based on the acoustic signal, thereby achieving lightweight friction plate detection process.

[0072] In order to ensure high accuracy in friction plate life prediction and damage identification generated by multimodal data, the present invention adopts a bidirectional coupling process in the physical calculation process. Forward coupling is that the stress / temperature field in thermodynamics can be used to calculate the damage increment of the friction plate, and reverse coupling is that the damage increment is used to calculate the material degradation parameter to correct the physical property parameters of the thermodynamic model. This forms a positive feedback chain of "damage evolution → material property degradation → thermal field redistribution", which can improve the detection accuracy of friction plate life prediction and damage identification.

[0073] Through limited actual monitoring and complex physical calculations, the present invention can accumulate a certain amount of real data for neural network training. However, this insufficient sample size makes neural networks trained with this data prone to overfitting and poor generalization. Therefore, the present invention requires expanding the dataset, and employs a Generative Adversarial Network (GAN) network to achieve this, generating data close to the real distribution. To expand the dataset, the GAN network is first used to generate temperature and strain field data close to the real distribution. Constraints are added to the generation process, creating a thermodynamic bidirectional coupling model. This ensures that the generated temperature and strain field data adhere to real physical laws, resulting in greater authenticity. However, there is no physical correlation between acoustic signals and temperature and strain field data, which can easily lead to randomness during the generation process, reducing authenticity. To address this issue, the present invention utilizes the domain conversion function of a cycle-consistent generative adversarial network (i.e., a CycleGAN network) to learn the conversion between multimodal data. This allows accurate conversion of temperature and strain field data into acoustic signals corresponding to real-world scenarios. This effectively establishes a constraint between multi-domain / multimodal data, avoiding randomness in acoustic signal generation. Finally, the GAN network is used to generate temperature and strain field data close to the actual distribution. The domain conversion function of the CycleGAN network is then used to generate acoustic signals close to the actual distribution based on the output of the GAN network. Combined with physical calculations, the life prediction results and damage identification results corresponding to the output of the GAN network are obtained to form a generated data set. This is combined with real data to form an overall training set for training the neural network, thereby improving the prediction performance of the neural network.

[0074] In the present invention, the strain field is obtained by using the DIC technology (Digital Image Correlation) or differential calculation after multi-point measurement using multiple strain sensors in actual measurement.

[0075] The analysis methods for life prediction results include:

[0076] Between the temperature field and the strain field, the heat conduction equation and the Chaboche damage evolution model are used for forward coupling through thermal stress calculation, and reverse coupling is performed through thermal physical property damage degradation calculation, thus obtaining a thermodynamic bidirectional coupling model.

[0077] The life prediction results of the friction plate are calculated based on the thermodynamic bidirectional coupling model using the Weibull distribution function;

[0078] The heat conduction equation is: ;

[0079] In the formula is the material density, c is the specific heat capacity, T is the temperature, t is the time, k1 is the thermal conductivity coefficient, is the frictional heat power, To dissipate heat for damage, , is the dissipation coefficient, D is the damage variable;

[0080] The Chaboche damage evolution model is: ;

[0081] Where N is the number of load cycles, is the equivalent stress amplitude obtained by inversion of the strain field, is the temperature damage threshold, is the mean stress correction function, , is the mean stress, p is the stress sensitivity index, k2 is the damage acceleration index, is the activation energy parameter, T0 is the reference temperature, is the Macaulay bracket, M0 is the basic damage resistance coefficient, and M1 is the average stress sensitivity coefficient;

[0082] The thermal stress calculation formula is: ;

[0083] Where, For thermal stress, is the thermal expansion coefficient, E is the elastic modulus, m is the damage sensitivity coefficient, For local warming;

[0084] The thermal physical damage degradation model is: ;

[0085] Where, is the effective heat transfer coefficient in the damaged state, is the heat transfer coefficient under lossless state, is the thermal conductivity damage coefficient, is the effective specific heat capacity in the damaged state, is the effective specific heat capacity in the lossless state, is the heat capacity damage coefficient;

[0086] The calculation expression of life prediction result is:

[0087] ;

[0088] Where, is the remaining life of the friction plate, is the critical damage value, is the current damage value determined in real time by the Chaboche damage evolution model, is the damage evolution rate.

[0089] The analysis methods of damage identification results include:

[0090] By introducing the weighted attention mechanism into the Weibull distribution function and based on the thermodynamic bidirectional coupling model, the damage identification results of the friction plate are calculated;

[0091] The calculation expression of damage identification result is:

[0092] ;

[0093] Where, is the failure probability of the friction plate at time t, is the median remaining life, D is the damage variable, N is the number of load cycles, is the damage evolution rate, is the Weibull shape parameter corresponding to the x term, A is the identifier of the temperature field, B is the identifier of the strain field, for The attention weight of .

[0094] Attention weight The setting methods include:

[0095] Using Sigmoid response curve setting ,in:

[0096] When x=A, , where for The attention weight, For local warming, is the temperature rise rate of the hot spot, is the Weibull shape parameter corresponding to the temperature field;

[0097] When x=B, , where for The attention weight, is the damage evolution rate, is the Weibull shape parameter corresponding to the strain field.

[0098] The present invention sets the weight in the calculation expression of the damage identification result as the attention weight, and can perform sensitivity adaptation of the damage identification result according to the temperature change degree and damage change degree in the temperature field and strain field. The temperature change degree of the temperature field is quantified by the temperature rise rate of the hot spot. The higher the temperature rise rate of the hot spot, the more attention should be paid to the temperature field. Therefore, it is necessary to improve the sensitivity of the damage identification result. The weight of the damage identification result is increased to increase the sensitivity of the temperature field. Similarly, the damage change degree of the strain field is quantified by the damage evolution rate. As the damage evolution rate increases, it is necessary to increase the attention to the strain field. Therefore, it is necessary to increase In summary, setting the weight as the attention weight can adaptively improve the sensitivity of damage identification when the temperature field and strain field change drastically, identify the probability of friction plate failure more quickly, and improve the timeliness of damage identification, so as to achieve fault response earlier.

[0099] Methods for generating training data for neural networks include:

[0100] Combine all the temperature fields, strain fields and acoustic signals acquired by the infrared thermal imager, strain sensor and acoustic sensor into a real data set;

[0101] The GAN network is used to learn the data relationship in the real data set, and the thermodynamic bidirectional coupling model is used as a constraint term to regularize the generator in the GAN network to generate new temperature and strain fields.

[0102] The CycleGAN network is then used to learn the modal conversion relationship between the temperature field, strain field and acoustic signal in the real data set, and the new temperature field and strain field are modally converted to generate new acoustic signals;

[0103] Calculate new damage identification results and life prediction results based on the new temperature field and strain field, and combine the new damage identification results and life prediction results with the new acoustic signal to generate a data set;

[0104] All acoustic signals acquired by the acoustic sensor, as well as damage identification results and life prediction results calculated based on the temperature and strain fields acquired by the infrared thermal imager and strain sensor, are added to the generated data set to form a training data set.

[0105] The generator of the GAN network is used to generate new temperature fields and strain fields, and the discriminator is used to distinguish whether the new temperature fields and strain fields are generated data or real data;

[0106] The generator of the first GAN network in the CycleGAN network is used to convert the temperature field and strain field into acoustic signals, and the discriminator is used to distinguish whether the acoustic signal is generated data or real data;

[0107] The generator of the second GAN network is used to convert the acoustic signal into temperature and strain fields, and the discriminator is used to distinguish whether the temperature and strain fields are generated data or real data;

[0108] The joint training loss of the GAN network and the CycleGAN network is:

[0109] ;

[0110] Where, L comb is the joint loss, G is the generator of the GAN network, z is the noise input of G, G(z) is the output of G, G c1 is the generator of the first GAN network in the CycleGAN network, G c2 is the generator of the second GAN network in the CycleGAN network, G c1 (G(z)) is G c1 The modal conversion output, G c2 (G c1 (G(z))) is G c2 The modal conversion output of is the L1 norm.

[0111] The present invention jointly trains the GAN network and the CycleGAN network. The joint training loss refers to the temperature field and strain field output by the GAN network, that is, , the sound signal G converted and output by the first GAN network in the CycleGAN network c1 (G(z)), and then the temperature field and strain field G are converted and output by the second GAN network c2 (G c1 (G(z))), and the initial The difference between them, using this as the loss function can ensure the output of GAN After conversion in both directions, it is restored to its original state to the greatest extent possible. , maintaining conversion consistency and minimizing conversion loss, thereby ensuring that the CycleGAN network can generate acoustic signals corresponding to the output of the GAN network in real scenarios. After joint training, it is ensured that the GAN network outputs temperature and strain field data close to the real distribution, while ensuring that the CycleGAN network outputs acoustic signals close to the real distribution.

[0112] The present invention uses a neural network to perform transfer learning on the detection results obtained by calculating the temperature and strain fields through bidirectional thermodynamic coupling. This method maps and associates the acoustic signal with the detection results obtained through bidirectional thermodynamic coupling. This allows high-precision detection results (i.e., damage identification results and life prediction results) to be obtained based on the acoustic signal, thus achieving lightweight friction plate detection. The details are as follows:

[0113] The construction method of the multimodal fusion model includes:

[0114] Divide the training dataset into a training set and a test set;

[0115] On the training set, the neural network is trained with acoustic signals as input and damage identification results and life prediction results as output to obtain a multimodal fusion model;

[0116] On the test set, the performance of the multimodal fusion model is evaluated;

[0117] The multimodal fusion model is:

[0118] ;

[0119] Where, is the failure probability output by the neural network, is the remaining lifespan output by the neural network, CNN is the neural network, and C is the acoustic signal.

[0120] The loss function of the multimodal fusion model is:

[0121] ;

[0122] Where, is the loss function of the multimodal fusion model, is the failure probability output by the neural network, is the remaining lifetime of the neural network output, is the failure probability calculated by the thermodynamic bidirectional coupling model, is the remaining lifetime calculated by the thermodynamic two-way coupling model, is the L1 norm.

[0123] This invention simplifies the complex physical calculation process into a multimodal fusion model, achieving lightweight processing. Similarly, by utilizing small-scale acoustic signals, it avoids the need for integrated devices to collect strain and temperature field data, as well as large amounts of data, achieving data lightweighting.

[0124] In this invention, the damage identification results and life prediction results obtained by physical calculation are used as the prediction gold standard of the multimodal fusion model. It is expected that the output of the multimodal fusion model is close to the high-precision output of physical calculation. This is used as the loss function for neural network training to ensure that the output of the multimodal fusion model reaches high-precision output, and finally a lightweight model is achieved to obtain high-precision damage identification results and life prediction results.

[0125] After generating the training data set, the training data set needs to be preprocessed.

[0126] The performance evaluation indicators of the multimodal fusion model may include mean square error, ROC curve, and AUC value.

[0127] like Figure 2 As shown, the present invention provides a friction plate damage identification and life prediction system based on multimodal fusion, which is applied to a friction plate damage identification and life prediction method based on multimodal fusion, including:

[0128] A data acquisition unit is used to synchronously acquire the temperature field, strain field and acoustic signal of the friction plate using an infrared thermal imager, a strain sensor and an acoustic sensor;

[0129] The physical modeling unit is used to analyze the damage identification results and life prediction results of the friction plate based on the synchronized temperature field and strain field through a thermodynamic bidirectional coupling model;

[0130] The deep learning unit is used to use a neural network to establish a synchronous mapping association between the acoustic signal and the damage identification results and the life prediction results, forming a multimodal fusion model for damage identification and life prediction of the friction plate based on the acoustic signal.

[0131] The present invention uses a neural network to migrate the friction plate detection process formed by large amounts of data and complex thermodynamic coupling calculation processes to acoustic signals with small amounts of data and easy to monitor. This can achieve rapid acquisition of high-precision detection results using acoustic signals, achieve lightweight friction plate detection, and improve timeliness.

[0132] The above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and scope of protection of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present application.

Claims

1. A friction plate damage identification and life prediction method based on multimodal fusion, characterized in that: The following steps are involved: Use infrared thermal imagers, strain sensors and acoustic sensors to synchronously acquire the temperature field, strain field and acoustic signals of the friction plate; The damage identification result and life prediction result of the friction plate are analyzed based on the synchronized temperature field and strain field through the thermodynamic bidirectional coupling model; Using a neural network to establish a synchronous mapping association between the acoustic signal and the damage identification result and the life prediction result, thereby forming a multimodal fusion model for damage identification and life prediction of the friction plate based on the acoustic signal; Methods for generating training data for neural networks include: Combine all the temperature fields, strain fields and acoustic signals acquired by the infrared thermal imager, strain sensor and acoustic sensor into a real data set; The GAN network is used to learn the data relationship in the real data set, and the thermodynamic bidirectional coupling model is used as a constraint term to regularize the generator in the GAN network to generate new temperature and strain fields. The CycleGAN network is then used to learn the modal conversion relationship between the temperature field, strain field and acoustic signal in the real data set, and the new temperature field and strain field are modally converted to generate new acoustic signals; Calculate new damage identification results and life prediction results based on the new temperature field and strain field, and combine the new damage identification results and life prediction results with the new acoustic signal to generate a data set; All acoustic signals acquired by the acoustic sensor, as well as damage identification results and life prediction results calculated based on the temperature and strain fields acquired by the infrared thermal imager and strain sensor, are added to the generated data set to form a training data set. The generator of the GAN network is used to generate new temperature fields and strain fields, and the discriminator is used to distinguish whether the new temperature fields and strain fields are generated data or real data; The generator of the first GAN network in the CycleGAN network is used to convert the temperature field and strain field into acoustic signals, and the discriminator is used to distinguish whether the acoustic signal is generated data or real data; The generator of the second GAN network is used to convert the acoustic signal into temperature and strain fields, and the discriminator is used to distinguish whether the temperature and strain fields are generated data or real data; The joint training loss of the GAN network and the CycleGAN network is: ； Where, L comb is the joint loss, G is the generator of the GAN network, z is the noise input of G, G(z) is the output of G, G c1 is the generator of the first GAN network in the CycleGAN network, G c2 is the generator of the second GAN network in the CycleGAN network, G c1 (G(z)) is G c1 The modal conversion output, G c2 (G c1 (G(z))) is G c2 The modal conversion output of is the L1 norm.

2. The friction plate damage identification and life prediction method based on multimodal fusion according to claim 1 is characterized by: The analysis method of the life prediction result includes: Between the temperature field and the strain field, the heat conduction equation and the Chaboche damage evolution model are used for forward coupling through thermal stress calculation, and reverse coupling is performed through thermal physical property damage degradation calculation, thus obtaining a thermodynamic bidirectional coupling model. The life prediction results of the friction plate are calculated based on the thermodynamic bidirectional coupling model using the Weibull distribution function; The heat conduction equation is: ; Where, is the material density, c is the specific heat capacity, T is the temperature, t is the time, k1 is the thermal conductivity coefficient, is the frictional heat power, To dissipate heat for damage, , is the dissipation coefficient, D is the damage variable; The Chaboche damage evolution model is: ; Where N is the number of load cycles, is the equivalent stress amplitude obtained by inversion of the strain field, is the temperature damage threshold, is the mean stress correction function, , is the mean stress, p is the stress sensitivity index, k2 is the damage acceleration index, is the activation energy parameter, T0 is the reference temperature, is the Macaulay bracket, M0 is the basic damage resistance coefficient, and M1 is the average stress sensitivity coefficient; The thermal stress calculation formula is: ; Where, For thermal stress, is the thermal expansion coefficient, E is the elastic modulus, m is the damage sensitivity coefficient, For local warming; The thermal physical damage degradation model is: ; Where, is the effective heat transfer coefficient in the damaged state, is the heat transfer coefficient under lossless state, is the thermal conductivity damage coefficient, is the effective specific heat capacity in the damaged state, is the effective specific heat capacity in the lossless state, is the heat capacity damage coefficient; The calculation expression of the life prediction result is: ； Where, is the remaining life of the friction plate, is the critical damage value, is the current damage value determined in real time by the Chaboche damage evolution model, is the damage evolution rate.

3. The friction plate damage identification and life prediction method based on multimodal fusion according to claim 2 is characterized by: The damage identification result analysis method includes: By introducing the weighted attention mechanism into the Weibull distribution function and based on the thermodynamic bidirectional coupling model, the damage identification results of the friction plate are calculated; The calculation expression of the damage identification result is: ； Where, is the failure probability of the friction plate at time t, is the median remaining life, D is the damage variable, N is the number of load cycles, is the damage evolution rate, is the Weibull shape parameter corresponding to the x term, A is the identifier of the temperature field, B is the identifier of the strain field, for The attention weight of .

4. The friction plate damage identification and life prediction method based on multimodal fusion according to claim 3 is characterized by: Attention weight The setting methods include: Using Sigmoid response curve setting ,in: When x=A, , where for The attention weight, For local warming, is the temperature rise rate of the hot spot, is the Weibull shape parameter corresponding to the temperature field; When x=B, , where for The attention weight, is the damage evolution rate, is the Weibull shape parameter corresponding to the strain field.

5. The friction plate damage identification and life prediction method based on multimodal fusion according to claim 4 is characterized by: The method for constructing the multimodal fusion model includes: Divide the training dataset into a training set and a test set; On the training set, the neural network is trained with acoustic signals as input and damage identification results and life prediction results as output to obtain a multimodal fusion model; On the test set, the performance of the multimodal fusion model is evaluated; The multimodal fusion model is: ； Where, is the failure probability output by the neural network, is the remaining lifespan output by the neural network, CNN is the neural network, and C is the acoustic signal.

6. The friction plate damage identification and life prediction method based on multimodal fusion according to claim 5 is characterized by: The loss function of the multimodal fusion model is: ； Where, is the loss function of the multimodal fusion model, is the failure probability output by the neural network, is the remaining lifetime of the neural network output, is the failure probability calculated by the thermodynamic bidirectional coupling model, is the remaining lifetime calculated by the thermodynamic two-way coupling model, is the L1 norm.

7. The friction plate damage identification and life prediction method based on multimodal fusion according to claim 6 is characterized by: After generating the training data set, the training data set needs to be preprocessed.

8. The friction plate damage identification and life prediction method based on multimodal fusion according to claim 7 is characterized by: The performance evaluation indicators of the multimodal fusion model include mean square error, ROC curve, and AUC value.

9. A friction plate damage identification and life prediction system based on multimodal fusion, characterized in that: A friction plate damage identification and life prediction method based on multimodal fusion as described in any one of claims 1 to 8, comprising: A data acquisition unit is used to synchronously acquire the temperature field, strain field and acoustic signal of the friction plate using an infrared thermal imager, a strain sensor and an acoustic sensor; A physical modeling unit, configured to analyze damage identification results and life prediction results of the friction plate based on the synchronized temperature field and strain field using a thermodynamic bidirectional coupling model; A deep learning unit is used to use a neural network to establish a synchronous mapping association between the acoustic signal and the damage identification result and the life prediction result, thereby forming a multimodal fusion model for damage identification and life prediction of the friction plate based on the acoustic signal.

Images